Grade Estimates and Statistical Models

Without exams, how do we estimate what grades students get?

Anthony B. Masters

--

The Coronavirus pandemic caused great social and economic disruption.

Educational institutions around the world closed and began mass online learning. Teachers and students are adapting to exceptional circumstances. The question is: what grades do students receive when there are no exams?

This article discusses the statistical model used the qualifications regulator in England.

The Direct Centre Performance model

The Office of Qualifications and Examinations Regulation (Ofqual) regulates qualifications in England. After school and college closures in March, Gavin Williamson MP (Education Secretary) wrote:

Ofqual should ensure, as far as is possible, that qualification standards are maintained and the distribution of grades follows a similar profile to that in previous years.

In simple terms, the Ofqual model has three stages for each subject:

  1. Calculate the grade distribution at each centre based on the prior three years. That calculation accounts for improvement at centres.
  2. Adjust for the GCSE performance of the 2020 cohort. Their performance at GCSE compares to past cohorts in each school and college.
  3. Use the relative rankings given by teachers. Place each student in the adjusted grade distribution.

For centres with low numbers in a subject, student grades depend more on the teachers.

Thanks to the Royal Statistical Society and others, there are six main problems:

  • Validity: do teacher assessments and exams measure the same thing?
  • Model assessments: Ofqual use actual results from 2019, not teacher predictions.
  • Uncertainty in relative rankings: the model treats rank orders as immutable.
  • Subject covariance: student performance is separate in different subjects.
  • The ungraded problem: some students must receive an ungraded mark.
  • Point estimates and prediction intervals: students get the estimated grade. They do not receive the interval estimate.

Validity

--

--

Anthony B. Masters

This blog looks at the use of statistics in Britain and beyond. It is written by RSS Statistical Ambassador and Chartered Statistician @anthonybmasters.