My thanks to Bobby Grewal at Four Dwellings Academy for the awkward question.
Let’s start by asking what it is we are trying to measure. At Yacapaca, we have opted for “current” grading, as that’s what the majority of our users want. This means that we are attempting to answer the question:
- If the student were to take the GCSE exam today,
- and they had studied all the content to the same level as the content of this quiz
- then what grade might they be expected to achieve?
Without actually throwing students as young as 11 into a real GCSE, how can we test whether our answers are accurate? Not only is that impractical; we can’t even (yet) back-calculate from real GCSE results. So we have to calibrate against some existing scheme for which we have data that can be validated.
Fortunately, there are two sources, NC Levels and GCSE A*-G. There are published conversion tables between each of these and the GCSE 9-1 grades. My preferred version is the QEGS Points Conversion Wheel.
The next question becomes “how do we know that our original NC and GCSE gradings were accurate. The answer is that we crowdsourced the calibration from thousands of teachers. As well as quizzes, Yacapaca allows you to add other grades from teacher-graded assignments. We correlate these to difficulty-adjusted quiz scores using linear regression. Thus, the accuracy of our quiz scores is based on the consensus in the teaching profession, rather than one arbitrary authority.
Now, we have to ask ourselves how accurate the result of one single quiz might be anyway. There are fundamental limits: consider the hypothetical case of a quiz that contains only one question, which may be got right or wrong. Does a correct answer equate to a Grade 9? An incorrect to a Grade 1? Of course not. A single answer tells you almost nothing. In practice, the absolute minimum you should rely on is 50 questions spread across at least three separate quizzes.
Finally, what accuracy can you expect from a single session with any assessment tool? Everyone has good and bad days; teenagers take it to extremes. If you really want Yacapaca to give you grades that you can rely on, my recommendation is that you use it little and often, building up a picture over time. Use the Progress Chart and the Parents’ Report to discover averages and trends in the data.
So is Yacapaca accurate? Yes, provided you use it intelligently.
Before half term, I sent a small survey to 10,000 teachers in UK secondary schools. I already knew that a popular “life after levels” alternative to NC Levels at KS3 was to extend GCSE grading all the way down to Y7, but I needed more detail to help plan our future grading strategy at Yacapaca. We got 201 responses – 2% is very creditable for an online survey. There were only three questions, and here are the results:
Just over half – 53.3% use GCSE 9-1 at KS3. I don’t know what proportion of the “in future” respondents will use it at KS3, but if we assume half of them, it pushes the total usage up to 60.5%. Like or lump, it’s becoming the default for KS3.
Oh dear me. How on earth are teachers supposed to communicate student progress to each other, when two thirds record it on one basis, and one third on an entirely different one?
What makes it worse is that, as you can see below, it is not at all clear that many teachers even know there are two bases in common use. Too many teachers are likely to be labouring under the delusion that their way universal. I leave the kind of disastrous cockups that might ensue
to your imagination.
What else is important to you about GCSE 9-1 grading?
This open question was designed to illuminate the thinking behind the raw statistics, and indeed it did. I don’t have permission to credit these quotes, sorry:
There is a lot of confusion over the grading system. Accurate mapping from A* – G is not available
- nor can it be, due to Grade 9 being norm-referenced. At the top end of the ability spectrum, you can only predict a GCSE grade if you have national data for that cohort.
The difficulty is that we don’t truly know the accuracy of what we are doing as the way the governmental has set this up has made it quite impossible to judge 1-9 outcomes. We can only really guess.
It’s grade inflation proof.
- They promised us that, didn’t they. Actually, only Grade 9 is norm-referenced and therefore grade inflation proof. After a few years, student results will just pile up at Grade 8 instead.
There is no link to the IDEAL / Blooms theory, or if there is nobody in my school is aware of it.
- This is a very interesting point. NC level descriptors focused attention and teaching effort onto high-order skills such as evaluation and understanding. One of the very unwelcome consequences of pulling GCSE grading further down the school is that it will focus teaching back onto knowledge – the very bottom of Benjamin Bloom’s pyramid.
It is a total Joke. Nobody appears to know what is going on. It is a grade now or a scaled down Y11 grade. I’m already completely sick of it and haven’t started to use it properly.