Our algorithm for converting quiz percentages into National Curriculum levels was originally based on a cross-grading exercise with some 2,400 students. We have just finished re-calibrating the algorithm with almost 100 times that number – 220,962 to be exact.
We collected the data by asking every teacher the approximate average level of their class. As a teacher, you don’t know this exactly, but we can be confident that if we gather enough data (8,093 different student sets in our case) then the errors will cancel each other out. We also eliminated those teacher gradings that were so far off our calculated value that we suspected they had been set without much thought. We then correlated this to the results from the ICT Baseline tests. These were selected because they are generally set and run in accordance with a tight prescription that delivers reliable results.
You can see the outcome in the graph: the new correlation is the orange line, the old is brown. The exercise largely validated the existing settings, the exceptions being levels 2 to 4 where our results had been a little optimistic.
We are working hard to gather, and validate, more data. As far as I am aware, Yacapaca is the only organisation in the UK capable of collating this much data on day-to-day student attainment, and I am determined that we put it to good use.
Yacapaca 