Misleading question statistics

I threw this question into the 2014 Christmas quiz with no real expectation that anyone would get it. The (possibility of a) Technological Singularity is something every kid needs to grow up being aware of, so this was my contribution to building awareness.

Fake certainty

But look! 40% got it! That is double the guess rate, so we can conclude that one student in 5 actually knew the answer.

Or can we? Continue reading

Pair Match analysis tutorial

Pair match questions can be used to test for understanding of sequence and relationship in quite subtle ways, but they serve you poorly if you do not know how to analyse the results.

Below is an analysis of a pair match (AKA Drag and Drop) question from the Computing Baseline (High level). This is the author’s view, using data from about 1,500 responses. The teacher’s view shows only the data from Continue reading

How to calibrate quiz percentages to levels or grades

calibrationIf you look in your gradebook for any given student set, you will see that all the results are reported in the grade scheme you chose for that set. It’s easy to take that for granted and not think about how it is achieved – at least I hope it is, because we have worked hard to make the enormous complexity of that task invisible to casual users.

Although teachers very rarely challenge the accuracy of Yacapaca results, I do occasionally get asked about their Continue reading