John Davies from Kings School in Gütersloh asked me today about the scoring for Checkbox questions. It’s not obvious because the colour-coding in the Detail markbook is simply red or green, leading one to assume that a question would score a simple 1 or 0. But not so. A Checkbox can return a partial mark.
Why?
A Checkbox question is a list of items. For each item, the student must answer whether or not it fulfils a certain condition. In the example screenshot here, it’s whether the substance named would be found in a comet. Is it generally reasonable to give a student no score at all even if they got three of the four answers right? Most teachers would say ‘no’.
What about the opposite approach? As a checkbox question is really just a collection of yes/no questions, why not just split the mark equally between each item? Get 3 out of 4 right, and you get 75%, for example.
This, too, has problems. It is a characteristic of all multiple-choice that a random guess has a statistical chance gaining the mark. In a yes/no question, it’s 50%. In a quiz full of y/n questions, you’d expect even a sleep-deprived monkey to score 50%. That would be fine if all quizzes were calibrated to this expectation, (and always had enough questions to smooth out the randomness) but they are not. The vast majority of questions in Yacapaca are 4-option choose-1s, which have a 25% chance of a random answer being correct.
What’s more, there is a big difference in most teachers’ minds between ‘all right’ and ‘some wrong’. If you tell me the Union Jack is red, white and green, I’m not going to want to give you more than half a mark.
So I had to come up with a scheme that resolved both of these problems. And here it is:
- Get all correct: 1 point
- Get one wrong 1/2 point
- Get two wrong 1/4 point
- etc, up to
- Get all wrong 0 points
Some careful statistical profiling demonstrated that this simple scheme has about the same probability curve as a 4-option choose-1 question. Hence it can safely be mixed with the predominant question type, without skewing the results.
It also has the benefit of providing a consistent scheme however many checkboxes there are in the question, and it acknowledges the central principle of each checkbox being a logically-separate yes/no question.
So the next time one of your students challenges you on their mark from a quiz containing checkbox questions, point them to this post. And don’t forget to test them on their understanding, afterwards.
Yacapaca 