Yacapaca

Dave Evans from Christleton High School is an excellent author you are going to be hearing a lot more of. The other day, he wrote in and asked:

Why is the score for a checkbox question a fraction of the number of possible responses rather than a fraction of the number of correct responses?

I’m pretty sure only about 1% of Yacapaquistas has noticed this, so if you have too, give yourself a pat on the back. Here’s why:

Reason #1

Suppose we only did count the number of correct ticks, and ignored items that were supposed to be left un-ticked? Your students would quickly (very quickly!) learn that they could pass these questions by ticking all the boxes. Students are so much more motivated to get good grades than they are to actually test their own knowledge, that checkbox questions would become worthless.

Reason #2

Logically, a Checkbox question is really a list of yes/no questions. You can see this more clearly with the following example.

Which colours does the Union Jack contain?

• [ ] Red
• [ ] Green
• [ ] White
• [ ] Blue

Can be re-written as:

• Does the Union Jack contain red? Yes/No
• Does the Union Jack contain green? Yes/No
• Does the Union Jack contain white? Yes/No
• Does the Union Jack contain blue? Yes/No

You can now see that each statement has a correct answer, either “Yes” or “No”. In a Checkbox question, this is indicated by either checking the box, or leaving it blank. The absence of the check is as significant as its presence.

So why not separate them into four discrete questions?

Sometimes, you want to group these yes/no questions just for neatness. Other times, it’s because you are testing knowledge of a combination. For example, would you want to give even a 50% mark to someone who said the Union Jack was red, white and green? Especially as the average mark from simply guessing yes/no questions is 50%?

There is a further refinement that you don’t notice until you analyse the final marks. The actual recorded score is calculating by halving your point for every incorrect option, thus:

• All correct: 1 point.
• One incorrect: half a point.
• Two incorrect: quarter of a point:
• etc, until
• None correct: no points.

This gives us a nice balance between over-rewarding simple guesswork and a punitive ‘no points unless completely correct’ approach. It also has the advantage that plain guesswork on 4-option checkbox questions (the most commonly-authored) will result in an average of 0.25 points per question. That’s the same average as you get with four-point multiple choice. This makes it much easier to calibrate a quiz comprising a mix of choose-one questions and checkbox questions.