Yacapaca

Checkbox questions (aka “choose-n”) are our second most-popular question type. They are simple and versatile, and you can often replace a whole list of choose-1 questions with a simple choose-n. They also contain a few booby-traps for the unwary author, so it is worth studying them diligently before attempting to write one.

Understanding the structure and scoring of a checkbox question

Technically, a checkbox question is a collection of simple yes/no questions. We could have written the example above as four independent yes/no questions: “Are forest fires a problem typically caused by quarrying? Y/N” etc. But note that the answer that would earn the mark would be “no”: what we want the student to do is leave this box unchecked. Consequently leaving a box unchecked is as significant an action as checking it.

This ‘collection of y/n’ aspect leads on to the question of how to score a choose-n. When designing the question type, I initially considered

1. All or nothing: you have to get every y/n sub-question right in order to score the point.
2. Treat each sub-question separately. If you get half of them right, for example, you get half the marks.

(1) fails to inform the teacher about students who know some but not all of the answers. (2) suffers the problem of any y/n question: random ticking is rewarded with, on average, 50% marks. Neither performs well as summative assessment.

I found the solution by asking myself how I would rate a student who demonstrated this level of knowledge conversationally. My evaluation of the student would fall away exponentially with each mistake, so we wrote an algorithm that matched that intuition. It works like this.

• Get all the sub-questions right, 1 mark
• Get one wrong, 1/2 mark
• Get two wrong 1/4 mark
• …
• Get all wrong, 0 marks.

Tips for writing checkbox questions

• State your question in the positive “which of these is…” rather than “which of these is not…”. Negative questions confuse some students, and this pollutes your summative results. Are they low because the students don’t understand the content, or because they cannot decode the logic of the question?
• Use checkbox questions to replace negatively-phrased choose-1 questions. “Which of these is not a reason why…” becomes “Which of these are…”.
• Keep secret the number of ticks required for a fully-correct answer. Once again, this is about improving the accuracy of your summative assessment. You want to force students to consider each yes/no sub-question on its own merits.
• Keep it short. Include only text that absolutely has to be there; explanations live elsewhere.
• A checkbox question has only two possible feedback statements. One is for an all-correct response, the other is for any other condition. Remember that a student reading this second response has not necessarily got the whole thing wrong.

Examples with commentary

Each of these examples was pulled from the Yacapaca question bank. I used the preview from the Add Question search page. I am quite critical of some of these, so to save embarrassment I have not credited any of them.

There is a lot to like in this. It’s referenced very intelligently to a diagram. The options are just one word. And note the feedback: it gives useful extra information without giving the game away.

Again, nicely referenced to an illustration, and with nice short options. I feel it is too easy to guess that the correct response is to tick all the options, especially if that is something the author is prone to. We all have our patterns as authors, and the kids quickly pick up on them. I was also less thrilled with the very generic feedback statements. Histograms are such interesting things that I’m sure something more could have been done here.

This does fall into several traps (I am sure you can work them out at this stage) but I include it to give you a calibration point on the amount of text.  For you or I it’s fine; some students will panic when faced with this much.

This pair of questions make a very interesting case study. You can see that the author has written one, then duplicated it and flipped it around, thus giving himself two bites at the same cherry. For almost no extra work, he has improved the summative value of his quiz. I wondered if ‘wasted energy’ contravened my “state positives” rule, but on balance I feel it is legitimate here.

Oh, no, no, no! This is actually two entirely separate questions, one about an LDR and the other about a thermistor. When you unpack them, they are actually simple yes/no questions. I am strongly against obfuscating questions for the sake of it, both on aesthetic grounds and because it undermines the summative validity of the question.

This was the original version of the question I use at the top of the page. I edited it before presenting it. You can see this is really a choose-1 questions “which one of these is not….” that in that form gains nothing from being a choose-n. However, flip it round into positives and suddenly it works quite well.