Yacapaca

I threw this question into the 2014 Christmas quiz with no real expectation that anyone would get it. The (possibility of a) Technological Singularity is something every kid needs to grow up being aware of, so this was my contribution to building awareness.

But look! 40% got it! That is double the guess rate, so we can conclude that one student in 5 actually knew the answer.

Or can we?

I looked in the Peer Feedback log and not one student had written a feedback statement for this question, out of 520 who got it right. Not one. That signifies that the ones who got it right had a remarkably low level of confidence in their answers.

Is there a plausible explanation? There is, and basically it’s “careless authoring”. I think I wrote the distractors in such a way as to signal the required answer. 1 in 5 students picked up the signal, either consciously or subliminally.

Here is how it works. Look at the options in sequence:

Error #1: with numeric responses, authors will intuitively avoid putting the key at the extremes. It looks ‘more random’ hidden somewhere in the middle of the sequence. Immediately we have restricted the field to just two – 31 and 78.

Error #2: authors will often attempt to distract students with a distractor that is visually (or phonetically) very close to the key. This is fine if all distractors are equally close, but otherwise it signals that the key is one of a small set that are clearly related. In this case, 13 and 31 are the same digits just flipped round.

So the author (me, to my shame) signalled that the key was a member of (13, 31) and also a member of (31, 78). Not hard to work it out, after that!

I could have avoided either of these two errors with a little more care. Looking at it now, it might have been better as a Cloze question in the first place.