Yacapaca

How many questions should the ideal quiz contain?

When we first built Yacapaca, a quiz was an encapsulated object of which every question would be presented to the student, albeit in random sequence. When improvements in schools’ infrastructure made it possible to move to dynamic delivery, we took advantage of that to allow authors to specify that only a proportion of the questions should be shown each time. This resulted in assessments that were more formative, more reliable and much better at measuring whole-class progression.

But it was also a mistake.

Specifically, the mistake was to ask the authors to make that decision on behalf of teachers they did not know. When I realised this, I put a fix on the development schedule and, finally, this has worked its way to the top. The software is now in the process of being written.

In future, teachers will be able to set the number of questions to be shown in a particular quiz to anything from 1 to the maximum number of questions in that particular question bank. Problem solved…

…almost. Knowing that new users nearly always accept defaults, I needed to make sure that the default value will give you the best teaching experience. Easy enough; just ask the existing users. I sent out a poll to a small sample of users, and here is what they said:

The chart looks like it has two horns, one at 10 questions per quiz and one at 20. Dilemma!

At first glance, it looks like 20 is the peak of the main curve, with 10 as an outlier that can be discarded. I nearly went with 20, until I looked at how Google Forms had based the chart. It only shows counts for which there are data. This looks pretty, but skews the interpretation. So I rebased it to a linear presentation.

Not so pretty, but a better decision aid. Preferences actually group more around the 10-15 range. To understand what these data really meant, I looked into the comments. I had asked users if they had ever had problems with quizzes that were the wrong length for their needs. Yes they had! 7 had found quizzes too long, 3 had found them too short, and 2 reported having had both problems. That is a small sample, but the bias is clear. Long quizzes are a bigger problem than short ones.

So, short it is. When you assign quizzes from now on, you will be able to decide the number of questions for yourself. If you don’t decide, you’ll get 10.

There is one extra enhancement, which I’m not sure will get into the feature launch straight away. Each time you manually set the question count, that number will be remembered as your personal default for the future. Questions counts will also carry over when you Assign Again.