We have just updated Yacapaca with what I think is one of the most profound changes we have made.
It’s called Self-Calibration. Your students will meet it as soon as they start their next quiz. Here’s why and how it works.
Charles Darwin said that “Ignorance more frequently begets confidence than does knowledge”. You probably know students who consistently overestimate their own abilities. It is an emotional defence mechanism known as the Dunning-Kruger effect. Students who over-estimate their own abilities are less likely to study and thus drive themselves into a vicious spiral of failure and denial.
Self-Calibration introduces a gentle but persistent way for students to build and ‘own’ a realistic view of their own abilities. At the start of each quiz, we ask the student to simply “Predict your score”.
At the end of the quiz we assign extra motivation points (that they can spend on new avatars) according to the accuracy of the prediction. Result? Over time, students start to care about self-calibration, and to get better at it.
Self-Calibration appears automatically on all Yacapaca quizzes; you do not need to do anything to enable it. Please do observe it in action, talk to the students about it and give me feedback below on how well it is working.
7 responses to ““Self-Calibration” is now a standard feature on all quizzes”
Something I have been reflecting on is relevant here. the questions I use are from a number of authors, in consequence the difficulty they present to the students varies. When I look at the charts, the display may not be reflecting a students progress, but the variability of the resources. Under these circumstances a students prediction is compromised.
i realise that the answer is to only use questions I have authored because in spite of their deficiencies they are consistent.
Chris, Yacapaca will calibrate the difficulty level of each quiz for reporting purposes, once it has enough data. You can improve the accuracy of the calibration by adding some Offline Assessment data to your gradebook for each student set. Yacapaca intelligently uses every scrap of data it can get to improve the reliability of its assessments.
In terms of the new student self-calibration feature, you have a point. Things may not be as bad as you fear, however. Authors generally select questions with a difficulty range, rejecting those that are too easy or hard for the intended audience. Also, most teachers allow the default two attempts. On the second attempt, the lure of extra points gives the student every motivation to be realistic about how much s/he can improve over the first, if at all.
[…] feature that asks students to predict their quiz score before they start. I have covered the theory and benefits of calibration elsewhere. What if we introduce a leaderboard for prediction accuracy? It avoids the […]
Hi Kara, I took a look at what you have done so far. I see two tags used in that quiz: mythology and dionysus.
I recommend tagging every question dionysus: this won’t help with analysis, but it will be very useful when searching for those questions in future. Mythology could be useful if you are planning to do more general quizzes across multiple myths.
If you were in conversation with a student, and you wanted to know how well they understand the underlying concepts you are trying to teach through Dionysus, what 2-3 issues would you try to draw them out on? Those should be your analysis tags. Working from concept to tag to question is a good discipline for making sure your quiz covers the required concepts with the right balance.
[…] At the start of each quiz, we ask students to predict their scores. After the quiz is over, we reward them with badge points, not for a high or a low score, but for the accuracy of their predictions. In this way we encourage students to really own a realistic expectation of their own ability, on which they can then build. We call it ‘self-calibration’ and you can read a fuller description here. […]
[…] scores, and reward them with badge points for accurate predictions. Over-confident students will use braggadocio to avoid revision, and thus not have to face themselves with their own limitations. Under-confident […]
[…] Students gain additional badge points for accurately predicting their score. Used consistently, this helps them develop a realistic estimate of the amount of work needed to get satisfactory results, and combats Dunning Kruger syndrome. […]