The ‘exam paper’ has been around for the last 14 centuries and it is still going strong. The way things are going, the exam-paper looks likely to outlast even the news-paper.
But it shouldn’t. Paper-based exams are a hideously-unreliable method of assessing competence. Here are some of the major reliability problems of paper-based exams:
No control for state of mind
I have put this first because it bugs me the most. Student A got a good sleep, had a good breakfast, got a great pep talk from his mum. Student B is on her period and had a row with her boyfriend last night. Guess which one is going to Oxford. Even if you control for long-term biases such as socioeconomic status, making a high-stakes decision based on a sample taken at a single point of time is criminally negligent.
Demand for consistency results in testing only knowledge
Exam boards place tremendous emphasis on ensuring that any two arbitrary moderators would return the same mark for the same piece of work. This results in mark schemes that specify the exact words that should written to attain a given mark. No matter the intent behind the question, these words can always be rote-learned. In fact, rote learning seems to be the most efficient route to a high grade in most exams. Exam boards are perfectly aware of this, but achieving consistency is an existential issue for them. They are trapped.
A single modality: writing
Whilst alternatives such as oral exams do exist, 90% of the questions in 90% of exams require a written response. Understanding of an issue might well be better demonstrated in some other way, but that is just hard luck because the exam system is built around the (hand)written word and any alternatives are tacked on the side in a way that makes them even more expensive.
Restricted sampling of content
Teachers who get good results spend considerable time weighing the probability that each given piece of content will be in this year’s exam. No need to teach something that won’t be. The exam tail wags the learning dog.
Slow and expensive
The only time a typical adult handwrites a communication and has it physically delivered to its recipient is at Christmas. The world has moved on simply because pen and paper makes everything so damned expensive. If we got rid of that expense, we could spend the money on a better system.
New technology gives us an enormous range of possible solutions to the high-stakes assessment problem. The objection that has so far prevented them from even being tried so far is “but they require technology that not everybody has”. Yes they do, but that technology is now so cheap and ubiquitous that it is time to make the commitment and demand that the laggards catch up. The size of the problem justifies some sacrifice.
So what should replace paper-based exams?
We don’t need to search for one perfect solution. To the contrary, we should be embracing a rich mix of assessment methods to fit particular subjects and circumstances. They should in general have the following characteristics:
- Start assessing from Day 1 of a course. Sample briefly, frequently and regularly to build an accurate picture of each student over time.
- Use methods that can test understanding and the ability to evaluate. These might for example include paired comparisons, chatbots or observed conversations.
- Routinely include non-written components such as speech or video. Use monitored peer assessment so as not to soak moderator time.
- Assess formatively. Summative testing eats into learning time, but formative assessment becomes part of the learning process itself and is essentially zero-footprint.
- Do it all online. Cut costs, shorten timeframes. Use the savings to improve quality.
I have seen and used (in some cases built) systems that achieve each of the above, and that have successfully resolved every objection that I could come up with. They exist, they work routinely in other assessment contexts, and they are ready to be applied to high-stakes academic assessments.
Resolving the big objection
“Schools don’t have enough computers and they can’t trust their connectivity.” Well shoot, everyone else has solved this; why haven’t schools? There is no valid excuse. I have enumerated elsewhere how much money schools are wasting on manual marking and laid out a strategy by which they can acquire all the hardware they need with a payback of one year.
What we still need is for an exam board to to step up and start offering qualifications that are assessed exclusively online and that are structured to offer clear benefits over traditional paper exams. It will only take one to break the logjam. The question now is, who wants first-mover advantage?
8 responses to “It is long past time to abandon paper-based examinations”
and btw here is reason #6, from 2008!
I teach A Level and am also a senior assessor for a leading exam board yet I agree with most of what you say. Universities ar moving away from exams and essays yet A Levels seem to be more and more focused on this. Definitely time for a change. The obsession with exams is driven by government ideology and ministers who seem to want to drive education backwards to the 1950s rather than embrace the oppotunities offered by new technology.
I remember sitting in a meeting hearing a colleague inform us that the maths department had ‘cracked the code’. They had come up with a ranking list so that pupils could be placed in groups according to ability in our 8-form school. Our eager deputy head immediately declared that future ‘setting’ in our science groups will follow maths ranking lists.
Great, said my key stage 3 co-ordinator, do you want to give you our ranking list based on our eci of module tests that our students take every half term. I have results in a spreadsheet! One look at that spreadsheet showed that we had come pretty close to formative assessment!
It was a pity that they never took up our recommendation that they should ditch the maths ranking lists and use science data in stead. That was in 1988 I think.
Good to see that things have not moved on much!
[…] It is long past time to abandon paper-based examinations | Yacapaca […]
[…] on from my rant about paper-based exams, Ty Goddard of EdTech UK asked me for some current examples of online assessment. Spoiled for […]
[…] is not a gimmick. Education has been gamified at least since the first exams were introduced around 600 AD. The current game does not work terribly well but, as the estate […]
Sorry. I’m a senior examiner and that is simply not my experience of examining science. Exams do not test “only knowledge”. A student only giving rote learned answers might get a maximum of 40% of the marks if they got everything right and answered the question set. Exams have to conform to assessment objectives and knowledge is only AO1.
Peter, thanks for your comment. How do you tell the difference between a student who understands the issue, and one who writes the same answer having rote-learned it?