Funny exam question answers

I thought you might enjoy some of answers your student have been giving to the short-text questions in our J276 GCSE Practice Exam module.

Actually, I don’t know whose students these are. The students’ responses are completely anonymised when we see them. As the first results to each question come in, we analyse the performance of the auto-marking algorithm and tweak the scoring rubrics to improve performance. In the process, we discover that student humour is alive and well…

    • Describe the features of RAM. (2)
      It can breed with sheep, they are fluffy and have good horns which is good because they may lose there memory but when playing games with it, it is always powerful.
    • Describe file management (2)
      Hire Wenger for the management and then put him in a folder.
    • Describe why multicore processors run faster than single. (2)
      More cores, more instructions can be processed at once, it runs faster, it grows legs and bolts it out the door.
    • Non-volatile memory holds information on a _____basis. (1)
      Permanent, non-temporary, infinite, till death do we part, long term, eg supercalifragalisticexpialidocious.
    • Volatile memory holds information on a _____ basis. (1)
      Temporary, finite, not for forever, short term, e.g. minuscule or a contractual wobbly knobbly bobbly.
    • Identify the two main types of CPU architecture. (2)
      Harvard, Von Neumann, Turing, Gothic, Baroque, Neo-Classical, Minimalist.
    • Explain how the performance of a CPU can be improved. (3)
      More RAM. More Cache. More Cores. Higher Clock Speed. Giving it a Cookie. Bribing it with Chocolate Cake. Adding some interesting lighting. Better Choreography.



How we cracked the auto-marking of GCSE short-text responses


Courtesy of

The mainstay of most GCSE exams is the short-text question. For our new GCSE Practice Exams, we have developed a free-text marking engine that is robust enough to deal with real GCSE questions and, more importantly, real students. That system is now (March 2018) in beta, and performing well above expectations.

Short-text automarking has traditionally been regarded as nigh-on impossible by the assessment community. Short-text occupies that difficult transition zone between ‘objective’ (e.g. multiple choice) methods that are relatively straightforward to mark, and long-text essay type questions that can be marked with reasonable accuracy using syntactical rules and semantic analysis.  Whilst several teams have tried to build a production-capable short-text automarker, we appear to be the first to have succeeded.

We came to the project with two advantages other teams did not have. Firstly, ten years experience of real-world automated assessment. Secondly, a huge community of willing volunteers to test our prototypes and provide feedback. Here, in simplified form, are the problems we faced, and the solutions we found to them:

  1. Deal with students’ spelling

Student spelling ranges from dictionary-perfect to random-character-generator. Experienced teachers become expert at extracting the intended meaning from even the most tortured English.

Yacapaca does the same. We took the most brutally efficient autospell available and enhanced it with multiple specialist dictionaries covering subject vocabulary, current student-speak, stop words and more.

  1. Extract the meaning

A human moderator, fundamentally, judges whether the student understands the subject of the question. Yacapaca uses artificial intelligence to emulate this process. Only emulate, mind you. Artificial Intelligence does not really work remotely like human intelligence.

  1. Score it by machine

There are thousands of possible ways to express any given concept. There are thousands more possible ways to say something superficially-similar that actually indicates a complete lack of understanding. To build a scoring mechanism that can finish between the two, we had to devise a unique scripting language.

  1. Iterate

The first lesson from our beta-testers was that working across multiple schools required a much wider vocabulary than working within a single school. Even after a question is published, we follow a strict schedule of manually reviewing and updating the scoring rubrics.

  1. Provide formative feedback

Of course students need to understand how to improve. After each quiz, we show them the scoring rubrics so they are able to improve. Because of the large number of questions and the randomised approach, a student can expect to only occasionally meet the same question twice. Knowing that this is a possibility is one of the central motivations to pay attention to feedback.

Setting up and managing automarked short-text questions requires significant training and resources. For those reasons they will remain limited to our professionally-produced premium content. While that is not free, it works out a very great deal cheaper than paying a suitably-qualified person to do the marking. And if that person happens to be you, it frees you up to do more important tasks.

GCSE Exam Practice modules coming in April

Exam results

This April, we are launching a new series of professionally written and edited online practice GCSE exams, entirely auto-marked.

The aim is to raise GCSE results by at least one grade level for students who practice regularly.


  • All GCSE question types, including multiple choice, short-text and long-text
  • Exam-timed questions
  • Instant, automatic marking
  • Formative feedback
  • Student motivation features
  • Select questions by keyword, topic and difficulty

There is a huge pile of work to do between now and then. New software modules to be written – some of them using advanced AI. A whole authoring and editing process has to be created and populated with talented individuals. But there is nothing we’ve not done before. What this project does is for the first time pull together skills we have developed over the last 25 years into a single, amazing, product.

I’m excited!

Is Yacapaca’s GCSE 9-1 grading accurate?


My thanks to Bobby Grewal at Four Dwellings Academy for the awkward question.

Let’s start by asking what it is we are trying to measure. At Yacapaca, we have opted for “current” grading, as that’s what the majority of our users want. This means that we are attempting to answer the question:

  • If the student were to take the GCSE exam today,
  • and they had studied all the content to the same level as the content of this quiz
  • then what grade might they be expected to achieve?

Without actually throwing students as young as 11 into a real GCSE, how can we test whether our answers are accurate? Not only is that impractical; we can’t even (yet) back-calculate from real GCSE results. So we have to calibrate against some existing scheme for which we have data that can be validated.

Fortunately, there are two sources, NC Levels and GCSE A*-G. There are published conversion tables between each of these and the GCSE 9-1 grades. My preferred version is the QEGS Points Conversion Wheel.


The next question becomes “how do we know that our original NC and GCSE gradings were accurate. The answer is that we crowdsourced the calibration from thousands of teachers. As well as quizzes, Yacapaca allows you to add other grades from teacher-graded assignments. We correlate these to difficulty-adjusted quiz scores using linear regression. Thus, the accuracy of our quiz scores is based on the consensus in the teaching profession, rather than one arbitrary authority.

Now, we have to ask ourselves how accurate the result of one single quiz might be anyway. There are fundamental limits: consider the hypothetical case of a quiz that contains only one question, which may be got right or wrong. Does a correct answer equate to a Grade 9? An incorrect to a Grade 1? Of course not. A single answer tells you almost nothing. In practice, the absolute minimum you should rely on is 50 questions spread across at least three separate quizzes.

Finally, what accuracy can you expect from a single session with any assessment tool? Everyone has good and bad days; teenagers take it to extremes. If you really want Yacapaca to give you grades that you can rely on, my recommendation is that you use it little and often, building up a picture over time. Use the Progress Chart and the Parents’ Report to discover averages and trends in the data.

So is Yacapaca accurate? Yes, provided you use it intelligently.

Disapplying National Curriculum levels: the official Yacapaca response

Yacapaca delivers criterion-referenced assessments. This makes it very useful for doing things like predicting exam grades, because exams are based on tightly-standardised criteria. A Grade C is the same in Barnsley as it is in Basingstoke. At the upper secondary stage (KS4 in England) we have been able to support a wide spectrum of qualifications, and given teachers a reliable tool with which to measure progression.

The English National Curriculum levels have allowed us to do the same at Key Stage 3. The levels are sufficiently well defined that we have been able to Continue reading