The exams regulator Ofqual is launching a competition to find out if exams can be marked by artificial intelligence.
In a blog published today, the organisation said it wants to “understand whether there might be a role for AI in marking”, and is “particularly interested in whether using AI as a second marker or as a way of monitoring marking might help improve marking”.
The announcement follows warnings that the ongoing teacher recruitment and retention crisis is affecting examiner recruitment, and also comes after a report from Ofqual found large numbers of examiners are unhappy with a system used to standardise their marking.
In the blog, Beth Black, Ofqual’s director of research and analysis, said the organisation would conduct research including an “AI competition”.
This will involve senior human markers marking several thousand student essays multiple times. The responses will then be used to run a competition for “individuals and organisations with expertise in AI” to attempt to train an AI system to mark “similarly to the training set”.
“We can then test these AI systems on another set of essays (for which we know the marks, but the AI systems do not). We very much hope this competition will help stimulate and identify the very best practice in this field,” said Black.
“The results from this competition will help us undertake further subsequent research work – for example, modelling the impact of AI as a second marker or as a marker monitoring system.”
It is not the first time the use of AI in exam marking has been explored. Cambridge Assessment English, an international exam board, has already created two auto-markers for writing and speaking tests, and Pearson has looked at using AI for university exam-marking in the US. However, this is thought to be the first time the idea has been seriously considered for English school exams.
Black insisted Ofqual’s exploration of AI in marking was in its “early days”, but said the organisation felt it was “important to take some first steps on this in England, by beginning this exploratory research”.
“If there are genuine potential improvements, ways which might enhance marking quality, of course we want to know, so we can encourage the system to adopt such practices safely.
“Similarly, we want to have a deep understanding of the potential risks in operating such technology in our high stakes examinations.”
Ofqual is now looking for schools to provide pupils’ essay responses to a particular past question in mock tests.
Ofqual is now looking for schools to provide pupils’ essay responses to a particular past question in mock tests. In return, schools will be given examiner marks and annotations.
The regulator needs around 3,000 responses. Interested schools can email AI.Challenge@ofqual.gov.uk
The alleged “recruiment crisis” for examiners can be solved in short order by the examination boards.
1 Pay adequately, so that a reasonable rate of marking is capable of earning £20+ per hour.
2 Trat examiners well, not as numbers, but as professional colleagues.
3 Ensure that schools encourage their staff to work as examiners, not deter them.