Can LLMs Reliably Mark Responses to Exam Questions?

Can LLMs Reliably Mark Responses to Exam Questions? Insights from Omar Afzal’s Talk at Computing at School (CAS)

Recently, Omar Afzal, our Co-Founder and Head of Products, spoke at an event organised by Computing at School (CAS), sharing insights on how AI, particularly Large Language Models (LLMs) like ChatGPT, is transforming marking and feedback processes in education.

🎥Watch the Full Talk

We’ve embedded the video of Omar’s talk below for those who’d like to watch the complete session:

📌 The Challenge for Teachers

Teachers face immense pressure to provide timely feedback, but the workload is overwhelming. Omar illustrated this with a striking example:

• A class of 20 students, each attempting 20 questions, that’s 400 responses to mark and provide feedback on.
• Managing this at scale is nearly impossible for teachers alone. Yet our education system expects us to do this.

🤖 What Are Large Language Models (LLMs)?

LLMs, such as ChatGPT, are AI tools trained on vast amounts of text data, like super-smart assistants that have read millions of books and articles. These models can understand, interpret, and generate human-like language with surprising accuracy. In education, LLMs can read student responses and provide feedback as a human would—only faster and at scale.

Now, what do we mean by faster and at scale?

LLMs can generate 400 pieces of feedback—in under 5 seconds. Compare that to the hours it would take a teacher to achieve the same. It’s practically impossible for a teacher to do this manually.

📊 AI as a Personal Teacher for Every Student

Omar highlighted how AI acts like a personal teacher, offering real-time, personalised feedback. This rapid feedback helps students quickly identify mistakes and improve their understanding—essential for effective exam preparation.

🧪 Demonstrating AI Marking and Feedback with Teepee.ai

During the session, Omar showcased how Teepee.ai automates marking and feedback:

• Learners submit responses: Including short answers or essays.
• AI assigns marks: Using predefined rubrics.
• AI provides personalised feedback: Highlighting correct answers, errors, and improvement suggestions.

💡 Teachers Test AI Feedback in Real-Time

Teachers experienced AI feedback first-hand through live quizzes:

• OCR P2 Assessment tool Computer Science (GCSE)
• Edexcel P2 Assessment tool Computer Science (IGCSE)
• Edexcel P2 Assessment tool Computer Science (GCSE)
• Bizai tool (GCSE)

⚠️ Understanding AI’s Limitations

Teachers experienced AI feedback first-hand through live quizzes:

• Inconsistency: AI may provide differing feedback for similar responses.
• Context Gaps: AI may misinterpret nuanced answers. However, these challenges encourage students to interact with feedback critically, enhancing their learning experience.

🧠 Learning Through Challenging AI Feedback

Omar emphasised that AI limitations present a valuable learning opportunity for students. Teepee.ai’s recheck feature allows students to challenge feedback, explain their reasoning, and request a reassessment. This engagement fosters critical thinking, deeper understanding, and active participation in their learning process.

🔗 Connecting with CAS Insights

CAS has also covered this topic in their blog: Can LLMs Reliably Mark Responses to Exam Questions? Explore their perspective to gain further insights into AI’s role in education.

🛠️ How Teachers Can Use Teepee.ai in Their Classrooms

Omar concluded with practical ways to use Teepee.ai in classrooms:

• Assign Tasks: Create quizzes, homework, and mock practice exams.
• Provide Instant Feedback: Automatically generate personalised feedback & marking.
• Generate Reports: Track performance trends with automated reports.
• Curriculum-Aligned Assessments: Use Teepee’s question bank aligned with educational standards.

📝 Final Thoughts

The session underscored that while AI is not infallible, it empowers students to engage deeply with their learning and supports teachers by saving time. Teepee.ai automates the marking process, enabling teachers to focus on teaching.

Explore more about Teepee.ai: Learn More

About CAS: CAS (Computing at School) is a grassroots organisation supporting teachers in delivering computing education. We are grateful to CAS for hosting this insightful event and for their dedication to advancing technology in education.