Can computers really mark exams? Benefits of ELT automated assessments

app Languages
Hands typing at a laptop with symbols

Automated assessment, including the use of Artificial Intelligence (AI), is one of the latest education tech solutions. It speeds up exam marking times, removes human biases, and is as accurate and at least as reliable as human examiners. As innovations go, this one is a real game-changer for teachers and students. 

However, it has understandably been met with many questions and sometimes skepticism in the ELT community – can computers really mark speaking and writing exams accurately? 

The answer is a resounding yes. Students from all parts of the world already take AI-graded tests.  aԻ Versanttests – for example – provide unbiased, fair and fast automated scoring for speaking and writing exams – irrespective of where the test takers live, or what their accent or gender is. 

This article will explain the main processes involved in AI automated scoring and make the point that AI technologies are built on the foundations of consistent expert human judgments. So, let’s clear up the confusion around automated scoring and AI and look into how it can help teachers and students alike. 

AI versus traditional automated scoring

First of all, let’s distinguish between traditional automated scoring and AI. When we talk about automated scoring, generally, we mean scoring items that are either multiple-choice or cloze items. You may have to reorder sentences, choose from a drop-down list, insert a missing word- that sort of thing. These question types are designed to test particular skills and automated scoring ensures that they can be marked quickly and accurately every time.

While automatically scored items like these can be used to assess receptive skills such as listening and reading comprehension, they cannot mark the productive skills of writing and speaking. Every student's response in writing and speaking items will be different, so how can computers mark them?

This is where AI comes in. 

We hear a lot about how AI is increasingly being used in areas where there is a need to deal with large amounts of unstructured data, effectively and 100% accurately – like in medical diagnostics, for example. In language testing, AI uses specialized computer software to grade written and oral tests. 

How AI is used to score speaking exams

The first step is to build an acoustic model for each language that can recognize speech and convert it into waveforms and text. While this technology used to be very unusual, most of our smartphones can do this now. 

These acoustic models are then trained to score every single prompt or item on a test. We do this by using human expert raters to score the items first, using double marking. They score hundreds of oral responses for each item, and these ‘Standards’ are then used to train the engine. 

Next, we validate the trained engine by feeding in many more human-marked items, and check that the machine scores are very highly correlated to the human scores. If this doesn’t happen for any item, we remove it, as it must match the standard set by human markers. We expect a correlation of between .95-.99. That means that tests will be marked between 95-99% exactly the same as human-marked samples. 

This is incredibly high compared to the reliability of human-marked speaking tests. In essence, we use a group of highly expert human raters to train the AI engine, and then their standard is replicated time after time.  

How AI is used to score writing exams

Our AI writing scoring uses a technology called . LSA is a natural language processing technique that can analyze and score writing, based on the meaning behind words – and not just their superficial characteristics. 

Similarly to our speech recognition acoustic models, we first establish a language-specific text recognition model. We feed a large amount of text into the system, and LSA uses artificial intelligence to learn the patterns of how words relate to each other and are used in, for example, the English language. 

Once the language model has been established, we train the engine to score every written item on a test. As in speaking items, we do this by using human expert raters to score the items first, using double marking. They score many hundreds of written responses for each item, and these ‘Standards’ are then used to train the engine. We then validate the trained engine by feeding in many more human-marked items, and check that the machine scores are very highly correlated to the human scores. 

The benchmark is always the expert human scores. If our AI system doesn’t closely match the scores given by human markers, we remove the item, as it is essential to match the standard set by human markers.

AI’s ability to mark multiple traits 

One of the challenges human markers face in scoring speaking and written items is assessing many traits on a single item. For example, when assessing and scoring speaking, they may need to give separate scores for content, fluency and pronunciation. 

In written responses, markers may need to score a piece of writing for vocabulary, style and grammar. Effectively, they may need to mark every single item at least three times, maybe more. However, once we have trained the AI systems on every trait score in speaking and writing, they can then mark items on any number of traits instantaneously – and without error. 

AI’s lack of bias

A fundamental premise for any test is that no advantage or disadvantage should be given to any candidate. In other words, there should be no positive or negative bias. This can be very difficult to achieve in human-marked speaking and written assessments. In fact, candidates often feel they may have received a different score if someone else had heard them or read their work.

Our AI systems eradicate the issue of bias. This is done by ensuring our speaking and writing AI systems are trained on an extensive range of human accents and writing types. 

We don’t want perfect native-speaking accents or writing styles to train our engines. We use representative non-native samples from across the world. When we initially set up our AI systems for speaking and writing scoring, we trialed our items and trained our engines using millions of student responses. We continue to do this now as new items are developed.

The benefits of AI automated assessment

There is nothing wrong with hand-marking homework tests and exams. In fact, it is essential for teachers to get to know their students and provide personal feedback and advice. However, manually correcting hundreds of tests, daily or weekly, can be repetitive, time-consuming, not always reliable and takes time away from working alongside students in the classroom. The use of AI in formative and summative assessments can increase assessed practice time for students and reduce the marking load for teachers.

Language learning takes time, lots of time to progress to high levels of proficiency. The blended use of AI can:

  • address the increasing importance of formative assessmentto drive personalized learning and diagnostic assessment feedback 

  • allow students to practice and get instant feedback inside and outside of allocated teaching time

  • address the issue of teacher workload

  • create a virtuous combination between humans and machines, taking advantage of what humans do best and what machines do best. 

  • provide fair, fast and unbiased summative assessment scores in high-stakes testing.

We hope this article has answered a few burning questions about how AI is used to assess speaking and writing in our language tests. An interesting quote from Fei-Fei Li, Chief scientist at Google and Stanford Professor describes AI like this:

“I often tell my students not to be misled by the name ‘artificial intelligence’ — there is nothing artificial about it; A.I. is made by humans, intended to behave [like] humans and, ultimately, to impact human lives and human society.”

AI in formative and summative assessments will never replace the role of teachers. AI will support teachers, provide endless opportunities for students to improve, and provide a solution to slow, unreliable and often unfair high-stakes assessments.

Examples of AI assessments in ELT

At app, we have developed a range of assessments using AI technology.

Versant

The Versant tests are a great tool to help establish language proficiency benchmarks in any school, organization or business. They are specifically designed for placement tests to determine the appropriate level for the learner.

PTE Academic

The  is aimed at those who need to prove their level of English for a university place, a job or a visa. It uses AI to score tests and results are available within five days. 

Read more about the use of AI in our learning and testing here.

Explore our courseware and exams

More blogs from app

  • A woman on her phone smiling outdoors in a city

    Understanding accents and their role in language learning

    By
    Reading time: 4 minutes

    Accents are a fascinating aspect of language that can reveal a lot about a person's background and experiences. For language learners, accents often become a central focus, sometimes to the detriment of more important language skills. In this post, we’ll explore what accents are, why people have them, and why intelligibility is more important than mastering an accent.

  • A female teacher sat in a classroom with a young child playing with toys

    Educating young learners: The importance of developing fine motor skills

    By Hawys Morgan
    Reading time: 5 minutes

    Teachers of young learners will be familiar with the importance of developing children’s fine motor skills. Building muscle strength, hand-eye coordination, and control are essential parts of students’ development during their early years.

    A holistic approach to education

    For young learners, their education is frequently holistic. A single game or activity might develop their speaking and listening skills, mathematical knowledge, social interaction, artistic development as well as motor skills. In the same way, improving fine motor skills can form a natural part of students’ English classes and can have the following benefits:

    • Builds concentration and self-esteem
    • Actively engages students in their learning
    • Develops the ability to switch between physical and mental activities
    • Improves social development and autonomy

    Below are some simple ways you can develop fine motor skills in your classroom.

    Building muscle strength

    Holding a pen for prolonged periods requires strong hand, wrist and arm muscles. If you’ve ever had to sit long hand-written exams, you will be familiar with tired and aching hand muscles.

    It is important that students develop their muscle strength so they have the stamina and control needed for writing. Kneading and rolling play dough is a fun way to build these muscles. Then, children can use their playdough creations to role-play scenarios such as having a picnic or visiting a farm. They could even model it into letter shapes.

    When singing songs or telling stories to young learners, teachers often incorporate actions to reinforce meaning. This is another opportunity to build those muscles. Children could also create shadow puppets with their hands to act out stories.

    Craft activities that involve scissors and gluing also help improve stamina and hand-eye coordination.

    Mark making

    Mark making is an important step in a child’s development, encouraging creativity and coordination.

    Try giving your students the opportunity to explore different mediums of mark making. For example, they could draw marks in trays of sand, jello, shaving foam, flour or rice. Talk to them about their sensory experience (Is it cold? Do you like it? What color is it?). They could start making marks with their whole hands and then, as their coordination improves, use an index finger. Then, they can start using a stick to make marks.

    As their fine motor skills develop, try using magic markers, chunky crayons and egg chalks to make large marks. Egg-shaped chalks are easier for young children to grip. Each straight line, wavy line and circle is another step on the road to learning how to write.

    Pincer grip

    As children move on from general mark making, it is important to establish a correct grip when using a writing tool. This is especially important if you expect your students to go on to use a cursive style of handwriting in the future.

    The pincer grip is when we hold something with our index finger and thumb. Peeling off and placing stickers, sorting building blocks and threading beads use this grip and provide opportunities for practicing colors, numbers, vocabulary and prepositions of place. Doing up buttons or zips uses this grasp too.

    While it may be second nature for adults, for children, this grip requires precise control of the small muscles in their hands, wrists and fingers. Challenge students to pick up items with large blunt tweezers or chopsticks and work on their English at the same time (What have you got? I have a car. It’s small.).

    Tripod grip

    The next stage of development for most children is the tripod grip. It uses three fingers: the thumb, index and middle fingers. It enables children to keep their wrist steady so they can make small, precise pencil movements.

    Some children find using a rubber pencil grip, or simply wrapping an elastic band or lump of playdough around the base of the pencil helps them maintain this grip.

    At this stage, students will be learning to hold and use writing tools such as pencils, crayons, markers, chalks and paintbrushes.

    Prewriting activities

    Prewriting activities offer more controlled fine motor skills practice. The usual progression is to start with straight lines, zig-zags, curved lines and diagonal lines. Then move on to tracing over circles and u-shapes. This is all essential preparation for writing letters and words.

    English courses for young learners are full of mazes, dot-to-dot, tracing and matching activities, all of which combine prewriting with learning English.

    When it comes to writing letters, it can be helpful if students begin by drawing the letter shape in the air or in sand. They then trace the letter shape with an index finger, before finally tracing over the letter with a pencil.

    Other writing systems

    When teaching students who use a different writing system in L1, establishing fine motor skills routines can make all the difference to students’ writing.

    It can be helpful to work on left-to-right orientation. For example, before they sit down to write, give students scarves to move from left to right in the air. These students will benefit from pre-writing activities that work on left-to-right, top-to-bottom patterns.

    Social development

    Doing up buttons, zips, and laces, turning on taps, cutting up food and opening boxes – all of these things improve students’ fine motor skills. They also promote autonomy and social development by helping students learn essential everyday life skills

    This has an added advantage for the teacher. The less time you have to spend helping students with these tasks, the more time you will have to work on other areas of their development. Not only that, it is also motivating for students to have that ‘I can do it all by myself!’ feeling.

    Students will be far better prepared to pick up a pen or pencil if they have developed strength, dexterity and stamina in their hands, wrists and arms. This will leave them free to concentrate on the language element of their classroom task, rather than the physical challenge it presents.

    AboutEnglish Code

    Support your young learners withEnglish Code,a 7-level course for 7-12-year-olds, offering 5 hours or more of English study per week. Available in both American English and British English versions, it promotes hands-on creative learning, investigation, fun projects and experiments.

    Focusing on project work and STEAM learning, children develop fine motor skills while learning how to collaborate and solve problems with their peers. Core functional language is at the forefront, giving students the vocabulary and tools they need to become confident speakers of English inside and outside the classroom.

  • A teenage girl smiling and doing her homework or studying at home.

    Creative writing exercises for English language learners

    By app Languages
    Reading time: 3.5 minutes

    Engaging in creative writing is one effective way to improve your English language skills. It allows for artistic expression and personal flair, making it a unique way to put your imagination into words. Additionally, creative writing can draw from the author’s life and experiences, providing a personal touch to the narrative. Whether you’re a beginner or an advanced learner, creative writing can help you enhance your vocabulary, grammar and overall language proficiency.

    Creative writing exercises can also benefit other forms of writing, such as business writing and technical writing, even though these are not typically considered creative writing. In this post, we’ll explore various creative writing exercises that are designed to make learning English fun and engaging, as well as improve writing skills.