Can computers really mark exams? Benefits of ELT automated assessments

app Languages
Hands typing at a laptop with symbols

Automated assessment, including the use of Artificial Intelligence (AI), is one of the latest education tech solutions. It speeds up exam marking times, removes human biases, and is as accurate and at least as reliable as human examiners. As innovations go, this one is a real game-changer for teachers and students. 

However, it has understandably been met with many questions and sometimes skepticism in the ELT community – can computers really mark speaking and writing exams accurately? 

The answer is a resounding yes. Students from all parts of the world already take AI-graded tests.  aԻ Versanttests – for example – provide unbiased, fair and fast automated scoring for speaking and writing exams – irrespective of where the test takers live, or what their accent or gender is. 

This article will explain the main processes involved in AI automated scoring and make the point that AI technologies are built on the foundations of consistent expert human judgments. So, let’s clear up the confusion around automated scoring and AI and look into how it can help teachers and students alike. 

AI versus traditional automated scoring

First of all, let’s distinguish between traditional automated scoring and AI. When we talk about automated scoring, generally, we mean scoring items that are either multiple-choice or cloze items. You may have to reorder sentences, choose from a drop-down list, insert a missing word- that sort of thing. These question types are designed to test particular skills and automated scoring ensures that they can be marked quickly and accurately every time.

While automatically scored items like these can be used to assess receptive skills such as listening and reading comprehension, they cannot mark the productive skills of writing and speaking. Every student's response in writing and speaking items will be different, so how can computers mark them?

This is where AI comes in. 

We hear a lot about how AI is increasingly being used in areas where there is a need to deal with large amounts of unstructured data, effectively and 100% accurately – like in medical diagnostics, for example. In language testing, AI uses specialized computer software to grade written and oral tests. 

How AI is used to score speaking exams

The first step is to build an acoustic model for each language that can recognize speech and convert it into waveforms and text. While this technology used to be very unusual, most of our smartphones can do this now. 

These acoustic models are then trained to score every single prompt or item on a test. We do this by using human expert raters to score the items first, using double marking. They score hundreds of oral responses for each item, and these ‘Standards’ are then used to train the engine. 

Next, we validate the trained engine by feeding in many more human-marked items, and check that the machine scores are very highly correlated to the human scores. If this doesn’t happen for any item, we remove it, as it must match the standard set by human markers. We expect a correlation of between .95-.99. That means that tests will be marked between 95-99% exactly the same as human-marked samples. 

This is incredibly high compared to the reliability of human-marked speaking tests. In essence, we use a group of highly expert human raters to train the AI engine, and then their standard is replicated time after time.  

How AI is used to score writing exams

Our AI writing scoring uses a technology called . LSA is a natural language processing technique that can analyze and score writing, based on the meaning behind words – and not just their superficial characteristics. 

Similarly to our speech recognition acoustic models, we first establish a language-specific text recognition model. We feed a large amount of text into the system, and LSA uses artificial intelligence to learn the patterns of how words relate to each other and are used in, for example, the English language. 

Once the language model has been established, we train the engine to score every written item on a test. As in speaking items, we do this by using human expert raters to score the items first, using double marking. They score many hundreds of written responses for each item, and these ‘Standards’ are then used to train the engine. We then validate the trained engine by feeding in many more human-marked items, and check that the machine scores are very highly correlated to the human scores. 

The benchmark is always the expert human scores. If our AI system doesn’t closely match the scores given by human markers, we remove the item, as it is essential to match the standard set by human markers.

AI’s ability to mark multiple traits 

One of the challenges human markers face in scoring speaking and written items is assessing many traits on a single item. For example, when assessing and scoring speaking, they may need to give separate scores for content, fluency and pronunciation. 

In written responses, markers may need to score a piece of writing for vocabulary, style and grammar. Effectively, they may need to mark every single item at least three times, maybe more. However, once we have trained the AI systems on every trait score in speaking and writing, they can then mark items on any number of traits instantaneously – and without error. 

AI’s lack of bias

A fundamental premise for any test is that no advantage or disadvantage should be given to any candidate. In other words, there should be no positive or negative bias. This can be very difficult to achieve in human-marked speaking and written assessments. In fact, candidates often feel they may have received a different score if someone else had heard them or read their work.

Our AI systems eradicate the issue of bias. This is done by ensuring our speaking and writing AI systems are trained on an extensive range of human accents and writing types. 

We don’t want perfect native-speaking accents or writing styles to train our engines. We use representative non-native samples from across the world. When we initially set up our AI systems for speaking and writing scoring, we trialed our items and trained our engines using millions of student responses. We continue to do this now as new items are developed.

The benefits of AI automated assessment

There is nothing wrong with hand-marking homework tests and exams. In fact, it is essential for teachers to get to know their students and provide personal feedback and advice. However, manually correcting hundreds of tests, daily or weekly, can be repetitive, time-consuming, not always reliable and takes time away from working alongside students in the classroom. The use of AI in formative and summative assessments can increase assessed practice time for students and reduce the marking load for teachers.

Language learning takes time, lots of time to progress to high levels of proficiency. The blended use of AI can:

  • address the increasing importance of formative assessmentto drive personalized learning and diagnostic assessment feedback 

  • allow students to practice and get instant feedback inside and outside of allocated teaching time

  • address the issue of teacher workload

  • create a virtuous combination between humans and machines, taking advantage of what humans do best and what machines do best. 

  • provide fair, fast and unbiased summative assessment scores in high-stakes testing.

We hope this article has answered a few burning questions about how AI is used to assess speaking and writing in our language tests. An interesting quote from Fei-Fei Li, Chief scientist at Google and Stanford Professor describes AI like this:

“I often tell my students not to be misled by the name ‘artificial intelligence’ — there is nothing artificial about it; A.I. is made by humans, intended to behave [like] humans and, ultimately, to impact human lives and human society.”

AI in formative and summative assessments will never replace the role of teachers. AI will support teachers, provide endless opportunities for students to improve, and provide a solution to slow, unreliable and often unfair high-stakes assessments.

Examples of AI assessments in ELT

At app, we have developed a range of assessments using AI technology.

Versant

The Versant tests are a great tool to help establish language proficiency benchmarks in any school, organization or business. They are specifically designed for placement tests to determine the appropriate level for the learner.

PTE Academic

The  is aimed at those who need to prove their level of English for a university place, a job or a visa. It uses AI to score tests and results are available within five days. 

Read more about the use of AI in our learning and testing here.

Explore our courseware and exams

More blogs from app

  • a woman stood in front of a noteboard, gesturing to it. The noteboard has different papers and graphs stuck to it.

    The importance of teachers professional development

    By Richard Cleeve

    There’s the saying, “There are two types of teachers with 20 years of teaching experience: the first are those with 20 years of experience and the second are those with one year’s experience repeated 20 times.”

    Some believe most teachers want to be the first kind of teacher – constantly evolving throughout their careers – rather than repeating the same classes. Additionally, taking professional development courses can help us reach these new heights.

    Doctors, engineers, lawyers and other professionals regularly have to seek training opportunities. This enables them to keep up to date with the latest technologies, innovations and changes in the wider world. It could be argued that teachers are no different.

    Benefits of professional development training

    There are several benefits to continued professional training for teachers. For instance, there’s always something new to learn as developments are happening regularly in the world of ELT. Whether it’s innovations in ed-tech, new research into how we learn languages or a new pedagogy, certain courses can help bring you up to speed with these!

    Undertaking further training will help you to deepen your knowledge and widen your expertise. So rather than simply repeating the same method of teaching a grammar point over again, you’ll learn new approaches. In turn, you’ll be better equipped to find the tools that work to help your students reach their learning goals.

    Another great thing about professional development is that it can lead to career progression and promotion. There are a number of courses that you can take to develop not only skills for teaching, but other roles in the ELT industry.
    For example, you can train to become a director of studies, specialize in business or academic English, or enter ELT publishing or management. There are many options to explore through further education!

    New trends in English language teaching

    To be the best teachers we can be, it's important to be aware of new trends in the field of ELT. Some of the latest ones include:

    Hybrid learning

    Gaining popularity in recent years, hybrid learning is an approach to teaching. Specifically, it involves some students attending class in person, while others join the class virtually from home. This means that teachers must be prepared to simultaneously teach both students in person and online.

    Flipped classroom

    A flipped classroom means students are introduced to content at home. Often teachers introduce this via videos and then practice working through the new knowledge in class. This is the opposite of the more traditional method where they are given new content at school, and complete assignments independently at home.

    Bite-sized learning

    One method of teaching that has become more common is bite-sized learning. For example, breaking down information into smaller chunks instead of having learners listen to long, uninterrupted sessions, and this helps students absorb information and keeps them engaged.

    If you’re not already familiar with these, there are lots of professional development courses that teach you how to use them in your classroom.

    Formal qualifications in ELT

    So what courses can you take? Here are some of the most common courses you can take for English language teaching.

    • CELTA (Certificate in Teaching English to Speakers of Other Languages) - Cambridge’s Level 5 qualification is an internationally recognized ELT course. While this certificate focuses on teaching adults, language academies accept it for both adults and young learners.
    • Trinity CertTESOL (Certificate in Teaching English to Speakers of Other Languages) - Like the CELTA, the Level 5 course is designed for people with little or no English teaching experience.
    • Trinity CertPT (Certificate for Practising Teachers) - Trinity’s Level 6 is designed for teachers who have already been teaching for a minimum of six months. It aims to support teachers' further development and help them improve their teaching practice.
    • DELTA (Diploma in Teaching English to Speakers of Other Languages) - Cambridge’s provides professional development for teachers with at least one year’s experience. This level 7 qualification is also designed for those who want to progress into more senior roles such as head of English and teacher training.
    • Trinity DipTESOL (Diploma in Teaching English to Speakers of Other Languages) - Trinity’s is designed for teachers who have reached a stage in their career where they’d like to progress to a more senior level. Like the DELTA, it’s an internationally respected level 7 qualification for experienced TEFL teachers.
    • Master’s degree - Master’s studies in English Language Teaching, TESOL, or applied linguistics, are great options to reach a more senior level in your career. Most people take them to become a director of studies or reach a management position.
    • Specialized courses - If there’s an area of teaching that you’d like to specialize in, plenty of short courses cover a range of topics. These include teaching online, teaching with technology, teaching business English and teaching very young children. Check out , or to see what they offer.

    If you’re unable to take a formal course, sharing ideas with colleagues is a great way to create professional development opportunities. You could exchange lesson plans and ask for feedback or input. There are plenty of podcasts, webinars and social media groups for teachers where you can find great tips and ideas from fellow teachers worldwide.

    Whichever route you decide, remember that learning is a lifelong journey – not only for your students but for yourself too! There are always new things to discover that will help you develop as a teacher.

  • Children sat at desks in a classroom with their hands up

    Lesson tips for Special Educational Needs

    By Richard Cleeve

    In this blog, James Laidler talks about his insights into how to plan lessons for neurodiverse students. James is a teacher and has been a Special Educational Needs (SEN) Coordinator for the past 18 years. He also discusses how important it is to consider your terminology, using phrases like ‘special learning powers’ or ‘neurodiversity’ to break down negative stereotypes. On top of this, he wants to help teachers and students recognize the strengths SEN students can bring to the classroom.

    James explores special needs education and what teachers can do to ensure their lessons are inclusive for all. A lot of these lesson tips are also great to apply to keep all students engaged, SEN or otherwise.

    Defining Special Educational Needs

    To define what Special Educational Needs (SEN) is, a child has SEN if they have a learning difficulty or disability that calls for special educational provision. Learners with such as autistic spectrum disorder (ASD), attention deficit hyperactivity disorder (ADHD), dyslexia, dyspraxia or anxiety disorders come under this framework.

    Inclusive lesson tips for neurodiverse students

    Although teachers want to create inclusive lessons, many feel ill-equipped to support neurodiverse students.To help, James offers some tips for lesson planning which aim to turn learning diversities into strengths:

    Attention Deficit Hyperactivity Disorder (ADHD)

    ADHD is a condition that can include symptoms such as inattentiveness, hyperactivity and impulsiveness. Students with this disorder may have a short attention span, constantly fidget, or act without thinking.

    Lesson tips for ADHD students:

    • Movement breaks – Students with ADHD may struggle to sit still for extended periods of time. Include short breaks in your lessons that offer them the opportunity to get up and move around at regular intervals.
    • Group work – To keep learners active and engaged, include group work in class. This means they don’t have to focus on the board for too long.
    • Dramatise lessons – A really effective activity is to . For example, students can act out role plays or other fun . It keeps them motivated, holds their attention and can be fun for all of the class.

    Dyslexia

    Dyslexia primarily affects the skills involved in accurate and fluent reading and spelling. It may affect a person’s phonological awareness, verbal memory and verbal processing speed. Lesson tips for dyslexic students include:

    • visual aids – Learners with dyslexia tend to have . Try bringing in pictures to illustrate ideas or add them to lengthy texts to help students when doing reading comprehension exercises.
    • font and spacing – When setting reading tasks, simply , enlarging font size, and double spacing is hugely beneficial to dyslexic students. Simply adapting the text can make their learning experience much easier.
    • text-to-speech software – Using a text-to-speech specialized software often provides significant support to those who struggle with reading or digesting text on computer screens – try or .

    Autism spectrum disorder (ASD)

    ASD is a developmental condition that involves challenges in social interaction, speech and nonverbal communication, and restricted or repetitive behaviors. The severity of symptoms is different in each person. Lesson tips for ASD students:

    • Encourage systematic skills – Often students with ASD may be more systematic than other students. This means they favor routines, regular processes, and predictable activities. Try bringing out these skills by asking students to spot patterns, analyze numbers or evaluate data.
    • Talk about interests – Autistic students may have specific interests they love to research. Engage them by getting them to or ask students to create projects on a topic they choose that they can present to the class.
    • Teaching online/blended learning – If you have a learner who is struggling socially at school, it may be an option to include hybrid or blended learning. This takes away the social and emotional challenges of school and people interaction, which can benefit ASD students.

    Anxiety disorders

    Anxiety disorders differ from normal feelings of nervousness or anxiousness, but rather involve intense fear or anxiety. in young people and greatly affects their ability to learn.

    Lesson tips for anxiety disorder students:

    • Changing language and terminology – Our education system is very exam driven, which can cause students to experience much stress. By simply offering reassurance, guidance, and motivation, you can help to reduce their feelings of anxiety.
    • Talk openly – Encourage learners to discuss their feelings if they struggle. They can do this with you, a classmate, or a support worker at the school. If they open up to you, focus on .
    • Mindfulness techniques – Try adding five minutes at the start of the day for guided meditation or breathing exercises. It may help students to begin the day in a calm and relaxed manner.
  • A overhead shot of a chalkboard with a cube on, with people around it with chalk and books

    The coding mindset: Benefits and activities

    By Richard Cleeve

    What is the coding mindset?

    Over the past decade, the ELT industry has placed more emphasis on soft skills. The focus has been on developing personal leadership qualities, creativity, problem-solving, teamwork and communication and collaboration skills. These are all essential skills for the future of work – and especially useful when students need to work better together and solve unexpected issues.

    A coding mindset encourages students to develop these essential soft skills – and practice them as a coder would. Teachers can use activities and tasks in the classroom that are based on this mindset to help students develop strategies to analyze, understand and solve problems.

    This is integral to computational thinking and is how computer programmers think when coding. Yes, the coding mindset is a way of thinking, but it does not directly relate to computer science. Instead, it follows the skills and mentality that coders and programmers use in their work. Following this mindset can and savvy when faced with challenges in their learning or daily lives.

    Four benefits of the coding mindset

    There are several benefits to developing this mindset:

    1. Gain creativity skills

    One significant benefit of this way of thinking is that students learn that not everything they try will work out just as they expect. In fact, it’s normal to fail several times when trying to solve problems.

    In working to find new strategies to work through challenges, students are also .

    Creativity was once synonymous with art, such as drawing or painting. However, this now means coming up with new ideas and is a skill that is .

    2. Learn how to cope in difficult times

    We live in a world where we experience constant change – and we need to be able to find ways of managing. A coding mindset teaches learners how to build resilience.

    By openly communicating with others, evaluating ideas and discussing a range of options, students will be able to work through uncertainties and confront challenges.

    Not only will this help students when coming up against stumbling blocks in their learning, but it will also benefit their day-to-day lives.

    3. Create risk takers

    We can all recognize that learning English isn’t easy and that students are bound to make mistakes.

    However, a coding mindset encourages students to take risks when approaching difficulties. It also helps language learners spot their mistakes and experiment with different options to find solutions.

    Ultimately, learners become more willing to take risks which they need to do to reach a higher level of proficiency.

    4. Develop the ability to overcome obstacles

    When approaching a task with a coding mindset, students will learn how to focus on the important information. They will filter out any irrelevant details and find ways around barriers.

    For example, if learners have to write a text about their last holiday in class, they could hit a wall if they don’t know how to use the third conditional to explain something. Rather than giving up, students with a coding mindset would use the grammar they do know to complete the task. For instance, they can continue with the past simple or past continuous, explaining their story in a different way.

    This encourages learners to focus on their strengths rather than weaknesses to overcome obstacles and keep going.

    Practical activities for use in the classroom

    There are several activities that teachers can use in the classroom to develop the coding mindset for their students. These include:

    Recognizing patterns

    If you teach in a classroom with a whiteboard, you can draw a series of colored circles on the board. The colors should follow a pattern that students must work out in small groups and then continue on the board.

    This simple exercise can be adapted for all levels and ages. You may even want to use flashcards with vocabulary, letters or number combinations.

    Giving instructions

    A great way to develop troubleshooting and problem-solving skills is by asking students to direct one another across the classroom. Put the learners into pairs and ask one of them to give directions and the other to follow.

    They can practice imperatives and language for directions, while they break down problems into smaller, more manageable parts.

    Treasure hunts

    Creating treasure hunts works particularly well with young learners. If you have access to an outside space, you can hide classroom objects or flashcards around the space and give students clues as to where to find them.

    You can also do this around the classroom or school if you cannot access the outdoors. This will help them to think systematically and follow instructions.

    Pixilation of pictures

    If you have access to an interactive whiteboard, another way to develop problem-solving skills is by selecting some pictures from the internet and blurring them with a pixilation tool.

    Choose vocabulary you’ve been working on in class, so students are already familiar with the topic. Show the pictures on the whiteboard and ask students to work in groups to guess what the pictures are.