Can computers really mark exams? Benefits of ELT automated assessments

app Languages
Hands typing at a laptop with symbols

Automated assessment, including the use of Artificial Intelligence (AI), is one of the latest education tech solutions. It speeds up exam marking times, removes human biases, and is as accurate and at least as reliable as human examiners. As innovations go, this one is a real game-changer for teachers and students. 

However, it has understandably been met with many questions and sometimes skepticism in the ELT community – can computers really mark speaking and writing exams accurately? 

The answer is a resounding yes. Students from all parts of the world already take AI-graded tests.  aԻ Versanttests – for example – provide unbiased, fair and fast automated scoring for speaking and writing exams – irrespective of where the test takers live, or what their accent or gender is. 

This article will explain the main processes involved in AI automated scoring and make the point that AI technologies are built on the foundations of consistent expert human judgments. So, let’s clear up the confusion around automated scoring and AI and look into how it can help teachers and students alike. 

AI versus traditional automated scoring

First of all, let’s distinguish between traditional automated scoring and AI. When we talk about automated scoring, generally, we mean scoring items that are either multiple-choice or cloze items. You may have to reorder sentences, choose from a drop-down list, insert a missing word- that sort of thing. These question types are designed to test particular skills and automated scoring ensures that they can be marked quickly and accurately every time.

While automatically scored items like these can be used to assess receptive skills such as listening and reading comprehension, they cannot mark the productive skills of writing and speaking. Every student's response in writing and speaking items will be different, so how can computers mark them?

This is where AI comes in. 

We hear a lot about how AI is increasingly being used in areas where there is a need to deal with large amounts of unstructured data, effectively and 100% accurately – like in medical diagnostics, for example. In language testing, AI uses specialized computer software to grade written and oral tests. 

How AI is used to score speaking exams

The first step is to build an acoustic model for each language that can recognize speech and convert it into waveforms and text. While this technology used to be very unusual, most of our smartphones can do this now. 

These acoustic models are then trained to score every single prompt or item on a test. We do this by using human expert raters to score the items first, using double marking. They score hundreds of oral responses for each item, and these ‘Standards’ are then used to train the engine. 

Next, we validate the trained engine by feeding in many more human-marked items, and check that the machine scores are very highly correlated to the human scores. If this doesn’t happen for any item, we remove it, as it must match the standard set by human markers. We expect a correlation of between .95-.99. That means that tests will be marked between 95-99% exactly the same as human-marked samples. 

This is incredibly high compared to the reliability of human-marked speaking tests. In essence, we use a group of highly expert human raters to train the AI engine, and then their standard is replicated time after time.  

How AI is used to score writing exams

Our AI writing scoring uses a technology called . LSA is a natural language processing technique that can analyze and score writing, based on the meaning behind words – and not just their superficial characteristics. 

Similarly to our speech recognition acoustic models, we first establish a language-specific text recognition model. We feed a large amount of text into the system, and LSA uses artificial intelligence to learn the patterns of how words relate to each other and are used in, for example, the English language. 

Once the language model has been established, we train the engine to score every written item on a test. As in speaking items, we do this by using human expert raters to score the items first, using double marking. They score many hundreds of written responses for each item, and these ‘Standards’ are then used to train the engine. We then validate the trained engine by feeding in many more human-marked items, and check that the machine scores are very highly correlated to the human scores. 

The benchmark is always the expert human scores. If our AI system doesn’t closely match the scores given by human markers, we remove the item, as it is essential to match the standard set by human markers.

AI’s ability to mark multiple traits 

One of the challenges human markers face in scoring speaking and written items is assessing many traits on a single item. For example, when assessing and scoring speaking, they may need to give separate scores for content, fluency and pronunciation. 

In written responses, markers may need to score a piece of writing for vocabulary, style and grammar. Effectively, they may need to mark every single item at least three times, maybe more. However, once we have trained the AI systems on every trait score in speaking and writing, they can then mark items on any number of traits instantaneously – and without error. 

AI’s lack of bias

A fundamental premise for any test is that no advantage or disadvantage should be given to any candidate. In other words, there should be no positive or negative bias. This can be very difficult to achieve in human-marked speaking and written assessments. In fact, candidates often feel they may have received a different score if someone else had heard them or read their work.

Our AI systems eradicate the issue of bias. This is done by ensuring our speaking and writing AI systems are trained on an extensive range of human accents and writing types. 

We don’t want perfect native-speaking accents or writing styles to train our engines. We use representative non-native samples from across the world. When we initially set up our AI systems for speaking and writing scoring, we trialed our items and trained our engines using millions of student responses. We continue to do this now as new items are developed.

The benefits of AI automated assessment

There is nothing wrong with hand-marking homework tests and exams. In fact, it is essential for teachers to get to know their students and provide personal feedback and advice. However, manually correcting hundreds of tests, daily or weekly, can be repetitive, time-consuming, not always reliable and takes time away from working alongside students in the classroom. The use of AI in formative and summative assessments can increase assessed practice time for students and reduce the marking load for teachers.

Language learning takes time, lots of time to progress to high levels of proficiency. The blended use of AI can:

  • address the increasing importance of formative assessmentto drive personalized learning and diagnostic assessment feedback 

  • allow students to practice and get instant feedback inside and outside of allocated teaching time

  • address the issue of teacher workload

  • create a virtuous combination between humans and machines, taking advantage of what humans do best and what machines do best. 

  • provide fair, fast and unbiased summative assessment scores in high-stakes testing.

We hope this article has answered a few burning questions about how AI is used to assess speaking and writing in our language tests. An interesting quote from Fei-Fei Li, Chief scientist at Google and Stanford Professor describes AI like this:

“I often tell my students not to be misled by the name ‘artificial intelligence’ — there is nothing artificial about it; A.I. is made by humans, intended to behave [like] humans and, ultimately, to impact human lives and human society.”

AI in formative and summative assessments will never replace the role of teachers. AI will support teachers, provide endless opportunities for students to improve, and provide a solution to slow, unreliable and often unfair high-stakes assessments.

Examples of AI assessments in ELT

At app, we have developed a range of assessments using AI technology.

Versant

The Versant tests are a great tool to help establish language proficiency benchmarks in any school, organization or business. They are specifically designed for placement tests to determine the appropriate level for the learner.

PTE Academic

The  is aimed at those who need to prove their level of English for a university place, a job or a visa. It uses AI to score tests and results are available within five days. 

app English International Certificate (PEIC)

app English International Certificate (PEIC) also uses automated assessment technology. With a two-hour test available on-demand to take at home or at school (or at a secure test center). Using a combination of advanced speech recognition and exam grading technology and the expertise of professional ELT exam markers worldwide, our patented software can measure English language ability.

Read more about the use of AI in our learning and testing here, or if you're wondering which English test is right for your students make sure to check out our post 'Which exam is right for my students?'.

Explore our courseware and exams

More blogs from app

  • Work colleagues talking together in the office

    Maximising workforce potential: The top benefits of language training

    Por app Languages
    Reading time: 9 minutes

    In today's globally connected business landscape, language training is not merely an added advantage but a critical component for companies seeking to maximize their team's potential. Being proficient in multiple languages enables effective communication, promotes cross-cultural understanding, and enhances networking capabilities.

    As businesses expand their operations globally, the proficiency to communicate in the native languages of clients and partners becomes crucial. This essential skill, proficiency in spoken languages, fosters trust and paves the way for more successful negotiations. Linguistic adaptability is becoming a pivotal requisite for organizations that aim to thrive in international commerce's dynamic and diverse environment.

    Understanding the concept of language training

    Acquiring a second language through language learning involves a systematic approach. It focuses on studying one language with the aim of enhancing communication skills in professional, educational and personal settings. This form of education transcends mere memorization and language study of vocabulary and grammar in a single language. It strives to foster a profound understanding of linguistic subtleties and cultural aspects across multiple languages.

    Language learning mastery accelerates critical thinking and adaptation to communication challenges, resulting in stronger, more meaningful connections in a multilingual, interconnected world. Mastering more than one language in today's diverse business world is not simply about translation; it's about gaining a competitive edge by understanding the nuances and cultural contexts that can inform and enhance global relationships.

    Consequently, language learning not only equips individuals with the tools required for effective dialogue but also sharpens cognitive abilities, paves the way for greater empathy towards other languages and cultures, and opens a doorway to a wealth of new personal and professional opportunities.

    The direct benefits of corporate language training for employees

    Enhanced communication skills

    Through language training, employees develop better listening abilities, pronunciation and vocabulary that collectively improve their overall communication proficiency. They gain a deeper understanding of the subtleties of language, allowing them to convey messages more clearly and comprehend others with increased accuracy.

    Language learning is crucial in fostering more profound connections with colleagues and clients.Learning languages goes beyond mere verbal exchange; it immerses individuals in the subtleties of different cultures, fostering a global mindset and sensitivity that are invaluable assets in today's international market. Individuals can cultivate more meaningful relationships by showcasing empathy and a genuine interest in cultural engagement with other cultures and other languages.

    Boosted confidence

    The fluency attained through dedicated language training significantly elevates an employee's self-assurance in their professional capabilities. Confident employees are more likely to engage in discussions, present ideas articulately, and assert their viewpoints in negotiations—key catalysts in driving innovation and business growth. Achieving fluency in learning a second language or foreign mother tongue is a commendable milestone that fosters employees' sense of accomplishment and self-worth.

    Furthermore, through additional language learning, employees not only enhance their communication abilities but also increase their propensity to establish rapport with international clients and partners. This, in turn, fosters more fruitful and harmonious professional relationships. In essence, language proficiency in more than just one other language, empowers individuals to operate beyond their comfort zones, thereby driving both personal and professional growth.

    Career advancement opportunities

    Language skills are not just tools for effective communication—they can be a catalyst for career advancement.

    Acquiring proficiency in a first or second language, a foreign tongue or an additional second language can also open the door to new roles within the company, such as international liaison or translator, which may have been unattainable otherwise. Mastering more than one language not only signifies an individual's ability to engage in varied linguistic exchanges but also demonstrates a commitment to personal development and an understanding of the evolving demands of the global marketplace.

    Employees with multilingual capabilities are often perceived as valuable assets who can seamlessly interact with diverse markets, making them prime candidates for promotions and leadership positions.

    Furthermore, as companies increasingly value global collaboration, being able to negotiate and network in a different culture and other languages can lead to being entrusted with more strategic projects and overseas postings. These opportunities broaden an individual's experience of the global economy and exposure to learning a second or third language, preparing them for a variety of influential roles within the ever-expanding global business arena.

    Cognitive benefits of language learning

    The process of acquiring and learning a new language thus does much more than merely expand an individual’s ability to communicate in only one language; it also provides significant cognitive enhancements.

    As individuals learn and use a second language, they engage in mental exercises that strengthen their brain and the new language's neural pathways, leading to better memory retention and recall of the new language.

    This cognitive exercise extends beyond language acquisition, enhancing the brain's ability to handle complex situations and solve problems more efficiently. Bilingual or multilingual individuals often demonstrate enhanced proficiency in recognizing essential components of complex problem-solving challenges and crafting innovative solutions. This ability stems from their adeptness and other skills acquired through language diversity.

    The practice of switching between languages enhances multitasking abilities, making language learners more effective in navigating tasks that require an attention span juggling multiple priorities. These cognitive side benefits of learning can have profound implications, contributing to an individual's mental agility and resilience in both personal and professional contexts. You can read more in our post 'Being bilingual can help keep your brain in good condition'.

  • A group of children at a desk playing a baord game with a teacher smiling looking over them

    Preparing for the PEIC YL oral test

    Por app Languages

    Younger learners may find speaking in English comes easily to them, more so than reading and writing. However, they need to be well-prepared and familiar with what they have to do in order to be successful in an exam. Let’s look at a few ways we can prepare our students for the International Certificate Young Learners (PEIC YL) oral test and have fun along the way!

    The test

    The PEIC YL oral test includes two speaking tasks that have an emphasis on real-life communication. Throughout the four levels, the format of the speaking tasks remains the same, with questions and topics suitable for each level and age group.

    Learners take the oral test in groups of five, with one examiner, who gives instructions and assesses the learners.

    • The first speaking task consists of a question and answer activity played as a board game. In their group of five, students take turns to throw a dice and move around the board/cards. When they land on a square, they read out a question which they should direct at another student, who then responds. Each question has a corresponding picture, which helps scaffold the task and give clues to what the question means.
    • In the second task, each student gives a short talk about a topic. Students take turns to pick a topic card and then talk about the topic for one minute. The other students in the group then ask questions related to the topic.

    For both parts of the test students are assessed on their vocabulary, grammar, and pronunciation.

    Classroom activities

    There are lots of ways in which you can prepare your learners for the test in class. The game-like format of the speaking test makes practicing the tasks an ideal end-of-lesson activity. Here are a few ideas:

    Play board games

    Playing board games will allow learners to get used to the functional language they will need to play games, e.g. "It’s your turn" / "Where’s the dice?".

    It will also give them plenty of practice in counting the squares in English, which they are expected to do in the test. You can use traditional board games designed for English language learners or make your own.

    Include questions on familiar topics such as family, clothes, or vacations.

    Make collaborative board games

    Young learners love to make things so why not have them make their own board games and cards? Put students in groups to make a board game for the other groups to play. You can provide a board template, have them write questions, and draw pictures in the squares.

    Make sure you have students write a rough draft of their questions first so that you can check for any errors. It can be helpful to write example questions and prompts on the board as a guide, e.g. 'What (sports) do you play at school?' 'How often do you …?'. Depending on the level, students will need to include questions in a variety of present/past/future tenses.

    Use student pictures

    To prepare for the short talk, give students plenty of practice at speaking for one minute. One way of making this more engaging is by having the students choose their own topics by talking about pictures they have drawn or taken.

    If students have mobile phones, you can ask them to choose a photo from the gallery and discuss it with a partner. If mobile phones are not an option, have them bring in some photos from home. These could be pictures taken on vacation, birthday parties, or at other celebrations.

    You could also write a list of topics on the board, have each student choose one, and draw a picture to illustrate it.

    Tips

    • Make sure students are used to playing board games and know rules and functional language.
    • Practice turn taking and asking/answering questions regularly.
    • Encourage students to listen carefully to questions and to read them thoroughly, paying attention to the verb forms used. They should aim to use the same verb form in their answer.
    • Give learners practice speaking in one-minute turns.
    • Use your course book unit topic to include similar speaking tasks in lessons.
    • Make a set of laminated picture cards for students to practice both parts of the test.
    • Write questions and topics on popsicle sticks or cards for fast finishers.

    The key to good test preparation is to make it a part of your regular lessons, rather than something you do in just the weeks before the test. The more familiar your students are with the tasks, the more relaxed they will be on the day of the test.

  • A woman sat in front of a laptop, with a pen in hand and a paper on the desk. She is looking at the camera

    International Certificate home testing: Benefits, rules and requirements

    Por app Languages

    Education has embraced remote and hybrid learning – and it looks like it’s here to stay as a more accessible option for students everywhere. For the same reasons, even higher-stakes English exams are proving it’s possible to throw off old-fashioned approaches and adapt to the changing needs and expectations of today’s test-takers. The app English International Certificate is available online (as well as in paper-based format) and can be taken in a home environment. It has a number of benefits for both teachers and test takers.

    In this post, we will look at why testing from home is a good option and explore the rules and requirements for the online app English International Certificate.

    5 benefits of remote language testing

    1. Flexibility to take the test anywhere

    One of the great things about taking the online International Certificate is that test-takers can do it from any private location with a reliable internet connection.

    While we offer the option to take the test in one of our authorized computer-based testing centers, we now also offer testing from home with the . Both delivery methods are backed by the same level of robust security, meaning International Certificate test results can be equally trusted regardless of where the test was taken.

    2. Fits into learners’ schedules

    The online International Certificate allows test-takers to schedule their 2-hour test at a time that fits into their timetable and is not restricted by location. This means that they don’t have to take time off work or education to take the test, or suffer the additional stress or inconvenience this can cause to their lives.

    Using just one system and one log-in, they register for the test, schedule it for whenever suits them and then, when the time comes, sign in to gain access to the online proctoring system.

    3. Easy-to-use testing platform

    The International Certificate is available through , the online assessment portal that brings teachers, test-takers and tests together in a single, user-friendly space.

    Test Hub is extremely straightforward to use for scheduling, taking tests and accessing results – while also delivering powerful, data-rich insights into test performance for both teachers and test-takers.

    Furthermore, we have clear resources showing new users how to navigate the platform, as well as online videos on to help learners familiarize themselves with the online International Certificate format.

    Also available through Test Hub is the Readiness Test, which predicts readiness to pass the International Certificate, as well as the Level and Benchmark Tests – assessments that teachers find invaluable for placing new students in the right class and tracking their progress to proficiency.

    4. Quicker to mark and issue results

    All tests are scored using an AI system. This speeds up the turnaround time for results, making the whole process much more efficient than the paper-based exam.

    Governments trust app’s world-leading scoring technology, businesses and universities globally and is powered by the expert input of thousands of skilled, experienced English teachers. Since 2022, the online International Certificate has been scored entirely using this system, bringing the score turnaround time to under 2 weeks – a fraction of the time taken to score the paper-based test.

    5. Accessible to all

    Running an assessment online means that many people from around the world can access it. So, this will allow people in some of the world's most remote areas to get a recognized English certificate and change their lives, as long as they meet the following requirements.

    app English International Certificate requirements

    There are a that test-takers need to be aware of when taking the exam from home. These include:

    • a computer- test-takers must have access to a computer or laptop for the test duration. We recommend using a personal rather than a work computer as the testing software uses a secure browser that restricts other programs from running. Therefore, firewalls or network settings on work computers may cause problems.
    • a government-issued ID - To sign in to the online testing system, test-takers must have their government-issued ID, such as a passport, identification card or driving license, ready to present.
    • internet access - As the test requires continuous audio and video streaming, test-takers need to maintain a good internet connection. Additionally, we recommend connecting to a hardwire ethernet cable. Alternatively, test-takers should ask others at home to avoid using the internet during the exam.
    • a headset- Test-takers need a wired headset to take the test, rather than a Bluetooth one. This needs to include a microphone. Make sure it is of good sound quality so that tasks can be completed effectively.
    • a webcam - Test-takers need a webcam as they’ll be monitored by our online proctoring system throughout. This is to ensure that they do not have help from another person, or their notes nearby.
    • a private testing space - Test-takers must ensure they have a quiet and comfortable location that is completely private while doing the test. The proctor must end the session if anyone appears – even momentarily – during the test. The candidate is the only person allowed to see and answer the test questions.

    The International Certificate testing rules

    There are a number of simple, yet critical rules to follow when taking the International Certificate test from home. These include:

    • minors need adult consent - Test-takers who are under 18, an adult must be present to show their ID and provide consent. Then they must leave the room for the test to begin.
    • test taker must remain in webcam view - The test is under 2 hours with one optional 10-minute break. Aside from this break, people taking the test must not stand up or leave their workspace.
    • remove watches - Test-takers must remove all watches and smart watches to be stored out of sight.
    • store personal items away - Test-takers must store all their personal items out of reach during the test. They can not use phones, books, pens, notes or whiteboards.
    • clear exam workspace - There shouldn’t be any clutter on the desk or workspace. The proctor will check that no prohibited items are in the room before beginning.
    • no handwritten notes allowed - Test-takers are not permitted to take handwritten notes during the test. They may use the notes feature in their browser to take notes.