Can computers really mark exams? Benefits of ELT automated assessments

app Languages
Hands typing at a laptop with symbols

Automated assessment, including the use of Artificial Intelligence (AI), is one of the latest education tech solutions. It speeds up exam marking times, removes human biases, and is as accurate and at least as reliable as human examiners. As innovations go, this one is a real game-changer for teachers and students. 

However, it has understandably been met with many questions and sometimes skepticism in the ELT community – can computers really mark speaking and writing exams accurately? 

The answer is a resounding yes. Students from all parts of the world already take AI-graded tests.  aԻ Versanttests – for example – provide unbiased, fair and fast automated scoring for speaking and writing exams – irrespective of where the test takers live, or what their accent or gender is. 

This article will explain the main processes involved in AI automated scoring and make the point that AI technologies are built on the foundations of consistent expert human judgments. So, let’s clear up the confusion around automated scoring and AI and look into how it can help teachers and students alike. 

AI versus traditional automated scoring

First of all, let’s distinguish between traditional automated scoring and AI. When we talk about automated scoring, generally, we mean scoring items that are either multiple-choice or cloze items. You may have to reorder sentences, choose from a drop-down list, insert a missing word- that sort of thing. These question types are designed to test particular skills and automated scoring ensures that they can be marked quickly and accurately every time.

While automatically scored items like these can be used to assess receptive skills such as listening and reading comprehension, they cannot mark the productive skills of writing and speaking. Every student's response in writing and speaking items will be different, so how can computers mark them?

This is where AI comes in. 

We hear a lot about how AI is increasingly being used in areas where there is a need to deal with large amounts of unstructured data, effectively and 100% accurately – like in medical diagnostics, for example. In language testing, AI uses specialized computer software to grade written and oral tests. 

How AI is used to score speaking exams

The first step is to build an acoustic model for each language that can recognize speech and convert it into waveforms and text. While this technology used to be very unusual, most of our smartphones can do this now. 

These acoustic models are then trained to score every single prompt or item on a test. We do this by using human expert raters to score the items first, using double marking. They score hundreds of oral responses for each item, and these ‘Standards’ are then used to train the engine. 

Next, we validate the trained engine by feeding in many more human-marked items, and check that the machine scores are very highly correlated to the human scores. If this doesn’t happen for any item, we remove it, as it must match the standard set by human markers. We expect a correlation of between .95-.99. That means that tests will be marked between 95-99% exactly the same as human-marked samples. 

This is incredibly high compared to the reliability of human-marked speaking tests. In essence, we use a group of highly expert human raters to train the AI engine, and then their standard is replicated time after time.  

How AI is used to score writing exams

Our AI writing scoring uses a technology called . LSA is a natural language processing technique that can analyze and score writing, based on the meaning behind words – and not just their superficial characteristics. 

Similarly to our speech recognition acoustic models, we first establish a language-specific text recognition model. We feed a large amount of text into the system, and LSA uses artificial intelligence to learn the patterns of how words relate to each other and are used in, for example, the English language. 

Once the language model has been established, we train the engine to score every written item on a test. As in speaking items, we do this by using human expert raters to score the items first, using double marking. They score many hundreds of written responses for each item, and these ‘Standards’ are then used to train the engine. We then validate the trained engine by feeding in many more human-marked items, and check that the machine scores are very highly correlated to the human scores. 

The benchmark is always the expert human scores. If our AI system doesn’t closely match the scores given by human markers, we remove the item, as it is essential to match the standard set by human markers.

AI’s ability to mark multiple traits 

One of the challenges human markers face in scoring speaking and written items is assessing many traits on a single item. For example, when assessing and scoring speaking, they may need to give separate scores for content, fluency and pronunciation. 

In written responses, markers may need to score a piece of writing for vocabulary, style and grammar. Effectively, they may need to mark every single item at least three times, maybe more. However, once we have trained the AI systems on every trait score in speaking and writing, they can then mark items on any number of traits instantaneously – and without error. 

AI’s lack of bias

A fundamental premise for any test is that no advantage or disadvantage should be given to any candidate. In other words, there should be no positive or negative bias. This can be very difficult to achieve in human-marked speaking and written assessments. In fact, candidates often feel they may have received a different score if someone else had heard them or read their work.

Our AI systems eradicate the issue of bias. This is done by ensuring our speaking and writing AI systems are trained on an extensive range of human accents and writing types. 

We don’t want perfect native-speaking accents or writing styles to train our engines. We use representative non-native samples from across the world. When we initially set up our AI systems for speaking and writing scoring, we trialed our items and trained our engines using millions of student responses. We continue to do this now as new items are developed.

The benefits of AI automated assessment

There is nothing wrong with hand-marking homework tests and exams. In fact, it is essential for teachers to get to know their students and provide personal feedback and advice. However, manually correcting hundreds of tests, daily or weekly, can be repetitive, time-consuming, not always reliable and takes time away from working alongside students in the classroom. The use of AI in formative and summative assessments can increase assessed practice time for students and reduce the marking load for teachers.

Language learning takes time, lots of time to progress to high levels of proficiency. The blended use of AI can:

  • address the increasing importance of formative assessmentto drive personalized learning and diagnostic assessment feedback 

  • allow students to practice and get instant feedback inside and outside of allocated teaching time

  • address the issue of teacher workload

  • create a virtuous combination between humans and machines, taking advantage of what humans do best and what machines do best. 

  • provide fair, fast and unbiased summative assessment scores in high-stakes testing.

We hope this article has answered a few burning questions about how AI is used to assess speaking and writing in our language tests. An interesting quote from Fei-Fei Li, Chief scientist at Google and Stanford Professor describes AI like this:

“I often tell my students not to be misled by the name ‘artificial intelligence’ — there is nothing artificial about it; A.I. is made by humans, intended to behave [like] humans and, ultimately, to impact human lives and human society.”

AI in formative and summative assessments will never replace the role of teachers. AI will support teachers, provide endless opportunities for students to improve, and provide a solution to slow, unreliable and often unfair high-stakes assessments.

Examples of AI assessments in ELT

At app, we have developed a range of assessments using AI technology.

Versant

The Versant tests are a great tool to help establish language proficiency benchmarks in any school, organization or business. They are specifically designed for placement tests to determine the appropriate level for the learner.

PTE Academic

The  is aimed at those who need to prove their level of English for a university place, a job or a visa. It uses AI to score tests and results are available within five days. 

Read more about the use of AI in our learning and testing here.

Explore our courseware and exams

More blogs from app

  • A woman walking around outdoors with a camera

    The benefits of taking part in foreign exchange programs

    By
    Reading time: 4 minutes

    If you are a student or one considering a gap year, the option of foreign exchange programs may have crossed your mind or been mentioned to you. More and more young people are taking advantage of exchange programs and reaping the transformative benefits that come with it.

    Let's have a look at some of the reasons why taking part in an exchange program can be a life-changing experience and how it can help you more than you think.

    What is a foreign exchange program?

    A foreign exchange program is an educational initiative where students have the opportunity to study abroad and immerse themselves in a different culture for a specified period, typically ranging from a few weeks to an entire academic year. These programs facilitate cultural exchange by allowing students to attend foreign schools or universities, live with host families or in dormitories, and engage in activities that foster intercultural understanding and personal growth.

    The primary aim of a foreign exchange program is to provide students with a global perspective, enhance their language skills, and expose them to different educational systems and cultural practices. By stepping out of their comfort zones, students can develop independence, adaptability and a deeper appreciation for diversity.

    What are the benefits of a foreign exchange program?

    1. Cultural immersion and global perspective

    Embracing diversity

    One of the most immediate benefits of an exchange program is the opportunity to immerse yourself in a different culture. Experiencing new customs, traditions and ways of life firsthand fosters a deeper understanding and appreciation of global diversity. This cultural immersion helps break down stereotypes and broadens your worldview, making you more open-minded and adaptable.

    Develop cultural understanding

    Living in a foreign country teaches you how to navigate and respect different cultural norms and practices. This cultural competence is increasingly valued in our interconnected world, enhancing your ability to work effectively in diverse environments and making you a more attractive candidate in the global job market.

    2. Language proficiency

    Enhanced language skills

    For language learners, an exchange program is an opportunity to achieve fluency. Experiencing a language firsthand is one of the best ways to learn a new language. Being surrounded by fluent speakers provides constant practice and exposure, accelerating your language acquisition in ways that classroom learning alone cannot match. You'll develop better pronunciation, expand your vocabulary and gain confidence in your speaking abilities.

    Real-world communication

    Using a new language in everyday situations—whether it's ordering food, asking for directions, or making new friends—helps solidify your language skills in a practical context. This real-world communication practice is invaluable, ensuring that your language proficiency extends beyond textbooks and exams to actual, meaningful interactions.

    3. Academic and professional growth

    Academic enrichment

    Exchange programs often provide access to unique academic resources and teaching methods that differ from those in your home country. Exposure to new perspectives and approaches can deepen your understanding of your field of study and inspire new areas of interest. Additionally, studying abroad can enhance your academic credentials, making your CV stand out to future employers or academic institutions.

    Career opportunities

    Experience abroad signals to employers that you possess qualities like independence, adaptability and cross-cultural communication skills. These attributes are highly sought after in today’s job market. Moreover, the networking opportunities during your exchange can open doors to international internships, job placements and collaborations that might not have been available otherwise.

    4. Personal development

    Building independence and resilience

    Living away from home in a completely new environment challenges you to become more self-reliant and adaptable. You'll develop problem-solving skills, resilience and the ability to thrive outside of your comfort zone. These experiences build character and prepare you for future challenges, both personally and professionally.

    Forming lifelong connections

    The friendships and connections you make during your exchange program can last a lifetime. You'll meet people from various backgrounds, creating a global network of peers and mentors. These relationships can provide support, inspiration and opportunities long after your exchange program ends.

    Boosting confidence

    Successfully navigating life in a foreign country, mastering a new language, and achieving academic success abroad can significantly boost your confidence. This newfound self-assurance can positively impact all areas of your life, giving you the courage to pursue further opportunities and take on new challenges.

    5. Engaging with the local community

    Volunteer and community projects

    Many exchange programs encourage participants to engage with their host communities through volunteer work or community projects. This engagement allows you to give back to your host country, gain a deeper understanding of local issues, and develop a sense of global citizenship. It's a rewarding experience that fosters empathy and reinforces the importance of contributing to the wider world.

    Gaining valuable work experience

    Participating in volunteer and community projects during your exchange program can provide significant work experience that is highly attractive to future employers. These projects often involve teamwork, problem-solving and project management, all of which are essential skills in any professional setting.

    By contributing to local initiatives, you can demonstrate your ability to adapt to new environments, work with diverse teams, and handle responsibility. Furthermore, these experiences can fill gaps in your CV, showcasing your proactive approach to skill-building and community involvement. Engaging in meaningful projects not only supports your personal growth but also highlights your commitment to making a positive impact, a quality greatly valued in any career field.

    Where can I sign up for exchange programs?

    There are many organizations and programs that offer exchange opportunities for students and language learners. Some popular options include:

    • Study abroad programs through universities or colleges, which often have partnerships with foreign institutions. Check with your institution.
    • Government-sponsored programs such as Fulbright or Erasmus, (depending on your location).
    • Check non-profit organizations in your country, they may also offer exchange programs.

    Conclusion

    Taking part in an exchange program is a life-changing adventure that offers myriad benefits for students and language learners alike. From enhancing language proficiency and cultural awareness to boosting academic and career prospects, the experiences gained through an exchange program are invaluable. By stepping out of your comfort zone and into a new world, you open yourself up to endless possibilities, personal growth and a broader, more inclusive perspective on life.

    If you're a student or language learner considering an exchange program, seize the opportunity. The world awaits, and the benefits are boundless.

  • Friends studying together around one big table

    8 things you should try to avoid when learning English

    By
    Reading time: 4 minutes

    Learning a new language is an exciting and rewarding journey, but it can also come with its fair share of challenges. As English learners, it's important to recognize and overcome the common pitfalls that could hinder your progress. Here, we provide advice and guidance on what not to do to make your English learning experience as smooth and successful as possible.

  • Students sat at a table with a teacher stood with them interacting with them

    Why don’t my students speak English in class?

    By Silvia Minardi
    Reading time: 3 minutes

    Last year, I contributed to a national research project with an article titled “?”. The title originated from a concern expressed by a language teacher involved in the project, highlighting a common challenge faced by numerous language teachers. The difficulty of developing learners’ production and interaction skills is a well-known issue in language education.

    Large and increasingly diverse classes, limited time, and learners’ reluctance to speak in class are significant hurdles. During pair and group work, students often revert to their first language (L1), they lack confidence in speaking activities and end up avoiding all interaction in English. These observations are consistent with recent Global Scale of English (GSE) research findings, which indicate that 52% of English learners leave formal education without confidence in their speaking skills.

    Factors contributing to learners’ reluctance

    Several factors contribute to students’ reluctance to speak English in class. Psychological barriers such as lack of motivation, shyness, low self-confidence, fear of making mistakes, anxiety and concerns about negative evaluation play a crucial role. Linguistic challenges, including limited vocabulary, poor pronunciation, and insufficient grammatical skills, further exacerbate the problem.

    Task-related issues can also hinder speaking, especially when tasks are not well-matched to the learner’s proficiency level or focus more on accuracy than communication. Additionally, the classroom environment may not always be conducive to speaking, particularly for learners who need more time to formulate their thoughts before speaking.

    Positive teacher impact

    Fortunately, teachers can positively influence these intertwined factors. By creating a supportive classroom atmosphere and implementing well-designed tasks that prioritize communication over perfection, teachers can encourage reluctant students to participate more actively in speaking activities.

    Leveraging technology: Mondly by app

    One effective tool that can help address these challenges is Mondly by app. This learning companion is especially beneficial for learners who are hesitant to speak in class. Mondly by app offers over 500 minutes of speaking practice, encouraging learners to use English in real-life situations and tasks that prioritize action and communication over accuracy. This approach allows for mistakes - they are part of the game - thus fostering a positive mindset, which is essential if we want to enhance our learners’ speaking skills.

    AI-powered conversations

    A standout feature of Mondly by app is its AI-powered conversation capability, thanks to advanced speech recognition software. This flexibility helps build self-confidence and allows students to experiment with various production and interaction strategies.

    Comprehensive skill development

    Mondly by app is designed not only for speaking but also to develop all four language skills—listening, reading, writing and speaking—and is aligned with the Global Scale of English. The vocabulary for each topic is selected from the GSE vocabulary database, ensuring that learners are exposed to level-appropriate words and phrases.

    Integration into classroom teaching

    To facilitate the integration of Mondly by app into classroom teaching, three GSE mapping booklets have recently been published. These booklets cater to different proficiency levels:

    • Beginner (GSE range: 10-42 / CEFR level: A1-A2+)
    • Intermediate (GSE range: 43-58 / CEFR level: B1-B1+)
    • Advanced (GSE range: 59-75 / CEFR level: B2-B2+)

    These resources provide practical guidance on how to incorporate Mondly by app into lesson plans effectively, ensuring that the tool complements classroom activities and enhances overall language learning.

    Conclusion

    Encouraging students to speak English in class is a multifaceted challenge, but it is not insurmountable. By understanding the various factors that contribute to learners’ reluctance and leveraging innovative tools like Mondly by app, teachers can create a more engaging and supportive learning environment. This approach not only boosts students’ confidence in their speaking abilities but also fosters a more inclusive and interactive classroom atmosphere.

    Embracing technology and aligning teaching practices with modern educational standards, such as the Global Scale of English, can lead to significant improvements in language proficiency and student engagement.