Explaining computerized English testing in plain English

app Languages
a pair of hands typing at a laptop

Research has shown that automated scoring can give more reliable and objective results than human examiners when evaluating a person’s mastery of English. This is because an automated scoring system is impartial, unlike humans, who can be influenced by irrelevant factors such as a test taker’s appearance or body language. Additionally, automated scoring treats regional accents equally, unlike human examiners who may favor accents they are more familiar with. Automated scoring also allows individual features of a spoken or written test question response to be analyzed independent of one another, so that a weakness in one area of language does not affect the scoring of other areas.

was created in response to the demand for a more accurate, objective, secure and relevant test of English. Our automated scoring system is a central feature of the test, and vital to ensuring the delivery of accurate, objective and relevant results – no matter who the test-taker is or where the test is taken.

Development and validation of the scoring system to ensure accuracy

PTE Academic’s automated scoring system was developed after extensive research and field testing. A prototype test was developed and administered to a sample of more than 10,000 test takers from 158 different countries, speaking 126 different native languages. This data was collected and used to train the automated scoring engines for both the written and spoken PTE Academic items.

To do this, multiple trained human markers assess each answer. Those results are used as the training material for machine learning algorithms, similar to those used by systems like Google Search or Apple’s Siri. The model makes initial guesses as to the scores each response should get, then consults the actual scores to see well how it did, adjusts itself in a few directions, then goes through the training set over and over again, adjusting and improving until it arrives at a maximally correct solution – a solution that ideally gets very close to predicting the set of human ratings.

Once trained up and performing at a high level, this model is used as a marking algorithm, able to score new responses just like human markers would. Correlations between scores given by this system and trained human markers are quite high. The standard error of measurement between app’s system and a human rater is less than that between one human rater and another – in other words, the machine scores are more accurate than those given by a pair of human raters, because much of the bias and unreliability has been squeezed out of them. In general, you can think of a machine scoring system as one that takes the best stuff out of human ratings, then acts like an idealized human marker.

app conducts scoring validation studies to ensure that the machine scores are consistently comparable to ratings given by skilled human raters. Here, a new set of test-taker responses (never seen by the machine) are scored by both human raters and by the automated scoring system. Research has demonstrated that the automated scoring technology underlying PTE Academic produces scores comparable to those obtained from careful human experts. This means that the automated system “acts” like a human rater when assessing test takers’ language skills, but does so with a machine's precision, consistency and objectivity.

Scoring speaking responses with app’s Ordinate technology

The spoken portion of PTE Academic is automatically scored using app’s Ordinate technology. Ordinate technology results from years of research in speech recognition, statistical modeling, linguistics and testing theory. The technology uses a proprietary speech processing system that is specifically designed to analyze and automatically score speech from fluent and second-language English speakers. The Ordinate scoring system collects hundreds of pieces of information from the test takers’ spoken responses in addition to just the words, such as pace, timing and rhythm, as well as the power of their voice, emphasis, intonation and accuracy of pronunciation. It is trained to recognize even somewhat mispronounced words, and quickly evaluates the content, relevance and coherence of the response. In particular, the meaning of the spoken response is evaluated, making it possible for these models to assess whether or not what was said deserves a high score.

Scoring writing responses with Intelligent Essay Assessor™ (IEA)

The written portion of PTE Academic is scored using the Intelligent Essay Assessor™ (IEA), an automated scoring tool powered by app’s state-of-the-art Knowledge Analysis Technologies™ (KAT) engine. Based on more than 20 years of research and development, the KAT engine automatically evaluates the meaning of text, such as an essay written by a student in response to a particular prompt. The KAT engine evaluates writing as accurately as skilled human raters using a proprietary application of the mathematical approach known as Latent Semantic Analysis (LSA). LSA evaluates the meaning of language by analyzing large bodies of relevant text and their meanings. Therefore, using LSA, the KAT engine can understand the meaning of text much like a human.

What aspects of English does PTE Academic assess?

Written scoring

Spoken scoring

  • Word choice
  • Grammar and mechanics
  • Progression of ideas
  • Organization
  • Style, tone
  • Paragraph structure
  • Development, coherence
  • Point of view
  • Task completion
  • Sentence mastery
  • Content
  • Vocabulary
  • Accuracy
  • Pronunciation
  • Intonation
  • Fluency
  • Expressiveness
  • Pragmatics

More blogs from app

  • Understand your English level

    Por Sam Colley
    Reading time: 4 minutes

    Learning English as a second language is a journey that can be fun and tough. A key part of this journey is knowing your current skill level. The Global Scale of English (GSE) helps learners check their skills. The GSE is a scale from 10 to 90 that measures English ability. It gives clear information about what learners can do at each level. In this blog, we’ll look at how to find out your English level using GSE scores, levels, and "I can..." statements.

    Why understanding your English level matters

    You may wonder, ‘Why is it important for me to know my language level’? If you start studying without knowing your skill level, you might feel overwhelmed, frustrated, or find learning too easy and be put off or not make any learning progress.
    Knowing your English level helps you in many ways, such as to:

    1. Set realistic goals: Tailor your learning objectives to your current abilities.
    2. Choose appropriate materials: Select books, courses, and resources that match your proficiency.
    3. Track progress: Measure improvement over time and stay motivated.

    GSE levels and what they mean

    The GSE levels range from Starter to Expert, each with specific "I can..." statements that describe what you should be able to do at that level. Here’s a breakdown of each level, along with some practical examples:

  • Students sat ina library studying with laptops in front of them chatting to eachother

    Teaching engaging exam classes for teenagers

    Por Billie Jago
    Reading time: 4 minutes

    Teachers all over the world know just how challenging it can be to catch their students’ interest and keep them engaged - and it’s true whether you’re teaching online or in a real-world classroom.

    Students have different learning motivations; some may be working towards their exam because they want to, and some because they have to, and the repetitiveness of going over exam tasks can often lead to boredom and a lack of interest in the lesson.

    So, what can we do to increase students’ motivation and add variation to our classes to maintain interest?

    Engage students by adding differentiation to task types

    We first need to consider the four main skills and consider how to differentiate how we deliver exam tasks and how we have students complete them.

    Speaking - A communicative, freer practice activity to encourage peer feedback.

    Put students into pairs and assign them as A and B. Set up the classroom so pairs of chairs are facing each other - if you’re teaching online, put students in individual breakaway rooms.

    Hand out (or digitally distribute) the first part of a speaking exam, which is often about ‘getting to know you’. Have student A’s act as the examiner and B’s as the candidate.

    Set a visible timer according to the exam timings and have students work their way through the questions, simulating a real-life exam. Have ‘the examiners’ think of something their partner does well and something they think they could improve. You can even distribute the marking scheme and allow them to use this as a basis for their peer feedback. Once time is up, ask student B’s to move to the next ‘examiner’ for the next part of the speaking test. Continue this way, then ask students to switch roles.

    Note: If you teach online and your teaching platforms allow it, you can record the conversations and have students review their own performances. However, for privacy reasons, do not save these videos.

    Listening – A student-centered, online activity to practice listening for detail or summarising.

    Ask pairs of students to set up individual online conference call accounts on a platform like Teams or Zoom.

    Have pairs call each other without the video on and tell each other a story or a description of something that has happened for their partner to listen to. This could be a show they’ve watched, an album they’ve listened to, or a holiday they’ve been on, for example. Ask students to write a summary of what their partner has said, or get them to write specific information (numbers, or correctly spelt words) such as character or song names or stats, for example. Begin the next class by sharing what students heard. Students can also record the conversations without video for further review and reflection afterwards.

    Writing –A story-writing group activity to encourage peer learning.

    Give each student a piece of paper and have them draw a face at the top of the page. Ask them to give a name to the face, then write five adjectives about their appearance and five about their personality. You could also have them write five adjectives to describe where the story is set (place).

    Give the story’s opening sentence to the class, e.g. It was a cold, dark night and… then ask students to write their character’s name + was, and then have them finish the sentence. Pass the stories around the class so that each student can add a sentence each time, using the vocabulary at the top of the page to help them.

    Reading –A timed, keyword-based activity to help students with gist.

    Distribute a copy of a text to students. Ask them to scan the text to find specific words that you give them, related to the topic. For example, if the text is about the world of work, ask students to find as many jobs or workplace words as they can in the set amount of time. Have students raise their hands or stand up when they have their answers, award points, and have a whole class discussion on where the words are and how they relate to the comprehension questions or the understanding of the text as a whole.

    All 4 skills –A dynamic activity to get students moving.

    Set up a circuit-style activity with different ‘stations’ around the classroom, for example:

    • Listening
    • Reading
    • Writing (1 paragraph)
    • Use of English (or grammar/vocabulary).

    Set a timer for students to attempt one part from this exam paper, then have them move round to the next station. This activity can be used to introduce students to certain exam tasks, or a way to challenge students once they’ve built their confidence in certain areas.

  • Students sat at a desk looking at a textbook together, with a teacher pointing to it

    Real-world English: How GSE Job Profiles bridge learning and work

    Por Sara Davila
    Reading time: 5 minutes

    Did you know that improving your English proficiency can increase earning potential by up to 50% and safeguard your career against AI? Recent research by app highlights that English is not just a skill but a career-defining advantage in today’s globalized workplace. For millions of adult learners, the journey from the classroom to the workplace requires more than general conversational abilities—it’s about gaining targeted, job-ready skills as quickly as possible. For English language educators, understanding what “jDz-𲹻” English is and how to identify “jDz-𲹻” skills can provide a significant advantage in ensuring learners are prepared to communicate effectively and collaborate with their future coworkers.

    That’s where the Global Scale of English (GSE) Job Profiles comes in. For educators and program developers, it offers a bridge between real-world job skills and the English learners need to perform them. Whether you’re creating programs for nursing assistants, hospitality workers, or IT professionals, this tool ensures that learners build the precise English skills they need to thrive in their roles.

    Let’s walk through how to create a GSE Job Profile and explore its practical use for building programs that align with today’s professional realities.