Explaining computerized English testing in plain English

app Languages
a pair of hands typing at a laptop

Research has shown that automated scoring can give more reliable and objective results than human examiners when evaluating a person’s mastery of English. This is because an automated scoring system is impartial, unlike humans, who can be influenced by irrelevant factors such as a test taker’s appearance or body language. Additionally, automated scoring treats regional accents equally, unlike human examiners who may favor accents they are more familiar with. Automated scoring also allows individual features of a spoken or written test question response to be analyzed independent of one another, so that a weakness in one area of language does not affect the scoring of other areas.

was created in response to the demand for a more accurate, objective, secure and relevant test of English. Our automated scoring system is a central feature of the test, and vital to ensuring the delivery of accurate, objective and relevant results – no matter who the test-taker is or where the test is taken.

Development and validation of the scoring system to ensure accuracy

PTE Academic’s automated scoring system was developed after extensive research and field testing. A prototype test was developed and administered to a sample of more than 10,000 test takers from 158 different countries, speaking 126 different native languages. This data was collected and used to train the automated scoring engines for both the written and spoken PTE Academic items.

To do this, multiple trained human markers assess each answer. Those results are used as the training material for machine learning algorithms, similar to those used by systems like Google Search or Apple’s Siri. The model makes initial guesses as to the scores each response should get, then consults the actual scores to see well how it did, adjusts itself in a few directions, then goes through the training set over and over again, adjusting and improving until it arrives at a maximally correct solution – a solution that ideally gets very close to predicting the set of human ratings.

Once trained up and performing at a high level, this model is used as a marking algorithm, able to score new responses just like human markers would. Correlations between scores given by this system and trained human markers are quite high. The standard error of measurement between app’s system and a human rater is less than that between one human rater and another – in other words, the machine scores are more accurate than those given by a pair of human raters, because much of the bias and unreliability has been squeezed out of them. In general, you can think of a machine scoring system as one that takes the best stuff out of human ratings, then acts like an idealized human marker.

app conducts scoring validation studies to ensure that the machine scores are consistently comparable to ratings given by skilled human raters. Here, a new set of test-taker responses (never seen by the machine) are scored by both human raters and by the automated scoring system. Research has demonstrated that the automated scoring technology underlying PTE Academic produces scores comparable to those obtained from careful human experts. This means that the automated system “acts” like a human rater when assessing test takers’ language skills, but does so with a machine's precision, consistency and objectivity.

Scoring speaking responses with app’s Ordinate technology

The spoken portion of PTE Academic is automatically scored using app’s Ordinate technology. Ordinate technology results from years of research in speech recognition, statistical modeling, linguistics and testing theory. The technology uses a proprietary speech processing system that is specifically designed to analyze and automatically score speech from fluent and second-language English speakers. The Ordinate scoring system collects hundreds of pieces of information from the test takers’ spoken responses in addition to just the words, such as pace, timing and rhythm, as well as the power of their voice, emphasis, intonation and accuracy of pronunciation. It is trained to recognize even somewhat mispronounced words, and quickly evaluates the content, relevance and coherence of the response. In particular, the meaning of the spoken response is evaluated, making it possible for these models to assess whether or not what was said deserves a high score.

Scoring writing responses with Intelligent Essay Assessor™ (IEA)

The written portion of PTE Academic is scored using the Intelligent Essay Assessor™ (IEA), an automated scoring tool powered by app’s state-of-the-art Knowledge Analysis Technologies™ (KAT) engine. Based on more than 20 years of research and development, the KAT engine automatically evaluates the meaning of text, such as an essay written by a student in response to a particular prompt. The KAT engine evaluates writing as accurately as skilled human raters using a proprietary application of the mathematical approach known as Latent Semantic Analysis (LSA). LSA evaluates the meaning of language by analyzing large bodies of relevant text and their meanings. Therefore, using LSA, the KAT engine can understand the meaning of text much like a human.

What aspects of English does PTE Academic assess?

Written scoring

Spoken scoring
  • Word choice
  • Grammar and mechanics
  • Progression of ideas
  • Organization
  • Style, tone
  • Paragraph structure
  • Development, coherence
  • Point of view
  • Task completion
  • Sentence mastery
  • Content
  • Vocabulary
  • Accuracy
  • Pronunciation
  • Intonation
  • Fluency
  • Expressiveness
  • Pragmatics

More blogs from app

  • A teacher helping a student at a table

    The Global Scale of English and planning: A perfect partnership

    Por

    As a teacher, I realized that planning had become an 'automatic pilot' routine from which I did not learn much. Like many others, I thought scales such as the Global Scale of English (GSE) or the Common European Framework of Reference are just that; references that are beyond the realities of their lessons.

    However, I've seen that the GSE is a very powerful resource to help us at the level of planning.

    If you're using a coursebook you may have noticed that, after completing one of the books in the series, students move up one level, such as from elementary to pre-intermediate or from intermediate to upper-intermediate.

    We all understand what it means to be an elementary or intermediate student. These levels are usually defined in terms of structures – conditional sentences, passive voice, and tenses – Simple Past, Future Continuous, etc.

    But why do students want to learn English? Using it means being able to listen or read and understand, interact with others, and communicate in writing. Even if it is parents who enroll their children in language institutes, what they want is for them to use the language. We can see a mismatch between how levels are defined and students' aims to study English.

    Here's how the GSE can help English language teachers

    First, you need the right scale for your group – Pre-primary, Young Learners, Adults, Professionals or Academic, which can be downloaded at:

    /languages/why-pearson/the-global-scale-of-english/resources.html

    Focus on your students' level. There you will see all the learning objectives that students need to achieve to complete the level at which they are and move on in their learning journey.

    What are learning objectives?They are can-do statements that clearly describe what students are expected to achieve as the result of instruction. In other words, these objectives guide teachers in our planning to help students learn.

    When we plan our lessons, rather than working at lesson level only, we should reflect on how the activities proposed are referenced against the learning objectives of the level. We may see that some activities need some adapting in order to focus on the selected learning outcomes.

    At the level of planning as well, I also use the GSE to analyze the activities proposed in the materials I am using. Let me tell you what I do. Let's take listening, for instance. You may use the downloaded scales or the Teacher Toolkit that the GSE provides. Let's run through how this works.

  • A teacher stood at the front of a class holding a tablet in front of adult students

    9 steps to teaching advanced business English

    Por Margaret O'Keeffe

    The challenge of teaching business English to C1 level students

    Once your English students reach a B2 level of English, they’re fairly competent communicators. For many learners, their motivation to improve starts to suffer when they reach this intermediate plateau. They understand almost everything and can express themselves clearly enough - so why would they want to continue learning English and achieve a C1 level of English?

    The CEFR describes C1-level learners as proficient users of a language. C1-level students have a high proficiency in English and perform well in an international work environment.

    How can we help our upper intermediate students reach this level and see the benefits in their own lives and careers? Here are nine steps you can take as an English language teacher to help your students achieve language proficiency.

  • a man with a backpack stood in a street with shops

    How to prepare for student life in the UK

    Por

    Deciding to study in the UK is a big step. Moving to a country with different food, language, culture and weather can make you nervous. It is completely normal to experience this.

    You may feel overwhelmed by the unfamiliarity of these things compared to what you are used to. Studying abroad can be a very rewarding experience once you get used to your new situation.

    So how can you conquer your fears? Here are some tips to prepare for life as an international student studying in the UK.

    Research student life in the UK

    Even though you're in the United Kingdom to study, you need to balance studying with a social life. Otherwise you risk boredom or worse, burnout. Luckily, students all over the world love to socialize, and British students are no exception.

    The most well-known time to join student life in the UK is at Freshers Week. This is when new students are welcomed to the university with various social events. There's a fair where university clubs and societies try to get new members. There are clubs for many interests, hobbies and sports, so it's an excellent way to meet people who share your interests. There are some pretty unusual clubs too. At Cambridge, for example, there's a .

    Freshers Week is also a great opportunity to talk to everyone you can. Whether it's in the queue to get your student card or to your neighbor in your first lecture, everybody is in the same situation as you - they know nobody and are trying to figure out who to be friends with. During Freshers Week, you can make good friends and converse with other students. It's all about meeting new people.

    Get to know the faculty and staff

    On your course website you'll be able to have a look at the academic staff and Ph.D. candidate profiles. Learn about their research interests and read a few abstracts of their past publications to get an idea of their work. This will help you understand their main focus during teaching and give you ideas for independent study projects.

    You can get to know the academic staff by signing up for online events before term starts. You'll hear them talking about the course and what you'll learn on it. Think about the questions you might ask.Before the event you should read your course syllabus. While you read, consider the reasons for choosing or arranging certain topics in a specific order. If you have any questions, make a note of them.

    You might even be able to watch past lectures or sign up for events that they are participating in. That way you'll have some familiar faces around the department when you arrive at your university.

    Find out about the student union

    Student unions are a big part of UK universities. Every university has a union that you can join. Sometimes there are a few different student unions, so you can choose the one you think would suit you best. Student unions are on-campus buildings where students can attend events, eat, study and socialize with others.

    But as well as organizing clubs, societies and social events, they can also help with any problems you might have. Every union has a welfare officer, typically a recent graduate elected to assist current students with personal issues they may encounter.

    An international students' officer will be available to assist with any issues overseas students may encounter. This officer will have personal experience as an international student, making them well equipped to provide support. Knowing there are people to listen and help with your problems can be comforting, even if you don't need their assistance.

    Learn about the local social scene

    University life is more than just what's happening on campus. You can investigate a whole world outside your institution. In addition to nightlife, dining and cultural events, there are also sports clubs, outdoor activities and community volunteering.

    has many volunteer opportunities. It's a good way to use your skills to help others. Meeting locals and making friends is important while studying in the UK. It helps build a strong network.The more you see of life in the UK, the more you'll get used to it and feel at ease.

    If you're worried about your level of English - don't be. People in the UK are very tolerant of how their language is used. It's a truly global language, so there's no right or wrong accent.

    If you join the local community, expect to pick up a slight accent from the place you study. It shows that you have become a part of the community.

    Follow your university on social media

    Social media is a key part of all our lives, and this also goes for university life. Schools use social media accounts to share positive information about their staff and students. They also post pictures from the campus.

    Occasionally, a student takes over their account, giving them a chance to show what it's like to study there.

    Student-run sites and social accounts also give an insider perspective on student life. Sites like cover news relevant to students, and have sections that cover news at specific universities. is also a place where current and former students discuss all aspects of student life in the UK, from exams to the TV shows students like to watch.

    Pay attention to the English language test

    You'll have to pass an English test for your visa to study in the UK. Treat it as a chance to learn. Focus on the test content, as it reflects the daily tasks of the UK and student life.

    For example, the speaking component of the PTE Academic test, which can be used for UK visas and to access 99% of British universities, tests your ability to summarise an academic lecture. You may have to summarize a lecture for a friend who couldn't attend, not just for your classes.

    Every task on the test has a real-world application, so if you can pass the test, you can be confident that you stand a good chance of making the most of life in the UK.

    Remember, although you can do as much research as you like, there's only so much you can learn online. To truly understand a place, you must be there and learn from the firsthand experience it offers. Be ready to be surprised and take advantage of things you never thought would happen. You only get to be an undergraduate once, so make the most of every opportunity.