Explaining computerized English testing in plain English

app Languages
a pair of hands typing at a laptop

Research has shown that automated scoring can give more reliable and objective results than human examiners when evaluating a person’s mastery of English. This is because an automated scoring system is impartial, unlike humans, who can be influenced by irrelevant factors such as a test taker’s appearance or body language. Additionally, automated scoring treats regional accents equally, unlike human examiners who may favor accents they are more familiar with. Automated scoring also allows individual features of a spoken or written test question response to be analyzed independent of one another, so that a weakness in one area of language does not affect the scoring of other areas.

was created in response to the demand for a more accurate, objective, secure and relevant test of English. Our automated scoring system is a central feature of the test, and vital to ensuring the delivery of accurate, objective and relevant results – no matter who the test-taker is or where the test is taken.

Development and validation of the scoring system to ensure accuracy

PTE Academic’s automated scoring system was developed after extensive research and field testing. A prototype test was developed and administered to a sample of more than 10,000 test takers from 158 different countries, speaking 126 different native languages. This data was collected and used to train the automated scoring engines for both the written and spoken PTE Academic items.

To do this, multiple trained human markers assess each answer. Those results are used as the training material for machine learning algorithms, similar to those used by systems like Google Search or Apple’s Siri. The model makes initial guesses as to the scores each response should get, then consults the actual scores to see well how it did, adjusts itself in a few directions, then goes through the training set over and over again, adjusting and improving until it arrives at a maximally correct solution – a solution that ideally gets very close to predicting the set of human ratings.

Once trained up and performing at a high level, this model is used as a marking algorithm, able to score new responses just like human markers would. Correlations between scores given by this system and trained human markers are quite high. The standard error of measurement between app’s system and a human rater is less than that between one human rater and another – in other words, the machine scores are more accurate than those given by a pair of human raters, because much of the bias and unreliability has been squeezed out of them. In general, you can think of a machine scoring system as one that takes the best stuff out of human ratings, then acts like an idealized human marker.

app conducts scoring validation studies to ensure that the machine scores are consistently comparable to ratings given by skilled human raters. Here, a new set of test-taker responses (never seen by the machine) are scored by both human raters and by the automated scoring system. Research has demonstrated that the automated scoring technology underlying PTE Academic produces scores comparable to those obtained from careful human experts. This means that the automated system “acts” like a human rater when assessing test takers’ language skills, but does so with a machine's precision, consistency and objectivity.

Scoring speaking responses with app’s Ordinate technology

The spoken portion of PTE Academic is automatically scored using app’s Ordinate technology. Ordinate technology results from years of research in speech recognition, statistical modeling, linguistics and testing theory. The technology uses a proprietary speech processing system that is specifically designed to analyze and automatically score speech from fluent and second-language English speakers. The Ordinate scoring system collects hundreds of pieces of information from the test takers’ spoken responses in addition to just the words, such as pace, timing and rhythm, as well as the power of their voice, emphasis, intonation and accuracy of pronunciation. It is trained to recognize even somewhat mispronounced words, and quickly evaluates the content, relevance and coherence of the response. In particular, the meaning of the spoken response is evaluated, making it possible for these models to assess whether or not what was said deserves a high score.

Scoring writing responses with Intelligent Essay Assessor™ (IEA)

The written portion of PTE Academic is scored using the Intelligent Essay Assessor™ (IEA), an automated scoring tool powered by app’s state-of-the-art Knowledge Analysis Technologies™ (KAT) engine. Based on more than 20 years of research and development, the KAT engine automatically evaluates the meaning of text, such as an essay written by a student in response to a particular prompt. The KAT engine evaluates writing as accurately as skilled human raters using a proprietary application of the mathematical approach known as Latent Semantic Analysis (LSA). LSA evaluates the meaning of language by analyzing large bodies of relevant text and their meanings. Therefore, using LSA, the KAT engine can understand the meaning of text much like a human.

What aspects of English does PTE Academic assess?

Written scoring

Spoken scoring

  • Word choice
  • Grammar and mechanics
  • Progression of ideas
  • Organization
  • Style, tone
  • Paragraph structure
  • Development, coherence
  • Point of view
  • Task completion
  • Sentence mastery
  • Content
  • Vocabulary
  • Accuracy
  • Pronunciation
  • Intonation
  • Fluency
  • Expressiveness
  • Pragmatics

More blogs from app

  • A group of university students outside smiling

    International students: Which UK university is best for you?

    Por

    People study abroad for many reasons - to see new places, to experience a different culture, and, most importantly, to get access to student life and educational opportunities not available at home. If you want to study at a UK university, you’ll not only have access to some truly innovative teaching and research, you’ll also be joining one of the most multicultural student bodies in the world.

    The UK has a vast range of universities and courses to choose from. So, how do you choose between over 160 institutions and thousands of courses? that half of students who drop out of university early do so because they chose the wrong course, so it’s important to be completely happy with your choice.

    So, here's what you need to consider before you make your university application so you can make the right choice.

    Choose your subject

    The first step in choosing a university is deciding what you want to study. Your will be sent to all your university choices, so it should be relevant to all the courses you apply for. This means that your course choices must be the same or in a similar field.

    Think about the school subjects you’re good at, but also think about what you could spend the next three or even four years studying.

    Is there a subject that you want to continue and deepen your knowledge in? Or do you want to study something completely new?

    Do you want to study a subject like law or medicine related to a specific job? Or do you want to take a course like history that can be used in a wide range of careers? Think carefully about these questions and the rest of the process will become much more manageable.

    Do as many events and online taster sessions, and sign up for as many free online courses as possible. has a great range of taster courses from universities all over the world. This can help you decide where your interests lie. Above all, keep notes on what interests you and what doesn’t.

    If you think you want a broader degree, then you’ve always got the option of doing a joint honors degree. This means you combine two different courses. Many joint honors courses combine a language with another subject, allowing you to use your learning around the world or even do a study placement in another country.

    Consider the competition

    When thinking about what course you could do, it’s important to consider the competition. Some courses have many more applicants than places, but there are also hundreds of ‘hidden’ courses that get far fewer applicants. For example, the University of Oxford receives 16.9 applicants per place on its Economics course, but it’s closer to 3 applicants per Classics place. Classics covers a range of areas like politics, history, linguistics and social sciences, and really challenges your intellect.

    Economics might be competitive but Classics might be just as valuable. There are thousands of ‘hidden’ courses, which don't attract as many applicants just because they’re not well known. Check out the full range of courses on the site and the and do it with an open mind.

    Once you’ve narrowed down your list of courses, it’s time to think about where to study.

    Think about location and compare places

    The UK might be a small country but it’s incredibly diverse. There are large, international cities like London or Glasgow, smaller towns and cities like Cardiff, Belfast or Nottingham, rural universities like Lancaster, coastal towns like St. Andrews or historic cities like York, Oxford or Cambridge.

    So, when deciding on a UK university, you must first ask yourself: what’s important to you? Would you prefer to live in a busy city or the quiet countryside? Will you be doing a lot of exploring or will you stay near campus? These factors will make a difference to your decision.

    You’ll also need to think carefully about costs. Different parts of the UK have different living costs, depending on things like rent, transport and the cost of entertainment. Check out to see what rents are like, to get an idea of how expensive life will be.

    Finally, the type of university itself is important. Some universities have campuses where the whole of university life takes place, like the . Others are more spread out, such as the , which has buildings all over the city. Each has its own atmosphere and you should think about which you prefer.

    Look at the university rankings

    Find out how your preferred universities rank in the league tables. Rankings of the top universities in the UK don’t just measure teaching quality. They show data on student satisfaction, post-graduation employment rates and staff-to-student ratios.

    The best universities in the UK and courses with higher rankings will be more competitive to gain entry to. You can apply for up to five courses on your UCAS form, so choosing a range of courses and universities is a good idea. Include an ambitious choice which you may or may not get the grades for and safe choices that you know will accept you.

    You can check the , or the to see which universities are the most highly rated.

    That being said, student life in the UK is about balance. You need to consider both rankings and what’s right for you. It’s no use going to a top-ranking UK university in a location you hate, studying things that don’t interest you and where the culture on campus doesn’t appeal to you.

    Sometimes it might be better to consider a lower university ranking if it can offer you plenty of chances to thrive and achieve your personal goals.

    Remember, the reason that there are so many different kinds of universities is because everyone is unique. If you’re applying to somewhere you know matches your values and interests, you’ll be much more likely to succeed in your university career.

    Research the teaching style

    It’s a good idea to look at the style of teaching and assessment at your chosen university. Some courses focus more on final exams, others on coursework, projects, dissertations and independent research.

    See whether your university’s exams are open book or closed book. If they’re closed you have to remember a lot more, but if they’re open you will probably have to give more detailed answers in your exams.

    If you can attend open days and chat with other students and potential professors, do so. But if you can’t, remember that each university course website will tell you about the course content and how it is assessed. You need to make a choice based on what type of assessment will suit you best.

    Some courses are taught in large lectures and seminars; others, such as many colleges, will teach you in small groups or one-to-one with a tutor. The teaching style that works best for you will be crucial in your choice.

    Consider your hobbies

    Every university has its own culture and social life based around the student unions and societies. For local and international students, university life is about more than studying.

    You will have a more balanced student experience if you make time for your hobbies. You can look on the university website to learn about student societies and clubs and see which activities are available. But don’t worry if you don’t see anything that appeals - you can always be proactive and start your own society.

    Do you have a sport that you love to do? If you do and are really good at it, you may be eligible for a . There are all sorts of funding and awards available, so even if you think you won’t get anything, it’s a good idea to check.

    Make sure you meet the English language requirements

    You’ll need to pass a test of English for your UK student visa requirements. You can take any Secure English Language Test (SELT) approved by the Home Office, the UK government department responsible for immigration.

    PTE Academic can be used for UK student visas for degree-level courses, and to access 99% of universities in the country. The test takes two hours, covers reading, speaking, listening and writing, and is done on a computer. Results usually come back within 48 hours, making it convenient and reliable.

    Choosing a university requires a lot of thought. Luckily, choosing an English test to get a visa is easy.

  • a woman sat in a library in front of a laptop and open books

    The importance of antonyms: Expanding your vocabulary

    Por

    Expanding and improving your English vocabulary can be done in various ways, but exploring antonyms can be a handy and unique approach. Antonyms, which are words that express opposite meanings, can help you enhance your language skills. Knowing a wide range of them can be incredibly helpful in everyday life and help you get your point across clearly.

    In today's language learning blog post, we share practical tips to help you harness the power of antonyms and improve your vocabulary.

  • A child sat at a desk with a pen in hand, looking up at their teacher and smiling

    Dyslexia and ELT: How to help young learners in the classroom

    Por Joanna Wiseman

    When you’re teaching English to young learners, you might find that there are a few students in your class who are struggling. But sometimes it can be hard to tell why. Is it because their language level is low? Or are they finding classroom work difficult because of a general cognitive difference, like dyslexia?