Research has shown that automated scoring can give more reliable and objective results than human examiners when evaluating a person’s mastery of English. This is because an automated scoring system is impartial, unlike humans, who can be influenced by irrelevant factors such as a test taker’s appearance or body language. Additionally, automated scoring treats regional accents equally, unlike human examiners who may favor accents they are more familiar with. Automated scoring also allows individual features of a spoken or written test question response to be analyzed independent of one another, so that a weakness in one area of language does not affect the scoring of other areas.
was created in response to the demand for a more accurate, objective, secure and relevant test of English. Our automated scoring system is a central feature of the test, and vital to ensuring the delivery of accurate, objective and relevant results – no matter who the test-taker is or where the test is taken.
PTE Academic’s automated scoring system was developed after extensive research and field testing. A prototype test was developed and administered to a sample of more than 10,000 test takers from 158 different countries, speaking 126 different native languages. This data was collected and used to train the automated scoring engines for both the written and spoken PTE Academic items.
To do this, multiple trained human markers assess each answer. Those results are used as the training material for machine learning algorithms, similar to those used by systems like Google Search or Apple’s Siri. The model makes initial guesses as to the scores each response should get, then consults the actual scores to see well how it did, adjusts itself in a few directions, then goes through the training set over and over again, adjusting and improving until it arrives at a maximally correct solution – a solution that ideally gets very close to predicting the set of human ratings.
Once trained up and performing at a high level, this model is used as a marking algorithm, able to score new responses just like human markers would. Correlations between scores given by this system and trained human markers are quite high. The standard error of measurement between app’s system and a human rater is less than that between one human rater and another – in other words, the machine scores are more accurate than those given by a pair of human raters, because much of the bias and unreliability has been squeezed out of them. In general, you can think of a machine scoring system as one that takes the best stuff out of human ratings, then acts like an idealized human marker.
app conducts scoring validation studies to ensure that the machine scores are consistently comparable to ratings given by skilled human raters. Here, a new set of test-taker responses (never seen by the machine) are scored by both human raters and by the automated scoring system. Research has demonstrated that the automated scoring technology underlying PTE Academic produces scores comparable to those obtained from careful human experts. This means that the automated system “acts” like a human rater when assessing test takers’ language skills, but does so with a machine's precision, consistency and objectivity.
The spoken portion of PTE Academic is automatically scored using app’s Ordinate technology. Ordinate technology results from years of research in speech recognition, statistical modeling, linguistics and testing theory. The technology uses a proprietary speech processing system that is specifically designed to analyze and automatically score speech from fluent and second-language English speakers. The Ordinate scoring system collects hundreds of pieces of information from the test takers’ spoken responses in addition to just the words, such as pace, timing and rhythm, as well as the power of their voice, emphasis, intonation and accuracy of pronunciation. It is trained to recognize even somewhat mispronounced words, and quickly evaluates the content, relevance and coherence of the response. In particular, the meaning of the spoken response is evaluated, making it possible for these models to assess whether or not what was said deserves a high score.
The written portion of PTE Academic is scored using the Intelligent Essay Assessor™ (IEA), an automated scoring tool powered by app’s state-of-the-art Knowledge Analysis Technologies™ (KAT) engine. Based on more than 20 years of research and development, the KAT engine automatically evaluates the meaning of text, such as an essay written by a student in response to a particular prompt. The KAT engine evaluates writing as accurately as skilled human raters using a proprietary application of the mathematical approach known as Latent Semantic Analysis (LSA). LSA evaluates the meaning of language by analyzing large bodies of relevant text and their meanings. Therefore, using LSA, the KAT engine can understand the meaning of text much like a human.
Written scoring |
Spoken scoring |
|---|---|
|
|
Learning English with music can enhance learning and create a more engaging and dynamic classroom environment. In a previous post, we discussed if music can help you learn a language; this post looks at how music can be incorporated into the classroom.
Using music in your classroom can help improve student motivation, focus, and retention of information. Here are some ways you can use music to enhance your classroom teaching:
Have you considered studying abroad? Moving to Canada to study? With more than 100 world-class universities across the country, there’s good reason to. From coast to coast to coast, it offers many international students a safe and supportive learning environment and the opportunity to work after graduation.
Want to learn more? In this guide, we cover ten things you should know about studying in Canada:
Teaching teenagers can be a rewarding yet challenging experience.
When planning classes, we need to consider the environment in which the students are growing up. Most of our teenage students do not know a life without the internet, instant messaging or Google.
Commonly referred to as (or Gen Z), they are the most tech-savvy generation to date; this should be reflected in how we teach them.
However, not all of us have access to computer labs, interactive whiteboards or class tablets. But there is something that many of our teens bring to class in their pockets and bags every day.
You’ve guessed it - a phone.
By bringing students’ phones into your classes, you can bridge the technology gap between the learners and the curriculum, keep them engaged for longer and make them feel more empowered at the same time.
Here are four low-tech activities that use phones in class.
Activity requirements: one phone per student (or group of students) and a pre-made quiz.
is an app designed to help you create quizzes online, which can be a fun, engaging way to challenge your learners in the classroom.
Quizzes are an ideal activity to use at the start of a class to recycle previous vocabulary or to pre-teach new language.
For example, you could choose 10 British English phrases or idioms. Write a series of four possible definitions for each word, phrase, or idiom (with only one correct answer).
In class, ask your students to take out their phones, find the app, and link to the quiz.
Tell the students that they must vote for the definition they think is correct. They can keep track of their scores directly on the app while competing with their friends.
Not only will it immediately engage your learners and help them interact with what’s being learned, but they’ll also have a lot of fun.
Activity requirements: a phone with the Google Translate app installed and a pen and paper per pair.
Did you know Google Translate has a feature where you can ? This is a perfect tool for text that you might find on a poster, in a book or uses an alphabet which you are unfamiliar with.
Bring in samples of different languages to class – the more complex, the better. If you don’t have anything suitable at home, find something online and print it out.
Get one student to take a photo of the text using the app and with their finger, select the section of the text they’d like to translate from the original language to English. (Note: if you download the Google Translate language file to your phone, it will translate everything automatically).
Then, have them dictate what it says to their partner, who has to write it down.
Finally, they work together to improve the English translation. After all, everyone knows Google’s English isn’t perfect (not yet, anyway).
Activity requirements: a mobile phone with a camera per student.
Do your students love taking photos? Tap into their love of photography and make activities more exciting by incorporating the photos on their phones into your classes.
Adapt speaking activities from common exams such as the B2 Cambridge First. Instead of using the pictures in the book, put the students in pairs and get them each to choose a more relevant photo from their phones. You may find they have more to say and will communicate naturally while still practicing the necessary language and skills required in the exam.
Another idea is to get students to take photos of things they encounter in English in the street. This could be on a sign outside a restaurant or in their favorite clothing shop. Use these photos to start the next class with a discussion about where they saw it, what it means and if the English is correct.
You could also nominate one student each week to bring a photo they took that weekend. Share it with the class and get everyone to write Instagram captions. You wouldn’t believe how many already do this in English, so why not help them?
If you are worried about the misuse of mobile devices in your class – don’t be! Here are our top three strategies to help avoid students getting distracted:
