Explaining computerized English testing in plain English

app Languages
a pair of hands typing at a laptop

Research has shown that automated scoring can give more reliable and objective results than human examiners when evaluating a person’s mastery of English. This is because an automated scoring system is impartial, unlike humans, who can be influenced by irrelevant factors such as a test taker’s appearance or body language. Additionally, automated scoring treats regional accents equally, unlike human examiners who may favor accents they are more familiar with. Automated scoring also allows individual features of a spoken or written test question response to be analyzed independent of one another, so that a weakness in one area of language does not affect the scoring of other areas.

was created in response to the demand for a more accurate, objective, secure and relevant test of English. Our automated scoring system is a central feature of the test, and vital to ensuring the delivery of accurate, objective and relevant results – no matter who the test-taker is or where the test is taken.

Development and validation of the scoring system to ensure accuracy

PTE Academic’s automated scoring system was developed after extensive research and field testing. A prototype test was developed and administered to a sample of more than 10,000 test takers from 158 different countries, speaking 126 different native languages. This data was collected and used to train the automated scoring engines for both the written and spoken PTE Academic items.

To do this, multiple trained human markers assess each answer. Those results are used as the training material for machine learning algorithms, similar to those used by systems like Google Search or Apple’s Siri. The model makes initial guesses as to the scores each response should get, then consults the actual scores to see well how it did, adjusts itself in a few directions, then goes through the training set over and over again, adjusting and improving until it arrives at a maximally correct solution – a solution that ideally gets very close to predicting the set of human ratings.

Once trained up and performing at a high level, this model is used as a marking algorithm, able to score new responses just like human markers would. Correlations between scores given by this system and trained human markers are quite high. The standard error of measurement between app’s system and a human rater is less than that between one human rater and another – in other words, the machine scores are more accurate than those given by a pair of human raters, because much of the bias and unreliability has been squeezed out of them. In general, you can think of a machine scoring system as one that takes the best stuff out of human ratings, then acts like an idealized human marker.

app conducts scoring validation studies to ensure that the machine scores are consistently comparable to ratings given by skilled human raters. Here, a new set of test-taker responses (never seen by the machine) are scored by both human raters and by the automated scoring system. Research has demonstrated that the automated scoring technology underlying PTE Academic produces scores comparable to those obtained from careful human experts. This means that the automated system “acts” like a human rater when assessing test takers’ language skills, but does so with a machine's precision, consistency and objectivity.

Scoring speaking responses with app’s Ordinate technology

The spoken portion of PTE Academic is automatically scored using app’s Ordinate technology. Ordinate technology results from years of research in speech recognition, statistical modeling, linguistics and testing theory. The technology uses a proprietary speech processing system that is specifically designed to analyze and automatically score speech from fluent and second-language English speakers. The Ordinate scoring system collects hundreds of pieces of information from the test takers’ spoken responses in addition to just the words, such as pace, timing and rhythm, as well as the power of their voice, emphasis, intonation and accuracy of pronunciation. It is trained to recognize even somewhat mispronounced words, and quickly evaluates the content, relevance and coherence of the response. In particular, the meaning of the spoken response is evaluated, making it possible for these models to assess whether or not what was said deserves a high score.

Scoring writing responses with Intelligent Essay Assessor™ (IEA)

The written portion of PTE Academic is scored using the Intelligent Essay Assessor™ (IEA), an automated scoring tool powered by app’s state-of-the-art Knowledge Analysis Technologies™ (KAT) engine. Based on more than 20 years of research and development, the KAT engine automatically evaluates the meaning of text, such as an essay written by a student in response to a particular prompt. The KAT engine evaluates writing as accurately as skilled human raters using a proprietary application of the mathematical approach known as Latent Semantic Analysis (LSA). LSA evaluates the meaning of language by analyzing large bodies of relevant text and their meanings. Therefore, using LSA, the KAT engine can understand the meaning of text much like a human.

What aspects of English does PTE Academic assess?

Written scoring

Spoken scoring

  • Word choice
  • Grammar and mechanics
  • Progression of ideas
  • Organization
  • Style, tone
  • Paragraph structure
  • Development, coherence
  • Point of view
  • Task completion
  • Sentence mastery
  • Content
  • Vocabulary
  • Accuracy
  • Pronunciation
  • Intonation
  • Fluency
  • Expressiveness
  • Pragmatics

More blogs from app

  • A woman on her phone smiling outdoors in a city

    Understanding accents and their role in language learning

    Por
    Reading time: 4 minutes

    Accents are a fascinating aspect of language that can reveal a lot about a person's background and experiences. For language learners, accents often become a central focus, sometimes to the detriment of more important language skills. In this post, we’ll explore what accents are, why people have them, and why intelligibility is more important than mastering an accent.

  • A female teacher sat in a classroom with a young child playing with toys

    Educating young learners: The importance of developing fine motor skills

    Por Hawys Morgan
    Reading time: 5 minutes

    Teachers of young learners will be familiar with the importance of developing children’s fine motor skills. Building muscle strength, hand-eye coordination, and control are essential parts of students’ development during their early years.

    A holistic approach to education

    For young learners, their education is frequently holistic. A single game or activity might develop their speaking and listening skills, mathematical knowledge, social interaction, artistic development as well as motor skills. In the same way, improving fine motor skills can form a natural part of students’ English classes and can have the following benefits:

    • Builds concentration and self-esteem
    • Actively engages students in their learning
    • Develops the ability to switch between physical and mental activities
    • Improves social development and autonomy

    Below are some simple ways you can develop fine motor skills in your classroom.

    Building muscle strength

    Holding a pen for prolonged periods requires strong hand, wrist and arm muscles. If you’ve ever had to sit long hand-written exams, you will be familiar with tired and aching hand muscles.

    It is important that students develop their muscle strength so they have the stamina and control needed for writing. Kneading and rolling play dough is a fun way to build these muscles. Then, children can use their playdough creations to role-play scenarios such as having a picnic or visiting a farm. They could even model it into letter shapes.

    When singing songs or telling stories to young learners, teachers often incorporate actions to reinforce meaning. This is another opportunity to build those muscles. Children could also create shadow puppets with their hands to act out stories.

    Craft activities that involve scissors and gluing also help improve stamina and hand-eye coordination.

    Mark making

    Mark making is an important step in a child’s development, encouraging creativity and coordination.

    Try giving your students the opportunity to explore different mediums of mark making. For example, they could draw marks in trays of sand, jello, shaving foam, flour or rice. Talk to them about their sensory experience (Is it cold? Do you like it? What color is it?). They could start making marks with their whole hands and then, as their coordination improves, use an index finger. Then, they can start using a stick to make marks.

    As their fine motor skills develop, try using magic markers, chunky crayons and egg chalks to make large marks. Egg-shaped chalks are easier for young children to grip. Each straight line, wavy line and circle is another step on the road to learning how to write.

    Pincer grip

    As children move on from general mark making, it is important to establish a correct grip when using a writing tool. This is especially important if you expect your students to go on to use a cursive style of handwriting in the future.

    The pincer grip is when we hold something with our index finger and thumb. Peeling off and placing stickers, sorting building blocks and threading beads use this grip and provide opportunities for practicing colors, numbers, vocabulary and prepositions of place. Doing up buttons or zips uses this grasp too.

    While it may be second nature for adults, for children, this grip requires precise control of the small muscles in their hands, wrists and fingers. Challenge students to pick up items with large blunt tweezers or chopsticks and work on their English at the same time (What have you got? I have a car. It’s small.).

    Tripod grip

    The next stage of development for most children is the tripod grip. It uses three fingers: the thumb, index and middle fingers. It enables children to keep their wrist steady so they can make small, precise pencil movements.

    Some children find using a rubber pencil grip, or simply wrapping an elastic band or lump of playdough around the base of the pencil helps them maintain this grip.

    At this stage, students will be learning to hold and use writing tools such as pencils, crayons, markers, chalks and paintbrushes.

    Prewriting activities

    Prewriting activities offer more controlled fine motor skills practice. The usual progression is to start with straight lines, zig-zags, curved lines and diagonal lines. Then move on to tracing over circles and u-shapes. This is all essential preparation for writing letters and words.

    English courses for young learners are full of mazes, dot-to-dot, tracing and matching activities, all of which combine prewriting with learning English.

    When it comes to writing letters, it can be helpful if students begin by drawing the letter shape in the air or in sand. They then trace the letter shape with an index finger, before finally tracing over the letter with a pencil.

    Other writing systems

    When teaching students who use a different writing system in L1, establishing fine motor skills routines can make all the difference to students’ writing.

    It can be helpful to work on left-to-right orientation. For example, before they sit down to write, give students scarves to move from left to right in the air. These students will benefit from pre-writing activities that work on left-to-right, top-to-bottom patterns.

    Social development

    Doing up buttons, zips, and laces, turning on taps, cutting up food and opening boxes – all of these things improve students’ fine motor skills. They also promote autonomy and social development by helping students learn essential everyday life skills

    This has an added advantage for the teacher. The less time you have to spend helping students with these tasks, the more time you will have to work on other areas of their development. Not only that, it is also motivating for students to have that ‘I can do it all by myself!’ feeling.

    Students will be far better prepared to pick up a pen or pencil if they have developed strength, dexterity and stamina in their hands, wrists and arms. This will leave them free to concentrate on the language element of their classroom task, rather than the physical challenge it presents.

    AboutEnglish Code

    Support your young learners withEnglish Code,a 7-level course for 7-12-year-olds, offering 5 hours or more of English study per week. Available in both American English and British English versions, it promotes hands-on creative learning, investigation, fun projects and experiments.

    Focusing on project work and STEAM learning, children develop fine motor skills while learning how to collaborate and solve problems with their peers. Core functional language is at the forefront, giving students the vocabulary and tools they need to become confident speakers of English inside and outside the classroom.

  • A teenage girl smiling and doing her homework or studying at home.

    Creative writing exercises for English language learners

    Por app Languages
    Reading time: 3.5 minutes

    Engaging in creative writing is one effective way to improve your English language skills. It allows for artistic expression and personal flair, making it a unique way to put your imagination into words. Additionally, creative writing can draw from the author’s life and experiences, providing a personal touch to the narrative. Whether you’re a beginner or an advanced learner, creative writing can help you enhance your vocabulary, grammar and overall language proficiency.

    Creative writing exercises can also benefit other forms of writing, such as business writing and technical writing, even though these are not typically considered creative writing. In this post, we’ll explore various creative writing exercises that are designed to make learning English fun and engaging, as well as improve writing skills.