Explaining computerized English testing in plain English

app Languages
a pair of hands typing at a laptop

Research has shown that automated scoring can give more reliable and objective results than human examiners when evaluating a person’s mastery of English. This is because an automated scoring system is impartial, unlike humans, who can be influenced by irrelevant factors such as a test taker’s appearance or body language. Additionally, automated scoring treats regional accents equally, unlike human examiners who may favor accents they are more familiar with. Automated scoring also allows individual features of a spoken or written test question response to be analyzed independent of one another, so that a weakness in one area of language does not affect the scoring of other areas.

was created in response to the demand for a more accurate, objective, secure and relevant test of English. Our automated scoring system is a central feature of the test, and vital to ensuring the delivery of accurate, objective and relevant results – no matter who the test-taker is or where the test is taken.

Development and validation of the scoring system to ensure accuracy

PTE Academic’s automated scoring system was developed after extensive research and field testing. A prototype test was developed and administered to a sample of more than 10,000 test takers from 158 different countries, speaking 126 different native languages. This data was collected and used to train the automated scoring engines for both the written and spoken PTE Academic items.

To do this, multiple trained human markers assess each answer. Those results are used as the training material for machine learning algorithms, similar to those used by systems like Google Search or Apple’s Siri. The model makes initial guesses as to the scores each response should get, then consults the actual scores to see well how it did, adjusts itself in a few directions, then goes through the training set over and over again, adjusting and improving until it arrives at a maximally correct solution – a solution that ideally gets very close to predicting the set of human ratings.

Once trained up and performing at a high level, this model is used as a marking algorithm, able to score new responses just like human markers would. Correlations between scores given by this system and trained human markers are quite high. The standard error of measurement between app’s system and a human rater is less than that between one human rater and another – in other words, the machine scores are more accurate than those given by a pair of human raters, because much of the bias and unreliability has been squeezed out of them. In general, you can think of a machine scoring system as one that takes the best stuff out of human ratings, then acts like an idealized human marker.

app conducts scoring validation studies to ensure that the machine scores are consistently comparable to ratings given by skilled human raters. Here, a new set of test-taker responses (never seen by the machine) are scored by both human raters and by the automated scoring system. Research has demonstrated that the automated scoring technology underlying PTE Academic produces scores comparable to those obtained from careful human experts. This means that the automated system “acts” like a human rater when assessing test takers’ language skills, but does so with a machine's precision, consistency and objectivity.

Scoring speaking responses with app’s Ordinate technology

The spoken portion of PTE Academic is automatically scored using app’s Ordinate technology. Ordinate technology results from years of research in speech recognition, statistical modeling, linguistics and testing theory. The technology uses a proprietary speech processing system that is specifically designed to analyze and automatically score speech from fluent and second-language English speakers. The Ordinate scoring system collects hundreds of pieces of information from the test takers’ spoken responses in addition to just the words, such as pace, timing and rhythm, as well as the power of their voice, emphasis, intonation and accuracy of pronunciation. It is trained to recognize even somewhat mispronounced words, and quickly evaluates the content, relevance and coherence of the response. In particular, the meaning of the spoken response is evaluated, making it possible for these models to assess whether or not what was said deserves a high score.

Scoring writing responses with Intelligent Essay Assessor™ (IEA)

The written portion of PTE Academic is scored using the Intelligent Essay Assessor™ (IEA), an automated scoring tool powered by app’s state-of-the-art Knowledge Analysis Technologies™ (KAT) engine. Based on more than 20 years of research and development, the KAT engine automatically evaluates the meaning of text, such as an essay written by a student in response to a particular prompt. The KAT engine evaluates writing as accurately as skilled human raters using a proprietary application of the mathematical approach known as Latent Semantic Analysis (LSA). LSA evaluates the meaning of language by analyzing large bodies of relevant text and their meanings. Therefore, using LSA, the KAT engine can understand the meaning of text much like a human.

What aspects of English does PTE Academic assess?

Written scoring

Spoken scoring
  • Word choice
  • Grammar and mechanics
  • Progression of ideas
  • Organization
  • Style, tone
  • Paragraph structure
  • Development, coherence
  • Point of view
  • Task completion
  • Sentence mastery
  • Content
  • Vocabulary
  • Accuracy
  • Pronunciation
  • Intonation
  • Fluency
  • Expressiveness
  • Pragmatics

More blogs from app

  • A businessman smiling outside

    Upskilling and re-skilling for work: Future-proofing your career

    Por Claire Chettiar

    In today's rapidly evolving work landscape, the importance of both upskilling and re-skilling has never been more evident. The skills required in the workplace are continuously shifting due to technological advancements, changing job roles, and increasing global competition. Learning new skills is the key to remaining relevant, competitive, and future proofing your career.

    A recentapp skills outlook reportshows how Indian workers are keen to improve their skills for career growth (with a focus on certain skills in technology and business English) to keep up with progression, and to stay relevant to employers. Following this, in today's post, we explore the significance of upskilling and re-skilling and how they can empower you to thrive in the dynamic world of work.

    Staying relevant in a changing job market

    The job market is in a constant state of flux, with traditional roles being disrupted and new ones emerging. Upskilling allows you to stay ahead of these changes in your current career, and reskilling allows you to adapt and remain relevant in the face of evolving skill requirements. By continuously updating and expanding your skill set, you enhance your employability and increase your chances of securing meaningful employment opportunities, even in the face of automation and job market fluctuations.

    Enhancing professional growth and advancement

    Upskilling is not just about maintaining your current job; it is also a catalyst for professional growth and advancement. By acquiring new skills, you broaden your horizons, open doors to new career possibilities, and position yourself for promotions and higher-level positions.

    Employers value employees who demonstrate a commitment to self-improvement and possess a diverse range of skills, making upskilling a powerful tool for career progression.

    Navigating technological advancements

    Technology continues to reshape the workplace, and upskilling is essential for effectively navigating these advancements. Automation, artificial intelligence, and digitalization are transforming industries, and workers must adapt to remain relevant. Re-skilling in areas such as data analysis, coding, cybersecurity, or digital marketing equips you with the expertise needed to leverage technology and work alongside it, rather than being displaced by it.

    Increasing job satisfaction and motivation

    Learning and growth are fundamental drivers of job satisfaction and motivation: research has shown that it is as significant a factor as low pay in peoples' decision to stay with a particular company or job role1. Upskilling not only expands your knowledge but also stimulates your intellectual curiosity and passion for your work. When you invest in your professional development, you can align your skills with your passions and find greater fulfilment in your chosen career path.

    Nurturing adaptability and resilience

    Upskilling fosters adaptability and resilience, two essential qualities in today's dynamic work environment. The estimates that a quarter of all jobs will be disrupted in the next five years alone; meaning that everyone needs to be prepared for change and uncertainty.

    By continuously learning and developing new skills, you become more adaptable to changing circumstances and can readily embrace new challenges. Upskilling enhances your problem-solving abilities, critical thinking skills, and ability to learn quickly, making you better equipped to handle unexpected situations and thrive in an ever-evolving workplace.

    Remaining competitive in a global economy

    As the world becomes increasingly interconnected, global competition for jobs is intensifying. By staying abreast of industry trends, acquiring in-demand skills, and demonstrating a commitment to self-improvement, you differentiate yourself from other candidates and increase your chances of securing coveted job opportunities, both domestically and internationally.

    The importance of upskilling and re-skilling for work cannot be overstated. In a rapidly changing world, they are the keys to remaining relevant, adaptable, and competitive. By continuously learning and developing new skills, you future-proof your career, enhance your employability, and open doors to new opportunities.

    Embrace skills development as a lifelong journey: by doing so, you empower yourself to thrive in the dynamic world of work and build a fulfilling, successful career.

  • Coworkers sat at a desk with their laptops, smiling and talking to eachother

    Boosting employee retention: Benefits of language learning in the workplace

    Por Claire Chettiar

    In the era of the Great Resignation, there are lots of factors that influence whether or not someone chooses to stay with their company or decides to accept a job offer from a new employer.

    Whilst fair pay remains a constant in this, feeling included, valued, and cared about by colleagues plays a significant role in the decision on whether to stay or go.

  • Three business people walking through a hallway smilng

    The value of language skills in the workplace

    Por Claire Chettiar

    Language skills have become increasingly important in the workplace as the world becomes more globalized and connected. Having the ability to communicate effectively in more than one language is very valuable, and with English being a leading lingua franca for businesses across the world, language skills sit at the heart of business success.

    Here we highlight four top reasons why language skills are so valuable in the workplace.

    Enhanced communication

    Effective communication is the number one soft skill that employers look for, . Having strong language skills enables better communication with clients and colleagues, and also helps to work effectively with people from different cultures. It builds solid relationships and reduces frustration where customers or colleagues may otherwise feel that they are not understood or listened to.

    Career opportunities

    Having different languages on your CV can really help to stand out among other candidates. For some industries it is essential, such as tourism, where it is necessary to serve clients from different countries, and this is also true where companies serve clients in international markets.

    Where businesses are increasingly moving to hybrid working practices that can include teams spread across different countries and timezones, it is also becoming more of an expectation and advantage to be able to connect through a central language, such as English.

    Furthermore, having English language skills can provide employees with better networking opportunities, in particular through conferences, trade shows and social media platforms such as LinkedIn.

    Personal growth

    Learning a new language can be a very enriching experience that enhances personal growth and skills development. Language learning doesn't just impact communication skills; it's also been noted that learning a language helps the brain process and remember information more efficiently[1], as well as developing new cognitive and problem-solving skills.

    Learning a new language requires discipline, dedication, and patience, which are qualities that can be applied to other areas of work and life.

    To summarise, language skills are highly valuable in the workplace and can open up new career opportunities for employees, as well as adding tangible benefits to businesses. Investing in language skills can increase an employee's value, and this can help them succeed in today's ever-evolving work landscape.