Explaining computerized English testing in plain English

app Languages
a pair of hands typing at a laptop

Research has shown that automated scoring can give more reliable and objective results than human examiners when evaluating a person’s mastery of English. This is because an automated scoring system is impartial, unlike humans, who can be influenced by irrelevant factors such as a test taker’s appearance or body language. Additionally, automated scoring treats regional accents equally, unlike human examiners who may favor accents they are more familiar with. Automated scoring also allows individual features of a spoken or written test question response to be analyzed independent of one another, so that a weakness in one area of language does not affect the scoring of other areas.

was created in response to the demand for a more accurate, objective, secure and relevant test of English. Our automated scoring system is a central feature of the test, and vital to ensuring the delivery of accurate, objective and relevant results – no matter who the test-taker is or where the test is taken.

Development and validation of the scoring system to ensure accuracy

PTE Academic’s automated scoring system was developed after extensive research and field testing. A prototype test was developed and administered to a sample of more than 10,000 test takers from 158 different countries, speaking 126 different native languages. This data was collected and used to train the automated scoring engines for both the written and spoken PTE Academic items.

To do this, multiple trained human markers assess each answer. Those results are used as the training material for machine learning algorithms, similar to those used by systems like Google Search or Apple’s Siri. The model makes initial guesses as to the scores each response should get, then consults the actual scores to see well how it did, adjusts itself in a few directions, then goes through the training set over and over again, adjusting and improving until it arrives at a maximally correct solution – a solution that ideally gets very close to predicting the set of human ratings.

Once trained up and performing at a high level, this model is used as a marking algorithm, able to score new responses just like human markers would. Correlations between scores given by this system and trained human markers are quite high. The standard error of measurement between app’s system and a human rater is less than that between one human rater and another – in other words, the machine scores are more accurate than those given by a pair of human raters, because much of the bias and unreliability has been squeezed out of them. In general, you can think of a machine scoring system as one that takes the best stuff out of human ratings, then acts like an idealized human marker.

app conducts scoring validation studies to ensure that the machine scores are consistently comparable to ratings given by skilled human raters. Here, a new set of test-taker responses (never seen by the machine) are scored by both human raters and by the automated scoring system. Research has demonstrated that the automated scoring technology underlying PTE Academic produces scores comparable to those obtained from careful human experts. This means that the automated system “acts” like a human rater when assessing test takers’ language skills, but does so with a machine's precision, consistency and objectivity.

Scoring speaking responses with app’s Ordinate technology

The spoken portion of PTE Academic is automatically scored using app’s Ordinate technology. Ordinate technology results from years of research in speech recognition, statistical modeling, linguistics and testing theory. The technology uses a proprietary speech processing system that is specifically designed to analyze and automatically score speech from fluent and second-language English speakers. The Ordinate scoring system collects hundreds of pieces of information from the test takers’ spoken responses in addition to just the words, such as pace, timing and rhythm, as well as the power of their voice, emphasis, intonation and accuracy of pronunciation. It is trained to recognize even somewhat mispronounced words, and quickly evaluates the content, relevance and coherence of the response. In particular, the meaning of the spoken response is evaluated, making it possible for these models to assess whether or not what was said deserves a high score.

Scoring writing responses with Intelligent Essay Assessor™ (IEA)

The written portion of PTE Academic is scored using the Intelligent Essay Assessor™ (IEA), an automated scoring tool powered by app’s state-of-the-art Knowledge Analysis Technologies™ (KAT) engine. Based on more than 20 years of research and development, the KAT engine automatically evaluates the meaning of text, such as an essay written by a student in response to a particular prompt. The KAT engine evaluates writing as accurately as skilled human raters using a proprietary application of the mathematical approach known as Latent Semantic Analysis (LSA). LSA evaluates the meaning of language by analyzing large bodies of relevant text and their meanings. Therefore, using LSA, the KAT engine can understand the meaning of text much like a human.

What aspects of English does PTE Academic assess?

Written scoring

Spoken scoring

  • Word choice
  • Grammar and mechanics
  • Progression of ideas
  • Organization
  • Style, tone
  • Paragraph structure
  • Development, coherence
  • Point of view
  • Task completion
  • Sentence mastery
  • Content
  • Vocabulary
  • Accuracy
  • Pronunciation
  • Intonation
  • Fluency
  • Expressiveness
  • Pragmatics

More blogs from app

  • A businessman smiling outside

    Upskilling and re-skilling for work: Future-proofing your career

    Por Claire Chettiar

    In today's rapidly evolving work landscape, the importance of both upskilling and re-skilling has never been more evident. The skills required in the workplace are continuously shifting due to technological advancements, changing job roles, and increasing global competition. Learning new skills is the key to remaining relevant, competitive, and future proofing your career.

    A recentapp skills outlook reportshows how Indian workers are keen to improve their skills for career growth (with a focus on certain skills in technology and business English) to keep up with progression, and to stay relevant to employers. Following this, in today's post, we explore the significance of upskilling and re-skilling and how they can empower you to thrive in the dynamic world of work.

    Staying relevant in a changing job market

    The job market is in a constant state of flux, with traditional roles being disrupted and new ones emerging. Upskilling allows you to stay ahead of these changes in your current career, and reskilling allows you to adapt and remain relevant in the face of evolving skill requirements. By continuously updating and expanding your skill set, you enhance your employability and increase your chances of securing meaningful employment opportunities, even in the face of automation and job market fluctuations.

    Enhancing professional growth and advancement

    Upskilling is not just about maintaining your current job; it is also a catalyst for professional growth and advancement. By acquiring new skills, you broaden your horizons, open doors to new career possibilities, and position yourself for promotions and higher-level positions.

    Employers value employees who demonstrate a commitment to self-improvement and possess a diverse range of skills, making upskilling a powerful tool for career progression.

    Navigating technological advancements

    Technology continues to reshape the workplace, and upskilling is essential for effectively navigating these advancements. Automation, artificial intelligence, and digitalization are transforming industries, and workers must adapt to remain relevant. Re-skilling in areas such as data analysis, coding, cybersecurity, or digital marketing equips you with the expertise needed to leverage technology and work alongside it, rather than being displaced by it.

    Increasing job satisfaction and motivation

    Learning and growth are fundamental drivers of job satisfaction and motivation: research has shown that it is as significant a factor as low pay in peoples' decision to stay with a particular company or job role1. Upskilling not only expands your knowledge but also stimulates your intellectual curiosity and passion for your work. When you invest in your professional development, you can align your skills with your passions and find greater fulfilment in your chosen career path.

    Nurturing adaptability and resilience

    Upskilling fosters adaptability and resilience, two essential qualities in today's dynamic work environment. The estimates that a quarter of all jobs will be disrupted in the next five years alone; meaning that everyone needs to be prepared for change and uncertainty.

    By continuously learning and developing new skills, you become more adaptable to changing circumstances and can readily embrace new challenges. Upskilling enhances your problem-solving abilities, critical thinking skills, and ability to learn quickly, making you better equipped to handle unexpected situations and thrive in an ever-evolving workplace.

    Remaining competitive in a global economy

    As the world becomes increasingly interconnected, global competition for jobs is intensifying. By staying abreast of industry trends, acquiring in-demand skills, and demonstrating a commitment to self-improvement, you differentiate yourself from other candidates and increase your chances of securing coveted job opportunities, both domestically and internationally.

    The importance of upskilling and re-skilling for work cannot be overstated. In a rapidly changing world, they are the keys to remaining relevant, adaptable, and competitive. By continuously learning and developing new skills, you future-proof your career, enhance your employability, and open doors to new opportunities.

    Embrace skills development as a lifelong journey: by doing so, you empower yourself to thrive in the dynamic world of work and build a fulfilling, successful career.

  • Two students sat on stairs smiling at a laptop

    How to keep your students motivated during summer intensive courses

    Por Richard Cleeve

    Summer is here and so are English language intensive courses. Adults and teens are lining up at your classroom doors, full of enthusiasm and ready to learn English. But soon they’re staring out the windows, looking up at the bright blue skies and wishing they were on the beach, or - frankly - doing anything but studying English for the next 100 hours.

    So how can you turn it around and keep your learners on track and motivated this summer?

  • A group of university students outside smiling

    International students: Which UK university is best for you?

    Por

    People study abroad for many reasons - to see new places, to experience a different culture, and, most importantly, to get access to student life and educational opportunities not available at home. If you want to study at a UK university, you’ll not only have access to some truly innovative teaching and research, you’ll also be joining one of the most multicultural student bodies in the world.

    The UK has a vast range of universities and courses to choose from. So, how do you choose between over 160 institutions and thousands of courses? that half of students who drop out of university early do so because they chose the wrong course, so it’s important to be completely happy with your choice.

    So, here's what you need to consider before you make your university application so you can make the right choice.

    Choose your subject

    The first step in choosing a university is deciding what you want to study. Your will be sent to all your university choices, so it should be relevant to all the courses you apply for. This means that your course choices must be the same or in a similar field.

    Think about the school subjects you’re good at, but also think about what you could spend the next three or even four years studying.

    Is there a subject that you want to continue and deepen your knowledge in? Or do you want to study something completely new?

    Do you want to study a subject like law or medicine related to a specific job? Or do you want to take a course like history that can be used in a wide range of careers? Think carefully about these questions and the rest of the process will become much more manageable.

    Do as many events and online taster sessions, and sign up for as many free online courses as possible. has a great range of taster courses from universities all over the world. This can help you decide where your interests lie. Above all, keep notes on what interests you and what doesn’t.

    If you think you want a broader degree, then you’ve always got the option of doing a joint honors degree. This means you combine two different courses. Many joint honors courses combine a language with another subject, allowing you to use your learning around the world or even do a study placement in another country.

    Consider the competition

    When thinking about what course you could do, it’s important to consider the competition. Some courses have many more applicants than places, but there are also hundreds of ‘hidden’ courses that get far fewer applicants. For example, the University of Oxford receives 16.9 applicants per place on its Economics course, but it’s closer to 3 applicants per Classics place. Classics covers a range of areas like politics, history, linguistics and social sciences, and really challenges your intellect.

    Economics might be competitive but Classics might be just as valuable. There are thousands of ‘hidden’ courses, which don't attract as many applicants just because they’re not well known. Check out the full range of courses on the site and the and do it with an open mind.

    Once you’ve narrowed down your list of courses, it’s time to think about where to study.

    Think about location and compare places

    The UK might be a small country but it’s incredibly diverse. There are large, international cities like London or Glasgow, smaller towns and cities like Cardiff, Belfast or Nottingham, rural universities like Lancaster, coastal towns like St. Andrews or historic cities like York, Oxford or Cambridge.

    So, when deciding on a UK university, you must first ask yourself: what’s important to you? Would you prefer to live in a busy city or the quiet countryside? Will you be doing a lot of exploring or will you stay near campus? These factors will make a difference to your decision.

    You’ll also need to think carefully about costs. Different parts of the UK have different living costs, depending on things like rent, transport and the cost of entertainment. Check out to see what rents are like, to get an idea of how expensive life will be.

    Finally, the type of university itself is important. Some universities have campuses where the whole of university life takes place, like the . Others are more spread out, such as the , which has buildings all over the city. Each has its own atmosphere and you should think about which you prefer.

    Look at the university rankings

    Find out how your preferred universities rank in the league tables. Rankings of the top universities in the UK don’t just measure teaching quality. They show data on student satisfaction, post-graduation employment rates and staff-to-student ratios.

    The best universities in the UK and courses with higher rankings will be more competitive to gain entry to. You can apply for up to five courses on your UCAS form, so choosing a range of courses and universities is a good idea. Include an ambitious choice which you may or may not get the grades for and safe choices that you know will accept you.

    You can check the , or the to see which universities are the most highly rated.

    That being said, student life in the UK is about balance. You need to consider both rankings and what’s right for you. It’s no use going to a top-ranking UK university in a location you hate, studying things that don’t interest you and where the culture on campus doesn’t appeal to you.

    Sometimes it might be better to consider a lower university ranking if it can offer you plenty of chances to thrive and achieve your personal goals.

    Remember, the reason that there are so many different kinds of universities is because everyone is unique. If you’re applying to somewhere you know matches your values and interests, you’ll be much more likely to succeed in your university career.

    Research the teaching style

    It’s a good idea to look at the style of teaching and assessment at your chosen university. Some courses focus more on final exams, others on coursework, projects, dissertations and independent research.

    See whether your university’s exams are open book or closed book. If they’re closed you have to remember a lot more, but if they’re open you will probably have to give more detailed answers in your exams.

    If you can attend open days and chat with other students and potential professors, do so. But if you can’t, remember that each university course website will tell you about the course content and how it is assessed. You need to make a choice based on what type of assessment will suit you best.

    Some courses are taught in large lectures and seminars; others, such as many colleges, will teach you in small groups or one-to-one with a tutor. The teaching style that works best for you will be crucial in your choice.

    Consider your hobbies

    Every university has its own culture and social life based around the student unions and societies. For local and international students, university life is about more than studying.

    You will have a more balanced student experience if you make time for your hobbies. You can look on the university website to learn about student societies and clubs and see which activities are available. But don’t worry if you don’t see anything that appeals - you can always be proactive and start your own society.

    Do you have a sport that you love to do? If you do and are really good at it, you may be eligible for a . There are all sorts of funding and awards available, so even if you think you won’t get anything, it’s a good idea to check.

    Make sure you meet the English language requirements

    You’ll need to pass a test of English for your UK student visa requirements. You can take any Secure English Language Test (SELT) approved by the Home Office, the UK government department responsible for immigration.

    PTE Academic can be used for UK student visas for degree-level courses, and to access 99% of universities in the country. The test takes two hours, covers reading, speaking, listening and writing, and is done on a computer. Results usually come back within 48 hours, making it convenient and reliable.

    Choosing a university requires a lot of thought. Luckily, choosing an English test to get a visa is easy.