Explaining computerized English testing in plain English

app Languages
a pair of hands typing at a laptop

Research has shown that automated scoring can give more reliable and objective results than human examiners when evaluating a person’s mastery of English. This is because an automated scoring system is impartial, unlike humans, who can be influenced by irrelevant factors such as a test taker’s appearance or body language. Additionally, automated scoring treats regional accents equally, unlike human examiners who may favor accents they are more familiar with. Automated scoring also allows individual features of a spoken or written test question response to be analyzed independent of one another, so that a weakness in one area of language does not affect the scoring of other areas.

was created in response to the demand for a more accurate, objective, secure and relevant test of English. Our automated scoring system is a central feature of the test, and vital to ensuring the delivery of accurate, objective and relevant results – no matter who the test-taker is or where the test is taken.

Development and validation of the scoring system to ensure accuracy

PTE Academic’s automated scoring system was developed after extensive research and field testing. A prototype test was developed and administered to a sample of more than 10,000 test takers from 158 different countries, speaking 126 different native languages. This data was collected and used to train the automated scoring engines for both the written and spoken PTE Academic items.

To do this, multiple trained human markers assess each answer. Those results are used as the training material for machine learning algorithms, similar to those used by systems like Google Search or Apple’s Siri. The model makes initial guesses as to the scores each response should get, then consults the actual scores to see well how it did, adjusts itself in a few directions, then goes through the training set over and over again, adjusting and improving until it arrives at a maximally correct solution – a solution that ideally gets very close to predicting the set of human ratings.

Once trained up and performing at a high level, this model is used as a marking algorithm, able to score new responses just like human markers would. Correlations between scores given by this system and trained human markers are quite high. The standard error of measurement between app’s system and a human rater is less than that between one human rater and another – in other words, the machine scores are more accurate than those given by a pair of human raters, because much of the bias and unreliability has been squeezed out of them. In general, you can think of a machine scoring system as one that takes the best stuff out of human ratings, then acts like an idealized human marker.

app conducts scoring validation studies to ensure that the machine scores are consistently comparable to ratings given by skilled human raters. Here, a new set of test-taker responses (never seen by the machine) are scored by both human raters and by the automated scoring system. Research has demonstrated that the automated scoring technology underlying PTE Academic produces scores comparable to those obtained from careful human experts. This means that the automated system “acts” like a human rater when assessing test takers’ language skills, but does so with a machine's precision, consistency and objectivity.

Scoring speaking responses with app’s Ordinate technology

The spoken portion of PTE Academic is automatically scored using app’s Ordinate technology. Ordinate technology results from years of research in speech recognition, statistical modeling, linguistics and testing theory. The technology uses a proprietary speech processing system that is specifically designed to analyze and automatically score speech from fluent and second-language English speakers. The Ordinate scoring system collects hundreds of pieces of information from the test takers’ spoken responses in addition to just the words, such as pace, timing and rhythm, as well as the power of their voice, emphasis, intonation and accuracy of pronunciation. It is trained to recognize even somewhat mispronounced words, and quickly evaluates the content, relevance and coherence of the response. In particular, the meaning of the spoken response is evaluated, making it possible for these models to assess whether or not what was said deserves a high score.

Scoring writing responses with Intelligent Essay Assessor™ (IEA)

The written portion of PTE Academic is scored using the Intelligent Essay Assessor™ (IEA), an automated scoring tool powered by app’s state-of-the-art Knowledge Analysis Technologies™ (KAT) engine. Based on more than 20 years of research and development, the KAT engine automatically evaluates the meaning of text, such as an essay written by a student in response to a particular prompt. The KAT engine evaluates writing as accurately as skilled human raters using a proprietary application of the mathematical approach known as Latent Semantic Analysis (LSA). LSA evaluates the meaning of language by analyzing large bodies of relevant text and their meanings. Therefore, using LSA, the KAT engine can understand the meaning of text much like a human.

What aspects of English does PTE Academic assess?

Written scoring

Spoken scoring

  • Word choice
  • Grammar and mechanics
  • Progression of ideas
  • Organization
  • Style, tone
  • Paragraph structure
  • Development, coherence
  • Point of view
  • Task completion
  • Sentence mastery
  • Content
  • Vocabulary
  • Accuracy
  • Pronunciation
  • Intonation
  • Fluency
  • Expressiveness
  • Pragmatics

More blogs from app

  • children holding hands in a line with a parent outside

    11 Offline English learning ideas

    Por app Languages

    In today's fast-paced digital era, online resources and language learning apps have become the popular means for mastering English. However, offline language learning has its own unique charm and benefits. Engaging our senses and connecting with the physical world around us can enhance our language skills in ways that no app can match. In today's language learning blog, we discuss offline language learning activities that can help you towards becoming a confident English speaker, even without an internet connection.

  • Buisness people in a workplace stood around a table laughing

    Business English idioms: Understanding corporate lingo

    Por app Languages

    When it comes to business, communication is key. Knowing how to use business English idioms can really help you express yourself, make connections and achieve success. These colorful expressions add depth and nuance to our conversations, making it easier to communicate confidently and effectively.

    Today, we explore several commonly used idioms you might come across when using English in the workplace, providing examples of how they can be used and shedding light on their meanings.

    Ballpark figure

    A ballpark figure means an estimate or rough guess of a quantity or value. It's a handy way to give a general idea or range without getting too into the details. This can be really helpful during negotiations or planning when you don't need exact numbers. Ballpark figures give you a sense of the bigger picture without bogging you down with specifics.

    In the red

    If someone says they are "in the red", it means they are having a tough time financially and are experiencing losses or negative cash flow. This happens when expenses are more than the money coming in, resulting in a negative balance or deficit. The term originated from using red ink in financial records to indicate negative numbers.

    Blue-sky thinking

    A way to describe a really creative and outside-the-box approach to problem-solving or brainstorming. Basically thinking or brainstorming without any boundaries or limits. A manager might encourage a team to do some blue-sky thinking, allowing them to come up with some very imaginative ideas and concepts.

    Square the circle

    An idiom for attempting an impossible task, originating from a math problem where you can't create a square with the same area as a circle using just a compass and straight edge. Someone may say trying to reach a solution all clients were happy with was like trying to square the circle, implying that it was a difficult and almost impossible task.

    Don't put all your eggs into one basket

    This saying suggests that relying on just one strategy or investment is not wise. It emphasizes the importance of diversification and spreading risks for long-term success. To achieve sustained success, a financial advisor may suggest to their client to diversify their investment portfolio and not solely rely on one option.

    Make ends meet

    The meaning of making ends meet involves being able to complete a task or project using the existing income or resources. It requires efficient management of finances to cover necessary expenses, despite having a limited budget or resources.For instance, if someone is working on a project with a restricted budget, they may need to make ends meet as they cannot obtain additional funds to support it.

    Cover all bases

    This means considering every possible factor and handling them carefully to prevent problems or risks. Making sure everything is examined, and anticipating any potential challenges or situations. A business doing a report or analysis may want to "cover all the bases" to prevent any surprise issues or gaps.

    The origin of this is likely a reference to baseball. When playing defense, the goal is to protect or 'cover' your bases and stop the other team from getting points.

    Get down to brass tacks

    To solve a problem or task, by focusing on the important details and practical steps needed for success. Avoiding unnecessary distractions and concentrating on what's essential.For instance, a project manager might say, "Let's focus on the necessary steps we need to take to complete this project on time by getting down to brass tacks."

    Playing hardball

    "Playing hardball" means being tough and unwavering during negotiations or business deals. It shows that someone is assertive and firm in defending their interests. If the other party is unyielding, a skilled negotiator may suggest "playing hardball" to get a better deal.

    These are just a few of a long list, but by adopting idioms and English phrases like these into your professional interactions, you can strengthen relationships with colleagues and showcase your understanding of the business environment more clearly. Plus, it can give you a boost of confidence in your language learning and in navigating the complex world of corporate English.

    If you want to refine your business English skills even further, make sure to check out our blog post5 ways to politely say no in Business English. You can also check out Mondly, an app developed to help you learn languages for various realistic scenarios.

  • A man smiling in a coffee shop, with a coffee in one hand and paying for it with his card in the other

    Studying in the UK: A students' guide to budgeting

    Por app Languages

    Studying at a UK university can be an amazing experience, but it comes with financial obligations. If you're thinking about studying abroad, it's essential to thoroughly research the cost of living in the UK and create a budget that works for you.

    We’ve created a simple guide to help you get an idea what to think about when it comes to researching the living expenses in the UK for international students. To make sure you're prepared before you head off to the UK.

    Consider all costs

    As you prepare for your trip abroad, there are various expenses you'll have to pay for before leaving your country. These one-time expenses include fees for a UK student visa, university fees for international students, and course materials.

    Although these costs can be significant, it's important to include them in your budgeting.

    Recurring expenses

    After moving to the UK, you will have regular expenses to take care of such as rent, transportation, household and phone bills. These expenses remain consistent throughout the year and can be easily planned for.

    Unplanned expenses

    As an international student in the UK, you may face various living expenses that can vary based on your individual circumstances.

    For example, If you often travel on weekends or spend a lot of time going out with your friends, it's best to overestimate those expenses.

    It's important to consider your expenses for socializing, shopping for clothes, and pursuing hobbies. If you have a limited budget, it's crucial to monitor these expenses closely.

    Researching the cost of living in your area

    If you're planning to study in the UK, keep in mind that the cost of living will depend on your location. Living in a big city versus a small town can result in significantly different accommodation costs.

    For instance, the average one-bedroom rental can range from £1100 in a smaller city like Oxford to £1,695 in metropolitan London.

    If walking isn't an option for you, then you should consider budgeting for transportation. In Glasgow, a monthly pass for public transportation costs £63, while in Manchester, it will cost you £74.

    If you're someone who likes to stay active and exercise, you may also need to account for workout costs. Even with student discounts, joining the university gym in Cambridge will cost students about £35 per month, compared to £22 at the on-campus facilities in Leeds.

    Where to find local information

    Before deciding where to study, it's important to research the cost of living in different cities and regions to ensure it fits within your budget due to the variation in costs.

    Do you know any international students who have studied at the university you plan to attend? Or is there a student organization you can contact for information?

    Reaching out to current students can provide insight into their daily expenses and they might even offer great tips on where to find good deals or places to avoid.

    The cost of student housing varies based on the type of accommodation you prefer. Are you interested in living on campus, renting your own place, or sharing with fellow students off campus? Most schools provide comprehensive details about housing options, including contact information for potential roommates, landlords, or homestay families.

    Handy tip:

    To help with budgeting as a student in the UK, you can use a like the one provided by the Universities and Colleges Admission Service (UCAS).

    The calculator also offers the convenience of selecting your school from a list for accurate local costs.

    Part-time UK jobs for international students

    With all these expenses, a bit of extra income would be helpful for any student. One possible way to supplement your finances during your studies is through tutoring. You can consider tutoring children or your fellow students at the university. It's important to note that international students are allowed to work up to 20 hours per week during the school year while on a student visa.

    Your university's career center can assist you in finding part-time work opportunities, especially if you are looking to gain experience in your field of study. Maybe you're looking to work in a pub or restaurant to improve your English skills and experience the local culture.

    Scholarships are available for both undergraduate and postgraduate studies, either directly from universities or through government initiatives. Non-EU nationals seeking postgraduate studies can apply for .

    An example of a student budget

    Although it may seem overwhelming, estimating your cost of living only requires a few calculations. Simply add up your fixed and variable expenses, as you would if you were living in your home country.

    As a helpful reference, we have provided an example of a monthly budget for a student living in Nottingham. The budget is calculated using average spending data obtained from a UK student budget calculator.

    Item

    Cost

    Rental apartment

    £385

    Transportation (monthly pass with student discount)

    £49

    Utility bills (electricity, gas, water, etc)

    £46

    Gym membership (university gym)

    £23

    Phone and internet

    £27

    Grocery shopping

    £73

    Bank fees

    £30

    Clothes and personal items

    £50

    Takeaway and eating out

    £70

    Total

    £753

    It's common to come across discounts for students, like deals on food at campus eateries, special phone plans, or lower prices at certain shops and restaurants.

    Don't miss out on these opportunities and make sure to ask about them. Keep in mind that this budget doesn't cover expenses for shopping or hobbies, so take those into account based on your individual situation.

    If you're able to work while studying abroad, it can help ease some of the financial strain. UK residents who require additional support may be eligible for .

    There are lots of regarding financing and budgeting whilst abroad, so make sure to do your research and check them out.