24 Nov Why the Naysayers are Wrong About Computerised Adaptive Testing

Computerised Adaptive Testing (CAT), is the latest technology innovation for improving talent assessment. Unlike traditional testing, CAT adapts the test to the test-takers. Traditional tests need to have questions with a range of difficulty so that they can fairly test candidates of all abilities. Unfortunately, this means that candidates in the high or low ranges are likely to be asked questions which are far too hard or far too easy for them. CAT solves this problem by adapting the questions asked based on candidate’s previous answers.  For example, if a candidate answers the first few questions correctly, CAT will be able to surmise that they’re on the higher end of ability, and jump straight into the meat of the test – asking the hard questions where they can show how smart they really are. Ultimately, this means that tests can be made shorter, harder to cheat, and less frustrating.

However, as with any new technology, CAT has its critics. Here are 3 of the most common attacks made against CAT – and why they’re unfounded:

• It’s too costly: In order to be maximally effective, Computerised Adaptive Tests need to be administered to a large sample of testers in order to ensure accuracy. Naysayers suggest that this higher upfront cost means that adaptive testing is not feasible. But the reality is that any accurate testing initiative will require thorough testing before its accuracy can be ensured. And while CAT may increase these costs initially, this is more than accounted for by speed of testing and improved accuracy in identifying candidates with the greatest potential to perform.
• It makes it harder to compare candidates: Some would say that comparing two candidates who have taken different tests is like comparing apples and oranges. However, each and every question in an adaptive assessment is carefully weighted so its difficulty can be compared to other questions. This is part of the reason why more upfront testing is required by adaptive testing—that testing is to assure that questions are balanced and fair to candidates. So no matter who you’re testing you can get an accurate picture of their performance.
• Shorter tests means less accuracy: A longer test should be more accurate right? Wrong. Psychometric tests will contain many questions which are outside of a candidate’s ability level. A super smart candidate will still see many questions which they have a 100% chance of getting correct, and vice versa for low performing candidates. By asking questions based on candidate’s previous answers, adaptive tests avoid redundant questions which are either trivially easy or much too hard for candidates to answer.