IIMs CAT & The Transition To Computer Based Testing

So what does a move to Computer Based Testing mean to a CATter? Career Avenues conducted an impromptu exercise at work. We ran two similar-level of difficulty tests to a group of test takers. We first administered a paper based test to the group and then a CBT format test to them.  We then recorded their experiences post the test. Also an analysis of their scores gave us an indication of the final outcome of their performance on the test. Finally, we compared it with published data and research articles on student experiences when GMAT went online from a paper based test to a computer based test. And the results are very similar. Please note that the CBT administered allowed the students to see all the questions in a section and move freely between sections. Questions asked were of equivalent level of difficulty to past CATs.

1. Yes, there is a overall drop in attempts and scores of the students when they take the CBTs compared to paper based tests. Also the error rate went up marginally.
2. The drop in scores is across all types of questions (sections) for the CBT.
3. Students with good computer literacy did far better than students with less computer literacy in the CBTs.
4. Students with good computer literacy did better in paper based tests than students with good computer literacy who took the CBTs.
5. Hence, while good computer literacy is an advantage in the CBTs, in general, paper based tests are easier to score than CBTs.
6. When compared to paper based tests, in CBTs the average question solving time increased significantly in Quant and DI, but not so much in Verbal and RC.
7. After getting the answer, while students spent 8 seconds on an average to shade an oval in the OMR sheet on the paper based tests, they took the same amount of time to click the answer in the CBTs. So more time was being spent on solving the question rather than marking the answer.
8. Students opined that they kept coming back to the same question more frequently in CBTs when they were unable to answer a question, compared to paper based tests.
9. Some students reported that even after attempting a question, they ended up reading the entire questions again when they saw the same questions after a significant period of time.
10. Students did find it more difficult to concentrate across all sections in a CBT scenario. This was most visible in DI and RC, and least in Verbal.
11. In RC, they reported that they lost the thread of the passage frequently, and had to re-read parts of the passage.
12. Some students reported that they did far more wild guesswork in CBT format compared to the paper based format.
13. Some students reported more time awareness and time pressure in a CBT format.
14. Some students complained a difficulty that some non-computer savvy students faced was using the mouse to scroll up and down, and also navigation between various pages of the test.

What does it mean to your preparation?

1. Familiarity with computers and computer based testing : Students are expected to have basic familiarity with the operations involved in using a computer. While using the mouse and the keyboard is a very basic function, the CAT test would involve a basic training keeping the least computer literate CAT aspirant in mind. The students need to understand the nuances that differentiate a computer based test (from a normal test) – precious seconds can be lost if the student is not familiar with computers and more importantly a student can slip in the advantage that he has, if she/he has not worked enough on CBT. For a student who has not been familiar with computers, it is going to be a definite disadvantage. Make sure that you solve atleast 30-40 full length CBTs across your CAT prep, starting NOW. Also try to do your preparation of fundamentals, or reading of newspaper on the computer going forward.
2. Adaptability : The number of tests that have been promised by most CAT coaching institutes will not be enough for a student to sufficiently practice and adapt themselves to the new format. We recommend around 30 – 40 tests under varying conditions in order to be sure of optimal performance during the CAT exam. It is quite clear that the surrounding conditions play an important role at the time of the test. Hence the student will do well to have practiced quite a bit and at different times during the day so that he can mould himself to the situation whenever required. CAT aspirants also need to move their learning online as much as possible. This will ensure that they are adapted to and are comfortable working on a computer. Skills such as quick reading of comprehension over the computer require a sufficient enough practice to ensure peak performance at the time of the CAT exam. They are many computer based test preparation modules available in the market. Students should try and prepare as much as possible over a computer to ensure that they develop a habit of working seamlessly on the desktop. And they need to ensure that they answer atleast 5 papers of each of the 4 types of question papers mentioned in the first article of this series.
3. Quick SWOT : In the new scheme of things, it is even more important to know your strengths and weaknesses well before actual CAT. We suggest that by August, map out your strengths and weaknesses topic-wise and chapter-wise across all sections of CAT. This will help you draw out a clear development plan to address your weaknesses and give sufficient time to fine tune your test taking strategy on live CBTs. And stress management and time management become the key strategic strengths to have. In a paper based format, you could easily search for easy questions for you, but to do the same in a computer based test takes that much more time. And if the test is of the format one question at a time, then you need to know which questions to quickly skip and which question to spend some time on. In a single question format CBT, very often you may end up with one of these two cases. A) Spending too much time on a difficult question early on and missing out even seeing an easy question, and B) Not spending sufficient time to answer a question and skipping it, and later realizing that you have time left but you have reached the end of the section. A SWOT and a plan assist you in addressing these errors.
4. Focus on a personalized strategy for each section : Strategies like speed reading are not likely to be of great help, but quick calculations or answering the questions mentally or concentration are good strategies to have. Also, you need to build a personalized strategy to address specific areas. So to improve your scores in Quant while someone would be able to work more on mentally solving questions, other would rely more on process of elimination or fitting in numbers. These strategies need to be tested in an actual CBT as a strategy may work on a paper based test but may be tough to implement on a computer based test. So figure out what strategies work for you by September end.