Numbers don't lie
A Data Forensics Case Study for State Assessments
THE INCIDENT OF THE "TEST KILLER"
The Value of Data Forensics
Caveon Data Forensics™ is essential for assessing test security threats to your program and protecting the validity of your exams. In addition to being one of the best sources for discovering test security threats and weaknesses, data forensics provides the evidence you need for taking action if your test’s security has been breached.
Key Takeaways:
An incident has been uncovered in a classroom and you want to understand the truth about what occurred, and whether the incident was isolated or widespread. Caveon can help. Using Data Forensics, we illuminate which test results should be trusted and which results may require additional scrutiny.
Numbers don’t lie.
They reveal the truth.
The Situation
During the routine monitoring of a state-wide test, Caveon Data Forensics detected a school that had a cluster of tests with similar response patterns and a high pass rate. The unusual results prompted the SEA to send in investigators, who discovered a “test killer” booklet of harvested questions and answers that had been used to artificially inflate some students’ scores. They confiscated the booklet, and pass rates returned to normal.
This was a test security success story, but Caveon’s data scientists couldn’t help but wonder: what would have happened if the test killer had never been found? Could Caveon Data Forensics pinpoint wrongdoing on its own?
The Question
Caveon’s Data Forensics experts took up the challenge. We decided to see if it was possible to determine the individuals involved in this situation using only data. The results were compelling…
By applying an iterative statistical model designed to analyze the differences and similarities between tests, Caveon experts not only extracted the answer-key that had been shared, but also confirmed the individuals who had used it. Out of more than 23,500 tests, Caveon’s data analysis identified nine tests that had a high probability of using the answer key.
The Impact
The Incident of the Test Killer shows that under appropriate conditions, data forensics can be used to find the source of exposed test questions, as well as the culprits who use it to inflate their scores.
The impact of this type of analysis is powerful. When Caveon analyzes clusters of extremely similar tests from a single location, a model of inappropriate testtaking may be developed. This model can then be used help identify other candidates who gained an unfair advantage through pre-knowledge.
Numbers don’t lie.
With Caveon, they reveal the truth.
READY TO TALK TO AN EXAM SECURITY EXPERT?
Reach out and tell us about your organization’s needs today!
