Security delusions in the name of technological innovation

Being an election year, the security of voting machines is coming under intense scrutiny. The article states, “With the presidential race in full swing, some U.S. states have found critical flaws in the accuracy and security of their electronic voting machines, forcing officials to scramble to return to the paper ballots they abandoned after the 2000 Florida debacle.”

The backlash against security flaws that have been discovered by the states may very well prompt a return to paper-based balloting. “Vendors of the electronic voting machines warn against a rush back to paper.” Industry representatives state that we should “not throw the baby out with the bath water” and “trying to design a voting system when you don’t know how it’s being judged is causing a lot of problems.” State election officials just want secure, tamper-proof election systems.

Security professionals have serious reservations about the security of all voting machines that are based on general purpose operating systems. Such systems provide too many points of entry for an attacker and cannot be made completely secure. Bruce Schneier has a lot of very good essays on this topic. Here’s one of his latest:

A return to paper may be viewed by some as being shortsighted. On the other hand, the security flaws in that system have been thoroughly studied and countermeasures are in place. We must learn that switching from paper ballot systems to electronic voting systems does not mean that security will be better. It means that the attacks and opportunities will be different.

Now, I have digressed. This is an essay on testing and exam security. The lesson to be learned from the introduction of voting machines is that technology does not automatically solve security problems. In fact, adoption of technologies introduces new security challenges. Managers of testing programs that migrate from paper-based testing to computer-based testing in the name of improved security need to be acutely aware that new, unforeseen, security challenges will emerge.

In the same way that most types of election fraud do not involve the actual mechanism by which the vote is cast, most types of test fraud do not involve the actual mechanism by which the test responses are recorded. If you accept the above statement, then you realize that security responses to test fraud transcend the test delivery mode.

Proponents of computer-based testing argue that the tests are more secure than paper-based delivery methods due to limited item exposure, strict time limits, sophisticated verification mechanisms, and strong encryption. After analyzing a lot of data I have come to realize that security is not the result of how the tests are deployed and delivered. I have learned that the above claims are groundless because good security comes from people who follow processes and procedures which have been designed to provide security. Security may be improved using technology, but good security is not derived from technological devices. Any computer system is hackable, especially if the attacker has unrestricted physical access to the system.

Here is a partial list of the “different” security issues that plague computer-based tests.

1 – If the tests are provided in an “on-demand mode” (i.e., scheduled for the test takers convenience in a prolonged testing window), organized test thieves may very quickly harvest the item banks. We have seen tests in this environment that are completely compromised within weeks or days of publication.

2 – Dishonest test site operators can completely reverse engineer the testing engine and database that are on the local machines.

3 – A video T-connector can be used to record any and all testing sessions, allowing easy theft of test items.

4 – Viruses and other forms of malware may be installed on the test delivery workstations allowing for keyboard logging or test result tampering (which would likely be undetectable).

The above attacks have their counterparts in the form of booklet theft, test form photocopying, or answer-sheet tampering in the paper-based testing environment. If you have individuals in positions of trust who you have not certified to be trustworthy, then the integrity of your testing process will be suspect. The point I am trying to make is that security attacks are not automatically thwarted when we use technology. Instead, good security comes from people who follow processes and procedures which have been designed to provide a secure testing environment.

Dennis Maynes

Chief Scientist, Caveon Test Security

Leave a Reply