Can unproctored online assessments be trusted?

As more and more online courses are developed and offered, instructors of online courses need to consider the potential for cheating on the assessments. The following article describes some measures being implemented by FGCU (Florida Gulf Coast University):

http://www.nbc-2.com/articles/readarticle.asp?articleid=16460&z=3&p=

One of the measures is to track IP addresses and determine if more than one test is being submitted from the same computer. Other measures include randomization of answer choices and random selection of items from an item bank. The software also prevents the test questions from being printed. Kathleen Davey, Dean of Academic Technology, said, “”You can’t prevent everything from happening. You must rely on the integrity of the individual students up to a certain point.”

Ultimately, the above statement is true. If a test taker is sufficiently determined he or she will be able to successfully cheat on the test or steal the test content.

I have been very interested lately in the security of online assessments. They are becoming more prevalent and indications are that they will become a dominant technology in testing if security concerns can be adequately addressed. The problem is that most online assessments are essentially unproctored assessments. Until unproctored Internet tests can be delivered securely, they should not be used for high-stakes exams. By definition, an exam has high stakes if passing or failing the exam has significant life consequences for the test taker. Usually this means getting a job, getting licensed in a profession, getting admitted to a school, getting a diploma, etc.

Recently, Boston Globe released an investigative report concerning Army Correspondence Courses. Yesterday, Senator Edward Kennedy M. Kennedy, Chairman of the Armed Services Committee, reacted strongly to the report, writing, “I was shocked to read of one website that provides answer keys and boasts that “[w]ith cheap prices and fast service, you can be wearing that E-5 [sergeant] rank before you know it.”

http://www.boston.com/news/nation/washington/articles/2007/12/19/kennedy_urges_army_to_deter_cheating_on_promotional_exams/

The essential problem is that the assessments being used for the correspondence courses are unproctored Internet tests.

I remember taking unproctored tests as a student at the university. We called them “take home” tests. Our take-home tests had implicit security built into them:

  1. They were really hard. You couldn’t just find the answer to the questions in the university library.
  2. You might find someone to take the test for you or help you out, but eventually you would take a few in-class tests (where you couldn’t use your friend).
  3. The tests were written in your own handwriting, which was easily compared with prior copies of your handwritten assignments.

Later, as an instructor at the university we added another twist to take-home tests: Every student got the same problems but with different data and different answers.

The above simple principles highlight the issues that must be addressed to administer a test securely online in an unproctored setting:

  1. Biometrics should be used to authenticate test taker identity.
  2. The questions must not be answerable using simple “Google” searches.
  3. A verification process needs to be in place that allows the unproctored test result to be trusted.
  4. Other security measures may assist with authenticating that the test taker actually did his or her own work.
  5. Algorithms that produce item clones or variants can reduce the ability of test takers to share test content or profit from another’s answers.

I remember the day that I took my oral exams. There was no faking. There was no cheating. I was in a room, face-to-face, with three professors. Each of them had taught me in at least one course. Of course, it is not realistic to do this for every single individual being certified in a profession or being admitted into the university. But, it demonstrates the importance of having several observations which together confirm that the candidate does indeed possess the requisite competence.There has been interesting progress in the area of secure administrations of unproctored Internet tests. I will mention just a few items that I can recall readily:

  1. Kryterion (www.kryteriononline.com) is using data forensics and biometrics to establish that a test is being taken properly.
  2. SHL (www.shlgroup.com) is using an initial unproctored test followed by a verification test in a proctored setting to ensure that the test results can be trusted.
  3. An instructor named Simon at the School of DCIT, University of Newcastle, used an innovative detection system with online unproctored tests that relied on font colors in Word documents to detect cheaters: http://crpit.com/confpapers/CRPITV42Simon.pdf

At this URL: http://www.westga.edu/~distance/ojdla/summer72/rowe72.html you will find a paper that is very interesting in this context.

Two things are clear: (1) online assessment is here to stay, and (2) ubiquitous security solutions are needed if online assessments are to be trusted.

Dennis Maynes

Chief Scientist, Caveon Test Security

Leave a Reply