Four Overlooked Test Security Questions

We’ve all felt it: the panic that sets in when we lose something that matters, whether it is a family keepsake or, in the case of testing programs, stolen exam content. In order to deter loss, we plan ahead, whether it involves locking the door when we leave home or creating a security plan for our exams.

Unfortunately, exam security plans often overlook early development considerations that can help to prevent loss:

  • Secure transfer and storage of exam development materials
  • Clear protocols for subject matter experts and reviewers
  • Wise selection of item types
  • Effective use of item cloning

When these considerations are paired with traditional security practices, including data forensics, web patrolling, professional certification, secure exam delivery, a well-informed security plan, and standards-based proctoring, the risk of catastrophic loss goes down substantially.

1. Does your security plan cover secure transfer and storage of exam development materials?

Over the last few months there have been multiple reports of lost laptops containing sensitive information. For example, a laptop containing personal health information for 9,300 University of Oklahoma patients made the news in October. In November, thieves acquired sensitive information that included personal testimonies and pending investigation data from University of Washington Center for Human Rights. If one of your contractors loses a laptop, are your items vulnerable or are your materials securely stored on an encrypted drive?

2. Do you know if your contractors or SMEs use email to share sensitive testing materials or are your sensitive documents tracked and transferred via a secure repository? A few years back a friend of mine was unhappy with items on a standardized exam that he had been asked to review. In order to gather support for his position, he emailed the exam questions to a limited number of peers at other universities. Of course, those peers forwarded his concerns to a few of their friends as well. While his intentions were good, the program had to incur the costs of a rewrite due to the breach and the exam was published almost 3 months late.

3. Do you have clear protocols for Subject Matter Expert item writers and reviewers? Do they sign non-disclosure agreements? Have you added background checks where appropriate? Once development is underway, it is important to keep security in mind. Do you allow writers and reviewers to view the entire exam as it is developed or does your item writing system allow them to only see the content they create? Even at the end of a project, there are steps to take. For example, are your participants reminded to delete all materials in their possession at the end of each project?

If you have answered no to any of the above, you are vulnerable to an internal breach during development.

4. Finally, when you initially select your exam’s item types do you have security in mind? Have you added a few Trojan horse items to augment your other investigative processes? Are you using Discrete Option Multiple Choice (DOMC) items in order to limit item exposure, deter memorization, and increase security? Do you create clones while you are in the item writing phase so that you have a way to respond quickly in the event of a breach? Even ensuring that you have a sufficiently-large number of items, limiting the exposure of individual items contributes to the security of your exam.

If you have answered no to any of these questions, don’t despair! Discovering your program’s gaps may leave you feeling worried, but simple and cost-effective solutions are available that can remedy those gaps. From web patrolling and data forensics, to secure test development and formal security plans, we have experts in each area who can help you avoid loss and the panic that sets in when loss happens.

Saundra Foderick

Exam Development Manager, Caveon Test Security

Leave a Reply