Test Security and Record & Review Proctoring
The Ultimate Guide
what is record & review?
Record and Review (R&R) proctoring is a model of online proctoring in which there are no proctors watching test takers while the exam is in progress. Instead, R&R uses recording equipment to capture the video and audio from the exam administration. The recordings are then reviewed at a later time by someone to determine whether any form of test fraud may have occurred. This review could occur a day, a week, or a year later.
If a security infraction is detected during R&R proctoring, the testing program has a variety of responses at its disposal. These responses can range from conducting an investigation, canceling the test score, initiating other sanctions, etc. But what are the security implications of R&R proctoring? Is R&R sufficient to keep your tests secure and your test scores valid, or is there more that needs to be done?
Live, Online Proctoring
A form of proctoring wherein a proctor observes one or more test takers live, at the time of test administration.
Continuous Monitoring
Launched by a Live Proctor
Instantaneous Incident Reporting
More Expensive
Immediate Response to Security Violations
Typically Requires a 3rd-Party Proctoring Vendor
Ability to Pause/Suspend an Exam
Recorded Test Session Reviews
A form of proctoring wherein a test session is audiovisually recorded and stored for later observation.
Monitored Later
Launched by an Examinee or Exam Administrator
Delayed Incident Reporting
Less Expensive
Delayed Response to Security Violations
Can Sometimes be Conducted In-House
No Ability to Pause/Suspend an Exam
A FRIENDLY REMINDER:
PROCTORING ISN'T ENOUGH
For centuries, we have relied on proctors as our main test security measure.
Unfortunately, proctors themselves just aren’t enough to prevent, deter, or detect the most dangerous security risks.
PROCTORS SHOULD CATCH
- Cheat sheets, phones, earpieces, etc.
- Examinees conversing with one another
- Obvious illicit materials in the exam space
Technically, proctors should be able to catch obvious cheating and theft. But proctors are people, too. Even the most well-trained and diligent proctors can be distracted. Worse, proctors can be offered bribes to look the other way while examinees break the rules.
PROCTORS CAN’T CATCH
- Having prior knowledge of test questions
- Hacking into servers to steal content
- Memorizing content to share with others
- Small hidden cameras that record content
WHAT ELSE DO YOU NEED?
PREVENTION
- Randomize item/option order
- Utilize secure test designs that randomly present test content
DETERRENCE
- Inform test takers of the rules and consequences
- Inform test takers of methods in place for detecting fraud
DETECTION
- Conduct data forensics
- Monitor the web for intellectual property
THE GOOD NEWS IS:
RECORD & REVIEW CAN STILL BE A VALUABLE PART OF A COMPREHENSIVE SECURITY PLAN—IF DONE CORRECTLY.
THE SECURITY BENEFITS OF R&R
Deterrence
Basis: With record and review proctoring, a test taker knows that their actions are being monitored and recorded, and that the recording will be reviewed at a later time. This will influence their behavior during the exam. For some examinees (perhaps most), simply knowing they are being recorded is enough to convince them not to cheat.
Caveat: Rather than be deterred, those test takers who are determined to cheat will likely adapt to be more covert and creative.
Bonus Points: Increase the deterrent effect of record and review proctoring by publicizing incidents of cheaters being caught and facing consequences. This will let examinees know that the security measures are effective, that test security is taken seriously, and that cheating simply isn’t worth the risk.
Detection
Basis: Well-trained and thorough auditors can catch certain instances when test takers violated the security rules. Examples of types of test fraud that can be caught by reviewers include catching someone using a cell phone to look up answers, observing an examinee taking pictures of test sessions, getting help from someone else in the room, etc.
Caveat: Many of the most important types of cheating cannot be seen by reviewing a recording—or even by a live proctor.
Bonus Points: Review sessions quickly after they are recorded. Catching fraud early enables you to cancel a test score or administer other sanctions. Inform examinees that their scores are provisional until final review, and that releasing official scores might take several weeks.
COMMON MISTAKES
Record and review proctoring has its drawbacks, leading to some preventable mistakes. What are they, and how can you avoid them?
Reviewing Never Happens
Often, the test sessions recorded as part of record & review proctoring are never actually reviewed. This happens because reviewing sessions is a time-consuming and difficult process. Sometimes, it’s easier to just not conduct the reviews at all. But this allows cheating and test theft to continue absolutely unchecked and with no consequences. Avoid this mistake by scheduling your reviews and treating them as a critical part of your assessment process.
Delayed Action
The delay from when the test is taken, to when the review is conducted, and when the decision about the score is finally made can be lengthy. This means security threats will continue unchecked until the review has been completed. For example, a student taking a clandestine video will be able to capture the entire test, share it online, and enable more students to cheat before the test program can know or do anything. Those examinees may even be able to realize the benefits of the higher score (such as grade promotion, increased GPA, etc.) all before the cheating incident is discovered and before their scores can be invalidated. Avoid this problem by reviewing recorded test sessions in a timely manner.
THE ONLY QUESTION REMAINS:
HOW DO YOU MAKE SURE YOUR TESTS ARE STILL SECURE?
5 WAYS TO BOOST SECURITY USING R&R
Audit: Use an R&R proctoring audit service where trained security professionals review recorded test sessions and efficiently flag suspicious incidents.
Probe: Use other security methods (such as monitoring the web for exposed test content and data forensics analyses) to detect potential clues in your test results.
Review: Be sure to make examinees agree to the fact that reviews of recorded test sessions may take weeks or months to conclude. Remind them that their test scores are “provisional” until the reviews have all been completed.
Design: Protective exam designs make many types of cheating and test theft ineffective (e.g., DOMC™ and SmartItem™ technology). These designs make reviewing test sessions efficient by greatly reducing the number of security threats.
Communicate: Use deterrent communications to convince would-be-cheaters they shouldn’t even try. You can require examinees to sign an oath not to cheat, provide them with a list of automatic sanctions for cheating, or simply publicize the vast array of security methods you are using.
1. AUDIT
WHAT IS A RECORDED TEST SESSION AUDIT?
In a Recorded Test Session Audit, trained security professionals use proven processes and procedures to review video-recorded sessions of online delivered exams. Audits can be done as a thorough scrub of all recorded test sessions, as a random or stratified-selection process, or as part of a pinpointed investigation based off of data forensics findings.
HOW DOES IT HELP?
Auditors quickly and efficiently screen for obvious signs of test taker misbehavior and other potential incidents, flagging those test administrations for further review. Combing through recorded sessions and following up on potential security incidents helps to ensure the validity of test results.
HOW DO YOU “PROBE” TEST RESULTS?
When using any method of proctoring, it’s important to shore up the security weaknesses by probing your test results for patterns and clues that may only be visible in the data (this process is called data forensics). It’s also best practice to use other security methods, such as actively monitoring the web for incidents of stolen and shared test content.
HOW DOES IT HELP?
Conducting forensic reviews of your data and scanning the corners of the internet for stolen content allow you to detect instances of test fraud and uncover vulnerabilities in your testing program. This kind of probing can reveal patterns of answer copying, proxy test taking, large-scale collusion, widespread exam content exposure, and more. Proctoring does none of those things.
WHY SET EXPECTATIONS?
It’s important to set candidate expectations prior to testing. Candidates should be aware not only of your security policies against cheating and theft, but also of the fact that your test sessions will be recorded and reviewed for violations. Test takers should agree to the fact that reviews of recorded test sessions may take weeks or months to conclude, and they should be reminded that their test scores are “provisional” until the reviews have all been completed.
HOW DOES IT HELP?
Once a score is official, it is very difficult to rescind it. Issuing provisional scores gives testing programs the breathing room necessary to conduct a thorough review and any necessary follow-up investigations. This all ensures that official scores are only issued to deserving test takers.
WHAT IS SECURE TEST & ITEM DESIGN?
Secure exam and item designs make many types of cheating and test theft ineffective. Examples of these types of designs include the SmartItem™, the Discrete Option Multiple Choice™ (DOMC) item, or computerized adaptive tests (CATs), which all reduce the amount of content that each individual examinee can see..
HOW DOES IT HELP?
Protective test designs actually have the power to reduce the overall number of test security threats. This in turn makes the review process a lot less burdensome. In addition to decreasing item pool exposure and eliminating many forms of cheating and test theft, these designs can also curb ballooning development costs, and more importantly, increase fairness and accessibility.
HOW DO YOU DETER TEST FRAUD?
One of the most powerful things you can do to amplify your security is to communicate its existence. Use deterrence communications to convince would-be-cheaters they shouldn’t even try. You can get creative with your methods, but examples include: requiring examinees to sign an oath not to cheat, providing test takers with a list of automatic sanctions for cheating, and simply publicizing the vast array of security methods you are using.
HOW DOES IT HELP?
There are all kinds of different fraudsters out there. Some can’t and won’t be stopped, especially not by deterrence communications. But there is an entire subset of opportunistic fraudsters who can be convinced not to cheat. No method of security is a catch-all, so this step is important.
In this guide, we’ve learned that Record & Review proctoring can play an integral role in any comprehensive security plan. The basic deterrence and detection benefits of R&R proctoring can be amplified by using a number of additional security methods to provide more comprehensive protection, while common pitfalls of the method can be avoided with proper planning. Now you know more about automated, R&R proctoring!
Curious What Strategies Can Enhance Your Proctored exams?
Reach out and tell us about your organization’s needs today!
