Record and Review (R&R) proctoring is a model of online proctoring in which there are no proctors watching test takers while the exam is in progress. Instead, R&R uses recording equipment to capture the video and audio from the exam administration. The recordings are then reviewed at a later time by someone to determine whether any form of test fraud may have occurred. This review could occur a day, a week, or a year later.
If a security infraction is detected during R&R proctoring, the testing program has a variety of responses at its disposal. These responses can range from conducting an investigation, canceling the test score, initiating other sanctions, etc. But what are the security implications of R&R proctoring? Is R&R sufficient to keep your tests secure and your test scores valid, or is there more that needs to be done?
A form of proctoring wherein a proctor observes one or more test takers live, at the time of test administration.
A form of proctoring wherein a test session is audiovisually recorded and stored for later observation.
RECORD & REVIEW CAN STILL BE A VALUABLE PART OF A COMPREHENSIVE SECURITY PLAN—IF DONE CORRECTLY.
Basis: With record and review proctoring, a test taker knows that their actions are being monitored and recorded, and that the recording will be reviewed at a later time. This will influence their behavior during the exam. For some examinees (perhaps most), simply knowing they are being recorded is enough to convince them not to cheat.
Caveat: Rather than be deterred, those test takers who are determined to cheat will likely adapt to be more covert and creative.
Bonus Points: Increase the deterrent effect of record and review proctoring by publicizing incidents of cheaters being caught and facing consequences. This will let examinees know that the security measures are effective, that test security is taken seriously, and that cheating simply isn’t worth the risk.
Basis: Well-trained and thorough auditors can catch certain instances when test takers violated the security rules. Examples of types of test fraud that can be caught by reviewers include catching someone using a cell phone to look up answers, observing an examinee taking pictures of test sessions, getting help from someone else in the room, etc.
Caveat: Many of the most important types of cheating cannot be seen by reviewing a recording—or even by a live proctor.
Bonus Points: Review sessions quickly after they are recorded. Catching fraud early enables you to cancel a test score or administer other sanctions. Inform examinees that their scores are provisional until final review, and that releasing official scores might take several weeks.
Record and review proctoring has its drawbacks, leading to some preventable mistakes. What are they, and how can you avoid them?
Often, the test sessions recorded as part of record & review proctoring are never actually reviewed. This happens because reviewing sessions is a time-consuming and difficult process. Sometimes, it’s easier to just not conduct the reviews at all.
But this allows cheating and test theft to continue absolutely unchecked and with no consequences. Avoid this mistake by scheduling your reviews and treating them as a critical part of your assessment process.
The delay from when the test is taken, to when the review is conducted, and when the decision about the score is finally made can be lengthy. This means security threats will continue unchecked until the review has been completed. For example, a student taking a clandestine video will be able to capture the entire test, share it online, and enable more students to cheat before the test program can know or do anything. Those examinees may even be able to realize the benefits of the higher score (such as grade promotion, increased GPA, etc.) all before the cheating incident is discovered and before their scores can be invalidated. Avoid this problem by reviewing recorded test sessions in a timely manner.
IF YOU’RE USING R&R PROCTORING—
HOW DO YOU MAKE SURE YOUR TESTS ARE STILL SECURE?
Audit: Use an R&R proctoring audit service where trained security professionals review recorded test sessions and efficiently flag suspicious incidents.
Probe: Use other security methods (such as monitoring the web for exposed test content and data forensics analyses) to detect potential clues in your test results.
Review: Be sure to make examinees agree to the fact that reviews of recorded test sessions may take weeks or months to conclude. Remind them that their test scores are “provisional” until the reviews have all been completed.
Design: Protective exam designs make many types of cheating and test theft ineffective (e.g., DOMC™ and SmartItem™ technology). These designs make reviewing test sessions efficient by greatly reducing the number of security threats.
Communicate: Use deterrent communications to convince would-be-cheaters they shouldn’t even try. You can require examinees to sign an oath not to cheat, provide them with a list of automatic sanctions for cheating, or simply publicize the vast array of security methods you are using.
In a Recorded Test Session Audit, trained security professionals use proven processes and procedures to review video-recorded sessions of online delivered exams. Audits can be done as a thorough scrub of all recorded test sessions, as a random or stratified-selection process, or as part of a pinpointed investigation based off of data forensics findings.
Auditors quickly and efficiently screen for obvious signs of test taker misbehavior and other potential incidents, flagging those test administrations for further review. Combing through recorded sessions and following up on potential security incidents helps to ensure the validity of test results.
When using any method of proctoring, it’s important to shore up the security weaknesses by probing your test results for patterns and clues that may only be visible in the data (this process is called data forensics). It’s also best practice to use other security methods, such as actively monitoring the web for incidents of stolen and shared test content.
Conducting forensic reviews of your data and scanning the corners of the internet for stolen content allow you to detect instances of test fraud and uncover vulnerabilities in your testing program. This kind of probing can reveal patterns of answer copying, proxy test taking, large-scale collusion, widespread exam content exposure, and more. Proctoring does none of those things.
It’s important to set candidate expectations prior to testing. Candidates should be aware not only of your security policies against cheating and theft, but also of the fact that your test sessions will be recorded and reviewed for violations. Test takers should agree to the fact that reviews of recorded test sessions may take weeks or months to conclude, and they should be reminded that their test scores are “provisional” until the reviews have all been completed.
Once a score is official, it is very difficult to rescind it. Issuing provisional scores gives testing programs the breathing room necessary to conduct a thorough review and any necessary follow-up investigations. This all ensures that official scores are only issued to deserving test takers.
Secure exam and item designs make many types of cheating and test theft ineffective. Examples of these types of designs include the SmartItem™, the Discrete Option Multiple Choice™ (DOMC) item, or computerized adaptive tests (CATs), which all reduce the amount of content that each individual examinee can see...
Protective test designs actually have the power to reduce the overall number of test security threats. This in turn makes the review process a lot less burdensome. In addition to decreasing item pool exposure and eliminating many forms of cheating and test theft, these designs can also curb ballooning development costs, and more importantly, increase fairness and accessibility.
One of the most powerful things you can do to amplify your security is to communicate its existence. Use deterrence communications to convince would-be-cheaters they shouldn’t even try. You can get creative with your methods, but examples include: requiring examinees to sign an oath not to cheat, providing test takers with a list of automatic sanctions for cheating, and simply publicizing the vast array of security methods you are using.
There are all kinds of different fraudsters out there. Some can’t and won’t be stopped, especially not by deterrence communications. But there is an entire subset of opportunistic fraudsters who can be convinced not to cheat. No method of security is a catch-all, so this step is important.
In this guide, we’ve learned that Record & Review proctoring can play an integral role in any comprehensive security plan. The basic deterrence and detection benefits of R&R proctoring can be amplified by using a number of additional security methods to provide more comprehensive protection, while common pitfalls of the method can be avoided with proper planning. Now you know more about automated, R&R proctoring!
