In this complete guide, you’ll learn about:
So if you’re curious about proctoring, whether it’s right for your program, or the security benefits it offers, then this is the guide for you.
Let’s get started!
For centuries, people have relied on proctoring as their main exam security measure.
In fact, the earliest known use of proctors occurred more than 2,000 years ago on the Imperial Civil Service Exams in China. You can learn more about the history of proctoring in this really neat white paper, but for now, let’s focus on the fundamentals.
In this chapter, you’ll get a handle on:
Then, in later chapters, we’ll touch on the most important question for our time: is proctoring a robust enough exam security measure on its own? We’ll also find out where proctoring needs help bolstering security and what you need to consider when choosing a proctoring vendor for your testing program.
Let’s dive right in!
Good question!
When we have proper exam security measures in place, the results a test produces will reflect fair and valid testing measures and test-taking practices, which are the factors that make the use of test scores trustworthy and meaningful in the first place.
Without exam security measures in place, we can’t really know if the people who are passing their exams really have the knowledge and skills they need to do so!
Examinations would be worthless if we couldn’t trust their results, and security measures are put in place to make sure that we can!
There are four different proctoring practices used today:
Live proctoring is when a test taker is monitored by testing or third-party staff in real time throughout the duration of the exam. This can be done in person or in remote settings, depending on the method of exam delivery.
When done remotely (oftentimes referred to as remote or online proctoring), the exam takes place live over the internet with the test taker and the proctor in separate locations. The test is usually monitored over a webcam, and sometimes screen-shared.
For online proctoring specifically, methods proctors use to protect exams may include:
Whether on-site or online, proctors in any form of live setting must be well-versed in the cheating and test fraud threats that face exams and comfortable with confronting any issues head on when they are spotted.
While each type of proctoring has strengths and weaknesses, overall, they perform a very similar service.
Let’s look more closely at the role of proctors…
Live proctors perform many of the same tasks, whether in person or remote.
Proctors are responsible to:
This is a two-part responsibility that requires proctors to:
In the end, much like the exam programs they help represent, proctors are hopeful that they won’t be called on to handle any test security incidents because they are, in general, not paid very much (and sometimes they are even volunteers!).
Training for proctors is oftentimes brief, but in other instances, proctors are trained for many weeks and must take a certification exam to begin monitoring.
But no matter the training, proctors are asked to do a difficult chore: to stop test fraud as soon as they detect it.
And there’s more to it than that…
To conclude chapter 1, a proctored exam is a test that is monitored by staff or by a neutral third party in order to prevent test takers from cheating or stealing test content.
There are 4 types of proctoring that can occur, whether in person or remotely:
Up next in Chapter 2, we’ll talk about the benefits of proctoring and uncover the different types of cheating proctors are able to catch.
Then, in Chapters 3-7, we’ll cover proctoring’s effectiveness, its drawbacks, whether un-proctored online exams can be trusted, and more.
Onward!
The technology used for online proctoring can improve with time and increase the effectiveness of proctoring. For example:
Cheat sheets used during the exam
Phones or other devices that can be used as cheating aids
Earpieces that examinees use to receive answers from a third party outside of the testing room (In-person proctors)
Conversations among other examinees
Other conversations with people in or near the testing space who aren’t taking the exam
Examinees leaving the testing space
Obvious illicit materials in the exam space (textbooks, notepads, etc.)
Copying answers from another test taker (in-person proctoring only)
With the above scenarios in mind, you can imagine the pressure is high for proctors, especially when they’re monitoring very high-stakes exams!
In the best of circumstances, proctors can detect many security breaches.
Important to note, however, is that proctors can’t catch everything…
In fact, proctors can’t catch many of the most nefarious and effective methods of cheating or test content theft (but it’s not their fault!).
And while some forms of proctoring are suitable for high-stakes testing, others actually increase the risk of cheating!
Up next in Chapter 3, we’ll talk about the drawbacks of proctoring and cover all the types of cheating proctors aren’t quite able to catch on their own.
Here we go!
In Chapter 2, we explored the benefits of proctoring. In this chapter, we’ll discuss some of the downsides of proctoring.
And you’ll also learn about the types of cheating proctors can’t catch!
Let’s get started!
Simply put: No.
Many testing programs still rely a bit too heavily on proctors—particularly when proctoring is a program’s main test security strategy.
As we know, there are always individuals (even in proctoring) who are willing to cut corners without understanding or disclosing the security problems to an unwary testing program. Then, it isn’t until later when widespread cheating comes to light that the testing program actually recognizes the flaws.
Therefore…
Until testing programs understand the many possible security risks and create technology and procedures to protect the exam, proctoring will remain appropriate for only low-stakes exams when used as the sole security measure.
(And even then, most forms of cheating can elude proctors!)
There are three examples of security risks associated with proctoring (and some that proctoring may even introduce):
A serious security and privacy risk, proctors who have this ability could view a test taker’s screen, see the questions presented, and identify the answer provided by the test taker. This provides opportunity for both collusion with proctor / examinee and harvesting of test content to sell it on the internet.
Many cameras don’t provide the proctor with a sufficient view of the testing environment during the exam. When this is the case, it is almost impossible to detect most attempts at cheating. Additionally, these cameras can’t spot hidden cameras or other testing contraband. Check out the picture at the end of this section and see if you can spot any hidden cameras!
Having examinees display a government-issued ID close to the webcam lens is not an effective authentication measure. Not only are there issues with privacy and ease in the falsification of government-issued IDs, but there are also difficulties in verifying IDs in the best of circumstances—let alone through a webcam lens. Such an authentication method would be easy to circumvent. You can view page 8 of this white paper to learn more.
Can you spot any hidden cameras? (There are five!)
Slide the arrow below to reveal the answers!
In addition to the security risks associated with proctoring, it is important to note that proctors have a limited number of test-taking malfeasances that they can identify and confront. Let’s take a look at what proctors can’t catch—no matter how vigilant they may be.
Pre-knowledge refers to examinees who have already seen test questions (whether from a braindump site or from previously taking the test) and use that prior knowledge to get a better score on their test. And no matter how vigilant the proctor may be, no human or machine can detect whether an examinee has seen test questions beforehand. You can learn more about pre-knowledge in this article.
Test takers can bring small hidden cameras to their in-person and remote exams to record test content and later sell (to braindumps online or to memorize). Hidden cameras can go on shirts, buttons, contact lenses, eyeglasses, brooches, watches, pens, and more. You can learn more about the dangers of hidden cameras in this white paper.
While less of a widespread issue, examinees can cheat by hacking into the testing server and stealing exam content. (Think Ferris Bueller’s Day Off—when he hacks into the school’s computer system to change his attendance records—but for exams).
The above scenarios outline just a few scenarios of cheating and test fraud that proctors cannot adequately detect.
(And if you’re more of a visual learner like me, check out this infographic for the types of cheating and test fraud threats proctors can and can’t identify during an exam.)
In conclusion, we learned that proctoring really isn’t an effective security method just on its own.
(Proctors cannot catch test theft, proxy test taking, pre-knowledge, small hidden cameras, or hacked exams.)
We also learned that understanding how proctors affect the quality of test security efforts during test administration is not as straightforward as it may seem. Some of the security risks associated with proctoring include being able to view the test taker’s screen during the exam; a limited head-and-shoulder-view of the examinee during testing; and authentication failures and privacy concerns.
Up next in Chapter 4, we’ll dive into the effectiveness of proctoring and teach you how to maximize its security benefits.
Then, in Chapters 5-7, we’ll find out whether un-proctored exams can be trusted, the specifics of Record & Review proctoring, and what to consider when choosing your proctoring vendor.
Let’s get going!
In the last two chapters, we went over the benefits and drawbacks of proctoring.
In this chapter, we’re going to show you exactly how effective proctors are at protecting your exam.
We’ll also show you how you can maximize proctoring’s benefits to have the greatest impact on your exam’s security!
Here we go!
A proctor’s most impactful test security benefit is in their ability to deter test takers from engaging in fraudulent behavior.
Simply by being present, whether it be online or in the room, proctors have a strong effect of convincing test-takers that it is too risky to attempt to cheat. Examinees know they are being watched, and they must modify their behavior accordingly.
This effect is just as strong—perhaps even stronger—when an examinee is taking an online test; they must assume that they are always being watched, and do not have the in-person advantage of waiting for the proctor to turn away or get distracted before attempting cheating.
Regardless of these nuances, it is clear that proctors create a strong deterrent effect.
Proctors are trained to consistently scan the room or testing area in order to discover any infraction of the rules, including the use of cheat-sheets or cell phones, communicating with others, etc.
If proctors discover cheating or harvesting, they are trained (hopefully) to follow a specific protocol to deal with the incident.
In this sense, proctors serve an important detection purpose, and are well suited to uncover certain types of fraud.
However, as we’ve already read, proctors are not equipped to deal with many of the most harmful forms of cheating, such as detecting the use of hidden high-tech devices, catching proxy test takers, or stopping individuals from using pre-knowledge to cheat on an exam. (See Chapter X for more information.)
For this reason, proctors cannot be viewed as a comprehensive solution for detecting all cheating or theft during test administration.
From a purely technical view, proctors cannot truly prevent (meaning: stop cheating or theft before it happens) any test security threats while monitoring test-takers.
This means that, regardless of what the proctor is currently doing in the testing room, the test-taker can consult cheat-sheets, look at a neighbor or receive help online, or carry out any other form of cheating at any time—none of which can be prevented by proctoring staff. They might get caught, but they can’t be stopped beforehand by proctors.
Thus, the proctor’s role is a relatively passive one, limited to scanning the room or examination area, looking for misbehavior, and attempting to discourage cheating by just being present.
These limitations make proctors relatively ineffective at directly preventing cheating or theft.
The above analysis is a useful one because it provides us with:
Let’s zoom in on that last point. How can we truly maximize the benefits of proctoring?
Programs that utilize proctors (pretty much all programs!) should start by:
Examining your current approach to proctoring
Analyzing the changes you could institute in order to maximize proctoring’s deterrent benefits (this could include increasing the number of your proctors, implementing technology to assist them, improving your proctor training, adding ways to make proctors’ jobs easier, and many others)
Considering how proctoring fits into your wider test security framework
Training your proctors continuously to update them on new threats, remind them of their customer service responsibilities, and more. (This could be done through quarterly training videos with short quizzes, intermittent webinars, email updates, etc.)
Since proctoring cannot be viewed as a one-and-done process for security during test administration, each assessment program that utilizes proctors needs to honestly evaluate whether it is relying on proctors as their primary test security tactic.
If proctoring is your primary test security tactic, then this is a great opportunity to institute other security strategies that will not only fill the gaps that proctoring simply cannot address, but also mitigate or lessen any of the risks presented by introducing proctors to a testing room (as discussed in the last chapter).
Let’s check out some of the security measures that help highlight the strengths and overall effectiveness of proctors.
The following security measures can be used in addition to proctors to increase their effectiveness in preventing, deterring, and detecting/reacting to security threats:
With more sophisticated testing systems, big data approaches, and more complex statistics, real-time data forensics is used to flag candidates for irregular behavior while the test is in progress.
For example, candidate response patterns, timing, and other information can be processed by cloud-based software, enabling the data to be analyzed in real-time. Such analysis could include response similarity, response latencies, and other measures. Some data forensics measures are even built right in to the testing platform!
As a real-world example, Caveon Data Forensics℠ uses latency-based statistics that can alert a proctor that the candidate is proceeding unusually quickly through their examination. The purpose of this forensic measure is to uncover those who could be due to cheating or attempting to steal items. Similarities indices have also been used to flag potential collusion among the candidates or find those who may have pre-knowledge. Such flags, particularly if used in combination with the technologies discussed below, make for a potent combination.
In addition, regular data forensics monitoring should be conducted on a routine basis to detect unlikely test aberrations. For example, running regular psychometric analyses can help you discern whether you have underperforming items and forms and give you an insight into p-values, point-biserial correlations, discrimination values, response latencies, and more.
In case anything gets past your proctors’ ever-vigilant efforts, web monitoring for stolen and distributed test content can be an excellent option to shore up your detection measures.
For example, Caveon Web Patrol® efforts have successfully uncovered XX (thousands) of items from proprietary exams that have been exposed online. And since those testing programs know what content is exposed from their exams and where online it has been posted, they’re able to narrow down who may have used that stolen content to artificially pass their exams and understand how much it has already impacted their program.
AIG is another technique that makes one element of the proctor’s job (addressing pre-knowledge) irrelevant.
A large enough item bank utilized properly has many benefits, including making it so that when exam questions are stolen and exposed online, any exposed items are unlikely to be seen by others on the exam. And even if a test taker has memorized test questions and answers, the odds of it appearing on their test are so low that their effort to cheat is ineffective.
A large item pool also means that testing programs can replace items quickly in the event of a breach, limiting the damage from examinees utilizing pre-knowledge, and makes secure test designs (such as CATs, LOFTs, or even a plethora of forms) possible for all programs. Consider using Automated Item Generation (AIG) as an easy and cost-effective way to expand your item pool.
While most AIG methods currently still have the items reviewed by subject matter experts prior to their use on an examination, it is likely that—in the near future—it will be common for items to be generated on the fly. (See: SmartItems)
Randomizing the order of your test questions and answer choices on each administration is a frequently-used way to prevent cheating from taking place on in-person exams or for virtual exams where test takers may gather and take the exam together.
The idea of randomization is that answer keys from static exams will be useless to cheat from on a randomized exam, and looking at the test of another examinee is worthless since their test form is different.
Test designs such as computerized adaptive tests and linear-on-the-fly testing reduce the overall exposure rate of items, making them difficult to harvest.
Tests with multiple equivalent or equated forms serve a similar security purpose.
Similarly, new types of security-sensitive items are being used that make pre-knowledge irrelevant. The most notable of such new item types is the SmartItem™. The SmartItem renders uniquely, on-the-fly, for every test taker and makes it virtually impossible for test takers to profit by either stealing test content or using stolen content for pre-knowledge.
Adaptive testing is another way to present unique content to each test taker.
For example, with Computerized Adaptive Testing (CAT), or Linear on the Fly Testing (LOFT) tests can be administered more quickly, with fewer items, and more securely.
One of the toughest jobs a proctor has is spotting hidden cameras and other devices. Programs spend extensive time training proctors on how to spot these devices.
But there are a variety of technologies that have been developed to detect hidden devices. Options include (but are not limited to):
Metal-detecting wands: Some organizations have opted to use metal-detecting wands, which detect metal-based devices on the person. However, this can create privacy concerns and is not possible in some countries/regions of the world; further, devices can pass below the sensitive range of such wands.
Radio Frequency (RF) and Camera Lens Detectors: Radio Frequency (RF) and Camera Lens Detectors are two devices that have not yet been deployed extensively. RF detectors pick up the transmission of radio waves that occur from most devices, while camera lens detectors attempt to pick up the reflection of light from a camera lens. Many devices now have both techniques built in, with amateur versions starting at around $100 and professional versions running into the thousands of dollars. However, a reasonably priced device would provide another critical tool in a proctor’s toolbox. It may also detect currently undetectable devices, including new contact lens-based cameras.
When combined with facial recognition technology and a standard high-definition webcam, AI tech can monitor the movements of candidates to recognize irregularities.
For example, facial recognition can read the positioning of the candidate’s head to identify potential aberrant behavior, so a candidate consistently looking down may be reading off a cheat sheet. Similarly, a candidate may manipulate an object on their person, such as a button camera, to take pictures of the screen. In either case, the behaviors can be trained into the system, and your proctor could be notified.
If proctors are on-site, monitoring may be simply a matter of deploying webcams to test center workstations. The use of such technology might also enable lower proctor-to-candidate ratios, and allow proctors to focus more on providing great service to candidates while reducing the number of test room walkthroughs required.
Of course, these technologies should only be employed when there is absolute confidence they do not introduce bias or privacy concerns.
To learn more about the security solutions that can aid proctors in preventing, deterring, and detecting/reacting to cheating, check out this infographic:
In conclusion, it is shocking how little most exam programs actually know about what proctors really do with regards to test security.
Thankfully, you have a leg up!
Now you know how to best enable your proctors to do their job and be vigilant in detecting cheating threats.
Remember: it is important to closely evaluate just how proctors affect the security of your tests—where proctors improve test security, where they have no impact on test security, and where they make testing less secure. We hope this chapter has been a good guide to help you do just that!
Now that you know how effective proctors are at preventing, deterring, and detecting/reacting to cheating, let’s zoom in even further to understand an important frequently-asked question…
Can un-proctored assessments be trusted?
That’s what we’ll uncover next in Chapter 5.
In the same chapter, we’ll also check out the other security measures that ensure trustworthy test scores on your exams.
Then, in Chapters 6-7, we’ll talk about record and review proctoring and the considerations you should keep in mind when choosing your proctoring vendor.
Forward march!
Now that you’ve got the fundamentals of proctoring down, it’s time to jump into some more advanced material.
Specifically, we’re going to show you whether un-proctored exams can be trusted.
You’ll also learn the security measures that bolster the effectiveness of measurement in certification, licensure, and educational exams.
This answer might surprise you: Yes—with the right technology and other security measures in place, high-stakes exams administered today can be trusted even without the added component of proctoring.
Let’s paint this picture with a story:
In 2007, the Boston Globe released an investigative report concerning Army Correspondence Courses.
Senator Edward M. Kennedy, Chairman of the Armed Services Committee, reacted strongly to the report:
The problem is that many online assessments are essentially un-proctored internet tests that don’t have robust enough security measures in place to prevent, detect, and deter cheating and test theft.
NOTE: Unless online tests can be delivered securely, they should not be used for high-stakes exams.
(By definition, an exam has high stakes if passing or failing the assessment has significant consequences for the test taker. Usually, significant consequences entail opportunities such as getting a job, getting licensed in a profession, getting admitted into a school, getting a diploma, getting a raise or promotion, etc.)
There has been interesting progress in the area of secure administrations of un-proctored internet tests. Below are just a few recommendations:
In answering this question, we can’t help but be reminded of the old days when taking unproctored tests as a student at university (back then, they were called “take-home” tests). These take-home tests had implicit security built right into them:
Those are just a few of the “build-in” security measures exams from the mid-20th century oftentimes employed.
There are many types of secure item and test designs that prevent cheating and theft from the start, before they even have the chance to happen.
Examples of secure test designs include tests that use SmartItem™ technology, CATs, and LOFTs, or even just having a large item bank and a plethora of forms. Examples of secure item designs is the Discrete Option Multiple Choice™ (DOMC) format. All of these reduce the amount of content that each individual examinee can see.
And in addition to decreasing item pool exposure and eliminating many forms of cheating and test theft, these secure test and item designs can also curb ballooning development costs, and more importantly, increase fairness and accessibility.
It is best practice to utilize biometrics to authenticate a test taker’s identity and prevent proxy test taking.
Another security measure many universities have added to their online assessments is the tracking of IP addresses. This enables you to determine whether more than one test is being submitted from the same computer.
Algorithms that produce item clones or variants can reduce the ability of test takers to share test content or profit from another’s answers.
Another security measure includes the randomization of answer choices, the random selection of test questions from an item bank, and the prevention of test items from being copied or printed. You can learn more about randomization in this article.
Some test delivery engines (like this one) offer proctor tools that automatically notify you when suspicious events take place during an exam, no proctors needed.
For example, you can receive warnings for examinee behavior such as:
Automated proctoring tools give room for an unplanned investigation into suspected testing malfeasances. It’s great because the test delivery software gives the test administrator the option of pausing, suspending, or resuming the exam at their discretion.
There are test security strategies available that are far more effective at detecting likely instances of cheating than proctoring. Two of the most effective solutions include monitoring the web for stolen and disclosed test content and routinely running data forensics analyses that detect cheating by looking for unusual patterns in your testing data.
Why? Because conducting forensic reviews of your data and scanning the corners of the internet for stolen content allow you to detect instances of test fraud and uncover vulnerabilities in your testing program that would have otherwise gone unnoticed.
This kind of probing can reveal patterns of answer copying, proxy test taking, large-scale collusion, widespread exam content exposure, and more. (Proctoring does none of those things.)
Remember the day that you took your oral exams? There was no faking. (Unless hiring a proxy to take the test in your place.)
You’re in a room, face-to-face, with one or more professors or exam administrators. (In some cases, with educational exams, these individuals may have taught you in at least one class or preparation course.)
Of course, it is not realistic to administer oral exams for every single individual being certified in a profession or being admitted into the university. But it demonstrates the importance of having several observations which together confirm that the candidate does indeed possess the requisite competence. You can learn more about oral exams in this article.
One of the most powerful things you can do to amplify your security is to communicate its existence. Think of deterrence as a communication plan meant to discourage and inhibit test takers from cheating.
You can get creative with your methods, but deterrence can be as simple as letting examinees know that there are consequences to cheating and test theft.
Deterrence messages should:
Examples of deterrence communications include:
For even more ideas, you can head to this article to discover ways you can prevent cheating in online exams.
So there you have it!
With the right technology in place, high-stakes exams can be trusted even without proctoring.
Up next in Chapter 6, we’ll zoom in on what record and review proctoring is and how it differs from other forms of proctoring.
Then, in Chapter 7, we’ll look at the considerations you should keep in mind when choosing your proctoring vendor.
Let’s go!
In the last chapter, we learned that un-proctored assessments can be trusted, so long as there are robust enough security measures in place to prevent, deter, and detect/react to cheating and test theft.
But another question that we frequently get asked is:
“How is record and review proctoring different from regular proctoring?”
Good news… That’s what this chapter is all about!
A form of proctoring wherein a proctor observes one or more test takers live, at the time of test administration.
A form of proctoring wherein a test session is audiovisually recorded and stored for later observation.
Continuous Monitoring
Launched by a Live Proctor
Instantaneous Incident Reporting
More Expensive
Immediate Response to Security Violations
Typically Requires a Third-Party Proctoring Vendor
Ability to Pause or Suspend an Exam
Monitored Later
Launched by an Examinee or Exam Administrator
Delayed Incident Reporting
Less Expensive
Delayed Response to Security Violations
Can Sometimes Be Conducted In-House
No Ability to Pause or Suspend an Exam
Important to note, the “review” portion of record-and-review proctoring could occur a day, a week, or even a year later.
Unfortunately, for many programs with limited resources, often the recordings are never reviewed.
In essence, the main difference between live proctoring and record-and-review proctoring is the ability of the proctor to immediately respond to a security incident.
If a security infraction is detected during record and review proctoring (R&R proctoring, the testing program can respond by conducting an investigation, canceling the test score, initiating other sanctions, or by doing nothing at all.
Because of the delayed nature of the review in R&R proctoring, these decisions occur long after the test was taken.
There are security advantages to using record and review proctoring, particularly when compared to using no form of proctoring at all. The benefits include:
Just remember that with record and review proctoring, it is always best practice to:
And keep in mind: if too much time has passed between the test being taken and the recording being reviewed, it will be harder for you to cancel a test score or administer other sanctions if cheating is detected.
The disadvantages of record and review proctoring include:
Let’s look deeper into these points.
One major limitation of record and review proctoring is the delay between when the test is taken, when the review is conducted, and when a decision is made to confirm or invalidate the test score.
These delays will mean that the security infractions that are occurring in real time will continue to happen until the review has been completed and the problem confronted.
A ripple effect from this could mean that test content has been unnecessarily exposed, which could facilitate further cheating by other examinees—all without the testing program even knowing it is happening.
For example, a student taking a video of each item on the test will be able to capture the entire test, share it online, and enable more students to cheat, all before the test program has caught anything with their record and review proctoring measures.
Furthermore, a person blatantly cheating by using their phone to look up answers will be able to continue to do so on other tests, and they will likely even receive a high score at the end of those exams as well. That examinee will have already realized the benefits of a higher score (such as a certification, grade promotions, etc.) before the cheating incident is discovered and confronted.
Often, the test sessions recorded as part of record and review proctoring are never actually reviewed. This happens because reviewing sessions is a time-consuming and difficult process. Therefore, programs find it easier to simply not conduct any reviews at all! But this is ill advised because it allows cheating and test theft to continue unchecked and with no consequences, thus undermining the validity of test scores.
So say it with me:
Make. Sure. To. Do. The. Review!
As we’ve discussed in-depth in earlier chapters, proctoring is an ineffective method of test security when conducted by itself. There are many security threats that simply cannot be detected by proctors, no matter how diligent and well trained (e.g., the use of hidden cameras, utilizing test content pre-knowledge, memorizing test content, and others).
Record and review proctoring shares these limitations, and it is often even less effective because the review process is either delayed or it doesn’t occur. (Sad, we know.)
Additionally, record and review proctoring has even greater security weaknesses when the reviewers aren’t trained to reliably detect security infractions.
Because of the limitations of record and review proctoring, a program that uses it must bolster its test security in other ways. (You will recognize some of these from the earlier chapter on unproctored exams.) These include:
What is an audit?
In a Recorded Test Session Audit, trained security professionals use proven processes and procedures to review video-recorded sessions of online delivered exams.
Audits can be done:
How does it help?
Auditors quickly and efficiently screen for obvious signs of test taker misbehavior and other potential incidents, flagging those test administrations for further review. Combing through recorded sessions and following up on potential security incidents helps to ensure the validity of test results.
Additionally, this step allows for trained test security professionals to review your exams for you. Here’s one example of an auditing service that covers all the steps. Security folks are specifically trained to detect cheating and theft, and they know what key behaviors to look out for in test session reviews.
What Is Secure Test and Item Design?
Secure exam and item designs make many types of cheating and test theft ineffective. Examples of these types of designs include the SmartItem™, the Discrete Option Multiple Choice™ (DOMC) item, or computerized adaptive tests (CATs). All of these reduce the amount of content that each individual examinee can see.
How Does Secure Item & Test Design Help?
Protective test designs actually have the power to reduce the overall number of test security threats. This, in turn, makes the review process a lot less burdensome.
Two of the most effective solutions for detecting cheating include monitoring the web for stolen and disclosed test content and routinely running data forensics analyses that detect cheating by looking for unusual patterns in the testing data.
How Do They Help?
When using any method of proctoring, it’s important to shore up the security weaknesses by probing your test results for patterns and clues that may only be visible in the data (this process is called data forensics).
It’s also best practice to use other security methods, such as actively monitoring the web for incidents of stolen and shared test content.
Why?
Because conducting forensic reviews of your data and scanning the corners of the internet for stolen content allow you to detect instances of test fraud and uncover vulnerabilities in your testing program.
This kind of probing can reveal patterns of answer copying, proxy test taking, large-scale collusion, widespread exam content exposure, and more. (Proctoring does none of those things.)
How Do You Deter Test Fraud?
One of the most powerful things you can do to amplify your security is to communicate its existence.
Think of deterrence as a communication plan meant to discourage and inhibit test takers from cheating.
You can get creative with your methods, but deterrence can be as simple as letting examinees know that there are consequences to cheating and test theft. Learn more about deterrence messages and see examples in this earlier chapter.
How Does It Help?
There are all kinds of different fraudsters out there.
Some can’t and won’t be stopped, especially not by deterrence communications.
But there is an entire subset of opportunistic fraudsters who can be convinced not to cheat! No method of security is a catch-all, so this step is important.
It’s important to set candidate expectations prior to testing.
At the start, candidates should be aware not only of your security policies against cheating and theft, but also of the fact that your test sessions will be recorded and reviewed for violations.
Just as important, make sure test takers are aware and agree to the fact that the reviews of their test sessions will take time to complete (sometimes several weeks or even months), and that their test scores are “provisional” until those checks have been completed.
How Does Setting Expectations Help?
Once a score is official, it is very difficult to rescind it.
Issuing provisional scores gives testing programs the breathing room necessary to conduct a thorough review and any necessary follow-up investigations. This all ensures that official scores are only issued to deserving test takers.
Use this handy infographic for an overview of these 5 additional ways to boost security when using R&R proctoring:
(Click to zoom or view the image in a new tab)
Record and review proctoring has a few important security advantages. It can deter and even detect some forms of cheating and test theft.
However, since since the nature of record and review proctoring includes having a delayed response, and since test sessions are recorded and viewed for security breaches weeks (or sometimes months) later, it is best suited as a backup form of proctoring when other types of proctoring are not possible (e.g., due to budget or time constraints).
Most importantly: as with all proctoring, record and review proctoring is best utilized with other security measures that shore it up (e.g., data forensics, web monitoring, etc.).
So there you have it!
In this chapter, we’ve learned that Record & Review proctoring can play an integral role in any comprehensive security plan.
The basic deterrence and detection benefits of R&R proctoring can be amplified by using a number of additional security methods to provide more comprehensive protection, while common pitfalls of the method can be avoided with proper planning.
Now you know more about automated and R&R proctoring!
Up next, we’ll look at the considerations you should keep in mind when choosing your proctoring vendor.
Let’s go!
In the last two chapters, we went over the benefits and drawbacks of proctoring.
In this chapter, we’re going to show you exactly how effective proctors are at protecting your exam.
We’ll also show you how you can maximize proctoring’s benefits to have the greatest impact on your exam’s security!
Here we go!
Traditionally, proctoring didn’t offer much peace of mind.
Nowadays, testing programs should ask for—and expect—great communication, accessible data, and hands-on control.
View the below table for more insight into the strengths and weaknesses of the traditional method of proctoring and the modern, online version of proctoring.
A simple, straightforward integration with your LMS should be a requirement for any online proctoring solution.
Gone are the days of spreadsheets and web forms; automation is expected and necessary when transferring exam details.
Are you researching different proctoring styles?
Consider a streamlined approach by selecting a solution that offers a variety of proctoring levels all within a central platform.
An online proctoring solution should never get in the way; it should simply support the process of facilitating an exam online.
If your test takers are experiencing delays (and likely agitation), this could impact their performance on the test.
What assurance do you have that your test integrity is protected?
Without sufficient security, your reputation may be at risk.
Look for a solution that builds additional security measures into their processes, such as machine learning technology and regular exam audits.
Test-takers and instructors must have 24-hour access to support in order to ensure a streamlined experience.
As you assess your online proctoring needs, consider your audience in relation to when and where support will be required.
Curious to see how many test-takers have scheduled their exams?
What violations have occurred to date?
Your solution should offer quick and easy access to all your data, from exam completion rates to violation reports and video files.
By now, you know proctoring alone isn’t enough to prevent, deter, and detect/react to security incidents.
Therefore, your proctoring vendor should be able to:
For example, in the case of data forensics, Caveon uncovers suspicious patterns in your testing data that can uncover threats and expose testing irregularities and other cheating efforts. That data help you learn whether your test takers are colluding or using pre-knowledge, whether proxy test takers are posing as regular examinees, and more.
No matter the route you choose, your test’s security should be thick enough to protect the validity of your exam scores. It should also be robust enough to ensure you won’t have to waste time and money re-writing your exam or responding to needless security breaches that could have otherwise been prevented.
Proctors should never be able to view an examinee’s screen or any of the test questions. Why?
Because proctors can be a security vulnerability in and of themselves!
You should always prevent your proctors and test administrators from:
And since one of the easiest ways your exam content can end up on the internet is through the unauthorized distribution of test content, every measure must be exhausted to ensure test content is viewed by as few eyes as possible.
Therefore, the proctors themselves should never be able to view the test questions.
As discussed in Chapter 3, it is nearly impossible to discern whether an individual’s ID is real, especially over a webcam.
Below is an ID… Can you tell whether it is real or fake?
That said, there are measures for authenticating test takers. Those include biometric signature analysis and other verification methods that make it far more difficult for would-be proxy test takers to gain access to the exam or falsify a passing score for another examinee.
This security feature gives room for an unplanned investigation into suspected testing malfeasances.
Less permanent than terminating an exam, the ability for proctors to pause or suspend the test at the first sign of cheating or test fraud provides proctoring staff the opportunity to take a deeper look into a situation. This prevents the need to move forward with more permanent measures, such as invalidating a score.
In an ideal world, proctors will always be well-trained and highly invested in every test session they monitor.
But that isn’t always the case.
Proctors are not always well-trained, they are not infallible, and like all humans, they can be easy to fool (see how easy it can be to use hidden cameras during a proctored exam in this video).
For example, perhaps your data forensics measures uncovered a potential cheating ring and you need to take a closer look into the individual test sessions of examinees, or perhaps you discovered a cache of live exam questions on the internet and need to find out how widely they were spread.
Whatever the scenario, access to recorded test sessions is a high-priority checklist item to ensure that test takers—and test monitors—are following the rules.
You should expect each of these features from your online proctoring solution. When implemented, your assessment program will have the resources needed to detect, deter, and prevent test fraud.
To learn more about the standards for online proctoring, please view this white paper:
We hope you’ve enjoyed our Ultimate Guide to Proctoring in 2023!
Now we’d like to hear from you:
Is it live, in person proctoring or online proctoring? Record and review proctoring or automated proctoring? Whatever it is, we’d love to hear from you.
