When the Secret Shopper Sits for the Exam

One of Caveon’s professional services is Secret Shopping, conducted at the request and with the cooperation of testing programs. During a Secret Shopping engagement, a trained evaluator books a real exam slot and behaves like an ordinary candidate while probing for gaps in live security. Some log in from a bedroom laptop, others check in at busy test centers.

Their mission is simple: behave like an ordinary candidate while attempting the kind of low-tech violations that any determined test-taker could try. The goal is not to embarrass proctors––most are diligent, polite, and doing their best––but to test whether the security measures that programs rely on actually work.

The data and stories below are typical of what we observe in these experiments day after day.

These observations, and the failings they expose, frustrate us, because they show how often well-meaning programs depend on safeguards that cannot deliver the security they promise.

What secret shopping is, and why testing programs use it

Think of ‘secret shopping’ as the assessment world’s version of an undercover health and safety inspector or a penetration test in cybersecurity. An inspector might try to purchase alcohol without showing their ID to evaluate whether everyone is being carded. In IT, a red-team specialist might see if a mock virus can make it through a company’s firewalls. These measures help vet the quality of security measures to ensure that they can effectively catch the threats they’re designed to stop.

In testing, ‘secret shoppers’ are brought in to evaluate the efficacy of proctors and the exam administration protocols they have in place. These trained agents enter an exam under genuine credentials, undergo the standard check-in process, and then attempt to perform certain simple violations throughout the exam. For example, they use their phones, sneak in earbuds or smart watches, open notes, dig in their bags, or leave prohibited items on their desks, etc. Throughout the exam, they log whether or not these attempts are caught by the exam proctor, and if so, which remedial actions are taken.

The Numbers Don’t Lie

Before diving into two representative stories, it helps to see the wider picture. At the 2024 National Council on Measurement in Education, Caveon shared aggregate results from our past secret shop missions. Across engagements, one pattern dominates: proctors—whether in person or online—miss the majority of visible cheating attempts, and when they do notice, the session often continues after a warning to the test-taker.

The numbers confirm that these incidents are not outliers but the norm.

Across all Exam Administration Modes (Remote or Test-center):

• More than 90% of scripted cheating or theft tasks were completed with no detection.
• When violations were noticed, proctors most often issued a verbal warning and allowed testing to continue.

Test-center Secret Shops:

• 97% of shoppers carried printed notes or other banned items through check-in.
• 100% accessed those notes during the exam without being stopped.
• 52% smuggled in at least one phone: 85% of that group used a phone to photograph exam items and only 18% were caught, always after the photos were taken.

Remote Testing Secret Shops:

• 91% kept printed notes within reach and used them unnoticed.
• Every shopper had a phone nearby; just over half used it and 96% of those captured photos undetected.
• 60% completed the exam with another person present in the room.
• 31% left camera view for an unscheduled break and returned without challenge.

These results underline how easily everyday tactics slip past proctors.

Secret Shopping of Remote Testing Sessions

Below are the results of two secret shopping engagements of remotely proctored exams. On each occasion, the fake test-taker attempted the infractions listed on Table 1, including: leaving and returning to the room, using notepaper to copy items, hiding a cell phone, using a cell phone to capture items, having someone enter the room, and taping notes to the bottom of the mouse. Of these, only two of the actions were even noticed by the proctor. Only a warning was issued, and the exam was allowed to continue. The rest of the “cheating” actions occurred without being detected.

Secret Shopping of a Test-Center Session

This secret shopping engagement detailed below took place in a testing center with an in-person proctor. The “undercover” test-taker was screened and then entered the testing environment, just like any other examinee.

In this instance, the test-taker was able to hide notes in the bathroom, take an unauthorized bathroom break to read the hidden notes, write down the exam questions they had seen and then return to the testing environment, hide a cell phone in their waistband, and record more than 25 minutes of the testing session.

Patterns we see over and over

Even when proctors are well trained and diligent, six recurring factors open gaps large enough for routine cheating to slip through.

1. Time pressure narrows focus.
   Admission queues, software glitches, and candidate questions pull attention away from the live feed or testing floor at the very moments a determined cheater needs.
2. Multitasking creates blind corners.
   Remote proctors juggle multiple video windows and chat logs. Test-center staff escort test-takers to breaks while watching several screens. Each added duty widens the interval in which misconduct can occur undetected.
3. Searches and scans rarely catch hidden items.
   Desk sweeps miss what sits just outside the webcam’s field of vision. Wand scans miss devices taped below the knee. As covert tech keeps shrinking, the gap between what is searched and what is actually present keeps widening.
4. Warnings replace consequences.
   Terminating an exam triggers paperwork, refunds, and heated disputes. A quick verbal caution feels easier, and most detected violations end that way.
5. Courtesy limits confrontation.
   Proctors are trained to be helpful and respectful. That professionalism can translate into hesitation when decisive action (confiscating a phone, ending a session) is required.
6. Proctors can’t see all threats.
   And most importantly, they can’t see some of the most damaging misconduct which have no visual cues at all: pre-knowledge of stolen items, remote proxy test-takers, AI answer tools running on a second device. Even the most vigilant proctor cannot detect what is happening off-camera, in another room, or inside a candidate’s memory.

Because these fault lines play out in almost every session, Caveon’s secret shoppers routinely pass nine out of ten visible cheating attempts. Had they been intent on real fraud, they could have walked away with high scores and high-value item banks, leaving the program exposed to costly score invalidations and reputational damage once stolen content surfaced online.

A Closing Note

Caveon shares these narratives not to blame or shame proctors, most of whom work hard within the limits of their tools, but rather to show why those limits matter. Programs that depend on proctoring alone may feel secure, yet the evidence says otherwise. Layered defenses—secure exam design, real-time analytics, continuous forensics—offer a clearer path.