Dennis on Data Forensics

The ACLU is opposing a pilot project in Rhode Island to track students as they enter and exit school buses. “Steven Brown, executive director of the Rhode Island chapter of the American Civil Liberties Union, [called] the plan ‘a solution in search of a problem’ and saying the school district already should have procedures in place to track where its students are.” “There’s absolutely no need to be tagging children,” he said. “The program raises enormous privacy and safety concerns, he added.”

If my research is accurate, there have been at least four previous projects for tagging children in schools with RFID (Radio Frequency Identification) chips in the United States:

1. Enterprise Charter School, Buffalo, New York (2003) – badges on children, tagging library books, school cafeteria purchases, and visits to the school nurse;
2. Spring Independent School District, Houston, Texas (2004) – school bus pass program that is still operational;
3. Brittany Elementary School, Sutton, California (2005) – badges on children, due to public outcry this project was cancelled which lead to the Senate of the State of California debated banning RFID chips to identify people in the state; and
4. Tucson Unified School District, Tucson, Arizona (2007?) – school bus pass program that may still be under discussion.

In the UK, two clothing manufacturers are sewing RFID tags into school uniforms for the express purpose of tracking students while they are in school. RFID tags sewn into clothing are not new, and neither are RFID badges in the work place. And, now RFID badges are being used in universities. The University of Chicago is revamping all their student id cards primarily to ensure secure building access. Another application of RFID technology is a label affixed to your cell phone at Slippery Rock University, north of Pittsburgh, Pennsylvania, which allows for payment processing.

The idea of chipping children is controversial. On one hand, privacy advocates warn of possible abuses and intrusions. On the other, security proponents promote increased safety. In between are administrators who want improved efficiency and convenience. No one is seriously considering implanting RFID chips into children yet. But, this is happening for patients with Alzheimer’s and dementia, as well as being seriously considered by the British government for prisoners. And, the University of Washington has started a human experiment in the computer science building to assess the possibilities of RFID tracking. Several states have passed legislation prohibiting a person from being forced to accept RFID implants, which are approved by the FDA. (Image source.)

There is no doubt that RFID tags can be abused. With an RFID reader, a bad person can gather information about you surreptitiously. A bad person with a database can profile you, and even create an inventory of your belongings. But this potential exists today, even without ubiquitous RFID tags and readers. Bad people with cameras can gather information about you surreptitiously and create an inventory of your belongings. They do it to children, families and the elderly. The concern about RFID is that this can be done more efficiently. If you use membership cards and discount shopping cards, your purchases may already be tied to you in some database, somewhere, that at sometime in the future may be hacked by someone. To a certain degree, the anti-RFID movement promotes the fear that at some time your information will be stolen.

We should be aware that the school district personnel who are investigating this technology are trying to solve real problems. It’s important to keep track of library books. And, it’s even more important to know that students are entering and exiting the school buses at the proper times and locations. We live in a changing world and what we dismiss as paranoia today may become essential tomorrow. For example, there were no lockers in my elementary school. The first time I saw a locker was in junior high. While in high school, we moved to a small town and the lockers did not lock in that school (unless you brought your own lock from home). Given our changing world, I would be extremely surprised if this situation still exists in my alma mater.

I’m somewhat surprised that the ACLU has not opposed cell phones in schools. Who would have ever imagined that in the name of privacy and safety we would allow everyone to carry a camera to school and take a picture of anything there (e.g., students spitting in a teacher’s water bottle or a teacher filming the girls bathroom)? But that is precisely what has happened with cell phones. We can’t pry cell phones away from students. There is also great potential to abuse cell phones, as demonstrated with the FBI’s ability to remotely activate a cell phone’s microphone and use it to eavesdrop on nearby conversations. If cell phones are ever fitted with RFID tags, this entire debate could be over.

Mr. Brown from the ACLU is right. There are safety and security concerns with RFID devices. However, he doesn’t seem to understand those concerns. RFID chips can be hacked and the information from those chips can be transferred to other chips. As an example if the RFID card allows access to a secured area, a bad person may pilfer the electronic codes and in essence make a copy of the electronic key, as demonstrated by James Van Bokkelen. If the chips do not have proper electronic safeguards the information may be overwritten or used illegitimately.

While I have not directly addressed testing, there are implications for using RFID chips in testing which I will discuss the next time I write. But today, I just couldn’t resist this topic. In my opinion, we need to ignore the fear mongering and we need to use this technology wisely. RFID technology is not a panacea, but it can solve real problems.

The Association of Test Publishers (ATP) Conference of 2008 ended yesterday. As always, it was a good conference. In 2004 we stated, “You can’t manage what you don’t measure.” Being a sponsor of the conference, we placed a bag of M&M’s (i.e., manage and measure) in each attendee’s conference packet. And, we printed the message on the hotel room key cards.

I have just completed analyses for three testing programs and I am so impressed with what they have done that I want to share their results with you. Good news concerning exam security is refreshing in the midst of so many cheating stories. We recognize dramatic acts of heroism, but often ignore the good that happens with steady, persistent progress. I am so proud of these three programs. They are achieving their common goals: “Reduce cheating, strengthen exam security and emphasize ethical test taking.” The data demonstrate this convincingly. Caveon’s message at ATP this year was, “The answer is in the data.” So let’s look at the data.

Figure 1: Percent of anomalous tests for three programs

Let me describe the data in Figure 1. The percent of anomalous tests for successive analyses are plotted for each program. A trend line has been fit to the data to aid your eye in visualizing the trend pattern. An anomalous test is one that deviates from normal test taking, and will exhibit at least one of the following: aberrance (answering hard questions correctly and missing easy questions), large numbers of erasures, inexplicable score changes from a previous test score, or excessive similarity in the selected answers with at least one other test. An anomalous test does not mean the test taker cheated. For example, when we observe excessively similar tests it is very likely that one person cheated (the copier) and the other person did not (the source). The percent of anomalous tests does not measure the precise number of people who have cheated, but it is highly correlated with that number.

These data are important because they demonstrate that all high-stakes testing programs, irrespective of industry or application, can effectively reduce cheating. They illustrate that reductions in cheating can occur with persistence and dedication. Let me briefly describe each program and some of the positive steps they have taken.

Program 1: This program provides a professional certification with high security. We estimate that there was a 45% reduction in cheating in three years. They have followed up on every case that appeared to be a security violation and every test site that appeared to have lax security. They have emphasized proctor training. They are now reviewing their test taker agreements, proctor training, identification procedures, and physical security with the intent of using the best known security protocols.

Program 2: This program is a public education program. We estimate that there was a 72% reduction in cheating in two years. They have rewritten their test administration manuals and have begun test administration monitoring. They assign a conditional status to extremely anomalous test results and require local review of those test results. They are receiving reports that the students being flagged are admitting to having cheated.

Program 3: This program administers tests in the service industry. We estimate that there was a 78% reduction in cheating in one year. They have stressed ethical test taking. They have revised their test taking agreements and strengthened test administration policies to allow for scores to be invalidated with an appeals process. They have refreshed test forms which appeared to be exposed. They are researching the next phase of security improvements: test site monitoring and appropriate disciplinary measures for test administration personnel who may be helping test takers inappropriately.

These very different programs were the same in one important way: They started where they were, they created a plan, and they were not discouraged. Each was taken back by the first data forensics report (we always find something disconcerting), but they pressed forward and executed their plan. Best practices used by these programs include: test site monitoring, emphasis on ethical test taking, invalidating scores as per policy, refreshing tests which appear to be over exposed, and updating their security procedures.

Let’s give credit where credit is due. The numbers are impressive and the data do not lie. These programs have earned our respect and admiration.

They say that cheaters only hurt themselves. In all honesty, I think that a cheater said that and we believed him. It is often the case that cheaters hurt the people who gave them the test more than themselves. If you are responsible for giving tests, some fool will eventually cheat on your test. How you handle cheating incidents can make or break you.

When you started out in your career and you began giving tests, you probably didn’t imagine that the most demanding aspect of your job might be what to do about cheating. The first time you encounter this and when the media spotlight is focused on you, you will probably wish you were a rattlesnake handler or a bomb disposal expert. You create and give tests. And, you’re good at your job. You never intended to become a test cop. Let me suggest that you anticipate and prepare for cheating incidents now, before they happen. We call it security incident response planning.

Speaking of cops, there have been a number of stories concerning police departments and cheating on tests recently. In the summer of 2007, information about the police promotion test in Boston was leaked to several officers, as reported by WBZTV. In another story, theState.com reported that twenty-one police officers in Columbia, South Carolina were implicated in cheating when they either cheated, helped others to cheat, or knew of the cheating but failed to report it. And, Houston’s crime lab was in the news twice for open-book cheating, which resulted in the shutdown of the lab, as reported by the Houston Chronicle on October 6, 2007, and January 26, 2008.

The above stories illustrate the importance of responding appropriately to cheating incidents and testing irregularities. You will not be judged harshly because a few miscreants decided to cheat on your test. But, you may be embarrassed completely if you do not address the problem adequately. Your program may suffer a loss of credibility. The public confidence in those you certify may be eroded. And, adding insult to injury, the media may portray you as a fool and a blunderer.

Your security incident response plan should be suited to your organization’s needs and requirements. There are a lot of questions that you should answer. Let me list a few:

1. What discipline should the cheater(s) receive?
2. Is the discipline appropriate? If it’s too harsh you may be perceived as being unfair. If it’s too lenient you may be judged as playing favorites.
3. When will you inform the public about the security breach?
4. What will you do if the media learns of the breach before you announce it? Or, before you learn of it?
5. Is an investigation needed?
6. If so, how will the investigation be conducted? Who will conduct the investigation?
7. What information will you share with the media?
8. What information will you keep confidential? What justification do you have for not sharing everything?
9. Who will be responsible for communications and media relations?
10. Is your security incident response plan recorded in policy form to guide you?

As you can see, my list focuses on “doing the right thing” not just on “looking like we are doing the right thing.” Reporters, in particular, are very quick to suspect a cover up or to suspect they are not being told the truth. And, if you are responsible for a testing program which is accountable to the public (e.g., tests in schools or tests involving public safety), it is vital that you maintain the public trust.

One way that you can sharpen your skills in this area is to “simulate” what you would do in specific cheating situations. During the course of a year, just about every type of cheating will be reported in the news. You can stay current with these stories by reading Caveon’s “Cheating in the News.” You can sign up to receive CITN notification by e-mail about twice a month on the lower right-hand corner of the main Caveon web page. Read the stories. Discuss the stories with your staff. Does your security incident response plan tell you how to handle the problem, if it happened to you?

Just as we expect our local emergency response teams (i.e., police, firefighters, and paramedics) to prepare for disasters, we should prepare to handle cheating incidents. A properly executed security incident response plan can keep an incident from becoming a disaster.

Ten years ago the New York Times criticized ETS, claiming that ETS elected to keep quiet rather than publicize exam security breaches. When you, as a testing program manager, have a full-scale security breach on your hands what will you do? I can imagine that it was a very difficult decision within ETS whether to “keep the lid on” the story or to let the story be told. This appears to be a “no-win” situation. If you publicize the security breaches you may seriously undermine your testing program. If you keep quiet and the word leaks out, like it invariably does, your own credibility may be questioned.

Read stories of cheating in the news to learn how the media might portray your cheating incident negatively. Journalists print newspapers and sell advertising. Sensational news is good copy for them. It is especially important, when under spotlight of the press, that your testing program be viewed as being fair, responsible, and ethical. In my experience, reporters will probe for any apparent contradictions, irregularities which could have been avoided, or supposed dismissal of the severity of the situation. If they find any thing that might be construed as an irregularity, it will probably be printed. In my opinion, it’s better to tell your own story first, rather than let reporters interpret the situation in a potentially harmful manner.

I wish you the best as you formulate your security incident response plan. If you could use additional guidance in preparing your security incident response plan, a Caveon test security director will be glad to consult with you.

The State of Florida recently imposed a cell phone ban on students while taking the FCAT. All the parents of school children in the state received a letter explaining the ban. On the other hand, the Legislature in the State of Utah voted down a bill that would require school districts to establish policies governing cell phone use. The sponsoring legislator said, “[Cell phones] can be used to cheat. We’ve had inappropriate photos transmitted. The problem is pervasive.” An opposing legislator was quoted as saying that “he thinks electronic devices could be better used in education and wouldn’t necessarily like to see policies that simply prohibit them.”

In another story last week reported by Wave3 of Louisville, we read: “Teachers at Oldham County High say they’ve had problems with students using their cell phones to cheat in class. ‘I saw a boy texting under his desk during a test. Then I picked it up. Clear as day it said number five –D- and I took it to the office and we were able to trace the number and it was to another student in the same class,’ said Newkirk.” Now contrast that experience with this column from the Muskegon Chronicle, where the writer claims that gadgets don’t help cheaters. The following points were made:

1. Yet research indicates that cheating in high school and college isn’t any more common today than it was 30 years ago.
2. “And 99 percent of cheating is still done the old-fashioned way, like copying from a neighbor,” said Scott Gomer, media relations director for ACT,
3. But in her four years at Northview High School in Plainfield Township, Kelsey Perras has heard of someone pulling out a cell phone to take a picture of a test “only once,” she said.
4. “High-tech cheating isn’t really something you see a whole lot of,” said Hudsonville High senior Travis Martin. “Most people won’t pull out their cell phone during a test. It’s tough to make that discreet.”
5. Perras said cheaters at Northview are caught more often than not.

Muskegon must be a very sheltered place with extremely astute teachers. The credibility of each of the above statements is easily challenged. I see a lot of data and from what I see, I feel very confident in stating that cheaters are rarely caught. The only way that I can explain some of the cheating I see is through wireless communications. And, research from the Josephson Institute and Center for Academic Integrity convincingly shows that cheating in school is rising and has been rising for the last two decades.

Confusion concerning cell phones in schools is raging throughout the whole country. The issue is being intensely debated in New York City where it has spilled into the court system. Last spring the New York State Supreme Court upheld a ban on cell phones imposed by New York City in 2006. The Supreme Court decision is now being challenged in appellate court.

Surprisingly, security arguments are given by both sides of this debate. Proponents of cell phones argue that parents and administrators need constant contact with students, because without constant contact student security is jeopardized. Opponents of cell phones in schools cite privacy violations with videos posted on the Internet of students in restrooms and teachers disciplining students. And, of course, they do not overlook the implications of cheating. As reported by WSAZ, the solution at Marshall University has been to allow each instructor to determine in their course syllabus whether cell phones during tests are banned, but to not restrict cell phone use on campus.

Penn State has addressed the issue by creating secure testing environments, where the computers do not have Internet access and where cell phone transmissions are silenced. The technology they are using includes: secure workstations, cameras and monitors on every test taker, and metal-lined testing rooms (known as faraday cages) that passively prevent wireless communications. While this may seem extreme, contrast this with the exam breach of 2004 in South Korea where 314 test results were invalidated after police discovered answer keys being transmitted using text messaging. As another example, consider the January 2, 2008 report by the Boston Globe where firefighters in Boston sent text messages from the restroom to cheat on their exams.

It is clear that cell phones are used to surreptitiously cheat on tests. People, in general, feel strongly that cheating shouldn’t be tolerated on tests. We don’t want doctors, lawyers, nurses, accountants, firefighters, police or any other person who provides a service to us to be an incompetent, bumbling cheater. On the other hand, the public sentiment appears to be confused when it comes to setting aside the cell phone while an exam is being given. The public seems unwilling to restrict the individual privilege of being able to communicate with a child in school while taking a test in order to prevent cheating.

The principle of fairness and integrity dictates that students should have a level playing field. It is very difficult to convince me that the playing field was level when cheaters in China were caught with radio receivers in their shoes:

Police in Jiutai, in the northeastern province of Jilin, became suspicious when a mini-bus remained parked outside a school hosting the exam on Thursday, Xinhua said.

Inside, they found three people, “two of them staring at a computer screen and talking into a walkie-talkie,” Xinhua said.

A student in the examination hall used a wireless microphone to read out the questions and received the answers from the van; Xinhua quoted their confessions as saying.

Police had found some 42 pairs of so-called “cheating shoes” with transmitting and reception ability, selling for about 2,000 Yuan each, in a flat in Shenyang, the provincial capital, state media said on Thursday, adding that they — along with “cheating wallets” and hats — had proved popular this year.

There is no confusion in my mind on this issue. But, I’m just a statistician and who am I to know differently?

There is a children’s game known by various names as “Whisper,” “Secrets,” or “Gossip” where a secret is shared and passed from one player to the next. The last player hearing the secret says it aloud, often with hilarious results. These same distortions happen in the news media, as journalists cite other reports or each other. Such a misquote from the Star-Telegram concerning additional security announced by the TEA (Texas Education Agency) for the TAKS (Texas Assessment of Knowledge and Skill) caused me to pause and reflect about using statistical evidence to “prove” that someone cheated on a test.

The reporter wrote, “Among other security measures: … Scramble field test questions on tests to provide proof if someone is copying someone else’s answer sheet.” (Italics added.) http://www.star-telegram.com/news/story/433614.html. Being well aware of the controversy surrounding the use of statistics, alone, to prove cheating, I immediately doubted the accuracy of the above statement. Actually, on June 7, 2007, Shirley Neeley announced that “the Texas Education Agency today will immediately initiate the following: … analyze scrambled blocks of test questions to detect answer copying…” TEA later clarified that the scrambling would only involve field test items. The Dallas Morning News was quick to criticize the scrambling plan, but I applauded TEA’s intent to detect cheating behavior using statistics.

We naturally ask whether statistical evidence can be relied on to detect cheating. Many authors have expressed the opinion that statistical evidence must be corroborated by eye-witness accounts before making allegations of cheating. I can understand this position if the statistics are not reliable. In my opinion, reliable evidence must meet the following conditions:

1. It must be factual,
2. It must be objective,
3. It must be credible, and
4. It must be defensible.

If statistical evidence meets the above conditions, I believe that it can be relied upon, whether corroborating eye-witness accounts are available or not. Statistical evidence is

1. factual when it is based on test result data (an actual record of the test event),
2. objective when it provides a statistic with a probability statement,
3. credible when the statistics have been shown to work because the models accurately depict actual test taking, and
4. It is defensible when the underlying science withstands scrutiny.

An additional fifth criterion the evidence must meet for taking action on a suspected instance of cheating is that the evidence must be strong. Statistical evidence is strong when the calculated probabilities are so small that we no longer believe the observed data are the result of normal test taking. Statistics can provide guidance for determining how strong is strong enough to take action, but ultimately the establishment of a probability threshold (i.e., the strength of the statistic) is a matter of policy that must be answered by the testing program administrator.

It is important with any statistical investigation to choose statistics that are well-suited and designed for the task at hand. For example, if the concern is that answer sheets are being modified, then erasure counts should be analyzed. Having analyzed over one hundred data sets for a wide variety of clients including state Departments of Education, admissions tests, certification programs, and licensure exams, I can unequivocally state that answer copying is the predominant means of cheating on tests. Therefore, it is especially relevant in this discussion concerning the reliability of statistical evidence to discuss answer copying and statistics that are designed to detect answer copying.

As you reflect upon the principles that I have outlined, I would ask you to consider the data in Table 1. The table contains differing probability values that a testing program administrator might be asked to evaluate. These are sampled answer-copying statistics (i.e., counts of identical answers) from a test having 240 items. With this many items on the test, the central limit theorem will generally apply so I have included a Z-Score in the table, as a point of reference.

Table 1: Sampling of test similarity statistics

At Caveon we deal with extremely small probability values, so we typically express those using “an index” where the probability is one in 10 to the power of the index (p=10^-index). The most extreme case in Table 1 has a probability of one in 10 to the thirtieth power. These data are definitely not due to normal test taking.

Assuming that you accept the statistical evidence as being reliable, the decision needed by you, the testing program administrator, is how low in Table 1 should you go? Where do you set the cut point? These data illustrate if you set the cut point too low, you might accuse some individuals of answer copying without having strong evidence. If you set the cut point too high, you might allow several individuals who have cheated to escape discipline.

I will elaborate more on this topic, next time. Until then, may your tests remain secure.

Kathleen Vohs of the University of Minnesota and Jonathan Schooler of the University of British Columbia have created a series of experiments involving cheating on tests as an attempt to associate behavior (specifically cheating behavior) with moral beliefs and free will. In essence, they ask the question, ” ‘If people came to believe that their behavior was the inevitable product of a causal chain beyond their control — a predetermined fate beyond the reach of free will’ how would their behavior change?”

http://www.psychologicalscience.org/media/releases/2008/vohs.cfm

http://www.sciencedaily.com/releases/2008/01/080129125354.htm

This is a fascinating subject and idea. I have personally felt that an individual’s code of ethics is a stronger deterrent against cheating than anything else. Don McCabe from the Center for Academic Integrity has shown through research the positive effect of honor codes (http://www.academicintegrity.org/educational_resources/honor_code_101.php) in maintaining academic integrity. From the Josephson Institute’s 2006 Report Card on the Ethics of American Youth, we read, “Widespread and deep youth cynicism often reflects itself in a rationalization process that nullifies ethical judgment and condones conduct that is contrary to stated moral convictions. Thus, the same youngsters that speak of the importance of ethics, character and trust, frequently lie, cheat and even steal without much guilt or hesitation.” http://www.josephsoninstitute.org/pdf/ReportCard_press-release_2006-1013.pdf

Although eighty-three percent of high school students believe “It’s not worth it to lie or cheat because it hurts your character” (please note that 17% feel that lying and cheating do not hurt your character), fifty-nine percent believe “In the real world, successful people do what they have to do to win, even if others consider it cheating.” In other words we can visualize these young people saying, “It may hurt me to cheat but I must cheat if I want to be successful.” The way the question is worded, definitely affects the response rate. For example, only forty-three percent agree that “A person has to lie or cheat sometimes in order to succeed.” Relevant dishonest behaviors from the Josephson study are shown below:

On September 19, 2006, The Chronicle of Higher Education stated, “More than half of the graduate business students surveyed recently admitted to cheating at least once during the last academic year.” (Survey Finds Widespread Cheating in M.B.A. Programs).

More recently, Caveon’s “Cheating In the News” referenced a story where Marianne Jennings of Arizona State University recounted recent and notable ethics lapses.

http://www.caveon.com/citn/?p=415

After thinking about these issues, I have the following personal observations:

• Honor codes and codes of ethics are critical to maintaining an ethical testing environment. Test taker agreements (e.g., non-disclosure agreements) are essential for all testing programs and should contain honesty covenants.
• Not all testing programs are in a position to require a code of ethics, but if you can do it as a part of your association or organization, you should.
• Even without a code of ethics, you should emphasize the importance of ethical test taking. This should be a fundamental message that is always championed.
• If you believe the adage that “the pen is mightier than the sword,” then you will direct significant resources towards ethics education, while at the same time making an example of miscreants.

In my opinion, the challenge to educate and inculcate ethics into test-taking behavior must begin with an understanding of cultural influences, moral beliefs, and perceived acceptability of such behavior. For example, our data analyses indicate that “cheating for the common good” on tests may be condoned in some Asian cultures. It seems reasonable to conclude that high-profile role models who have been caught cheating will provide others with an excuse or rationalization to “cheat.”

Here are a few other recent articles that are relevant to this topic:

Ethical Conduct continues to Spiral out of control

F for Conduct

Cheaters never win – Unless Penn lets them

Federal investigators are investigating falsified weapons certifications from 2001 where security guards who were supposedly trained to protect federal buildings have been failing weapons tests. Evidently, the contractor who provided the training was also responsible for certifying that the guards could use the weapons. The contractor knowingly did not train, did not test, but did falsify documents attesting to both.

http://www.zwire.com/site/news.cfm?newsid=19011441&BRD=1641&PAG=461&dept_id=594835&rfi=6

The accused has pled guilty to mail fraud in order to avoid more serious criminal charges. However, the breach was the result of a built-in security flaw. It’s easy when the trainer is also the certifier to falsify data. The lie may not be detected for a long time (in this case we are talking about 6 years). The story illustrates that checks and balances are needed in all security settings. Independent reviews and evaluations are good and should be required.

Another huge cheating incident illustrating the same security weakness was broken by KCNC’s Brian Maass. Servisair has the contract for training workers to de-ice airplanes at Denver and eleven other airports. After receiving a tip, a producer at KCNC went undercover and gathered video evidence of a trainer feeding answers of test questions to trainees.

http://www.rockymountainnews.com/news/2007/nov/24/de-icing-meltdown-needs-fast-resolution/

http://www.poynter.org/content/content_view.asp?id=133478

I have knowledge of other situations similar to these, where trainers have given answers to examinees and teachers have tampered with answer sheets. The question becomes, “Who will guard the guardians?” When the stakes are high for both the test taker and the program administrator, test security may be easily breached by program personnel unless opportunities to gain an unfair advantage are minimized and even eliminated.

It is well known that individuals in positions of trust may easily abuse that trust and do a lot more damage to a testing program than a single test taker who cheats. For some reason, these stories keep showing up. We don’t like to believe that people we trust may actually be engaged in test fraud or covering up a few peccadillos (from Spanish meaning ‘little sins’).

A student at the Dayton School of Law is suing the school because it did not fix a glitch in the test administration software that allowed other students to upload pre-written exam answers during the exam. He feels disadvantaged because he had to type in his answers, while others uploaded their answers electronically.

http://www.daytondailynews.com/n/content/oh/story/news/local/2007/11/22/ddn112307lawschool.html

The student said, “I’m not upset with my actual grade, but that other students used the technique to get better grades and the law school didn’t try to prevent it.” (quote from article.)

This position seems rather tenuous to me. Without further information concerning the test, there is no reason to suspect that the other students would have received lower grades if they were required to type in their answers also. Cheating occurs when one or more individuals gain an unfair advantage. From the information presented, it is not obvious that these students have gained an unfair advantage in being awarded a higher grade.

(The company that produced the software used by the Dayton School of Law is ExamSoft: http://www.examsoft.com/)

The lawsuit does raise an interesting question. What conditions need to be present to hold an organization which administers tests accountable for preventing cheating?

On the other hand, last March several high school students filed suit against TurnItIn.com for adding their term papers to a massive anti-plagiarism database. Ostensibly, the students claimed copyright infringement. The link to the article is below.

http://www.washingtonpost.com/wp-dyn/content/article/2007/03/28/AR2007032802038.html

From the article we have the following quote: “Kevin Wade, that plaintiff’s father, said he thinks schools should focus on teaching students cheating is wrong. ‘You can’t take a person’s work and run it through a computer and make an honest person out of them,’ Wade said. ‘My son’s major objection is that he does not cheat, and this assumes he does. This case is not about money, and we don’t expect to get that.’”

Admittedly, this is old news. I couldn’t determine the current status of the law suit. When I first saw this story, I thought to myself that the use of metal detectors at airports doesn’t presume everyone is a terrorist and, similarly, that scanning term papers does not presume that all papers are the result of plagiarism. I was left wondering the question, “What rational motive do students have for preventing a term paper (which could be republished and redistributed despite having been added to TurnItIn’s database) from being detected as a potential source of plagiarism?” I thought that perhaps the students were planning careers as ghostwriters and wanted to amass their collection of papers, but the students say that they don’t cheat, so it couldn’t be that. (http://en.wikipedia.org/wiki/Essay_mill)

But, I find it interesting that one student sues to prevent cheating and others sue to allow cheating.

Other references on the TurnItIn lawsuit:

http://volokh.com/posts/1175270161.shtml

https://turnitin.com/static/pdf/us_Legal_Document.pdf

I am often asked, “What is cheating?” And sometimes I ask others the same question. My succinct reply is usually: “Cheating is when someone gains an unfair advantage.” And, naturally the next question is, “How do you define unfair?” “Well, when you break the rules,” I reply. THE RULES are often unstated and poorly defined. Sometimes it seems like the rules are made up as we go along. In this short essay, I provide my attempt at listing the rules for taking a test.

Before I list my rules, I’m reminded of the film “Butch Cassidy and the Sundance Kid.” In that film Harvey Logan challenges Butch to a knife fight, waving a 12 inch blade menacingly. Butch says, “No, no, not yet. Not until me and Harvey get the rules straightened out.” Harvey bellows out, “Rules? In a knife fight? No rules.” And Butch immediately kicks Harvey in the groin. (For a script excerpt see http://www.imdb.com/title/tt0064115/quotes.)

Harvey forgot the rule, “There are no rules.” He thought that Butch would use a knife, because it was a “knife fight.” When there are no rules, there is no cheating.

As I pondered the question of rules that might define fair test taking, I came up with the following list.

1. Be honest: Test taker statements are to be true and accurate. Two ways in which some individuals attempt to gain an unfair advantage are lying in order to obtain an accommodation, such as being granted extra time to take the test, and employing an imposter who takes the test in their behalf.
2. Communicate clearly: Test takers are to be duly informed of test taking rules. Communication of the rules should be done carefully so as to not intimidate test takers, nor create an adversarial test-taking environment. All exam policies should be clearly stated, including the ability to retest and prohibition from sharing the test content with others.
3. Be fair: The test-taking environment is to be the same, as much as it is possible, for all test takers. An exception to this rule is made for those who require an accommodation, such as reading or answer-responding assistance. All test takers have access to a uniform set of test material. Electronic devices with memories that allow for the smuggling of information useful in taking the test are specifically prohibited.
4. Behave appropriately: The testing event is to remain free from disruptions and distractions. This allows test takers the opportunity to perform their best in a fair and equitable manner. Test takers who create distractions may be subject to discipline. Except for medical reasons, test takers may not bring food, liquids or other items into the test administration area.
5. Don’t steal: No individual shall remove test materials from the testing session or make a copy of the test, without authorization. This includes any copy of the test content that could be made and taken from the test administration area.
6. Do your own work: There is to be no communication between the test taker and any other individual during the test, except for questions regarding the test administration, but not the test content, directed at the test administration officials. This includes copying, receiving, and sharing answers between test takers. An exception to this rule is made if group collaboration is permitted.
7. Respect others: The physical needs of test takers must be accommodated during the testing session. These include restroom breaks and other breaks that may be scheduled. A reasonable schedule should be established and posted so that all may adjust their behavior appropriately. Test takers who are unable to abide by the testing session schedule should be given an opportunity to request an accommodation, or exception.
8. Respect property: All test taking materials are to be returned, intact and without excessive wear or abuse, immediately following the test administration. Test administrators are responsible to verify that all test taking materials have been returned.
9. Don’t lie or fabricate: No act is allowed by a test taker, or other individual, which results in the unauthorized change of an individual’s test result. Such acts include changing answer sheets, disclosing answers during the test, providing answers or hints to a test taker, or allowing a test taker to break any of the rules.

Well, my list only contains nine rules so I don’t believe that we could call it the “Ten Commandments of Testing.”