2018 marks the 15th anniversary since Caveon Test Security first opened its (virtual) doors. Caveon has come a long way since David Foster and the small band of colleagues first recognized and decided to address the industry-wide need for test security. What started as a small team of enterprising pioneers has grown into a one-stop shop for test security solutions and thought leadership. In this article, Caveon founding member John Fremer, Ph.D., speaks to his experience at the forefront of this burgeoning field, reminisces on the past decade and a half of progress, and ruminates on what the future might hold.
Here at Caveon, there developed very quickly a "family" atmosphere and the sense that we were going to face all challenges together. I have worked in good places before, but the quality of the personal interactions was beyond what I have ever experienced.
We underestimated how difficult it would be to establish a place for ourselves in the testing industry. In the beginning, few companies had a test security budget or a person whose job it was to make decisions about test security. We had to draw on our own resources for quite a while.
The Caveon Family in 2003.
I have served in different roles over Caveon's lifespan. The most difficult was being Caveon's President and COO at a time when Caveon and the testing industry were facing a significant downturn. We had evidence that clients really liked our products and services, so I was confident that we would succeed, and part of my responsibilities turned out to be serving as a "cheerleader" and counselor. Many of us played multiple roles to keep expenses down. At one point, I had three jobs within the company, including being the President and the top sales professional. Even when I feel very busy at times now, it does not match the frenetic pace of that period.
To celebrate my winning of the ATP Career Award, my colleagues threw a party for me at the ATP Conference and a wonderful singing group performed. The lead singer was Caveon's Jamie Mulkey and the lead guitarist was Caveon's Steve Addicott. It was as good as it gets, and I treasure that memory.
The Caveon Family in 2003.
At Caveon, we try to treat each client as we would like to be treated. That means listening to what they want and trying our best to provide it in a friendly, productive, and professional way. We care about the impression we have on them, and we try to be helpful and responsive with everyone that we deal with. It is also the way we aim to treat each other within the company. All staff facilitate our weekly company meeting; any staff member has access to our CEO, Dave Foster; colleagues routinely help across programs and projects to meet client needs. Every person is important and merits our respect and regard. It is how we treat both our colleagues and our clients.
All the hard work and expense that goes into developing and administering tests can be undermined by breakdowns in test security. Suddenly, at what is often a truly terrible time, you have to deal with high-pressure demands on money, people, and the reputation of your testing program.
My dad was a New York City firefighter. I learned from him that the time to practice fire prevention should begin when you are designing and building structures. After that, you need a maintenance program carried out on a regular schedule and under conditions that help you make productive decisions. It is the same with test security: you need to plan and monitor at a measured pace, and you need to get help to make sure that you have a thorough and productive set of test security policies, procedures, and materials in place. You must be nimble and able to adapt as threats and risks keep evolving. The plan that was adequate even a year ago may need to evolve in the face of new test security threats.
In the musical The Music Man, the line "You've got to know the territory" is presented in a comical way. It's quite true though. Unless I have learned how a testing program functions and how the staff view their roles and constraints, I cannot do a good job trying to find a way that Caveon can help. It is not enough that we have acquired important clients who value our help. That is often a "door opener" for us, but it is not enough. A new potential client has to see where Caveon fits in their setting. Only if our values and orientation match the client’s perspective do we get a chance to help.
The world has changed in many critical ways that very much impact test security. The pace of change is one of the most important. Many of the test security threats that our clients face are closely linked to developments in technology. This is true both of the way that tests are delivered and of the way that test takers and organized groups try to undermine our policies and procedures. For example, the miniaturization of recording devices, including the development of contact lenses that embody video recorders, has greatly increased our vulnerability to item theft and communication of item answers within the test setting.
Additionally, more and more testing programs in the United States are expanding globally. There are many testing programs already being administered across the world and others are exploring the possibility of expanding their scope to other countries. In the United States, we have a significant and growing level of testing misbehavior, but most test takers accept the idea that cheating on tests is wrong. However, you cannot automatically count on that orientation among test takers, test proctors, or even the legal system as you learn about conditions in each country that you consider for testing. Therefore, it is essential to revisit your assumptions about critical issues such as the life span of a test, asking yourself how long you will be able to use it before the results become undependable. In some settings, you start to lose test items the very first time they are downloaded or otherwise delivered to a testing site.
Finally, the importance of testing has continued to grow as its value in collecting actionable information becomes increasingly recognized. If you are making important decisions about people and programs, test-based information can be very helpful. However, it is only helpful if you can trust the fairness and accuracy of the results. You must have first-class test security policies and procedures for this to be effective—think about these four test security enhancements.
We have established ourselves as trusted partners in the design and management of particular testing programs in many fields, and through our work, we have been able to demonstrate substantial reductions in testing misbehavior. This is an impact that we are very proud of and try our best to replicate with every new client.
In addition, we have developed test security standards that are closely followed by many testing entities and even by national agencies such as the American National Standards Institute (ANSI). Developing formal standards is part of the maturation process of a field, and it has been wonderful to watch the test security standards we have drafted, piloted, and shared take hold.
We have also been very active in professional organizations, such as the Association of Test Publishers (ATP), the National Council on Measurement in Education (NCME), the National Conference on Student Assessment (NCSA), and the Institute for Credentialing Excellence (ICE). In the case of the one conference devoted exclusively to test security, the Conference on Test Security (COTS), we have had the privilege of being a co-sponsor since its start several years ago.
In my previous comments, I have touched on a number of lessons that I have learned. Perhaps I will just note that you should frequently be looking for help from others.
I have found that not only within my colleagues at Caveon, but across our industry and profession, I am more effective when I reach out and ask for the advice and help of others. Have they ever dealt with the kind of issue that one of my clients is facing? What did they do? Not only do I often get good ideas that I can credit and share, but it can also be valuable to learn about approaches that did not work and why that appeared to be the case.
I tell my grandchildren (when they seem disposed to listen) that you can't avoid making mistakes in life, but you don’t have to make all the mistakes yourself. Instead, you should try to learn from the experience of others whenever you can. This is the case in a lot of settings, including in test security.
We have wonderful clients, first-class services, and a highly conscientious and professional staff. Because of this, I am very positive about our prospects as a company. We need to remain faithful to our orientation of treating clients and prospects the way that we would personally like to be treated. We need to keep up with developments in our field and profession. We need to learn from others and be part of the solution to problems wherever we can. I don't know if Yogi Berra ever said this, but I think it is in his spirit:
"It is hard to predict the future, because no one knows what is going to happen. However, if we can stay faithful to our core values and do our best to help our clients protect their items, tests, and reputations, we will be on the right path to continue to be a good company to do business with and a great place to work."
The Caveon Family in 2018.
Topics from this blog: Test Security Stories
