Risk Management: The Keys to Secure Data Management
Originally Published in Certification Magazine, 12/2004
Even after the test items are safely locked away, your program’s exam results and its associated credential can still be compromised by data mismanagement issues. Secure data management means that the integrity of the program’s test results is protected as well as the examinee’s privacy. Keep in mind that results data are used to make credentialing and screening decisions as well as program decisions. A range of policies is needed to prescribe how this type of information is collected, stored and transmitted.
The costs associated with data mismanagement can be high. For example, a loss of data integrity can expose your program to charges of fraud, incompetence and, in some states, unlawful disclosure of candidate information.
Consider your program’s vulnerability in the scenario described below:
Case Study: An examinee was told that that her test result has been invalidated due to an incident that occurred at a test site. The report stated that the program had strong evidence that she copied from her neighbor’s test paper and that this constitutes professional misconduct. She was also told that the case is closed and they informed her that they don’t have the data anymore.
Based on this case study there are many potential data mismanagement concerns such as:
- Did the program have a policy that states that a score will be invalidated if the test administrator reports that cheating may have occurred?
- What is the program’s policy regarding what type of data associated with the test event will be retained?
- If the program retains additional test event data, what is their policy regarding length of time to retain this data?
To strengthen the ability of your organization to defend itself in situations like the one described above, your program needs to create data management policies and communicate them to their examinees. Below I have provided a sampling of some of the policies you may want to include (Note: The first two policies did not exist in the case study described earlier):
- Test taker agreements should include a statute of limitations; how long test results are retained, and how long an examinee has to make a challenge.
- Program policy requires that any other information collected during or incident to test administration is stored with the candidate’s test result and will be retained for a specified period of time. Examples include demographic information, survey results, and test administration reports.
- Scores are not reported until it is confirmed that the test taker met eligibility and identification requirements.
- If integrity of a test result or group of test results cannot be confirmed, test scores and decisions based on them are withheld until the score accuracy is confirmed.
- Information regarding test results and examinees is shared in a manner consistent with policies and procedures that protect the integrity of the information and the rights of the examinees.
Creating and documenting data management procedures to implement policies are also critical to the security of your data. Examples of procedures for your consideration include:
- Test scores and test score handling (before, during and after test administration) should adhere to prescribed program policy to ensure their integrity. Examples of how to do this include log and incident reports filed by the test administrator.
- Procedures exist for secure transport and handling of test documents such as answer sheets, test booklets and examinee identifications.
- Handling procedures for non-electronic test results such as scan sheets are created to reduce potential fraud, such as erasures and modification of answers after the fact.
- Encryption and secure transmission are used with all electronic information.
- Access to test results is controlled and monitored. Procedures are in place for detecting unauthorized access and tampering with the data.
- The program’s testing procedures are audited regularly to detect security breaches and their effects on the exams, scores, and program decisions made. This audit follows the test administration window for those tests with set testing administration windows.
- All data, including items, tests, test results, and examinee information; are archived in a manner consistent with program business and security policies.
- Handling and implementation procedures are created for disposal of test results. Procedures need to ensure protection of privacy and data.
For those of you awarding credentials, you should be aware that credential verification is also a form of data management. This shows that the credential exists, that it’s not just made up. Requirements for credential verification include identity verification, proper documentation for information release and secure transmission of authentication information. You need to develop a process such as phone call, written request, website with passwords to communicate only the relevant information such as if the credential exists for the person in question along with the date the credential was awarded. Generally you do not report a score.
As I stated at the opening of the article, secure data management policies and procedures are critical to maintaining the integrity of your program. Data mismanagement puts your whole investment at risk.