Test Security: Expanding Our Ideas to Support Score Integrity

From Defensibility to Equity in Educational Assessment

In traditional test security, the focus has been on preventing, detecting, and acting on irregularities to maintain the integrity and comparability of scores. But as the field reckons with the historical implications of measurement—rooted in eugenics and marginalization—we must ask ourselves: What are we reinforcing, and what are we dismantling? 

I had this question on my mind when I attended a recent conference that redoubled my commitment to challenging conventional ideas about test security: It must expand to consider equity. 

These issues are also at the heart of my work to support the NCME Educational Measurement and Civil Rights Task Force. That work centers on ensuring that measurement promotes, not suppresses, civil rights. This shift calls for a critical look at how we design, develop, and evaluate assessments to uphold fairness, particularly for marginalized groups.

All of these ideas were circulating in my head as I headed to the 13th annual Conference on Test Security (COTS) in Salt Lake City. There, I participated in a plenary discussion with colleagues. My role was to challenge traditional concepts of test security and address how they intersect with equity and fairness. In this post, I’ll explore themes that emerged as we discussed evolving our current conceptions of test security.

A Broader Vision for Test Security

I propose that we reconceptualize test security as a necessary but insufficient part of a broader framework of test integrity. Test security must expand as we consider potential innovations of summative testing in K-12. 

For example, how do we validate individualized experiences? How do we develop security protocols that protect individual-level data while maximizing trust in personalized results? The best assessments require students to transfer knowledge to novel contexts, making them less vulnerable to manipulation.

How do we ensure that fairness represents the context and history students bring with them? Test security should consider whether assessments accurately capture a student’s context, socio-cultural background, and educational opportunities. The process and protocols must be defensible if the testing experience authentically reflects these factors.

How do we safeguard against misuse? Extending test security means protecting scores and ensuring that results are interpreted and used correctly. This involves addressing the use of test scores—how educators and policymakers use them and their impact on students and schools.

Future-Focused Considerations for Supporting Score Integrity 

Looking ahead, this broader vision should inspire us to reexamine ways to approach test security in view of promising new developments to create more fair and equitable opportunities for students. As testing models evolve to include personalized and adaptive assessments, test security must move beyond focusing on standardized conditions and support authentic, individualized experiences. 

This evolution means protecting the validity of scores not just as a measure of knowledge but also as a reflection of student opportunities—or the lack thereof. And that means focusing on protecting the validity and fairness of scores while using test security methods to support equity. 

Below, I offer a few reflections on things we need to consider as we look toward supporting score integrity in the future, based on conversations I had at the conference: 

• Consider the validity of scores in remote settings: As more assessments shift to unsupervised or remote formats, it’s essential to develop security protocols that protect against threats while ensuring student performance is still accurately measured and reflective of true learning outcomes.
• Explore reliability across diverse settings: The emphasis on reliability must evolve to accommodate diverse testing environments and individualized experiences. Future efforts may focus on ensuring consistency within a student’s own performance over time, rather than strictly comparing results across different students.
• Securing adaptive environments: As adaptive testing continues to gain traction, security protocols must be robust enough to support these flexible environments while protecting the validity and fairness of the results. We need to ensure adaptive features enhance the credibility of assessments rather than compromise them.

Test security should not only defend the technical integrity of assessments, but also serve as a critical component in promoting fairness and equity in our decisions. This requires considering more than just the test administration event; we must also consider the conditions supporting the test. By reinforcing the secure and responsible use of assessment data, we can transform test security into a tool that supports closing opportunity gaps and fostering more equitable educational outcomes. 

We can use test security focused on score integrity to more effectively:

• Identify disparities: Valid scores from secure assessments provide a better— if incomplete —view of inequities, allowing educators and policymakers to identify opportunity gaps and allocate resources to improve student access and support. What’s key here is that even if measures are imperfect, we can still use them as signals for where opportunity improvements should be made. 
• Promote accountability and shared responsibility: Ensuring that assessments are reliable and secure upholds a consistent standard for all students that can be used to evaluate how well schools are supporting them. It enables systems to be held accountable for their role in addressing educational inequities and ensuring that all students receive fair opportunities to succeed. However, as Scott Marion writes about shared responsibility, states and districts should be expected to provide schools with the resources and autonomy to act on assessment results. 
• Drive equity-focused interventions: When assessments are designed with security measures that protect against bias and uphold integrity, they can be used to drive interventions that dismantle systemic barriers. However, this also requires critical attention to the biases within assessment designs and the standards they measure against, ensuring that equity is central in every process step. We cannot assume that assessment scores are perfect; we must keep pushing to improve measurement practices and expand what we can claim about student knowledge, experience, and context.

Test Security and the Future of Educational Equity

Ultimately, test security is about defending the validity and fairness of assessments, and that requires us to continue pushing forward on whether we are evaluating assessments comprehensively to represent what students know and can do fairly. 

I suggest that we keep the following three key issues at the forefront of our design, implementation, and evaluation of assessments:

• The intended uses of assessments: Designers, users, and partners must understand the appropriate applications and limits of assessments. Allowing unintended uses disadvantages the system—and its students—from the start. 
• The limitations of available assessments: Transparent communication about assessments’ inherent limitations is necessary to prevent misuse. This should be coupled with the first point. 
• The consequences of assessments: Even when used correctly, assessments can reify systemic barriers. Understanding these impacts is crucial for designing better, fairer systems. For more details, see our recent RILS presentations, which explore the consequences associated with assessments and assessment systems. 

Test security is not just about the integrity of scores. It should be about the integrity of the entire decision-making process surrounding those scores. By evolving our practices, we can create more equitable, defensible assessment systems that support students’ opportunities for success.