Accountability as a Roadblock to Assessment Reform
Advocating for Changes to Federal Accountability Requirements to Enable Innovative Assessment
Many of us at the Center for Assessment have led assessment reform for decades, including such noteworthy programs as the Kentucky writing and math portfolios, Wyoming’s Body of Evidence System, and Rhode Island’s Proficiency-Based Graduation Requirements. Most recently, we have supported several state leaders in gaining approval for the Innovative Assessment Demonstration Authority (IADA) under the Every Student Succeeds Act (ESSA) to design assessment systems that elevate teaching and learning. I have written previously about the challenges of trying to be innovative under the IADA, but many IADA assessment rules—particularly comparability—are required because these scores must be used in each state’s ESSA-approved school accountability system. The accountability roadblock, however, is not limited to IADA.
Several states have expressed interest in a through-year assessment design—where the end-of-year summative assessment is split into multiple parts using a variety of design options—to reduce the emphasis on the end-of-year test and to provide instructional information throughout the year. Such a design is permitted under ESSA without pursuing an IADA, but is still subject to rigorous accountability-related requirements. I am not endorsing through-year assessment designs, but that’s for another blog post.
Further along the assessment reform continuum, several states are attempting to implement balanced, or at least re-balanced, assessment systems that move away from the singular focus on the end-of-year accountability test to support instructional in addition to accountability uses. We have outlined the challenges of implementing balanced assessment systems previously, and both Brian Gong and I have written about design features necessary for assessments to support instruction.
IADA, through-course, and balanced assessment systems all face considerable challenges in fulfilling aspirations of producing instructionally-useful information while meeting the requirements necessary for accountability purposes.
The Accountability Roadblock to Assessment Reform
Early advocates for balanced assessment systems, expressed in the seminal publication Knowing What Students Know(KWSK), envisioned various assessments at different levels of the system coherently linked yet serving different purposes. However, KWSK was written prior to the enactment of the No Child Left Behind Act (NCLB). The intense focus on federally-mandated high-stakes accountability determinations changed the nature of district and classroom assessment systems.
Two decades of NCLB and ESSA highlighted how accountability policy ripples through the assessment system. School and district leaders directed considerable attention toward test preparation and related activities designed to improve summative test scores. Assessment initiatives designed to improve instruction and student learning were squeezed out in an effort to raise summative test scores.
Understanding this history is the first step to avoid repeating it, but assessment innovators still find themselves between a rock and a hard place. They can choose to count the assessments designed for instructional purposes in accountability determinations, or they can firewall accountability from instructional components.
Two Bad Choices for Innovation in Assessment
Counting as part of accountability.
In a balanced system, results from local assessment may support monitoring or accountability functions, while state summative results can be used to support district decision-making. But there is a risk. Donald Campbell pointed out almost 50 years ago that increasing the consequences associated with a social indicator escalates the likelihood of corrupting the indicator’s original purposes. In other words, using assessments intended for instructional purposes as part of accountability runs the risk of limiting the instructional utility of these assessments.
Not counting in accountability.
Separating instructional and accountability assessments in the system allows educators the space to build their assessment knowledge and skills. This separation protects the instructional component from accountability influences, but it means the accountability determinations are based solely on the end-of-year assessment. Some states are starting to use shortened summative assessments in an attempt to shrink the “footprint” of the end-of-year test. Depending on how this separation is done, it may reduce the emphasis on the state test, but it places all of the accountability weight on a short, largely selected-response test.
There is a long history of negative effects of accountability tests on classroom instruction because, in part, teachers create activities to mimic the types of test items on the summative test, and district leaders overemphasize interim tests as part of test preparation. Therefore, firewalling the accountability test, while well-intentioned, could work at cross purposes to the innovative reform.
Accountability Flexibility Must Accompany Assessment Reform
How do we get out of this conundrum? At a minimum, the IADA must be expanded to include accountability flexibility for participating states and districts. States should determine an appropriate approach for holding participating districts and schools accountable for performance while participating in the IADA, such as evaluating the effectiveness of professional development programs, changes in teacher practices, and, of course, improving student learning defined in ways coherent with the aims of the reform.
However, we cannot achieve our longer-term goal of instantiating balanced systems of assessment without major changes to federal requirements for state accountability systems. My colleagues and I offered several suggestions for expanding accountability options under a revised Elementary and Secondary Education Act, including providing considerably more flexibility in how states hold schools accountable and expanding the focus of accountability to districts and states.
Given our tapestry of state contexts, it does not make sense for every state to have to follow the same accountability formula. No question, the federal government should hold states to account for providing all students with sufficient opportunities to learn. After all, the purpose of Title I is “…to provide all children significant opportunity to receive a fair, equitable, and high-quality education…” But states can be held accountable and can hold their schools accountable without the federal government dictating the specific means of accountability. Further, states can design reciprocal accountable systems that support the development and implementation of balanced assessment systems.
Fostering balanced systems of assessment requires moving away from accountability determinations based largely on end-of-year tests. I am not advocating a laissez-faire approach to school accountability. Conversely, accountability to support balanced assessment systems will be more comprehensive than current systems and likely more rigorous as well. This approach to accountability could involve requiring a state to submit a detailed theory of action describing the design, implementation, and evaluation of the balanced assessment systems, including things like appropriate professional learning opportunities and calibrating the quality and scoring of local assessments. The theory of action should also include a continuous improvement plan along with a set of indicators for which the school or district will be accountable. The state should describe how it will measure the various indicators along with defining the criteria for success, such that the indicators transparently communicate how the balanced assessment system is supporting the intended learning and assessment reforms.
I offer this one example of a reformed accountability system in an attempt to illustrate a key point. Accountability must shift from a top-down approach that relies heavily on a single end-of-year test to one that recognizes and supports the more holistic nature of learning and assessment reform.
Further, my vision of accountability recognizes that real and sustained change only occurs when actors in the system take ownership of the need to change, as well as the methods necessary to bring about that change.