Supporting A Principled Approach to Restarting School Accountability in Spring 2022

Operational Best Practices for Accountability

Recently, my colleague Chris Domaleski reminded us of the complexity associated with restarting school accountability in Spring 2022.  It is unlikely that this problem will have a simple or straightforward solution. He presented four principles states should consider as they chart their own contextually appropriate courses moving forward: 

1. Minimize the long-term influence of ‘fragile’ indicators,
2. Prioritize utility,
3. Don’t overload your ESSA accountability system, and 
4. When in doubt, support.

I couldn’t agree more. I highly value guiding principles for most problems, initiatives, and even personal projects. They provide key guardrails that can help provide early indications for when things, well, go off the rails. In this post, I attempt to supplement Chris’s advice by revisiting a resource that can serve as a reference to help states operationalize these principles. 

I have written before about the idea of moving toward specificity after establishing a broad-brush stroke direction for a complex project (see my post here about logic models extending theories of action). As part of a project for the Chief Council of State School Officers in support of the Accountability Systems and Reporting (ASR) state collaborative, I developed Operational Best Practices for Accountability. 

The OBPA is intended to provide guidance and criteria for the design, development, implementation, evaluation, and potential revision of accountability systems with which states may interact or operate. These criteria describe practices that are applicable across context-dependent systems while providing concrete benefits to users. The best practices are intended to: 

• Establish a common set of operational criteria that correspond with guidance reflected in other areas of measurement in education (e.g., large-scale assessment, evaluation, psychological testing, etc.).
• Reflect ‘lessons learned’ from state experiences with No Child Left Behind and local accountability models.
• Reflect expectations that improve accountability system design, development, and implementation in non-partisan and unbiased language consistent with other professional standards.
• Represent the breadth of the considerations and decisions associated with accountability system design, development, and implementation.
• Provide multiple examples or methods of how to achieve a goal, objective, or criterion. 
• Articulate a range of best practices and criteria that can be approached by states with varying levels of capacity and expertise. 

The OPBA is intended to be a choose-your-own-adventure style reference document, with best practices in accountability distributed across 9 chapters. This organization acknowledges the reality that there is no ‘one-size-fits-all’ way to approach the work. Readers may elicit guidance from selected sections in order to meet their specific needs. While I’m sure one could read it cover-to-cover, that approach might be best suited for a “from scratch” development of an accountability system (or light bedtime reading). 

The 9 best practices discussed in the chapters of the OPBA are 

1. Establishing an Accountability System’s Theory of Action
2. Identifying Stakeholders and their Involvement
3. Accountability Roles, Responsibilities, and Program Management
4. Selecting and Integrating Measures for Accountability Systems
5. Establishing Performance Standards for Accountability Systems
6. Articulating Operations and Quality Control in Accountability Systems 
7. Reporting and Communicating Accountability Results 
8. Monitoring and Evaluating Accountability Implementation
9. Engaging in Accountability System Change Management

Within each chapter, readers are provided introductory text that clearly describes the importance and role of the information provided and how it is intended to be used. The chapter then presents detailed recommendations and considerations for accountability designers in the form of best practice statements (e.g., 1.1, 1.2., etc.) and supporting actions (bullets). Where applicable, each chapter includes or references illustrative examples or scenarios. 

Applying Operational Best Practices to Restarting Accountability in Spring 2022 

The restart of accountability following COVID-19 disruptions marks a particularly specific context where states will need to consider potential revisions to their systems that will require thoughtful implementation and evaluation. While there may be best practices that are worthwhile to review throughout all chapters in the OBPA, states may be particularly interested in reviewing specific chapters depending on plans for restart. 

More broadly, states will likely fall into one of three categories when restarting their accountability systems (see D’Brot, Landl, Domaleski, and Brandt, 2020):

1. Restarting their legacy system in Spring 2022; 
2. Revising their legacy system to start a new accountability system in Spring 2022; or 
3. Introducing a transitional system in Spring 2022. 

Depending on which path is most appropriate, states may wish to review specific chapters to ensure their decisions and actions are appropriate and defensible. The table below offers some suggested review chapters depending on plans for restart. ​

Conclusion

States must manage multiple requirements as they restart accountability while considering varied stakeholder perspectives. Leveraging a core set of guiding principles, in conjunction with specific operational best practices, can help state leaders avoid potential problems as they restart accountability in spring 2022. I hope that this resource is helpful to states, whether they are revising, restarting, or developing accountability systems from scratch.