How to Choose Local Assessments

A Guide for District (and State) Leaders

School district leaders have a new guide to help them choose assessments for their districts. Erika Landl, a senior associate at the Center for Assessment, and Susan Lyons, principal consultant for Lyons Assessment Consulting, co-authored the District Assessment Procurement Protocol (DAPP) in partnership with EdReports. They discussed the new guide with the Center’s editorial director, Catherine Gewertz.

You’ve created a new guide to choosing district assessments. Who is it for and what does it do?

Erika: 

The guide is for district leaders and others involved in the selection of assessments for district use. It’s intended to provide some guidance on how to be thoughtful in decision-making rather than reactive when it comes to selecting a new assessment.

Susan:

Our intention is to arm district people with resources so they can ask their test vendor good questions about what the product is and its technical quality. 

The guide has three steps. The first two are all about internal district processes. We ask district leaders to think about what assessment information they need, who will use that information, and what decisions they’re going to make based on that information. 

We ask district leaders to articulate how those decisions will positively impact teaching and learning. So really, this is an assessment literacy tool that asks district leaders to think through what they need an assessment for, and why, before they engage with vendors about procuring an assessment.

The third step supports district leaders in evaluating the evidence of test quality provided by the vendor. 

What led the two of you to this work? Didn’t it stem from a project with EdReports to evaluate interim assessments?

Susan:

EdReports is in the business of evaluating curricular materials. They knew Erika and I had worked together on an evaluation methodology for summative assessments. So they reached out to us to ask if we’d work with them to evaluate interim assessments. We collaborated to develop a pretty extensive methodology and review criteria for interim assessments. The District Assessment Procurement Protocol is an offshoot of that work.

Erika:

The original work for EdReports was extremely comprehensive and technical; it wasn’t intended for use by a lay audience. We created the DAPP for district leaders. It says, here’s the evidence of assessment quality you want to look for, but even before that, it asks, how do you know what you need, and what are the characteristics of the assessment that you’re looking for?

Wasn’t all of this work part of an EdReports project to evaluate interim assessments? And that project never launched, right? 

Susan: 

Right. That work was officially paused in May because not enough publishers participated.

What are you hoping district leaders will do with this new assessment protocol?

Erika:

Our biggest hope is that they will interrogate their decisions about the need for a commercial interim assessment in light of what they already have and their vision for how students learn.

We hope they really engage in a thoughtful process to understand what they want, why they want it, and what the assessment should look like to support their goals before they go out and look at things that might make them think they need something that they actually don’t need.

The label “interim” can really be problematic and get in the way of making good decisions about assessments and your assessment needs. Because there are so many different types of interim assessments designed for different purposes, just knowing that you’re selecting an interim assessment doesn’t tell you anything. 

So we want to encourage people to not look for the label, but instead focus on what they need, and the specific information these tools provide the uses that they’re validated to support, regardless of what they’re called. 

Does this guide have any applicability to states and how they choose assessments?

Susan:

Yes. Many states require or encourage districts to adopt interim assessments. Often, states will provide an assessment they’ve developed or acquired for that purpose, or they’ll allow districts to select from a menu of options. Sometimes they just leave it completely open to the districts. 

As states are thinking about enacting policies around balanced assessment systems, it’s useful for them to be thinking about the resources they’re equipping districts with so they can make good decisions about assessment procurement. We’re hoping this tool will help with that.