Choosing a Growth Model: Far More Than Technical Factors
States Should Start by Examining What They Value
The Center’s latest paper offers guidance for states that are adopting or revising a growth model for their accountability systems. Co-authors Scott Marion, Chris Domaleski and Will Lorié advise state leaders to begin the process by exploring their values. Marion discussed key ideas in the paper with the Center’s editorial director, Catherine Gewertz.
What led the three of you to write this short paper?
Actually, it was my friend Andrew Ho, the president of the National Council for Measurement in Education (NCME), who suggested that we write it. I was presenting the results from a particular growth-model initiative in one state to the state’s technical advisory committee. Andrew is on that TAC and suggested we write it up.
The process we describe in the paper captures something we think is really important: When we work with our clients and their constituents, district and school educators and leaders, policymakers, and others, we want people to engage in rich discussions about important design decisions. But to do so, they need to understand some fundamentals.
So, there’s a period of education, and then we have deep discussions before we move into decisions. We do that whether it’s an assessment system, an accountability system, or a component of an accountability system: in this case, a growth model.
Do states often think that choosing a growth model is all about the technical factors?
Not so much anymore. They’re aware other things are involved, but those things have been operating under the hood without people articulating what they are. We’ve tried to surface those other factors in this paper.
You emphasize in the paper that states need to consider their values when they’re deciding on a growth model. What do you mean?
Here’s a very straightforward example of a value: does the model favor lower-performing or higher-performing schools? Or is it completely neutral?
You could make the case that neutral is good; it should be the same for everyone. Others could say well, we want to make sure that lower-performing schools have a chance to show growth, and we’ll bend over backward to make sure we do that.
There are certain types of growth models that are highly correlated with prior student achievement. By definition, they tend to favor higher-achieving schools. So those schools are getting a double bonus, or the poorer-performing schools are getting whacked twice.
Unless you make that kind of information explicit in the process of discussing growth models, people might not think about it. But when you make it explicit, most people will say, oh, we want a model that’s neutral, one that doesn’t favor higher- or lower-performing schools.
In the paper, we lay out seven different potential considerations that reflect values. They’re not completely independent of one another. In certain states, we’ve used fewer than seven. In other states, we’ve used more than seven. But these seven tend to be the ones that show up most commonly.
Whether to include “background factors” in the growth model is another big values decision. Value-added models include background characteristics, such as whether the student is economically disadvantaged, or a special-ed student. Whether a model includes these kinds of background characteristics is a value, something a state should explicitly surface and discuss when choosing a model.
What are a couple of messages you hope people take away from this paper?
When you engage in these types of design decisions, the worst way to do it is to have a few people in a state office room picking models based on cost or ease of use or other things like that.
The other thing I’d like people to take away, particularly as it relates to growth, is that these values and principles undergird and are somewhat implicit in all growth models. It’s important to have people clarify what they value before they start picking a model.
It helps to keep you from getting into a “shopping trip,” where people start saying, “This is my favorite kind of growth model.” We never talk about types of models right away. We talk about the things we care about in a model, and then we say, let’s look at the models to see how they align with our priorities.
That usually narrows down a list of potential models. Then we and the state folks engage in empirical work where we run the model with state data to see how well it performs relative to the things we value.
We’ve found that clarifying values and priorities provides a useful framework for state leaders to communicate about their growth model and how to use it well.
Image by Allison Shelley/The Verbatim Agency for EDUimages
