Accountability Systems Must Offer More Than School Grades
School Grades Obscure Important Information About School Quality
One of the goals of school accountability systems is to communicate important information about school quality to education stakeholders, especially parents and community members. At the urging of advocates, many state leaders have been using grades for schools, like the ones teachers use for students, for many years in hopes of effectively and transparently communicating school-quality information to the public.
But I believe school grades—like student grades—obscure more meaning than they convey and carry significant unintended negative consequences. School grading and typical approaches to student grading have many of the same problems, particularly the lack of a shared understanding of what particular grades really mean, and the generally arbitrary ways the cutoffs between grades are established.
What is a “B School” Anyhow?
A commonly cited reason for using school grades is that parents and other stakeholders understand them. Everyone went to school and got grades, so we all understand that an “A school” is better than a “B school,” and a “B school” is better than a “C school.” Right? But what does “better” actually mean?
To understand the challenge of defining “better” or, more specifically, the meaning of school quality associated with specific school grades, I did an eye-opening exercise with the members of a state board of education several years ago. I asked them to write and share a short description of an “A school.” Not surprisingly, many members included characteristics such as high overall achievement and student growth, high graduation rates, and strong performance on all other indicators in the system.
But when I asked the board members to describe “B” and “C” schools, there was very little shared understanding. They were divided on whether achievement or growth should carry more weight. Some board members thought gaps in performance among identified groups of students should be included in their descriptions; others didn’t. This exercise demonstrated that the letter grades assigned to schools do not necessarily carry the information accountability system designers think is being carried – at least not to a good proportion of the intended users.
School Grades are Not the Only Way to Meet Federal Accountability Requirements
All school accountability systems designed to meet the requirements of the Every Student Succeeds Act (ESSA) include academic achievement, another academic indicator (almost always student longitudinal growth), English language proficiency, an indicator of school quality and student success, and graduation rate (for high schools).
Most states combine the various indicators to arrive at an overall determination for each school each year. ESSA requires states to weigh achievement and growth more than the other indicators, but states otherwise have some leeway in how they value the other indicators in the system.
School grades are generally based on this overall weighted average of indicator scores. Even though these weightings are communicated in accountability documentation, people focus on the grade and add their own interpretations of what’s important or heavily weighted, as was clear in my state board exercise.
But A-F Grades Work for Students, Right?
School grades are often defined by an arbitrary number or percentage of points, such as 90 percent of possible points being the cutoff for an “A” rating. I say arbitrary because there is no inherent meaning in 90 percent, for example. It’s understanding the “90 percent of what” that helps us make sense of what 90 percent should represent. Unfortunately, many people think, “Well, that’s how we assign grades to students, and that seems to work.”
Except it doesn’t.
There are far too many misconceptions about student grading scales, such as thinking that grades are separated by 10 points—except in schools where it’s 7 points—and somehow accepting a 60-point range for an F grade. These and other hidden problems with school grades have been well-documented by authors such as Tom Guskey and Robert Marzano.
My colleague Carla Evans also wrote a terrific blog series about student grading, convincingly illustrating that even newer approaches to student grading, such as competency-based grading, do not easily erase these deeply ingrained misconceptions.
Can We Add Meaning to School Grades?
The state board I mentioned above realized through our exercise that school grades mean different things to different people. But they still had a legislative mandate to assign school grades. They found a way to produce school grades that are more defensible and hopefully meaningful.
The board used a standard-setting process that’s similar to the way we establish achievement levels on large-scale assessments. My colleagues Chris Domaleski, Juan D’Brot, and Leslie Keng developed this approach and describe it in this paper. In a nutshell, accountability standard-setting engages participants in a deliberative process to match agreed-upon descriptions of school quality with scores on the indicators in an accountability system. Participants review indicator scores from a broad range of schools and individually and collectively determine which school performance profiles best match the agreed-upon descriptions of schools at various levels (grades). This activity leads to more of a shared understanding of school performance associated with various grades.
Beyond School Grades: A Better Way to Support School Improvement
Using a standard-setting approach is a marked improvement over the grading approaches most states use. But I’m still concerned that grades or other overall school designations, such as stars, hide more information than they reveal.
So far, we’ve been discussing how school accountability systems are supposed to convey information to stakeholders. But it is critical that they also support school improvement efforts. To do so, we need finer-grained information without the distraction of overall grades.
In fact, several states do not calculate an overall score for each school, choosing instead to show separate scores for each indicator so stakeholders can easily understand specific aspects of a school’s performance. This is a step in the right direction. It’s similar to the way Consumer Reports provides information about cars.
As it happens, I’m in the market for an electric vehicle. I care about the car’s range, reliability, performance, and interior space. I also care about all-wheel drive because I drive in the snow a lot, so it’s important to me that Consumer Reports rates these qualities separately. If all I had was an overall rating, I wouldn’t be able to figure out which of the car’s qualities factored into the rating and how. And that means I wouldn’t be able to judge whether the editors’ idea of quality aligns with mine.
We need a system like the one used for Consumer Reports that provides information that’s detailed and meaningful enough to help leaders understand what they should focus on to make their schools better. Accountability systems that draw attention to performance on specific indicators, such as student growth in 5th grade mathematics, for instance, tell school leaders a lot more than those that simply give schools an overall grade that blends multiple indicators in ways the public—and educators—often can’t untangle.
Nearly all school-grading accountability systems provide this level of detail in their full accountability reports. But when overall grades are posted, they become the shiny object that draws everyone’s attention. We know this from extensive research on student grades, and from our own conversations with district personnel in many states.
If we want school accountability systems to support school improvement, then leaders and educators need actionable information without being led astray by the limited and vague information school grades provide. For example, when indicators are combined via some sort of weighted average, school leaders could strategically focus on the indicator that might yield the most points in the overall grade rather than focusing on where the need for improvement is the greatest.
Rethinking the Function of Grading Systems
Grades may be intuitively appealing, but they are fraught with many problems that can get in the way of school improvement and stakeholder communication. Dangerously, people think they know what grades mean and will often act on this misunderstanding, which could be worse than no information at all. Further, grades can pervert incentives to focus on meaningful improvement. And the grade levels are generally established using arbitrary and often naïve methods.
Yes, grades offer the hope of simplicity and transparency, but when dealing with organizations as complex as schools, we owe it to students and educators to provide an accurate and useful depiction of school quality.