Transitioning Toward Holistic Assessment Systems

Many changes are being considered for schools in 2020-2021 and beyond in the era of COVID-19 and disruptions to schooling. Some of these include later school-year starts, blended instructional models for all students, full-on virtual delivery of content, and an extended school year to account for lost instructional time. Add to this the very real social and emotional needs of both students and adults and we need to leverage holistic assessment systems when considering school restart.

In the face of challenges associated specifically with lost instructional time, two arguments appear to be bubbling to the surface:

1. We have a blueprint for addressing learning gaps and learning loss with students who enter a with certain foundational or content losses.
2. We have not done a good job of systematically helping students mitigate learning gaps or learning losses.

Persistent learning gaps on virtually all measures of student academic achievement provide strong evidence that the second argument is true. Therefore, regardless of whether the first argument is true, it is clear that we need more information to determine why a blueprint for addressing learning gaps and learning loss has not been effectively implemented with wide-scale success. I am convinced that more complete information is needed to understand what barriers are in place that support the second and impede the first.

Assessment Systems

Before considering what a more complete assessment system looks like, it’s important to first understand what a traditional assessment system is. An assessment system can be conceptualized as a system that consists of the following properties (NRC, 2001; Chattergoon & Marion, 2016):

• Comprehensiveness: A range of measurement approaches should be used to provide a variety of evidence to support educational decision-making (NRC, 2001).
• Coherence: The system comprises compatible assessments that conform to models of student learning (NRC, 2001); which are both vertically coherent (consider the coherence across classroom assessment, common assessments, district assessments, and the state assessments) and horizontally coherent (alignment among curriculum, instruction, and assessment along a common set of learning goals).
• Continuity: Measure student progress over time (NRC, 2001).
• Efficiency: Strive to get the most out of assessment resources and eliminate redundant, unused, and untimely assessments (Chattergoon & Marion, 2016).
• Utility: Utility refers to the degree to which the system provides the information necessary to support its intended goals, which follows from a well-articulated theory of action that specifies the intended outcomes and the processes and mechanisms by which these outcomes will be realized (Chattergoon & Marion, 2016).

The term assessment system is used almost exclusively to refer to the assessment of academic content in K-12. Furthermore, there are subsystems (e.g., formative assessment practices, formal classroom assessments, district-wide assessments, and large-scale summative assessments) within the larger system that varies in characteristics based on at least the following:

• Design.
• Intended use.
• Role in the instructional process.
• Types of information they provide.
• The primary audience served.

States, districts, and schools are rightfully focused on assessment systems that zero in on teaching and learning. My colleagues Carla Evans and Jeri Thompson (2020) offer a series of concrete resources that focus on classroom assessment—the most actionable and impactful aspect of an assessment system. Carla describes the topics educators should consider here:

• Using pre-assessment to identify student knowledge and skill gaps.
• The basics of differentiated instruction.
• Formative assessment processes.
• Accelerated learning approaches and methods.
• Summative classroom assessment and re-learning.
• Systems of assessment.

Furthermore, my friend Susan Lyons argues that educators should focus not on “external instruments that provide little insight into student understanding…” but instead on assessment practices that have the highest chance to improve student learning (i.e., formative assessment practices).

Resetting the Focus on the Hierarchy of Needs

I absolutely believe that assessment systems, when well developed, can help create system-level efficiencies to accelerate student learning with a focus on identified learning gaps. However, as I’ve argued before, it is insufficient to think about assessment systems as only focused on content. This expands upon the notion that there are many prerequisites to which we must attend before we can target student outcomes, which I depict in the following adapted hierarchy (note, student outcomes should be more than just academic outcomes—they include social and emotional outcomes as well)

I raise this idea because our assessment systems need to include more than just information on achievement of academic outcomes so that we can more fully understand why those outcomes are or are not being achieved. The Every Student Succeeds Act sought to expand the notion of school quality by including measures beyond academic performance. While noble, constraints such as comparable, systematic, and defensible information mean that we’re limited to proxy data that approximates concepts like educator capacity, instructional behaviors, and learning processes. This is also true with large-scale assessment data. While large-scale data are defensible, focused on accessibility, and informative at the system level, they are outcome-focused, where our impact occurs at the process level.

To an extent, large-scale assessment provides a coarse quantitative indicator of equitable access to educational opportunities vis-à-vis an examination of student performance against grade-level standards. With the loss of large-scale assessment data in Spring ’20 and the increasing threats to interpreting performance on Spring ’21 assessment, we need to leverage finer-grained, more comprehensive information gleaned through holistic assessment systems.

Our focus should be on not only content-focused assessment, but also on information that speaks to safety, well-being, and the support being provided to both students and educators. This brings me back to the hierarchy presented above. A focus only on content-based assessments will inhibit us from understanding whether the layers of the hierarchy that lead up to the student outcomes are sufficiently addressed. If those are first understood, we can determine whether there is access to and engagement with content. Only then can we focus on grade-level content, with an emphasis on classroom practice. These types of assessments that supplement a traditional assessment system can be informal and less reliable. They can help contextualize learning gaps and gains once school reopens (in-person, virtually, or both). For example, items could focus on constructs like

• The ability to focus on school during COVID disruptions.
• The level of access to content during COVID disruptions.
• The availability of devices and broadband at home.
• The availability and completion of teacher-led work.
• The availability and completion of student-led work.
• The impact of reduced social interaction and opportunities for collaboration.
• The role of device-mediated (and required) communication.

We need to include, as much as possible (and in ways that protect student and family privacy), these types of contextual data to understand the well-being of students and educators before we even get to teaching and learning.

Once there, we must then elevate the value of educator judgment using curriculum embedded and curriculum aligned assessment practices. This can help remove the perceived reliance on a silver bullet external assessment that likely won’t provide value to student learning needs. That is not to say that formalized assessments do not have value. Rather, formalized assessments should be used (when appropriate) to corroborate instructional decisions and interpretations in a systematic and aggregated way. These more formalized and coarser-grained assessments can provide school- and district-level insights for higher-level decision-making. Similarly, and near the end of the year, the large-scale assessment can help us understand how students are progressing over time on the full set of grade-level standards. After addressing the nearest terms needs of students and teachers, we need to turn our focus on how to interpret information from holistic assessment systems to help us make decisions about the next instructional steps strategically.

A friend reminded me about my penchant for writing about program evaluation, and holistic assessment systems are perfect fodder for the topic. I plan on expanding the idea of how to think about multiple information points along each layer of the hierarchy as part of a larger program evaluation effort in my next post.