Dealing with Fallout from COVID-19 School Disruptions

Tags:
Apr 16, 2020

What to do Next in Assessment and Accountability?

In late winter-early spring 2020, COVID-19 school disruptions became mainstream across the U.S. States and communities implemented policies for “social distancing,” resulting in students schooling from home, which interrupted state assessment schedules in every state. 

Canceling state assessments and local schooling have implications for several other state policies and programs, ranging from federal school accountability designations to determining how students could earn credits and graduate.

States and their partners need, of course, to deal with the immediate decisions raised by COVID-19 school disruptions for Spring 2020, but they also need to anticipate the implications of these disruptions for assessment and accountability next year and even into 2021-2022.  

We outline below several things states may need to attend to immediately, in the short term, and over the next two school years. 

What’s Been Impacted?

The following is a summary of the major impacts of the COVID-19 school disruptions organized under the categories of:

  • Assessment
  • School Accountability
  • Student Accountability
  • Other State Policies, and
  • Help States May Provide  

In the final section, we provide checklists of things states should consider in these areas in four time periods: Spring/Summer 2020, Fall 2020, Spring 2021, and Spring 2022, but we ground this discussion in an examination of the current impacts.

Assessment

No state completed its planned Spring 2020 state assessment, which means that the state needs to collect and document information on:

  • Whether each state test had all, some, or none of the eligible students assessed (in a previous post, Chris Domaleski outlines various categories of partial testing and their implications). The condition under which some students were tested may be the most interesting and complex. It is interesting because the state potentially has some data to use in the future. It is complex because the state will have to evaluate how complete and credible the data are.
  • States will need to decide what to report regarding the spring 2020 assessment. If states move forward with some reporting, how can they support appropriate interpretation and use of the information (e.g. flag limitations of comparing to prior years)?  

School Accountability

States would have used assessments from Spring 2020 and other data from 2019-2020 to evaluate and inform actions for schools identified for support and intervention from the state and/or districts. All states have applied for a waiver from the U.S. Department of Education to keep school accountability for the 2019-2020 school year the same as from 2018-2019.  

All federal reporting requirements related to state assessment and accountability results were waived in conjunction with the requirements to administer state assessments in 2020. A state, however, may wish to report results for some accountability indicators or measures.

States will need to decide what to report regarding 2020 school accountability results and how to support appropriate interpretation and use.  

  • States will need to determine which of the state measures are complete, partial, totally incomplete, or perhaps different than other years. This determination is relevant to situations that depend on a single measure, but it is critical for things that involve multiple measures, such as the calculation of growth, any measure with a multi-year average, or school accountability. It also affects any measure that may be less comparable to other years—for example, relaxing requirements for seniors may result in 2020 high school graduation rates being different than in other years.
  • A state that had a 2020 state school accountability system that differed from the federal school accountability system will need to decide how to proceed.

Student Accountability

Many states or districts have policies requiring students to do something in order to be eligible for some consequence, such as high school graduation. These policies are distinct from school accountability. They may be related to student performance on a state assessment or some other activity such as a graduation exhibition or completing a community service project.    

Some student accountability policies directly related to the state assessment may involve opportunities that were disrupted, either during the original administration or in retesting: 

  • Pupil progression (e.g., a student may need to pass a state reading test to qualify to advance to the next grade)
  • Course grades or credits (e.g., an end-of-course test may count for a certain percentage of final grade), 
  • Earning an endorsement (e.g., a diploma seal of biliteracy may require passing a foreign language test); 
  • Being considered for a state scholarship (e.g., may require scoring at or above a certain level on a state-recognized test); 
  • Receiving college placement or credit (e.g., may require scoring at or above a certain level on a state-recognized test).

Other State Policies

Other state policies may be dependent on assessment or may affect how assessment results or accountability measures are interpreted in future years. 

  • Have requirements been changed for the 2019-2020 school year from what was required in 2018-2019 and will likely be required in 2020-2021? For example, is the state waiving the requirement regarding the minimum number of minutes of instruction or waiving certain graduation requirements regarding course credits?
    • It is important to document things such as a) what exactly is the policy, b) when was the policy communicated, when did it take effect, and who in the field attended to it, c) how many students used the different requirements; d) which students used the different requirements.
  • What requirements were relaxed or implemented differently in Spring 2020 regarding student support, such as for students with disabilities, English learners, or economically disadvantaged students?  
    • Such differences should be documented so that there are data to interpret or adjust if need be, how the 2019-2020 data are used in the future.
  • What are the impacts of disrupted schooling on people other than K-12 students?  
    • Does the state use student assessment information as part of a state educator evaluation system?
    • State teacher credentialing programs may require that a student-teacher spend a certain amount of time in a supervised classroom – student-teachers may not have been able to meet that requirement in Spring 2020.

We are not questioning any choices made by states for this year; rather we are arguing for deliberate consideration and clear documentation to help interpret performance going forward.

Optional State Policies to Help Partners

States may be asked to help districts, schools, or other partners in ways not directly related to ordinary state responsibilities in assessment, accountability, or other state-regulated actions.  

  • For example, what will the state do if a district asks for an option to administer a state test just to inform the district in the fall? What if a district asks for information about which non-state assessments the district might use in the fall to inform curriculum planning and student placement?  
  • A state may wish to conduct or assist districts and other partners to conduct special studies such as to estimate the statewide, district, school, grade, or class relative “learning loss” engendered by 2020 school disruptions, or differential effects on certain groups of students (“gaps”). Special studies might also be aimed at identifying specific content knowledge/skills that are lower than expected in Fall 2020.  
  • A state may wish to conduct or assist districts and other partners in special program evaluation studies to identify programs and/or entities that are effective at decreasing learning loss and/or gaps during the 2021 and 2022 school years.

These are just some of the possible topics or questions the Spring 2020 COVID-19 school disruptions might stimulate regarding Assessment, School Accountability, Student Accountability, Other State Policies, and Optional State Policies to Help Partners. These areas should be considered not only for their immediate effects but also for longer-term or unintended consequences.

States should also work with their technical advisory committees, other technical assistance partners, and assessment vendors to plan specific analyses and options for the next two years.

What to Do Next in Assessment and Accountability

Spring/Summer 2020

  • Make adjustments to policies that are fair, legally defensible, and intended to have good consequences, especially with consideration to meeting the needs of students who may be underserved or at-risk.
  • Keep up-to-date on USED policy; communicate the USED desired policy either directly or through CCSSO or other channels.
  • Communicate with partners about policy and operational changes, and get detailed information about responses from partners, particularly those who might be impacted by changes in contracts or data (e.g., if the state waives certain graduation requirements, how will colleges respond?).
  • Decide what the state recommends that districts do to assess and counter any “learning loss” from disrupted schooling and a “summer learning loss,” particularly if that loss is differential. Will the state assist?
  • Gather accurate information about every measure used in state assessment and accountability and carefully document policies, practices, and – where possible –  individual students (e.g., not just that “some students were able to complete ELP assessment in 2020”, but which students did complete it and whether that completion took place under comparable circumstances as previous years; or which students did not complete, and what they did do?)
  • Start the planning process about what will be done in 2020-2021 and 2021-2022 in terms of assessment, accountability, and any special studies.
    • Will the state “bank” the state assessment that wasn’t administered in Spring 2020 and use it intact in Spring 2021?
    • How did the failure to test in Spring 2020 affect future assessment plans (e.g., are field-testing and development of test forms on track or do assessment contracts need to be adjusted)?
    • Is the performance of groups of students affected from 2019-2020 in a way that might affect equating in Spring 2021?
    • How will growth be calculated in 2020-2021 and 2021-2022?
    • What if analyses show that certain indicators are less technically sound in Spring 2021?
    • How will identification and exit determinations be made in 2020-2021 for CSI, ATSI, and TSI schools, particularly where multiple years of data were to be considered?  
    • How will the state assist districts/schools to identify and help students/student groups who need more support?
    • States may have been counting on Spring 2020 data for external validation studies, IADA comparison studies, evaluation studies for grants, etc. What can/should be done?
    • If states were considering resetting assessment or accountability standards or other program changes that might involve the breaking of trendlines, consider whether Spring 2021 might be an opportune time or may not be an appropriate time.
    • Are the state values, theory of action, and claims for assessment interpretation and accountability use impacted by the 2020 school disruptions and any lasting effects (e.g., are schools responsible for meeting long-term state goals regardless of external factors)?

Fall 2020

  • Implement policies regarding Fall 2020 students and schools, such as December graduation.
  • Implement any “make-up” data collection where Fall 2020 data may be used in place of Spring 2020 data for 2020-2021 school or student accountability.
  • Implement any data collection that may be needed to evaluate the impact of disruptions on and/or inform adjustments to assessment and/or accountability in 2020-2021 or 2021-2022.
  • Work with technical assistance partners and assessment contractors to perform analyses needed to inform assessment and accountability decisions.
  • Communicate any revised plans for assessment and/or accountability for 2020-2021 or 2021-2022.
  • Provide any planned support to districts and schools to help them assess needs and provide appropriate support, such as helping districts/schools and their partners evaluate the extent of learning loss and learning gaps, and provide information to help educators determine how to bridge these gaps and evaluate the effectiveness of supports.

Spring 2021

  • Implement planned analyses of Spring 2021 data for comparability, fairness, etc. For example, each state should carefully evaluate the technical quality of its student growth measures, particularly for schools with high student turnover, students at the tails of the achievement distribution, and other disadvantaged student groups.
  • Implement planned assessment and accountability (plans should include options if certain measures or indicators are insufficiently technically sound to include).
  • Determine if adjustments should be made to accountability determinations and/or reports (e.g. set new performance expectations).
  • Use 2020-2021 to update plans for 2021-2022.

Spring 2022

  • Implement planned analyses of Spring 2022 data for comparability, fairness, etc.
  • Implement planned assessment and accountability programs (plans should include transition options if certain measures or indicators were insufficiently technically sound to include in 2021).
  • Use 2022 to update plans for 2023.

We attempted to illustrate how state assessment and accountability leaders must recognize how the Spring 2020 COVID-19 school disruptions present challenges with implications for multiple years. Those challenges may be lessened by careful planning, analyses, and actions to help avoid potential negative consequences.

Share: