Supporting the Improvement of Educators’ Assessment Literacy

Developing Needs-Based Sociocultural Approaches to Professional Development and Assessment Literacy

This post is a summary of the main ideas from a symposium focused on educators’ assessment literacy presented at the 2021 NCME Classroom Assessment Conference. Slides from the session are available here. 

As Carla Evans wrote in a recent post, if we have not been successful in improving educators’ assessment literacy it has not been due to a lack of trying. Countless books have been published describing assessment literacy and its importance in the instructional process. There are just as many books, if not more, discussing the use of formative assessment and classroom assessment; and an internet search will reveal a plethora of offerings designed to improve educators’ assessment literacy. Perhaps our lack of success in improving educators’ assessment literacy is due, in part, to the misalignment between our approaches to professional development and our evolving understanding of sociocultural approaches that consider the impact environment and experience have on educators’ assessment practices and perceptions.

Perhaps we’ve focused too much on assessment literacy as the acquisition of assessment-related knowledge and skills rather than the development/refinement of intellectual tools and strategies consistent with core assessment principles. 

Perhaps we’ve failed to recognize the “supremely contextual” nature of teaching and learning, and the ways teachers resist reforms that superimpose rules or homogenize the craft of teaching. Who the teacher is matters. Who the students are that we teach matters, and what we are teaching matters.

Perhaps we’ve focused on the development, adoption (i.e., widespread use) or replication (i.e., widespread implementation with fidelity) of standardized assessment literacy resources, without thinking enough about the need for adaptation (widespread use that is modified according to the needs of local users) (Morel et al., 2019).  

Professional development models that consider context and are designed for adaptation are better aligned with current thinking about assessment literacy:

Assessment literacy is a dynamic context-dependent social practice that involves teachers articulating and negotiating classroom and cultural knowledges with one another and with learners, in the initiation, development and practice of assessment to achieve the learning goals of students (Willis et al., 2013, p. 242).

How would our approach to professional development change if we focused less on identifying what is generalizable, and more on understanding the potential impact of sociocultural factors on assessment literacy efforts?

Shifting The Focus of Assessment Literacy Efforts

If we accept that assessment literacy is a “dynamic context-dependent social practice that involves teachers articulating and negotiating classroom and cultural knowledge” then it follows that teachers will approach assessment differently given their experience and the environment within which teaching occurs.

Given this scenario, how can we come to establish and provide useful and generalizable professional development that supports improved assessment practices? How can we structure professional development in a way that accounts not only for the goals and purposes the training is intended to serve but also takes into account who it is trying to serve, who is providing it, and the outcomes that are expected?  

A Framework to Support Planning for Professional Development

In our recent RILS conference we talked about needs assessment as a way to identify and approach “wicked problems” of practice: the formal process of identifying needs as gaps between current and desired results, placing those needs in priority order based on the cost to meet each need versus the cost for ignoring it, and selecting the most important needs (problems or opportunities) for reduction or elimination [Kaufman 1992, 1998].”

Taken together, this definition of needs assessment and the revised conceptualization of assessment literacy as a social process provide a foundation upon which to design effective professional development. In this context, effective professional development in assessment literacy is that which targets the information and practices stakeholders need to use assessment to help them achieve desired results.  

Consequently, the needs assessment process must determine who needs what information and why so that the how can be articulated and appropriately represented in the professional development approach. Questions that can serve to support this process are provided below:

• What results are we looking to achieve?
• What is the nature/degree of discrepancy between the current state and the desired result (i.e., What is missing)? 
• What factors are contributing to that discrepancy? 
• Who is responsible for or will influence whether the result is met?
• What is the role of assessment (or knowledge related to assessment) in achieving the desired result?
• What specific information/practices do educators (and other stakeholders) need to have in order for assessment to effectively play this role? 
• Is this information generalizable across stakeholder groups and contexts?
• What aspects of a professional development approach focusing on this information will need to be adaptable (i.e., to address local context and goals)?

As you can see from this list, it is not until the 5^th bullet that the role of assessment in achieving the desired result is discussed. The intent is to pose questions that identify the need and the theory of action underlying the design of your professional development approach. Once these questions have been addressed the focus and design of required assessment literacy efforts can be established. 

There are four ways to classify the information need or gap to be addressed through professional development (i.e., What is missing?). Each class of information necessitates a different level of adaptivity in the professional development approach in order to ensure the sociocultural nature of assessment literacy is appropriately addressed.

• Foundational: General information that everyone needs in order to effectively communicate and engage with each other (e.g., language, definitions, processes).
• Targeted: Information focused on filling a specific gap in knowledge/understanding that is necessary to achieving a well-defined goal.
• Situated: Information necessary to support the attainment of a specific goal that facilitates understanding and implementation of assessment in light of a number of highly relevant contextual factors.
• Individualized: Information needed at a given moment to address an immediate concern, question, or misunderstanding. 

Adapting Professional Development Approaches to Fit the Need

Professional Development approaches fall on a continuum of adaptability (Koellner, K. & Jacobs, J., 2015). At one end of the continuum are Highly Adaptive approaches, which are designed to be modified (or adapted) in response to contextual factors that might influence efficacy if not addressed. These approaches are based on general guidelines rather than specific content. 

At the other end of the continuum are Highly Specified approaches, which use standardized content and facilitation to achieve a clearly specified goal. Some approaches fall along this continuum, including adaptive and static elements. 

In the remainder of this post, we briefly describe several approaches that range from Highly Specified to some that are still specified but can be adapted to meet state and district needs.

Approach 1: Open Educational Resources

• Highly Specified but Open-Source and Able to be Adapted with Attribution – Assessment literacy professional learning modules for leaders and teachers to support foundational knowledge.
• Example: Center for Assessment – Classroom Assessment Learning Modules. The Center created a series of open-source modules for educational leaders and teachers that support the development of educators’ classroom assessment literacy in areas critical to student academic success and reducing achievement gaps. 

Approach 2: State Supported Resources

• Specified but Adapted for the State’s need – Assessment literacy professional learning modules for teachers and leaders to support foundational knowledge and development of balanced assessment systems.
• Example: Wyoming Department of Education and Center for Assessment – Wyoming Teacher Assessment Literacy Modules. A series of teacher and school leader assessment literacy modules provided as an open educational resource to all educators in Wyoming. 

Approach 3: State Agency Supported Professional Development Resources

• Specified but Adapted for the State’s need – Assessment literacy professional learning modules for teachers in support of foundational knowledge focused on accelerated learning using formative assessment processes in the classroom.
• Example: A series of micro-courses on the use of formative assessment in the classroom developed for new and novice teachers, advanced teachers, and school leaders. 

Approach 4: State Supported Professional Development and Resources

• Specified but Adapted for the State’s need – Assessment literacy professional development and learning modules for leaders and teachers in support of foundational knowledge aligned with Tier 1 curriculum and focused on school/district improvement
• Example: Louisiana Department of Education – Assessment Literacy Initiative. The goal of the initiative is to cultivate assessment literacy among Louisiana educators in order to improve student learning. This goal will be accomplished through a series of learning modules developed with teams of educators from across Louisiana. 

Approach 5: State Supported Professional Development and Resources

• Specified but Adapted for the State’s need – Assessment and data literacy professional development and learning modules for district leaders, state coaches, and higher education leaders supporting pre-service teachers in support of foundational knowledge focused on educator/school/district improvement.
• Example: South Dakota Department of Education and Center for Assessment – South Dakota Assessment & Data Literacy Initiative. The South Dakota Department of Education is recruiting regional coaches, school/district practitioners, and university/teacher preparation faculty to participate in a two-year Classroom Assessment and Data Literacy Initiative. 

Approach 6a: State Supported Professional Development and Resources

• Specified but Adapted for the State’s need – Assessment literacy professional development and learning modules for teachers is targeted to support the development of performance assessments measuring state competencies.
• Example: Michigan Department of Education and Center for Assessment – A series of open-source modules used by the Michigan Assessment Consortium and Michigan Department of Education to develop a bank of performance assessments to measure state competencies in English language arts and mathematics. 

Approach 6b: District Supported Professional Development and Resources

• Specified but Adapted for the District’s need – Assessment literacy professional development and learning modules for district leaders and teachers in support of foundational knowledge-focused educator evaluation.
• Example: Polk County Public Schools and Center for Assessment – Professional development and modules include understanding foundational assessment literacy information and how the information supports Student Achievement Objectives.

Conclusion

Conceptualizations of assessment literacy as a social practice are more consistent with the characteristics of Highly Adaptive approaches, but that does not mean all professional development focused on supporting assessment literacy has to tend toward this end of the continuum. For example, if the goal is to establish a common language and understanding among stakeholders to facilitate collaboration (i.e., foundational information) a Highly Specified approach may be appropriate. As the examples above demonstrate, the appropriate degree of adaptivity depends on the class of information required and how it is intended to be used.