Stop Training the Trainers

Moving Toward a Research-Based Approach for Improving Assessment Literacy

“Train-the-trainers” is commonly employed as an approach for attempting to spread new learning from individuals who attended a professional development experience to others who did not attend. Unfortunately, this model often works like the children’s game of “telephone,” where the message is mangled by the time it gets around the circle. I am struck that “train-the-trainers” continues to be so popular with so little evidence that it works to improve the implementation of complex knowledge and skills. A district or school leader may think they can get a bang for their buck by spending money for just a few teachers to participate in professional development, as long as they come back and spread the word. This approach might work for simple changes, but creating widespread and deep-seated reform requires time-intensive efforts based on how people really learn. At the Center for Assessment, we have applied such approaches to the development of assessment literacy. 

Fundamental Change Takes Time, Support and Flexibility

We draw from the research on teaching literature to help us understand the nature of professional learning opportunities that can support significant shifts in teachers’ understandings, beliefs, and practices. Putnam and Borko (2000) summarized this research, noting that teachers are most likely to make and sustain fundamental changes in instructional practices if provided time, conceptual and strategic support, and opportunities to try new practices in the context of their own teaching. 

Conceptual and strategic support extends typical high-quality professional development approaches of sustained learning embedded in actual practice by relying on experts, to the extent necessary, to help guide complex reforms. Shepard (in press) notes that teaching and assessing in “fundamentally different ways is a complex and daunting task,” and it is misguided to believe that teachers can engage in this work alone or without significant support. 

The Center for Assessment has relied on a sociocultural perspective in our work, particularly Lave and Wenger’s (1991) concept of “legitimate peripheral participation,” to improve assessment literacy at scale. The term “legitimate peripheral participation” describes how novice members of a community of practice become fully-participating and eventually expert members of a community. The experts support peripherally-participating members (novices) to develop the strategies and skills associated with expertise (Lave and Wenger, 1991). This approach to learning is notably different than the more intuitive, but much less successful, “train-the-trainer” approaches. 

We first used this approach to professional learning with Wyoming’s Body of Evidence system about 20 years ago, and subsequently in Colorado, Utah, and Rhode Island. We have continued to refine this approach and are now using it for developing science assessment expertise with the Alabama Mathematics and Science Teachers Initiative, competency-based education strategies with the Arkansas Department of Education, and supporting assessment expertise with New Hampshire’s Performance Assessment of Competency Education (PACE) as I highlight below (see https://www.nciea.org/current-initiatives/innovative-assessment-and-accountability-systems for some examples of the Center’s work in this area). 

Supporting Assessment Literacy in New Hampshire’s PACE Initiative

Figure 1 below portrays the multi-level approach used to develop assessment literacy in PACE. Each group plays an important role in the development of assessment expertise and the success of PACE. 

Assessment Experts: Assessment leaders from the Center for Assessment serve as experts to provide conceptual and strategic support (Putnam and Borko, 2000). The 60 or so content leads (in ELA, math, and science) are the linchpins in the entire process. They are the ones who start as apprentices and shift into the role of masters. 

Content Leads: The content leads have an ongoing working relationship with the Center for Assessment professionals to build their assessment knowledge over multiple years of embedded practice. These content experts lead the task development workshops for other participating teachers. Initially, the Center experts provided considerable support before and during these workshops, but our role has increasingly shifted to the sidelines as the content experts increased their knowledge and skills. Now, three years later, the content leads provide essentially all of the task development leadership. However, these content leads still meet three times each year with Center professionals to discuss emerging issues and to deepen their assessment knowledge around key topics. This ongoing mentoring and support as teacher leaders build expertise and apply their skills is the key difference between this approach and a “train-the-trainers” model. 

 

Assessment Developers: The assessment developers—approximately 350 teachers from multiple districts working on common performance tasks—work directly with the content leads and to a lesser extent with the Center for Assessment. These assessment developers have the opportunity to develop deep levels of assessment literacy through their work on task development, reviewing student work, and participating in calibration and scoring activities. Some of these members are recruited into the content lead ranks to increase their assessment expertise. 

Local Educators: The outer circle includes those educators (approximately 1500) who have not been directly involved in PACE task development activities but are expected to implement performance-based assessments. The PACE theory of action posits that by creating widespread assessment literacy among participating teachers, those teachers not directly involved in these professional learning activities will have the opportunity to learn in their schools and districts from colleagues who have developed true assessment expertise. We are in the process of collecting data to evaluate this assertion.

There is no question that our research-based approach to professional learning requires considerably more time and investment than a “train-the-trainer” approach. However, helping educators develop a deep understanding of assessment is a complex undertaking that must be based on sound conceptual and practical underpinnings.

References
Lave, J., & Wenger, E. (1991). Situated learning: Legitimate peripheral participation. Cambridge, MA: Harvard University Press.
Putnam & Borko (2000). What do new views of knowledge and thinking have to say about research on teacher learning? Educational Researcher, 29, 1, 4-15
Shepard, L. A. (in press). Assessment for classroom teaching and learning. Workshop on Educational Assessment as Useful and Useable Evidence. National Academy of Education and American Academy of Political and Social Science. September 13-14, 2018