Educating Pablo: A Play About Through Course Assessment Systems

Dissecting the Conundrum of Testing Throughout the Year or Once at the End in a Conversation Between Teacher and Student

Various states have advanced through course assessment proposals, in which states administer tests throughout the year rather than only once at the end. Some of these proposals come through the Innovative Assessment Demonstration Authority (IADA). Others are externally funded initiatives or proposed as a one-time spring 2021 replacement for traditional end-of-year state tests. Although seemingly promising, through course innovations face an uphill battle in proving their coherence and superiority to a single summative test. Through course systems require both: they require that earlier tests contribute substantively to final determinations of student proficiency; and that entities held accountable for student results (such as schools, districts, and sometimes, teachers) commit to a reduced timeframe for beginning to demonstrate such results.

This scene from Educating Pablo, a fictitious play, illustrates the through course conundrum as it plays out in classroom grading practices. The teacher highlights “professional judgment” as a hallmark of her grading policy, but she applies a consistent algorithm to arrive at final grades. For all practical purposes, the professional judgment is built into the algorithm. A consequence of this practice is that final grades signify eitherwhat a student knows and can do at the end of yearor how they performed throughout the year. Despite this peculiarity, it strikes both parties as fair.

The vignette poses a challenge for designers of through course systems. Can large-scale assessment tolerate the ambivalence inherent in otherwise fair and valid classroom grades? Or does this demand too much of traditional educational measurement models for accountability tests?

EDUCATING PABLO, Act 2, Scene 1: Pablo and Ms. A.

Split stage. Pablo’s kitchen, where he has set up a makeshift classroom. Ms. Álvarez’s home office. For the audience, an enlarged version of Pablo’s screen.

An online session is ending. Pablo glances at his mid-term test on-screen, graded and with many red x’s, in disappointment.

Ms. Álvarez chats back at Pablo’s private “can we talk about my test?” with a “let’s talk after class.” At 10:45, Ms. Álvarez dismisses the class. Pablo’s classmates’ faces blink away three or more at a time until only his and Ms. Álvarez’s remain.

MS. ÁLVAREZ: Go ahead, Pablo.

PABLO: Does this one have to count?

MS. ÁLVAREZ: It’s 30% of the grade like the syllabus says. [Pablo puts his head in his hands.]

MS. ÁLVAREZ: Look, I know you’re disappointed. The important thing is that you can make it up by the end of the year. The rest of the course is all about integrals; this is what stumped you, and you will get better at those. You’ll see.

PABLO: The important thing is we understand it in the end.

MS. ÁLVAREZ: That’s right. That’s what I told the class today.

PABLO: So if I do really well on the final…

MS. ÁLVAREZ: Yes?

PABLO: It doesn’t seem fair! You’re going to test us on everything at the end anyway!

MS. ÁLVAREZ: Pablo, calma.

PABLO: I’m sorry.

MS. ÁLVAREZ: Let’s talk about this. Pablo, you are taking calculus. When you start college next fall, you will be taking more advanced math classes. You will get graded just like in this class, and you need to keep up with the pace, just like here. This is preparing you.

PABLO: If I do really well in the end, then I learned my lesson about that too, right?

MS. ÁLVAREZ: You make a good point.

PABLO: What point?

MS. ÁLVAREZ: That everything essential to learn is reflected in your performance on the final.

PABLO: Yes, what you just said. [Both chuckle.]

MS. ÁLVAREZ: But Pablo, don’t you want to know how well you’ve mastered the material at some point before the very end? How will you know that without a test?

PABLO: The homeworks, the self-assessments, the daily challenges?

MS. ÁLVAREZ: And you’re doing great on those. But the AP test is not like them. You’ll see problems that can come from any part of the course, and the test doesn’t tell you how to solve them. It doesn’t say, “On this problem, use LʹHôpital’s rule because you just learned LʹHôpital’s rule in class.”

PABLO: I know. That’s a big difference. In the self-assessments, I can rely on, um…What is it? It’s not there to clue you when you take the test.

MS. ÁLVAREZ: The immediate learning context.

PABLO: Yes, that’s it.

OK, Ms. A., you convinced me that the mid-term test is important because it helps us prepare for the AP test.

But why should it count?

Like, let’s say I did terrible on this one – I mean, terrible. Worse than the 55 I got. Ugh! 55! (And I’m telling you, I studied and really thought I was ready for this.) Anyway, let’s say I got a 25 instead. (And I’m telling you I prepared, Ms. A.)

But then, I suddenly finally get the thing about the um, the learning, um…

MS. ÁLVAREZ: The learning context.

PABLO: Right, the learning context. And then, I take the sample tests to check my understanding because those tests have a whole mix of things from different parts of the course. And then I do really well on the final. 100%. Haven’t I learned something important? I went from 25 to 100. I’d be a real success story, Ms. A.! Would a D or a C be fair?

Ms. ÁLVAREZ: If that happened, you wouldn’t finish the course with a D or a C. The syllabus also says that I take improvement into account when I calculate the final grades.

PABLO: How do you do that? I mean, how does it work? Do you have a formula?

MS. ÁLVAREZ: The formula, Pablo, is professional judgment.

PABLO: Does 55 to 100 get any professional judgment?

MS. ÁLVAREZ: Hm! You do realize I know that you are doing a unit on debate with Mr. Thomas?

PABLO: I’m not trying to argue. Really, no. I just want to get a better understanding of your… um, system.

MS. ÁLVAREZ: [Recognizing and accepting the challenge] All right. I count the final twice as much as the mid-term. You know that part. But, if someone does better on the final, then I drop the mid-term from the calculation and count the entire 90% as coming from the final.

PABLO: Yazzzz!

MS. ÁLVAREZ: I thought you weren’t trying to argue.

PABLO: No, I’m not! It’s just…a relief!

MS. ÁLVAREZ: Good. So don’t worry about this setback. Learn what you missed. You’ll do well.

PABLO: OK.

MS. ÁLVAREZ: Are we good?

PABLO: [Pauses.] Yes. But…

MS. ÁLVAREZ: What?

PABLO: No, it’s nothing, it’s…well, what about the people who do well on the mid-term but then go down on the final?

MS. ÁLVAREZ: Well, the mid-term helps them.

PABLO: But… Should it?

MS. ÁLVAREZ: They get credit for doing well on the mid-term. It would be mean not to let them get that.

PABLO: You’re right, yes. I wasn’t thinking about that. It’s just that, the grade that they get.… It doesn’t really say how they ended up in the class, right? I mean, what they know at the end is really less than the grade they got. It’s like it’s inflated or something.

MS. ÁLVAREZ: That’s one way to look at it. Here’s another: The course grade is not just about where you are in the end; it’s also about how you did throughout the year.

PABLO: Except for the kids that went up. Because of the, um, professional um…

MS. ÁLVAREZ: Professional judgment.

PABLO: Professional judgment. [Having an aha moment] Oh! For some kids, their grade is more about how they end the year. It’s about what they know at that time. But for the others, it’s more about how they did throughout the year.

MS. ÁLVAREZ: Very observant, Pablo. I know the season has already started, but you should seriously consider joining the debate team.

PABLO: Thanks, Ms. A.

MS. ÁLVAREZ: There are two kinds of course grades. Or maybe better, two different interpretations of course grades. Sometimes one interpretation applies; sometimes, the other.

Here’s a question for you: Now that you realize this, does it make you think about blowing off mid-terms?

PABLO: No! Wait. [Less impulsively] No, because it’s a chance to learn from them. Also, if I do well on them, that cannot hurt me.

MS. ÁLVAREZ: What about the final? Would you blow off the final if you didn’t do well on the mid-term?

PABLO: No way! Everything would be riding on the final.

MS. ÁLVAREZ: What if you did well on the midterms? Would you blow off the final?

PABLO: Nope, that wouldn’t make sense either. The final counts for more.

MS. ÁLVAREZ: Good. Always put in your best effort, no matter where you are in the course.

PABLO: Exactly.

MS. ÁLVAREZ: By the way, this reminds me of an issue that came up in a committee on which I serve. I’m interested in your opinion, Pablo. This um, system, as you put it, of having a single course grade, but with a different interpretation depending on whether the student improved from an interim test to the last test: Do you think this system is bad or unfair to anyone?

PABLO: Hmmm.

MS. ÁLVAREZ: Would the students who worked hard throughout the course think it’s not fair to them that some other students did well by quote un-quote [does air quotes] waiting to the very end?

PABLO: They might. And the group that did well in the final but not the mid-term might say it’s not fair that people are getting credit from the mid-term. If what’s most important is what they know at the end of the year, then, well…

MS. ÁLVAREZ: Yup, Mr. Thomas should put you on the school’s debate team.

PABLO: Seriously, though. It’s fair to both sides, isn’t it? Who would complain?

MS. ÁLVAREZ: You’d be surprised…Ever talked to a psychometrician?

PABLO: A psycho muh what?

MS. ÁLVAREZ: [Laughs. Suddenly becomes aware of another appointment.] Oh shoot, I’ve got a meeting with the principal in two minutes about my mid-term.

PABLO: Ha! OK, Ms. A. Good luck with that.… I mean, [more deferentially] I’m sure you’ll do well.

MS. ÁLVAREZ: Thanks, Pablo. I’ve truly enjoyed our discussion. I have to see about getting you on this committee. And the debate team. The debate team and the committee. Hasta luego.

PABLO: Thanks, Ms. A. ¡Hasta!