Getting a Grip

A little while ago, I wrote about cognitive rigor and the different ways we have of measuring it. I called the post, “Climbing the Pyramid.”

Towards the end of that post, I said:

Our goal should be that final level—the ability to take all the things you’ve been given and make your own, wild and wonderful and weird creations—the ability to carry your learning into life and use it in ways that no one had planned and that no one could have been able to predict, to enrich your life and empower your ambitions.

I still think that’s a worthy and lovely goal. But…maybe it’s not entirely fair? I don’t know. I’m working through it. Not every student is going to climb to the top of the pyramid in every subject area, every year of school. Some might never get to the top in any subject area. Not every kid has that level of curiosity or drive or intellectual ambition. Every kid has some, but maybe it’s unfair to expect every young learner to find a passion while in school and pursue it to the Nth degree.

And let’s be fair to the teachers. Maybe they can’t push kids to the tippy-top—not every teacher, with every kid, every day. They might not have the right resources; they might not have the right background; they might not have enough time in their day; they might simply not have enough fuel in their tank.

I still believe we should set high expectations. We should shoot for the moon, even if we don’t always land on it. Aim for 100; settle for 90. That’s so much better than aiming for 70 and getting 50.

That’s me. But is the dream of aiming for the top of the pyramid really the school’s goal? You’d like to think it is, but can it be, when schools are held accountable first and foremost to meeting the state’s learning standards, wherever they may be pitched and however they may be assessed? Don’t they first have to guarantee the legislated and mandated floor before climbing towards the ceiling?

If they even know where that floor is?

Let’s take a look.

Subscribe now

The What

Let’s start with a little thought experiment adapted from the excellent book, Driven by Data, by Paul Bambrick-Santoya. And let’s look at mathematics, since it’s allegedly so clear and concrete, at least when compared to language and literature*.

In the state of Nevada (nothing personal—just choosing a state), students are expected to be able to do the following by the end of Grade 7:

Use proportional relationships to solve multistep ratio and percent problems. Examples: simple interest, tax, markups and markdowns, gratuities and commissions, fees, percent increase and decrease, percent error.

Nevada Mathematics Standards: Ratios & Proportional Relationships 7.RP.3

Clear? Concrete? Cool.

Imagine four classrooms, each teaching a unit aligned to that standard. Each classroom assesses the skill slightly differently, using questions like the ones below. For the sake of argument, assume nobody is mixing it up: Classroom 1 is asking a whole bunch of questions just like Question 1, Classroom 2 is pitching everything at the level of Question 2, and so on.

So, which classroom is assessing the standard appropriately?

Identify 50% of 20.

Identify 67% of 81.

Shawn got 7 correct answers out of 10 possible answers on his science test. What percent of questions did he get correct?

J.J. Redick was on pace to set an NCAA record in career free-throw percentage. Leading into the NCAA tournament, he made 97 of 104 free-throw attempts. In the first tournament game, Redick missed his first five free throws. How far did his percentage drop from before the tournament game to right after missing those free throws?

They’re all questions about percentages. So, from a purely topic-level alignment, they’re all teaching what they’re supposed to be teaching. But standards are more than mere topics, as you can see from the detailed language. And of these questions, only #4 seems to align with any the examples they provide, specifically “percent increase and decrease.” The others, even the word problem, are only asking the student to find a percent, which happens to be the standard from the previous grade:

Find a percent of a quantity as a rate per 100 (e.g., 30% of a quantity means 30/100 times the quantity); solve problems involving finding the whole, given a part and the percent.

Nevada Mathematics Standards: Ratios & Proportional Relationships 6.RP.3

Are there schools in Nevada (again, not trying to pick on them in particular) dumbing-down expectations and teaching Grade 6 content as though it’s aligned to Grade 7 standards? I have no idea. From my experience pretty much everywhere else in the union, I wouldn’t be surprised.

Let’s get back to Question 4. Does that one meet the Grade 7 standard? It addresses percent loss, which is one of the examples they provide for guidance. And it asks students to find both the “before” and the “after” percent, and then to do some subtraction. So it’s a multistep problem, which is a requirement of the standard. So, it’s aligned. A teacher using questions like that in her classroom is likely to be teaching the skill and applying it at the right level of rigor.

Except. Hold up. What if I have another classroom whose teacher looks at Question 4, thinks it’s “too hard,” and simplifies it like this:

Subtract 50% of 20 from 67% of 81.

Doesn’t that still meet the standard? It’s exactly the same math, and mostly the same task. Percent loss; check. Multi-step; check. In theory, the instruction in the two classrooms should be equally well-aligned. But…is it? Do those two questions meet the standard in the same way? Do the kids working in those two rooms really know the same things?

Questions 4 and 5 both meet the standard, but they require different levels of thought and rigor in application. Question 4 is more challenging, because it demands reading and interpretation; it requires students to figure out the math equation that Question 5 simply gives them. It also mixes numerals (97 and 104) with a spelled-out number (five) to create more confusion. You have to find your way to the actual math problem in Question 4. That’s a whole, separate task—and a reasonably complex one, too.

But that’s not anywhere in the standard! Is it fair to judge a student’s math knowledge based on their reading ability? There’s nothing in the standard that says “using simple equations only” or “using complex, real-world problems.”

So, if Classroom 4 asks a ton of questions like Question 4 and Classroom 5 never does, even if the math is the same, are they both teaching to the standard? Do the kids move on to 8th Grade with the same level of knowledge and skill? How is a teacher supposed to know how students are meant to apply what they know?

The Theoretical How

Well, for the state of Nevada, and for any other state that has adopted or adapted the Common Core State Standards, there is a second set of math standards that sits alongside the content standards. They’re called the Mathematical Practice Standards, and they address the “how” in eight, explicit statements that remain constant across all grade levels.

The job of the teacher—and the creators of instructional resources—is (in theory) to teach kids how to apply these eight enduring habits of mind to their particular grade-level content. A teacher should (again, in theory) be able to take Standard 7.RP.3 on proportional relationships and have students practice doing all eight of these things with that math.

So, this should provide an answer to my question above: is it fair to make students have to read and interpret a wordy and confusing question in order to produce an equation before solving it? Sure—it’s right there in Practice Standards #1 and #2. Simply performing a calculation isn’t sufficient.

Triangulating between the content and practice standards should set clear expectations around cognitive rigor—what and how much to do with the math you learn, and how far to take it in order to be able to say that you “know it.”

In theory.

The Actual How

Some teachers create their own curriculum, but most rely on some kind of published program, at least part of the time. Published curriculum, whether provided by a Big Company or offered through something like Teachers Pay Teachers, certainly instructs at a particular level of rigor. But is it always the right level, just because someone says it is?

Caveat emptor, as I seem to have to write in every blog post—don’t assume they’ve done their homework completely or correctly. Even the big guys.

Teachers will also look at tests or sample items provided by their state. State assessments are where the rubber meets the road, after all. Teachers can see exactly how rigorously a standard is going to be assessed by the Ultimate Authority. And if we’re talking about the floor, not the ceiling, what the test-makers say has to rule the day. That’s what everyone is held accountable to.

Well, in the case of Nevada, that Ultimate Authority is the Smarter Balanced Assessment Consortium, of which Nevada is a participating state. Using this helpful website, I can see exactly what sample items aligned to Standard 7.RP.3 look like:

Remember, we’re assessing the floor—the minimum level of performance expected of students who will be thought to have “met the standard.”

So, what do we notice? All of these are real-world problems, or at least quasi-real-world. All of them require moving from words to equations.

But that first question, about Sara: it’s not exactly multi-step, is it? Find 25% of 30. Yes, you have to express the answer in dollars, but so what? You’ve been in dollars the whole time. You’re not converting from something else. One step. It’s not even a subtraction question: the correct answer is the percent saved, not the new total. In Nevada, at least, that should be a 6th grade question.

Question 2 is better. It requires you first to figure out Jenny’s original hourly rate, then to figure out the percent increase, and then to figure out her new rate. Fair game.

The last question—what do you think? It seems to be multi-step and reasonably complex.

But notice something: all of these are open-ended questions, not multiple-choice. Open-ended questions are, by themselves, cognitively different challenges from multiple-choice questions, and that, alone, sets a different expectation for student performance. The standards say nothing about question types, but the Practice Standards seem to point in the direction of doing more problem-solving work on your own. But if a teacher has given multiple-choice tests all year long, some of her kids are going to be in some amount trouble on the state test, even if they know their math.

But even with open-ended questions, how many of those Practice Standards do you see being assessed? Maybe just the first two. Mostly it’s just straight computation.

So what is actually required to meet the standard: what a standard says or what the test assesses? The latter is what will be reported to the state, the school, and the parents. The test rules the day. And it often rules the curriculum—not what’s printed, necessarily, but what gets fit into the schedule. “Meeting the standards” doesn’t mean covering the entire textbook; it means providing enough instruction to succeed on the assessment. This is how it has been for nearly a quarter of a century, since the passage of No Child Left Behind and the flowering of state assessments at every grade level.

Subscribe now

Gaming the System

Even with all of these tests, we probably know less than we think we know about what our kids actually know. After all, even if we could somehow, magically, get to a place where every single student in a state was judged as “meeting the standard,” would we know, for sure, that every one of those children could do everything listed in the standards documents?

No. We’ve already seen that.

Well, would we at least know that all of those “meeting the standard” students had gotten 65-70% of the questions on their state assessments correct (or whatever the “passing” grade is, normally, in their schools)? That they had at least mastered the test?

Not really.

Just as there’s wiggle room between the standards and the tests, there’s wiggle room between the actual scoring of the test and the reported scores. There can be a lot of secret math that goes on between raw scores and scale scores, and there can be different expectations set for when a student has to retake a state test again and again. I’m not saying state agencies game the system to get the results they want, or at least the results their voters will tolerate, but…I’m not not saying it, either.

And it’s not just them. Between the scores that get pushed out and what schools do with that data, there’s even more wiggle room. Between what schools may try to do and what parents will allow: same.

We have what looks like an accountability system, but it’s one in which nobody is really held accountable. We’ll allow some kids to slip through without knowing all they’re supposed to know, because holding their feet to the fire means holding everybody’s feet to the fire, from the parents all the way up to the governor, and that’s…really hard.

There was a year, a few decades ago, when some New York City schools decided that they couldn’t, in good conscience, promote to high school students who hadn’t passed their Grade 8 state tests. So they created an 8+ program to hold onto the kids who hadn’t been promoted. What was the result? I’m working from memory here, but as I recall, new sixth-graders moved into the middle schools and a bunch of old eight-graders didn’t move out, and the population swelled. The schools became angry and big and chaotic, and it was just going to get worse the following year, because nobody had a real solution for how to help those 8+ kids learn what they had failed to learn when they were regular 8th graders. So, the schools canceled the program and the students were moved along. I don’t remember if they gave the kids an easier test to pass, or simply gave up. But the idea of 8+ just…stopped.

Sometimes, if you can’t solve a problem, you just redefine it and call it normal.

All of this is why the notorious and embarrassing A&W Third-Pound Hamburger fiasco of the 1980s, in which many, many adults seemed not to know that a third was larger than a quarter, could just as easily happen today. It’s why public reporting on student achievement may look like things are getting a little bit better or a little bit worse, each year, but nothing really seems to change.

To understand what students actually know and can do means having the courage to look at reality—open-eyed and without flinching—and speak the truth about what we see. And if we do that, I don’t think the main question will be, “how can we get them from the floor up to the ceiling?” It will be, “how can we get them all up to the floor?”

Share

Leave a comment

*I was an English teacher, so I know I’m venturing into dangerous territory here by talking about math AT ALL. If I’m screwing anything up here, I accept full responsibility…except for that little bit of responsibility my teachers had, a million years ago, for making sure I developed a tiny bit of number sense.