The Myths of Standardized Tests: Why They Don't Tell You What You Think They Do

by Phillip Harris, Bruce M. Smith, Joan Harris

Pundits, politicians, and business leaders continually make claims for what standardized tests can do, and those claims go largely unchallenged because they are in line with popular assumptions about what these tests can do, what the scores mean, and the psychology of human motivation. But what most of what these opinion leaders say-and the public believes-about standardized testing just isn't so. However, few members of the general public, not even concerned parents, have the time or the background to keep up with the latest findings of testing experts, psychometricians, and researchers. That's where The Myths of Standardized Tests comes in. In simple, accessible language, Harris, Smith, and Harris spell out the assumptions underlying standardized tests and point out what's true about them and what's just plain mythical. But they not only debunk common assumptions; they propose better ways to judge the success of our schools. They also offer readers suggestions for ways they can help reduce the burden of tests on their children.
Appendixes offer readers contact information and suggestions for actions they can take to become part of the solution to the problem of overusing and misusing standardized tests.

Genres: Education, Nonfiction

206 pages, Hardcover

First published January 16, 2011

Reviews

[C.S. Malerich]

Explains the problems with emphasizing test scores as the sole indicator of good schools and student learning. Tests are actually poor indicators of creativity, complex & critical thinking, and success in other areas of school and life -- not to mention empathy, honesty, or self-confidence. That much has always seemed intuitive to me. This book does a pretty good job of explaining the whys and hows of testing to show that, for most of what we expect tests to indicate and predict, they just don't work. Among their points:

1) Tests can only sample knowledge. Standardized tests are mostly limited to math and language arts, and yet they are still expected to test a broad range of competencies. In order to fit the test into a reasonable amount of time, only one or two questions, for instance, might be given on a whole test to determine if a 3rd-grader knows their multiplication tables from 1-7. But missing 8 x 9 says nothing about whether a student knows 7 x 5. Yet missing that single question would be enough to signal to evaluators that the student is lacking in the whole area of multiplication tables.
2) Tests ARE subjective -- the selection of questions, setting of standards, and cut-off grades are all based on human judgment, not hard-and-fast laws of nature.
3) Tests encourage superficial learning--knowing just enough to guess the right answer, not thorough understanding of complex concepts.
4) Test questions are tested and selected for the greatest score spread. Ideal questions, which separate the "losers" from the "winners," are those which, on average, 50% of students answer right, and 50% answer wrong. If schools effectively teach the test content, however, close to 100% of students would answer every question correctly. As tests are re-tooled, this amounts to making students and teachers play a shell game: as soon as a teacher is effectively teaching one content area (so that nearly 100% of students get those questions correct), that content area is phased off the test in order to give the evaluators the point spread they want to see.
5) Rewards and punishments are not effective encouragements. Some studies show that as soon as a person is rewarded for taking part in an activity, the inherent joy, satisfaction, and interest of that activity lessens. This seems to be because a person's focus shifts from the activity to the reward. So, if I'm rewarded $5 for every A on my report card, my attention to what I'm actually supposed to be learning in school falls off, and, once the incentive is removed, I'm less likely to pick up a book or think about a scientific problem on my own.
6) Standardized testing has shifted school curriculum in demonstrable ways, including cutting time from science, art, music, social studies, recess, physical education, and lunch time. The curriculum content that remains is "standardized," i.e., made superficial and rote.

It turns out that tests are only really good at predicting how well a person will do on future tests (i.e., people who do well on tests are proven to be good test-takers, and THAT'S ALL the tests reliably prove). Many assumptions are made about how well teachers are teaching & how well schools are educating whenever standardized test scores make the headlines. Hopefully, after reading this book, we'll be able to argue and win when we hear that "teachers have to be held accountable" and "schools are failing," when all policy-makers are considering are standardized test scores.

[Claudia]

This book left me wanting to stand up and cheer. The authors (from Indiana University, but I swear that's not why I loved it!!) systematically explain what standardized tests can do (predict performance on other tests) and what they cannot do (show achievement, label students' learning, be used to retain students or punish teachers or schools). I have six pages of quotes from the book that I will be using in my work about testing abuses.

They talk about accountability, narrowed curriculum, grading schools and teachers. They discuss the high-stakes efforts that are sweeping the nation as policy makers think they're doing the right thing.

They discuss the cheating that goes on when tests become high stakes, identifying any test prep as a form of cheating, since these tests are normed on students who were NOT prepped for the tests. They talk about how reliance on standardized testing is dumbing down curricula, narrowing it, and taking learning opportunities away from students...they even talked about one elementary school that completely did away with recess.

Their discussion of the SAT and GRE were telling. SAT as a measure doesn't predict grades for the first year of college any better than high school grades...and that's all the SAT is designed to do. Many colleges have stopped requiring it for that reason. In fact, a better indicator of success after high school is involvement in outside leadership opportunities...but no one can make money on that like Pearson and other testing companies can.

One of the authors had the same experience with the GRE that I did. GRE claims to predict success with graduate work. We both had masters' degrees, a strong indication that we would be successful in graduate work, but we both were required to take the GRE anyway...nothing more than another box to check off.

Ultimately, the authors show how we should NEVER put our faith in tests; instead we should put our faith in teaching and authentic learning, and leadership, and service.

I'll be sharing this with everyone I know!

[Lynn]

This book is a good overview about why standardized tests are a bad judge of students, teachers and schools. They need objective, but were created subjectively from the beginning when people decided what the children should know and created the questions. While tests go through a standardizing process, testing them out on children in the same grade level and choosing what questions 50% of the sample students get right and 50% get wrong. Most tests don't predict very far into the future and can't predict who will be successful in life. SAT tests are claimed by the College Board to predict grades in the 1st year of college when students usually have large classes and assessed often by multiple choice tests. GRE tests don't make claims for graduate school where students receive As and Bs anyway if they pass their coursework. Tests given in K-12 don't make the claims for future success but are used to evaluate teachers and schools without explanation on how they do that or use numbers that doesn't prove their claim. A good primer that I hope to follow up with more readings and information.

[Susan D'Entremont]

This book was an excellent outline for a layperson on how tests are made and what they can and cannot tell you. The authors explain a lot about how to interpret the statistics of testing without using a lot of technical language. Especially interesting to me was the discussion of Campbell's Law, which basically says that if you rely too much on one factor to evaluate something, that factor no longer is a valid indicator because people are consciously focusing on it.

I felt the second part of the book was not as strong as the first part. There wasn't as much discussion as I thought there would be, based on the dust jacket, about how to deal with the fact that policy-makers relying too heavily on tests to make decisions. I also would have liked more discussion on how to evaluate teachers if you don't use tests. Yes, teachers are in the best position to evaluate their students' work, but some are much better at this than others.

All in all, though, worth reading.

[Elise]

Three educators debunk several commonly held beliefs held sacred about the US educational system today such as:

* High test scores of students at any particular school are reason enough to assume that they have high student achievement and quality teaching

* Improvement in scores on "high stakes" tests is a sound indicator of improvement in learning

* Punishments or rewards to schools, teachers, or individual children based on their test scores motivate them to do better

A good complementry read to Diane Ravitch's "The Death and Life of the Great American School System: How Testing and Choice Are Undermining Education". It also contains a good resource guide for concerned parents, teachers and citizens to take next steps.

[Esther]

This is a very readable book. It tackles the assumptions behind why we use standardized tests, what they actually mean, and more importantly, how we could have a system that achieves our educational goals without such absurdly narrow reliance on them. The tone if very folksy and informal, and suddenly you realize you're reading paragraphs discussing "all quantifiable predictor variables" and not blanking out. Nicely done.

[Kate]

I may have run out of steam on my standardized test readings, only because the truths seem so obvious to me that it's amazing to see the necessity for their repetition, but Harris, Smith, and Harris do a great job with data and discussion of tests and the small pieces about testing by ed names move the text along nicely.

[Casey]

An interesting look at how standardized tests are made and the problems with using them to evaluated student learning and teacher effectiveness. There was also a helpful discussion on the goals of public education.

[Scott]

A good read full of useful information, although unless you are interested in this at all, you probably won't find it very engaging. In a day of "Blinks" and "Freakanomics," the book feels a little bland.

[Qwerty]

A good primer on the uses and misuses of standardized testing.