Measure What You Value

In the quantum world, observing a thing can affect or change it, which makes it hard to gain any solid or objective knowledge. In the macro world, where a tree is a tree no matter what angle you look at it from, measuring and tracking can still affect the thing being observed.

When we decide to care enough about something to test it, measure it, analyze it, and report our findings about it, we can affect what the thing actually is. We can warp it by prioritizing it. We can warp other things by deprioritizing them.

Subscribe now

As I wrote recently, when we decided to institute high-stakes academic tests across the country, many schools responded by changing their curriculum to hew closer to what was being tested—and deprioritized things that were not being tested, like reading whole novels or writing research papers.

But it’s not only a school issue. Here’s one obvious example: the quality of Hollywood films seems to matter less to people these days than the opening weekend box office numbers—or, at least, the box office figure has come to be a stand-in for the film’s quality and value. Not because any of us have a financial stake in the movie, but because those figures are blared out at us now in ways that they never used to be. You can go find reviews and ratings, but you can’t escape the revenue figures.

Beyond Hollywood, you can see how products created by all kinds of companies, and the quality of life enjoyed by the people who work there, become less important in our discourse than the company’s revenue, because the stock market is the sole piece of data used to describe the success and health of businesses. That’s the data point we all see, so that’s the data point we decide must be important.

We change the world by deciding what to look at.

Inside School

Do our test scores really tell us whether students can read and do the math expected of them in school? Maybe. I used to have my doubts, but as curriculum has shrunk to fit the tests, maybe they’re a decent match for each other nowadays.

I do not take that to be good news.

But do our test scores tell us whether students can read and do the math expected of them out in the world beyond school, in their careers and their pastimes and their civic responsibilities? That is debatable.

How often will our young graduates be forced to read a handful of paragraphs that they didn’t choose to read, about a topic they don’t care about, presented to them devoid of any meaningful context and with no expectation that they’ll do anything important with the information they’ve read?

I mean, I guess it could happen.

But after they read these things, how often will their sole task be to select a correct answer from among four choices, three of which have been deliberately designed to confuse or distract them?

What about math? How often are any of us forced to solve a ready-made equation? I don’t know about you, but I’ve never walked into a basement that needed carpeting and found the area equation just sitting there, waiting for me to solve it, with one value represented by an X. When I have to do math, I often have to do a lot of work to get to the equation that I need to solve. The mathematical thinking is the hard part for me; performing a calculation at the end is a lot simpler. But I was trained on calculations far more than I was taught to think mathematically.

We measure what we care about. Do we care about the right things?

Do we measure a student’s ability to sift through dense information and separate facts from nonsense?

Do we assess the kinds of questions that students might ask of a text, to gauge their curiosity or their skepticism?

Do we ask students to list additional things they might want to learn about after having completed a reading?

We do not.

Do we track the ways in which students’ abilities to handle math change when the same concepts or skills are presented purely numerically vs. in text vs. through charts and graphs?

Do we ask whether their ability to handle math changes or falls apart when ambiguity and unpredictability are woven into the problem?

We do not.

Regardless of subject matter, do we test and measure and report a student’s ability to persevere through a challenge, tracking where and why a student might give up? I’d love to see test scores that told me how deep into a test a student was willing to wade, and at what level of cognitive difficulty they gave up.

Do we take surveys of student happiness and satisfaction, and put them up alongside state test scores? Do we report participation in extracurricular activities? Do we try to learn about a student’s willingness to collaborate with peers, and their effectiveness at doing so (and look to see if it is different in different subject areas, or different between academics and extracurriculars)? Do we know how to measure a student’s ability to be creative and innovative? Do we track instances of ethical and responsible behavior?

By and large, we do none of those things in any serious or public way. And yet, in our “Portrait of a Graduate” infographics, we claim to care deeply about all of those things.

I have news for you: we don’t really care. Maybe particular schools and teachers and administrators care, but as a society, we don’t. If we did, we would ask our schools to measure those things and share the results with us. Schools would analyze the data. They would identify gaps and provide remediation. They would spend money on programming. That’s what happens we care about something in school.

Beyond School

I’m not an economist (I’m probably as radically not-an-economist as a person can be), but I think this same kind of thinking has an effect on the world of business.

What do we measure to tell us whether a company is good and healthy? Shareholder value may not be the sole driver of executive decision-making, but, when challenged with any other metric, it’s hard for anything other than revenue and shareholder/owner value to win an argument. Why? Because we decided to make that the one metric that matters.

That was a human decision, and like any human decision, we could change it. Nothing is carved in stone. Corporations didn’t exist before people.

There have been efforts for years to encourage states to adopt a second set of metrics to evaluate corporate health alongside revenue—some kind of corporate citizenship framework that evaluated whether the company was a good steward of its human resources and whether it lived happily and healthily within its community—not polluting the local environment, participating in the arts and culture and life of the area in positive ways, and so on. There would be independent evaluations, and results would be published alongside revenue figures. I remember reading about these efforts decades ago.

And it wasn’t just a pipe dream. Many corporations do collect and publish this kind of information. As of 2021, 90% of the companies in the S&P 500 published some kind of corporate social responsibility (or CSR) report, up from 20% just eight years earlier.

Did you know that? I didn’t. Would you know where to go get that information? I wouldn’t—not right away. I’d figure it out, but it would take a little work. Whereas looking up a stock price takes no more than a click.

What if those two kinds of metrics sat side by side whenever you wanted to learn something about a company? Imagine if, in our regular, daily discourse about business, we looked at revenue and this other framework together, and gave both sets of metrics, if not equal weight, then at least some meaningful consideration. No one would be forced to care about one or the other more (or at all), but they would exist, side by side. By reporting both, we would show that both were important. How important? That would be up to the citizenry—and even the shareholders—to decide.

When I was a child, the Ford Motor Company decided not to pull a car out of production, even though they knew it was prone to potentially fatal accidents. They calculated that the cost of recall and retooling would far exceed the cost of payouts in lawsuits. On the plane of dollars, the decision was simple and clear-cut. And if that is the only plane of existence, their decision was the correct one.

“But they should have cared about human life!” you might say.

Really? Why? Because we want people in positions of power to be kind and caring people? Sure. I mean, of course. We do.

But we seem to expect that value entirely on the honor system, without ever actually demanding any honor. Do we measure care? Or do we just say things like, “people should care about their customers,” and leave it at that?

I hate to break it to you, but we don’t live on Planet Should.

Step Up and Be Counted

If we care about something, we should say so.

Wait, no, sorry—my bad: we must say so.

If we are about something, it’s our job to make it real in the world we want to live in. State it, measure it, publish it, and hold people to it.

I understand—not everything is easily quantifiable. Revenue numbers are easy to tally. Good corporate citizenship is harder. Multiple-choice questions are easy to score. Assessing deep understanding requires time and effort, and perhaps money. So we do what’s reasonable, and what’s affordable, and what doesn’t take too much work. I get that.

We’ve been doing things a certain way for so long that they seem inevitable; unchangeable. But that’s not true. Human things can be changed by humans any time they so desire…as long as they desire it enough. And folks…we get what we pay for.

Simple is not the only value. Avoiding time, effort, and expense is not the only good. When things matter, they’re worth some effort. We just have to decide that they do matter.

Share

Leave a comment

Friendly Reminder:

My new book, Box of Night, is available in paperback and Kindle eBook formats here. A little bit mystery, a little bit science-fiction, a little bit dystopian thought experiment. I hope you’ll give it a try.

If you do, and you enjoy it, let me know. And if you really like, it, please write a review at Amazon or Goodreads. Every little bit helps.