Amanda Ripley's Blog, page 9
September 17, 2012
Old-School Drama
Chicago’s children are trapped in a time warp. In the year 2012, in a city where only 6 out of 10 kids graduate from high school, in the richest country in the world, local leaders shut down the schools to fight about who dissed whom first. It’s a story so familiar worldwide that it feels almost Old Testament, and it’s one that always ends badly.
Here’s my take on the situation in the Wall Street Journal.
The bottom line: there are many countries that have revolutionized their education systems. But there is no precedent for any country doing so through strikes, walk-outs or righteous indignation. Not one.
I wish there were. I have no sympathy for politicians who do not understand politics; I have no patience for the obfuscations coming from union leaders and their defenders in Chicago. If breaking the union would serve the interests of Chicago’s 400,000 students, this problem would be easy to fix.
But alas. “One of the clearest lessons looking around the world is that you can’t build a great system with pissed-off teachers,” says Benjamin Levin, a former deputy minister for education from Ontario, a place that has learned that lesson the hard way. “You can’t bludgeon your way to greatness.”
What happened in Chicago is about more than just Chicago. It’s about education reformers trying to backwards engineer a professional workforce—which is like trying to turn a rowboat into a space ship—and union leaders fighting to get back to a placid sea that no longer exists.
Here’s my take on the situation in the Wall Street Journal.
The bottom line: there are many countries that have revolutionized their education systems. But there is no precedent for any country doing so through strikes, walk-outs or righteous indignation. Not one.
I wish there were. I have no sympathy for politicians who do not understand politics; I have no patience for the obfuscations coming from union leaders and their defenders in Chicago. If breaking the union would serve the interests of Chicago’s 400,000 students, this problem would be easy to fix.
But alas. “One of the clearest lessons looking around the world is that you can’t build a great system with pissed-off teachers,” says Benjamin Levin, a former deputy minister for education from Ontario, a place that has learned that lesson the hard way. “You can’t bludgeon your way to greatness.”
What happened in Chicago is about more than just Chicago. It’s about education reformers trying to backwards engineer a professional workforce—which is like trying to turn a rowboat into a space ship—and union leaders fighting to get back to a placid sea that no longer exists.
September 11, 2012
Making Sense of the Chicago Strike
Negotiations about teachers’ contracts happen in secret. That is madness, of course. The negotiators are making huge decisions about taxpayers’ money and kids’ lives.
Nevertheless, this shroud of secrecy helps explain why the Chicago dispute seems so confusing. Because no one can say exactly what it is about. So all kinds of misinformation floods the void, further entrenching both sides and sending reporters off in search of vapid-but-passionate quotes. (Like this one in today’s New York Times: “‘Clearly the teachers’ unions are under attack and under siege,’ said Diane Ravitch, an education historian at New York University who often defends the unions.”)
Here are 2 things we can say for sure:
1. Reporters are Getting an Important Fact Wrong
Like many teachers and parents and humans everywhere, most reporters do not understand how test scores would be used to evaluate teachers. They assume—understandably—that teachers will be judged based on how high their students score on tests. So reporters repeat claims like this one—from Chicago union president Karen Lewis:
“[U]nion president Karen Lewis, who has sharply criticized [Mayor] Emanuel, said the standardized tests do not take into account of the poverty in inner city Chicago as well as hunger and violence in the streets…..‘Evaluate us on what we do, not the lives of our children we do not control,’ Lewis said in announcing the strike.”
In fact, as you can see by reading this PDF from the web site of Chicago Public Schools, the proposed model was designed to “take into account” the poverty that Lewis mentions—and evaluate teachers on what they do, not on things they can’t control.
The algorithm compares how kids perform on a test—compared to their performance the year before. It looks at growth, not absolute scores. Then, to judge whether that growth was relatively strong or weak, the model controls for the effects of gender, race, ethnicity and poverty. So let’s say a 4th grade boy performs poorly on the end-of-year math test in Chicago. Now let’s imagine that he qualifies for free or reduced-price lunch, like most kids in Chicago. Well, his teacher could still be judged to be among the best in the city! How? Because the child’s score would be compared to his score from the previous spring—and compared to his expected score, given his demographic profile. If his score went up only a little bit, he could still have performed better than most other Chicago kids from a similar socio-economic background. See why this is complicated?
In fact, these models were designed to promote fairness and assuage union and teacher objections; but facts are secondary in education debates. (See next thing-we-know-for-sure…)
2. Nothing Good This Way Comes
There is no country in the world that has an outstanding education system and a toxic relationship with its union. There are many, many countries whose leaders have fought with their teachers’ unions, and many more who have rolled over for unions, agreeing to policies that doom children to years of boredom and incompetence. But the best results have come from places that have managed to raise the professionalism of teachers—while not going to war with them. How is this possible? Sometimes it takes a crisis; we have that in Chicago. Always, it requires a deft political hand; we don’t have that in Chicago, at least not that I can tell.
For a real-life example, consider Ontario, Canada.
Like the U.S., Canada is a decentralized system in which locals control the schools. Canadian schools are also very diverse, with a higher immigrant intake than that of the U.S. And until recently, Canadian schools were not getting great results for all kids. In Ontario in the 1990s, there were several teacher strikes—including a two week shut down in 1997. “Morale was extremely low and the relationship between the government and teachers was highly acrimonious,” as this OECD report describes in detail.
What happened? In 2003, a new leader came into power. Premier Dalton McGuinty appointed an education minister named Gerard Kennedy (a critic of the previous regime, it should be noted), and together they tried a radically new approach:
“We needed to create a new political consensus on education. the current level of politicisation of the system was taking a huge toll on public confidence. in the preceding eight years of conservative government hundreds and hundreds of hours of school had been lost to strikes and lockouts, and this level of disruption was at the core of public discontent with the system. We felt we had to change that dynamic if we were going to have any chance of successfully moving our reform agenda. We needed to re-establish trust between the government and the profession, and between school boards and teachers.”—Gerard Kennedy quoted in Lessons from PISA for the United States
Check out the report for the rest of the story, but the point is that no major improvement can occur in an atmosphere of distrust and animosity. I wish it could, but it can’t. To get past that kind of poison—while still standing up for kids’ interests—is the Holy Grail of education reform. Are the likes of Union president Karen Lewis and Chicago Mayor Rahm Emanuel capable of such transcendence? I’d say No—if I didn’t know that 350,000 kids were counting on it.
Nevertheless, this shroud of secrecy helps explain why the Chicago dispute seems so confusing. Because no one can say exactly what it is about. So all kinds of misinformation floods the void, further entrenching both sides and sending reporters off in search of vapid-but-passionate quotes. (Like this one in today’s New York Times: “‘Clearly the teachers’ unions are under attack and under siege,’ said Diane Ravitch, an education historian at New York University who often defends the unions.”)
Here are 2 things we can say for sure:
1. Reporters are Getting an Important Fact Wrong
Like many teachers and parents and humans everywhere, most reporters do not understand how test scores would be used to evaluate teachers. They assume—understandably—that teachers will be judged based on how high their students score on tests. So reporters repeat claims like this one—from Chicago union president Karen Lewis:
“[U]nion president Karen Lewis, who has sharply criticized [Mayor] Emanuel, said the standardized tests do not take into account of the poverty in inner city Chicago as well as hunger and violence in the streets…..‘Evaluate us on what we do, not the lives of our children we do not control,’ Lewis said in announcing the strike.”
In fact, as you can see by reading this PDF from the web site of Chicago Public Schools, the proposed model was designed to “take into account” the poverty that Lewis mentions—and evaluate teachers on what they do, not on things they can’t control.
The algorithm compares how kids perform on a test—compared to their performance the year before. It looks at growth, not absolute scores. Then, to judge whether that growth was relatively strong or weak, the model controls for the effects of gender, race, ethnicity and poverty. So let’s say a 4th grade boy performs poorly on the end-of-year math test in Chicago. Now let’s imagine that he qualifies for free or reduced-price lunch, like most kids in Chicago. Well, his teacher could still be judged to be among the best in the city! How? Because the child’s score would be compared to his score from the previous spring—and compared to his expected score, given his demographic profile. If his score went up only a little bit, he could still have performed better than most other Chicago kids from a similar socio-economic background. See why this is complicated?
In fact, these models were designed to promote fairness and assuage union and teacher objections; but facts are secondary in education debates. (See next thing-we-know-for-sure…)
2. Nothing Good This Way Comes
There is no country in the world that has an outstanding education system and a toxic relationship with its union. There are many, many countries whose leaders have fought with their teachers’ unions, and many more who have rolled over for unions, agreeing to policies that doom children to years of boredom and incompetence. But the best results have come from places that have managed to raise the professionalism of teachers—while not going to war with them. How is this possible? Sometimes it takes a crisis; we have that in Chicago. Always, it requires a deft political hand; we don’t have that in Chicago, at least not that I can tell.
For a real-life example, consider Ontario, Canada.
Like the U.S., Canada is a decentralized system in which locals control the schools. Canadian schools are also very diverse, with a higher immigrant intake than that of the U.S. And until recently, Canadian schools were not getting great results for all kids. In Ontario in the 1990s, there were several teacher strikes—including a two week shut down in 1997. “Morale was extremely low and the relationship between the government and teachers was highly acrimonious,” as this OECD report describes in detail.
What happened? In 2003, a new leader came into power. Premier Dalton McGuinty appointed an education minister named Gerard Kennedy (a critic of the previous regime, it should be noted), and together they tried a radically new approach:
“We needed to create a new political consensus on education. the current level of politicisation of the system was taking a huge toll on public confidence. in the preceding eight years of conservative government hundreds and hundreds of hours of school had been lost to strikes and lockouts, and this level of disruption was at the core of public discontent with the system. We felt we had to change that dynamic if we were going to have any chance of successfully moving our reform agenda. We needed to re-establish trust between the government and the profession, and between school boards and teachers.”—Gerard Kennedy quoted in Lessons from PISA for the United States
Check out the report for the rest of the story, but the point is that no major improvement can occur in an atmosphere of distrust and animosity. I wish it could, but it can’t. To get past that kind of poison—while still standing up for kids’ interests—is the Holy Grail of education reform. Are the likes of Union president Karen Lewis and Chicago Mayor Rahm Emanuel capable of such transcendence? I’d say No—if I didn’t know that 350,000 kids were counting on it.
September 5, 2012
7th in the World
The World Economic Forum released its annual rankings of global competitiveness today. The United States fell for the 4th year in a row, coming in 7th.
1. Switzerland
2. Singapore
3. Finland
4. Sweden
5. Netherlands
6. Germany
7. United States
Many things figure into a country’s economic competitiveness. And 7th is a perfectly respectable ranking (the real wailing should be reserved for Burundi, which comes in 144th out of 144 nations). But for any country, the trend is more important than the number, and the trend is not inspiring:
U.S. Competitiveness
2008: 1st
2009: 2nd
2010: 4th
2011: 5th
2012: 7th
Education matters more than it once did. It’s not a coincidence that the top 3 countries on this list also perform at the very top of the world on international math tests. Or as the report puts it, in report language:
“Workers who have received little formal education can carry out only simple manual work and find it much more difficult to adapt to more advanced production processes and techniques….Lack of basic education can therefore become a constraint on business development, with firms finding it difficult to move up the value chain by producing more sophisticated…products.”
Or as I might put it in simpler language: duh.
1. Switzerland
2. Singapore
3. Finland
4. Sweden
5. Netherlands
6. Germany
7. United States
Many things figure into a country’s economic competitiveness. And 7th is a perfectly respectable ranking (the real wailing should be reserved for Burundi, which comes in 144th out of 144 nations). But for any country, the trend is more important than the number, and the trend is not inspiring:
U.S. Competitiveness
2008: 1st
2009: 2nd
2010: 4th
2011: 5th
2012: 7th
Education matters more than it once did. It’s not a coincidence that the top 3 countries on this list also perform at the very top of the world on international math tests. Or as the report puts it, in report language:
“Workers who have received little formal education can carry out only simple manual work and find it much more difficult to adapt to more advanced production processes and techniques….Lack of basic education can therefore become a constraint on business development, with firms finding it difficult to move up the value chain by producing more sophisticated…products.”
Or as I might put it in simpler language: duh.
August 14, 2012
Surviving Disaster Movie
Many of you have asked me how you can get a copy of the PBS documentary, Surviving Disaster, based on The Unthinkable. I am sorry it has taken so long to become available to the public… Maybe they were trying to build suspense? I don’t know, but I finally have an answer for you!
To buy a copy of the film, please email Chris Tiano at Santa Fe Productions at . I’m told that if you are in North America and just want to watch it as an individual, then it will cost $24.95 plus $4.95 shipping & handling. If you are going to use it in a classroom, then it will cost $75 plus shipping for the public performance rights and so on and so forth.
Thank you to all who have expressed interest. I am insanely biased, but I loved the film. It captures the mystery and drama of human behavior in a disaster—and explains why our behavior matters—better than any TV show or film I’ve seen.
To buy a copy of the film, please email Chris Tiano at Santa Fe Productions at . I’m told that if you are in North America and just want to watch it as an individual, then it will cost $24.95 plus $4.95 shipping & handling. If you are going to use it in a classroom, then it will cost $75 plus shipping for the public performance rights and so on and so forth.
Thank you to all who have expressed interest. I am insanely biased, but I loved the film. It captures the mystery and drama of human behavior in a disaster—and explains why our behavior matters—better than any TV show or film I’ve seen.
August 13, 2012
Olympic Predictions
Economists, it turns out, can predict Olympic medal counts with uncanny accuracy. They don’t get the exact figures right, but they get weirdly close to predicting the ranking of countries worldwide—well before the Opening Ceremonies.
First let’s just acknowledge that there is something irritating about this; aren’t the Olympics supposed to be games of chance and romance? Economists do not belong in the Olympics. Is no place sacred?
And yet. In June, PricewaterhouseCoopers issued an economic briefing paper (pdf here) that modeled the performance of countries based on 4 critical characteristics:
1. Population
2. Income
3. Host country
4. Whether the country was once part of the Soviet bloc
I would have guessed the first two inputs, but the last two are more surprising—and help explain the outliers in the medal counts finalized yesterday with the closing of the London Olympics.
The model made some mistakes: the U.S. won 9 fewer medals than predicted, and Japan won 10 more. But the model did predict the ranking order, more or less, prophesying that the U.S. would come in first, followed by China, Russia and Great Britain—in that order. And that is exactly what happened.
The host country, as the model had foreshadowed, got a heady boost. Why? The economists say this is partly this is due to the greater investments that countries make in their Olympic teams when they know they are going to host. I suspect other more subtle forces are at work, too; there is a great psychological advantage to playing on the home court, after all. Imagine the thrill of having all those royals cheering for you—just a short distance from all those hooligans!
But interestingly, the home-court advantage varies depending on the country (China saw a huge bump when it hosted in 2008, winning 37 more medals than it had in Athens, but Greece only won 3 additional medals compared to its performance in Sydney).
The dominance of former Soviet bloc countries might be the most mysterious pattern of all. The economists explain this as a historical relic—a lingering Cold War respect for sports and Olympic triumph that motivates countries to invest and athletes to perform. They also see this as a victory for public policy, which is kind of adorable:
“This shows that sport is one area where state planning and intervention can produce results, which still persisted in Beijing almost two decades after the fall of the Berlin wall. However…these effects are gradually fading for the ex-Soviet countries, except for China which still has a strong state policy of promoting Olympic sport.”
Of course, try as I might, I can’t help but look at this ranking of medals and compare it to the ranking of education around the world. There is not a lot of overlap; the smartest countries in the world are not necessarily the most athletic and vice versa, as the U.S. has shown time and time again.
However, there are a few countries that perform exceptionally well—in mind and body. They are not the ones that spend the most per student (or per athlete). They are the ones that care a lot about both sports and learning, for whatever combination of reasons, and have the results to prove it:
Japan (No. 6 in the world in Olympic medals 2012; No. 6 in the world on PISA math 2009)
Australia (No. 7 in the world in medals; No. 12 in the world in math)
South Korea (No. 9 in the world in medals; No. 2 in math)
Netherlands (No. 11 in the world in medals; No. 8 in math)
Compare that to the U.S: No. 1 in the world in medals…and No. 26 in math. Something doesn’t add up; and all the king’s economists and all the king’s models can’t entirely explain it, try as they might.
First let’s just acknowledge that there is something irritating about this; aren’t the Olympics supposed to be games of chance and romance? Economists do not belong in the Olympics. Is no place sacred?
And yet. In June, PricewaterhouseCoopers issued an economic briefing paper (pdf here) that modeled the performance of countries based on 4 critical characteristics:
1. Population
2. Income
3. Host country
4. Whether the country was once part of the Soviet bloc
I would have guessed the first two inputs, but the last two are more surprising—and help explain the outliers in the medal counts finalized yesterday with the closing of the London Olympics.
The model made some mistakes: the U.S. won 9 fewer medals than predicted, and Japan won 10 more. But the model did predict the ranking order, more or less, prophesying that the U.S. would come in first, followed by China, Russia and Great Britain—in that order. And that is exactly what happened.
The host country, as the model had foreshadowed, got a heady boost. Why? The economists say this is partly this is due to the greater investments that countries make in their Olympic teams when they know they are going to host. I suspect other more subtle forces are at work, too; there is a great psychological advantage to playing on the home court, after all. Imagine the thrill of having all those royals cheering for you—just a short distance from all those hooligans!
But interestingly, the home-court advantage varies depending on the country (China saw a huge bump when it hosted in 2008, winning 37 more medals than it had in Athens, but Greece only won 3 additional medals compared to its performance in Sydney).
The dominance of former Soviet bloc countries might be the most mysterious pattern of all. The economists explain this as a historical relic—a lingering Cold War respect for sports and Olympic triumph that motivates countries to invest and athletes to perform. They also see this as a victory for public policy, which is kind of adorable:
“This shows that sport is one area where state planning and intervention can produce results, which still persisted in Beijing almost two decades after the fall of the Berlin wall. However…these effects are gradually fading for the ex-Soviet countries, except for China which still has a strong state policy of promoting Olympic sport.”
Of course, try as I might, I can’t help but look at this ranking of medals and compare it to the ranking of education around the world. There is not a lot of overlap; the smartest countries in the world are not necessarily the most athletic and vice versa, as the U.S. has shown time and time again.
However, there are a few countries that perform exceptionally well—in mind and body. They are not the ones that spend the most per student (or per athlete). They are the ones that care a lot about both sports and learning, for whatever combination of reasons, and have the results to prove it:
Japan (No. 6 in the world in Olympic medals 2012; No. 6 in the world on PISA math 2009)
Australia (No. 7 in the world in medals; No. 12 in the world in math)
South Korea (No. 9 in the world in medals; No. 2 in math)
Netherlands (No. 11 in the world in medals; No. 8 in math)
Compare that to the U.S: No. 1 in the world in medals…and No. 26 in math. Something doesn’t add up; and all the king’s economists and all the king’s models can’t entirely explain it, try as they might.
June 22, 2012
School-Life Balance
Just finished reading Anne-Marie Slaughter’s Atlantic essay on “Why Women Still Can’t Have It All.” I read it, I should say, from home, where I was waiting in a 90-degree room for the air-conditioning repair man to come after I’d dropped my child off at “camp”—since school is, for some reason, closed all summer long.
Having it ALL!
My one fix would have been to update the headline to read, “Why Men & Women Still Can’t Have It All.” Personally, I don’t know any men with young children who don’t experience the same wrenching pull, even if (as Slaughter notes) they may not feel the same level of shame and guilt when they go on business trips. (Besides, who’s to say men don’t feel more guilt than their wives when they fail to deliver at work? Guilt is a fungible commodity.)
Anyway, I applaud Slaughter for wading into this cauldron in an honest and nuanced way—and for proposing at least a couple of tangible solutions, instead of just listing laments, which is how these things normally go.
Here is my favorite tangible solution from the essay:
My longtime and invaluable assistant, who has a doctorate and juggles many balls as the mother of teenage twins, e-mailed me while I was working on this article: “You know what would help the vast majority of women with work/family balance? MAKE SCHOOL SCHEDULES MATCH WORK SCHEDULES.” The present system, she noted, is based on a society that no longer exists—one in which farming was a major occupation and stay-at-home moms were the norm. Yet the system hasn’t changed.
Want to reduce strain on families? Try, for one thing, running schools as if they operate in the real world—and as if what they do matters so much that it has to happen fairly consistently, just like work.
I just checked the calendar for the upcoming public school year in Washington, DC. Kids have 32 days off during the school year. Not counting the summer! That’s over 6 work weeks that parents somehow have to scramble to manage.
Most Americans get about half that in holiday and vacation time. The math is obvious: the tension parents feel isn’t just caused by employers not giving enough vacation time; it’s caused by education leaders who are willfully blind to the realities of families.
Every time my kid has a day off from school, I wish I had a magical Harry Potter map that could show the ripple effects being silently endured all across the city: the tense negotiations with spouses, the white lies to employers, the fingers crossed as mothers and fathers walk away from dropping off their children at dubious childcare centers where they don’t know anyone…. hoping for the best. All this for what? For “professional development” that most teachers will tell you is utterly useless.
Here’s an idea! Perhaps days off should be earned. If a given form of professional development is proven to actually make teaching more effective, then OK, it’s worth the time off. I’ll do anything to support that worthy goal.
Likewise with summer vacation: if, by the end of the year, students are all performing at levels required for young people to thrive in the modern economy, then great! Go home for the summer, kids and teachers, too. Break out the sprinkler. If not, we’ll see you back here on Monday. Which is to say, we’ll see every damn one of you here on Monday.
And now I’m off to the office. Air conditioner still broken. “Camp” ends at 3:30, at which time something called, for some reason, “aftercare” begins!
Having it ALL!
My one fix would have been to update the headline to read, “Why Men & Women Still Can’t Have It All.” Personally, I don’t know any men with young children who don’t experience the same wrenching pull, even if (as Slaughter notes) they may not feel the same level of shame and guilt when they go on business trips. (Besides, who’s to say men don’t feel more guilt than their wives when they fail to deliver at work? Guilt is a fungible commodity.)
Anyway, I applaud Slaughter for wading into this cauldron in an honest and nuanced way—and for proposing at least a couple of tangible solutions, instead of just listing laments, which is how these things normally go.
Here is my favorite tangible solution from the essay:
My longtime and invaluable assistant, who has a doctorate and juggles many balls as the mother of teenage twins, e-mailed me while I was working on this article: “You know what would help the vast majority of women with work/family balance? MAKE SCHOOL SCHEDULES MATCH WORK SCHEDULES.” The present system, she noted, is based on a society that no longer exists—one in which farming was a major occupation and stay-at-home moms were the norm. Yet the system hasn’t changed.
Want to reduce strain on families? Try, for one thing, running schools as if they operate in the real world—and as if what they do matters so much that it has to happen fairly consistently, just like work.
I just checked the calendar for the upcoming public school year in Washington, DC. Kids have 32 days off during the school year. Not counting the summer! That’s over 6 work weeks that parents somehow have to scramble to manage.
Most Americans get about half that in holiday and vacation time. The math is obvious: the tension parents feel isn’t just caused by employers not giving enough vacation time; it’s caused by education leaders who are willfully blind to the realities of families.
Every time my kid has a day off from school, I wish I had a magical Harry Potter map that could show the ripple effects being silently endured all across the city: the tense negotiations with spouses, the white lies to employers, the fingers crossed as mothers and fathers walk away from dropping off their children at dubious childcare centers where they don’t know anyone…. hoping for the best. All this for what? For “professional development” that most teachers will tell you is utterly useless.
Here’s an idea! Perhaps days off should be earned. If a given form of professional development is proven to actually make teaching more effective, then OK, it’s worth the time off. I’ll do anything to support that worthy goal.
Likewise with summer vacation: if, by the end of the year, students are all performing at levels required for young people to thrive in the modern economy, then great! Go home for the summer, kids and teachers, too. Break out the sprinkler. If not, we’ll see you back here on Monday. Which is to say, we’ll see every damn one of you here on Monday.
And now I’m off to the office. Air conditioner still broken. “Camp” ends at 3:30, at which time something called, for some reason, “aftercare” begins!
June 21, 2012
Value-Added Doctoring?
Medicare is starting to reimburse physicians based in part on the “quality” of their care. To incentivize better results, the theory goes, doctors whose patients’ health improved could get reimbursed at a higher rate
Ah, but how to measure quality fairly? What about all the things that doctors can’t control? Patients who are obese, patients who don’t even bother to fill their prescriptions… Surely a doctor can’t be blamed if these patients fail to thrive.
Sound familiar?
Mathematica released a new paper yesterday on whether docs could be evaluated based in part on value-added models similar to the ones designed for teachers (pdf here). Doctors everywhere shuddered, no doubt.
Here’s the part that makes theoretical sense: Doctors, like teachers, have not historically been paid based on how much they actually help their charges. In fact, doctors are often rewarded to do things that may be directly counter to patients’ interests (like administering tests they don’t need). It would make sense if doctors got a small bump if, for example, their diabetic patients’ glucose levels improved over time. That is a hard thing to do—and it should be rewarded.
The Mathematic paper suggests that Medicare could, for example, compare a patient’s glucose levels from one year to the next, and see how the change compared to the change in levels of similar patients in the same region. After that, of course, things get complicated—just as they do with teachers.
What about patients who have many doctors? What about doctors who see the same patients for many years?
The answer might be the same as the proposed fix for teachers, the paper suggests: Multiple measures!
In education, approaches to increasing the precision of performance estimates include using test scores from multiple years of classes (in other words, increasing the sample size for the estimate); combining value-added scores with other, independent measures of teacher performance, such as principals’ evaluations; and calculating scores at a higher level of aggregation (e.g., for all the teachers in a given subject or for all the teachers in a school, which, again, increases sample size). The following similar approaches could be taken in health: using multiple years of patient’s outcomes; combining value-added measures with other measures of physician performance, such as their scores on clinical process measures; and calculating value- added scores for groups of physicians in a practice.
Soon the model becomes very complex; and if most doctors do not buy in, and I suspect they will not, many will find a way to game the system. They will resent the intrusion. Some may drop low-income, low-performing patients. Others may indeed respond to the incentives—for ego or financial reasons—and actually improve their practice to lead to better outcomes.
Who knows? Maybe the value-added model could work even better with doctors than teachers. Unlike teachers, many doctors get into the profession partly to make money; they may, theoretically (again!), be more motivated by pay-for-performance schemes than teachers. They also have enough schooling that many of them may actually understand the models—unlike most people (including most teachers). And there may be more visibility into what they need to do in order to get better outcomes than there is for teachers.
Or not. So far, there’s not a lot of evidence that using value-added models to evaluate teaching actually improves outcomes for kids in the real world. There’s not a lot of evidence that it doesn’t. Time may tell.
Regardless, thinking about this analogy made me realize that teachers have way more time to try to influence students than doctors have to affect patients. On the high end, let’s say a doctor sees a patient every two months for 15 minutes. Teachers, meanwhile, spend about 5 to 25 hours a week with their students. Over the course of two months, then, teachers have somewhere between 16,000% to 80,000% more time with students than doctors have with patients. (Granted, teachers don’t see their students one-on-one for all that time, but you get the idea: teachers spend a LOT of time with their students.) Opportunity lurks in that time….
Time is something that adults underestimate, I find. Perhaps we think it moves more swiftly because we have less of it left. Kids on the other hand know what it feels like to sit in the same room for 5 hours for 185 days. Much can happen in that time, they will tell you. (Or not.)
Ah, but how to measure quality fairly? What about all the things that doctors can’t control? Patients who are obese, patients who don’t even bother to fill their prescriptions… Surely a doctor can’t be blamed if these patients fail to thrive.
Sound familiar?
Mathematica released a new paper yesterday on whether docs could be evaluated based in part on value-added models similar to the ones designed for teachers (pdf here). Doctors everywhere shuddered, no doubt.
Here’s the part that makes theoretical sense: Doctors, like teachers, have not historically been paid based on how much they actually help their charges. In fact, doctors are often rewarded to do things that may be directly counter to patients’ interests (like administering tests they don’t need). It would make sense if doctors got a small bump if, for example, their diabetic patients’ glucose levels improved over time. That is a hard thing to do—and it should be rewarded.
The Mathematic paper suggests that Medicare could, for example, compare a patient’s glucose levels from one year to the next, and see how the change compared to the change in levels of similar patients in the same region. After that, of course, things get complicated—just as they do with teachers.
What about patients who have many doctors? What about doctors who see the same patients for many years?
The answer might be the same as the proposed fix for teachers, the paper suggests: Multiple measures!
In education, approaches to increasing the precision of performance estimates include using test scores from multiple years of classes (in other words, increasing the sample size for the estimate); combining value-added scores with other, independent measures of teacher performance, such as principals’ evaluations; and calculating scores at a higher level of aggregation (e.g., for all the teachers in a given subject or for all the teachers in a school, which, again, increases sample size). The following similar approaches could be taken in health: using multiple years of patient’s outcomes; combining value-added measures with other measures of physician performance, such as their scores on clinical process measures; and calculating value- added scores for groups of physicians in a practice.
Soon the model becomes very complex; and if most doctors do not buy in, and I suspect they will not, many will find a way to game the system. They will resent the intrusion. Some may drop low-income, low-performing patients. Others may indeed respond to the incentives—for ego or financial reasons—and actually improve their practice to lead to better outcomes.
Who knows? Maybe the value-added model could work even better with doctors than teachers. Unlike teachers, many doctors get into the profession partly to make money; they may, theoretically (again!), be more motivated by pay-for-performance schemes than teachers. They also have enough schooling that many of them may actually understand the models—unlike most people (including most teachers). And there may be more visibility into what they need to do in order to get better outcomes than there is for teachers.
Or not. So far, there’s not a lot of evidence that using value-added models to evaluate teaching actually improves outcomes for kids in the real world. There’s not a lot of evidence that it doesn’t. Time may tell.
Regardless, thinking about this analogy made me realize that teachers have way more time to try to influence students than doctors have to affect patients. On the high end, let’s say a doctor sees a patient every two months for 15 minutes. Teachers, meanwhile, spend about 5 to 25 hours a week with their students. Over the course of two months, then, teachers have somewhere between 16,000% to 80,000% more time with students than doctors have with patients. (Granted, teachers don’t see their students one-on-one for all that time, but you get the idea: teachers spend a LOT of time with their students.) Opportunity lurks in that time….
Time is something that adults underestimate, I find. Perhaps we think it moves more swiftly because we have less of it left. Kids on the other hand know what it feels like to sit in the same room for 5 hours for 185 days. Much can happen in that time, they will tell you. (Or not.)
May 11, 2012
Call for Nominations: The Rick Rescorla Award
This year, for the first time, the U.S. Department of Homeland Security will recognize a regular, non-governmental human (or organization) for acts of superior leadership and innovation—through a new honor called the Rick Rescorla National Award for Resilience.
This is a big deal. For years, schmucks like me have been haranguing the federal government for failing to highlight the stories and wisdom of the regular people who make our country more resilient. Instead of talking about how government is going to make us safe, we ought to start listening—to the t-shirt vendors, the flight attendants, the survivors and the guy in the aisle seat, to the Rick Rescorlas of the world who have shown us how the public can prevent and respond to disasters with grace, courage and initiative.
Well, now DHS is doing it, in at least one symbolic and important way. Please send your nominations asap to rescorlaaward@hq.dhs.gov. More details and the nomination form can be found here. The deadline is June 1, 2012.
The award was named after Rick Rescorla, the head of security for Morgan Stanley Dean Witter in the World Trade Center. I wrote about Rescorla in The Unthinkable, and I’ve talked about him around the country. His story is impossible to forget once you’ve heard it. So let me share some of it here, now that we have a good excuse…
Rick Rescorla was one of those thick-necked, former soldier types who spent the second halves of their lives patrolling the perimeters of marble lobbies the way they once patrolled a battlefield. He was disciplined in everything he did, and he understood the power of the human brain to get better through practice.
After the 1993 bombing and the fiasco of an evacuation that followed, Rescorla decided that Morgan Stanley employees had to take full responsibility for their own survival— something that happened almost nowhere else in the Trade Center. He knew it was foolish to rely on first responders to save his employees. His company was the largest tenant in the World Trade Center, a village nestled in the clouds. Morgan Stanley’s employees would need to take care of one another.
From then on, Rescorla started running the entire company through frequent, surprise fire drills. He trained employees to meet in the hallway between the stairwells and, at his direction, go down the stairs, two by two, to the forty-fourth floor. He noticed they moved slowly, so he started timing them with a stopwatch—and they got faster.
The radicalism of Rescorla’s drills cannot be overstated. Remember, Morgan Stanley was an investment bank. Millionaire, high-performance bankers on the 73rd floor chafed at Rescorla’s evacuation regimen. They did not appreciate interrupting high-net-worth clients in the middle of a meeting. Each drill, which pulled the firm’s brokers off their phones and away from their computers, cost the company money. But Rescorla did it anyway. He didn’t care whether he was popular.
When guests visited Morgan Stanley for training, Rescorla made sure they all knew how to get out too. Even though the chances were slim, Rescorla wanted them ready for an evacuation.
On the morning of 9/11, Rescorla heard an explosion and saw Tower 1 burning from his office window. A Port Authority official came over the public address system and urged everyone to remain at their desks. But Rescorla grabbed his bullhorn, his walkie-talkie, and his cell phone and began systematically ordering Morgan Stanley employees to get out. They already knew what to do, even the 250 visitors who were taking a stockbroker training class and had already been shown the nearest stairway.
Rescorla had led soldiers through the Vietcong-controlled Central Highlands of Vietnam. He knew the brain responded poorly to extreme fear. Back then, he had calmed his men by singing Cornish songs from his youth. Now, in the crowded stairwell, as his sweat leached through his suit jacket, Rescorla began to sing into the bullhorn. “Men of Cornwall stand ye steady; It cannot be ever said ye for the battle were not ready; Stand and never yield!”
Moments later, Rescorla had successfully evacuated the vast majority of Morgan Stanley employees out of the burning tower. Then he turned around. He was last seen on the 10th floor, heading upward, shortly before the tower collapsed. His remains have never been found.
Rescorla taught Morgan Stanley employees to save themselves. It’s a lesson that had become, somehow, rare and precious. When the tower collapsed, only 13 Morgan Stanley colleagues—including Rescorla and four of his security officers—were inside. The other 2,687 were safe.
This is a big deal. For years, schmucks like me have been haranguing the federal government for failing to highlight the stories and wisdom of the regular people who make our country more resilient. Instead of talking about how government is going to make us safe, we ought to start listening—to the t-shirt vendors, the flight attendants, the survivors and the guy in the aisle seat, to the Rick Rescorlas of the world who have shown us how the public can prevent and respond to disasters with grace, courage and initiative.
Well, now DHS is doing it, in at least one symbolic and important way. Please send your nominations asap to rescorlaaward@hq.dhs.gov. More details and the nomination form can be found here. The deadline is June 1, 2012.
The award was named after Rick Rescorla, the head of security for Morgan Stanley Dean Witter in the World Trade Center. I wrote about Rescorla in The Unthinkable, and I’ve talked about him around the country. His story is impossible to forget once you’ve heard it. So let me share some of it here, now that we have a good excuse…
Rick Rescorla was one of those thick-necked, former soldier types who spent the second halves of their lives patrolling the perimeters of marble lobbies the way they once patrolled a battlefield. He was disciplined in everything he did, and he understood the power of the human brain to get better through practice.
After the 1993 bombing and the fiasco of an evacuation that followed, Rescorla decided that Morgan Stanley employees had to take full responsibility for their own survival— something that happened almost nowhere else in the Trade Center. He knew it was foolish to rely on first responders to save his employees. His company was the largest tenant in the World Trade Center, a village nestled in the clouds. Morgan Stanley’s employees would need to take care of one another.
From then on, Rescorla started running the entire company through frequent, surprise fire drills. He trained employees to meet in the hallway between the stairwells and, at his direction, go down the stairs, two by two, to the forty-fourth floor. He noticed they moved slowly, so he started timing them with a stopwatch—and they got faster.
The radicalism of Rescorla’s drills cannot be overstated. Remember, Morgan Stanley was an investment bank. Millionaire, high-performance bankers on the 73rd floor chafed at Rescorla’s evacuation regimen. They did not appreciate interrupting high-net-worth clients in the middle of a meeting. Each drill, which pulled the firm’s brokers off their phones and away from their computers, cost the company money. But Rescorla did it anyway. He didn’t care whether he was popular.
When guests visited Morgan Stanley for training, Rescorla made sure they all knew how to get out too. Even though the chances were slim, Rescorla wanted them ready for an evacuation.
On the morning of 9/11, Rescorla heard an explosion and saw Tower 1 burning from his office window. A Port Authority official came over the public address system and urged everyone to remain at their desks. But Rescorla grabbed his bullhorn, his walkie-talkie, and his cell phone and began systematically ordering Morgan Stanley employees to get out. They already knew what to do, even the 250 visitors who were taking a stockbroker training class and had already been shown the nearest stairway.
Rescorla had led soldiers through the Vietcong-controlled Central Highlands of Vietnam. He knew the brain responded poorly to extreme fear. Back then, he had calmed his men by singing Cornish songs from his youth. Now, in the crowded stairwell, as his sweat leached through his suit jacket, Rescorla began to sing into the bullhorn. “Men of Cornwall stand ye steady; It cannot be ever said ye for the battle were not ready; Stand and never yield!”
Moments later, Rescorla had successfully evacuated the vast majority of Morgan Stanley employees out of the burning tower. Then he turned around. He was last seen on the 10th floor, heading upward, shortly before the tower collapsed. His remains have never been found.
Rescorla taught Morgan Stanley employees to save themselves. It’s a lesson that had become, somehow, rare and precious. When the tower collapsed, only 13 Morgan Stanley colleagues—including Rescorla and four of his security officers—were inside. The other 2,687 were safe.
April 19, 2012
Where Does the $ Go?
Thanks to the folks at USC’s Master of Arts in Teaching Program for this nice graphic on $ and education around the world.
Via: MAT@USC | Master’s of Arts in Teaching
But this raises another mystery: We’ve known for a long time that more money does not tend to lead to more learning, once you get past a bare minimum (which we did a long time ago). So here’s my question: Where does all that money go in the U.S.??
Why do we spend so much more? Has anyone seen a good answer to this? I’d love to see what percentage of our spending goes to things that other countries’ education budgets don’t have to cover (i.e. health care for teachers). One report (PDF here) states that this difference alone could account for up to 8% of the variation between our expenses and those of other nations. Well, if that’s true, that’s not actually very much.
Has anyone tried to compare countries’ spending while controlling for differences in how non-salaried benefits get distributed from place to place? Also, I’d love to see what percentage of our spending goes to technology compared to the spending in other countries… Anyone ever seen anything that reveals the story behind the money? I may be missing something, but I can’t seem to find any really strong analysis of the money story—even though we are talking about huge sums of money…
Via: MAT@USC | Master’s of Arts in Teaching
But this raises another mystery: We’ve known for a long time that more money does not tend to lead to more learning, once you get past a bare minimum (which we did a long time ago). So here’s my question: Where does all that money go in the U.S.??
Why do we spend so much more? Has anyone seen a good answer to this? I’d love to see what percentage of our spending goes to things that other countries’ education budgets don’t have to cover (i.e. health care for teachers). One report (PDF here) states that this difference alone could account for up to 8% of the variation between our expenses and those of other nations. Well, if that’s true, that’s not actually very much.
Has anyone tried to compare countries’ spending while controlling for differences in how non-salaried benefits get distributed from place to place? Also, I’d love to see what percentage of our spending goes to technology compared to the spending in other countries… Anyone ever seen anything that reveals the story behind the money? I may be missing something, but I can’t seem to find any really strong analysis of the money story—even though we are talking about huge sums of money…
April 18, 2012
Women & Children First?
On the Titanic, 70% of the women and children survived—but only 20% of the men. Cue the orchestra!
But was the Titanic the exception? A new study investigates whether women and children really do have an advantage on a sinking ship.
It is so refreshing, first of all, to see a study focus obsessively on the thing that matters most in a disaster—the behavior of the humans involved. Naturally, the results show that life is more complicated than the movies.
The study, out of Sweden, concludes that it is in fact worse to be a woman on a shipwreck, based on a study of 18 maritime disasters involving over 15,000 people. The survival rate of women was 27% vs. 37% for men (see Table C1). But children have the lowest survival rate of all at 15%. And crew members have the highest rate of anyone at 61%!
The authors have some compelling data, but their conclusion jumps the shark:
Taken together, our findings show that behavior in life-and-death situation is best captured by the expression ‘Every man for himself’.
Um, really? I look at the same set of facts and make a very different conclusion.
The most important detail in the study is actually the crew survival rate. To me, these figures show that the most valuable asset in a disaster is not gender; it’s experience.
The crew members knew where the life boats were. They knew how to operate them. And they knew how to swim.
They weren’t afraid to take action; they weren’t waiting for instructions; they weren’t down below trying to save the children (a likely explanation for the death of at least some of the female passengers.)
This doesn’t mean that crew members are all cowards who flee in the life boats while passengers die. That may happen sometimes, but the opposite also happens. Crew members, given their roles, may go to extreme lengths to help rescue passengers. And who knows? The passenger survival rate might be even worse if crew members did not have this inclination.
Personally, I think the question of chivalry on a sinking ship is less interesting. There are too many compounding factors in a real disaster to be able to isolate whether people were being gender neutral or not. (Indeed, even more women might have died if the women-and-children-first slogan had never existed. Who knows?)
Anyway, the good news here is that knowledge matters. Under strain, the brain reverts to what it knows best. If you’ve got muscle memory for getting into a life boat, you’ll be better off than someone who doesn’t. This kind of study should encourage cruise ship safety directors (not to mention building and airplane personnel) to give people physical experience trying on life jackets and releasing life boats. These are not onerous tasks; you do them with crew members all the time. Now do them with the rest of us.
Thanks to Freakonomics and @DaniloBalu for noticing the study!
But was the Titanic the exception? A new study investigates whether women and children really do have an advantage on a sinking ship.
It is so refreshing, first of all, to see a study focus obsessively on the thing that matters most in a disaster—the behavior of the humans involved. Naturally, the results show that life is more complicated than the movies.
The study, out of Sweden, concludes that it is in fact worse to be a woman on a shipwreck, based on a study of 18 maritime disasters involving over 15,000 people. The survival rate of women was 27% vs. 37% for men (see Table C1). But children have the lowest survival rate of all at 15%. And crew members have the highest rate of anyone at 61%!
The authors have some compelling data, but their conclusion jumps the shark:
Taken together, our findings show that behavior in life-and-death situation is best captured by the expression ‘Every man for himself’.
Um, really? I look at the same set of facts and make a very different conclusion.
The most important detail in the study is actually the crew survival rate. To me, these figures show that the most valuable asset in a disaster is not gender; it’s experience.
The crew members knew where the life boats were. They knew how to operate them. And they knew how to swim.
They weren’t afraid to take action; they weren’t waiting for instructions; they weren’t down below trying to save the children (a likely explanation for the death of at least some of the female passengers.)
This doesn’t mean that crew members are all cowards who flee in the life boats while passengers die. That may happen sometimes, but the opposite also happens. Crew members, given their roles, may go to extreme lengths to help rescue passengers. And who knows? The passenger survival rate might be even worse if crew members did not have this inclination.
Personally, I think the question of chivalry on a sinking ship is less interesting. There are too many compounding factors in a real disaster to be able to isolate whether people were being gender neutral or not. (Indeed, even more women might have died if the women-and-children-first slogan had never existed. Who knows?)
Anyway, the good news here is that knowledge matters. Under strain, the brain reverts to what it knows best. If you’ve got muscle memory for getting into a life boat, you’ll be better off than someone who doesn’t. This kind of study should encourage cruise ship safety directors (not to mention building and airplane personnel) to give people physical experience trying on life jackets and releasing life boats. These are not onerous tasks; you do them with crew members all the time. Now do them with the rest of us.
Thanks to Freakonomics and @DaniloBalu for noticing the study!
