Amanda Ripley's Blog, page 11

December 22, 2011

Calling All Data Nerds…

A few people have asked me to explain in more detail why I think the PISA index of socioeconomic status is a better way to compare the performance of rich and poor kids around the world (versus the breakdown of scores based on how many kids qualify for free or reduced price lunch at a US school). So I'll do my best for those of you looking to get deep in the weeds on this….

OK, first let's talk about the PISA index on socioeconomic status. The data for that index is indeed self-reported by the students taking the test, as some of the commenters have noted. I can see why people would wonder if that is reliable. In fact, I had the same question when I first heard about this.

Two things:

First, the research suggests that students are surprisingly accurate when asked specific questions about their family's situation (for example, see this report on students' reliability on such questionnaires.)

Secondly, the students are not asked to give their parents' income per se; they are asked a long list of questions about their parents' education levels, occupations, the number of books and computers in the home, etc.—all things that give a holistic sense of SES (and some of which, including education level, can better predict educational success than income alone).

Alright, as for the Free/Reduce Price Lunch (FRPL) breakdown of the PISA data referenced by people who insist our low-poverty schools are "No.1" in the world: this data comes from a totally different survey done in the U.S. only. Principals at U.S. schools where some number of students took the PISA were asked this question. They were told to respond in reference to the entire school—not just the students who took the test. So this is already a different unit of measurement than the average PISA scores for, say, Finnish students.

Moreover, the number of principals who said that between 0 and 10% of their students are eligible for FRPL is small; only about 10% of the 2009 U.S. PISA sample attended these schools.

But that's all well and good. This FRPL data surely gives us a sense of the huge gap between the performance of the 10 percenters and the rest of the schools in the U.S.

But those last three words are key. This data is collected only to look at variance in the U.S. I agree that it would be fascinating to compare these figures to the same figures in Finland and around the world. However, we don't have that information. We don't know how Finnish schools with 0-10% of students from families earning less than 185% of the U.S. poverty level do on PISA.

We do know that Finland overall has far less poverty than the U.S. But the oft-cited figure—that Finland has about 4% child poverty—refers to a totally different definition of "poverty" than the FRPL definition. That 4% figure refers to the percent of people who earn less than 50% of the median income in Finland. (The comparable figure for poverty in the U.S. is about 20%—whereas under the FRPL definition of "poverty," it's about 40%, to give you a sense of the difference.)

Just to be sure, I spoke to the data experts who crunch this FRPL data in the U.S. and know it far better than I ever will, and they confirmed that it is inappropriate to use this data in the way that Ravitch is using it. You can't compare the FRPL data from US schools to an entire country; it's apples to oranges. The best option that I know of to compare apples to apples is PISA's own ESCS index. And again, on that index, our richest kids do fine in reading—and not well in math and science.

OK, now back to writing the book! If you've read this far, you are probably trying to procrastinate doing something, too… Thanks for the company!
 •  0 comments  •  flag
Share on Twitter
Published on December 22, 2011 09:50

December 14, 2011

Why Do Our Rich Kids Rank 23rd in Math…?

The other day, I posted the country rankings you never hear about—the only legitimate ones to show how countries' most privileged 15-year-olds do on the PISA test of what kids know around the world.*

Our richest kids rank No. 7 in reading. OK, so it is not No.1, as others keep insisting, and we spend way more money per student to get there. But I'll take it. No. 7 is still a perfectly respectable performance—well above the OECD average for rich kids.

But it got me thinking: What about math and science? How did our most privileged kids (who are, by the way, more privileged than most countries' well-off children) do in math and science?

Oh Lord…Brace yourselves, suburban parents:

With thanks to the folks at the Education Trust who helped me ferret out this data from the PISA results, here we go:

MATH ACHIEVEMENT of the most privileged teenagers around the world:

1. Belgium

2. Netherlands

3. South Korea

4. Finland

5. New Zealand

6. Japan

7. Switzerland

8. Czech Republic

9. Canada

10. Australia

11. Germany

(Still going…)

12. Denmark

13. France

14. Sweden

15. Austria

16. Hungary

17. Slovak Republic (!)

18. Iceland

(Hang in there…)

19. Luxembourg

20. Ireland

21. Norway

22. Poland

23. UNITED STATES

There it is, No. 23 out of 29 countries in math, according to the 2003 PISA exam (which was the last time math was the primary focus of the test, yielding enough data to make such comparisons).

Wow. How to explain this? Our most privileged kids attend, on average, the most well-resourced schools in the world with some of the smallest class sizes and among the most credentialed, experienced, well-paid teachers. They have educated parents, books at home and computers to use, and this sample includes our private-school students.

And yet they score below the OECD average in math when compared to other countries most-privileged students. What is going on here?

SCIENCE ACHIEVEMENT of the most privileged teenagers around the world:

1. Finland

2. New Zealand

3. Netherlands

4. Canada

5. Australia

6. Germany

7. United Kingdom

8. Czech Republic

9. Belgium

10. Switzerland

11. Japan

12. France

13. Austria

14. Hungary

15. Ireland

16. Sweden

17. South Korea

18. UNITED STATES

In science, our most privileged students ring in 18th out of 30 countries, per the 2006 PISA test (the last one that had science as its primary focus.) This is, as in math, just below the OECD average for similarly affluent kids.

Why does it matter?

I bring this up just to point out that it is possible for kids to learn at much higher levels than our kids are learning—even our most-advantaged kids. I am not (repeat, not!) saying that poverty doesn't matter; it obviously matters enormously. Let's just stop talking about poverty as if it is some dark force that acts in isolation from the rest of our institutions.

Even if we could magically eliminate poverty in America (which would be a beautiful thing and something we should try much harder to do), then we still would not have world-class education outcomes.

Anyone care to offer a theory for why our most affluent kids score 23rd in math and 18th in science? Is it a lack of motivation? An overabundance of wealth? If so, why aren't we below average in reading, too?

*And remember, before you send me links to wildly misleading blog posts and demand a recount: these rankings listed here rely upon PISA's own carefully administered survey of students' socioeconomic status known as the index of Economic, Social and Cultural Status—not a hijacked table regarding free-or-reduced price lunch ratios that was never ever intended to be used for international comparisons.
 •  0 comments  •  flag
Share on Twitter
Published on December 14, 2011 09:14

December 12, 2011

Reality Distortion Field

This is the story of how wishes come true in the strange, upside-down world of education.

Edu-pundits like Diane Ravitch like to say that America's education problems have everything to do with poverty. This is actually a debate that goes back centuries in American schools. It takes different forms at different times, but it almost always follows the same equation: poverty (or race) is a problem so intractable that schools cannot be expected to overcome it. (Fun fact: the same debate was used to defend low-performing, segregated public schools in New York City in the 1960s. Check out this New York Times story from 1963.)

Interestingly, this is not the kind of talk you hear in places with higher-performing education systems. In those countries, the very same countries that Ravitch says should be models for US schools, educators also think poverty is a big problem. But they think it is their problem. They think it is a problem so intractable that our schools must be outstanding in order to help overcome it. See the difference?

Of course, if we think about it calmly for more than 5 minutes, we can probably agree that poverty interacts with schools, like a chemistry experiment. Bad schools make poverty worse, and great schools make it possible to overcome poverty. In fact, great schools are among the most effective anti-poverty measures known to humanity. Neither schools nor poverty work in isolation.

And yet this debate rages on, with a stunning lack of sophistication. To show you what I mean, let's consider the latest talking point.

Ravitch and others have been saying over and over again that America's low-poverty (i.e. affluent) schools do even better than Finland. "Low poverty schools, low poverty districts in the US perform just as well in the US as schools with similar demographics in the top nations in the world. They're number 1. In fact, our children are number 1 in the low poverty districts."

So if we took away our pesky poverty problem, we'd rank at the top of the world! This point is meant to defend all schools, but mostly it makes upper-income parents feel better about their own kids' schools.

Too bad it's not true.

The most respected international tests of teenagers around the world (PISA) has consistently shown that our most-affluent kids do not perform as well as the most-affluent kids in the highest-performing countries around the world (even though our rich kids are richer than their rich kids). PISA measures students' economic, social and cultural status to get a sense of their socio-economic background. In reading, American kids' best subject, our most affluent students still rank behind the most affluent kids in six other countries. (Even though we spend far more money per student than all of those countries.)

Rich Kids Ranking (PISA Reading 2009)

1. New Zealand

2. Korea

3. Belgium

4. Finland

5. Canada

6.  Australia

So where is Ravitch coming from? She is, after all, a professor at New York University. Surely she can't just make these things up, right?

Here's what's happening. Bear with me, because it is revealing.

Ravitch's claim can be traced back to a small table on page 15 of a government report that broke down the PISA results based on the percentage of kids who qualify for free or reduced-price lunch. When you do that, you see that kids at U.S. schools where less than 10% of the students qualify for free/reduced-price lunch score on average very high—indeed higher than the average for, say, Finland.

But then she makes the magical leap. She says that since Finland has less than 10% poverty, and those schools do, too, then…ta da! Our low-poverty schools are best in the world—when you compare rich kids to rich kids.

Here's the problem: she is using 2 different definitions of "poverty."

The free/reduced-price lunch figure measures the number of kids from families making 185% of federal poverty line, right? So that means a family of 4 needs to make less than about $40,000 to qualify. Under this measure, roughly 40% of American kids qualify as "poor."

OK. Then the other measure is the measure usually used in international comparisons of poverty. That is the percentage of kids from families earning less than 50% of the median income in that country. (In the US, this comes out to about 22%. NOT 40%.)

In other words, Ravitch is comparing the test scores of kids from families that earn more than $40,000 in the U.S. to the scores of all kids in Finland (where the median household income is about $40,000).

I don't have tenure, but even I know you can't mix and match data like this. Unless you are really, really desperate to find a certain answer, that is. [Edit: After this post went up, an alert reader informed me that Ravitch does not have tenure either; NYU confirms that she is a nontenured "research scientist."]

Conversely, PISA's own measure of socio-economic background, the one you can find detailed in Table II.3.1 in PISA Volume II, offers a more valid comparison. And yet Ravitch does not cite it—because it does not show what she wants it to show.

I have been to Finland, Korea and Poland working on this book, and I have the luxury of spending hours reading PISA results. Most writers do not. They just repeat what Ravitch and others say. And so the magical thinking continues.

On Friday, David Sirota repeated this myth in Salon. And it ran again yesterday in the Oregonian.

As 2011 draws to a close, we can confidently declare that one of the biggest debates over education is — mercifully — resolved. We may not have addressed all the huge challenges facing our schools, but we finally have empirical data ruling out apocryphal theories and exposing the fundamental problems.

We've learned, for instance, that our entire education system is not "in crisis," as so many executives in the for-profit education industry insist when pushing to privatize public schools. On the contrary, results from Program for International Student Assessment exams show that American students in low-poverty schools are among the highest achieving students in the world.

What interests me is not so much that fiction gets reported as fact. That is an old story. What interests me is why so many people—particularly liberals—seem to want to believe that poverty is even more intractable than it is…  Why would this be?
 •  0 comments  •  flag
Share on Twitter
Published on December 12, 2011 04:52

November 12, 2011

Playing the Odds in Vegas

I'll be in Vegas this week giving a talk to the International Association of Emergency Managers. Incredibly, I've never been to Vegas before. I know, I know. I'm an embarrassment to my country. So it's time, and I'm looking forward to catching up with old friends and sources. Plus, what better place to talk about risk, denial and panic?
 •  0 comments  •  flag
Share on Twitter
Published on November 12, 2011 12:47

October 21, 2011

A Round-the-World Guide to Kids, Tests & Scapegoats

Here's a little screed I wrote about testing in Finland and other countries for NBC's Education Nation blog, the Learning Curve. As Congress debates a rework of No Child Left Behind and our own culture of testing, it's worth considering what really matters.

Before Jesus Christ was born, human beings were taking tests. Civil service exams date back to China's Han Dynasty (206 BC – 220 AD.) Hiring test-prep tutors - and cheating - go back about as far, by the way.

U.S. students now take more standardized tests than ever. Under the federal No Child Left Behind law, our kids get tested in grades three through eight, and at least once between tenth and twelfth grade.

Have we lost our minds? Many teachers and critics of school reform insist that we have, citing other, higher-performing nations as evidence of our relative insanity…

First of all, let's be clear: Finland does have standardized testing. They have had it for at least 159 years. They have less of it, for sure. (Which is not to suggest that they have less testing overall, but more on that later.) In fact, in every high-performing nation, tests are embedded in the wiring of schools - particularly in high schools. In the developed world, 76 percent of students attend high schools that use standardized tests, according to the OECD.

Read more here.
 •  0 comments  •  flag
Share on Twitter
Published on October 21, 2011 07:15

October 18, 2011

Is Occupy Wall Street Really about Education Reform?

The following is a dispatch written by guest blogger Marie Lawrence, a researcher at the New America Foundation. As a recent college graduate watching the Wall Street protests, she saw a connection that I had not considered. Here is her take:

A few days before my 6th-grade graduation in Richardson, Texas, my teacher asked us to write poems about the jobs we hoped to have in 10 years. In clumsy rhyme and loopy cursive, we proclaimed our intentions to become singers, pilots, doctors, race car drivers and pastry chefs. With the audacity of youth, I predicted my own success as an author, lawyer or architect. (I was keeping my options open.)

Mrs. Babb affixed a gold star to each page and lovingly pinned them to the bulletin board, silently affirming that yes, these jobs are waiting for you if you work hard. Not a single child prophesied his future as a barista, a telemarketer or a perpetual job-seeker.
Since then, I have graduated from college and been fortunate to find a job that allows me to use my brain and pay the bills. But some of my highest-achieving friends are still grasping for the very bottom rung of the career ladder.

We know that the Occupy Wall Street protest is partly a response to corporate greed, but I suspect it also reflects the disconnect between our aspirations and our reality. It feels like the engines of social mobility (namely education) are failing us. After talking with the protesters in Zuccotti Park, the Washington Post's Alexandra Petri described the sentiment this way:

"Growing up, we were told: You are special. You are brilliant. Go to school, get a degree, pursue what you love. Four years later, we are mired in debt. Jobless, with no prospects. This is not what it said on the motivational poster."

It's as if we are catching up to the data, which has for years shown a mismatch between our academic performance and our occupational aspirations. In its 2007 report Child Poverty in Perspective, UNICEF evaluated countries' performance along 40 indicators of child well-being, six of which measured educational well-being. Among 25 "economically advanced" nations, the U.S. ranked 21st in educational achievement of 15 year-olds in reading, math and science. The U.S. also had higher drop-out rates than similarly prosperous countries. Of the 23 countries ranked, the United States ranked 21st in "percentage of 15-19 year-olds in full-time or part-time education." In fact, the United States ranked second-to-last (20th of 21 countries) in child well-being overall.

But at the same time, U.S. kids trounced all others when it came to optimism about their careers. Just 14% of 15 year-olds surveyed said they expected to go into low-skilled occupations—the lowest rate in the world. Although many could not compete with average students elsewhere in core academic subjects, very few believed they would pay a price for this mediocrity. (By contrast, over half of Japanese 15-year-olds expected to be doing low-skilled work—while the country ranks fourth in overall academic achievement and has a lower unemployment rate than we do.)
Can we continue to peddle the American Dream in classrooms that don't prepare students to compete in a globalized labor force? One anonymous blogger wrote on the "We Arethe99 Percent" tumblr page:

"I have a bachelor's degree from a top-ranked liberal arts college and a master's from an Ivy League university. After graduation, all I could find was a year-long internship that only pays about 1/4 of my living expenses. The fellowship ends in under three months, and I still don't know if they plan to hire me on permanently."

Occupy Wall Street is not just about deadlock, dysfunction and disenfranchisement. It is about our nation's willingness to over-promise and under-educate. It is about the urgent need to finally get serious about making our education system worthy of our ambition.

 
 •  0 comments  •  flag
Share on Twitter
Published on October 18, 2011 04:00

October 17, 2011

Steve Case, Steve Jobs & Sweet (dis)Solve

I got the chance to spend an evening interviewing two American entrepreneurs in Times Square recently. One was Steve Case, the co-founder of AOL, and the other was Zoe Damacela, a 19-year-old who runs her own clothing company. We were the halftime show during the national business competition held by the Network for Teaching Entrepreneurship—an organization that goes into low-income high schools to help kids learn to start their own businesses. We watched four kids pitch their business ideas to a panel of accomplished entrepreneurs, and then, while the judges deliberated, we realized a few things:

* Entrepreneurs come out of the closet early. Both Steve and Zoe started businesses before they could be legally hired as employees. Zoe sold greeting cards as a little girl, and Steve did, too. But both had trouble getting taken seriously. We say we love innovation in this country, but we don't always celebrate the just-this-side-of-crazy risk-taking and hard work it takes to start a business.

* Steve Jobs could have ended up in a jumpsuit instead of a black turtleneck. We talked a bit about Jobs, since he had just passed away. He knew the value of a great education—and how it can make or break a child. Here is what Jobs said in 1995:

"I know from my own education that if I hadn't encountered two or three individuals that spent extra time with me, I'm sure I would have been in jail. I'm 100% sure that if it hadn't been for Mrs. Hill in fourth grade and a few others, I would have absolutely have ended up in jail. I could see those tendencies in myself to have a certain energy to do something. It could have been directed at doing something interesting that other people thought was a good idea or doing something interesting that maybe other people didn't like so much."

* The recovery of the U.S. economy and U.S. jobs will be led by entrepreneurs. We can choose to help them—by changing immigration laws to help attract and retain talented entrepreneurs from around the world, by making it easier for people to start businesses without worrying about losing their health insurance, and by helping successful companies grow more quickly. Or not.

Case and others recently met with President Obama to push him to pursue a 16-point plan for energizing entrepreneurship in America. I don't know if this strategy is the right one—or if it has any chance of succeeding, given what has happened with Obama's jobs bill.

But I agreed with Case when he says this:  "If we're worried about the economy, and everybody should be, if you're worried about employment, and everybody should be, the answer is really doubling down on entrepreneurship as a core American value….We have to. Because there really isn't a Plan B."

After that, the judges announced their winner: Congratulations to Hayley Hoverter, CEO of Sweet (dis)SOLVE from Los Angeles, CA, the 16-year-old winner of the NFTE 2011 Challenge. Hayley won $10,000 in venture money to grow her business selling dissolvable sugar packets (no paper, no nonsense) to high-end coffee shops. Watching Hayley pitch her business plan, which was meticulous and smart, I started to think there may be hope for America after all…
 •  0 comments  •  flag
Share on Twitter
Published on October 17, 2011 11:36

September 22, 2011

How to Stop Studying: Korea's Quest

When I visited Korea for the book this summer, I met a teacher who makes $4 million a year; I interviewed kids who study 16 hours a day; I had long, fascinating talks with principals, politicians and teachers, all of whom patiently and generously tutored me in the ways of the Korean education system.

But the strangest moment came when I did a ride-along with the local study-crackdown squad. Check out my new Time Magazine story about Korea's crusade to get its kids to chill out.

On a wet Wednesday evening in Seoul, six government employees gather at the office to prepare for a late-night patrol. The mission is as simple as it is counterintuitive: to find children who are studying after 10 p.m. And stop them.

In South Korea, it has come to this. To reduce the country's addiction to private, after-hours tutoring academies (called hagwons), the authorities have begun enforcing a curfew — even paying citizens bounties to turn in violators.

The raid starts in a leisurely way. We have tea, and I am offered a rice cracker. Cha Byoung-chul, a midlevel bureaucrat at Seoul's Gangnam district office of education, is the leader of this patrol. I ask him about his recent busts, and he tells me about the night he found 10 teenage boys and girls on a cram-school roof at about 11 p.m. "There was no place to hide," Cha recalls. In the darkness, he tried to reassure the students. "I told them, 'It's the hagwon that's in violation, not you. You can go home.'"

Cha smokes a cigarette in the parking lot. Like any man trying to undo centuries of tradition, he is in no hurry. "We don't leave at 10 p.m. sharp," he explains. "We want to give them 20 minutes or so. That way, there are no excuses." Finally, we pile into a silver Kia Sorento and head into Daechi-dong, one of Seoul's busiest hagwon districts. The streets are thronged with parents picking up their children. The inspectors walk down the sidewalk, staring up at the floors where hagwons are located — above the Dunkin' Donuts and the Kraze Burgers — looking for telltale slivers of light behind drawn shades.

At about 11 p.m., they turn down a small side street, following a tip-off. They enter a shabby building and climb the stairs, stepping over an empty chip bag. On the second floor, the unit's female member knocks on the door. "Hello? Hello!" she calls loudly. A muted voice calls back from within, "Just a minute!" The inspectors glance at one another. "Just a minute" is not the right answer. Cha sends one of his colleagues downstairs to block the elevator. The raid begins.


You can find the rest of the story here.
 •  0 comments  •  flag
Share on Twitter
Published on September 22, 2011 11:21

September 2, 2011

Hurricane Irene

One of many smash-ups left in the storm's wake. Washington, DC, Aug. 28, 2011
 •  0 comments  •  flag
Share on Twitter
Published on September 02, 2011 13:12