Scrum: The Art of Doing Twice the Work in Half the Time

by Jeff Sutherland

I’ve seen this duplicated at almost every company I have worked with. That tall stack of futility is one of the reasons Scrum can be such a powerful change for people. No one should spend their lives on meaningless work. Not only is it not good business, it kills the soul.

“Agile Manifesto.” It declared the following values: people over processes; products that actually work over documenting what that product is supposed to do; collaborating with customers over negotiating with them; and responding to change over following a plan. Scrum is the framework I built to put those values into practice. There is no methodology.

Of course Johnson’s twelve-month promise was somewhat misleading. Because, in actuality, they didn’t know; they couldn’t know. The FBI didn’t know how fast their teams could actually work. It’s something I tell executives all the time: “I’ll know what the date will be when I see how much the teams improve. How fast they’ll get. How much they’ll accelerate.”

one of the key concepts that Ohno introduced is the idea of “flow.” That is, production should flow swiftly and calmly throughout the process, and, he said, one of management’s key tasks is to identify and remove impediments to that flow. Everything that stands in the way is waste. Ohno gives waste a moral, as well as a business, value in his classic book, The Toyota Production System: It is not an exaggeration that in a low-growth period such waste is a crime against society more than a business loss. Eliminating waste must be a business’s first objective.

And that’s why Jeff Johnson needed a few months before he could really tell how long the project would take. He wanted to measure the velocity of each team measured over a few Sprints and then see how much they could improve—how much faster they could go. Once he looked at how many work items each team had finished in each Sprint and then checked how many they had remaining until the end of the project, he could forecast a completion date.

Besides learning how fast the teams were going, he also wanted to know what impediments were slowing them down. What he really wanted to do was accelerate those teams so they were producing faster—not by working longer hours (I’ll go into why that’s a fruitless rat hole that ends up making things take longer later) but by working better and smarter.

And that was what made it work, says Johnson. “Scrum is not about the developers. It’s about the customers and stakeholders. Really, it was an organizational change. Showing the actual product was the most powerful part.”

Creating something that complex is a massive human endeavor. Whenever people are involved in a complex, creative effort, whether they’re trying to send a rocket to space, build a better light switch, or capture a criminal, traditional management methods simply break apart.

We see echoes of our real lives captured in fictional workplace dystopias like those depicted in the cartoon Dilbert or the movie Office Space. We’ve all gone home and told our partners or friends of the madness that is modern corporate “organization.” We’ve all been told that filling out the form correctly is more important than doing the work, or that we need to have a meeting to prep for the pre-meeting meeting. It’s madness. And yet we keep on doing

The launch of Healthcare.gov, the website where Americans are supposed to be able to sign up for health insurance, is a great example. The front end was beautiful. It was clever, clear—a great design. It was completed in three months using Scrum. The back end, though—that was the debacle. It simply didn’t work. It was supposed to hook up databases in the IRS to state databases, to insurance company databases, to the department of Health and Human Services. This is a complex piece of work. It involved more than twenty contractors working on different bits and pieces, and they pla...

They never looked at the site from the user’s point of view, merely from their own. The reason they could do that was that they weren’t aligned—weren’t united in a common purpose. What Scrum does is bring teams together to create great things, and that requires everyone not only to see the end goal, but to deliver incrementally toward that goal.

Fail Fast So You Can Fix Early. Corporate culture often puts more weight on forms, procedures, and meetings than on visible value creation that can be inspected at short intervals by users. Work that does not produce real value is madness. Working product in short cycles allows early user feedback and you can immediately eliminate what is obviously wasteful effort.

From day one, it was a mystery to me why people insist on working in ways they know are inefficient and wasteful and that are dehumanizing and depressing. I guess they figure that’s the way everyone does it, so it must be the best way.

Then one day, one of the developers came in with a Harvard Business Review paper from 1986, written by two Japanese business professors, Hirotaka Takeuchi and Ikujiro Nonaka. It was titled, “The New New Product Development Game.” Takeuchi and Nonaka had looked at teams from some of the world’s most productive and innovative companies: Honda, Fuji-Xerox, 3M, Hewlett-Packard, and others. They argued that the old way of doing product development, typified by NASA’s Phased Program Planning system—a Waterfall system—was fundamentally flawed. Instead, the best companies used an overlapping development process that was faster and more flexible. The teams were cross-functional. The teams had autonomy. They were empowered to make their own decisions. And they had a transcendent purpose. They were reaching for something bigger than themselves. Management didn’t dictate. Instead, executives were servant-leaders and facilitators focused on getting obstacles out of their teams’ way rather than telling them what and how to do product development. The Japanese professors compared the teams’ work to that of a rugby team and said the best teams acted as though they were in a scrum: “… the ball gets passed within the team as it moves as a unit up the field.”

They insisted on the Waterfall method, insisted that everything could be planned ahead of time, even insisted that things wouldn’t change over the course of a multiyear project. That’s just insane on the face of it.

Scrum has its roots in Japanese thought and practice. When I travelled to Japan recently to meet with Professor Ikujiro Nonaka, he made it clear to me that in Japan Scrum isn’t seen as the latest work fad. They regard it as a way of doing, a way of being, a way of life. When I teach people how to do Scrum, I often talk about my own personal experience studying the Japanese martial art of aikido over the years.

Work doesn’t have to suck. It can flow; it can be an expression of joy, an alignment toward a higher purpose. We can be better. We can be great! We just have to practice.

If the best team could perform a task in one week, how long do you think it took the worst team? You might guess the same ratio as was observed at Yale—10:1 (that is, the slow team took more than two months to accomplish what the fast team knocked off in a week). The actual answer, though, is that there is a much larger difference in team performance than there is in individual performance. It actually didn’t take the slow team ten weeks to do what the best team could do in one week. Rather, it took them two thousand weeks. That’s how great the difference is between the best and the worst. So where should you focus your attention? At the level of the individual, where you might be able to get an improvement of ten times if you can magically make all your employees geniuses? Or at the team level, boosting productivity by an enormous magnitude even if you merely make your worst teams mediocre?

It was as if a state of grace had descended upon those players, and for a moment they could do no wrong. Larry Bird would drive down the court and pass the ball without looking toward what seemed to be empty hardwood. But just as the ball was headed out of bounds, Kevin McHale would simply appear exactly where he was supposed to be. And then he’d throw the ball to the side—again, seemingly without looking—and Robert Parish would just happen to be perfectly positioned for a shot. That absolute alignment of purpose and trust is something that creates greatness.

So how do you create a team that aims for a higher goal, organizes itself, and constantly feeds off each member’s skills? I spent a lot of time pondering that. After all, you can’t just yell at people to be more self-organized and transcendent; the motivation has to come from within. Imposing it will kill what you’re trying to do. Might there be a simple set of rules that encourage the formation of magic?

One of the key concepts in Scrum is that the team members decide themselves how they’re going to do the work. It’s management’s responsibility to set the strategic goals, but it’s the team’s job to decide how to reach those goals.

In a classically organized structure you might have the team of planners, followed by the team of builders, followed by the team of testers, followed by the production team, followed by the shipping team. Each team along the way has to finish its piece of the action before the project can move to the next step. No one team by itself can actually get a product out the door.

The fact is, when you look at the best teams—like the ones that existed at Toyota or 3M when Takeuchi or Nonaka wrote their paper, or the ones at Google or Salesforce.com or Amazon today—there isn’t this separation of roles. Each team has all the people on it do everything, soup to nuts.

Then in 2007 General David Petraeus led what became known as the “Surge,” which involved putting tens of thousands more troops into the country and having them live among the populace. This new strategy had a remarkable impact. One reason was that it got the Iraqi people to believe that the Americans were on their side, fighting the insurgents who were blowing up bombs in their neighborhoods and conducting ethnic cleansing. Another reason was that the American military, using a program called the “Sons of Iraq,” succeeded in bribing tens of thousands of former insurgents to come over to the US side. But there was a third component of the strategy—something that journalist Bob Woodward called as revolutionary as the invention of the tank or the airplane. This weapon wasn’t a new gizmo or unmanned drone. It was what General Stanley McChrystal, the commander of the Joint Special Operations Command at the time, called “collaborative warfare.” It involved using cross-functional teams from across the entire US government to target and disrupt Al Qaeda networks.

What they did was create a cross-functional team that had all the skills necessary to get the job done. Instead of having all these experts on separate teams rarely sharing information, they all worked together, in the same room, sharing all their intelligence and planning to track down and kill Al Qaeda operatives. Before this, an intelligence organization would designate the target, then hand off the actual operations to a Special Forces team. Then that team would hand over any intelligence it gathered to yet another team for analysis.

Managers often don’t want other managers, their own teams, or other people within the power structure to know exactly what they’re doing or what is being accomplished and how fast. They see keeping that knowledge secret as critical to their power. Instead of being aligned with the interests of the greater good, they’re aligned with their own motivations, which often boil down to greed and ambition. It’s the same kind of thinking that led to the massive management failure that caused the most recent economic collapse. At many companies, actions were based solely on what was in it for the individual on a short-term basis. There was no thought of what would benefit everyone, or of limiting harm to the global economy.

The team dynamic only works well in small teams. The classic formulation is seven people, plus or minus two, though I’ve seen teams as small as three function at a high level. What’s fascinating is that the data shows that if you have more than nine people on a team, their velocity actually slows down. That’s right. More resources make the team go slower.

Which leads us back to Brooks. When he tried to figure out why adding more people to a project made it take longer, he discovered two reasons. The first is the time it takes to bring people up to speed. As you’d expect, bringing a new person up to speed slows down everyone else. The second reason has to do not only with how we think but, quite literally, with what our brains are capable of thinking. The number of communication channels increases dramatically with the number of people, and our brains just can’t handle it.

Just as on a Special Forces team, everyone on a Scrum team has to know what everyone else is doing. All the work being done, the challenges faced, the progress made, has to be transparent to everyone else. And if the team gets too big, the ability of everyone to communicate clearly with everyone else, all the time, gets muddled.

So we set up this framework of Sprints and Daily Stand-up meetings and Reviews and Retrospectives, and I realized we needed someone whose job it was to make sure the process itself was effective. Not a manager—more of a servant-leader, something between a team captain and a coach. As we were watching the All Blacks every day, I asked the team what we should call this person. They decided on “Scrum Master.” He or she would facilitate all the meetings, make sure there was transparency, and, most important, help the team discover what was getting in their way.

Ideally, at the end of each iteration, each Sprint, the team would look closely at itself—at its interactions, practices, and processes—and ask two questions: “What can we change about how we work?” and “What is our biggest sticking point?”

And you know what? When you’re talking about yourself, you’re absolutely right. When talking about others, though, you’re making one of the most common—and destructive—human errors in judging other people’s actions. It even has a name: “Fundamental Attribution Error.”

This way of perceiving the world is funny when you see it in others. It’s so obvious that they’re making misjudgments. But before you laugh, you need to own up that you do it all the time as well. Everyone does. We all perceive ourselves as responding to a situation, while we see others as motivated by their character.

It’s the system that surrounds us, rather than any intrinsic quality, that accounts for the vast majority of our behavior. What Scrum is designed to do is change that system. Instead of looking for blame and fault, it rewards positive behavior by focusing people on working together and getting things done.

Milgram summarized the implications this way in his 1974 article “The Perils of Obedience”: Ordinary people, simply doing their jobs, and without any particular hostility on their part, can become agents in a terrible destructive process. Moreover, even when the destructive effects of their work become patently clear, and they are asked to carry out actions incompatible with fundamental standards of morality, relatively few people have the resources needed to resist authority.

The plant was closed by GM in 1982. Management considered the workforce the worst in America. People drank on the job, didn’t show up for work, and subtly sabotaged the cars (by, for example, putting a Coke bottle inside a door, where it would rattle and annoy customers). Toyota reopened the plant in 1984. GM told them about how awful the workers were but that the managers were great and they should rehire them. Instead, Toyota declined to rehire the managers and rehired most of the original workforce—even sent some of them to Japan to learn the Toyota Production System. Almost immediately the NUMMI plant was producing cars with the same precision and as few defects as cars that were produced in Japan. Same people, different system.

When a team starts to align and synchronize, it can seem magical. You feel it when you walk into a room with them.

But how do we do that? It’s easy enough to shout “Carpe diem!” from a stage to inspire a crowd, but how do you actually pull it off?

When I sat down to develop Scrum, I had no intention of creating a new “process.” I simply wanted to gather together all the research that had been done for decades on how people work best and emulate that. I wanted to incorporate best practices and steal any better ideas I came across. Right before the first real Scrum at Easel in 1993 I was working at a company just blocks from the MIT Media Lab, and I stole an idea from the lab that has become the core of Scrum: the Sprint.

Their speed owed itself to a policy that the Media Lab had for all its projects. Every three weeks each team had to demonstrate to their colleagues what it was working on. This was an open demonstration; anyone could come. And if that demo wasn’t both working and cool, lab directors killed the project. This forced the students to build neat stuff fast and, most important, to get immediate feedback on it.

And so my team embarked on what we called “Sprints.” We called them that because the name evoked a quality of intensity. We were going to work all out for a short period of time and then stop to see where we were.

Sprints are what are often called “time boxes.” They’re of a set duration. You don’t do a one-week Sprint and then a three-week Sprint. You have to be consistent. You want to establish a work rhythm where people know how much they can get done in a set period of time. Often that quantity surprises them.

One crucial element of an individual Sprint, though, is that once the team commits to what they’re going to accomplish, the tasks are locked in. Nothing else can be added by anyone outside the team. Later, I’ll get further into the reasons why, but for now just know that interfering and distracting the team slows its speed dramatically.

The Scrum Master, the person in charge of running the process, asks each team member three questions: 1. What did you do yesterday to help the team finish the Sprint? 2. What will you do today to help the team finish the Sprint? 3. What obstacles are getting in the team’s way?

What he did was map all the communication flows within the team—who was talking to whom, where information was flowing, and where it wasn’t. This type of mapping is a tool that can be used to spot bottlenecks or information hoarders. The greater the communication saturation—the more everyone knows everything—the faster the team. Basically, the metric spun off by this type of analysis measures how well everyone knows what they need to get their work done. Borland had the highest rating ever: 90 percent. Most companies hover around 20 percent.

And that’s how the daily meeting came to operate. We had certain rules. The meeting was held at the same time every day, and everyone had to be there. If the entire team wasn’t present, communication simply didn’t happen. And it didn’t matter what time of day the meeting took place, as long as it was at the same time every day. The point was to give the team a regular heartbeat.

This is the reason such a meeting is often called the Daily Stand-up or Daily Scrum. It doesn’t really matter what you call it. It has to be at the same time every day, with the same three questions, with everyone standing up, and last no more than fifteen minutes. The problem that I frequently see crop up is that people have a tendency to treat the Daily Stand-up as simply individual reporting. “I did this … I’ll do that”—then on to the next person. The more optimum approach is closer to a football huddle. A wide receiver might say, “I’m having a problem with that defensive lineman,” to which an offensive blocker might respond, “I’ll take care of that. I’ll open that line.”

My standard speech to teams large and small is: “Do you really want to suck forever? Is that what your motivation is in life? Because it’s a choice, you know—you don’t have to be that way.” A team has to demand greatness from itself.

Think about your job. How much of your time is wasted while you’re waiting for someone else to finish their work, or for information to be delivered, or because you’re trying to do too many things at once?

The heart of Scrum is rhythm. Rhythm is deeply important to human beings.