Scrum: The Art of Doing Twice the Work in Half the Time

by Jeff Sutherland

Up to that point—and even as late as 2005—most software development projects were created using the Waterfall method, where a project was completed in distinct stages and moved step by step toward ultimate release to consumers or software users.

The trouble is, once that beautifully elegant plan meets reality, it falls apart. But instead of scrapping the plan, or the way they think about the plan, managers instead hire people to make it look as if the plan is working. Essentially, they’re paying people to lie to them.

Two decades ago I was desperate. I needed a new way of thinking about work. And through tons of research and experimentation and looking over past data I realized we all needed a new way of organizing human endeavor.

Traditionally, management wants two things on any project: control and predictability.

Scrum embraces uncertainty and creativity. It places a structure around the learning process, enabling teams to assess both what they’ve created and, just as important, how they created it. The Scrum framework harnesses how teams actually work and gives them the tools to self-organize and rapidly improve both speed and quality of work.

I’m writing this book to show you how to do it. And not just in software companies. I’ve seen Scrum used successfully to build cars, run a laundry, teach students in a classroom, make rocket ships, plan a wedding—even, as my wife has used it, to make sure that the “honey-do” list gets done every weekend.

“Agile Manifesto.” It declared the following values: people over processes; products that actually work over documenting what that product is supposed to do; collaborating with customers over negotiating with them; and responding to change over following a plan. Scrum is the framework I built to put those values into practice. There is no methodology.

In Scrum we call these cycles “Sprints.” At the beginning of each cycle there is a meeting to plan the Sprint. The team decides how much work they think they can accomplish during the next two weeks. They’ll take the work items off that prioritized list of things that need to be done and often just write them out on sticky notes and put them on the wall.

“How can we work together better in the next Sprint? What was getting in our way during the last one? What are the impediments that are slowing our velocity?” You can find a more detailed explanation of how Scrum works in the appendix.

Once he looked at how many work items each team had finished in each Sprint and then checked how many they had remaining until the end of the project, he could forecast a completion date.

They were going three times as fast once they got moving as compared to when they started. Why? They got better at working together, yes, but most important, they figured out the things that were slowing them down, and each cycle, each Sprint, they’d try to get rid of them.

The launch of Healthcare.gov, the website where Americans are supposed to be able to sign up for health insurance, is a great example.

This is a complex piece of work. It involved more than twenty contractors working on different bits and pieces, and they planned it all using Waterfall techniques. They only tested the site at the very end for a few days, rather than doing incremental testing along the way.

The people who fixed Healthcare.gov? They used Scrum.

In this book you’re going to learn some of the fundamental ways that people work best, why we’re awful at estimating, and why working overtime will make your project late. I’m going to take you through all the research and applications that people and scientists and organizations have diligently done for years, and how Scrum ties it all together in a way that you can implement tomorrow.

Years later it occurred to me that organizations, teams, and people are all complex adaptive systems. The same things that move cells from one state to another are also what move people from one state to another. To change a cell, you first inject energy into the system. At first there’s chaos, there seem to be no rules, everything is in flux.

How, I wondered, can we figure out some simple rules that will guide teams to settle into a more productive, happier, supportive, fun, and ecstatic state? I spent the next fifteen years trying to figure that out.

Scrum teams that work well are able to achieve what we call “hyperproductivity.” It’s hard to believe, but we regularly see somewhere between a 300- to 400-percent improvement in productivity among groups that implement Scrum well.

When I train people how to do Scrum, that’s what I use: paper airplanes. I divide people up into teams and tell them that the goal is to build as many paper airplanes as they can that will fly across the room. There are going to be three roles on the team. One person will check how many planes are built that can actually fly. Another will work as part of the assembly process but will also pay attention to the process itself and look for ways that the team can make better planes and speed up their production. Everyone else will concentrate on building as many planes that can actually fly the distance in the assembly time allowed.

And finally they’ll have two minutes to Check. In this phase the team looks for how they could improve their paper airplane–building process. What went right? What went wrong? Should the design be changed? How can the construction process be improved? And then they will Act.

They regard it as a way of doing, a way of being, a way of life. When I teach people how to do Scrum, I often talk about my own personal experience studying the Japanese martial art of aikido over the years.

Scrum is a lot like that. It requires practice and attention, but also a continuous effort to reach a new state—a state where things just flow and happen. If you’ve ever watched great dancers or gymnasts, you know that their motion can almost seem effortless, as if they’re doing nothing but simply being. They seem as if they couldn’t be anything else but what they are in that moment. I experienced that one day when a diminutive aikido master threw me effortlessly through the air, and yet did so in a way that caused me to fall gently to the mat, as though I were a baby being gently placed in a cradle.

Don’t Guess. Plan, Do, Check, Act. Plan what you’re going to do. Do it. Check whether it did what you wanted. Act on that and change how you’re doing things. Repeat in regular cycles, and, by doing so, achieve continuous improvement.

Take, for example, the process by which students receive grades in a class. At Yale University a computer-programming course taught by Professor Stanley Eisenstat, CS 323, is notoriously hard. When students began complaining about how long each assignment took, the professor didn’t make his assignments easier, but he did start tracking how long each student needed to complete them.

Some people work quickly and get an A, and some people work meticulously and get the same grade. The only difference is the amount of time spent. So what is the application for business?

When I was designing Scrum, I looked at what super-performing teams had that other teams didn’t. Why is it, I wondered, that some teams change the world, and others are mired in mediocrity? What are the common elements that truly great teams have? And, most important, can we reproduce them? The answer, it turns out, is yes.

Transcendent: They have a sense of purpose beyond the ordinary. This self-realized goal allows them to move beyond the ordinary into the extraordinary. In a very real way the very decision to not be average, but to be great, changes the way they view themselves, and what they’re capable of. 2. Autonomous: The teams are self-organizing and self-managing, they have the power to make their own decisions about how they do their jobs, and are empowered to make those decisions stick. 3. Cross-Functional: The teams have all the skills needed to complete the project. Planning, design, production, sales, distribution. And those skills feed and reinforce each other. As one team member that designed a revolutionary new camera for Canon described it: “When all the team members are located in one large room, someone’s information becomes yours, without even trying. You then start thinking in terms of what’s best or second best for the group at large and not only where you stand.”1

Among those descending on Cairo was National Public Radio, one of the premier journalism outfits in the United States. Caught flat-footed at first, producers and reporters for NPR blew deadlines, missed stories, and scrambled to meet the demands of executives back in Washington.

In a very real way, they were on their own. With constant direct oversight by Washington being impossible and events happening so fast, the team had to organize themselves to get the work done. One of the key concepts in Scrum is that the team members decide themselves how they’re going to do the work. It’s management’s responsibility to set the strategic goals, but it’s the team’s job to decide how to reach those goals.

They succeeded by using Scrum. Their deadlines called for them to report every twelve hours, on Morning Edition and All Things Considered. Each cycle J.J. would talk to the team and ask three very simple questions: What did you do since the last time we talked? What are you going to do before we talk again? And what is getting in your way?

Now Scrum is being used all over the place at NPR, from web design, to data journalism, to creating new radio shows. Teams at the Chicago Tribune, New York Times, Washington Post, and ProPublica are all using Scrum. When deadlines are tight, they just need the speed.

The third leg of the stool for great teams is that they have all the skills necessary to get things done. In a classically organized structure you might have the team of planners, followed by the team of builders, followed by the team of testers, followed by the production team, followed by the shipping team.

What she looks for in a team is diversity—of skill set, thinking, and experience. She wants teams that are unselfish and autonomous, but she also needs them to be cross-functional. Teams that can get a whole project done.

Special Forces don’t want any Challenger disasters. As a result there’s constant communication among the people collecting the intelligence, those planning what to do with it, and those who’ll be going through the door. During the Iraq war, Special Forces

But just because cross-functionality can achieve great results, you shouldn’t play Noah and throw two of everything into a team. The team dynamic only works well in small teams. The classic formulation is seven people, plus or minus two, though I’ve seen teams as small as three function at a high level. What’s fascinating is that the data shows that if you have more than nine people on a team, their velocity actually slows down.

So we set up this framework of Sprints and Daily Stand-up meetings and Reviews and Retrospectives, and I realized we needed someone whose job it was to make sure the process itself was effective.

They decided on “Scrum Master.” He or she would facilitate all the meetings, make sure there was transparency, and, most important, help the team discover what was getting in their way. The key part of that was to realize that often the impediments aren’t simply that the machine doesn’t work or that Jim in accounting is a jerk—it’s

It was the Scrum Master’s job to guide the team toward continuous improvement—to ask with regularity, “How can we do what we do better?”

“What can we change about how we work?” and “What is our biggest sticking point?” If those questions are answered forthrightly, a team can go faster than anyone ever imagined.

We all perceive ourselves as responding to a situation, while we see others as motivated by their character.

Are ordinary Americans so different from Germans? Would they have reacted differently in the same situation? And the uncomfortable answer is that no, Americans wouldn’t have behaved differently. In fact, given how many countries and cultures have replicated the experiment, no one would have. Given the right situation, we’re all capable of being Nazis.

What Scrum does is accept this reality, and, instead of seeking someone to blame, it tries to examine the system that produced the failure and fix it.

Yet people want to blame individuals, not systems. It

Autonomy. Give teams the freedom to make decisions on how to take action—to be respected as masters of their craft. The ability to improvise will make all the difference, whether the unit is reporting on a revolution in the Middle East or making a sale.

Blame Is Stupid. Don’t look for bad people; look for bad systems—ones that incentivize bad behavior and reward poor performance.

While I have nothing against promotions, sales, or projects, it’s just a fact that humans are absolutely terrible at working that way. We’re lousy focusers, we spend far more hours in the office than needed, and we’re horrible estimators of how long things will take. This is all people I’m talking about—it’s how we humans simply

So when I started the first Scrum at Easel and told the CEO I wasn’t going to show him a long and detailed Gantt chart that we both knew was wrong, he said, “Fine. What are you going to show me?” And I told him that each month I’d show him a piece of working software. Not something that works in the back end. Not some piece of architecture. A piece of software that a customer can actually use. A fully implemented feature. “Okay,” he said. “Do that.”

And so my team embarked on what we called “Sprints.” We called them that because the name evoked a quality of intensity. We were going to work all out for a short period of time and then stop to see where we were.

Sprints are what are often called “time boxes.” They’re of a set duration. You don’t do a one-week Sprint and then a three-week Sprint. You have to be consistent. You want to establish a work rhythm where people know how much they can get done in a set period of time. Often that quantity surprises them.

Every day each team gathers in front of a floor-to-ceiling whiteboard stretching from one end of the wall to the other. Just like at Team WIKISPEED there are a few columns drawn on the board: Backlog, Doing, Done.