The Art of Agile Development

by James Shore

First, business analysts would interview stakeholders and document the system requirements. Next, software architects would read the requirements documents and create detailed design documents specifying every component of the system and how they related to one another. Then programmers would convert the design documents to code. In some organizations, this was considered low-skill work—just a mechanical translation exercise.

Agile is three things: the name, the values, and the principles. That’s it. It’s not something you can do. It’s a philosophy.

Agile Development is adaptive rather than predictive; people-oriented rather than process-oriented.3 Martin Fowler

Agile teams define success as delivering value, not conforming to a plan. In fact, truly Agile teams actively look for opportunities to increase value by changing their plans.

Agile says people are the most important factor in software development success. Not just their skills, but all aspects of their humanity. How well team members work together. How many distractions they encounter. How safe they are to speak up, and whether they’re motivated by their work.

Agile teams have a process—every team does, even if it’s implicit—but the process is in service of the humans, not the other way around. And Agile teams are in charge of their own process. When they think of a better way of working, they change it.

You need to get feedback about what’s missing or wrong, then change your plans based on what you learn.

Deliver Value. Seek feedback, experiment, and adapt your plans. Focus on producing valuable results. Consider partially done work a cost, not a benefit. Deliver frequently.

Eliminate Waste. Work in small, reversible steps. Embrace the possibility of failure and design your plans to fail fast. Maximize work not done. Pursue throughput rather than efficiency.

Seek Technical Excellence. Enable agility via technical quality. Design for what is known, not what is speculated. Start simple and add complexity only in response to actual needs. Create systems that are easy t...

Improve Your Process. Experiment with new ideas. Tune and adapt what works. Never assume the established, popular wa...

As that success grew, Agile’s momentum shifted from the underlying ideas to hype. Rather than saying, “Let’s get better results by adapting our plans and putting people first,” organization leaders started saying, “Everybody’s talking about Agile. Get me some Agile.”

The tragedy of the Cargo Cult is its adherence to the superficial, outward signs of some idea combined with ignorance of how that idea actually works. In the story, the islanders replicated all the elements of cargo drops—the airstrip, the tower, the headphones—but didn’t understand the vast infrastructure that enabled airplanes to arrive.

The name Agile is everywhere. The ideas of Agile aren’t. It’s become self-perpetuating: for many, the only Agile they know is Cargo Cult Agile.

Every team has a way of working—a process, or method—that it follows, even if it isn’t formally written down. The method reflects an underlying philosophy of software development, although that philosophy is rarely articulated and isn’t necessarily self-consistent.

As a result, although it’s tempting to customize your Agile method from the beginning, it’s best to start with a by-the-book approach. The practices that are the least familiar are the ones that are most tempting to cut, but they’re the ones you need most, if you’re really going to be Agile. They’re the ones that involve the biggest change in philosophy.

There is no last step. Agile software development is an ongoing process of learning and improvement. Never stop practicing, experimenting, and evolving.

Daily planning. If you struggle with frequent interruptions, try adopting day-long iterations (see “Task Planning”). Use the planning game (see “The Planning Game”) and your team’s measured capacity (see “Capacity”) to conduct a joint planning session at the beginning of each day, then defer all interruptions until the next day’s planning meeting. Be sure to have people estimate their own tasks.

and tests. See “Continuous Integration” for details. Test-driven development. Although test-driven development (see “Test-Driven Development”) isn’t as easy to adopt as the other practices, it’s very powerful. Test-driven development is the basis for reducing bugs, increasing development speed, improving your ability to refactor, and decreasing technical debt. It can take some time to master, so be patient.

Each zone has several levels of maturity: Learning. The team is learning the skills. Proficient. The team is able to exhibit the skills when it concentrates on them. Fluent. The team exhibits the skills automatically, without conscious effort, as long as it has a coach as part of the team. Independently Fluent. The team exhibits the skills automatically without needing a coach or any one team member.

Focusing Zone The Focusing zone is about Agile fundamentals: focusing on business results; working as a team; taking ownership. Teams that are fluent in this zone focus development on their team’s core purpose, release their most valuable features first, and change direction in response to changing business needs. They’re constantly Focusing on their organization’s most valuable priority.

If you are able to get buy-in, Focusing fluency will take each team about 2–6 months of dedicated effort to achieve. With proper support, they’ll exceed their prior levels of performance within 1–4 months.

Delivering teams prevent this problem through technical excellence. They design their code to respond to frequent changes. They keep code quality high, so they don’t waste time hunting bugs. They refine their production lifecycle so releases are painless and operations are manageable. They’re capable of Delivering reliable, low-defect software whenever it makes the most business sense.

If your company makes these investments, Delivering fluency will take each team 3–24 months to develop, and you’ll see improved performance within 2–6 months. The exact amount of time each team needs will depends on the quality of its existing code and how much coaching team members receive.

This requires a shift in organizational structure. Creating optimal plans requires constant attention by people with deep business and product expertise, and that means having product and market experts join development teams full-time. It also means giving those teams full responsibility for their product budgets and plans.

Teams typically need to spend at least a year building trust via Delivering fluency before they get permission for these investments. Once that happens, Optimizing fluency takes another 3–9 months to develop, although you’ll see improvements within 1–3 months. But even then, Optimizing is a never-ending process of experimentation, learning, and discovery.

Briefly, the Strengthening zone involves distilling teams’ collective insights and channeling them into organizational improvements.

SUMMARY OF AGILE FLUENCY ZONES Focusing: Main benefit: Focus on business priorities; visibility into teams’ work; ability to change direction Investments: Team structure; management; work environment Approximate timing: 1-4 month performance dip; 2-6 months until fluency Delivering: Main benefit: Low defects and high productivity; technical longevity Investments: Development skills; integrating testing and operations Approximate timing: 2-6 month performance dip; 3-24 months until fluency Optimizing: Main benefit: Higher-value releases and better product decisions Investments: Embedded product management; team ownership of budgets and plans Approximate timing: 1-3 month performance dip; 3-9 months until fluency

As Martin Fowler said:1 I see a startling parallel between DHH [David Heinemeier Hansson, creator of Ruby on Rails] and Kent Beck [creator of Extreme Programming]. For either of them, if you present them with a constrained world, they’ll look at constraints we take for granted, consider them to be unessential, and create a world without them…they just stick some intellectual dynamite under them and move on. That’s why they can create things like Extreme Programming and Rails, which really give the industry a jolt.

Agile teams’ performance changes in other ways, too. Agile teams focus on getting features completely done before moving on to the next feature. This is particularly true for Delivering teams, which build quality in from the beginning rather than fixing bugs at the end. This improves throughput and performance, but—ironically—it feels like a slow-down to people who are used to seeing multiple features in progress at once.

Your organization needs to invest in teams that are: Cross-functional. The people on the team collectively have all the expertise needed for the team to fulfill its purpose. Fully dedicated. Specialists can come help from time to time, but core team members are solely dedicated to their team. Collaborative. Team members have friendly, collegial relationships and work closely together. Long-lived. It can take months for team members to figure out how to work most effectively together, so keep teams together for as long as possible.

Each team needs a coach to help team members learn how to be an effective Agile team. “Coaching Skills” has the details, but briefly: Every team needs someone who can help team members learn how to be an effective, jelled team. Focusing teams need someone who can teach them the planning practices described in Part II. Delivering teams need someone who can teach them the technical practices described in Part III. Optimizing teams need someone who can teach them the business development practices described in Part IV. Some coaches can cover multiple categories. Each coach can work with one or two teams.

A related issue is traceability. Some companies require that every commit be traceable to its original authors. You can meet this requirement by adding the authors’ initials or email addresses to the commit comment. Git has a convention of adding Co-authored-by lines to the end of the commit message.3

Some companies require that all code be reviewed prior to release. Pairing and mobbing meet this requirement, but you may need to modify tooling to allow code to be released without a separate review phase. If removing the requirement entirely isn’t an option, you might be able to modify the tool to skip commits that have coauthors.

Large changes—those that directly impact more than 30-70 people—require professional change management. Depending on the size of your organization, your HR department may have change management staff who can help. If you hire Agile consultants to help with your change, ask about their change management experience and approach.

Kaizen (rhymes with “I win”) is a common term in the Agile community. It’s a Japanese word meaning “improvement.” In the Agile community, it specifically means continuous, gradual improvement.

Kaizen is for improving your existing ways of working. If you have a document-oriented culture, kaizen will help you streamline your documents. If you have a blame-oriented culture, it will help you place blame more accurately. But it won’t help you make the leap from either culture to Agile.

If you have a lot of teams, it’s probably safest to proceed gradually, but even then, kaikaku is your best bet. Rather than using kaizen to gradually introduce Agile to a lot of teams, use kaikaku to fully introduce Agile to a subset of teams.

The stakeholders who are most likely to resist are your teams’ business partners: product management, marketing, and sales. Agile represents a major shift in how teams interact with these groups. They’re used to a predictive approach, with a focus on commitments and deadlines. Their interactions with development teams are typically focused on documents, progress reports, and sign-offs.

Some stakeholders love it. Finally, they know what’s really going on, and they have the ability to influence the outcome. Others, particularly those who have been burned by missed commitments in the past, see Agile as a political maneuver: a complicated way for teams to avoid making promises. They fight Agile tooth and nail.

You’ve got to balance multiple interests, and you’re providing visibility and control, not predictability, but you’re there to help.

Agile uses adaptive planning, which involves changing your plans as you go to achieve the best possible results. You can commit to a specific date and steer your plans to meet that date, as described in “Predefined Release Dates”, but you can’t predict exactly which features will be done on that date.

Look to smaller “boutique” and independent consultancies that specialize in Agile change and coaching-the-coaches. The people you hire matter more than the vendor, especially for these specialized skills, and small consultancies do a better job of recognizing this.

Fluency is the basis for successfully scaling Agile, but it isn’t enough on its own. Unless every team works completely independently, you also need a way of coordinating teams’ work. This is harder than it sounds, because teams have dependencies on one another that tend to result in bottlenecks, delays, and communication errors. Successfully scaling Agile is a matter of figuring out how to manage those dependencies.

Because multiple LeSS teams could end up touching the same code, they’re expected to coordinate with one another to prevent problems. The coordination is typically ad hoc and peer-to-peer. Team members know when they need to coordinate because they worked together to choose stories, and part of the discussion involves considering how and when to coordinate.

Rather than creating detailed story breakdowns, FAST teams create a “discovery tree” for each valuable increment. (A valuable increment is something that can be released on its own—see “Valuable Increments”.) A discovery tree is a hierarchical, just-in-time breakdown of the work required to release the increment. It’s represented with sticky notes on a wall or virtual stickies on a virtual whiteboard.

Teams work for two days, or whatever cadence the tribe has chosen. They’re not expected to finish anything specific in that time. Instead, they just make as much progress as they can.

Someone may also volunteer to be a “feature steward” for a particular discovery tree, if neede...

The real problem is that it’s easy for collective code ownership to turn into no code ownership.