David Scott Bernstein's Blog, page 22

The Importance of Technical Practices

Software development has undergone many revolutions over the last few decades and the way we build software today is fundamentally different than the way we did just a few years ago. Our industry is maturing and we are beginning to pay attention to the quality of our work. No longer is just fulfilling the specification enough. We have to write code that’s easy to maintain and extend so we can reduce the cost of ownership. This requires thinking about building software in some fundamentally different ways.

When I ask developers what they can do to make their code more maintainable or conversely, what other people do that makes code difficult to maintain, I rarely get a clear answer because it’s something most of us really haven’t thought about. But there are things we can do to make our code more maintainable, and there are definitely things we must avoid when building software as well.

Many of the most important and valuable technical practices have been embraced in Agile software development, including Scrum, Extreme Programming, and continuous integration.

At its core, Scrum simply says to take complex software problems and build them incrementally. This has the effect of taking a difficult task and breaking it down into several much more easily manageable tasks. The idea works in all fields, and works especially well in software development.

Scrum went on to discover that with an active product owner, the ambiguity in software development—especially in requirements—goes away and again the process of building the right software is simplified. We stay focused on the most important issues and get lots of feedback from our customers.

These two aspects of Scrum software development are largely responsible for its enormous success, but there are other things that can help the development process, specifically the technical practices from Extreme Programming, which goes very nicely with Scrum development.

Extreme Programming, and before that the Smalltalk community, have proven time and time again that big, complex software problems can be broken down into simpler elements and can be made more manageable by a handful of straightforward techniques such as continuous integration, test driven development, and pairing.

Again, these are very simple practices that help guide us in doing the right thing at the right time. The key is to understand their purpose so you can use the practices to their fullest. This is the subject of my book, Beyond Legacy Code: Nine Practices to Extend the Life (and Value) of Your Software.

Two Big Things

I’d like to say that there are two big things we must achieve in software development. We must build the right thing and we must build the thing right.

Building the right thing starts with what the customer wants so they can gain value from our work.

The way we build the right thing for the customer in Agile software development is with frequent feedback. We build a little and then we take it to the customer and ask, “Is this what you want?” And we listen.

Software development is a discovery process for the developer as well as for the customer, and by partnering with the customer and discovering together the best way to achieve their goals we can build a better product. This is really the essence of Scrum and why having a short iteration cycle is so valuable for building the right thing. People can’t always come up with solutions to problems but they can recognize that they have a problem and when the solution is presented they can see that it will help. With constant feedback most teams get to build something of value to the customer.

When we order our backlog then the things that aren’t as important just drop off and we’re focused more on the things that provide the highest value to the customer. An ordered backlog is not really to help us build the most important things first, as much as it is to help us not build the least important things.

In Waterfall, customers know they get one bite of the apple and so everything becomes critically important. But with the ability to give constant feedback, customers can be more focused and identify the next most important thing that gives them more information about what is and what isn’t important for future iterations.

If we don’t build the right thing then all bets are off because no matter how beautiful our code is the customer will not find value in it. But just building the right thing is not enough either, we have to build the thing right.

Something that is often overlooked is that Agile and Scrum are more than development processes, they’re ways of building software. To do Agile successfully doesn’t just mean having standup meetings and doing sprints, there’s a lot more to it.

Building software right means understanding what good software is and how to create it.

Unlike other engineering disciplines, which are true disciplines, software development is often fly by the seat of your pants, make it up as you go along, clump something together that seems to work, and hope for the best.

You can’t just hang a shingle outside your door and say you’re a doctor, it’s illegal. But you can hang a shingle out your door and say you’re a programmer. There is no standard body of knowledge that’s recognized as prerequisites for becoming a software developer like there is in medicine or the law.

As such, it’s not easy evaluating a programmer’s capabilities. We’re not a homogeneous group. If teams do possess a set of general standards they often go unspoken so it’s typical to see huge differences between coding styles among developers in the same team. This promotes individual code ownership rather than collective code ownership and keeps knowledge of a system from disseminating to all team members. When we silo knowledge and foster specialization, projects suffer and teams become dependent on specific individuals to maintain aspects of the system.

Building the thing right means employing good engineering practices and elevating software development to a true engineering discipline. This means that developers must have a core set of knowledge and common goals: the sort of principles and practices I cover in detail in Beyond Legacy Code: Nine Practices to Extend the Life (and Value) of Your Software.

This doesn’t necessarily mean that software development becomes rote. Every problem needs unique solutions but we can find common ways of approaching problems that give us traction and we can build our common vocabulary. That gives us the ability to communicate better among ourselves, gives us a better ability communicate what a program does.

Building software is among the most complex activities humans can engage in. We need guidelines to help us and that really is what the Agile software development movement is bringing to the table.

Success to Standish is Failure in Agile

The Standish Group’s CHAOS Report is the largest and longest running research study in the software industry. Standish looks at approximately 34,000 software projects in a range of areas. It includes shrink-wrap applications, operating systems, custom apps, etc. They picked a range of diverse projects to look at. Every ten years the study drops about 3,400 projects from ten years ago and picks up another 3,400 projects in the current year. The analysis of the data is massive. Unfortunately, the study is blemished by many inherent flaws.

One of the flaws of the CHAOS Report is how it defines success. It defines success as “delivered on time and on budget with all the features and functions as originally specified.”

If you deliver exactly what you predicted it means you didn’t learn anything through the process of developing that software. In Agile, software development is a discovery process, with frequent feedback from our customers. We’re learning how to create a more valuable and useful product as we build it. This is one of the primary advantages to doing Agile software development.

But worse still, the Standish definition of success is all about prediction. It encourages bad practices such as padding estimates. Estimates are just guesses, no matter how artfully they’re applied, and sometimes guesses are wrong. Holding teams accountable for their estimates encourages people to be conservative, which erodes trust and discourages innovation.

We tend to throw around terms like “success” and “quality” without validating that everyone shares a common definition of them. These terms sound appealing but when everyone has different definitions of them then it can be very difficult to achieve them consistently.

Any real definition of success in software must encompass both definitions of success:

Success in the marketplace: the software delivers the features customers want, and maybe some features they didn’t know they wanted but find valuable.
Success as software: code that supports future extensions so it continues to add value in the future.

These two dimensions of “doing the right thing” and “doing the thing right” are at the core of my definition of success.

Successful software requires many things to be successful, from the code itself to its marketing operations, sales, and so on. It’s a complex process, and with any complex process there are far fewer ways to do it right than to get it wrong.

Success can’t usually be planned, it’s discovered. And success certainly has little to do with how accurately we were able to estimate a task upfront. To me, success as definied by Agile is real success: We want to make our customers happy through early and continuous delivery of valuable software. We want to build features that customers use and find value in. Likewise, we want our teams to be happy and healthy so they can continue to produce valuable software, and we want stakeholders to be happy by keeping the cost of ownership low for the software we produce. Everything else is just playing with numbers.

Propagate Knowledge

There is a huge amount of professional wisdom in software development. These are things that we have never bothered to write down but are standards nonetheless.

This general knowledge is communicated through individual interactions as well as the code we write, so it’s really important that we have high quality interactions within the team and that we do frequent code reviews.

Pairing and mobbing are two practices that support the propagation of knowledge across a team. I find that information spreads like wildfire on teams that pair and it spreads even faster in a mob. It often turns out that the information spread is the most important information the team needs. This includes programming standards and practices as well as a range of conventions.

We want to support collective code ownership so that any developer can go to any part of the code and work on it, but in order to do this we have to adopt common coding conventions and practices. This adoption can’t take place among just one or a few of the team members but rather must propagate across the whole team.

We also don’t want to be developing specialists who are the only people that know a particular technology or part of the code. We don’t want the company to become dependent on one or a few individuals. It’s safer to spread this knowledge around so there is no one person who holds all the knowledge for a particular piece of a system. The “hero” mentality on a non-Agile project, where someone specializes and is responsible for just one section of the code, could quickly become a liability. If those individuals, who were the only ones with critical knowledge of a certain part of the system, went on vacation or left the company then there would be big problems.

Some people think that having specialized knowledge equates to job security but it doesn’t. If the knowledge you possess and only you possess is important, you’ll find that your managers are reluctant to send you on vacation and you might quickly find that you “owe your soul to the company store.”

Just a few decades ago, in the midst of the Industrial Revolution, job security meant having specialized knowledge, knowledge that no one else had that gave you a competitive advantage and kept your manager from letting you go during cutbacks.

But in today’s Agile world job security is not about having and holding on to specialized knowledge. It’s just the opposite. It’s about sharing knowledge and propagating information across the team. The most highly valued team members are the ones who help others and support the team in being the best they can be.

Developer on Fire Podcast

I had a very interesting conversation with Dave Rael (@raelyard) on the #DeveloperOnFire #Podcast about what excites me as a developer. Listen to it here: http://developeronfire.com/podcast/episode-206-david-bernstein-beyond-legacy-code

Swarm on Showstoppers

Swarming is a practice where the entire team works together temporarily to solve a problem.

Swarming is most useful for certain kinds of problems, problems that impact the entire team. If no one can do anything until the problem is resolved then it may make sense, and improve morale, if everyone on the team has an opportunity to work on resolving it.

I remember I was sailing to Bimini from Miami with some friends and later the following day, we had run aground on a sandbar. The captain had the crew and the rest of us swarm on finding a solution. Some of us jumped into the water to help push the boat free. Others stayed on the boat and used their weight to shift the position of the hull. A few minutes later, we had broken free and we did it because we all worked together.

Swarming works best if everyone has a vested interest in the outcome. Of course, there are some problems that just require one person. For example, if the build breaks it’s typically the last person who ran the build that caused it to break so that person should be responsible for fixing or backing out any problematic changes so the build can run again. The entire team is dependent upon a good continuous integration server and when it goes down it can be of paramount importance to get it back up as quickly as possible. But this is usually a “one person job” since the person who broke the build is best suited to fix it.

Some teams find that they can become hyper productive when working in a swarm and for these teams, I recommend mobbing as a technique that they use on an ongoing basis. Mobbing is a technique first described by Woody Zuill. Woody describes mob programming as “All the brilliant minds working together on the same thing, at the same time, in the same space, and at the same computer.”

Mobbing is like keeping a full time swarm where everyone on the team works together all the time. It may sound inefficient for the whole team to work in a mob but it turns out that for certain kinds of tasks and on some teams it can be highly efficient. Mobbing and swarming can be a good way to start out a new team working together and then after a few days or weeks of working in the group, individual members can break out and work on their own. This can be a great way to get everyone on the same page with common practices and coding conventions.

Swarming can be a great opportunity for building and solidifying a team. The next time you’re faced with a showstopper problem, consider calling, “All hands on deck!”

Podcast on The Agile Uprising

I had a great conversation with Ryan Lockard (@AgilePHL) and James Gifford (@scrummando) on the @AgileUprising podcast. Listen here:

https://coalition.agileuprising.com/t/podcast-released-beyond-legacy-code-with-david-bernstein/627

Spike on Unknowns

As an Agile developer, I often start projects without a clear sense of exactly what I’m building. This used to be very difficult for me. I wanted to start with a complete specification for what I was to build, but today I know that it’s more efficient and effective to discover exactly what needs to be built just in time.

But this sometimes leaves me with a set of unknowns in my critical path, like boulders blocking the road in front of me, that have to be addressed before I can move forward. These unknowns can be anything from how an API behaves to finding a framework that does things I need to have done.

But that’s okay. When I find an unknown in my critical path, I simply spike on it.

Spiking is when one or more team members focus on answering one or more questions within a given time box. The goal is to answer the questions before the allotted time expires. If I can’t answer the questions then I’ll try to answer at least part of the questions and formulate a plan and timeline for answering the remaining critical parts.

I find that spiking on unknowns is an invaluable practice during iterative development. It’s easy to get lost in the weeds, especially when you’re doing research. Having a defined goal within a fixed period of time can be very helpful. When the time expires, I check in and see how close I’ve come to answering my questions. If I got satisfactory answers, then I move on. If I still need to do more research I will set up another spike.

Spiking on unknowns ensures that I don’t get lost for any length of time. I can always step back and reevaluate. Sometimes 80% of the resolution is sufficient and I don’t have to go for a total solution. Using time boxes to check in and evaluate my progress has become a useful practice that I have extended to many areas of my life.

If you hang around my office for any length of time, you’ll probably notice a Pomodoro timer going off every twenty-five minutes. Every time the timer goes off I stretch but I also check in with myself and ask myself three questions: “What have I learned? Where am I stuck? What am I going to try next?”

Unknowns are only scary if they catch you off guard. Spiking on unknowns is a way to prepare and address issues before they come up. Generally my spikes are anywhere from a few minutes to a few hours but I have spiked for several days in the past. I try to keep my spikes as short as possible because this gives me more opportunities to check in and evaluate my progress.

The key to a successful spike is to have the right questions to research and enough time to get good answers.

Buddy Programming

Sometimes the practices of Extreme Programming can be a bit too extreme when first introducing them to a team. This is especially true with pair programming.

Of all the practices that I teach software developers, I get the most resistance from pair programming, Both developers and their managers are skeptical about pair programming, although each have different concerns.

Managers tell me they can’t afford to cut their developers’ productivity in half but that actually doesn’t happen. Productivity increases for teams that do pair programming well, and for a number of reasons, but most fundamentally because when people who are working together they stay focused on a task more consistently than when they’re working on their own. This has several side effects, including fewer bugs, higher quality code, and better work habits.

The resistance I get from developers is different. Their main concern is often compatibility between partners but it usually turns out we can be most productive with people who are different from us rather than the same as us. It’s the differences between us that makes for a good pair. Our differences can often complement each other. And when everybody understands the ground rules and the purpose of pairing, it can grow into a very positive experience.

I know teams who pair on everything because they find so much value in it most of the time. Of the hundreds of teams I’ve introduced pair programming to, many of them continue to use the practices. Developers love pair programming when they’re introduced to it correctly.

The trick to getting a team to adapt pair programming is often just getting them to try it. When I encounter a team that’s reluctant to try pairing, sometimes I’ll suggest another technique, which I call buddy programming.

Buddy programming is not pair programming. With buddy programming you work on your own, by yourself, for most of the day, the way traditional knowledge workers typically work. But at the end of the day you get together with your buddy and walk through the code you each wrote since the last time you were together.

You’re checking in with another person, either at the end of the day or at the beginning of the next day for maybe an hour or so and reviewing each other’s work including decisions, designs, code, questions, etc. I find that this approach can often be a gateway for developers to adopt pair programming. They find it so productive working with their buddy that they do more and more of it throughout the day until it eventually goes from code reviews to code creation.

Some teams prefer to stay with buddy programming and use it as a way of enforcing code reviews among the team. But I have seen other teams soften to the notion of pairing and move from buddy programming to pair programming as the way they work most of the time.

Strong-Style Pairing

Pair programming is not about taking turns using the computer or being someone else’s typist. It’s about engaging two minds on the same problem so that a solution can be developed more quickly and with higher quality than if one person was working on it alone.

The two roles in pair programming are referred to as driver and navigator. The driver is the one with hands on the keyboard, It’s the driver’s job to get ideas into the computer and deal with the minutiae of typing. But the driver’s main goal is to trust and support the navigator. As the driver, I often think of myself as in service to the navigator. I still voice my opinion or challenge things if I have questions, but as the driver my hands are full, both literally and figuratively, so it’s hard for me to think several steps ahead. This requires being comfortable working with an incomplete understanding of the task, but that’s often a more efficient way to work.

The navigator, on the other hand, is free from the minutiae of having to type in code and so is able to see from a broader perspective. One of the main jobs of the navigator is to share information with the driver at the right level of abstraction. The overall goal is to present information so it’s easier for the driver to process. The navigator should give the driver the next instruction needed to implement just in time and at the correct level of abstraction so the driver can enter it most efficiently. The navigator is thinking about the problem at a higher level of abstraction so has a sense of the direction they want to go.

It’s the navigator that tells the driver what to do next, but what if the driver has an idea for what to do next? Simple, just pass the keyboard. This helps keep both pair partners in sync with each other.

There are several different kinds of configurations you can adopt when doing pair programming.

One of my favorite approaches to pair programming is what Llewellyn Falco calls “strong-style pair programming.” He defines strong style pair programming as follows “In order for an idea to go from your head into the computer, it must go through someone else’s hands.”

The best advice I can give both the driver and the navigator is to take a break and ask questions when you have them. When you ask questions of each other, you engage each other in meaningful conversation and you also help to understand each other.

These natural breaks are also ideal for passing the keyboard back and forth. We want to trade driver and navigator frequently throughout the day. Typically this is anywhere from just a few minutes to twenty or thirty minutes, though some purists would say that naturally trading the keyboard every two to six minutes is ideal.

When I see teams stuck in the roles of driver and navigator for more than an hour at a time I’ll sometimes introduce an external timer initially set for twenty-minute intervals so they have a reminder to trade off. Since timers can interrupt the workflow of a pair it isn’t ideal, but it helps form the habit of trading off frequently, which tends to boost overall productivity among the pair.