David Scott Bernstein's Blog, page 25

Story Splitting Techniques in a Nutshell

There are many techniques for splitting stories depending upon the situation you’re in. The goal is to break down work into the smallest increments that still provides some value, but the value can simply be a step in the right direction—as long as it produces visible results.

Most big stories are compound stories, meaning they’re made up of other smaller stories. If this is the case, simply break down the compound story into its component stories. This may have to happen repeatedly in order to break down a story to its smallest valuable component.

If the story includes unknowns, break out the known from the unknowns and begin iterating to the unknowns until they’re better understood and the domain of the unknown shrinks to nothing. In situations like these, it’s good to spike an unknown by researching specific questions in time boxes. At the end of the time box, evaluate your progress on answering your question. In this way you can eliminate the unknowns.

If a story represents a work flow or sequence of steps needed to accomplish a task, you can either thin-slice the sequence so you’re doing a very simple version of each step then devise additional stories for implementing each step in more detail. Alternately, you can create stories for each step in detail and string them together, if that makes more sense.

If the story acts on different data in similar ways, then you can make it work for a simple case and then have subsequent stories that work on the different data sets.

In some stories, it makes sense to write the happy path first and then write additional stories to handle exceptions, while in other stories it makes sense to peel off exceptions as individual stories that drive you to the happy path. Whether you choose one approach or the other depends upon the preconditions required and the complexity of the exception handling.

Regardless of the techniques you employ to split stories, the goal is that each story be a step toward overall completion and that it satisfies some acceptance criteria. It’s important to always have a clear criterion for when your story is done so you know when to move on to the next story.

And that’s it in a nutshell.

Splitting Stories

A number of Agile methodologies rely on artificial time boxes such as sprints or iterations. The purpose of these time boxes is to get you good at breaking down work so you’re only building a small piece of functionality at a time.

One of the critical skills in Agile development is being able to split up stories so they’re easy to build then assemble later. Small units are the core of all Agile practices so getting good at splitting stories is critical for being able to take best advantage of the Agile software development process. This is not some form of black magic nor is it a string of lucky guesses. Time boxing is a set of skills that, like all skills, need to be practiced to be mastered.

The goal is to build small units of value that we can deliver frequently to our customers. This should take place as we’re building features. Each task and each step in each task should be optimized so it provides maximum value as quickly as possible. The smaller the units we can work in that still deliver value, the better. Working in small units means we get feedback faster.

There are lots of techniques for splitting stories. Ideally, we like every story to just require one to four hours of effort to complete. But some teams starting out with these practices have a hard time imagining how they can create value for their customers in less than half a day’s work.

But always keep in mind that when splitting stories for development you don’t have to produce a complete, major feature. Ideally, we like to break down major features into a series of smaller features. Sometimes this is practical, in some situations it’s not. But in most situations Pareto’s Law does prevail and 80% of the value can be found in 20% of the effort—so it’s a good idea to identify what that 20% is and work to accomplish that first.

Technically Speaking Podcast

Here’s a link to the Technically Speaking Podcast I did at Agile 2016: https://soundcloud.com/teamdaugherty/david-bernstein.

In addition to talking about my conference session, my book, and the state of Agile, we also discussed my previous career as a filmmaker and the youth environmental video I made in the 1990s called Connect: A New Ecological Paradigm. This video was the 1997 MTV Earth Day Special and has inspired millions to care more for the environment. Here’s a link to the video, it’ll be the most inspiring 23 minutes of your day, so check it out: https://youtu.be/2EUu959IntA.

 

The Single Level of Abstraction Principle

It’s virtuous to separate out different perspectives in code because it helps make code more testable, extendable, and understandable. We want to use entities at the same level of perspective so that code is easier to read and understand code.

The same thing is true within entities where we want to do tasks at the same level of abstraction.

As human beings, we operate on many levels of abstraction. We think about the world in a series of nested abstractions that help us organize and understand the world around us.

We do the same thing in code. Well-written code is organized around abstractions that represent the main concepts of the domain. However, in software we usually focus on just one level of abstraction because that’s most useful and relevant when building a program, whereas when we think about things in the physical world we generally have many levels of abstraction that we could potentially consider.

One of the main shifts when going from journeyman programmer to a master is the move from thinking about creating programming constructs to creating behaviors in the system. We still understand and use programming constructs but we use them like a painter using paint and canvass as a means of conveying something more important. It’s important for code to focus on creating testable behaviors and hiding as much implementation detail as possible. This is how we make software more testable, more maintainable, and more dependable.

When we mix perspectives in code, when parts of the method delegate behavior and other parts of the method dive into implementation. It makes it hard to read and understand. If it’s done just once or twice, it’s a small thing and easy to overcome. But doing it over and over in code increases the viscosity of the software and makes it harder to make connections about what’s going on in the code.

When I write in API, some publicly available service, it only delegates. The only thing in that method is calls to other methods. It may have some high level control flow as well but I paid particular attention to keeping out any implementation details. If any of those methods are used only by this API then I’ll make those methods privates. Breaking these behaviors out into private methods gives me the opportunity to name them, and I try to give them a good intention revealing name that clearly describes exactly what the method does. I find this far more effective than writing comments in code because the code itself becomes expressive. This connects what the code does to the code itself rather than providing a separate narrative in comments.

Time and time again it has been shown that highly commented code is often a liability rather than an asset. Of course, there are many reasons for commenting code, and some of those reasons are quite valid. But if the code itself can express what it’s doing then that’s far more effective than a comment.

Keeping code within a method at the same level of abstraction helps ensure that there are no surprises when reading code. It helps us organize our code in ways that make sense and support the clear representation of the domain model. Following the single level of abstraction principle helps keep code focused around behaviors and easier to read and understand.

Automate Deployment

Different people have different ideas of what it means to be Agile. For some, Agile means doing standup meetings and two-week iterations. For others,it means doing all the practices from Scrum, Lean, and Extreme Programming.

For me, what it means to be Agile comes right out of the Agile Manifesto, which says: “Our highest priority is to satisfy the customer with early and continuous delivery of valuable software.”

For me that statement says it all. It states the goal of satisfied customer and the means of reaching that goal: continuous delivery of valuable software.

This is easier said than done in most situations. There are several prerequisites for reaching continuous delivery including writing testable code and a suite of reliable unit tests that can quickly regression test the system without any human intervention.

Why is this so vital? Because without the infrastructure of continuous delivery we’re doomed to build software in phases, which throws us back into Waterfall development.

Virtually every Agile implementation I see requires a separate QA effort on code that was finished in the previous iteration. That’s Waterfall.

We’ve known since the 1960s that as the time elapses between when a bug is written and when a bug is fixed, the effort and cost required to resolve the bug grows exponentially. But this is just the tip of the iceberg. Bug reports from previous iterations interrupt developers’ flow and force them to reacquaint themselves with code that is no longer in their short-term memory.

Having a separate QA effort can shift the responsibility of bugs away from the developer who wrote them and introduce a great deal of instability in the software development process. One bug can prevent a system from compiling or running properly and cost several more bugs to be hidden. When software isn’t tested as it is written, it’s most likely not being integrated immediately after writing it. And that means the most pernicious bugs often go unnoticed until software is integrated and tested. This is one of the big problems in Waterfall development.

The center piece of true Agile software development is in continuous deliverability, but that doesn’t mean we have to release continuously to our customers. Releasing code to customers is a business decision. But virtually all software should be built to be releasable all the time. Because only when code is integrated and tested as it’s built do we get an accurate measure of our progress and our risks.

It’s the infrastructure for continuous delivery that creates the context for all the other technical practices in Agile software development. I always recommend to teams that moving toward continuous delivery is the best place to improve their Agility.

Agile Amped Podcast

Check out my Agile Amped Podcast at Agile 2016 on Creating Implementations of Intent and What Makes Great Developers Great.

Linear versus Exponential

If I offered you a thousand dollars a day for thirty days, or a penny for the first day but I’ll double it every day for thirty days, which would you take? The former is a linear progression: 1000+1000+1000+1000… and the latter is an exponential progression: .01x2x2x2 . . . At the end of the thirty days with the thousand dollars a day you end up with $30,000. Starting with a penny, after thirty days of exponential growth, I’d have to pay you $10,737,418.20.

That’s the power of exponential growth.

This is very different from the way we normally think, and there’s always a learning curve. This is true of learning anything new, and what I’m proposing for building software is something radically new to most developers and business people. So if you only look at the first time increment in an exponential progression, it’ll appear that the linear path is winning, so we just see that linear is winning and go with that. But what we miss is the enormous growth that happens later in the exponential model. And this is the path of the software industry.

If you were playing the short-game and only looked at your rate of return for the first week you’d find that linear growth would have given you almost a thousand times more than exponential growth ($5000 verses $5.12). But as the power of compounding starts to take effect, the exponential growth quickly catches up to and surpasses linear growth so that by the end of the 30 days, exponential growth is over 350 times greater than linear growth ($10,737,418 versus $30,000).

Actually, we should be wired for exponential scales. For example, light, sound, gravity, and other natural forces fall off as a square of distance. In terms of sensory input, we tend to normalize exponential change into what appears to be linear. But somehow we seem to be wired to understand linear scales more clearly than exponential ones.

Linear growth is more in keeping with human experience. I get one year older every year, I don’t get twice as old every year. As creatures we tend to think in that way. If our brain is hardwired in any way it’s in a linear progression. When Ray Kurzweil, in his book The Singularity is Near, put forward that technological growth has followed an exponential progression, some people made fun of him for thinking that way, saying that that means in five years we’ll be living in some kind of science fiction universe…

But we are.

So how do you break through that thinking?

It’s easier to plan for linear growth. Exponential growth is frightening. But true innovation, true progress happens in exponential growth, not in linear growth. Innovation is by its very definition a discontinuous event. It is often not so much steady exponential growth as a sporadic series of quantum leaps.

Good practices can result in exponential improvements in software development. Bad practices can do the opposite. And I would say that we can’t have a software industry if we’re just linear in our thinking. We need an equivalent of Moore’s Law to provide pressure to innovate in software development the way the limits to transistor size provided—and continues to provide—pressure to innovate in hardware. I’ve come to see Moore’s Law as Intel marketing hype. It never really existed but the idea of scalability is important and something we are hoping for in software. But we’ll find it in different places than we found hardware scalability.

True innovation is exponential, but the reason we see linear growth in the software industry is that something’s pulling us back. And that something is the weight of our decisions—of our bad choices.

Those bad choices are being made by software developers.

It’s us.

I would love to lay the blame on someone else, but it’s our fault. That’s the bottom line. We’re writing the code. We wrote the bugs. We made the code unchangeable. It is our fault and we have to both take responsibility for it then recognize that there is a better way.

Telling Smaller Lies

People like certainty but we live in an uncertain world.

The future is uncertain and therefore it’s scary, so we tell ourselves stories about the future. We know they’re lies, but they’re good lies. They help us set our expectations.

We can’t help but lie to ourselves. If we didn’t, if we had to live completely in the unknown, then we probably wouldn’t get out of bed in the morning. Moving into the future with a sense of security is a good thing as long as we recognize that it’s a sense, an educated guess, and a projection to be validated and verified with reality.

As soon as we validate and get feedback, we learn the truth. The trick is to validate frequently, and when you get feedback, do something with the information.

I’ve seen a lot of teams do retrospectives and write up list of great things they could do to solve immediate problems, and then never get around to doing any of them.

If you are in a team that faces this problem then consider looking for small ways to improve that are easy to implement, and that can accumulate and compound over time. A few small changes consistently made over time can end up having a more lasting effect than a radical change that is tried and then abandoned.

If we can’t help but lie to ourselves then let the stories that we make up be empowering and drive us toward good action. We get to tell ourselves anything we want, so let’s tell ourselves stuff that ultimately makes us proud of ourselves and of our accomplishments.

A Culture of Inefficiency

The web site Mental Floss recently reprinted a series of helpful instructions written in Japan for Japanese tourists planning to visit America (http://mentalfloss.com/article/55140/...). These ten sometimes funny, sometimes confusing bits of advice ends with this:

“In Japan, there is great fear of failure and mistakes in front of other people. It is better to do nothing and avoid being criticized than to taste the humiliation of failure. As a result, there are things we wanted to do, but did not, and often regret.

“In America, you can make mistakes, fail, and it doesn’t matter. It is a fundamental feeling that to sometimes be incorrect is natural. In addition, rather than thinking about mistakes and failures, Americans have curiosity and say, ‘Let’s try anyway!’ ”

The Japanese are surprised by the fact that we’re not afraid of failure, even embrace it. Meanwhile, from a similar missive, the Russians are perplexed by our positivity, expectation of happiness, and our insistence on smiling all the time (http://mentalfloss.com/articles/63896...

“Americans and Russians say different things when faced with the same situation. Seeing the man who had fallen in the street, an American asks, ‘Are you all right?’ Russians will inquire: ‘Are you ill?’ We see a victim of the incident; they see survivors. Survivors are perceived as heroes. Where we ‘aren’t sick,’ they ‘stay well.’ We discuss the problem. They discuss issues and items on the agenda.

“However, it would be wrong to believe that the Americans with their smiles only create the illusion of well-being and that their smiles are stretched with false joy. This is not so. Americans: they are a nation that truly feels happy. These people get used to smiling from the cradle onwards, so they do not pretend to be cheerful. The desire for a successful happy life is inculcated from childhood.”

The common thread is that we Americans go about our lives with unbridled enthusiasm, smiling our way through mistakes in a constant effort to find happiness in everything we do.

So if Americans are perceived from the outside as being these hyper-enthusiastic, problem-solving, risk-tolerant people, why then do we, in reality, suffer from a culture of inefficiency, a culture of fear that says, “Oh, we could try it a new way, but what if it doesn’t work?” And I think we’ve all worked with the person who’s there to maintain his or her job—that’s what this person does in the organization, simply not get fired.

Even though we maybe are more risk tolerant than other cultures, we’ve certainly experienced the other side of it, and recognize the validity of caution—some sense of trying to do the right thing—not trying to fail, but accepting when we do.

We do understand that not everything we try will work perfectly the first time, but we need to get into the habit of failing fast, and failing in ways that will teach us something. We’re not going to try to do the stupidest thing we can, we’re going to try to take the best options we have available to us and learn a little at a time as we go, exploring the territory between start and done rather than drawing a straight line and trying to force our way through. By fixing the little mistakes as we go we can prevent those little mistakes from accumulating only to cascade into some sort of gigantic disaster that can’t be untangled at the end.

Incentivizing the Wrong Things

History is full of examples of institutions incentivizing the wrong things.

In 2007, in response to an increase in the population of wild pigs in the area surrounding Fort Benning in Georgia, the US Army sponsored a bounty program, paying local hunters from $25 to $40 per wild pig killed. All a hunter had to do to collect the bounty was present the pig’s tail along with a simple form.

But what the local hunters didn’t quite bargain for was that a team of Auburn University researchers led by Robert W. Holtfreter ran a population study of wild pigs in the same area, concurrent with the Army’s bounty program. And they quickly discovered that the number of pig tails turned in for the bounty didn’t quite match their field estimates of the wild pig population.

So they started making phone calls. They called ten local businesses that processed wild and/or domestic animals to see if they’d been asked to sell pig’s tails since the start of the 2007 bounty program, and if they’d sold any pig tails prior to the program. The researchers were eventually told that not only had one local business been recently asked to provide pig’s tails, but so had the other processors in the area, and the fact that “hunters” were buying tails from farm-raised pigs to turn in to the Army for the $40 bounty was “common knowledge.”

Of course, the Army couldn’t directly supervise every one of these private hunters, actually witnessing the legitimate killing of a wild pig in the bounty zone. They had to trust the hunters, and built only a simple, blunt instrument—their bounty form—to track where the pig was killed, when, and so on. There was no realistic way to check the veracity of the form, so at least a few of the locals started to game the system.

It’s fair to say that incentives drive behavior, so our goal should always be to develop processes in a way that incentivizes our teams to do the right thing. In the case of the Fort Benning bounty program, the motive for the hunters was money, full stop: Bring me a pig’s tail and I give you $40. But that didn’t necessarily align with the actual goal of the program: Reduce the wild pig population in the area until the population was normalized.

Are your processes encouraging software developers to stop working once a day’s arbitrary work goal is reached? Are your heavyweight processes encouraging developers to look for work-arounds, to spend their time looking for ways to get around the system in order to write quality code? Are you encouraging creativity in code, or creativity in managing your process?

It’s good to have goals, both short term and long. And it’s good to have certain metrics and some method to track accountability. But those goals, metrics, and processes should be balanced with the ultimate goal of enabling–of incentivizing–software developers do what they do best: develop software.