David Scott Bernstein's Blog, page 18

Think Small

The key to unlocking agility is to think small.

The whole purpose of building software in time boxes is to help us get better at scope-boxing, which is working in smaller pieces—breaking large tasks down into smaller tasks.

I’m not a big believer in using story points. It’s better to be able to break down all tasks into smaller tasks that can be accomplished in about half a day. When tasks become both small and homogeneous they’re much more straightforward to process. So rather than focus on task estimation, I focus on breaking down tasks into manageable sub-tasks. This gives a different focus to planning meetings. We spend our time working through examples of what we want to build rather than estimating tasks, which helps move a project forward more rapidly than spending a lot of time estimating tasks.

Being able to think small and break large tasks down into sub-tasks that are more manageable actually requires a whole range of key Agile skills and techniques.

I’m a big fan of behavior-driven development (BDD) or as it’s sometimes called acceptance test-driven development (ATDD). Unlike test-driven development (TDD), which is about building testable behaviors in the system, ATDD is about defining features and their acceptance criteria so that we know when it’s complete.

Automating acceptance criteria for features can be amazingly valuable for organizing a project and keeping it on track. Knowing rather than hoping we’re done with a feature allows us to move on and work on the next feature with confidence.

When I say “think small” I mean to think in terms of independently testable behaviors that are also independently verifiable so their acceptance criteria can be automated.

Abstractions are another way to think small. Abstractions let us put lots of similar things in the same category so we can treat them in conceptually similar ways. This is how we imbue our programs with meaning. Software, like all language, is a communication medium. It just so happens that software also has the side effect of making a computer do something. But when a programmer reads a program she is extracting a very different kind of meaning from that code than the computer does.

Because software is about creating behavioral models, every feature in the system can be seen as a separate behavior and every behavior should have some kind of an effect on the system. For example, you can receive some data, process it and change it in some way, then return the changed data. Or an object can simply change its own state or the state of another object in the system that can later be observed.

That’s important because without the ability to observe a change in the system we can’t verify that the software worked. And working with unverifiable software is a drag. I advise against it.

Instead, think small and build small, testable behaviors that are independently verifiable. If you do, you will be able to experience true agility. You will be able to automate product verification and thereby drastically reduce the time and cost of bringing software products to market. And isn’t that the Holy Grail of Agile?

It starts with thinking small and ends with building small.

Domain Language

Software is a model.

We’re used to thinking of models as physical models that occupy space, but software represents a different kind of model, not a physical model but rather a behavioral model. Behavioral models take place in time rather than in space. You can think of them as black boxes where some input comes in and then drives an output that goes out. What happens in between can be given a label and abstracted out.

Software is only clay. It’s a medium for creating virtual models based on our linguistic constructs and our understanding of the way things work.

Remember the myth of the Tower of Babel? Humankind wanted to build a tower so high that it reached up to Heaven, but God didn’t like the idea of associating with riffraff like us so he made everyone speak different languages. Because we couldn’t understand each other we couldn’t collaborate effectively and the tower fell.

Communication is central to collaboration. We must have a shared vocabulary and a shared set of standards and practices. Far too often, we use the same words but have very different meanings for them. We think in terms of our specific experiences, but our words represent generalizations. We’re constantly going back and forth between the abstract and specific, between the generalizations of what the words represent and our specific experiences.

This is where ambiguity and misunderstanding come in. Language is inherently ambiguous. But the kind of rules you need in order to articulate behavior in software have to be very precise. This is one of the reasons that requirements have failed as an effective way of conveying the features to be built in a software system. It’s far better to make software development a discovery process between the Product Owner or customer representative and the development team.

It’s nearly impossible to visualize an entire software system before it’s built. It’s far easier to visualize each piece as it’s being built with feedback from the development team. This kind of iterative development, where we’re constantly getting feedback from our customers and refining what we’re building so that we’re continually building the most valuable pieces, turns out to be a highly effective way for constructing software systems.

The key is to make the software that we build understandable by using effective metaphors that are centered in the domain language of the domain we’re working in. The domain language holds valuable information about the domain so we want to incorporate it into our designs. Our domain model should be understandable to domain experts even if they haven’t been exposed to programming languages.

If a class diagram of your system reflects an understanding of your domain that a domain expert would appreciate, you have an understandable design. If not, perhaps it’s time to refactor.

Writing Good Tests

As Misko Hevery, the author of AngularJS, says in his outstanding Google Tech Talk series “Clean Code Talks,” most developers assume they know how to write a good test but they don’t.

Knowing how to write a good test is paramount to being successful with TDD. Tests must be unique, expressive, and independent—easier said than done.

Doing TDD correctly is a discipline. It takes a deep understanding of a large body of knowledge as well as a lot of experience doing it. You can’t learn TDD from a 500-word blog post any more than you can learn brain surgery from one. It takes dedication, time, and practice.

I’m hooked on TDD but I guess I was predisposed to that since I’ve written automated test programs from the very start of my career as a developer and throughout the past three decades.

As I grow older my memory fades. This happens to all of us, although I didn’t believe it would happen to me when I was younger. Fortunately, wisdom replaces energy.

I lean on TDD to help me manage all those details I need to manage when I code. I rely on instant feedback from my builds to tell me if what I just did was a good thing or a bad thing. And with that kind of instant feedback, I make the connection between what I just did and how it affects my code.

Stimulus and response have to go together in order for an association to be made. If Pavlov rang the bell then fed his dogs an hour later they’d never make the connection. That’s why we don’t make the connection between the bugs we write when they’re found, weeks later, by QA.

But good unit tests help us make this connection by giving us immediate feedback whenever we make a change to the system. They also help us articulate the behavior we want to create and then validate that that behavior is working as expected.

Writing implementation independent tests by asserting against acceptance criteria or known behaviors gives us the freedom to change the implementation later without breaking our tests. Most developers weren’t taught how to do this, but writing good tests are central to success with TDD. Writing a test for every public method often drives us to write implementation-dependent tests, which can break when we refactor code.

Good tests are extremely valuable because they help you build out behaviors in ways that make sense and they support you in refactoring your code later. But in order for tests to have these amazing benefits and more they must be good tests. They must be unique, at the right level of abstraction, based on acceptance criteria, and be implementation independent.

I have discussed some of the major pitfalls to doing TDD in my book, Beyond Legacy Code: Nine Practices to Extend the Life (and Value) of Your Software, as well as on my blog. I also teach classes for professional software developers. Where you get experience doing TDD correctly you can see the benefits for yourself.

Early Automation Experiences

We didn’t do a lot of automated testing as an industry back in the late 1980s, but I did.

I was working for IBM on one of the first hypertext implementations, which later became HTML. Our authoring system created very complex data structures that required us to do extensive testing to validate they were correct.

I’m a lazy guy and the thought of running a series of manual tests—over and over, each time we made a change—to verify that the data structures our authoring system created were correct, was unappealing to me. So I wrote an automated test runner that was data driven. In many ways, my test runner was much more complex than the code it was testing. It took a long time to write and debug, but once I got it working, life was great. I could generate a new testing scenario in just a few minutes so I could test our data structures very deeply. And I found some obscure bugs I probably never would have found if I was only doing manual testing.

This was one of my early experiences automating testing, circa 1988. Three years earlier I wrote a data-driven report writer for the banking application I built. My test runners had a similar design.

Today, I wouldn’t think about writing code without building it test first. A suite of good unit tests is more than a safety net for catching bugs, they inform my designs and help me write code that’s more maintainable. Writing tests before writing implementation drives me to write functionally independent code that’s independently verifiable and testable. They keep me focused on implementing a single intention at a time, and they help separate tasks in ways we naturally think.

But unfortunately, when I visit clients or go to conferences and talk to other people who are doing TDD, I don’t always find the same enthusiasm for the practice. Many developers I’ve met have tried TDD and abandoned it. They found it impractical, that it requires too much overhead to maintain tests.

This is not a fault of TDD but rather a fault of the way people apply TDD. We have to know how to wield TDD correctly. When we don’t it goes from being an asset to being a liability.

Unfortunately, there aren’t many good books on doing TDD well. Kent Beck’s book, Test-Driven Development by Example is a great book but there needs to be many more great books on TDD.

Doing TDD well isn’t something you can learn from a blog post or in a few minutes. It’s a discipline that requires a great deal of study and practice. The path to doing TDD is fraught with misunderstandings and poor practices. People simply don’t understand what it is and therefore they use it improperly.

I’ve done my best to write about how to do TDD and other development practices, as well as try to explain some of the pitfalls. My book, Beyond Legacy Code: Nine Practices to Extend the Life (and Value) of Your Software, discusses the motivation, value, and purpose of TDD as well as several other practices. I’ve also written hundreds of posts on this blog about effective ways to build software, but there’s no substitute for doing it in an environment that supports you in building good habits, like my CSDE and ADET classes do.

Contact me if you’d like to learn more.

Changeability

What does changeability mean in the context of software? Software is soft so shouldn’t it all be changeable?

Not exactly.

It turns out we have to build changeability into code, but this is rarely done. To do this we have to understand the nature of software and the nature of change.

Software doesn’t behave like physical things. Software is a virtual thing and subject to the laws of the virtual universe, which are different from the laws of the physical universe. The lifecycle of virtual things is different from the lifecycle of physical things as well.

We talked about “bit rot” but of course, bits don’t rot. Software doesn’t change on its own. In fact, it’s one of the few things in the universe that doesn’t decay. Relative to the rest of the world, which is in a constant state of change, it only appears as if software is rotting or going stale. Our needs are ever-changing and in order to keep up with those ever-changing needs we have to write changeable software.

An NIST study showed that 80% of the cost of software happens after the initial release. On average it costs half as much to build software as it costs to fix bugs that escaped into production after release. The reason it costs so much to maintain and extend software is that we don’t write changeable code. We create un-cohesive classes and methods that are concretely coupled, leak implementation details, produce unwanted side effects, and repeat code found in other places. In short, we write dirty code.

I’ve written a lot about what I call CLEAN code qualities like cohesion, loose coupling, encapsulation, assertiveness, and non-redundancy. These five code qualities underlie all good software and help make code more changeable. Likewise, “Uncle Bob” Martin’s SOLID principles also support changeability. The disciplines that comprise Extreme Programming (XP), like test-first development and refactoring, are all about supporting changeable code.

Why? Because code needs to change but changing it can cost a small fortune. But if we write code that supports changeability, when the inevitable happens and code needs to change, we can handle it with less complication.

The biggest impediments to changeable code, aside from code that lacks CLEAN code qualities and SOLID principles, is code that’s not supported by unit tests. This code is hard to change because there’s no fast way to validate that it works after changes are made.

Bad developer practices are rampant in our industry and there’s very little consensus on the “right” way to do things. This is because there is no one right way to do things. There are many equally valid ways to approach writing software, but we need touch points that let us build software the way we want to but still allow us to easily integrate new features and refactoring into the system.

I believe we already have a solution to this problem and it’s called test driven development. But not just any test-driven development, the kind of test-driven development that comes from using tests as specifications. And that is the subject of my next couple of posts so stay tuned!

 

An Object Model

If you were to draw out all the objects in your system and their relationship to each other at runtime, that would be a graphical representation of your object model.

Object-oriented programming is about modeling the behavior of entities. It is a programming paradigm that, when used correctly, helps us build resilient systems that are straightforward to understand and extend. But we gain these benefits only by correctly using the object-oriented paradigm, which involves having a domain model that reflects what’s being modeled.

Designs often start out with a good domain model but then degrade over time as new features are added. Features get hacked in and new behaviors are bolted onto existing objects rather than expanding the object model to include these new classes. This distorts the object model and makes it more difficult to understand. Behaviors, and even entirely new classes, can be hiding in long methods. Pulling them out makes the code clearer and cleaner.

Object-oriented programming languages allow us to define classes and instantiate objects, but I find some developers resistant to doing that, having the false impression that creating objects is expensive and makes a system less efficient. But creating object instances are cheap, and modern runtimes are optimized for this. We should be defining classes and making objects all the time.

Any group of behaviors that converge around a set of values or state should be defined as a class. Since a class can represent anything, we can think of it in many ways, but most fundamentally as a concept that aggregates behaviors, often around a common set of instance data. For example, a savings account aggregates behaviors like deposits and withdrawals around an account balance.

Classes and objects, their runtime representations, can contain data that represents its state. They can also contain methods that represent its behaviors.

Though classes may only contain data and methods, object-oriented languages provide a rich environment for modeling anything. Classes can represent anything from tangible objects to ephemeral ideas. They can be a part of a larger whole, or they can represent the relationship between different entities.

Classes can be anything.

And there’s the rub.

We have to name our classes well. If we don’t create good, intention revealing names, our object model becomes distorted. If we stuff behavior into another object that belongs in its own object, the object model gets distorted. If we fail to call out classes in our domain or spread responsibilities across multiple, unrelated classes, our object model gets distorted.

When an object model gets distorted it becomes hard to read and understand. Worse still, a distorted object model will often lack flexibility precisely where flexibility is most needed.

Going against a design to special case features can often lead to maintainability issues, so we want to keep our object model robust and clear. We want it to reflect reality: the thing we’re modeling. The way we do this is by calling out classes when we find them. We must actively look for classes and continue to expand our object model as our knowledge of our system expands. This is how we keep a system maintainable.

Acceptance Criteria

Acceptance criteria are simple tests to tell us when we’re finished implementing a feature.

Knowing when we’re done building a feature is extremely important to developers because we’re not always sure how robust a feature should be.

We don’t like it when our code breaks in the field so we want our code to be robust, which often leads us to overbuild in one area, taking time we need to give other areas the attention they deserve. Knowing when we’re done with one feature helps us move on to the next with confidence.

Acceptance criteria should be seen from the perspective of how the user evaluates a feature. It can be jotted down as a conditional statement on the back of a story card, or it can be automated with an acceptance testing tool like FIT, FitNess, SpecFlow, Cucumber, etc. These tools help us create acceptance tests in a form that the computer can execute while still being easy for a non-developer to read.

But there’s another major benefit to coding to acceptance criteria. It keeps code focused on providing something useful to our users and puts acceptance tests at the right level of abstraction.

Creating the right tests for code is paramount. If tests are too fine grain then we’re likely writing implementation-dependent tests. If they’re too broad then we’re likely to miss edge cases and other exceptional conditions.

Good code depends on building it at the right level of abstraction, and the right level of abstraction is often defined in the acceptance criteria itself. When we write our acceptance test around the acceptance criteria of a story and our acceptance test passes, we know our story is finished.

Just like literature, code can tend to ramble when it should be succinct. When code is focused around passing some acceptance criteria then it’s more straightforward and to the point. We spend far less effort building cruft and put far more effort into code that provides value to our users. We also end up with a more assertive system that’s more straightforward to test and extend.

Another important aspect of coding to an acceptance criteria is that it helps us write code that produces an observable result, which means it’s generally more testable and modular. Acceptance criteria speak to the very core of what makes a feature valuable to the user. Keeping acceptance criteria in mind as we’re building a feature helps keep us focused on producing code that provides value.

“How will we know we’re finished?” is the question we should always ask before we begin building a feature. It’s starting with the end in sight. Defining this up front ensures that everyone starts on the same page with what’s being built, and stays there until it is built.

Use Examples

Requirements have gone from lengthy specifications that state “The system shall…” for dozens of pages, ad nauseam, to a single sentence story. But stories aren’t meant to capture all the details needed to build a feature, so where do those details come from?

Alistair Cockburn describes user stories as, “a promise for a conversation.” Stories move requirements from a written document to a conversation between the Product Owner and the development team. They aren’t meant to replace requirements, they’re meant to help create a context for requirements, which are then discovered by the Product Owner and development team working together.

This has several advantages. An NIST study found that more than half of all development effort is spent writing and interpreting written requirements, and more than half of all bugs in a system can be traced back to poor or inadequate requirements.

Turning requirements into discovery helps us build a better product. It keeps us focused on finding solutions throughout the development process instead of just following a plan. It helps us constantly ask if there’s a better way to do things, and that often leads us to better results.

But there’s a challenge to using stories. They speak in generalizations. For example: “As a moviegoer, I want to purchase tickets online so I can avoid lines at the theater.” This story tells me what is wanted, who it’s for, and why they want it, but it lacks specific details.

So how do we go from stories to code?

There is no straightforward answer to that question. A lot has been written on story breakdown but much less has been written on how to code a story incrementally.

One trick that I use when coding stories is to start with an example. Examples are concrete, so they’re understood in a different part of the brain than generalizations. They also help bring up different issues so we can understand a feature from different angles.

We think concretely, so working through simple examples can ferret out issues that I may not have noticed until I was in the middle of coding. Examples also let me get a strong sense of the feasibility of an idea before I get too deeply committed to a particular solution.

For me, examples are the middle ground between stories and code. That’s why I love the book, Specification by Example: How Successful Teams Deliver the Right Software by Gojko Adzic. This book discusses how to use examples effectively for specifying what behavior to build.

Rather than try to come up with exhaustive examples of a system, the approach that Gojko Adzic advocates is to find key examples that cover the main concepts. This keeps examples more general-purpose so they’re more straightforward to repurpose in the future.

Good examples, like good unit tests, should be as implementation independent as possible. They should cover the main cases but not be concerned with covering all exceptional paths. The result is a set of key examples that are easy to read and give you a sense of the essence of a system.

Developers Love Development

We software developers are a fortunate lot. We love what we do, and a lot of the time, when we’re actually building software, we’re very happy people. Unfortunately, this isn’t universally true since so many software development projects require developers to do a lot of other things aside from writing code such as sitting through boring meetings and building documentation or other artifacts. This is not as enjoyable to us as facing the challenge of solving unknown problems, learning and discovering as we go. This is what attracts us to writing software.

Professional software development is fundamentally different than most people think. It has nothing to do with using software or networking or virtually anything else. Building software is an immensely creative activity.

One of the questions I ask almost every group of students I teach is do they think writing software is more art or science. The majority of developers say that although there are elements of both, that software development is more art than science. By this they mean that there’s more creative and abstraction skills required to be a good software developer than there is explicit process. Of course, it takes a tremendous amount of discipline to write even the simplest of programs but this is an area where we get addicted to growing. Developers love the challenge of building things that have never been built before, solving problems and providing tools that improve people’s lives.

We really don’t conform to the typical stereotype of the antisocial nerd or geek who found friendship in silicone circuitry. Modern software developers come from all walks of life and all backgrounds.

\

Writing software is perhaps the most engaging and challenging profession of all. It takes a great deal of study to become a brain surgeon, for instance, and there’s a lot of pressure in having someone’s life in your hands. But a skilled surgeon draws on a very limited skill set. The most important skill for a good surgeon is to be dexterous and able to tie really good knots. Perhaps this is one thing they have in common with sailors. A good lawyer is able to present a compelling argument but again, they rely on a very limited set of skills including reason and persuasion.

Software development requires a greatly diverse set of skills and we have to be good at all of them to be successful. To design software takes a tremendous amount of creative visualization—after all, we’re using our imagination to understand a problem and model its solution. To write software requires a great deal of tenacity—we must keep track of a large number of details and use a whole variety of techniques to manage the enormous complexity of even a relatively trivial program. To debug code takes exceptional analytical skills—completely different from the skills required to design software, but developers must be good at both. As a result, we tend to use both sides of our brain in the process of building software, and this can make for a highly satisfying experience, but also a highly challenging one.

I’ve asked a number of non-software developers what they think the process of writing software is and I’ve heard of lots of different answers, none of which are anywhere near what the process really is about. I supposed that’s true for other fields. Professional acting is much more than just pretending. Great actors become their characters and transform into another person. It’s a great skill to have and certainly very few people have it, but what they do is not anything like the title of their profession implies. They are not act-ors, they are “be-ors.”

I know people who have gone into the restaurant business because they love the process of sharing food with their friends. But preparing five hundred meals a day is very different from sitting down with your friends and enjoying a scrumptious dinner. Chefs are some of the hardest working people of any profession. It’s a highly intense environment, which is why a lot of people drop out of the profession. I think a lot of people in a lot of fields feel like they have to make compromises because that’s just the way life is, and perhaps it is for many of them, but software developers still find fulfillment in our work every day we actually get to build software.

Of course it takes an enormous commitment and it’s certainly not easy to break into the profession. Most of the developers I know learned what they needed to on the job or through a lot of self-study. As such there’s a huge divergence in skill sets and knowledgeability in this industry. There is no set of clear standards so it can be challenging to work on teams when everyone has their own ideas on how to do things.

Writing software is a group activity. Most software development projects do not have coders siloed from each other just turning code out. Most of the code built for business today happens in a subdued environment that requires entire teams of individuals to work together. Of course, developers aren’t really known for their social skills, but a lot of that is changing as we realize the desperate need for communication among teammates.

How do you evaluate a design?

This is a question I often ask software developers in my senior advanced software design classes. I tend to get blank stares in response, not because they don’t know how to evaluate a design but they rarely have a common rubric to apply. This can be a challenge for teams, making it hard for us to communicate and collaborate when building software. So I spend a lot of time in my classes defining terms that will serve us in being able to evaluate what’s virtuous in software designs.

Developers love my classes because they immediately recognize the value of talking about and thinking about these things. I’ve had the chance to work with many senior software developers and I’ve made it my mission to find out why the successful ones are so successful then I share that with other developers I work with. It seems as if we each have a piece of the puzzle, and when we put it together we get the big picture. That’s precisely what I do in my work, which is amazingly satisfying.

Podcast on Legacy Code Rocks

Join me and the owners of Corgibytes, Andrea Goulet and M. Scott Ford, on their podcast Legacy Code Rocks where we talk about how to avoid legacy code:

https://www.stitcher.com/podcast/legacy-code-rocks/e/51642611?autoplay=true