David Scott Bernstein's Blog, page 13

Object Thinking

We refer to object-oriented programming as a paradigm or way of thinking because it’s more than just a set of techniques and practices. It’s a different way of modeling behaviors. It’s not necessarily as easy to learn as the syntax of a programming language.

Object-oriented programming is subtly different than procedural programming and it’s easy to overlook those subtleties if we’re not paying attention to their benefits. Really, the goal of good object-oriented code is to make the software accurately reflect the important features of what we are modeling. In other words, we should design our software to reflect as closely as possible whatever it is we are modeling.

We create models by defining classes and their relationship to each other. These are the “paints and brushes” of software development. They allow us to model anything—anything that we can name. It’s important that our domain model be clear, consistent, and complete.

It seems that wherever we find “bad code” it is a product of poor understanding. We must understand the problems that we’re trying to solve and model them accurately. When I say understanding, what I mean is that we are able to see how to create a behavior in a system through the interaction of objects. It can be easy to identify the need to create a new behavior but not so easy to identify where in the domain model that behavior belongs. However, keeping a design clear as it evolves is critical for maintainability and future extensibility.

To “codify” something is to understand it deeply enough to be able to express it in code. Writing software requires attention to the most minute details. If we are able to embody a process in code then we understand it in great detail. But just translating a process into code is often not enough to make it maintainable.

Good software is also straightforward to work with and extend. Good software encapsulates behaviors from each other through a well-designed object model where each object in the system has a single, well-defined responsibility. That way, when a change happens, it only impacts a small number of simple objects in the system, thus minimizing its impact overall.

However, functional decomposition is only part of what’s important in an object-oriented system, we also need to organize code in ways that make it straightforward to extend. We do this by recomposing or calling out commonalities between behaviors with abstractions and design patterns. Together, these two approaches, decomposing and recomposing, help us build more maintainable software that costs less to work with and extend.

There are many ways to write software. Conventional wisdom for development practices is totally different today than they were just a few years ago. This is a good sign that our industry is maturing rapidly and we are finally gaining the right insights to be able to create more consistent success throughout our industry.

The Virtues of Laziness

Ok, I admit it. I am lazy.

But laziness can be a virtue. I think it is humanity’s basic laziness that let us create computers in the first place. Computers take the drudge-work out of many tasks.

My basic laziness has motivated me to find better ways to build software because I simply got tired of maintaining poor code.

Over the years I started to get a feel for what kinds of enhancements were difficult to make in a system and why. It turned out that I would see the same patterns over and over again play out in each of the code bases of my clients.

Systems were frail and difficult to change. When developers tried to make a change or add a feature to their system it would become unstable.

Physicians use the term pathology to describe the symptoms of a particular disease. Poor code can also be seen in terms of pathologies but they are not caused by a disease, they are caused by one or more code qualities that are missing.

So what is code quality? It’s easy to identify quality in material goods or services but quality in software is more difficult to define. Some say the quality of software is meaningless because the user has no direct experience of it.

I beg to differ.

Users of software systems may not see the code but they definitely experience it. Even if the feature performs well initially, it may need to be modified or enhanced later and the ability to do this easily and without breaking other parts of the system directly relates to code quality.

I usually talk about six key qualities that code should have or be: cohesive, non-redundant, encapsulated, assertive, testable, and explicit. I see them as facets of the same gem. One leads to another so even if you only focus on one or two you are bound to get improvements in all of them.

Kent Beck tends to focus on eliminating redundancy and in doing so tends to improve the overall design of his code. I like cohesion because I find that if I can name something well it is usually cohesive. Different people gravitate to different code qualities.

You don’t have to focus on all of them. Pick one or two that make sense to you and focus on those. When you improve one and you see the other code qualities improve then you know you’re on the right track.

I don’t consider laziness as a quality but maybe as a motivator. I’ve found ways to automate redundant or repetitive tasks because I’m lazy. I hate redundancy in my code so you know I especially hate redundancy in my life. If I have to go back to the store because I forgot to buy something I needed—I hate that. So, I try to find ways of streamlining the work that I do so that I’m automating the rote tasks and focusing on doing the creative stuff.

Rebuild or Refactor

Sometimes, when an important project is going poorly there’s a desire to start over. Sometimes this comes from management but often this comes from the developers themselves. They say if they only had a second chance and could start over then they can build the right system.

But that almost never happens. Take it from me. I’ve seen companies try many times and I can say that without exception, when a team sets out to rebuild the same system with basically the same approach, they end up with roughly the same system the started with, including the same problems only this time they have two systems they have to maintain.

Boy, I could tell you stories…  Want to guess how many CRMs Microsoft had under development in the early 2000s? At one point, I counted a dozen. Twelve projects that basically did the same thing!

Legacy code gets a bad rap. People are scared to touch it but it embodies business rules and procedures that are time-tested, and even the most intractable legacy code may still hold value when used to refactor a system.

Refactoring gets a bad rap as well because developers and managers are unfamiliar with it. They don’t know that there are safe ways of reorganizing their “big balls of mud” that their software has become into more manageable chunks that are independently verifiable and therefore less costly to maintain and extend.

Refactoring legacy code is often a process of breaking it up. It might have been written procedurally and worked fine for years but then, in order to improve the build and support continuous integration, code needs to be broken up into independently verifiable parts.

Don’t worry. The hard part is over. The software does what it needs to do so refactoring it is really just a matter of reorganizing it. That doesn’t mean it isn’t easy or risk-free but there are ways of dealing with the issues that come up and the benefits of making these changes can oftentimes give companies the competitive edge.

Think of it this way, most software is written procedurally and takes a global perspective. That’s fine for simple programs but as systems grow their complexity skyrockets so we needed a way to manage that complexity.

Object-oriented programming gives us a way of managing complexity by taking the behavior we want to create and putting it in a collection of “objects” that interact to create that behavior.

By doing this, we hide different parts of the system from each other. Instead of having one global perspective, object-oriented programs are composed of a collection of objects that interact to create the desired behavior.

By taking the additional step of encapsulating behavior into the right objects, we allow our systems to become more modular and independently testable through automation, which drops the cost of change for the software we produce.

This is by no means an easy task in most situations. Your legacy code got that way from years of neglect and it may take a bit of effort to clean it up but if it already does the right thing then often it’s a matter of reorganizing and restructuring the code so it’s in different places but the functionality remains the same. Reorganizing code is usually more straightforward than rewriting it from scratch.

Buy or Build

One question that I hear a lot of people asking is whether they should buy or build the software that they need to run their enterprise. This is often a difficult question to answer. One thing I can say having lived through many major software purchases is that the main cost was understanding a system that was purchased and that cost of understanding turned out to be far greater than expected. Large, expensive software products are often more costly to purchase then the price tag would indicate.

If you can gain a competitive edge through embodying it in software then it’s almost always better to build rather than to buy. This applies to entire products. Technology companies are sometimes purchased because of their customer base or inroads into a particular market segment or any number of other reasons. Of course, I’m not talking about off-the-shelf software. I’m talking about integrating essential components that were not developed in-house.

When we purchase a codebase we also often purchase all of the headaches and shortcomings in that system. It’s very difficult for most engineers to discern where a system’s strengths and weaknesses are and therefore they are not able to accurately evaluate the value of the system. Of course, a system’s value depends on many things, not least of which is the market, but the quality and flexibility of the system and how it was written is certainly an important aspect of its value.

On the other side of the spectrum, developers often “borrow” snippets of code from the development community to build projects rather than reinvent the wheel each time. The Open Source movement allows developers to reuse other people’s code and learn from how they do things. Thanks to the Internet we can access source code in virtually any language and draw from repositories that let us quickly build systems.

The main problem with buying a foreign codebase is that you can get the source code and even the documentation but the process of building the software gave us the greatest benefit, which is understanding the problem so that we could build the solution. Without gaining that intimate understanding of the problem needed to code a solution, software is often not as valuable. Software that is opaque is difficult to change in the ways needed to fix bugs or add features. Without the ability for the software to change, its usefulness is often greatly reduced.

Outsourcing software development is the same problem. Often the understanding gained through building software is not propagated throughout an organization when it outsources its software development as broadly as when development is done in-house. Outsourcing often works best when remote teams partner with local teams and do things like pair programming and writing acceptance tests together.

Software development is engagement and discovery. When you’re building something that has never been done before it is useful to get as much feedback as possible so building a system in-house makes a lot of sense. If what you need is something that is fairly common and you’re certain you can find exactly what you need then it makes sense to buy it.

More on Test-Driven Development

One of the most valuable development practice that has become popular recently is test-driven development. When done correctly, unit tests can dramatically drop the cost of maintaining software while increasing its value. All the things that management wants and needs from the processes that are built around software development are embodied in this simple practice including consistency, reliability, and quality.

Code that has good unit tests tell us that it’s working correctly now and also tells us it’s working correctly in the future. Tests allow us to not only exercise the code in exactly the same way the application uses it but also documents its usage so we know exactly what to expect from the software that we write. Unit tests are contracts that express the expectations of the software they’re testing. Having a suite of unit tests in place for code allows us to automate a great deal of regression testing.

We need to eliminate the regression testing delay between the time a developer writes a defect and the time it takes to identify it. If our regression tests can identify defects then we must run them frequently to get rapid feedback. With this immediate feedback we can begin to see that many software development practices are not so good. Even with a very rudimentary continuous integration system and coarse-grained tests, developers can get immediate feedback on integration errors and thereby rapidly reduce the time to correct.

Developers begin to write code differently and that improves the quality of the system being built. Development teams that use continuous integration also build software differently and if they have good unit tests they have a high confidence that their code works as expected. Continuous integration with a reliable suite of unit tests is perhaps the most important practice to implement of all the Agile practices and it possibly provides the greatest value.

But as valuable as unit tests are for the software we write, they are even more valuable in the process of designing the software in the first place. The practice called test-first development has developers write a test even before they write a line of production code. It turns out that writing the test first is an incredibly valuable discipline for designing testable software and has become widely adopted within the development community.

I want to stress that test-first software development, contrary to its name, is really not about testing. Test-first software development is about designing software so that it is testable and understandable. Test-first software development is one of the more advanced development practices. It requires that the team understand and use the discipline for building software.

A lot of developers say to me that they don’t have time to write test code as well as production code but it turns out that doing test-driven development, even though you are writing significantly more code, is often far faster in the long run because of the things you are not doing.

Doing TDD, I spend far less time debugging code because my tests find problems in my code before then can become defects. I also spend less time reading and interpreting requirements because my tests allow me to build code with concrete examples, which is often more straightforward than trying to write code from abstract requirements. Most importantly, my tests give me confidence that my code still works correctly when I refactor or enhance my software and there’s no way to overestimate the value that confidence gives us.

Code Transformations

A lot of poor designs can be attributed to sticking with an existing design as changing requirements show us the need for a better one. Oftentimes, an initial design is just a stab in the dark. We might not know enough to make an informed decision but we have to get something done, so we do what Agile says and we code up the behavior that we need right now and not worry about future requirements.

For most teams, the problem comes when they start to enhance that behavior and go back into the code to extend it. Now they’re asking the system to do something that it couldn’t do before and, instead of redesigning a feature to accommodate the new behavior, developers might try to hack in the new behavior while minimally impacting the existing design. But this can degrade the quality of the code when done over and over again in a system.

What if we don’t have to get it right the first time? What if we could start off with a crappy design but improve it over time as we learned more about how the design should be and what behaviors it should be able to accommodate?

It turns out that often a “learn as you go” approach is more efficient and effective than the “figure it all out upfront” approach. It turns out that most of the time, in order to go from one design to another design in code requires a minimal effort. Code refactoring is a reliable, safe and effective way of transforming the design of software, which we typically need to do in order to accommodate new features.

We often stick with an existing design because it’s what we know but this is something that’s going to have to change in our industry. We have to be willing to change our designs in light of new information and pay off technical debt when we do it.

The book Refactoring: Improving the Design of Existing Code by Martin Fowler is an essential text for all developers. It teaches us about common transformations in code that allow us to clean up existing software so that it’s more extensible and reliable.

Reading this book and learning how to refactor code has been very freeing for me. It means I don’t have to get the design right the first time. In Agile, that’s rarely possible anyway because we often start building software without a complete set of requirements so we’re always dealing with unknowns. Knowing what it takes to go from one design to another in software means that we don’t have to get it right the first time. It means we can start out with the wrong design and change it later without paying a high price.

So, our initial design doesn’t really matter, as long as we can change it later to a better design without a lot of rework. This takes the pressure off of us to get it right up front and is actually a very efficient way of building software. It means that we can emerge features as they’re needed and this turns out to be a highly effective way of building many kinds of software systems.

Work Through Examples

One of the biggest challenges in building software is specifying what needs to be built. A blueprint captures all of the valuable information we need in order to build a building and details such as the tensile strength of the material and how to build a foundation are available through other sources. Blueprints have no ambiguity. They are clear specifications.

But software is much more difficult to specify than the structure of a building. Programming languages are fundamentally different than spoken languages and there’s rarely any one-to-one mapping between them. We speak in generalizations much of the time, but computers are very, very specific all of the time. This is a challenge in the way we think and communicate.

Communication by its very nature is a serial activity. When we describe processes we do so sequentially. This often implies a procedural approach and can lead to overly concrete requirements when we talk about software.

Good programmers know that there are other approaches to software design than procedural and that modeling software with objects can improve maintainability. But object-oriented programming adds additional levels of abstraction and complexity in order to give that additional flexibility.

All of this can be very abstract and so I find it incredibly helpful to work through examples.

Rather than spending time laboring over requirements, I now think of a couple of examples I can work through with the Product Owner and developers so that we get on the same page for exactly what a feature should be. I often like to start with a simple example—the simplest that I can think of. Once I get a sense of how I can work through that example, I have a much better sense of what I need to code.

For example, instead of saying, bids from logged-in users higher than the current bid become the high bid on active auctions, we can say, given an active auction and a logged in user, when the bidder bids over the current bid then they become the high bidder and their bid price becomes the current price.

Examples concretize the abstract requirements and shift the conversation from “what if” to “what is.” This is really important because we can spend a lot of time in “what-iffing.” And that time is usually wasted—the future is anyone’s guess. Instead, if we focus on solving a real problem that is in front of us then we can take the idea out of the hypothetical and bring it into the immediate, which is where our brains are used to living.

We naturally think in examples. When I give you a generalization, such as all people have feelings, you visualized a specific person, consciously or subconsciously, in order to make sense of my words. Since we always think in examples, when we specify behaviors with examples, it’s more straightforward to understand and there’s less room for ambiguity.

I find examples especially valuable for helping to ferret out some of the details that would otherwise need to be answered during development. In my planning sessions, we often spend our time working through examples rather than making estimates.

Good examples should specify generalizations and not be too specific. They are not meant to capture every little detail or edge case. They are simply meant to call out the main distinctions in a system and keep us focused on what does and not how the feature does it. I find that working through examples helps developers and Product Owners quickly get on the same page and understand a feature in enough detail to start building it.

Making Code Testable

I believe that testability is one of the key characteristics of good, maintainable software. But what do I mean by testability?

Testable code is code that’s written in such a way that it is independently verifiable. It has a well-defined programmatic interface and it can be fully tested based on that interface. Testable code receives dependencies as input parameters so that during testing fake dependencies can be injected instead. Testable code is made of small and functionally independent behaviors that make up a system.

When I say code should be independently verifiable I mean programmatically. In other words, we should be able to write code that exercises the specific behaviors that we want to verify.

Testable code is well-defined code so that specific inputs creates specific outputs. This often forces us to separate out our business rules from the way we display them and/or process them. This is a good thing because business rule shouldn’t be intertwined with the user interface or at the persistence layer. Pulling out business rules from UI or the persistence layer makes code more testable.

When objects obey the Single Responsibility Principle and do just one thing then testing is far more straightforward. Combinatorial explosion is one area where testing can be difficult. To get around this, reduce the number of code paths through each method. This is also known as cyclomatic complexity.

A method with no conditional logic has a cyclomatic complexity of one because there is only one path through the code. Therefore there is only one test that’s required to verify the code. If a method has one conditional statement that alters the path of the code in two different ways then we would say that it has a cyclomatic complexity of two and there are two tests required to verify or validate the code.

This can grow very quickly. Two conditional statements mean four paths through the code. Three conditional statements mean eight paths through the code, four conditionals mean 16 paths. It grows exponentially.

We want to keep the number of code paths in a method small so they are straightforward to test. Unit tests should exercise every code path which means that every code path should have its own unit test. Therefore, I try to keep the cyclomatic complexity of my methods as low as I can.

It’s also shown that the higher the cyclomatic complexity, the more prone a method is to have defects. That’s only natural because as the complexity of a method grows, our ability to manage it diminishes.

Sometimes people ask me how do you test the user interface and by and large my answer is that you don’t. If you build your user interface elements without business rules so they’re dumb then you don’t need to test them. Extract out the business rules from your display elements and put them in their own testable objects. Not only does that help make UIs more pluggable but it also allows us to swap out different kinds of UIs if we choose. So, we can have a web-based UI or a Java-based UI, etc.

When I’m testing code I want my test to be meaningful, of course. My test should be based on some meaningful behavior that’s created in the system as a result of the thing that I’m testing.

We often use the syntax of set up, trigger, verify. The setup puts the system into a state where we can run the test. The trigger triggers the behavior that we are trying to test. The verify verifies the result against some expected results. This approach to unit testing is very powerful, clear and straightforward.

I find that when I think along these lines and build software along these lines that not only do I make my code more testable but I also make my code more focused and I write far less cruft and far more code that’s about fulfilling acceptance criteria. And that makes my users happier.

Complex Systems

When I look at the systems around me I noticed three kinds. There are simple systems, there are complex systems, and there are complex adaptive systems.

Most of us are familiar with simple systems. They have a direct cause and effect relationship that can be easily traced. Other systems are less direct.

Complex systems are systems that have emergent properties. They don’t follow a simple cause-and-effect relationship. The weather is an example of a complex system. Complex systems often sit on the edge of chaos. I find that very few people understand the nature of complex systems.

Finally, there’s a third kind of system that I notice in the world, which is complex adaptive systems. Living organisms or communities of organisms can have this emergent property. Interestingly, biologists are quite skilled at dealing with these kinds of domains.

Currently, computer systems are complex systems. But many people think of computer systems as simple systems. As a result, there’s a great deal of misunderstanding about computer software and how it is supposed to operate.

Complex systems are defined by how they’re organized. Their elements are oriented in ways that make them most accessible for their desired purpose. Therefore complex systems have a purpose or tendencies toward certain behaviors. Understanding the nature of complex systems will be an important part of building software in the future.

Complex systems, like software, have a lifecycle and we’re just beginning to tune in to the importance of the lifecycle in our software. As we learn more about how to build maintainable software, our code will be able to be more easily repurposed in the future as the needs of the user changes.

Complex systems are composed of simple elements that are combined in complex ways. If we have three different choices, each with two options then we have 2^3rd or eight potentially different outcomes.

This can grow rapidly. For example, 10 options mean 1024 outcomes, potentially. That’s 1024 tests required to guarantee that we’re combining these options in ways that are valid.

Most programming languages give us the capability of encapsulating both data and behavior so we can better manage combinatorial issues. This is one of the most powerful aspects of objects because we can guarantee that they only interact with the rest of the system in predefined ways. This limits the kind and number of bugs that can be introduced to a system and it also limits our testing requirements.

If we can break complex systems down into composite parts then we’re often better equipped to manage these systems and understand them so we can reproduce them.

Virtually all complex behavior can be seen as composed of simpler behaviors that are enabled to combine in complex ways. When a complex system is understood in this way then it becomes much more straightforward to understand and work with.

Trade-Offs

Of course, we want the best of all worlds. We want our code to be clear and performant and extendable. But what happens when benefits are at odds with each other? What happens when, for example, in order to gain some clarity and improve maintainability we have to sacrifice some performance?

Of course, the answer is “it depends.” These days performance is rarely the bounding issue. It’s much better to think about building code in ways that are understandable to each other than to try to be clever for the sake of performance, most of the time. There are exceptions, of course.

Trade-offs in any field are inevitable and it seems like understanding when making the right trade-offs is a central part of every aspect of software development.

Rarely is there a right way and a wrong way to build software and different developers have different impressions on how to build code, which is fine as long as we’re able to interface our work together. One way we can do this is to have a common set of practices that we apply when building software. This is not to say that there is just one right way to do something. Often, there are many ways to approach a problem that are equally valid but yield very different kinds of designs. Understanding these trade-offs can be greatly beneficial.

Every design has trade-offs, it has strengths and weaknesses. Every design or piece of code is open for extension in some areas but closed for modification in the others. When thinking about and discussing designs, I find it extremely beneficial to call this out so that other developers are aware of how a design or a piece of code conforms to the Open-Closed Principle.

The Open-Closed Principle states that “Software entities (modules, classes, methods, etc.) should be open for extension but closed for modification.” What this means is that we want to make it so that adding new features in the future does not require changing a lot of existing code and mostly involves adding new code. When I ask developers why this principle is important they immediately know, because changing existing code is error-prone and difficult. Instead, we’d like to add new features by simply adding new code. And we can do this if we build our software to support the Open-Closed Principle.

Understanding where a design or piece of code is open for extension or how to make it open for extension in various ways gives us a way of extending features in a system without degrading the design or architecture.

Our goals when building a system are not around coming up with the perfect design. Very often when we start a project we know too little about what we’re building to determine the right design. Allowing a design to evolve over time can be highly efficient and effective. However, in order for a system to remain maintainable, we have to be willing to continually improve the design, as needed.

To me, software development is all about making trade-offs. It’s all about coming up with the right abstractions that do the job but don’t do more than is needed.

Every decision has consequences and when building software, one makes literally hundreds of decisions a day that affects the outcome of the product. In order to do this efficiently and effectively, to make the right trade-offs, we must first of all be aware of our options and then we must understand the consequences of each one. Becoming informed may not cause us to take any actions differently than we would have before but at least it helps us understand our options.