David Scott Bernstein's Blog, page 5

Drive to Testability

There are patterns, things that developers do when writing code that support testability of code and there are anti-patterns, which are things that developers do that lead to writing code that is difficult to test. As a developer, I’ve found it helpful to become aware of some of these testability patterns and anti-patterns that I frequently see in code.

There are more technical details needed in order to build high-quality software than any one book can cover. Beyond Legacy Code discusses the what and why of good developer practices, so that both developers and the non-developers on our team, like our managers and executives, can understand the reasoning and purpose of these practices to improve their processes. It really helps when developers understand the purpose behind the practices and they use the practices more effectively.

If the techniques and practices of Extreme Programming, such as test-first development and refactoring produced just incremental improvements in code then I’d be sort of so-so about it myself. But creating CLEAN, testable code is clearly the better way to build software. I have firsthand experience helping thousands of developers in companies around the world who now spend less effort developing software that’s more cost-effective to maintain.

Building software with these practices is far less stressful because there are fewer unknowns to deal with and we are able to better meet deadlines because we find fewer surprises that throw our estimates off. We’re focusing on one thing at a time and everything else just sort of fades into the distance so we can do that one thing really, really well.

We’re also building software that can be automatically validated so we can know immediately if there are any problems with it. This helps us elevate testability as a core part of the context for software development and this has deep implications on how we build and maintain code.

Testability is at the core of all good developer practices. We’re still exploring the nature of software in search of a better understanding of the context for software development, and a vital element of context for software must be testability—we must strive to produce testable code.

I recently tweeted this quote from Stanford University Professor Emeritus Donald Knuth: “Programming is the art of telling another human what one wants the computer to do.” I got seventy-six retweets in a few days, but I see too few developers actually following Professor Knuth’s advice.

Managers attempt to answer the question: What are “the right things?” by trying to measure productivity—that old production line manager’s stopwatch—or by imposing more complex schedules even though this often has the opposite effect and ends up demotivating people.

The more process we add the worse it gets because process cannot dictate creativity. And fundamentally we have to recognize that software development is a creative process.

But the truth is that most developers (and people in general) aren’t just looking for ways to game the system and slack off. Far from it! We want to help, to make a difference, yet often we feel it’s impossible to do, so we lower our expectations and fall back on approaches that are less ideal.

What I’ve discovered in my career as a software developer is that information aggregates in specific ways depending on its nature. When we understand the nature of information that we’re modeling then we can model it more accurately and from multiple perspectives and that will give us both understandability and extensibility in the code that we write, which is generally missing in most of the code I see in industry today.

The post Drive to Testability appeared first on To Be Agile.

The Importance of Continuous Integration

Perhaps the most important yet easiest to implement of all the software development practices in Agile is continuous integration. Continuous integration is simply creating an infrastructure where the code that is being built can immediately be integrated into a project.

One of the biggest challenges when writing software is the dependency relationship between the code that’s been built and the rest of the code in the system. Sometimes these dependencies are not at all obvious and only show up when we begin to integrate code from multiple developers into the entire project.

In traditional waterfall development, the software is integrated as one of the final steps of the process but by doing so it obscures many of the potential defects and we often end up in “integration hell” that delays projects and compromises code quality.

However, this can all be avoided through continuous integration. Defects are found while they are still fresh in the mind of the developers who wrote them so that they are easier to fix and they don’t get compounded by other defects through the development process.

A feature is not considered done until it runs and passes all tests on the build server. Just getting a feature to run on the developer’s machine who wrote it can hide a lot of problems that are not uncovered until integration. Integrating early and often is the cure for this. The sooner we can identify and fix defects the cheaper it is to do so.

But something magical happens when developers use continuous integration, it changes the way they approach problems. We can immediately see if our standards are not strong enough or if our approach is not the right approach for the problem by integrating our code with the rest of the system as it is being built. It teaches us in the only way that we really learn which is with immediate feedback.

As a result, the team just naturally flows to building high-quality software for the project and there are far fewer surprises at the end of the project. Continuous integration is easy to implement. Most of the tools needed to make it work are free and relatively painless to set up. Despite this, I’m always surprised to find that many development shops do not use continuous integration, even the ones that called themselves Agile.

Continuous integration also creates the technical infrastructure for other Agile practices, such as running a suite of unit tests and incrementally building features. I do not think it’s possible to have an Agile project without continuous integration and if you do, no other technical practices then I highly recommend continuous integration as the first and foremost Agile technical practice you start with. You will be reducing the risk of your project failing and be able to measure progress much more accurately when the features that are being built get fully integrated into the system, as they are built.

The post The Importance of Continuous Integration appeared first on To Be Agile.

The Importance of Technical Practices (Again)

Software development has undergone many revolutions over the last few decades. The way we build software today is fundamentally different than the way we did just a few years ago. Our industry is maturing and we are beginning to pay attention to the quality of our work.

No longer is just fulfilling the specification enough in software. Today, we must also write code that is easy to maintain and extend so that we can reduce the cost of ownership. To do this requires thinking about building software in some fundamentally different ways.

When I asked developers what they can do to make their code more maintainable or conversely what other people do that makes code difficult to maintain, I often don’t get clear answers because it’s something that most of us really haven’t thought about but there are things that we can do to make our code more maintainable and there are definitely things that we must avoid when building software as well.

Many of the most important and valuable technical practices have been embraced in Agile software development. Extreme Programming and before that Smalltalk and the development communities Smalltalk has proven time and time again that big complex software problems can be broken down into simpler elements and can be made more manageable by a handful of very straightforward techniques.

At its core, Scrum simply says to take complex problems and build them incrementally. This has the effect of taking a difficult task and breaking it down into several much more easily manageable tasks. The idea works in all fields and they work especially well in software development.

Scrum went on to discover that with an active product owner, the ambiguity in software development, especially in requirements goes away, and again the process of building the right software is simplified because we stay focus on the most important issues and get lots of feedback from our customers.

These two things in Scrum software development are largely responsible for the enormous success of Scrum development. However, there are other things that can also help the development process, specifically the technical practices from Extreme Programming which goes very nicely with Scrum development. Again, these are very simple practices that helped guide us in doing the right things at the right times.

The post The Importance of Technical Practices (Again) appeared first on To Be Agile.

Summary of Seven Strategies Series

I finished my “Seven Strategies” series of 72 blog posts with seven strategies for implementing each of the nine practices from my book, Beyond Legacy Code: Nine Practices to Extend the Life (and Value) of Your Software. These posts are filled with practical advice for implementing the nine core practices from Scrum, Extreme Programming, and Lean.

Here are the posts, enjoy!

Why Practice 1: Say What, Why, and for Whom Before How

Be the SME
Use Development for Discovery
Help Developers Understand Why and for Whom
Describe What You Want Not How to Get It
Answer Questions Quickly
Remove Dependencies
Support Refactoring

Why Practice 2: Build in Small Batches

Measure Time-to-Value
Measure Time Spent Coding
Measure Defect Density
Measure Time to Detect Defects
Measure Customer Value of Features
Measure Costs of Not Delivering Features
Measure Efficiency of Feedback Loops

Why Practice 3: Integrate Continuously

Use Version Control for Everything
One-Click Builds End-to-End
Integrate Continuously
Define Acceptance Criteria
Write Testable Code
Keep Test Coverage Where it is Needed
Fix Broken Builds Immediately

Why Practice 4: Collaborate

Try It, You’ll Like It
Engage Driver and Navigator
Swap Roles Frequently
Put in an Honest Day
Try All Configurations
Let Teams Decide on the Details
Track Progress

Why Practice 5: Create CLEAN Code

Get Crisp on the Definition of Quality
Share Common Quality Practices
Let Go of Perfectionism
Understand Trade-Offs
Hide How with What

Keep Code Testable

Why Practice 6: Write the Test First

Get Clear on the Benefits of What You are Building
Know Who it’s for and Why They Want it
Automate Acceptance Criteria
Specify Edge Cases
Use Examples
Split Behaviors on Acceptance Criteria
Make Each Test Unique

Why Practice 7: Specify Behaviors with Tests

Instrument Your TestsUse Helper Methods
Use Helper Methods
Show What’s Important
Test Behaviors, Not Implementations
Use Mocks to Test Workflows
Avoid Over-Specifying Tests
Use Accurate Examples

Why Practice 8: Implement the Design Last

Understand Object-Oriented Design
Understand Design Patterns
Understand Test-Driven Development
Understand Refactoring
Focus on Code Quality
Be Merciless
Practice Good Development Habits

Why Practice 9: Refactor Legacy Code

Refactor to Learn an Existing System
Refactor to Make Small Improvements
Refactor to Retrofit Tests in Legacy Code
Refactor to Clean Up as You Go
Refactor to Redesign When You Know More
Refactor to Clean Up Before Moving On
Refactor to Learn What Not to Do

The post Summary of Seven Strategies Series appeared first on To Be Agile.

Refactor to Learn What Not to Do

One of the things that I was not expecting when I started refactoring other people’s code was that I started to see improvements in the way I wrote my own code. I didn’t make those same mistakes.

Refactoring other people’s code has taught me a great deal about things to do and things not to do in software. We learn through imitation and unfortunately, good example code can be hard to come by. Example programs in books and frameworks tend to be simplistic and it can be hard to see how simple examples can scale up to enterprise systems.

Many developers that I meet tend to shy away from working with legacy code but I find that when I work with legacy code I learned a tremendous amount and my skills as a developer improve significantly. This is one of the reasons why I advocate for other developers to spend some of their time cleaning up legacy code.

A lot of refactoring is rote and it would be no fun at all except for the fact that modern IDE’s have many refactorings that are automated so when I’m working in Java or C#, I find that I can quickly and easily refactor my code with a few clicks of my mouse. In fact, I do a lot of my development with my mouse alone (or hotkeys) by selecting refactoring options to emerge my designs piece-by-piece. This can be a lot of fun but when I’m in more of a hurry then I resort to using the keyboard.

Having the automated refactorings that are available in IDEs is like Eclipse, IntelliJ, Visual Studio, and Code allows me to quickly and easily start simple with a design and emerge it as I wish. I first put the functionality I want into a method and then I extract that method into its own class and then I make that class abstract and add a Strategy Pattern, and so on. I find that this is a fun and safe way to build software.

I learn a lot by reading other people’s code. I haven’t found many good sources but of course, there is a lot of code on GitHub. One can learn a lot by reading other people’s code but only to a point. I wish there were more publicly available examples of good code, especially examples of large systems but unfortunately, there aren’t (or I haven’t found them yet). I’ve seen a lot of examples of big systems both good and bad that are the major systems in production today but most people don’t have access to that information.

A lot of the software that’s publicly available in repositories are not the best examples of high-quality code but there’s still much that can be learned from them. More and more resources are becoming available for learning about more advanced developer practices. I find that there are a lot of sources available for beginners and a few resources that are available for intermediate developers but virtually no resources are available for advanced software developers. This is a gap that I’m trying to fill while at the same time trying to boil down advanced practices to their simplest components so anyone can understand them.

Every once in awhile I will come across a project on GitHub that is beautifully written. I love to read elegant code. It makes my day. But honestly, I learned equally as much now when I read code that’s poorly written because I get clearer and clearer on the techniques that I want to avoid.

This not only concludes my seven blog posts on helping you justify refactoring, but it also concludes nine series of blog posts, each consisting of eight posts for a total of 72 blog posts that dive into strategies related to the nine practices from my book, Beyond Legacy Code: Nine Practices to Extend the Life (and Value) of Your Software. I hope you found it interesting and informative, and if so, consider reading my book.

The post Refactor to Learn What Not to Do appeared first on To Be Agile.

Refactor to Clean Up Before Moving On

Of course, the best time to refactor code is while it’s fresh in your mind, right after having worked with it. Once I get a feature to work, I go back and think about how I can make the code easier to understand. I’ve done this so often that it’s natural for me, almost of reflex.

Cleaning up code at this phase generally involves changing names of methods and attributes to more clearly state their purpose. Naming is my first line of defense in communicating what my code does. I want my names to be clear but when I’m in the process of discovering an implementation, I may not be as tuned in to refining what the name should actually be. Once I have an implementation that works then I’ll go back and see if I need to clarify any of my names.

Another cleanup activity that I’ll do at this point is to extract small methods from larger methods. I extract methods because it gives me the opportunity to name little bits of functionality instead of describing what I’m doing in a code comment. I find that if I can name some functionality, that it’s useful to wrap that functionality in a method with that name. I do this instead of using block comments in code because it makes my code more expressive. I want readers to read my code and not focus on reading comments.

Once I get a feature working as I want, instead of writing up what I did in a comment or external document, I go back to the code and see if I can let it tell the story, again by using good intention revealing names for the methods and attributes letting them say what the code is doing.

I want the code itself to say what it is doing but oftentimes code cannot say why it is doing what it is doing. To express why our code is doing what it is doing we often use block comments. This is the good use of comments whereas when we use comments to say what the code is doing we are just repeating ourselves from what the code says. Instead, I rather make the code clear and then I won’t feel like I have to have a comment saying what the code is doing. This will eliminate most of the comments in code and reserve them for only expressing the unexpected or for providing context around a large block of code.

The other thing that I will take a close look at when I have an implementation working but before I move on is to make sure that I don’t have any unnecessary dependencies. I want the dependencies in my code to represent real needs and I want to inject them into my code so that when I’m running tests I can inject test doubles or mocks instead. I want to try to avoid using singletons or service locators or other global states so that my software is as independently verifiable as possible.

Ideally, before moving on I want to make sure that I not only have clean code but also that I have good tests that express the intention of the code that I just built. Once I have these things I feel comfortable moving on.

Note: This blog post is based on one of the “Seven Strategies…” sections in my book, Beyond Legacy Code: Nine Practices to Extend the Life (and Value) of Your Software.

The post Refactor to Clean Up Before Moving On appeared first on To Be Agile.

Refactor to Redesign When You Know More

One of the reasons that I’m such a proponent of doing emergent design or just-in-time design, is that this is how we learn a system and this is typically how a system needs to be built. It’s fallacious to believe that you can easily envision every aspect of a complex system and come up with a comprehensive set of requirements that don’t need to change. Really smart people can’t do this so the rest of us shouldn’t be expected to do this either.

Imagine what the movies would be like if actors had to get every shot perfect on the first take. We would see far fewer risks being taken and far more mistakes would end up in the final version of movies. Being able to have multiple takes gives an actor and a director the freedom to try different things and often they can come up with better ideas than they originally imagined. The same thing is true in the software profession.

If software developers believe that they have to come up with the right design initially because it can’t be changed later then they have a lot of pressure to get it right. Not only do I find that this is an enormously difficult constraint, but I also find that it is typically an unnecessary one because you don’t have to come up with a design and stick with it evermore in software.

After all, software is soft. This means that we can take advantage of the changeability of code.

The key to doing emergent design successfully is being willing two change your design in the light of new requirements. When we get good at doing this we discover that most of the time, changing a design is a trivial activity that’s both safe and straightforward to do, when we know how to refactor.

When we reach that level of craftsmanship then software development can be a lot more fun because suddenly the pressure is off. We don’t have to come up with the perfect design because we know that we can refactor our code and change the design anytime in the future that it makes sense to do so. When we realize that going from one design to another design is usually a simple matter of repackaging existing functionality and that oftentimes the hard part is getting the functionality to work in the first place, then we can focus our refactoring efforts on finding a more extensible design for code that already works.

As the expression says, hindsight is 20/20. We often know things in retrospect that we wish we knew ahead of time. Refactoring code gives us a second chance. It gives us that opportunity to incorporate what we didn’t know at the start and this can vastly improve the design of our code, as well as the opportunities for extending it in the future.

We often learn in chunks. Even though there is the refactor step when doing test-first development, so we are constantly doing refactoring in the small, we still need to occasionally do refactoring in the large. It’s because we learn in chunks that we have to do refactoring in the large. We want to take our learning and bring it back into our code, and in order to do this when we have new and significant insights often requires taking some extra time and refactoring code so that the design reflects our current thinking.

Note: This blog post is based on one of the “Seven Strategies…” sections in my book, Beyond Legacy Code: Nine Practices to Extend the Life (and Value) of Your Software.

The post Refactor to Redesign When You Know More appeared first on To Be Agile.

Refactor to Clean Up as You Go

We should be refactoring the code we write and all the code that we encounter all the time. Making little improvements to code should be just something that developers do without even thinking about it.

I find that many of the anti-patterns the developers engage in when writing code are easy to spot and after I refactor code that uses those anti-patterns, I’m less apt to make the same mistakes in my code. This is the hidden benefit I found when I refactor existing code, it has made me a far better developer by teaching me not to use those anti-patterns in the code I write.

There is now a whole movement around people who like working with legacy code. They call themselves menders. In my experience, it’s about one in every 50 developers who like working in legacy code and these people really like it. There’s even a group and podcast called Legacy Code Rocks composed of self-proclaimed menders who prefer to remodel existing code over creating new code. Thank heavens there are people who like working with legacy code because most developers don’t and there’s a lot of legacy code out there that needs to be cleaned up.

When I write code, I try to think about the people who will come after me who have to look at my code and change it. I try to make things clear and sensible even if it requires a little extra effort on my part because I know how grateful I am when I have to maintain code when developers have taken the time to make their code understandable.

I find that the best time to improve a piece of code is in the moment while I’m looking at it. I may have gotten there in a debugging session or while I am learning a new piece of code but however I got there if I see an opportunity to improve code I want to take it and I want to encourage other developers to do the same. This should be part of our community of practice, that we are willing to go in and clean up code when it makes sense to do so.

Cleaning up code is not the hard part. The hard part is understanding the code and if software does what we need it to do then we don’t have to understand it most of the time. We only need to understand code if we need to change it.

So, I will make little changes to existing code all the time whenever I see an opportunity because it doesn’t take very long and I know that little changes can ultimately add up to significant improvements in code.

And when I say little changes I mean what we call the ‘safe refactorings’ like Rename Method or Move Method. These safe refactorings are typically used through an automated tool, like in many IDEs so there is little chance of making a mistake and introducing a defect.

But big changes, larger refactorings that could introduce defects, especially if the code is not under test, I reserve only for when I have sufficient justification. This might include needing to extend an existing system or add features to an existing system that doesn’t do something that I want it to do. If the system is poorly designed then I may have to take time and clean it up first. I often find that this is the most efficient way to add features to existing software.

In these situations, I use a technique called Refactoring to the Open-Closed where I first refactor the code to make space for the new feature and then add the new feature. I use this technique whenever I need to refactor an existing system and I also use this technique to emerge the design of a new system as I’m building it. In both cases, I’m simplifying the integration of new software into an existing system by first refactoring the existing code to easily accommodate the new feature.

Whenever I’m working in an existing system I’m typically doing two things, which is changing the existing code to make space for new features and then adding the new features. These are actually two distinctly different activities and I find that when I separate them out and do them at different times the process of working with legacy code becomes much more manageable and the opportunity to introduce defects becomes far less.

 

Note: This blog post is based on one of the “Seven Strategies…” sections in my book, Beyond Legacy Code: Nine Practices to Extend the Life (and Value) of Your Software.

The post Refactor to Clean Up as You Go appeared first on To Be Agile.

Refactor to Retrofit Tests in Legacy Code

The so-called safe refactorings are a subset of the refactorings from Martin Fowler’s book, Refactoring: Changing the Design of Existing Code. We call them safe because they require only straightforward changes to code that can be automated and proven to be correct with no unintentional side-effects.

Safe refactorings are refactorings like Rename Method that lets you change the name of a method or Extract Method where you can take a large method and extract out smaller methods from it. Conversely, you can take small methods and inline them to make larger methods. Each refactoring typically has an opposite refactoring, for example, extracting or inlining methods.

Some refactorings, however, are more complex so there are more opportunities for something to go wrong. Before attempting to do major changes to existing code we often would like to have that code under test with good automated regression tests so that if we make a mistake or change the behavior of the system as we are refactoring it then our tests will tell us and we can back out right away.

But when we have legacy code that is not under test it becomes dangerous to do complex refactorings and so the strategy that we typically take is to do safe refactorings to give us the opportunity to inject dependencies, which lets us write betters tests so that we can then do more complex and riskier refactorings having the code now under test.

Often times, the process of getting legacy code under test is a process of creating what Michael Feathers talks about in his book, Working Effectively with Legacy Code, as creating seams. A seam is an insertion point where you can inject a dependency and this is typically one of the things that allow us to make code more testable.

Therefore, much of the process of refactoring and working with legacy code is a process of first changing the relationship to dependencies. Instead of directly creating and calling dependencies in our code, we create seams in legacy code that allow us to inject those dependencies and then, in our tests, we use the seams to inject fake dependencies that allow us to test our code’s interaction with those dependencies without having to have those dependencies present.

This can be a slow and involved process but once we have good unit tests for the behaviors of an existing system, it’s like having a safety net that allows us to make changes to the system knowing that if we make a mistake and mess up then our tests will tell us and we can immediately recover.

In the process of retrofitting tests into legacy code, we are often doing many other things. We are also often rewriting the code, making it more straightforward to understand and work with in the future. When legacy code is difficult to test it will need to be rewritten or refactored in order to make it more testable which often has the benefit of making the code more extensible to adding new features.

If you need to refactor or extend the behavior of an existing system that is not under test then often the best first step is to retrofit tests into the system by adding seams, because having tests allows us to safely do more risky refactorings to improve the design even further. And that, in a nutshell, is how we improve legacy code.

Note: This blog post is based on one of the “Seven Strategies…” sections in my book, Beyond Legacy Code: Nine Practices to Extend the Life (and Value) of Your Software.

The post Refactor to Retrofit Tests in Legacy Code appeared first on To Be Agile.

Refactor to Make Small Improvements

Another strategy to help you justify refactoring is to do it in order to make small improvements in an existing system. Don’t try to bite off the whole Apple at one time. Really bad legacy code got that way over long periods of time and in my experience, a similar, gradual approach can be good for making it better.

Rather than trying to fix an existing system all at once, consider just making it a little bit better each time you have to go in and work with it. You don’t have to do much but just change the direction of decay. Instead of letting a design get a little bit worse each time you have to go in and add new features, find ways of improving the design to accommodate the new features. Improve the design in the area that you’re working and then also improve other parts of the design while you’re there if you can do it easily.

As a high ranking tenderfoot in the Boy Scouts of America, I was a devoted camper but not into merit badges. My scoutmaster told me words that I will never forget. He said, “Always leave a campsite cleaner than when you arrived.”

We didn’t just clean up after ourselves when we left a campsite, we also did a little bit more. We looked for other trash or garbage that other people had left and we just took a few minutes to clean that up too. If every time we go into code we make it just a little bit better then our software would no longer decay but rather it will start to take an upward spiral and improve as we continue to work with it. Wouldn’t that be great?

People think that refactoring an existing system should be a gigantic process but I find it’s best to work on it a little bit at a time and as a background activity. This is often a better approach than shutting down the entire organization to clean up an existing system but sometimes that’s unavoidable.

My preference is to make small improvements on existing systems as needed. If an existing system is poorly written but still serves its function then I’m apt to leave it alone. It’s really only when I have to extend an existing system by adding features that I want to first clean up the code so adding the new feature is more straightforward.

Sometimes, the most direct route to a solution in code is to first clean up a mess before adding new features. Sometimes it makes sense to add features by first refactoring an existing design so that it’s straightforward to accommodate the feature when we add it later. This can make adding the feature much simpler and safer.

This is actually a technique that we call “refactoring to the open-closed.” The open-closed is a reference to the Open-Closed Principle that states that “software entities (such as classes, methods, modules, etc.) should be open for extension and close for modification.”

What this means in practice is that we should write software in such a way so that when we add new features we’re minimally changing existing code, maximally writing new code. When we have existing code that doesn’t allow us to easily add a feature then it can be far more cost-effective and efficient to first refactor the existing code so that you can easily add the new feature later.

There’s a reason that we refactor in tiny steps. When you take a tiny step and something goes wrong then the chances are that you can figure out what happened quickly. There’s only a limited number of things that can go wrong in a tiny step.

Small improvements add up. I’m always surprised at the end of a day when I spend it refactoring existing code using the safe refactorings, like Rename or Move Method, to learn an existing system. I always walk away knowing a great deal more about the system but also I can see real improvements in the clarity of the code and in the design, making it a win for both me and the code.

Note: This blog post is based on one of the “Seven Strategies…” sections in my book, Beyond Legacy Code: Nine Practices to Extend the Life (and Value) of Your Software.

The post Refactor to Make Small Improvements appeared first on To Be Agile.