Markus Gärtner's Blog, page 11

The situational foreman

A couple of days ago, Uncle Bob Martin blogged about the foreman in software development. In it he makes some claims with regards to a role in software development that acts like a foreman on a traditional construction site. Uncle Bob claims that the foreman should be the master of having commit rights, and granting those to team members that prevailed long enough. While the idea reminded me by-and-large to the practice on some open source projects, I am not so sure whether we should start installing foremen (and forewomen) all over the place. Let’s see why.

Batch Size

Imagine the picture painted by Uncle Bob. There is a single person responsible to review all the commits to the main line. I assume an agile team limited to 3-9 people here. That person would become the single source of approval for all the stuff that makes it to the version control system.

In Lean thinking terms there would be delayed feedback for the features introduced. Depending on how often programmers send out pull requests to their foremen, the batch would be larger or smaller. The larger the batch size, the more potential problems could accrue. One can argue that this educates the novice programmer to commit smaller changes to the foreman, while the more advanced programmer could go with larger batch sizes, since there are hardly any changes necessary. Interesting idea.

Unfortunately that delayed feedback would lead also to more problems building on other problems in the change set that is sent to the foreman. With the delayed feedback from the foreman there would actually be fewer learning happening at the moment the problems are introduced. There would also be more learning piled together.

Unfortunately humans are terrible when it comes to learning in large batches – at least to my experience. I quickly feel overwhelmed with lots of feedback, close to being unable to respond at all. Of course, different people handle negative feedback differently, and certainly one can learn to deal with that kind of feedback.

However, what’s the outcome here? Why would a foreman be interested in the commits from any individual programmer? Ideally the programmers would have paired on most meaningful production code, anyways, wouldn’t they?

As Uncle Bob also points out, the foreman would be interested in the individual commits in order to help them solve problems of different nature. In his second blog entry he expands on this. The reasons he lists include personal coaching, and the team’s ability to provide that programmer some relief.

Reviewing all submissions is truly one way to become aware of personal problems, like your wife being diagnosed of cancer. I think there are more effective ways to become aware of these. Trustful 1-on-1 meetings are one way to deal with them.

Responsibility

Over the years I have experienced varying behavior from developers whenever there was a testing in place. Some programmers knew how to include them in their development efforts. Others, well, didn’t, and were rather fighting them. I even saw programmers that submitted their code without trying it on their own. In the end, there was a testing department taking a look, right?

The same could probably happen with a foreman having a look on each and every commit. But there is another social aspect in this dynamic. What I learned from the drama triangle, is that the foreman would eventually become an offender for the programmer that wants to improve, and get a higher salary. A programmer would have an easy time playing the role of the victim in this dynamic, rather than seeing the potential in the feedback from the more senior programmer to excel in his own skills. So, there is a high social cost attached to this, and I have heard about this same dynamic in many open source projects where the bad master committer won’t pull certain requests.

But also consider the workload for a large team on the foreman. Of course depending on the output of the programmers involved, I think the foreman could become overloaded by commits quite easily. This is speculation, since I never worked in such an environment. In my past, though, I took on the habit of working through the commits on our version control system to see which changes I could pull without risking to loose my test system. I gave that one up after a while since it was too much work working through all the commits of three different teams. I felt overloaded.

So, besides the workload, there is lack of responsibility for the individual work, and a weird dynamic from the drama triangle as social costs in play in that model.

Manager Teachers

Another thing I learned from the Toyota Production System is the idea of manager teachers. At Toyota, managers do not advice people on what to do. Rather they understand their managerial role as teaching others to become a better employer. These managers try to teach everything they know to their direct-reports. Thereby, a culture of growth is created to the extent that people need more time to advance in the career system, but that advance for good.

Instead of rejecting all the bad commits that come their desks, master programmers probably should work closely with apprentices to teach them the tools of their trade. If masters grow journeymen, and journeymen teach apprentices which tools work good, and how to working habits reflect the culture in the code, the overall code base does not have to start to rot.

Unlike foremen, people in programming should create a culture of learning from their peers. We should create a safe environment where everyone feels save to learn new stuff every day.

Situational Leadership

After all, I think the foreman model has a context where it makes sense. The Situational Leadership from Hersey and Blanchard reminded me of a possible context. When you learn something knew, you might need more direction. This holds true for apprentices in their first few days. This also holds true for cross-trained employees when they learn a new skill.

I think the foreman model makes sense in the telling quadrant of the situational leadership model where the employee needs more direction. Over time, though, the team member becomes more capable, and over time also more willing. The underlying leadership model should shift at this point to more and more degrees of delegation.

The situation that Uncle Bob describes in his second blog entry can be addressed by nurturing a culture of openness and more direct mentoring for example from the ScrumMaster or coach. The leadership in the code probably is better addressed by a “Go Gemba” attitude – go see the workplace. Unlike physical production, for software development “Go Gemba” probably still needs to take a look in the code. Whether you address that by creating a foreman bottleneck for changes to your main line, or by pair programming, and coaching on the job, should be a question better answered by the underlying situation for the individual team member. Some need more direction, others more delegation. Pick your battles.

The Power of the Team

Agile methodologies favor effective team work. Team work is fun to do, hard to grow, and at times unpredictable to nurture. I might be biased here, but I have seen way too few good teams working effectively together. Let’s see what can happen with bad and with good team work in place.

The Bad

A few years back I found myself part of a team at a client. Overall we were six people on that team. There was a second team working on the same product, but in a different area. We had pretty good engineering practices in place, including pair programming, continuous integration, we started with unit tests and TDD on the 2 million lines of code legacy code base, and we were constantly balancing between time pressure from the customer and good design and architecture.

Around the second month into that client gig, we faced a difficult issue. We didn’t deliver anything in one Sprint. Back then we were applying Scrum, and we had a terrible time to tell the ProductOwner that we didn’t manage to deliver anything in the past two weeks. What puzzled us even more was that fact that we didn’t had a good theory why that happened. You might call it bad luck, maybe there was a story causing us so much trouble that kept us from the other stories – we didn’t know, and we didn’t have a clue.

So, in the retrospective at the end of this disaster we sat together in order to find out how we could solve that problem. But where do you start to improve stuff if whatever you did thus far didn’t work at all? Well, you can start improving anything since whatever you did probably contributed to your mess. Eventually we introduced to focus on finishing one story first, before going on to the next story. Thereby we tried to establish a single-piece flow, and hopefully we could make at least the most valuable story work first rather than starting all stories right from the start.

So, no matter how bad you are doing, make sure to find something you want to experiment in order to make your life better – a tiny piece at a time.

The Worse

So, we started with the next Sprint. We were energetic about our tiny improvement. It just felt right. Skip forward two weeks, and we still hadn’t delivered the most valuable story for our ProductOwner. If my brain remembers back correctly, I think we eventually finished one or two items that we planned, but certainly not the one that was sort of overdue, since we had worked on it for two Sprints straight now.

I still remember the ProductOwner from the retrospective that day. It seemed that his heartbeat was pulsating, and if you turned out the light, you could see his head glow red in the dark. He was pretty upset with us. The other team continued to deliver in the end. How could it be we were not delivering at all? There was something wrong. He made us feel his problem by telling us that he had a problem with the situation. It was unpleasant to be part of that team at that point in time. To say the least, I would have preferred to run away from that meeting. The ProductOwner was calm enough not to blame anyone on the team, though he was courageous enough to make us aware that we had a problem somewhere in our team that kept us from delivering.

Notice that things usually go worse before they can start to improve.

It turned out that this situation helped us become aware there was something wrong that we couldn’t solve by any programming practice, any strategy, or any framework. When the ProductOwner raised his unhappiness we were eventually able to surface the underlying problem that was – at least to me – hidden under the surface.

There were two team members that couldn’t work together on the same computer. It somehow did not only affect the two of us, but also dragged down our team’s productivity to a degree that kept us from delivering anything at all. When we constructively realized that underlying problem, we were able to deal with it.

Of course, during that retrospective there were tears shed, people felt uncomfortable, and we sort of had to deal with the situation. We were able to derive two experiments for the next Sprint: the two with the problem started to consult and mediate with a facilitator and mentor in order to solve there problem, and we agreed that it will be ok if the two will not work on the same computer any more.

Whenever something goes wrong, communicate congruently, openly, and leave out the blame. Make sure that everyone can see there is a problem, and feels safe to contribute solving it.

The Gordian Knot

When we fixed that underlying problem, we were able to deliver twice the amount of stories that we planned on the next day during Sprint Planning. We were even able to work as a team by engaging in high-level design and architecture discussions in our Sprint Plannings. Last, but not least we were able to convince the ProductOwner of the benefits of test-driven development in the following month because we were able to create enough slack so that we could cross-teach other team members in it.

After we fixed the problem that silently held the whole team back from performing, we were able to leverage the full power of our team. I have seen few teams since then that were able to out-perform themselves from Sprint to Sprint. Yet, this experience shaped.

Whenever someone asks me about team and team performance, and how this whole Scrum thingie can work, I think back about that story of that team. It took us roughly two months to go from Forming to Storming to Performing in the third month. Based on my experiences with other companies, and other teams, I think we were quite fast in that.

Last year I was interviewed by a client that was transitioning to Scrum in a new project. They asked me how long it would take to create a performing team. I told them that it takes roughly two to three months to get familiar with the Scrum mechanics (how I would like to call them), another two to twelve months to create an awareness for better coding practices, and eventually implement them, and maybe another twelve to twenty-four months to face the next organizational boundaries.

My message didn’t appear to be the message my interviewers were looking for. I haven’t heard from that company for six to nine months – until about a month ago. Maybe I was lucky, maybe the team had bad luck – I don’t know.

All I know is that the team in my initial story went from good to great in three months – thinking back I consider the surrounding environment being supportive for team building. I have never seen team building dynamics go that fast again.

The Curse of Refactoring

Last week I sat in a meeting with a ProductOwner, a ScrumMaster, and the Development Team. The team works on a legacy code base with about 2 million lines of code together with 13 other teams. Thus far there has been little to decouple the various components. In that meeting the topic of refactoring came up. The bottom line was that the Development Team needed to ask the ProductOwner for time to refactor their code for each story.

What a waste of your time.

Personally, I believe that if you have to ask your ProductOwner to plan in time for refactoring, the ProductOwner should stop all work on the product, and send you on a class on working effectively with legacy code if you are an internal employee. If you are an external employee, your client should fire you immediately.

Wait, what? That’s harsh? I don’t think so.

Let’s take a look at the underlying system dynamics.

Prioritization

The primary responsibility of the ProductOwner is to prioritize the Product Backlog in order to develop the most successful product that’s possible with the given circumstances. Clearly, the ProductOwner optimizes the flow of backlog items to generate the biggest possible value out of the product.

If the team asks the ProductOwner to prioritize between technical polishing and business value, what would happen? First of all, in most cases I have seen, business value had the greater priority for the ProductOwner. Most of the times the ProductOwner was not even able to distinguish the value of refactoring items. (Most of the times, ProductOwners are not even able to distinguish the value of business items either, but that’s a different story.)

So, how would you choose if you could have 10,000 Euros now with a dirty kitchen, or a clean kitchen, and maybe 10,000 Euros later, what would you pick? Of course, the dirty kitchen could be a bit short-sighted. That’s probably why good ProductOwners understand the necessity for polishing work. The problem is that good ProductOwners do not fall from heaven. They need to learn. If you have a learning ProductOwner, you shouldn’t have him prioritize between polishing work and business value.

Technical Debt

The primary responsibility of the team members is to deliver a technical sound product. At the end of the iteration, the product should be potentially shippable. Scrum teams have to face the perfection challenge: deliver something of value fast.

If you leave out necessary refactoring, you introduce a delayed feedback loop into your system that is clearly not necessary. If there is refactoring work, you should not put that work in a backlog – either obviously or non-obviously by delivering a lower estimation under this or that condition for leaving out the refactoring step. That would be accumulating technical debt. Your ProductOwner needs to pay for this debt later. Oh, you will also have to pay for that debt later as well.

The problem with technical debt is the delay introduced in the feedback loop. The more and more technical you accumulate, the more the necessity for the grand redesign in the sky grows. On the same note you are slowly boiling yourself by introducing more and more debt that slows you down.

Over time, you will deliver new features slower. This will either lead to the ProductOwner trimming the tail sooner since the product generates less revenue than necessary. Or you can convince your ProductOwner of a refactoring sprint. I have seen this a couple of times. What refactoring sprints mean in fact is that you haven’t build the best possible product before. Of course, there are blind spots for all of us. In my experience refactoring sprint provide an excuses for the Development Team to not deliver the best possible product. In the end you will find yourself with more technical debt than is really necessary.

Instead, consider challenging your team to learn to really deliver an increment of working software. The ProductOwner should be able to say “ship it!” during the Sprint Review, and it’s live two seconds later. Ok, maybe ten seconds, but no more. No “wait a minute, there is this thing” or “oh, we really can’t do that now”. Thereby you help your ProductOwner to fill his role without delays by providing the necessary transparency of where the product really is.

Empirical processes

The whole story boils down to empirical process control. Empirical process control consists of three major elements: inspection, adaption, and transparency. Inspect & Adapt underlies Scrum in large portions. Without it you wouldn’t be able to improve over time.

But Inspect & Adapt without transparency leads to erratic improvements. If you try to tune a system that you don’t have the relevant information from because people are hiding critical information like how good the product really is, leads to improvements that could eventually increase the demand for refactoring.

Your ProductOwner will only receive a complete picture for the underlying problems in the software, if you can provide her full transparency about the problems that you know of when introducing. You are not delivering the best job you can, if you avoid that.

Also note that technical problems that lead to fewer business functionality is not something your ProductOwner should worry about. That would be similar to a taxi-driver asking a passenger for the best route to take in a foreign city. That’s unlikely to work out – if the passenger does not know the city.

The same holds true for most ProductOwners when you ask them to decide for priorities of technical solutions when they don’t know the code base well enough to estimate the risks around that. In most implementations that I have seen that ProductOwner couldn’t know because he had other daily business to turn to.

So, clearly, as a member of the Development Team it’s your responsibility to be able to deliver the best product. Your ProductOwner is unlikely to know about the technical trade-offs that you put in your estimates, and the amount of technical debt you accumulate when rushing through the code base, or the amount of legacy code you have to tackle with.

Instead, provide a reasonable estimate for the work you see, and have a chat with your ProductOwner on the why. In the end, consult with her in order to split too large items smaller – but avoid cutting off the “refactoring” in a separate part. Your product will thank you in the long run, and you will deliver a more professional job by doing so.

The “I don’t want to” attitude

We live in a cruel world. Our profession of software development is very young compared to other fields such as banking, hotels, or carpenters. I truly believe we have taken a couple of wrong turns in our short history. In this blog entry I try to shed some light on it by some seemingly unrelated stories.

The woman in the bus

A woman heads home from work. She has spent all day working hard – actually 10 hours straight with hardly a break in it. She needs to drive back home for 60 minutes. She waits at the bus stop for the bus to take her home.

She enters the bus. There are no free seats available to her. She goes to a restricted area of the bus, and takes a seat there. Immediately the bus driver points out that she is not allowed to sit there. Other passengers start to rally.

The woman leaves the bus, and walks home.

Outrageous story, isn’t it? How would you react if the bus driver was black, and the woman was white? Keep that thought for a minute.

Now, how would you react if the bus driver was white and the woman was black? Keep that thought for another minute.

To be frank, this one was paraphrased from the story around Rosa Parks that started off the whole movement to give blacks the same rights as whites in the USA in the 1950s and 1960s.

Rosa Parks was able to deliver passive resistance. She initiated that blacks went home from work by feet for several months. Just to indicate that they are treated differently.

Eventually they were able to change the public view on blacks and whites in America – though I think America still needs way to go on this.

Not in my contract

I heard the following story from a colleague who was coaching teams in a larger company a few years back. They talked to team members about unit testing, how to do it, and tried to motivate them to essentially apply coding practices from the 21st century, rather than the 19th.

One of the developers went to his desk, opened the drawer, pulled out his contract, and told my colleague that it didn’t state to write unit tests in the contract.

Now, observe your reactions to this story. Keep that thought for a minute before reading on.

Seriously, how can the programmer dare to refuse testing his code? Seriously, he can’t. He is not caring about quality in the code. That guy should be hung, drawn, and slaughtered – shouldn’t he?

Related, he is handing off work to his fellow testers. Thereby he is postponing feedback for the work he is doing. He will have a hard time finding out about those bugs later. It’s even worse than that, because he might start to justify his coding decisions later in order to avoid the tremendous amount of re-work later.

Of course, you scream that that guy should report to his manager – and maybe get fired. Rightly so. That guy does not care about the product, the quality, and the underlying system dynamics in play.

Another story

Here’s another story that I experienced one way or another. I consult with a development team for the first time. I notice that several of the team members have a problem with one or two others. It’s even worse. They are close to a witch hunt – of course an instance of the identified patient pattern. Everyone hates those one or two guys on the team since they are clearly not contributing, or helping the team to deliver better software.

I approach the one or two guys. I point out that they need to learn a certain skill to improve the overall team performance. I point out that they can learn from their team members. I point out that they can even teach their teammates some of their special skills.

They still refuse to do so.

Again, take a minutes to examine your reactions to this story before reading on. I will wait here for the time being.

Just a bit.

Ready?

Ok, here it goes. I was speaking about several situations I faced when dealing with testers on agile teams. The skill they refuse to take on is the programming skill. They are not willing to pair up with a programmer – to learn some of that – to teach them some of their testing skill.

How do you react now to that story?

I wonder how we started to treat testers differently to programmers. The programmer that does not want to learn anything about testing is cursed, hates, and brought to the stake. The tester that refuses to learn anything about programming is promoted to test manager.

Seriously? Where does this underlying value system come from?

To me, there are two major factors involved here: history, and cognitive dissonance.

Historically our industry suffered from good people in the 1970s and 1980s. That was also when we started to automate a couple of business process. At that time that people that were eventually replaced by automation were asked to become the testers of the software they were replaced with. Maybe they liked it, and became testers themselves in the end.

Typically these testers don’t have much programming know-how since they came from a totally different background. Skip forward twenty years, and we ask them to learn programming. Unlikely to work.

Cognitive Dissonance is cited by Leeds & Weinberg in Computer Programming Fundamentals. My paraphrasing of it is that a programmer cannot test effectively his own code if he knows what he has been coding, anyways. The historical solution that our industry came up with was to separate the person, team, and even the department that tested the software compared to the one that wrote it. Nowadays this is the prevalent solution to deal with cognitive dissonance. Some call it independent testers. Others call it the four eyes principle.

The problem I have with this, is that cognitive dissonance is taken as an excuse in order avoid learning programming, while programmers are expected to learn testing on their end.

Note, that several experts in the testing field recognize that programming skills help testers to large extents. I have read about it in Basiswissen Softwaretest, a German ISTQB book, in Lessons Learned in Software Testing, the context-driven foundation. I am sure there are others out there. The bottom line is that all these experts state basic programming knowledge helps a tester to improve their testing. How come we still use that excuse to not learn basic programming skills? On face value, it would improve our testing skills even further.

One thing I learned from my father was that learning does not end after your first education, rather it starts. My father was a car mechanic. He didn’t take university courses. He didn’t study computer programming. Though he knew that he needed to learn all his life – even after finishing his apprentice years. Unfortunately some of the higher educated folks do not seem to have the same attitude when it comes to learning. Sadly so.

So, what additional skill did you learn today? Which one are you going to learn?

Fixed price, fixed scope is broken contracting

Over the past week I came to realize that fixed prices, fixed scope contracts are broken. I am not sure who came up with the concept, but I sense a larger dynamic at play. Let’s explore what’s causing so much trouble around fixed price, and fixed scope contracts.

One of the last activities in a previous job was to work on a tender offer for one of our potential clients. The tender surrounded around a company whose major shareholder was the government in an Eastern country. The country had two different calendars: the Gregorian, and the Sambat calendar. The former one should be well-known. The latter one is based upon moon phases where a committee gets together to decide which upcoming months will have 28 or 32 days.

Since the client worked under the laws of the government, they had one requirement in their tender documents. All input and output dates should support both calendars. For outputs that meant to list both, the Gregorian calendar date, as well as the Sambat one. For inputs we would need to figure out how to separate the data by either the Gregorian calendar, or the Sambat one.

While working through the documents, I sensed a large risk for this requirement. Needless to say that few developers (and architects) ever think about supporting a different calendar implementation. Our system also dealt with three different programming languages. Since all of them had interfaces to the outside world, all of them needed to deal with these conversions. For the Java component it took me two days to find a working implementation – up to the year 2040. Our system supported dates up to the year 2200 internally.

Of course, this was one tiny piece of requirement. The client was rightly asking for that implementation. After having thought of it for a couple of days, I decided to pad my estimate, and put down a note about the upcoming effort – knowing that I won’t be with the company any more to implement. (I already had quitted my job at the point of time – for other reasons.)

What happened? The tender application asked for a fixed price, fixed scope contract. Needless to say there were about 10 documents with 100 to 500 pages, listing several such requirements. There was a large batch of requirements thrown at us. The project would take a couple of years to implement fully. We were asked to tell our sales department whether our software product already supported some of the requirements, and where the risks lie to determine the final price that we could bid on this.

Overall the tender application took several weeks at a point in time where we had a shallow understanding about the requirements. Also pricing in these situations is merely one factor. There are always political reasons involved as well. I did not track whether my former employer actually won the bid. What I know is that I was sub-optimizing. I was sub-optimizing because we were at the bidder side. I had a shallow understanding of the underlying calendar system. I needed to trust the third-party implementation. Maybe our requirements analysis would come to a different conclusion when working on it. All we needed was give out a price tag for the final bid.

Of course, there are competitors. In order to win such a bid, you need to keep their pricing in mind. Maybe you have a second channel into the organization, and know how to trick the system – thereby neglecting all the information that you can get from your staff. But these more political reasons come with a price. You are probably giving out a system that is costing you more to implement in first place.

What usually happens with contracts like these is that the bidder does not want to deal with all the risk and losses on their own. So, they put a clause in there that makes changes to the initial scope more costly. Thereby the bidder can still earn some amount of money.

Queueing theory taught me that the larger the batch sizes, the more likely there will be changes. So, on a systemic level, if you receive a huge tender application, there will be more changes attached to it. So, underbidding a competitor comes with a risk, unless you recognize that you can charge more for change requests. Since the likelihood of change requests with a large batch of requirements like the ones found in tender applications raises, as a bidder you can dramatically reduce your implementation risk by bidding on large tenders – with high costs for change requests.

For the customer, the more expensive such change requests become, the more likely the customer will ask for more detailed requirements, put more staff in the tender, and thereby increase the batch size. Since he becomes disappointed by the amount of change requests with these large batches, he will try the next time to get down more detailed requirements, thereby increasing the batch size even further – only to be disappointed even more.

Notice that there is a vicious cycle involved in this game. The bidder is sub-optimizing for his costs. The customer is sub-optimizing for more value. In the worst case this is a lose-lose game. A terrible idea for the whole industry.

But how can we fix this? I think we need to recognize where fixed price, fixed scope contracts come from. My imagination about this is that customers want to have insurance about their budgets. Thus, the origins for fixed price, fixed scope contracts probably lie in the underlying budget constructs that are widespread in the industry.

If we want to solve the situation, we need to work on two frontiers. First of all, a tender shouldn’t cause a bidder to produce crappy software. The current construct found in our industry does just that. Deadlines, fixed scope, and high ambitions together with wishful estimation lead to more rushed implementations that eventually lead to less quality for the customer in the software that we create.

But crappy software is not the origin. We need to work on our trust relationship with our customers. Over time we can achieve that they see a trustworthy software development shop in ourselves that is able to steadily add value for them. Only then can we break the positive reinforcing feedback loop that creates the vicious cycle in the overall scenario. If the customers receive the feeling that they can still step out of a contract after three months because it stopped working for them, and they still can maximize the value they get out of it, the whole industry would be better off.

So, why don’t we take the first step to create that trust-worthy relationship with our clients by inviting them every couple of weeks to present them our latest features? Maybe that could be helpful after all.

How do you know you’re improving?

I remember a lively discussion at DEWT 4 around self-education, and how you would know whether or not you are improving. There are lots of ways to engage with self-directed learning – in software testing, software development, leadership, and other areas surrounding this field. But with all these methods around, a single question remains: How do you know whether you’re improving with whatever technique you follow?

Deliberate Practice

According to Malcolm Gladwell’s Outliers, it takes 10,000 hours of deliberate practice (20 hours per week for 50 weeks per year for 10 years) to become an expert at anything. So, in order to become better at what you are doing, you better put some additional effort into it.

Unfortunately putting 10,000 hours into anything will not automatically make you an expert at anything. For example, if I learn how to dance my name, that won’t make me an expert at dancing, no matter how much effort I put into it.

With that said, we should pick things for our 10,000 hours of deliberate practice that help us move forward. What options are there for the different fields?

For software programmers, there is a variety of stuff out there. You can listen to a podcast on your way to work. You can read about the basics of your programming language. You can learn a new programming language every year. You can exercise in code katas and coding dojos, at times varying the constraints that you enforce on yourself. You can visit conferences, and user groups, and exchange thoughts with others.

For software testers, there is also a variety of stuff out there. Of course, there are podcasts. There is free learning material out there. For example the BBST course series, and the RST course have their material posted publicly so that you can work through in your own pace. Of course, you can also take one of the courses, and challenge yourself. Beyond that there are testing challenges put onto you publicly, or in Weekend Testing sessions, or in Testing Dojos. There are also coaches out there that can help you advance in your profession. Oh, and don’t forget the various testing related conferences, user groups, and meet-ups.

For other fields, there are similar things that you might pick. I am not aware of other fields. If you are, others probably would be glad if you can leave a comment with your addendum.

Does it help?

Now comes the difficult question. How do you know which one to pick? How do you know which one helps you at a particular topic? How do you know you are advancing in your craft?

Think about it for yourself right now. How do you know whether you are advancing?

I let this sit for a moment.

Ready?

The problem is, you don’t.

Alright, here is the longer explanation.

Some folks relate higher pay with advancement in their craft. Just like the school system though, a higher pay merely relates with someone else telling you that you are delivering the results that he is looking for thus providing you more money for these results. So, a higher pay from your employer merely tells you whether you are able to produce more of what he is seeking. So, you are referring the answer to your boss. How does your boss know whether you are advancing? How does your boss know whether he is on the right track with evaluating you?

The problem is, he doesn’t. He might claim so, but probably relies on a bunch of second-order surrogate measurements to do so. Oh, and of course, even if he doesn’t, how does he know whether the community he is involved in really moves forward, and you’re not merely adhering to a set of standards that does more harm than good?

Hard question.

The same holds true if you pay attention to your outside world, like professional organizations, different communities, etc. Of course, the combined view of several “experts” provides some clues whether you could be advancing. Still, I find this measurement rather shallow. The problem probably is that we are mostly relying on other people telling us. Oh, and don’t forget that all these indicators are trailing indicators. You can tell when you have gone way off-road. But the indicators can’t tell you which one thing to pick now in order to put your energy from the 10,000 hours budget on, in order to become an expert.

Now, enter my critical self. A few years back, I was asked similar questions by James Bach. He provided me the solution that I have a back-up of a larger community that shaped me, and that has answers to all the questions at heart. My critical self still questions that. How do I really know that I am not misleading all these folks? In the end, there have been folks before me that told some things, that turned out to be wrong in our current understanding.

The Advancement Fallacy

I truly think we cannot know what will be advancing ourselves. Us humans are terrible at predicting the future. We cannot tell whether our current picture of ourselves and the problems that we face is the correct picture. We cannot know whether a particular topic will attack the struggles we currently have.

That said, the only option I see is to pick something, anything, and start experimenting with it. You will find out whether it is advancing yourself – over time. So, make sure that you can opt out of it. Make sure that you put such an amount of energy into that, that helps you to evaluate yourself. If your energy drains, leave it. Probably you’re done with it. That’s ok.

When it comes to stuff like Weekend Testing, if you have become a facilitator, you should watch for other people that can step into your footsteps. This is the harder problem. There might be volunteers, there might not. If there are not, it might mean that you are actually shutting down all the effort that you have put into it, and no one is stepping in your footsteps. That should be ok with you as well. Don’t get too much hung up on your deliberate practice. Remember, it’s your exercise time.

However, always remember that you should enjoy the amount of work and energy that you put into it. It’s your free time. That should be fun. If it isn’t, step out of it. It surely will not help you advance any more than necessary.

Of course, there are times where you need to push yourself. I have this when exercising for running right now. That does not mean that I don’t enjoy every single run. It needs some stretch for me, but it’s worth my time. Keep that in mind before prematurely leaving some of your deliberate practice behind.

The No-Fake-Tester Rule

Have you every worked with a fake tester? How would you notice? How would you notice how much they are faking? Triggered by a discussion back at DEWT 4, I had an insight triggered by a book that I read earlier in my life: The No-Asshole Rule from Bob Sutton. Let’s see how fake testers and the no asshole rule connect in our workplaces.

The No-Asshole Rule

Bob Sutton makes a point in his brilliant Harvard Business Review article on assholes in our workplaces. He claims that assholes infect your company. If you hired an asshole, get rid of her. If you work with an asshole, switch jobs – now. Why?

Assholes around you infect your own habits. They infect others around, and inspire you to become an asshole yourself. By being treated from an asshole, you react in a similar manner. Over a short period of time, you will notice that you become an asshole yourself. You react the same way. You treat others as being minor. You treat yourself in an egoistic way.

The best solution that Sutton found in his work, is to get rid of the root-cause of the problem quickly. That means to either fire the asshole in your organization if you are the one with that power. Or to leave that workplace before you become infected yourself with the asshole virus. Sutton makes the claim that many organizations would be way better off if they were relentless about firing assholes. They in fact should install a rule to hire (or keep) no assholes in the organization because the overall productivity will go way up without all that asshole fighting happening all the time. Morale will be better in a workplace without assholes. Productivity will be way better. Fighting will be less. Economically a workplace without assholes will be way better compared to one that has installed a single asshole.

Fake Testers

What are fake testers? Fake testers are testers that take the requirements documents, use Microsoft Word’s search & replace function to replace all “should” to “verify”, and save that document as their test plan. Fake testers are testers that steadily increase the amount of tests executed counter on their desk for management reporting. Fake testers are testers that surf the web while executing test cases.

What’s the problem with fake testers? They make the impression that they are working while they actually aren’t. Fake testers maximize the impression that they are busy all day to the extent that you rarely want to provide them with more work while they are actually fetching new coffee all day.

Fake testers have an impact on your team morale. Of course, whether management attention focuses on fake testers or not, their team mates will notice over time whether they executed the amount of tests that they report in their numbers. Team colleagues will notice that they are faking. Most of the time they will even know how to make procrastination look like work – even more so if they are measured by the wrong numbers with surrogate measurements, but don’t get me started on these.

So, your loyal testers will know that someone is faking work, but then what? These loyal testers will wonder why they get the same raise as that other guy that is faking work. Why are they putting so much effort into their work while they could actually do that other fun stuff over there. Yeah, right, some of you, dear readers, will tell me “but I ain’t faking”, “I am morally giving my best”, blablabla. But I tell you – and so does Sutton – the majority of you will become a fake tester on their own in such an environment. They will find ways to deliver the same results with fewer effort – just because someone else can do so. Oh, they won’t actually deliver the same outcome, just the same output. They will make the impression to deliver the same results, while not providing the same value.

Over time, you will get a working environment of fake testers. They will drive down morale to the point where good people leave. At that point, you will have a fake testing organization, while still wondering why your bug metrics look correct. That will be too late – for you and your customers.

Are you faking?

I think to some extent we might be faking at times. It’s hard to tell whether you are fake testing right now while reading my blog entry, or whether you are using your power of procrastination. Call it whatever you would like to call it, but I think to some extent we are all faking work at times.

Just as we all are acting as assholes at times – especially so during emotionally stressful challenges that life might put on our own. The same applies to fake testing. Us humans can’t perform the same way every day at work. There are phases where we are more productive, and phases where we are less productive. If your productivity shows no variation, that would make me suspicious.

The thing is, you shouldn’t cross a certain boundary when you notice you are faking or becoming an asshole. At that point, you might become the asshole that draws down productivity in the overall company, and should probably leave.

But how do you notice that you are the problem? Just as the classic joke with the ghost driver goes, if you find yourself surrounded by ghost drivers all over the road, chances are, it’s you that is the ghost driver. If you find yourself surrounded with assholes, chances are it’s you that is the asshole. If you find yourself surrounded with fake testers, chances are it’s you who started that.

Don’t go down that road. Remember to do a good job, and try to minimize negative effects as best as you can.

The No-Fake-Tester Rule

But my claim goes further. I think we should go way further. Instead of throwing out fake testers from our companies, I think we need to fire fake testers from our profession. We should publicly announce their names, and tell the whole industry that they should not hire that guy.

The profession of software testing has taken serious damage by these fake testers. Programmers, project managers, and customers are suspicious about the value that we can provide. And I think they are rightly so. We should be able to explain ourselves. We should be able to explain what we are doing, and why we are doing it, and how we are providing value to them by doing it. If we are not, then we are setting up ourselves to disappoint more people around us, and leave a negative impression on the profession for the generations yet to come. We shouldn’t do that.

We should install a No-Fake-Tester Rule deeply into our profession, and get rid of anyone who is providing a harmful disservice to their clients, their stakeholders, and our profession. It’s not too late to face the problem, as long as we decide against becoming that asshole or fake tester on our own.

Do you want to join?

On Specialization

Over the past few months I have become more and more convinced about the fact that specialization in software development – which includes programming, design, architecture, and testing to me – is a terrible idea. I don’t know when this came up in the history of our craft, and I think we need to stop it – now. I also know that I am probably upsetting some folks with these statements, still I think you should critically read this blog entry, and make up your own mind about the matter.

Relays vs. self-organization

The agile community uses an analogy when comparing specialization and cross-functional teams. A cross-functional team consists of everyone that you need to get a particular job done. For software development that includes skills in business analysis, programming, software architecture, and design as well as software testing. For football (in the European sense of the game) this includes defenders, strikers, and probably a goal keeper. Even though a defender’s main job is to get the ball from the goal zone in a game, he is also able to score a goal. The same holds true for other team-based sports.

Compare this to sequential activities. (Hope I am not drawing a false dichotomy here.) In sports, when running a relay, you have a more sequential activity. One runner starts, passes on the baton to the next one, until all four runners have finished their job. Most of the same points hold true for sequential software development.

As you may know, I have been involved in swimming a bunch of years. We also had relay competitions back then. In comparison to running, in the swimming world there are a couple of reasons a swimmer might be disqualified. For example, if he’s swimming breast-stroke, and ends the lane by putting just one hand on the wall, the swimmer is disqualified. In relays, if a swimmer starts before the swimmer before him has finished, and put the hand on the wall, the whole team will be disqualified. There are about 100 rules why a swimmer or team might be disqualified.

Now, the thought that occurred to me just last week was, that in relay swimming a single point of failure exists. If you have a bad swimmer who’s making a mistake, he can disqualify the whole team. The same is true for other relay sports – even for other sequential activities. The specialist becomes the single point of failure. One guy screws up, bang, the whole team will be cursed. (Maybe that is why there is so much focus put on blaming each other in these cultures.)

Now, compare this team sports like football, rugby, or the cooperative game of software development. There are occasions where the whole team might get disqualified – but these are rare cases like someone from your team taking steroids. Yeah, I know these happen, too, and they are heavily discussed by the larger public. (Maybe that’s good.)

But if an ice hockey player is thrown out of the game for a penalty, then the team needs to find ways to adapt, and still either score another goal, or avoid getting goals scored against them. They need to collaborate to compensate for the situation. That’s probably when you need some specialist skill, but you will need more generalists – people that can score a goal, and defend the own goal.

You need to shift your thinking from blaming each other (which would paralyze you from doing anything good) towards constructively finding ways to win the game, still. You need to work together with your teammates in order to win as a team.

Also notice that a team in sports has more degrees of freedom – within the constraints set by the coach, and the referee (and the underlying rule system). Within these constraints the teams need to navigate through the complex problem of getting the ball to score a goal, and defending against players of the other team.

Also notice that it does not make sense to ask every player in a team sport to run as fast as they can all the time. For football, you would end up with a team that is tired out after 30 minutes, probably. Of course, they need to run fast, but they also need to take their time for tactics. In relay sports it makes more sense to ask everyone to run as fast (and as efficient) as possible. In a team sport more efficient on the individual level means being less effective on the whole team level.

The same holds true in software development. With team-based software development you have more degrees of freedom to navigate through the complex field of the cooperative game of software development. You also increase your flexibility with a more cross-skilled team that has fewer specialists on board. More and more this factor becomes crucial in order to have a competitive advantage in the modern world.

Not that I am not saying that you should have a team consisting of generalists alone. That would be a false dichotomy. Instead, you should have “just enough” specialization, and diverse skills on the team. The problem, though, is an over-reliance we currently face in most companies I have seen with specialists that are no longer able to work as part of a cross-functional team. I think we need to move away from that deep-specialization-thinking in our craft.

Specialization and ego

There is one problem with the transition from specialized experts to cross-functional teams. Most of these specialists have followed a career-path towards this specialization. For example, a tester started as tester, and became a test manager over time because that was the way to move forward in her career.

Now, if you confront these folks with the idea of generalization, and cross-functional teams, you might be telling these folks that the path they followed for the past ten years was pretty much worthless to get the job done. Ouch. I would hate that, too.

The problem is that the ego kicks in for these folks. People want to feel recognized, people want to know a clear path to their advancing in their career. I certainly can understand that. What I learned from egoless programming is that we shouldn’t adhere to our past career path too much. Just as we should detach from the code that we once wrote in order to constructively work with the feedback from reviewers, we should also be open to overthink our current career direction, and try to take a different perspective on how to actually build software that solves a problem for someone that matters again.

I know this is hard. I know this will be a long journey for all of us. I know this will also mean to re-think some of our underlying assumptions about software development, management, and whatever specialization we followed (thus far). I truly believe the re-evaluation of some of our assumption will lead to some serious innovations in our field – and eventually we might find out how to constructively work together with our customers again. I think this goal is worth giving up some of my ego. How about you?

What is Agile Testing?

While I was cleaning u my bag that I carry around with on various travels, I found the following notes. They were part of my introduction for the keynote at the Agile Testing Days 2012 in Potsdam, Germany. My talk was on “adaptation and Improvisation”, drawing an analogy from the famous geek movie “The Matrix”. I wanted to keep these notes somewhere. That’s why I decided to put them up on my blog. Hope you like it. Imagine Morpheus’ and Trinitiy’s voice.

I don’t know if you’re ready to see what I want to show you. But unfortunately you and I have run out of time.

Let me tell you why you’re here. You’re here because you know something. What you know, you can’t explain. But you feel it. You felt it your entire life. That there is something wrong with the world. You don’t know what it is, but it’s there. Like a splinter in your mind driving you mad. It’S this feeling that brought you here.

It’S the question that drives us. It’s the question that brought you here. You know the question… just as I did:

Whats it Agile Testing?

The answer is out there, and it will find you, if you let it to.

Agile Testing is everywhere. It is all around us. Even now in this very room. You can see it when you look out your window, or when you turn on your monitor. You can feel it when you go to work, when you go to lunch, when you pay your technical debt. It is the world that has been pulled over your eyes to blind you from the truth.

What truth?

That you are a slave. Like everyone else you were born into bondage, born into a prison that you cannot smell or taste or touch. A prison for your mind.

Unfortunately no one can be told what Agile Testing is. You have to see it for yourself.

What is Agile Testing? Control.

Remember, rules, some of them can be bet. Others can be broken.

Like any rules, all good rules should not be applied with unthinking faith.

ATDD in scaled agile environments

Years ago, I wrote a book on ATDD. Only years later I notice the value that this practice can bring to an environment with multiple teams working together on the same platform. The connection to scaling agile in the larger enterprise wasn’t that obvious than it became when I started to dive deeper into how to scale agile. Though, most larger enterprise struggle with getting one team running, scaling has become the latest fuzz in the agile community. Let’s see why ATDD is relevant for this sort of environment, and how you can challenge your team colleagues about it.

Continuous Integration

In XP explained second edition Kent Beck writes about different practices from extreme programming, and raises the concern that some practices need certain others to become more successful. There are a bunch of practices that I consider crucial to have in place before you will receive crucial benefits from implementing ATDD.

For example, you should have an automated build in place. A couple of years ago, I was working with a team where this was an issue. I really hope that nowadays programmers have proper coding practices in place to no longer rely on their IDE alone when it comes to building their software system.

But even more relevant with ATDD is to have proper continuous integration in place. What is continuous integration? Continuous integration does not necessarily involve a continuous build system. Continuous integration means that you integrate your working environment with the source code repository regularly. Continuous integration brings transparency to your colleagues that work on the same system.

A few years ago, I worked with a client where we had to deal with some confusion around continuous integration. It appeared that some programmers thought they needed to have a build pipeline in a build system. If you use a CI tool to have a working build environment, and you can’t reproduce those build steps on your programmer machine, then you are doing something wrong.

Integrate early, integrate often. Often means that you should be able to check in your code more than once per day. Usually that is already a large problem in most environments that I have seen. And these problems are solvable. They need discipline from the programmers, and they necessitate a paradigm-shift for most programmers in more traditional environments.

When I was a swimming trainer, I learned about the concept of over-correction. Maybe that’s why us programmer coaches challenge programmers to be able to check in their changes every two minutes – simply because so many programmers are so terrible at this.

Conway’s Law and bounded contexts

Now, think about a multi-team environment. With multi, I mean 20 teams working on the same code base. There are probably a couple of team members working on the same foundation. And they check in their changes a couple of times per day.

Ouch, that might cause a lot of trouble. Right.

In order to help these team members coordinate between each other better, you need to have good coding practices in place if you want to overcome Conway’s Law. Conway’s Law?

Malcolm Conway described this law in 1968. He reported about two different teams he had observed. One was developing a COBOL compiler, the other an ALGOL compiler. The COBOL team consisted of five team members, and the resulting compiler had five phases. The ALGOL team consisted of three members, and the resulting compiler had three phases.

Interesting.

If you want to avoid that your software architecture replicates your organization architecture, then you need to be able to work on the same basic building blocks. In domain-driven design Eric Evans talks about so called bounded contexts in which you solve the business problem in a separate building block. You also re-use pieces supporting contexts – like a commonly used infrastructure.

If 15 of the twenty teams work in parallel on these same supporting infrastructure, chances are high that they change similar portions of the code. This raises the question how to avoid integration conflicts? And how do you avoid that you break a particular functionality in an unfamiliar bounded context of another team?

The answer is that you need something that transports the tacit knowledge of the other code base to your environment.

ATDD

Enter ATDD. With ATDD you document what your particular piece of code should solve from a business perspective in a way that other teams might understand. After you identified different business scenarios, you automate these. The more and more you execute these business scenarios, the more they become the documentation of your software.

This documentation will be up-to-date as long as these tests run successfully. They will document the underlying assumptions about your bounded context, and they will make some of your tacit knowledge explicit for other team members and teams.

That is also why these automated acceptance tests will help you solve the problem of multiple teams working on the same code base. They no longer need to guess how to solve a particular problem they introduced for another requirement area when adapting a particular functionality in the supporting domain. They will be able to execute the same set of business scenarios that you used when you introduced the feature a couple of iterations back.

Thus, in a multi-team environment, given some disciplined practices in place, you will be able to use ATDD to coordinate between teams, and find important problems quickly.

So, make sure that you have proper continuous integration in place before starting with ATDD. You should be able to commit your changes to the production multiple times per day. If you are not yet able to do so, fix that problem in your development process first. Otherwise you won’t have the transparency soon enough to trigger the right set of actions when you break some unknown code. Problems will crawl in unnoticed until it is too late. You will become more defensive with your coding practices, and slow down since you have lost trust in many of your changes. Don’t go down that path.

When you get started, be aware that you will collide with some of your colleagues – maybe even oftentimes per day. Remember that this is good. You notice problems before shipping your code. That is good. By feeling that pain, you now know better where improvements could provide better results. Use that input instead of fighting it.

All that these pain points are really telling you, is where your current team setting is causing a lot more coordinate overhead. You have to deal with that because you haven’t (yet) found a way to deal with it better. Sit together with the teams that you oftentimes have conflicts with, and try to find a solution to these problems instead of fighting and hating each other. Your software will benefit from it, your product owners will benefit from it, and your customers will benefit from it.