Markus Gärtner's Blog, page 24

"Say, how many books did you read this year?"

At the beginning of November I attended a conference together with my boss Henning Wolf. While flying back to Hamburg, waiting for our plane, we talked about things, and I mentioned some lessons from a book that I was reading at that time. "Say, how many books do you read within a year?" he asked. I couldn't answer that question directly, as keeping in mind that this was my seventeenth book would distract me from reading the content. So, I looked it up, and was amazed.

I am sure, I do not exceed the number of books that for example Michael Larsen read this year, but I was still amazed about the number – having estimated it at about ten or twelve. I decided to visit back the books I read, and see which lessons stayed current even after having read them. This list is based upon my notes over at Library Thing, where I looked up which books I finished this year. Some of them I started back in 2010. Some of them are also in German. some have an English translation, others don't. Maybe time learning some German for some of my readers. :)

Training From the Back of the Room! – Sharon L. Bowman

I made this an expedite in my reading list. My colleague Stefan Roock visited the course back in 2010, and reported great things from it. I started to read this book back in January, read through in one week, and immediately started to incorporate the lessons into my training. I received some new inspiration from Sharon back in November when I visited her course as well. Great book, great insights, a must-read for any trainer and workshop moderator.

How to Read a Book – Mortime J. Adler, Charles Van Doren

I started to read this back in 2010. I got inspired by it on how I could test better. Initially this was a recommendation from Ilja Preuss with whom I exchanged some e-mails back before my time at it-agile – and also his. It explains why you shouldn't bother reading every book, because some of them are not worthwhile to read cover to cover. This is especially the case when you face a book that is written badly, or that has some beginners content which you are already familiar with. The book also explains how to find out about it. Recently I worked through my to read list, and prioritized the 30 or so to read books based on the recommendations from this book. While the title might be confusing, definitely a book worth reading – maybe not cover to cover, but that's how I did it.

Please Understand Me II – David Kirsey

Having dug deeper in the first part of this series, I was wondering what I would face when starting this one. This is also a relict from 2010. I enjoyed the insights about the Myers-Briggs Type Indicator from this book, but also the different temperaments which you may find in the different SP, SJ, NT and NF preferences. I learned some new stuff which I also incorporated in my trainings on a meta-level. For example, I remember one training where I was facing a detailed-oriented S-preference and knew based on this book what I had to do. Today, I might question the lesson I tried back then, but it helped in this particular situation. If you're unsure about other people surrounding you, you should probably read one of these – but be prepared that this is a psychology book. I had no problem with that, since I had educational science back in school with a psychology teacher. But you might find it dry to read.

Kanban – David J. Anderson

This is another expedite book in my reading list for 2011. I decided to jump deeper into the topic of Kanban back in February. My colleagues were discussing how to extend our knowledge about Kanban and our ability to coach companies adopting Kanban. I felt I had to dig deeper. I found this book interesting to read, but at times the lessons felt a bit too managerial. Being a tester that is suspicious about any metrics, I also had problems reading this part. I think Kanban is an interesting method, and it surely will evolve over the course of the next years even more.

Exploring Requirements – Quality before Design – Don C. Gause, Gerald M. Weinberg

Since I read Bridging the Communication Gap back in 2009 I had this on my book shelf. Finally I got to read this book. I enjoyed it. Especially I enjoyed reading about different requirements elicitation techniques, as well as the Mary Had a Little Lamb Heuristic. I applied it from time to time on some particular pieces on my blog – which readers thought to be a bit nitpicking. Unfortunately I was already aware of most of the lessons. Yet the authors achieved to teach me some new things as well.

Weinberg on Writing – The Fieldstone Method – Gerald M. Weinberg

A book on writing by an author that has written more than 50 technology books in the past 50 years. This was an awesome read. Since I am working on some pieces of writing myself, Jerry could give me some great hints on what to try out, and how to digest ideas. I also watched two different Alive in Wonderland movies as a result of reading this book. Great tips from a book writing guru.

Pömpel, Patt und Pillepoppen – Matthias E. Borner

This is a book on the German dialect as it is used in the region of Bielefeld – the place where I grew up and studied. It is covered with some stories about Bielefeld, a small city with a 300k population. I enjoyed diving out of the usual readings in English, and reading some humor back at that time.

Visual Meetings: How Graphics, Sticky Notes and Idea Mapping Can Transform Group Productivity – David Sibbet

Back in February we organized a Visual Facilitation course in our company. I finally learned how to draw flipcharts nicely, and what to do to make them colorful and engaging. As a result, I ordered several books on visual facilitation afterwards. This is the first one I started to read. Oh dear, I have to train my flipchart skills some more. Recently I found out that I still have to work more on them. Maybe a resolution for 2012.

Behind Closed Doors: Secrets of Great Management – Johanna Rothman, Esther Derby

Before attending the Problem-solving leadership course back in May, I already had this book. I asked both Johanna and Esther to sign it for me back in Albuquerque, NM. I enjoyed the management story alongside the explanations about good management. It's not a business novel alone. It's a business novel aided with theoretical insights about management. I think I read this in two maybe three weeks, which is quite fast for me. And you really had to force me to turn this down.

Pömpel, Patt und Pillepoppen 2 – Matthias E. Borner

The second volume from the Bielefeld dialect. This is the volume where I ran into the following saying for the first time:

Bielefeld is the nicest city in the world. Everyone who says other lies – or has been somewhere else.

Having been somewhere else, I think the author is right about it. :)

Rethinking Systems Analysis and Design – Gerald M. Weinberg

I enjoyed reading this book on system analysis and design. In one of the first few chapters, Jerry claims that you will think differently about design once you finish this book. He was right about it. He raises many questions about what we call software design, and really makes you think about it differently by doing so. A must read for any wannabe designer. I think Design Thinking could be originated based on this book.

xkcd – Volume 0 – Randall Munroe

This is an awesome geek comic on the web. I had to read the book once I saw it came out. It covers many comics based on science, geek-wisdom, and other stuff. A must read for geeks.

Give and Take – Chester L. Karrass

This was a recommendation from Jerry Weinberg at PSL to me. At dinner I joined Esther Derby and Jerry Weinberg for some personal coaching. On our way back to the hotel, we discussed the issue of the psychological contracts. Jerry recommended this book to me. I enjoyed reading it, and it cleared up my mind for things I can do to set expectations about my work more clearly, and how to work with concessions. At times I found the structure a bit unnatural, but as a lookup reference it's awesome.

5 Very Good Reasons to Punch a Dolphin in the Mouth (And Other Useful Guides) – Matthew Inman

This is the first Oatmeal book. Another one in the series of the comics I enjoy reading. Humorous at times, thought-provoking at other times.

Runtime Error: Not Invented Here Book 1 – Bill Barnes, Paul Southworth

Another web comic I read regularly. This one covers the first few episodes of the Not Invented Here comic. A strip on a software development comic which seems dysfunctional at times.

The Gift of Time – Fiona Charles (ed.)

This was a gift – both for me as well as for Jerry Weinberg. Back in 2010 I asked Michael Bolton whether he had written any book, as I wanted to get a sign from him on one of his pieces. He said, there was a chapter from him in this book. I couldn't find a copy, so he brought it to me when we met for the first time at the Agile Testing Days in Berlin. The book is a gift for Jerry Weinberg's 75th birthday in 2008. It covers a lot of stories about his life, his courses on PSL and the SHAPE forums, and how he helped to gain the testing profession more momentum. It's a must read for any self-claimed student of Jerry.

Agile in a Flash: Speed-Learning Agile Software Development – Jeff Langer, Tim Ottinger und Susannah Pfalzer

This is not necessarily a book, but a set of cards. Each card has one aspect of Agile software development on it. It gave me some insights, and re-taught things I already had forgotten. I had the urge to introduce it to a client of mine for lunch and learn events. It worked to some degree. Since then I am reconsidering how to use it with clients. I think it's awesome, but might be too dense for Agile beginners.

Computer Programming Fundamentals – Herbert D. Leeds, Gerald M. Weinberg

This book is 50 years old. Yet, I found it interesting to read about things that still hold true in the first few chapters. The book seems outdated at times, but it covers the field of software testing for the first time in our industry. After reading James Bach's chapter in the Gift of Time, I had to dive into this book. I was glad to find a used copy, as you can't buy a new copy of it. While the technology seems a bit outdated, the psychological insights about programming are still up-to-date. If this doesn't scare you, then maybe one of these quotes I noted down while reading it:

All good rules should not be applied with unthinking faith.

One of the more dangerous occupational hazards in computing is the habit of working out a set of diagrams, formulas, and figures until some impressive statement like "twice as efficient" emerges.

If we produce a new vocabulary word (subroutine) we must not produce something sloppy or something that may be misunderstood or misinterpreted.

Knowledge of only the instructions on a computer will permit one to code; but to be a programmer of any professional standing, computer operation must be understood.

Facilitating Organization Change: Lessons From Complexity Science – Edwin E. Olson, Glenda Eyowang

When Diana Larsen visited us back in July, she spoke about a program called Human Systems Dynamics. One of my colleagues, Jens Coldewey immediately booked this certification course as well. I decided to first dive into the topic some more before coming up with a decision about it. This book covers some aspects of complexity thinking. It states that uncertainty is fine in self-organized systems, that you can work with containers, differences, and exchanges to bring in organizational change. The model seems to have some power. Currently I am working on getting the pieces from PSL together with this model from Complexity Thinking. I hope to get my thoughts down soon on it.

Ten Years of Userfriendly.Org – Illiad D. Frazer

I think userfriendly was the first comic I read online. This book covers ten years of it – nearly completely. It's a pity that Illiad had to turn to different priorities recently. I don't know when I started reading this book. It must have been back in 2009. I finally read through all the comics. It's thick, heavy, and I liked it a lot.

Geschichten vom Scrum: Von Sprints, Retrospektiven und agilen Werten – Holger Koschek

This is a little novel on Scrum. My first thoughts were that this book started a bit too slow for me. Over time I enjoyed the tiny tale about the land of scrum where unicorns use Scrum for any project. You join a dragon trap team which uses Scrum to build a flexible dragon trap. Along the way the unicorn – the Scrum coach – explains some insights about Scrum to you. An awesome story, a nice book, unfortunately just available in German. The Power of Scrum could be an alternative for business people, but this is one goes deeper on the topic.

Die menschliche Seite des Projekterfolgs: Was Softwerker über (verborgene) Denkautomatismen und -modelle in der Projektarbeit wissen müssen – Peter Siwon

Another "only German" book. I think this is a must read for any German speaking student of Jerry Weinberg. It explains some of the things that we encounter in projects in a way that was easily accessible for me. It is nicely structured for an NT-type like myself, but might be too mechanic for an NF one. It covers the human side of project success as the title claims. The author digs really deep into the material, and tells a lot of stories about his studies of projects from about twenty years. A great read, but I think it's a bit biased towards the German culture – which is why it's ok to have it not translated, maybe. :)

Dilbert 2.0: 20 Years of Dilbert – Scott Adams

A selection of 20 years of Dilbert comics. Being comic fanatic, I enjoyed the insights about comic author life here. Adams describes how he got started as a comic writer, and how Dilbert took off. In Weinberg on Writing, Jerry writes that Scott Adams and Jerry had the same inspiration for some of their stories. Jerry for his writing, Adams for Dilbert. I love both of their styles.

Hope I could bring you some inspirations. I don't mind about the count. If you would like to read more about my insights from different books, please drop me a line, and I might publish them more often in 2012.

Some insights about TDD

At the recent meeting of Hamburg's Softwerkskammer – the German Software Craftsmanship movement – we worked through a Coding Dojo on the Roman Numeral Kata. Michael Norton wrote today about one piece that worried me as well. I think Michael did a fantastic job on tackling a different approach. But he reminded me that I wanted to put up some of the thoughts from the Coding Dojo.

We had about 12 participants in the dojo. After explaining some pieces about the format and the kata, we started the dojo. As the kata started, we had one participant asking questions up to the degree that the pair in front of the keyboard stopped doing anything.

Eventually we got the team back up to continue working on the problem. The claim of the interrupter was that we didn't yet understand the problem well enough to design the solution. Another claim was that TDD was not a design technique. Let's take a closer look at both of these.

We have to understand the problem before using TDD

You could argue that the Roman Numerals kata is not very complex. Yet it seems that it's possible to not grasp the domain of the future code model. Let's take a look on the problem.

The romans counted differently than we do now. They had letters for different numbers. "I" is 1, "V" is 5, "X" is 10. You combine two "I"s to yield a 2: "II". You combine two tens to yield a twenty: "XX". You combine five and three to yield an eight: "VIII". There are two exceptions to this concatenation. The four and the nine are generated by subtracting one from five, respectively ten: "IV" and "IX".

We started the kata with a test for one, then for two, then we were heading for three. The question was whether we should go for a test for four or five at that point. I recalled Kent Beck's exercise at his advanced TDD course in November 2010. He had us performing a kata. Then he asked us performing the same kata, but working the tests in a different than the usual order. The thing is that once you have a final design in mind, you work towards it. Now, the situation in the dojo was differently, as we didn't discuss a design.

Now, we were stuck at implementing a large construct of guard clauses. When we introduced the test for the five, then the test for the four, we finally made some progress. We had three instances of guard clauses that returned some value. All looked similar. We tried to see a different design when we were facing the situation with two similar blocks. The design we worked towards at that point seemed to get in our way once we introduced the four.

The case when we had three distinct blocks was easier, and led to a better design. Now, the question is, if I do not have a particular design in mind, can I still explore possible designs?

I think so. The thing is, that I have to stop thinking while refactoring. I have to start thinking again when I stop refactoring to see whether the design I'm heading towards is any good at all. Using continuous integration, I can then revert back to the old design, and go a little bit further.

TDD is not a design technique

This insight is mostly triggered by the realization that Exploratory Testing is not a test technique, but a test approach. This means that you can combine Exploratory Testing with any testing technique. It's orthogonal to testing techniques.

Similarly, TDD is not a design technique. It does not solve the problem of designing the source code for you. It helps you to think about the design though. You can combine TDD with any other design technique like UML, prototyping, and whatever you can think of. TDD is an approach to design. It helps you finding a design in your source code, rather than designing it. It's you that designs the source code.

I'd be interested in your thoughts on it in the comments.

A Christmas gift from a tester for a tester – Part Five

This is the fifth part of a series of blog entries on a gift I got from Matt Heusser. Today, I'm heading for tying together all the loose ends.

I was too lazy in the last entry to upload an update of the mindmap I created. In order to head for the final testing where I get together all the loose ends, I need to consider it, though. Here is the latest mindmap after the fourth part.

I was planning to go for the timing, and take a closer look on the inner parts. Let's start with timing – just in case I can't figure how to assemble the tiles after tearing them apart. Yesterday I found out that my old stop watch from my swim trainer time was out of batteries. So I had to use my iPhone. This means that it will have less precision as I could have with a stop watch.

For game 1 I measured a total time of one minute fifteen seconds when you don't do anything.

The manual claims that it adds five more seconds each time you find a five letter word. I validated that with one five letter word. Sorry, didn't take the time for the randomness to work for my advantage and validate it with two or three five letter words.

I went next to game 2. While trying to do so, I could reproduce the bug from my first actual test session. I decided to investigate it a bit more. It turns out that while you see the summary from the previous game, and press the button on the tiles with the 1, 2, or 3 on it, you might get into another game of the previous game. Timing is crucial here. If you press the button just when the summary will be displayed again, you can find yourself in the previous game despite the game you intended to choose. This seems to happen more frequently when your batteries got lower and lower on power, as the reaction time of the tile increases dramatically. I reproduced this also with the change from game 2 to game 1 while digging deeper. I also raised my priority for different lightning conditions, maybe even in combination with lower battery power. But this won't be part of my current mission.

Really jumping to game 2 I measured one minute thirty seconds for the initial round.

When you re-arrange and re-arrange more and more five letter word, your time will decrease. In between I measured roughly one minute.

I didn't go deeper in the differences between each round like I did for game 3. This was mainly caused by a problem I noticed. I had just re-arranged a five letter word, just before time was up. Then the timer started to tick down, the tiles blinked to indicate I formed a correct word, and the game was over. This seems to indicate that I got one minute thirty overall, rather than having some time for each five letter word. The manual does not state it like that. I was a bit confused, but noted this thing for a future session, maybe.

For game 3 I could reproduce my timings from yesterday. That is you get roughly 20 seconds for the first word, then it decreases by roughly one second each round. Nothing interesting.

Now, on to the next part. I was really curious about disassembling the tiles – finally some destruction. I was also curious whether I could reprogram the tiles. There was a German version of the game, so there must be something like that. Also, for game 1 it seemed to have some memory about the words you already formed. So, how does one tile look from the insight? Oh, one final thought before going for destruction: What about children taking the toy in their mouth? Was it water-proof? Let's see.

One piece of advice before we get started. I disassembled one piece unsure whether I will be able to re-assemble it afterwards. I did play around with electronic devices when I was younger, so I seemed a bit confident. Don't try this at home for yourself, kids. You might either destruct your game, or burn your house down – maybe. :)

Now, let's see. The manual states that I need a screw-driver to replace the batteries.

After opening the cover, I see two additional screws. So far there just seems to be the battery in there, nothing that really interests me. So, I continue to remove the two screws.

After tearing the tile into pieces, I see one main board, the button on the front has an additional plastic piece there, and I found two springs – unsure where they were originally placed.

I see four additional screws on the main board holding a cover for the display. It seems that there is some protection for the display so that little fingers won't crush the LCD. I remove that one as well.

I start to explore the pieces I can recognize. The black pieces to the left and right from the display are the optical (I suppose) connectors to interact with the other four tiles. The board has some interesting stuff in there, but nothing I can recognize on my own. I can't recognize a programmable interface here, and I don't see memory, but I am quite certain that it's too tiny for me to spot.

The connectors for the battery are not soldered with the board. I can remove them and place them back again. I really hope that I didn't screw up the piece now. I'll see.

I reassemble the tiles. The two springs seem to fit to the sound chip on the back panel. After putting all the screws back in, I try out the game, and still seems to work. Phew.

Tearing this piece apart didn't reveal any new dramatically information to me. But I learned some bits about the tiles, and what safety functions the inventor put into place. It doesn't seem to be water-proof. That's maybe why it's for ages eight and above.

Debriefing

So far, I have spent roughly two hours with the tiles. Another two hours for the write-ups. The game seems interesting, but also contains some inconsistencies. I saw one or two bugs there, one I couldn't really reproduce. There is still some testing open for the tiles, but I'll leave this to the ambitioned reader of my blog. I'd love to read about your approach if you do so.

The game seems to be ok for English speaking kids, but not so likely for foreign kids – even as an adult I had difficulties finding the words. If this is to provide a unique challenge for the game, it might be ok. The three different variants are also quite interesting, but in the long-run I don't expect much curiosity about it. There is just one multiplayer game, that's it. considering todays game consoles, kids might get bored pretty well. The tactile nature of the game, though, is a contrast to TV or computer games. For more multiplayer games, it would be interesting to combine two or three games to yield a bigger game with ten to fifteen letter words. Another dimension could be for kids to collect tiles, and yield more and more different variations. For example making explicit that you win one tile if you win the multiplayer game, then you could start collecting the pieces just as kids use to collect yugioh cards, or whatever is in fashion in your local area.

I hope you enjoyed this series. Looking forward to your thoughts and notes on it.

A Christmas gift from a tester for a tester – Part Four

This is the fourth part of a series of blog entries on a gift I got from Matt Heusser. Today, I'm heading for multiplayer games with our 11 year-old who has just same basic knowledge about English words.

As I started to prepare for this session, our 11 year-old, Leon, entered the room to play a bit on the computer of my wife. I asked him if I could use his service as a real usability tester of the product, and he agreed. Here you can see him in action.

We headed for the multiplayer game variant 3. The rules are simple. You get five letters, and some seconds to form a word out of them. If you achieve this, the next player takes turns. If you don't, you're out, the word is displayed, and the next player takes turn. If you're the last player, the game is over. With every new round, you get less and less time.

That's it.

Now, we went for our session. I explained Leon the rules of the game, and we started. Keeping in mind there are variants of five and four letter games, we started with five letters, and changed that later to four letters. Interestingly it was no possible to change within the game from a five letter to a four letter game, but it was impossible to change from four to five letters back without getting the main menu, and starting all over.

Leon claimed afterwards that the four letter game was easier, as there seemed to be easier words for a foreign speaker. I had the same experience the last time I played with the game, and it felt the same now as well.

We played a bunch of rounds. We didn't timebox it. Timeboxing with kids is difficult and easy at the same time. Once they get bored, you'll know that the game is pretty dull or they're demotivated. If they keep their attention, you'll have hard problems ending the timebox. Overall we used roughly half an hour to experiement with the product.

On usability, I agreed with Leon to help him with his turns. The five letter words seemed too hard to find for him – and even for me at times.

As the game started to bore us, we tried to measure a claim. The timing should decrease over time. So we started to notice whether the time we had to arrange the tiles was actually feeling to decrease as we played. After playing 20 or so rounds, we didn't notice that the time got shorter and shorter. At that point we started to try to actually measure the time we got left.

We found that we were having 10 seconds left at the point we started to measure. After a while this had come down to roughly 8 seconds, but didn't seem to decrease any more. After the session we formulated the hypothesis that the time just decreases if you find a word in the time. If you get thrown out, nothing happens. We didn't validate this hypothesis, so I put it down on my todo list for next time.

We also wanted to know how much time we have in the first round, and how much it decreases in each round. Now, there were some measure inconsistencies based on reaction times, and the like. We found the first round, we got roughly 21 seconds, and it seemed to decrease for one second each round.

There were two more things that were interesting in this session. First, Leon claimed there is an ad for the game on German TV as well. So it seems there is a German version of the game. I'd like to see how this one works compared to the English version. We could also run the German through someone with English as primary language. Nothing I may achieve over the holidays, but something for the long-run.

The second thing is we accidentially achieved to break the game, and had to reboot it. We couldn't reproduce the problem afterwards, though. We played around with removing a piece once the time was up, and the word was going to be displayed. Removing one tile at that point led to a stall up until I put the tile back, and the word was displayed again. The same happened for the "out" message. In one game we passed the game to the other played, maybe accidentially removing one piece thereby, the game got stalled, and we couldn't bring back the game to a normal, unstalled state. We tried to reproduce that, but didn't achieve it. This is a situation where I missed video recording of what we did, in order to reproduce our behavior.

In the next and final blog entry in this series I am going to head for the timing, and taking a closer look on the inner live of the tiles. I also want to see that ad on German TV. So, stay tuned for the next entry.

A Christmas gift from a tester for a tester – Part Three

This is the third part of a series of blog entries on a gift I got from Matt Heusser. Today, I'm heading for the first actual games.

First of all, I brought in a Time Timer. I am going for a timeboxed session of 25 minutes of uninterrupted testing. The Time Timer will indicate to me how much time I have left.

Game 1

I start studying the manual. What is game 1 all about? It seems that I get five letters, and have to re-arrange them in order to gain a time credit of five seconds each time. I can re-use the same words as well, but the tiles will not flash then. I start playing three to four times with game 1. I get some medium score in the beginning, eventually improving over time. But it seems that I won't make the maximum score.

I get bored – notice my felling – and get on. There was a variation stated in the manual regarding four tile games. I start game 1 with only four tiles, hoping to be able to improve my success with the game. After some more round with four tiles, I eventually find out, that I really can improve my English treasury of words – especially the two, three, four and five letter ones.

As it turns out, game 1 allows me to arrange words with two, three, four and five letters. I get scored by these. After time is up, I get my current score for this round alongside with the maximum possible score. Everything seems fine with game 1 for me so far. Taking a closer look on the time left, I go for game 2 as well since I got 15 minutes left.

Game 2

Game 2 is a bit different. I refuse to read the manual up-front, but instead go for the game directly. The first round seems to be the same as game 1. After that round I wonder what the actual difference to game 1 is supposed to be. I read the manual, only to find out that the letters now get re-shuffled once I formed a valid five letter word. Interestingly this didn't seem to happen. I note this down as an area to look deeper into, and start to play game 2 again. This time everything seems fine.

After time is up, I get one possible word for the current setting of letters alongside with my score. I try a few more games, even with four tiles, but get pretty bored fast. My English seems to be too bad for me to go along. I wonder what our 11-year-old will come up with when he plays the game.

Debriefing

Besides the additional session on the problem when I started game 2, but actually received game 1, I didn't notice a problem. On internationalization I found out that it doesn't seem to recognize German words like "Leid".

This is going to be an interesting session. I am looking forward to game 3 in the next blog entry. Here is the updated mindmap with the current state of my testing.

To be continued in the next part of this series.

A Christmas gift from a tester for a tester – Part Two

This is the second part of a series of blog entries on a gift I got from Matt Heusser. Today, I'm heading for touring the product.

Well, I know scrabble. I played quite some times, but I don't like it that much. Now, this seems to be an electronic version of it. I started to get down my thoughts in a mind map. I identified three different things I would be interested in before opening the package. I plan to use session-based test management to test this product over the few hours.

My initial mission was to seek knowledge about the product, and see if I need additional missions that I didn't think of initially. Let's see where the first pursuit leads me to.

What's the product about?

I open the package. There does not seem to be any documentation included. Just a tiny black box, covered with a bit of dust from the cardboard box, which has the name of the product printed on it.

I open the black box, and find the documentation in the cover.

There are two instructions in there. One is for the rules of the game. It's labeled "Rules for 1+ players Ages 8+". This makes me wonder whether there was something about the amount of players indicated on the original cardboard box. I take a look, no it wasn't. My sense for the similar products heuristics strikes me. Other games tell you on its box how many players can play with this. Considering I had bought this in a regular shop, and just knew the rules of Scrabble, the board game, I would have thought this is a 2+ players game, too. Just after unpacking I found out, I can actually play this alone. I suspect there could be a problem with potential customers who are unaware of this fact – just as I was.

I continue to look on the second piece of paper. It's heading tells me to read this before playing – I wonder what would happen if I didn't, and read on to find out.

It states that I should remove the plastic covers from the tiles, always put them in a single line, and finally there are troubleshooting steps. In the first part I notice that the paper states that recycling the plastic is optional. I may also put it in my regular trash. Considering the world's ecological problems, I note that it should state to always recycle it which could make this world a bit of a better place to live in.

For the sake of the moment I largely ignore the troubleshooting guidelines, besides the fact that each tile has some sort of battery in it. I put this one up in my mindmap on testing this product, as I am now interested in whether I am able to actually get replacement batteries for the tiles here in Europe. The backside of the manual tells me I need a particular type of battery, but I make this a separate charter. Another thing from the safety instruction indicate me, that there is a screw and a reset button. There is no screwdriver included (the manual also says that on that back), and how good is the reset button accessible are two more things I consider worthwhile to check.

Regarding the screwdriver, you may use a cross head or a straight head screwdriver. That's also what the manual tells me. I think I'm done with this consideration. I keep the reset button as an additional charter for the future, as I want to play first.

Next, I wonder what would happen if I did not put the tiles in one line. In order to find out, I have to read more on the rules about the game, actually playing a round or two. I am now interested in how precisely I have to get the tiles in line. I'm going to find out.

From the manual I find out that there are actually three different games I can play. I put them in my mindmap to find out more about them later. Game 1 and 2 are for single player, game 3 is for three players. The manual states that the tiles will power-down after three minutes of inactivity automatically. I wonder whether this is a problem for me as a non-native speaking person. I also note that there is a four-tile variation of the game. I get that one down as well.

I continue with my self-set mission to see for the line-up boundaries. I remove the plastics from the tiles, and start playing around. There is a menu for picking game 1, 2, 3 that should appear when I get the pieces in line.

The tiles show some animation while not connected together. I start playing around with the tiles, but nothing really indicates a menu unless it get at least four of them in line.

I start moving individual items out of line. Once the connection get lost, all five pieces start with the animation again.

Another thing I'm interested in the connection is the distance between individual pieces. I can get them apart from each other and still maintain the connection. This seems to be robust enough for 8 year-olds to me, but I plan on some real action with some kids later.

Accidentially I also notice another thing. When I put the piece with the "1″ on it apart, the "2″ gets a "1″, the "3″ gets a "2″, and the second to last piece in the line gets a "3″, thereby indicating a four piece game to be started. I test around this for some time, to find out what happens. I notice that there is a short delay between the pieces finding each other, and getting lost again. I notice that the animation starts on the lost tile first then on the others. It's about a split second delay, but I can still notice it.

Finally, I put the tiles up to extremes. I put them in line with as much offset as I can. Here is the final picture of the tiles still maintaining the connection.

I take a closer look on the interface between the pieces. One thing I notice is that it's a tiny hole in there. Seems to be an optic interface. I take a note to visit this topic under different lightning conditions.

One final thing for this session is what happens when I turn one piece around. Since all pieces have two connectors, this could be possible. But it seems that the inventor thought about this as well. There seems to be an inbound and outbound connector, or something.

I end my session after this test.

Debriefing

After this first session with Scrabble Flash, I have now gained some knowledge about the product. There are three different game types with two different numbers of tiles. I plan to explore each of the games in a separate challenge. There are some subtle things I crossed over while testing. I have noted them in my mindmap. Here's the updated one which also contains the priority of the sessions yet to come.

I also take notice of my thoughts about the product so far. I think it will be hard for me to come up with four- and five-letter words in English. This scares me a bit. Probably for the 8-year-old testing opportunity, I may ask my 11-year-old son who has had four years of English classes so far. I also noticed that the product seemed robust to me in this first session. This makes me curious about the first actual need of the troubleshooting instructions.

After all, testing games always is fun, so I look forward to my next mission, which will be around Game 1. To be continued in the next blog entry in this series.

A Christmas gift from a tester for a tester – Part One

A while back I received a gift from Matt Heusser, my early mentor in the Miagi-Do school. It asked me to test it. So, I decided to make a series of blog entries over the holidays out of it. Today we will start with the most obvious thing: unpacking.

As you can see in the picture, I received a cuboid package in wrapping paper. There is a card attached. There is also a small greeting label added. The card it turns out to be empty. No manual, no further information about my mission.

The little piece of paper on the wrapping paper revealed a basic inquiry for my mission.

Merry Christmas, man!

Here's a challenge: How would you test this?

That's my basic mission. I'm wondering whom to turn to when I got any additional questions. And I wonder what the target group for the gift will be. I denote these questions, as I might get back to my client with it.

I start to unpacke the wrapping paper. The bow goes first. It's glued to the paper. So I need some careful removal in order to keep the paper in shape. My wife usually likes to keep wrapping paper to reuse it. So I take special care about that.

When it comes to unwrapping the main package, I face a problem. I can't unwrap the paper without damaging it. I note down my thoughts about it. Though I was not explicitly told what to test here, I tested the amazon gift wrapping paper. It seems to be of good quality, not too thin. And actually quite well glued together.

I see a brown box out of cardboard inside. I pull off the wrapping paper completely to reveal the product.

It turns out to be a toy. "Electronic Scrabble Flash, Slide, Swap & Shuffle to find words fast. Ages 8+" the box claims. As I look on the box, I am amazed that it actually comes from the US, but still has the German "Grünen Punkt" on it, a symbol indicating that it can be recycled.

I turn the box around, to see if I can get some more out of it before opening the box. The only other claim I can find is the trademark on one side of the box. It states that Scrabble is a trademark of Hasbro in the US. I will keep that in mind while going further.

I will unpack the main package in my next blog entry in this series.

Test levels in Agile testing

In the past week Ulrich Freyer-Hirtz on the Agile Testing mailing list asked about different testing levels in an Agile project. He found the definition of unit, component, and acceptance tests appropriate based on several books. This discussion reminded me about the ambiguity that we face today in terms of testing. There have been several renaming attempts in the past few years. For example Dale Emery refers to a blog entry from 2004. Gojko Adzic made a similar renaming attempt. Let's take a step back, and see what all those names really want to tell us. Afterwards we will be able to make a more informed decision about names.

Test Levels

Why do we need to think in different test levels? I think we don't need to do that anymore. Back in the 1970′s when the distinction between programmers and testers had to do with different responsibilities most of the terms floating around today have been coined. Back then, it mattered whether you were testing on the unit level, on the component level, or on the system level. You had different missions for either of these levels.

In 2010 Mike Hill commented on my chapter in How to reduce the cost of software testing that he prefers the term microtest for unittests. The term unittest it seems has different meanings for different developers. Microtests make it transparent that we try to tests a little code as possible.

I like this term. It makes transparent that we aim to test as little as possible with these unittests. When I work with programmers, I often find the practice of unittesting producing tests focused on the functional requirements. The application with its database, mostly enterprise application servers, and any additional component is utilized. While this sort of tests is necessary as well, these are certainly no microtests. Microtests focus on one class of the production code – if possible. But what about that area of functional tests? And what about the remaining levels of testing, i.e. component, integration, system, sub-system, etc.? Let's take a look on other heuristics that might reveal insights about those. Enter the testing quadrants.

The Testing Quadrants

The testing quadrants as written down by Brian Marick provide a classification scheme for different tests as well as for the needs you might face. Rumor has it that most of the thoughts came actually from Cem Kaner, but Brian was the first to write them down, followed by Lisa Crispin and Janet Gregory in their Agile Testing book.

The testing quadrants describe a consultant's universal tool: four quadrants in a rectangular shape. The dimensions of the rectangle are technology-facing vs. business-facing on the x-axis, and team-support vs. product-critique on the y-axis. I think the term team-support does not ring immediately for certification-harmed testers. Yet, the dimensions describe quite well what helps a team, and makes a profound testing approach.

Microtests are found in the quadrant describing technology-facing tests that support the team. These clearly help drive forward the development. Over time they become change detectors, or checks as Michael Bolton would put it.

Functional tests usually lie in the quadrant with business-facing tests that support the team. These tests help the team, too. But they are written on a different level for a different purpose. They also help drive forward development, but are focused on a different level. Rather than making sure that you can exercise a network connection or connect to a database, these tests express business expectation and requirements. Some teams call them integration testing, acceptance tests, or system tests.

Whatever you call them, they address the issue that you don't necessarily know that you fulfilled the requirements for your customer. Acceptance tests may not reveal that information as well, but it informs you definitely that you fail to fulfill them in the future if one of them fails.

On the product critique side of the quadrants, performance, load, and the *ilities are found in the quadrant with technology-facing tests, while User Acceptance tests, usability testing, and the like are found on the business-facing quadrant.

With all these tests at my finger tips, what should I do in an Agile project? Enter risk analysis combined with the Testing Quadrants.

Risk analysis

In an Agile project, you face two constraints. On one hand, you want to serve the project as a tester. You want to provide feedback on time, to meet the two week iteration length, or whatever your time constraint there is. On the other hand you want to test the software thoroughly so that end users will not be harmed or confronted to a bug. So, you face time vs. test coverage – like in most traditional projects as well, but with a shorter timeframe.

So, you have to pick tests that you can fulfill now within the given timeframe. You want to make that as transparent as possible. So, depending on where your team currently is, you might be able to run tests for every quadrant, or you might be able to cover just the microtests and the functional tests.

But you have to make sure to include compensation for the tests that you currently cannot run in one iteration. This will take down your velocity, or you will have to plan from time to time for performance, load, or usability tests. However, over time you should be able to expand the tests that you can perform in one iteration, and fill these gaps. Over time you will gain a more balanced test set within your iteration.

Test Levels are an anti-pattern

Test levels are an anti-pattern. They are remains from a time long gone. On an Agile team everybody has the same responsibility to make the team successful. If you think about test levels, you have a more profound problem in your team. How come your team is not thinking as a team about issue? Why don't all team members contribute to the testing necessary? I claim thinking about test levels on an Agile project as an anti-pattern. When transitioning to Agile, you often start with explanations that testers are used to. So, that's where we start to classify different testing practices on the quadrants. But over time the team should make a mind-shift towards a Whole Team Approach to testing. If it's not, you should fix that problem, not the problem about levels or terms.

Alternative Paths to Certification

I received quite some feedback on my last blog entry on certification. One of the feedbacks made me wonder what an alternative to certification is. This question struck me hard enough to write a follow-up on that. I think this question can be answered solely in a certain context. I'll try to answer it under several contexts, one by one.

What's the problem?

In order to find out what an alternative option for a certification program could be, we have to find out more about the reasons behind the certification. Certification seems to be a solution to some problems. But what are these problems? What's the problem that certification is going to solve? And whose problem is it anyways?

In an open space session at the Agile Testing Days 2010 Elisabeth Hendrickson collected some of these alternative. Afterwards she created entaggle out of it. The problem that entaggle solves is one of peer recognition. If you want to find out more about a person, with whom she or he has worked, then you can see their profile there, and get some feedback about that person. The feedback is individual recognition of peers. Since you can create your own tags on entaggle, you can provide your own certification – if you want to claim it as such.

Among the other reasons for certifications, there were reasons such as, as a person working in the personnel department, I want to digest a pile of applications in a short time. A filter like a mandated certificate is one way to achieve this. Other reasons include to sell training course, and educate people.

One thing I learned from Jerry Weinberg in the Quality Software Management series was that things have exactly one reason, and we can tell which one is which. Regarding certification there are several reasons why certification programs are called to live. Having said that, let's take a look at some of the certifications out there. I'll stick to some popular ones. For the remaining ones, check out this comic from xkcd. It should yield an explanation.

Programming Certificates

For some programming languages like Java, C++, Python, Ruby, there are certifications that make you aware of the language. The language and how it's intended to be used is a distinct set of construct. While you might need a programmer sufficiently aware of some programming skills, you might look for a filter of programming certificates.

Programming certification solves a problem of skills. An alternative to this could be a programming exercise as entry criteria to your hiring process. Let people who apply submit a program they wrote. Or have them implement a solution to some simple kata like tic-tac-toe or minesweeper. This solution has the advantage of showing you how they code, so you can see how much work you will have in order to teach them about SOLID, design patterns, and responsive design. Oh, by the way, did they submit their unit- and acceptance test together with the code?

Process certification

There are many certification programs out there on some development process or methodology. Rather than needing to write down your own process in your own documents, you can rely on a profound basis of methods.

For the development process you may want to have a common understanding about the approach that your organization takes. From my experience this comes with the believe that every project will run the same way once you have trained everyone the process. My first reaction to such nonsense is: How boring to work in such a factory!

Instead of relying on a common way to develop software, you should transport the thoughts to a higher level. Indeed many Agile programs do this. By making the teams responsible for their own processes, they can adapt and self-organize to their situation. This approach is way more efficient than a prescribed process which might put you into the illusion of control. You hired highly educated people. They can work without your command & control, yet they still need guidance from you from time to time. If you start with a regular retrospective – like every two weeks – and have the team responsible for their process, the remaining technical skills will show up, anyways – maybe with some hints from a consultant, but they will show up.

Testing

This yields the discussion to my favorite field: testing. In the 1980s and 1990s there was a huge tendency to educate former workers as testers. In such times where people unaware of programming at all, there was a huge need for testers who knew some of the basics. These people knew the domain of their work, but didn't know which tests to run. That's when testing certification was born. The goal was to educate people on some basics. Sometimes even to educate students to exercise some application.

This field has turned in the past decades. Now we face testers professional testers who need to find the bugs that unit test automation and acceptance test automation have not covered before the software gets on their computer.

A recent blog entry by Andreas Simon reminded me of models. While programming we build up mental models, and put them into code. The task of a tester is to come up with other mental models to break these models – or maybe to confirm them. This is why testing uses so many diverse fields as basis, as epistemology, cognitive psychology, decision theory, philosophy, and many others. We need to think diverse.

Certification provides one set of models to break software. Boundary values (no analysis), decision tables, black-box, white-box, etc. build a foundation. Many such courses are called foundations. But you need more as a profound tester in the field nowadays. That is how to come up with your own models about test coverage, test planning, and test heuristics. Sure, you need basics, just as you need the basics in programming. And above that you also need critical thinking, systems thinking, complexity thinking, etc. in order to come up with heuristics and approaches on your own.

An alternative to this kind of certification would be to expose your testing skills to others. As you can submit your code along with your CV, you can also submit videos where you expose your testing skills. Take some application, and show how you would test it. Take a bug that you found recently, and submit it alongside. Or you might want to submit your bug reports as well.

Let's talk certification!

These thoughts made me curious about the topic. As I am looking for the topic of the next GATE workshops, I now picked a topic: certification. I want to learn more about certification in testing, why we need it, where we need, who needs it, what your experiences with courses have been, where you see alternatives, and what helps you about being (or not being) certified in any kind. I especially want to hear both sides, the pros and the cons, and we will provide a moderated discussion on these topics.

I aim for a date in March, somewhere in Germany. I hope to get to some more details after Christmas, so stay tuned. But I surely want to talk certification. Let's talk!

Certification

On the Agile testing mailing list there is currently a discussion on-going about the value of certifications and certificates. I have a strong opinion on it, and I would like to provide them on my blog. The basis has been the upcoming Certifiaction program for Agile testers. I have provided my critics to their courses as far as I could. I admire the efforts people put in such courses. That said, I don't intend to offend anyone involved in certification programs, and will try to raise my objections as constructive as possible. But I also know that I will fail from time to time.

An anecdote

In September 2010 I took the Scrum Master certification from my colleague Stefan Roock. It was a two day course, I learned some new stuff, but most of the content I was already aware of. Afterwards I took the online questionnaire only to pass with zero errors in the questionnaire. Flawless.

My first gig as a Scrum Master was a failure in many circumstances. After the five months of being engaged at the client, the contract was ended. I held a retrospective with my colleagues, I discussed this topic as part of my peer group discussion, and learned some lessons from it. Heck, I also reflected on it together in a personal coaching session at PSL in May with Esther Derby and Jerry Weinberg. The Hardest Law had fully hit me:

Helping myself is even harder than helping others.

(The Secrets of Consulting, page 18)

Remarkably there was no question in the CSM questionnaire on what to do if you fail. No one taught me this lessons directly during the two day course. Yet I knew it was the right thing to learn from it, taking into account several opinions from colleagues. While you could say that "Inspect and Adapt" taught me this lesson, I still doubt the questionnaire that made me pass without any errors certified anything or could keep me from failing.

Shallow agreements

This is a common theme for certification programs I am aware of. I think the biggest issue with certification is that they lull people into a shallow agreement. "He's got that CSM certificate, I got it too, so we agree on stuff." "He's got an ISTQB cert, so he knows all the terms about testing." This sort of agreement is shallow. If you don't believe me, and yet don't believe my story from above, go to any two testers and ask for their definition of "integration testing" and check the results. Personally, I think this most ambiguous term in software testing should be banned completely. Heck, there should be a fee of 200,000 USD each time someone uses it.

Shallow agreements make our life sad and disappointed. For example I recently asked my father-in-law to get ourselves some wood for our oven. He got us wood for the oven, but I still have to cut it down to fit into our oven. Now I got several cubic metres of wood in my car port which I have to cut down, resulting in a lot of work to be done in order to get our home warm during the winter months. All of this was based on a shallow agreement. I had wood ready to fit in our oven in mind, while my father-in-law had wood as cheap as possible in mind. Probably a mismatch in underlying values maybe.

The same holds true for the contents of certification programs. While the contents are part of every training, you're not guaranteed at all that all trainers agree to it. Yet, after leaving the course you might think that people have the same understanding as you have from the topic. Giving people a common understanding is a shallow agreement. You won't know whether you reached the same understanding until actually challenging their understanding. There are some folks out there who did challenge some of the certification programs, and their represents, but without any success so far. It seems that shallow understanding is a pattern found in this field.

Needs

This leads the discussion for me to the actual necessity for certification programs. I am truly convinced that SJ-types (see Myers-Briggs Type Indicator for a grasp on the model) are the main driver for certification programs. By their preference these folks seek safety and reliance. They bring stability to the organization. Remarkably they fail to bring to necessary changes to the work environment if these undergo the challenge for change in the market.

The need for certification arises from people seeking safety and reliance. They seek to hire good testers, yet they base their safety and reliance upon a shallow agreement as a certification. Even worse, in case certification programs indeed provide a common ground for understanding, hiring people only by their certifications results in a confirmation bias strategy for hiring people. You could actually hire certified monkeys instead.

Yet, how should you oppose such needs as a claim for safety and reliance? The first step to overcome the certified monkey hiring fallacy is to realize that it's uncertain what will happen once you hire anyone. This is a key lesson I learned from complexity thinking. If you're in the position to hire someone, you should be able to take the responsibility to deal with the consequences. As I discussed earlier it's uncertain whether the terms of the certification program reached a similar understanding in your colleagues as in the newly hired person. The human system dynamics which set in once the new hire reaches the team also provide an uncertain future. No piece of paper can change that. If you hired that person, you should provide the ground for their growth, and seek to exchange with your colleagues how to self-organize your team.

Containers

Finally, certification programs provide an artificial container for people. Certification not only splits up people into certified vs. not certified, but also provides barriers for communication and exchange. IF your department consists solely of people being certified, you won't know what you miss from the knowledge of the uncertified people. In terms of complexity thinking these different groups of people are called containers. They shape the environment for people. In a self-organized system you need trandformational exchanges to overcome the difference between these two containers.

This brings me back to the discussion of exchanges of thoughts between certification people and non-certification people. Based on my personal history it seems that certification people get offended quickly by arguments against their programs. From a human standpoint this seems natural. Certification people seek to serve their need to make money form the program. That's why they put a whole lot of effort into creating the program. Yet, this makes them blind to criticism from other areas. Human nature tends to neglect some critics when it comes to serving necessary needs in their value system.

At face value, this makes transformational exchanges more challenging. So, we got a catch-22. In order to get along with good certification programs, we need exchange between non-certification people and certification people. One of the differences between these folks is a different value system as well as a difference in the needs they're trying to serve with their decisions. So, an exchange won't happen whereby certification programs will become worse as the craft continues to advance.

Conclusion

There are a bunch of reasons for and against certifications. I provided some of my thoughts, yet I still know that these are my personal opinions. Yet I also know that there are exceptions, and I have met many people involved in providing certification courses who actually do a fantastic job. My personal preferences call for independence to the degree that I neglect any authorities. As James Lyndsay remarked in the tagging on entaggle healthily so. My only hope is to get certification and non-certification people together for an exchange – but I doubt this will happen based upon the reasons described above.