Jay Fields's Blog, page 10

Recent Thoughts On Hiring and Being Hired

The job market is insane right now. It got to the point that I was receiving job-related email so often that I changed my LinkedIn profile to say I lived in Jacksonville, Florida (I don't - I'm still happily in NYC). However, I do read every job related email that makes it through Google's spam filter, and a few things did catch my eye recently.

Recruiters doing it right:
One recruiter emailed me and didn't ask for a resume, but did ask to see my github account. Setting aside the fact that plenty of code lives outside of github, this request impressed me. If they actually have people doing a bit of research on applicants via their open-source contributions, I imagine they're much better at hiring than their competition.

A different recruiter asked if I was available or if I knew anyone I'd be interested in referring, but the conversation was unique because the recruiter offered a $5,000 referral fee if a friend of mine was hired. Occasionally I will pass a recruiter's contact information along if the job sounds interesting, but I never spend more than 2 minutes thinking about who I know and if they are a match. Obviously, when 5K is the reward, I spent significantly more time considering who I knew with appropriate skills and desires.

Recruiters doing it wrong:
Don't bother telling me that you're offering a $500 referral fee. It's not that $500 is insignificant, but referring a friend is such an unlikely event that the payoff needs to be much higher due to how infrequently things come together. Also, if your competition is paying 5K and you're paying $500, that can give a false impression on what you'll provide as salary. I'm guessing some cool companies only offer $500, and that's cool - but, I think you'll be better off focusing on where you are superior to your competition and omitting where you're definitely behind.

Programmers doing it wrong:
If you're out of work right now there should be a significant reason why. Perhaps you are tied to a small town or a specific domain - those are valid reasons. However, if you find yourself without the skills you need to get the jobs that are available in your area, you've likely been neglecting your craft. This summer, a family member asked me if I could tell his son my secret to success. The best advice I could come up with was: Hit the job boards and see what the largest technology need is and get to work learning it.

Once upon a time the technology I knew the best was Pegasystems. I was laid off and I ended up spending plenty of time on monster.com and the various IT specific clones. There were jobs available, but everyone wanted someone with a Microsoft background. I ended up turning down some well paying jobs working with dead-end technologies and took a decent paying job that allowed me to use .Net. Twelve months later I accepted a job making 125% of the decent salary - which was the most I'd ever made. If you are passionate and have high demand skills you should always be paid well.

Another thing that surprises me is programmers who are "too busy to read about new technology or attend conferences". Perhaps I value innovation more than other programmers, but I simply can't understand this perspective. I see my job as more than whatever feature I'm currently working on. To me, my job is to provide the most features in the shortest amount of time possible. On occasion that means needing to get something out as soon as possible. However, my general working mode is annual production, and in that timeframe inefficiency adds up. On occasion innovation can offer productivity boosts that I couldn't match if I worked 24 hours a day using more dated solutions.

As an employee, I feel it's my responsibility to the company to ensure I'm not ignoring any innovation that could drastically impact my delivery speed. As such, I need to be on top of as many relevant innovations as possible. Interestingly, a side-effect of this attitude is that I should also have the necessary skills to find another job should I find myself looking around. This is an everybody wins situation, but only if you choose to do the right thing.

That's all I have off the top of my head. If you have any interesting experiences with hiring or being hired, please leave me a comment.
© Jay Fields - www.jayfields.com

Life After Pair Programming

When I first joined DRW I noticed that the vast majority of developers were not pair-programming. I was surprised that a company that employed so many smart people would just simply disregard a practice that I considered to be so obviously beneficial.

note: it's so easy to judge people who don't pair-program, isn't it? You can write-off their behavior for so many reasons:They simply haven't done it enough to see how much value it provides.

They've used the wrong setup, and turned a positive ROI into a negative ROI situation in the past.

They must be anti-social or not team players.

They must guard their code because they are too proud or believe it's job security.Never once did I consider that their behavior might be correct based on their context. There was no context in which pair-programming wasn't the right choice, right?

So, I did what any responsible employee who wants to help would do: I gave a presentation. I took pictures of a traditional (read: broken) pairing station setup - the setup where one employee looked over the shoulder of the person doing all the work. Then I took a picture of a dual monitor, dual keyboard, and dual mouse setup. I made the distinction between pairing and watching someone work. I explained how ping-pong-pair-programming worked. I think I said the right things, and the presentation was well received. Management was on board and offered to setup pairing stations for any developers in the company that wanted to give it a try.



A few people embraced it, but most went back to their traditional habits. I did manage to convert the guys on my team, and we paired the vast majority of the time.



Almost 3 years later, and on a different team, I basically never pair. In fact, when my lone teammate (and boss) asks if we should pair on something my initial reaction is always 'no'. The 2 of us are the only people working on the applications we support, and when we pair I often find myself very distracted and not really participating. Sometimes we ping-pong-pair and it keeps us both involved, but I don't feel like we're going faster than if I'd been working solo. Don't get me wrong, we sit directly next to each other, talk constantly and even look at code together at times when we're just starting to talk about a new problem. That level of collaboration feels beneficial, but actually coding the implementation together feels wasteful.



I know how I felt 3 years ago (when I joined DRW), and I know how I feel now. The fact that I've changed my tune so drastically made me think it was worth putting a few ideas on the web. I was also motivated a bit by the recent publication of Pair Programming Benefits. The Pair Programming Benefits article actually gives me a perfect starting place, by examining the benefits of pairing and how they apply to my context.



I feel like I can safely summarize the Team and System Benefits as: transfer of knowledge, superior quality, and collaboration. The Programmer Benefits can be summarized as: expansion of knowledge, truck factor reduction, superior quality, and team support. The Management/Project Management Benefits can be summarized as: aiding addition and removal of employees, knowledge transfer, and the ability to hire any skilled generalist.



So, our full list of benefits is: expansion and transfer of knowledge, superior quality, collaboration, truck factor reduction, team support, eased hiring and firing. I would agree with these benefits of pair-programming, which explains why I was such an avid believer in pairing in the past. However, if those pros don't apply to my situation then pair-programming becomes nothing more than using twice as many man hours to get the same amount of work done.



I'll start by addressing the easiest "benefits" that are effectively irrelevant for my team: eased hiring and firing and truck factor. Like I said before, my team size is 2. Mike and I are both very happy and there's almost no chance that either of us leave anytime soon. Even if we did choose to leave, we would give enough notice that someone could easily come in and replace us - and either one of us could support the applications on our own if necessary. The only real risk is that we both are killed at the same time - at which point it wouldn't matter if we paired or not. Statistically it's much more likely that only one of us would suffer something causing us to not be able to return to work, and as Chris Zuehlke (manager at DRW) once pointed out to me: if you build small, well written processes the time to understand or rewrite any software you know nothing about is small - often smaller than the cost of keeping 2 people informed on the details of the process. If I asked the stakeholders if we should use 2x as many man hours to ensure that we're covered in case one of us is suddenly killed they'd laugh at me - and I'd deserve it.



We're also not looking to add to our team, and if we decided we did want to grow I expect we wouldn't have any trouble finding someone who was looking for a well-paying job in manhattan using very cool technology.



The previous discussion was meant to address team size, but it also touches on "transfer of knowledge". Transfer of knowledge is valuable for teaching new techniques and enabling people to do maintenance. However, if little teaching is going on (often the case as Mike and I add features to an app we know fairly well, using technologies we know fairly well) and we both tend to do the maintenance on the code we previously wrote then "transfer of knowledge" is either not occurring anyway or is unnecessary. Also, on more than one occasion I've had to take a first look at some code Mike had recently written. It's always been straightforward and taken no more than an hour to dig-in, make, and test a change. Had we paired on everything he worked on I would have unquestionably spent far more than an hour here or there. Again, the ROI on knowledge transfer doesn't look good for our situation.



The "superior quality" benefit is an interesting one. I consider myself to be a good engineer who on occasion writes something beautiful. I like to share what's beautiful with everyone. This is something that's going to happen whether or not I pair (evidenced by my blogging). I know Mike does the same. I expect we're all proud of our most elegant solutions. However, more often than not I write fairly good software that isn't excitement provoking, but solves the problem at hand. I'm not sure anyone is benefiting from working with me at those moments. And, while it's possible that we'd get a slightly better solution if Mike & I paired, spending twice as many man hours for a 10% better solution doesn't seem like a great choice in the short term. In the long term those 10%s could add up, but what if I see the superior solution on my 2nd pass through and end up there while expanding a feature anyway? If I have a massive solution that might be a big cost. However, with small, focused processes I can tend to make a massive change to a process in ~4 hours. Delete is still my favorite refactoring and I do it as often as possible on my quests for superior solutions. Therefore, I don't believe those 10%s add up, I believe the 90% superior solution is often good enough or deleted.



While I believe "expansion of knowledge" is often undervalued and under-practiced in our profession, I don't think pair-programming is the most effective way to expand your skills. Blogs, books, and presentations allow you to work at your own pace and dig in to your desired level. My experience pairing with people using tools or languages I don't know have been painful. I did learn things, but I couldn't help but feel like there were much more effective ways for me to get up to speed.



The final 2 benefits that I need to consider are "collaboration" and "team support". As I previously stated, Mike and I collaborate quite closely. We're always both happy to stop what we're doing to provide an opinion or direction. I honestly can't imagine a more collaborative environment. Basically the same thing can be said of "team support" - we constantly watch each-other's back. We're both looking to succeed and we know we need each-other to get to that goal, so it only makes sense that we look out for each other. We don't need pairing to force us to support each-other, the desire to succeed is more than enough.



When considering my context and the "benefits" of (or, in my situation, lack of benefits of) pair-programming, I'm no longer surprised that I basically never want to pair these days. Also, looking back at the other teams I was previously judging - their context back then looks a lot like my current context. With a bit more experience under my belt I have to say I'm honestly a bit embarrassed for never even considering that they already "got it" and were already making the right choice. Oh well, live and learn.



While it's easy for me to examine my situation and make a determination I think there is a more general context where these same ideas apply. The aspects of an environment where pairing may not be beneficial seem to look something like this:small team (< 4 people)small software (components/processes can be rewritten in < 2 days)motivated teammatestalented and well-seasoned teammates (similar skill level, and skilled overall)the design of the application is fairly mature and stable
© Jay Fields - www.jayfields.com

Clojure: Check For nil In a List

The every? function in Clojure is very helpful for determining if every element of a list passes a predicate. From the docs:

Usage: (every? pred coll)



Returns true if (pred x) is logical true for every x in coll, else false.

The usage of every? is very straightforward, but a quick REPL session is always nice to verify our assumptions.Clojure 1.2.0

user=> (every? nil? [nil nil nil])

true

user=> (every? nil? [nil 1])

falseAs expected, every? works well when you know exactly what predicate you need to use.



Yesterday, I was working on some code that included checking for nil - similar to the example below.user=> (def front 1)

#'user/front

user=> (def back 2)

#'user/back

user=> (when (and front back) [front back])

[1 2]This code works perfectly if you have the individual elements "front" and "back", but as the code evolved I ended up representing "front" and "back" simply as a list of elements. Changing to a list required a way to verify that each entry in the list was not nil.



I was 99% sure that "and" was a macro; therefore, combining it with apply wasn't an option. A quick REPL reference verified my suspicion.user=> (def legs [front back])

#'user/legs

user=> (when (apply and legs) legs)

java.lang.Exception: Can't take value of a macro: #'clojure.core/and (NO_SOURCE_FILE:8)Several other options came to mind, an anonymous function that checked for (not (nil? %)), map the values to (not (nil? %)) and use every? with true?; however, because of Clojure's truthiness the identity function is really all you need. The following REPL session shows how identity works perfectly as our predicate for this example.(when (every? identity legs) legs)

[1 2]For a few more looks at behavior, here's a few examples that include nil and false.user=> (every? identity [1 2 3 4])

true

user=> (every? identity [1 2 nil 4])

false

user=> (every? identity [1 false 4])

falseAs you can see, using identity will cause every? to fail if any element is falsey (nil or false). In my case the elements are integers or nil, so this works perfectly; however, it's worth noting so you don't see unexpected results if booleans ever end up in your list.


© Jay Fields - www.jayfields.com

Clojure: partition-by, split-with, group-by, and juxt

Today I ran into a common situation: I needed to split a list into 2 sublists - elements that passed a predicate and elements that failed a predicate. I'm sure I've run into this problem several times, but it's been awhile and I'd forgotten what options were available to me. A quick look at http://clojure.github.com/clojure/ reveals several potential functions: partition-by, split-with, and group-by.



partition-by

From the docs:

Usage: (partition-by f coll)



Applies f to each value in coll, splitting it each time f returns

a new value. Returns a lazy seq of partitions.

Let's assume we have a collection of ints and we want to split them into a list of evens and a list of odds. The following REPL session shows the result of calling partition-by with our list of ints.user=> (partition-by even? [1 2 4 3 5 6])

((1) (2 4) (3 5) (6))The partition-by function works as described; unfortunately, it's not exactly what I'm looking for. I need a function that returns ((1 3 5) (2 4 6)).



split-with

From the docs:

Usage: (split-with pred coll)



Returns a vector of [(take-while pred coll) (drop-while pred coll)]

The split-with function sounds promising, but a quick REPL session shows it's not what we're looking for.user=> (split-with even? [1 2 4 3 5 6])

[() (1 2 4 3 5 6)]As the docs state, the collection is split on the first item that fails the predicate - (even? 1).



group-by

From the docs:

Usage: (group-by f coll)



Returns a map of the elements of coll keyed by the result of f on each element. The value at each key will be a vector of the corresponding elements, in the order they appeared in coll.

The group-by function works, but it gives us a bit more than we're looking for.user=> (group-by even? [1 2 4 3 5 6])

{false [1 3 5], true [2 4 6]}The result as a map isn't exactly what we desire, but using a bit of destructuring allows us to grab the values we're looking for.user=> (let [{evens true odds false} (group-by even? [1 2 4 3 5 6])]

[evens odds])

[[2 4 6] [1 3 5]]The group-by results mixed with destructuring do the trick, but there's another option.



juxt

From the docs:

Usage: (juxt f)

              (juxt f g)

              (juxt f g h)

              (juxt f g h & fs)



Alpha - name subject to change.

Takes a set of functions and returns a fn that is the juxtaposition

of those fns. The returned fn takes a variable number of args, and

returns a vector containing the result of applying each fn to the

args (left-to-right).

((juxt a b c) x) => [(a x) (b x) (c x)]

The first time I ran into juxt I found it a bit intimidating. I couldn't tell you why, but if you feel the same way - don't feel bad. It turns out, juxt is exactly what we're looking for. The following REPL session shows how to combine juxt with filter and remove to produce the desired results.user=> ((juxt filter remove) even? [1 2 4 3 5 6])

[(2 4 6) (1 3 5)]There's one catch to using juxt in this way, the entire list is processed with filter and remove. In general this is acceptable; however, it's something worth considering when writing performance sensitive code.
© Jay Fields - www.jayfields.com

Clojure: Apply a Function To Each Value of a Map

I recently needed to update all of the values of map, and couldn't find an individual function in clojure.core that did the trick. Luckily, implementing an update-values function is actually very straightforward. The following function reduces the original map to a new map with the same elements, except the values are all the result of applying the specified function and any additional args.(defn update-values [m f & args]

(reduce (fn [r [k v]] (assoc r k (apply f v args))) {} m))The code is concise, but perhaps a bit terse. Still, it does the trick, as the REPL session below demonstrates.Clojure 1.2.0

user=> (defn update-values [m f & args]

(reduce (fn [r [k v]] (assoc r k (apply f v args))) {} m))

#'user/update-values

user=> (update-values {:a 1 :b 2 :c 3} inc)

{:c 4, :b 3, :a 2}

user=> (update-values {:a 1 :b 2 :c 3} + 10)

{:c 13, :b 12, :a 11}

user=> (update-values {:a {:z 1} :b {:z 1} :c {:z 1}} dissoc :z)

{:c {}, :b {}, :a {}}The last example is the specific problem I was looking to solve: remove a key-value pair from all the values of a map. However, the map I was working with actually had a bit of nesting. Let's define an example map that is similar to what I was actually working with.user=> (def data {:boxes {"jay" {:linux 2 :win 1} "mike" {:linux 2 :win 2}}})

#'user/dataEasy enough, I have 2 linux boxes and 1 windows box, and Mike has 2 linux and 2 windows. But, now the company decides to discard all of our windows boxes, and we'll need to update each user. A quick combo of update-in with update-values does the trick.user=> (update-in data [:boxes] update-values dissoc :win)

{:boxes {"mike" {:linux 2}, "jay" {:linux 2}}}As you can see, the update-values function plays nice with existing Clojure functions as well.



Disclosure: I am long AAPL and own every device Apple puts out. Don't take my example numbers to literally. Also, much to my dismay, DRW has yet to discard all of our windows boxes.
© Jay Fields - www.jayfields.com

Clojure: memfn

The other day I stumbled upon Clojure's -> (\r \h \y)-- clojure.orgAt first glance it appeared to be something nice, but even the documentation states that "...it is almost always preferable to do this directly now..." - with an anonymous function.

(map #(.charAt %1 %2) ["fred" "ethel" "lucy"] [1 2 3])
-> (\r \h \y)-- clojure.org, again

I pondered memfn. If it's almost always preferable to use an anonymous function, when is it preferable to use memfn? Nothing came to mind, so I moved on and never really gave memfn another thought.
Then the day came where I needed to test some Clojure code that called some very ugly and complex Java.
In production we have an object that is created in Java and passed directly to Clojure. Interacting with this object is easy (in production); however, creating an instance of that class (while testing) is an entirely different task. My interaction with the instance is minimal, only one method call, but it's an important method call. It needs to work perfectly today and every day forward.
I tried to construct the object myself. I wanted to test my interaction with this object from Clojure, but creating an instance turned out to be quite a significant task. After failing to easily create an instance after 15 minutes I decided to see if memfn could provide a solution. I'd never actually used memfn, but the documentation seemed promising.
In order to verify the behavior I was looking for, all I'll I needed was a function that I could rebind to return an expected value. The memfn macro provided exactly what I needed.
As a (contrived) example, let's assume you want to create a new order with a sequence id generated by incrementAndGet on AtomicLong. In production you'll use an actual AtomicLong and you might see something like the example below.(def sequence-generator (AtomicLong.))
(defn new-order []
(hash-map :id (.incrementAndGet sequence-generator)))

(println (new-order)) ; => {:id 1}
(println (new-order)) ; => {:id 2}While that might be exactly what you need in production, it's generally preferable to use something more explicit while testing. I haven't found an easy way to rebind a Java method (.incrementAndGet in our example); however, if I use memfn I can create a first-class function that is easily rebound.(def sequence-generator (AtomicLong.))
(def inc&get (memfn incrementAndGet))
(defn new-order []
(hash-map :id (inc&get sequence-generator)))

(println (new-order)) ; => {:id 1}
(println (new-order)) ; => {:id 2}At this point we can see that memfn is calling our AtomicLong and our results haven't been altered in anyway. The final example shows a version that uses binding to ensure that inc&get always returns 10.(def sequence-generator (AtomicLong.))
(def inc&get (memfn incrementAndGet))
(defn new-order []
(hash-map :id (inc&get sequence-generator)))

(println (new-order)) ; => 1
(println (new-order)) ; => 2
(binding [inc&get (fn [_] 10)]
(println (new-order)) ; => 10
(println (new-order))) ; => 10With inc&get being constant, we can now easily test our new-order function.
© Jay Fields - www.jayfields.com

The High-Level Test Whisperer

Most teams have High-Level Tests in what they call Functional Tests, Integration Tests, End-to-End Tests, Smoke Tests, User Tests, or something similar. These tests are designed to exercise as much of the application as possible.
I'm a fan of high-level tests; however, back in 2009 I decided on what I considered to be a sweet-spot for high-level testing: a dozen or less. The thing about high-level tests is that they are complicated and highly fragile. It's not uncommon for an unrelated change to break an entire suite of high-level tests. Truthfully, anything related to high-level testing always comes with an implicit "here be dragons".
I've been responsible for my fair share of authoring high-level tests. Despite my best efforts, I've never found a way to write high-level tests that aren't filled with subtle and complicated tweaks. That level of complication only equals heartbreak for teammates that are less familiar with the high-level tests. The issues often arise from concurrency, stubbing external resources, configuration properties, internal state exposure and manipulation, 3rd party components, and everything else that is required to test your application under production-like circumstances.
To make things worse, these tests are your last line of defense. Most issues that these tests would catch are caught by a lower-level test that is better designed to pin-point where the issue originates. The last straw - the vast majority of the times that the tests are broken it is due to the test infrastructure (not an actual flaw in the application) and it takes a significant amount of time to figure out how to fix the infrastructure.
I've thrown away my fair share of high-level tests. Entire suites. Unmaintainable and constantly broken tests due to false negatives simply don't carry their weight. On the other hand I've found plenty of success using high-level tests I've written. For awhile I thought my success with high-level tests came from a combination of my dedication to making them as easy as possible to work with and that I never allowed more than a dozen of them.
I joined a new team back in February. My new team has a bunch of high-level tests - about 50 of them. Consider me concerned. Not long after I started adding new features did I need to dig into the high-level test infrastructure. It's very complicated. Consider me skeptical. Over the next few months I kept reevaluating whether or not I thought they were worth the amount of effort I was putting into them. I polled a few teammates to get their happiness level. After 5 months of working with them I began my attack.
Each time my functional tests broke I spent no more than 5 minutes on my own looking for an obvious issue. If I couldn't find the problem I interrupted Mike, the guy who wrote the majority of the tests and the infrastructure. More often than not Mike was able to tweak the tests quickly and we both moved on. I anticipated that Mike would be able to fix all high-level test related issues relatively quickly; however, I expected he would grow tired of the effort and we would seek a smaller and more manageable high-level test suite.
A few more weeks passed with Mike happily fielding all my high-level test issues. This result started to feel familiar, I had played the exact same role on previous projects. I realized the reason that I had been successful with high-level tests that I had written was likely solely due to the fact that I had written them. The complexity of high-level test infrastructure almost ensures that an individual will be the expert and making changes to that infrastructure are as simple as moving a few variables in their world. On my new team, Mike was that expert. I began calling Mike the High-Level Test Whisperer.
At previous points in my career I might have been revolted by the idea of having an area of the code that required an expert. However, having played that role several times I'm pretty comfortable with the associated risks. Instead of fighting the current state of affairs I decided to embrace the situation. Not only do I grab Mike when any issues arise with the high-level tests, I also ask him to write me failing tests when new ones are appropriate for features I'm working on. It takes him a few minutes to whip up a few scenarios and check them in (commented out). Then I have failing tests that I can work with while implementing a few new features. We get the benefits of high-level tests without the pain.
Obviously this set-up only works if both parties are okay with the division of responsibility. Luckily, Mike and I are both happy with our roles given the existing high-level test suite. In addition, we've added 2 more processes (a 50% increase) since I joined. For both of those processes I've created the high-level tests and the associated infrastructure - and I also deal with any associated maintenance tasks.
This is a specific case of a more general pattern I've been observing recently: if the cost of keeping 2 people educated on a piece of technology is higher than the benefit, don't do it - bus risk be damned.
© Jay Fields - www.jayfields.com

Individuals Over People

I've been pondering a few different ideas lately that all center around a common theme: to be maximally effective you need to identify and allow people to focus on their strengths.
I hear you: thanks Captain Obvious.
If you are reading this it's likely that you're familiar with the phrase "Individuals and interactions over processes and tools" from the Agile Manifesto. I'm sure we agree in principle, but I'm not sure we're talking about the same thing. In fact, it's more common to hear "people over process" when discussing Agile, which I believe is more appropriate for describing the value that Agile brings.
Agile emphasizes people (as a group) over processes and tools. However, there's little room for "individuals" on the Agile teams I've been a part of. I can provide several anecdotes - When pair-programming with someone who prefers a Dvorak layout a compromise must be made.When pair-programming with someone who prefers a different IDE a compromise must be made.Collective code ownership implies anyone can work on anything, which often leads to inefficient story selection. (e.g. the business accidentally gives a card to someone who isn't ideally skilled for the task. Or, a developer decides to work on a card they aren't ideally skilled for despite other better suited and equally important outstanding cards)Collective code ownership requires a lowest common denominator technology selection. (e.g. If 3 out of 5 people know Ruby and 5 out of 5 know Clojure, selecting Ruby for any application, even when it's the appropriate choice, is likely to be met by resistance)Collective code ownership requires a lowest common denominator coding style selection. Let's be honest, it's easy to code in a language such as Ruby without a deep understanding of metaprogramming and evaluation. Both metaprogramming and evaluation are powerful; however, you can only take advantage of that power if you are sure everyone on the team is comfortable with both techniques.I could ramble on a bit more, but hopefully you get my point.
I'm still a believer in Agile. It's the best way I know how to take an average performing team and put them on a path to becoming a well performing team. However, I think the Agile practices also put a ceiling on how effective a team can be. Perhaps my favorite anecdote: Ola Bini believes he is 10x faster when using Emacs as compared to IntelliJ - when writing Java! 10x is huge, so what does he use when he's pairing: IntelliJ. If there's knowledge transfer occurring then perhaps the 10x reduction in delivery speed is a good decision; however, if he's pairing with someone of a similar skill level who isn't statistically likely to maintain the code in the future it's a terrible choice. Ola is programming at 1/10 of his possible efficiency for no reason other than it's the Agile way.
Clearly, it's gray area - the knowledge transfer level will vary drastically based on who he's pairing with and what they are pairing on. That's the important point - people are more important than processes, but individuals are the most important. If you want to achieve maximum productivity you'll need to constantly reevaluate what the most effective path is based on the individuals that make up your team.
If you already agree with the ideas above then you're probably familiar with the idea that you need to learn all the rules to know when to break them. That's an old idea as well. Unfortunately, I don't see much written on this topic in the software development world. The last 3 years of my life have been lesson after lesson of how smart people can break the Agile rules and get much larger gains as a result. I've decided to tag posts that cover this subject as "Individuals over People". There are even a few historical entires available at http://blog.jayfields.com/search/label/individuals%20over%20people.
Hopefully these ideas will spark a few discussions and inspire other post-Agile developers to post their experiences as well.
© Jay Fields - www.jayfields.com

Undervalued Start and Restart Related Questions

How long does it take to start or restart your application?

Start-up time tends to be a concern that's often overlooked by programmers who write unit tests. It will (likely) always be faster to run a few unit tests than start an application; however, having unit tests shouldn't take the place of actually firing up the application and spot checking with a bit of clicking around. Both efforts are good; however, I believe the combination of both efforts is a case where the sum is greater than the parts.

My current team made start-up time a priority. Currently we are able to launch our entire stack (currently 6 processes) and start using the software within 10 seconds. Ten seconds is fast, but I have been annoyed with it at times. I'll probably try to cut it down to 5 seconds at some point in the near future, depending on the level of effort needed to achieve a sub-5-second start-up.

That effort is really the largest blocker for most teams. The problem is, often it's not clear what's causing start up to take so long. Performance tuning start-up isn't exactly sexy work. However, if you start your app often, the investment can quickly pay dividends. For my team, we found the largest wins by caching remote data on our local boxes and deferring creating complex models while running on development machines. Those two simple tweaks turn a 1.5 minute start-up time into 10 seconds.

If your long start-up isn't bothering you because you don't do it very often, I'll have to re-emphasize that you are probably missing out on some valuable feedback.

Not time related, but start related: Does your application encounter data-loss if it's restarted?

In the past I've worked on teams where frequent daily roll-outs were common. There are two types of these teams I've encountered. Some teams do several same day roll-outs to get new features into production as fast as possible. Other teams end up doing multiple intraday rollouts to fix newly found bugs in production. Regardless of the driving force, I've found that those teams can stop and start their servers quickly and without any information loss.

My current team has software stable enough that we almost never roll out intraday due to a bug. We also have uptime demands that mean new features are almost never more valuable than not stopping the software intraday. I can only remember doing 2 intraday restarts across 30 processes since February.

There's nothing wrong with our situation; however, we don't optimize for intraday restarts. As part of not prioritizing intraday restart related tasks, we've never addressed a bit of data-loss that occurs on a restart. It's traditionally been believed that the data wasn't very important (nice-to-have, if you will). However, the other day I wanted to rollout a new feature in the morning - before our "day" began. One of our customers stopped me from rolling out the software because he didn't want to lose the (previously believed nice-to-have) overnight data.

That was the moment that drove home the fact that even in our circumstances we needed to be able to roll out new software as seamlessly as possible. Even if mid-day rollouts are rare, any problems that a mid-day rollout creates will make it less likely that you can do a mid-day rollout when that rare moment occurs.

Tests and daily rollouts are nice, but if your team is looking to move from good to great I would recommend a non-zero amount of actual application usage from the user's point of view and fixing any issues that are road-blocks to multiple intraday rollouts.
© Jay Fields - www.jayfields.com

Clojure: Get All Nested Map Values (N levels deep)

I recently needed to pull all the values from a nested map. The map I was working with was designed to give easy access to data like so (formatted):user=>
(def my-data {"Jay"
{"clojure"
{:name "Jay", :language "clojure", :enjoys true},
"ruby"
{:name "Jay", :language "ruby", :enjoys true}}
"Jon"
{"java"
{:name "Jon", :language "java", :enjoys true}}} )
#'user/my-data
user=> (get-in my-data ["Jon" "java"])
{:name "Jon", :language "java", :enjoys true}
user=> (get-in my-data ["Jay" "ruby"])
{:name "Jay", :language "ruby", :enjoys true} This worked for all of the ways the data was accessed, until someone asked for a list of all people and all languages.

I needed a function that grabbed all the values nested to a point in the map (grabbing only the values instead of the deepest map wouldn't have provided valuable information. I created the following function that should grab all the values up to the nesting level specified.(defn nth-vals* [a i m]
(if (and (map? m) (> i 0))
(reduce into a (map (fn [v] (nth-vals* a (dec i) v)) (vals m)))
(conj a m)))

(defn nth-vals [i m]
(if (nil? m)
{}
(nth-vals* [] i m)))The nth-vals function can be used as the following example shows. (assuming the same map for my-data)user=> (nth-vals 2 my-data)
[{:name "Jay", :language "clojure", :enjoys true} {:name "Jay", :language "ruby", :enjoys true} {:name "Jon", :language "java", :enjoys true}]For reference, here's what's returned if we reduce the map all the way to it's values.user=> (nth-vals 3 my-data)
["Jay" "clojure" true "Jay" "ruby" true "Jon" "java" true]That list of values may be helpful for someone else, but it wouldn't have solved our current problem.

And, if you're interested, here's what's returned if you ask for 1 level deep of values. (formatted, again)user=> (nth-vals 1 my-data)
[{"clojure" {:name "Jay", :language "clojure", :enjoys true},
"ruby" {:name "Jay", :language "ruby", :enjoys true}}
{"java" {:name "Jon", :language "java", :enjoys true}}]I wouldn't at all be surprised if something like this already exists in Clojure core, but I haven't found it yet. Hopefully
© Jay Fields - www.jayfields.com