Andrew Harmel-Law's Blog
April 12, 2021
Chapter 3: A Reassuring Co-Incidence
NOTE: This post is part of my extended series on the ���Experience of Development���. If you���ve not read the two-part introduction yet, it���s perhaps a good idea to hop over and check out part 1 and part 2 before continuing.
I wasn���t sure if my ideas regarding the experience of development would make sense to anyone else. But then I saw a post on twitter from Sergey Bykov and I realised there were at least two of us thinking along similar lines. His tweet was something he said in a 2015 Microsoft article (the italics are mine):
���People tend to equate computer science with mathematics, but I think of programming as more similar to physics,��� he explains. ���[���] I learned that when you have a problem in physics, first you must understand what pieces are involved, how they connect and how they influence each other. You build a mental model and only then would you try and solve the problem and eventually apply math.
���When you program systems, it���s the same. You identify the pieces involved, how they interact, build a model, and then you apply mathematics and programming. In reality you spend a very small percentage of time writing code and more time figuring out why things don���t work as intended.���
W.o.w.
It was all there, in microcosm, but still all there. The problem-solution mental model which is constructed using substrate models. Even the conveying of those models to the outside world in the form of code, and the subsequent wrestling with the difference between what you thought would be the result, and what the result turned out to be.
In the article he was quoted in Sergey went on to use this realisation as a jumping off point to argue for simpler cloud programming models. My goal is to take this in a different direction. I want to interrogate further these elements which Sergey and I have both identified and see what more we can learn from them about the act of development and our experience of it. Perhaps this will also lead to improved programming models and languages, libraries and frameworks, but I really want to give we developers a better understanding of what happens to us when we code.
It was thinking about this specifically human, and consequently personal, aspect which made me pay closer attention to what Sergey described. While all the elements of my conjecture, as well as the supporting concepts, were there, was it all completely relatable?
Well no. And the differences between us are, I think, worth examining. I know this because of the interviews which I did with various friends from the Winter Tech Forum and their personal insights which they shared with me. I learned then that what you might think we would all do in the same way, is in fact a source of great variety.
The first thing which jumps out from Sergey���s quote is the suggestion (although I could be interpreting it a little literally) of a good deal of up-front mental-model manipulation - what he describes as the selection, connection, and interaction - all prior to his writing any code. I don���t doubt that this is how he does it. People I have interviewed have articulated similar approaches. If I were to describe this way of approaching things I might use verbs such as ���construct���, ���invent���, ���shape��� and ���create���. It is as if Sergey and his ilk have an intimate knowledge and grasp of their substrate mental models, and from those, carefully combine these consciously and methodically to get to the result they need.
I however go about things in a different way. When I code I always feel as if I am working somewhat in the dark. Verbs I might use for a significant part of my process are ���discover���, ���find���, ���explore���, and ���recognise���. My approach to things has more in common with starting with a large lump of clay and then slowly but surely paring pieces away, sub-dividing, cleaving, differentiating, until the detail I need reveals itself.
Now it should be made clear that neither of these is better than the other. Nor am I suggesting that these are the only two types of experience of this aspect, nor that they even comprise the vast majority of the experience of developing code. But I do believe it is very interesting to think about why these differences might arise. Does it spring from how we are most comfortable thinking creatively to make things or solve problems? Does it highlight a more structured versus free-form ways of approaching matters? Might we argue one is more scientific and compositional while the other more sketchy and reducing? Could it indicate a predeliction for verbal or visual thinking? Or linear compared to non-linear thinking? It���s interesting to consider all these possibilities, but not yet, there are other posts for that, and we have another personal difference between myself and Sergey to consider.
This second difference is in the getting of the mental models into the outside world. Sergey seems to indicate he first thinks in terms that seem completely mental, and then, in a distinctly separate step expresses this by ���applying mathematics and programming���. It sounds here as if (and again I could be way off base) the constructed problem-solution model is, for Sergey, truly an abstract one. That is to say, he���s not thinking in the constructs of the language or framework which he is going to use to encode his models outside of his mind. The conveying seems to be both a distinct step, and an act of translation into a specific syntax.
I however, am not so pure, nor so distinct. I still think largely in the constructs of the language which best represents the programming model I am targetting. For example, if I���m writing object-oriented, memory-managed code, there will be a lot of Java constructs and Java libraries swilling around in my mind.
So what do I mean by ���not so distinct?��� I mean, frequently I will use the act of drafting the code, of working with the syntax itself, on the screen in front of me, to help push along my mental-model manipulations. Sometimes this will be simply to get the fast feedback of the red squiggly lines telling me my conception won���t compile; but other times there is something about seeing the constructs out there on the screen in front of me, with their names and their syntax-highlighted, which gives me a clarity I need to lock something down and move to the next element.
In the first difference I shared what I knew from interviews: many folks think far more like Sergey in that regard and far less like me. Is it the same with this second difference? Of course. And just as before, I am in no way arguing for one way being superior to another, nor for these two approaches being the only ways it is done.
So again, what might these external manifestations of difference lead us to uncover? I believe that my reliance on the syntax more directly, and consequently on much, much tighter loops out of my head and into the IDE, are due to a far fuzzier way of thinking.It could also be because I���m just not as good a developer as many others I���ve worked with and spoken to. One thing I do know is that as I code, my mind frequently transitions rapidly from small-scale to large-scale then back again, and this means if I don���t flush my buffers to the outside world regularly I am in danger of losing things - of losing pieces of mental model.
You can see this very clearly when I code. I���m always impressed when people write their functions starting at the top and working their way down to the bottom. I meanwhile jump around, between classes, between functions, between lines in a function. I drop TODOs everywhere when ideas come to me, and consequently the act of tidying my code is frequently an explicitly separate and significant step.Pairing with me can be quite a labor of love. Does it work for me? Yes, it seems to. Does it work for everyone? Definitely not.I���ve had feedback on just this specific point.
Having looked a little at the two differences, what is most striking however is the similarity between Sergey���s ���figuring out why things don���t work as intended��� and my conjecture that the experience of this surprise is where much (though not all) of the emotion experienced in development comes from. How so? It doesn���t seem very significant when expressed how Sergey puts it does it.
Well, what is exciting to me, is that despite the significant differences in thinking and coding styles which I believe exist between Sergey and I (and backed up by experiences I���ve had with others) Sergey still thinks this mismatch aspect worthy to be called out. In fact, it���s so important that it justifies his proposal for a new approach to cloud-computing models. If someone who relies on syntax and code and the external loop as little as Sergey thinks the mis-match is important, then thats good enough for me.
That���s it for this post. I���m going to keep sharing these musings, and hopefully also begin to bring in some specific case-studies of how development is experienced by specific individuals. In the meantime I���d love to know what you think. If you have any thoughts or any other kind of feedback please leave a comment below, or reach out to me on Twitter. Thanks!
April 6, 2021
Implications of the Silicon Valley Autonomous-Engineer Model
In my previous post I suggested a new way of looking at (and therefore comparing) organisations: considering them as a collection of services which they offer their members, and then looking at how each of those was delivered.
In the back-and-forth on twitter which followed, Krisztina suggested I have a look at a blog post on how Silicon Valley-like companies ���get��� software engineers, while others don���t. After a quick read, it seemed like a great opportunity to try out my approach. This post is the result.
Written by Gergely Orosz, the main argument of the post is that the way Silicon Valley companies organise results in faster innovation at a company level, better professional growth for engineers, and more bang for their salary-buck. It���s well written and well-argued. From his biography it is clear Gergely has extensive experience working for multiple SV-type companies and I see no reason to doubt the accuracy of his conclusions on how they operate.
Gergely structures things by listing seven things which makes them realise these benefits. I���ll follow the same structure, tackling each in turn.
First up is autonomy . This is autonomy to select what to work on, autonomy to subdivide that work, and autonomy to decide how to tackle it. There is also autonomy as to how much time to invest in a given task (e.g. paying down tech debt or delivering new features). Unsurprisingly there is little / no micro-management. Engineers are self-managing.
Looking back at the services I���ve identified, which ones are at play here? We clearly have skill/resource allocation - this service is being performed for the engineers, by the engineers. It also seems as if decisions are happening (e.g. ���should we pay this debt, or work on a new feature?���) and again, this is done by and for the engineers individually, or perhaps in teams. The post is generally individual-centric, but not exclusively.
Reading on a little we can also spot (long term) direction with the call to ���solve problems that the business has��� (how we know what these problems are is partially addressed later).
In section two we get our first explicit comparison with non-SV/traditional companies. Banks being mentioned specifically. Here we see how hierarchy is the solution in traditional companies for the services we have seen delivered by autonomous individuals (and potentially autonomous teams) in SV-co���s. In traditional companies decisions are taken higher up, by managers. No decisions are taken by developersI���d argue that there are, given that development is all design decisions, but I get the author���s drift here. . I too have seen structures like this where decisions required lower down the hierarchy had to be packaged and sent up the line to the appropriate echelon, where a decision would be taken, and passed back down again to be implemented, most likely to a specific team which had (at least theoretically) the appropriate skills/resources to allocate.
This is compared (still interestingly using a pyramid-graphic) with the SV-co doing decisions at the bottom level, but taking into account ���business context��� which is passed down to them from higher up. That makes more sense. It could be argued that this ���business context��� is the (long term) direction which we could see was acted on above, but were unsure where it came from. This might also be input into the decisions.
We then move on to the next critique of non-SV/traditional companies: the problems around skill/resource allocation which arise from the fact that software engineers are expensive and need to be constantly fed with work. I have little argument with this diagnosis, and it���s interesting to see here the introduction of another key service (called out here as a skill that���s explicitly hired for) and that���s communication. In SV-co���s, where work is not being spoon-fed to engineers, and where engineers are expected to have an impact worthy of their salary, they have to ensure that they have the information they need to maximise this. I���ve met many engineers where this was clearly a skill of theirs, and (until the global pandemic) it was one of the articulated reasons Netflix didn���t open another development office - the impact of cross-site working on engineer-to-engineer communications was deemed to be too high to bear.
In the examples which follow, another service can be discerned in phrases such as ���have we checked with the legal team?��� and also ���if we combined project X and project Y��� and again ���can we delay the project by a month, until the new infra is ready, to avoid double work���; this service is co-ordination. Clearly, if this information is not acted on, then it is only communication, but we can assume it is, and that co-ordination results.
In section three we get lots more information which could all be labelled ���communication���. Compared to the non-SV/traditional co���s we are to believe there is a lot. It is interesting to note that one older SV-co, Apple, does not have such a transparent approach. It is also worth pondering if this is all the information, if it is truly open, or if there is an element of selection and boosting happening.
Section four introduces aspects which again touch on services we have already mentioned; namely (long-term) direction, communication and co-ordination. While it���s undoubtedly the case that these interactions and guidance-seeking do happen far more in SV-style co���s, I can���t help wondering if some of this is down to size, and location. I have known many people at non-SV/traditional places gain very similar insights and inputs but via un-official routes: meeting up after work, chatting in the canteen, etc.
Section five continues this theme for me. I���ve rarely witnessed the ���up and over��� style of communication being mandated, but it is true that when you do see peer-to-peer, probably cross-team communications and co-ordination it will be less frequent, more ad-hoc, and not relied upon as a definitive means of collaboration.
Gergely does have a point here about the hierarchy���s role in communication however. One problem I see again and again is that hierarchies typically communicate down relatively efficiently, but are terrible at facilitating communication upwards. He is also right that peer-to-peer communication is faster, and higher-bandwidth. But is frequently also has the blessing / curse (select those which apply) of being un-targetted and unfiltered, leading to information overload. I agree here that managers can play a role in mitigating these issues by facilitating the right conversations. This sentence adds nuance to the predominant view in his post that ���engineers just talk directly to other engineers��� and gives the hierarchy some residual role in comms. It���s also worth pointing out that it���s refreshing to see a post which argues for increased autonomy but without assuming that means the end of hierarchy. I���m agnostic about it, but it does show a degree of independent thought and nuanced consideration.
Skipping section six as it���s not relevant to this view, we arrive at the seventh. This is all about recognition, another of the services, although viewed through the lens of attraction of talent and the salaries they are paid. It could be argued that this is the section which closed the argument and is intended to justify the vast salaries devs can command. I���m not sure this section really lives up to that promise however. Admittedly, many SV-style co���s have infrastructure (discussed in section six which I intentionally skipped) to optimise the delivery lead time of developers features, but I���m not sure every SV-style co is jam-packed with devs delivering features of the calibre suggested, all day, every day. I also know that for many, the salaries can be a major source of stress when things aren���t working, and when a company is solely structured to provide recognition along these lines, many other things fall by the wayside, leading eventually to many products which are definitively a ���success��� but also of questionable social good. The ���like��� button mentioned in the article could be argued to fit this profile for example.
Given all this , I have little issue with the underlying argument of the closing paragraphs (although I do take issue with the characterisation of devs in non-SV co���s as ���factory workers��� as much as I do the ���10x engineer��� trope). Software is indeed eating the world, and those companies which see software as delivering value, rather than as a cost centre are storing up trouble for themselves. I do find it hard to believe however that these SV-co���s are comprised soley of devs and ���managers���. Now, admittedly I haven���t looked at job postings recently, but I���m pretty certain there will be many other folks playing roles directly related to delivery of software not mentioned here. This tunnel-vision severely undermines the argument of this article in my mind, because by placing all the credit in the hands of a blessed few it makes you wonder about what���s missing.
That thought is what takes me onto other issues which arise from this incomplete view. What about the other services which the author doesn���t choose to mention? Namely, constitution, conflict management, and learning. I can take a guess at the latter. In many SV-co���s they hire devs who are deemed ���senior��� and beyond, so a majority of the fundamental learning has been achieved earlier in their career. What further learning does take place is typically ���on the job���. (Netflix define their approach to this in their Culture deck (slides 119 and 121). I won���t go into this any further as the other two are far more important.
Now, Netflix are clearer on these also, most notably in their calling out their intolerance for ���brilliant jerks���, but they are singularly notable for taking this standpoint. Elsewhere in SV-co���s there is frequently an ad-hoc, or worse still, lip-service approach taken to these essential elements. Susan Fowler���s Whistleblower goes into these topics in great detail, specifically in the case of Uber, which is one place the author has also previously worked. What Fowler calls out in her book is that not only were these aspects terribly implemented, they were set up in a way to protect the company, and those working there who were senior / had greater tenure / were cis, white, hetrosexual men, all at the expense of the individual. Subsequent events at the company have shown that what had grown up was in many ways a monoculture. Undeniably, the engineering was world-class. The products shipped helped Uber become one of the largest and most valuable companies in the world. The goals outlined in Gergely���s post were achieved. But at what cost? After writing this post, I���m more convinced that all services are provided by all organisations, but it is important to get them all right, not just the ones which seem to be directly making the company money.
Well , that���s my first end-to-end deployment of this lens on organisations. It was interesting to write, and helped me think about things in a methodical way. It also felt as if it made sense, allowing me to pull apart a few concepts into their constituent parts, and see some parts in light of others. But that���s just me. I���d love to know what you think. If you have any thoughts or any other kind of feedback please leave a comment below or alternatively reach out to me on Twitter. Thanks!
April 2, 2021
Chapter 2: On Bit Depths and Fine Wines
NOTE: This post is part of my extended series on the ���Experience of Development���. If you���ve not read the two-part introduction yet, it���s perhaps a good idea to hop over and check out part 1 and part 2 before continuing.
There were some great sessions at JavaOne 2008. Among the stand-outs for me were Josh Bloch and Neal Gafter���s Java Puzzlers (returning for the umpteenth time) and Kathy Sierra���s talk about reverse engineering passionate usersYou���ll need to rely on my memory for the contents of Kathy���s amazing talk because some people are horriffic jerks and that means the rest of us can���t hear from people like Kathy any more, despite the fact they���re awesome. .
What struck me at the time about Kathy���s session was how she had no words on her slides. None. At. All.
What���s stuck with me far longer was her use of the concept of ���bit depth���, which wasn���t the traditional technical one you���re probably thinking about. Kathy used it to differentiate the kinds of users of a given service or piece of software. The argument of her talk was to zone-in on one particular kind of user; the power users, and reverse engineer your system from what they would think were excellent features backed by a solid and coherent system model.I believe this was perhaps an early version of her Badass Users book because reading her site now, a bunch of things definitely resonate.
She described these peeps as ���high bit-depth users���. To explain this she explained how she understood wine. She had a low bit depth appreciation she saidApologies Kathy if I���ve got this the wrong way round . That is to say, she could tell red wine apart from white wine, but that was it. Others had a higher bit depth, and the higher their bit depth, the deeper and richer their concepts of wine were. The greater their awareness of things like grape varieties, the wine-making process, the regions wine came from and the influence of ���terrior���, as well as their knowledge and their sensitivities to subtle cues and differences. Consequently, their enjoyment was more subtle and nuanced too. Whereas for a low bit-depth drinker wine might be ���good��� or ���bad���, for a higher it depth aficionado might be ���forward on the nose��� with ���good legs��� but still ���not quite as good as the previous vintage���Can you guess which end of the field I���m placed at yet? A little Google is a wonderful thing. .
So what about Josh and Neal? Well, if you���ve not familiar with their regular sessions (and accompanying book) let me fill you in. Both Josh and Neal had worked on key core aspects of Java. On top of this, Josh had also written (and frequently updated) the classic ���Effective Java��� while Neal worked on Google Calendar and led the Java lambda specification team. This meant both of them were ideally placed to host an annual session where they posed a series of questions to the assembled audience which largely ran along the lines of ���given this code, what would you expect the output to be?���
The reason they did this was to illuminate some of the darker, more subtle, and (some would argue) less obvious areas of the Java language and core libraries. It was intended to educate, but it was also terribly good fun.
And, just like Kathy���s talk, years later it���s come to strike a new resonance in me.
Now, in light of my theory of what makes up ���the experience of development��� I think they were actively working to improve the bit depth of those attending their session. They weren���t just presenting facts. They���d explain why, draw out differences between Java versions, give a bit of history, and explain idioms which had come and gone, frequently as a result of things which were hard (or easy) in the Java development world at the time.
In expanding the Java dev bit-depth of those present, they were expanding the substrate mental models of those in the audience. Sometimes it was as low-level as how fundamentals such as primitives worked. Other times we moved up a little to syntactic sugar behind various bits of syntax. Other times still it was about the internals of various collections libraries, and other still attempted to help people grokk that great darkness which is co/contra-variance.
With this all in mind, let���s loop back to Kathy. As bit-depths increase, mastery does too. That in itself must surely bring additional elements to the experience of development. To have more detailed and nuanced substrate mental models must surely make them easier to work with, purely due to the ability it gives us to pick elements which do what we need more quickly, and consequently have fewer instances of surprise (especially puzzler-level surprises which in the many-sharp-edged world of Java can be real head-scratchers)I���ve even seen the pass-by-reference vs pass-by-value issue confound really solid devs for example. .
But there���s something else which arises from greater bit-depth and it���s something which has a direct impact on enjoyment. I���ve seen this with wine, with golf, and with programming. The more expert a person becomes, the less they can like certain things unless absolutely everything is perfect. Wine experts are far less able to just enjoy a glass of ���really nice plonk��� as some call pretty good wine, because a few subtle elements - which those with a lower bit-depth experience don���t even register - to connoisseurs are glaring issues. Terrible golfers (me) are happy to have hit a few decent shots in a round of eighteen disastrous holes, while I���ve seen a friend who played off a single-figure handicap have a bad week after they shot a double-bogey on a single holeThey still won. That didn���t seem to matter. .
The same goes for many high-bit-depth aficionados of programming languages. I���ve frequently seen them pick languages which are more ���pure��� and balk at having to sacrifice this purity in the name of getting things done, which generally means using a library or framework which does what they need, but which also forces them to do things in a style they feel is not their favourite.
I���m not saying this is bad. (I���d love to be that great at any language.) What I���m saying is, low bit-depth understandings of languages is almost always something which we will be motivated to improve on, for all but the languages (and frameworks / libraries) which we almost never touch. The higher levels of bit depth however, while they almost inevitably bring far greater knowledge and mastery of a language, can also frequently bring a reduced enjoyment of use, and perhaps even reduced effectiveness, when purity is not sacrificed sufficiently on the altar of pragmatism. (It can also lead in some cases to code which is nebulous and impenetrable to the average dev who isn���t at the same level of mastery. But that���s straying into the realms of the experience of developing with others which is a who other set of posts.)
That���s it for this post. I���m going to keep sharing these musings, and hopefully also begin to bring in some specific case-studies of how development is experienced by specific individuals. My next post begins this journey by attempting to do this from a personal perspective (and perhaps even finds a supporter). In the meantime I���d love to know what you think. If you have any thoughts or any other kind of feedback please leave a comment below, or reach out to me on Twitter. Thanks!
March 26, 2021
Your Organisation Viewed as a Collection of Capabilities
(This post is second in a series. Part one, ���Is Your Organisation Broken?��� sets the scene for this post���s proposal of a new way to look at organisations.)
UPDATE (Sat, 22nd January, 2022) - Changed ���service��� to ���capability��� and added another: ���Org Design��� to the table.UPDATE (Sat, 29th May, 2021) - Added another service: ���Membership��� to the table.
I���ve wondered for a long time if there���s value in looking at all organisations through a lens which treats employees as consumers of capabilities the organisation provides, and then asks ���how does the way you organise provide these capabilities?���
This might seem a bit meta, so here���s two examples. The multinational conglomerate Acme Corp follows a ���traditional��� model of organisations in that it is hierarchical, and bureaucratic. This model offers capabilities such as ���work allocation��� and ���decision making��� and in both cases (at least officially) these capabilities are provided via a ���performed by a named individual at the appropriate level of management who follows a documented process��� approach.
Meanwhile the small startup Mom & Pop Soda Shop Inc follows a ���trust��� model in that anyone is expected to spot things which need to be done, and are permitted to do them. This model comprises the same capabilities (���work allocation��� and ���decision making���) but here both are provided via a ���all employees are empowered to do the right thing ad hoc and whenever required��� mechanism.
How different organisations provide these ���internal��� capabilities will clearly vary greatly, but I think (bear with me) that there is a finite set of capabilities that all organisations provide purely by dint of their being an organisation. Sometimes an organisation will provide one or more of these capabilities explicitly, and other times it will provide some of them implicitly (or even by accident). Furthermore, some capabilities will be provided officially, and others unofficially. Sometimes both types will be at play at once, one most likely working more efficiently than another.
Finally, please note that this breakdown says nothing about the quality or fit of any of the capability implementations in a given organisation. The capabilities as delivered might be terrible, or they might be excellent. They might conflict with or contradict one another.
With this in mind we can get to the capabilities themselves. I am no org-design expert (I���ve just read a lot, and consulted in a lot of places, and like to nerd-out and find how other folks organisations work) but I hope that the items on this list are both collectively exhaustive and mutually exclusive. Finally I���ve tried to put them in order of importance. It is likely that in all organisations the first two capabilities are explicit. Only in larger organisations might later / all other capabilities be so.
I���ve tried to use a representative way of representing this which names the capability, gives the need it meets, the outcome it provides, and the value it delivers.
Skill / resource allocation
Right people with the right tasks and required resources to perform them
Tasks and resources allocated.
Tasks performed
Quality outputs.
Personal sense of purpose
Accountability
Communication
Timely and accurate information required to perform a task
Information at the right place at the right time
Efficient and productive organisation
(Long term) direction
Strategy/vision/mission/goals based on sensing world outside the organisation
Alignment and focus of effort (decision criteria)
Organisational purpose
Decisions
Good decisions made efficiently and effectively
Timely decisions rapidly made
Progress aligned with direction.
Efficient use of people���s efforts
Co-ordination
Groups of people (and resources) doing the right thing at the right time.
Keep in sync with others
Co-ordinated activities and allocation of resources
Reduction in waiting / blocking
Constitution
Clarity on what the organisation values and what it will (and won���t) tolerate
No ���rogue��� (i.e. contrary to the constitution) elements
Clear checks and balances
Recognition
Recognition for my skills and contribution (my value to the organisation)
Salary, job title, etc.
Sense of self-worth
Conflict management
People held to account for their actions (or lack of them)
Alignment to ���rules of the game��� and social norms.
Conflict resolution
Increased collaboration.
Predictability.
More efficient delivery of organisational goals
Learning
Personal development, additional / improved skills
Better skills available within the organisation
Feeling of personal satisfaction
Greater organisational productivity. Increased loyalty to the organisation
Org design
Skills and insights in the membership deployed effectivel and efficiently
Org achieves its goals as closely as possible
Org structure is an enabler / multipler
That���s it for now. In my following posts I want to look at various organisations and organisational styles and see how these capabilities might map onto them. The next one takes these capabilities as a lens and looks at a descripton of the ���Silicon Valley Autonomous-Engineer Model��� with all its pros and cons.
March 25, 2021
Is Your Organisation Broken?
Organisations are tools[An] organisation [...] is a tool for making people productive in working together.
Peter Drucker, "Management���s New Paradigms��� from ���Management Challenges for the 21st Century��� (1999)
The vast majority of our organisations are broken, and they are broken because they have the wrong structure and function for the job.
Those which have, by some stroke of luck, both the right structure and function, will fail to change at the required time, and will end up with the wrong structure and function, and will therefore end up broken.
Even those which do change, and change when required, will change in the wrong way, and end up with the wrong structure and function; and will consequently become broken.
Our organisations are wastefulWhy do I say ���broken���? Surely this is too bold a claim? Is it too strong a statement? Are they not simply poorly-suited or ill-used? I would argue not.
Why do the organisations we work for (by investing our time and more) exist? Surely it is to produce things - either products or services - which have value to users, and then (if they are a commercial enterprise) to capture some of that value back in the form of revenue. And yet, to adapt John Yorke:
���[these products and services] have no value until they are in the hands of a user and being used for a productive effort. So any activity not spent getting the next most valuable feature into the hands of a user quickly is just waste���
Can any of us read this and honestly answer that they are not engaged in their work, day upon day, in the creation of a significant amount of waste? By ���significant waste��� I mean not simply waste due to slight inefficiencies; inefficiencies which, as by the sharpening of a blade or employment of a slightly different stance, can be smoothed away by kaizen-style small and targeted improvement. No, our waste is far, far grosser. It is the waste of producing things which have no definite value, and of working with and performing entire job roles, of complete departments, who cannot claim to be contributing to the organisation���s goal.
���Do you mean supporting roles?��� you challenge. ���Not everyone is engaged in the daily task of making the next widget, and yet without my hiring / managing / planning / etc., the widget-making would itself be terribly inefficient, if not impossible. Your declaration is due to a misapprehension; an over-reduction of the complexities of modern business, and the ways in which they have evolved to efficiently operate.���
To which I would reply, ���Are you so sure? Do you have ways to measure your impact? How can you be so sure that your efforts, both indirectly and directly related to production of your organisation are contributing?���
And then you might counter, ���Just because there are no metrics��� (for there are never any metrics) ���does not mean we are not successful in our actions.���
And to this I would reply ���In which case you must surely all be able to state your common goal, for if you at least know this then you are assured in being aligned and confident that you are all pulling together.���
And at this point inevitably, besides repeating the meaningless platitudes that these days take the place of real vision and strategy, there is silence; because these circumstances are the case everywhere, and if, by some miracle they are not at present, they will be soon enough, for the reasons I set out at the beginning.
The wasted potential of peopleThis waste that we are all engaged in has a name. It is the worst waste of Lean; the wasted potential of people.
But it is worse than this. Not only are our organisations wasteful, but also this waste is not what we (myself included) want. We want to contribute. We do not wish to be engaged in the generation of this waste, and yet if we are honest we know we are to a painfully significant degree.
Why is it so prevalent? It is because, as I have illustrated, despite all of our best endeavors, it is the structure and function of the organisations in which we work which prevents us from succeeding. If the means to resolve these issues were in the hands of the individuals who suffer from them, then there would not be such a magnitude of this kind of waste.
This is the reason why I can confidently state that our organisations are broken; because we all know it; because we see and feel it every single day.
Should bad strategy not take the blame?Yet, if not already depressed by this dialogue, some fight on, raising the challenge ���surely a great deal of these failures are strategic and not organisational; due to poor focus - or if we���re being kind - lack of clarity?���
But if this were the case, Simon Wardley would not have had to talk so directly and repeatedly to organisational matters in his key doctrines. I am thinking here of the following (and would at encourage the challenger to take a look at them in detail) ���be transparent���, ���remove bias and duplication���, ���optimise flow���, ���think small���, ���distribute power and decision-making���, ���provide mastery, purpose and autonomy���, ���there is no one culture��� and ���design for constant evolution���.
And here we must acknowledge the assistance that the respondent has given us by their challenge. Why is there so much on organisation, both structure and function, in the Wardley method? It is there because the problem of organisation echoes very closely the problem Simon Wardley identified regarding strategy; viz organisations and those who run them have no idea what structure and function they should cultivate. And just as with strategy, leaders are terrified of admitting this, and so blindly copy others, and make incredibly costly mistakes.
But it is worse - at least in strategy there are many different options to copy from, and this flailing around at least provides for some diversity of sport for Wardley as he discusses his approach. Meanwhile, in western organisations there is only one structure, and one set of functions, and what that structure and those functions are is where we shall turn next.
A potted history of the ���One Right Organisation���All organisations are fundamentally the same structure, and operate the same functions. It need not be this way; but it is.
Let us consider how we got here. Why are organisations always the same in these regards? Astonishingly there are few who, if challenged on the street to extemporise on why organisations have developed into their current form and function, could articulate even part of an answer. This is despite the fact already mentioned that there is so little diversity, and that ���organisation��� of our kind is a relatively recent phenomenon.
We must turn to the great management thinker, Peter Drucker for a potted history. Firstly, the origins, and with them the hierarchy, and idea of a means of communication:
���Back in 1870, [���] large enterprises were first beginning to take shape. At that time, the only large permanent organisation around was the army. Not surprisingly, therefore, it���s command-and-control structure became the model for the men who were putting together transcontinental railroads, steel mills, modern banks, and department stores. The command model, with a very few at the top giving orders and a great many at the bottom obeying them remained the norm for nearly one hundred years.���
Peter Drucker, from ���Management as Social Function and Liberal Art��� in ���The New Realities��� (1988)
Then came the subdivision into departments and beyond that into specialisation of individual roles:
���By World War I the standard functions of a manufacturer had been developed: research and engineering, manufacturing, sales, finance, and accounting, and a little later, human resources (or personnel). During World War I, large numbers of unskilled, pre-industrial people had to be made productive workers in practically no time. To meet this need, businesses in the United States and the United Kingdom began to apply the theory of scientific management developed by Frederick W. Tayor. [���] They analysed tasks, and broke them down into individual, unskilled operations that could be learned quite quickly���
Peter Drucker, ibid
To many, myself included, this history is a revelation. That it rings true is important, but more significant is the lack of general awareness as to the ubiquity of this single model, or of the why. Surely these facts in themselves go a great way to explaining why so many competent, skilled, and motivated people persist in working with the structures and functions in which they find themselves, and with their every action, reinforce the status quo. For when there is only one, the natural assumption is that it must be ���right��� and the ���only way��� and so the best way to get on is to ���work with the system���. Within this world, who has the chance to examine the system itself, and perchance entertain the possibility that it is, in fact, only one of many possibilities.
Beyond ���One Right Organisation���This is not a recently identified problem. Drucker sounded the alarm on the issue way back in 1999 (emphasis is the authors):
���From the very beginning more than a century ago, the study of organisation has rested on one assumption: There is���or there must be���one right organisation. What is presented as the ���one right organisation��� has changed more than once. But the search [���] has continued, and continues today. ���
Peter Drucker, ���Management Challenges for the 21st Century��� (1999)
He continued, in terms which still form a stinging accusation:
���Immediately after World War I [��� there] developed the principle of decentralisation. And now, in the last few years, we have come to tout the team as the one right organisation for pretty much everything.���
Peter Drucker, ibid
But If our organisations are failing us, and they are all the same, what is the solution? Drucker concludes, and here gives us hope:
What are the alternatives?���There is no such thing as the one right organisation [���] It has become clear that organisation is not an absolute. It is a tool for making people productive in working together.���
Peter Drucker, ibid
So we see that the situation is not as bleak as it may seem, and there are other reasons for growing optimism.
For one, the notion of organisational change is nowadays a commonly accepted, if always misapplied one, if for no other reason than software is continuing to eat the world. Secondly, the concept of ���Organisational design��� has, of late, become a buzzword, signaling that the concept has reached a point where in-depth consideration and investigation is likely possible. Thirdly, the continued successes of the lean movement - which bears with it a new product-focus - coupled with the dominance of DevOps in all things software-delivery - underpinned and bolstered the solidly statistical conclusions of Dr. Nicole Forsgren et al���s ���Accelerate��� - indicates that the ground may be set for what needs to come next: An extensive, diverse, and effective move to a new way of thinking about, and working with our organisations as tools - both in structure and function.
Given all this, how might we set about fashioning these new organisation-tools?
I have argued elsewhere that the proto-skills for such a revolution are already widely available - systems thinking, distributed software design, evolutionary design. To this I would add further ingredients of service design, promise theory, and pattern languages as well as lessons learned from the various social and political movements of recent times.
The elements of a new approach are also to hand. Massive and collaborative co-design, and experimental rather than prescriptive discovery and adoption techniques will have significant roles to play. The change in approach will matter far more fundamentally than the destination, for we know from vast experience that we love to copy others, rather than finding what is right for us (just look at the ���Spotify Model��� and the Agile backlash).
But we must additionally change the cadence of our approach. There will never be a ���one size fits all��� solution, but it is clear that the ���massive annual re-org��� suits no-one. Smaller, more targeted, and near-continuous change will become the name of the day, dictated by the needs of the organisation.
The degree to which these elements differ from how we do things today will vary on a case-by-case basis, guided by their specific needs as well as the times they find themselves in. I am not calling for the next ���best way of doing things��� - to do so would be to fail to escape the trap that we have just described - rather I am arguing for an honest and full appraisal of where we find ourselves, and a collective and sustained effort to drive relevant and effective change.
So what to do first? How about a return to first principles? How about directly identifying and considering the needs we have as people working in organisations? If we did, what might this look like? If you���re excited to find out, so am I.
I���ll get into all these aspects and more in the later parts of this series. Next up, a humble suggestion for a list of services that all organisations provide.
March 24, 2021
The Experience of Development - Part 4: The Timeless Way of Building
But there is one more aspect I want to bring to the table. The experience of the wholly-internal castle construction. ���..
The detail, granularity, and explicit-ness [SURELY THERE IS A BETTER WORD?] of our mental models is also highly significant. We are alo limited by how much we can ���hold in our heads��� [ALTHOUGH THE AMOUNTS VARY GREATLY FROM PERSON TO PERSON]. Admittedly, we do not need to hold all details as we have committed it to the screen [SCREEN? THERE IS AN ALMOST-IMMEDIATE RE-READING ASPECT HERE] but that does not mean we don���t need to be able to add to it, change it, improve it, and build on top of it.
Now we move to pleasure and confidence. [I���m NOT SURE WE DO.]
To joy.
To satisfaction.
[IF I WAS TO DESCRIBE MY EXPERIENCE OF DEVELOPMENT IN THREE WORDS I WOULD CHOOSE ���CREATIVE���, ���SATISFYING��� AND ���FRUSTRATING���].
In coding [���DEVELOPMENT���, OR ���MAKING CODE-THINGS���?] we are engaged heavily in tool-use. Tools which are complex, multi-faceted, nuanced, and which must gel [���GEL��� DOESN���T FEEL LIKE IT LINKS STRONGLY ENOUGH BACK TO WHAT WENT IN THE FIRST SECTION] with our wider expectations [BUT ���EXPECTATIONS��� DOES, SO PERHAPS�������] of what they are, how they are structured, and how they work.
As we become more familiar with a language, it���s associated mental model/philosophy, and it���s associated [REPEATS ���ASSOCIATED���] libraries, frameworks and tooling we see not only how it succeeds, but how it fails; we see it���s limitations and shortcomings.
[WOULD THE VIRGINIA WOOLF QUOTES FIT IN HERE?]
DON���T FORGET ABOUT BIT DEPTH OF MENTAL MODELS (AFTER KATHY SIERRA)
[IT FEELS WE LEAP FROM WRITING TO READING HERE. AND FROM ��� ]
The perception of these things is different for different people. Their mental models of the language may vary greatly in their ���bit depth���. [REFER OUT TO CATHY SIERRA.] So too may be someone���s comfort with the issues [AND SHORTCOMINGS?] which a language or associated ecosystem might throw up [NOT SURE I LIKE THIS IMAGE - ���� ���PRESENT���?].
There is also the fact of someone���s comfort with a language���s syntax (or even style) [N.B. IN DEVS THIS IS ALSO VERY PECULIARLY FLOWS OUT TO THE PHYSICAL ENV - CODE-COLOUR, KEYBOARD, LIGHTING, ETC ALL CAN PLAY A ROLE HERE] and the ease with which they can consume this and conjoure up [I LIKE THIS PHRASE. BUILD IN MORE SKY-CASTLES LIKE THIS?] the associated mental model from it.
In this there is a strong association with how much detail can be represented as a ���short-hand��� and therefore manipulated effectively and effortlessly with confidence and skill. [THERE IS A MENTAL-EFFORT SIDE-BAR / DIGRESSION IN HERE TO PULL OUT.]
But we have not spoken of joy. The joy must come in great part from mastery, [WHAT ABOUT AUTONOMY AND PURPOSE? DO I EVEN WANT TO USE THIS WORD WITH ALL THE DISTRACTING CONNOTATIONS IT WILL BRING?] and the development of the same. But it can also come from both a lack of friction [MENTAL EFFORT AGAIN] and deployment [���MANIPULATION��� / ���DEFEAT��� / ���SUBDUGATION���?] of vast [VAST? REALLY?] complexity with little [���LITTLE���? HOW ABOUT ���APPROPRIATE���? OR SOMETHING ELSE. THIS NEEDS TO BE UNPACKED TOO] mental effort.
There is also the potential joy in experiencing the pleasant surprise when something built from simple elements combines as hoped [���EXPECTED���? ���HOPED��� IS NICE HERE AS IT HAS THE ELEMENT OF FAITH IN IT���] (i.e. without a great deal of solid confidence) and works as planned, revealing a greater power [���POWER���? AGAIN THIS IS QUITE OVER-BLOWN] than had been imagined. [���THOUGHT POSSIBLE���?]
[**CHECK THE JOYS BELOW AREN���T ACTUALLY COVERED LATER ON**]
[THERE IS ALSO THE JOY OF HAVING USED ONE���S TOOLS EFFECTIVELY. MENTAL AND PHYSICAL. TO HAVE HAD THE RIGHT THINGS TO HAND TO HAVE APPROACHED IT ALL IN THE BEST WAY. TO HAVE FELT THE EXCITEMENT OF ALL THE PIECES OF THE MENTAL AND PHYSICAL PROCESSES WORKING TOGETHER AS ONE. ����< TDD COMES INTO THIS.]
[THERE IS ALSO HERE THE JOY OF A PROBLEM UNDERSTOOD AND BESTED, ]
[AND BEYOND THIS, THERE IS THE JOY FELT IN PRIDE OF CRAFTSMANSHIP - FIND ANOTHER WORD GIVEN THE CONNOTATIONS OF THIS ONE; OF A TIDY, JUST-ENOUGH SOLUTION.]
[AND THERE IS ALSO THE JOY IN AND OF A JOB COMPLETED. OF NEVER HAVING TO GO BACK SOMEWHERE. OF HAVING CONQUERED FRUSTRATION.]
[AND THE JOY OF HAVING LEARNED. OF HAVING UPDATED A MENTAL MODEL.]
How much of this is consious? It is not wholly correleated with mastery. [THERE���S THAT WORD AGAIN] You can consciously be struggling with something and be aware at [���OF���?] the lack of strength of depth of mismatch in mental models. [KEEP BRINGING IT BACK TO THE MENTAL MODELS.]
And this brings in [���BRINGS IN��� SUCKS AS ENGLISH] another joy, in the construction of a new set of mental models and experiences in using them. In the experiencing of what is easier and harder, possible and impossible, in this new mindset. In the surprise at failing, and the surprise at learning. In the surprise in contrast between a solid, comfortable, established model and a new, unfamiliar, effervescent one. [AND THE JOY OF HAVING CAUGHT AND PINNED-DOWN/EXPRESSED THE LATTER.]
What about different modes of thinking? Both the ���Happy Path��� [CALL OUT TO BRUCE AND JAMES HERE] and the other paths? What about mode-swapping, both within a stack (e.g. from core language, to libraries, to framework, to tooling, etc.) and outwith (e.g. databases, etc.) Is there joy in these changes, in deploying multiple tools [AND ASSOCIATED MENTAL MODELS] to solve an overall problem? [DON���T LOSE THE THING-PROBLEM HERE. I���M LEAKING MY ANTI-DB BIAS HERE] Is there joy in staying in one place? [ONE HEAD-SPACE?] Is there joy in being challenged and overcoming complexity, and is there joy in understanding how it worked, and then finally distilling it down to the simplest, most elegant, most intentional and explicit solution? [LINK IN THE THREE DIFFERENT PERSPECTIVES OF TDD HERE TO.]
[IT FEELS LIKE THIS COULD GO RIGHT UP AT THE TOP - OR PERHAPS WHEN WE START GETTING TO TOIL/FRICTION/EFFORT.](Programming) Languages are built to provide abstractions. Sometimes they are more explicit than others. Sometimes they are more consistent than others. Sometimes they are more complete than others. Sometimes they are high level, other times they are low level. [SOMETIMES THEY ARE BOTH, EITHER CONSISTENTLY OR INCONSISTENTLY.] Sometimes they [AND THEIR ECOSYSTEMS] are fitted to [THE ENTIRETY OF] our task perfectly. Other times they are not - being incomplete, or having [���CAUSING���?] friction with our mental-solution model [IS THIS MY NAME FOR THE OTHER MODEL?] (Is it easier when we think in their model? ���Thinking in Java��� for example���)
[LANGUAGES WHICH FIT:]���To begin with, there is the technical difficulty [���] that the very form of the sentence does not fit her. It is a sentence made by men; it is too loose, to heavy, too pompous for a woman���s use.��� from the essay ���Women and Fiction��� by Virginia Woolf
���And this a woman must make for herself, altering and adapting the current sentence until she writes one that takes the natural shape of her thought without crushing or distorting it.���from the essay ���Women and Fiction��� by Virginia Woolf (bold mine)
How much do we think of the mental models in the minds of others? How much do we care to express what is in our heads clearly, consistently and explicitly? How much do we think about reading again the code we have just output [THIS IS A HEAVY WORD, BUT I LIKE IT HERE] just now in order to get something to work?
[CONSIDER WHY I WANT TO QUOTE FROM WRITERS (AND OTHER ARTISTS?)]
[ANOTHER THOUGHT FOR EARLIER ON?]How too does this play into Bruce���s ongoing question about control? Do we prefer to be falsely unsurprised or truthfully [?] (and frequently) surprised (and disappointed). IS THIS WHY SO MANY HATE TO EXPERIENCE THEIR CODE RUNNING AS IT WILL SHATTER THEIR ILLUSIONS?
[ALSO: WHAT ABOUT SYSTEM METAPHOR?]
N.b. ���The Programmers Brain��� from Manning https://livebook.manning.com/book/the...
N.b. TED: How Your Language Shapes the Way You Think https://www.instagram.com/tv/CKXQYJwH...
March 14, 2021
Chapter 1: The Fine Balance
���My phrases have to be crammed with information. They have to be crammed with content. Not that a compromise is necessary. I think that so much of what I do with language is instinctive. You see I don���t really think of myself as creating the language, because language pre-exists; I just have to go where it is. I have to find it and it has to find me; and it���s as much about listening as it is about being active. You have an inner voice where the perfect novel is kept; the perfect sentence is kept; and your task is to mediate that to the world outside; and that���s the writer���s daily business.
But of course you don���t think of it like that, you just type fast. And because you���re accustomed, because you���ve been doing this for years and years it���s like starting a song. The rhythm won���t falter. The internal logic of the prose won���t let you down. In the same way I think that with a long book you sometimes have to forget about structure and trust that it���s doing itself; and it is.
But there���s such a fine balance between exerting absolute control, and letting go, so that the thing can happen.���
Hilary Mantel interviewed by Michael Rosen for ���Word of Mouth���, BBC Radio 4.
Each of us approaches software development in our own individual way, based on both our understanding of the-thing-we-are-trying-to-create-slash-the-problem-we-are-trying-to-solve, and on our grasp of the tools with which we are working.
In doing so, three things���the mental models that we construct, the pre-existing mental models which we call upon, and the means by which we mediate them to the outside world���are potentially called into question every time the code itself, that external manifestation of these mental models and bearers of our expectations, tells us what it can and can���t do. It is at this nexus that our internal conception of the-thing-we-are-trying-to-create-slash-the-problem-we-are-trying-to-solve is confronted with an alternate reality; the reality of syntax and compilers and CPU registers and electrons.
(For more detail on this loop, see part one of the introduction to this series: ���A Conjecture���. For detail on the nature of the models and some clarifying terminology, see the second part: ���Models, All the Way Down���.)
The state of this alternate reality is made manifest to us through our senses, and there then arises an overlaying of this input with our mental models and their predictions. The two are compared, and there are a variety of outcomes which can result.
If what our models predict matches up with what we have sensed then the reward of having crafted a cloud castle that we have successfully mediated into the world outside hits. We feel good. This is an experience of development. One of the primary ones which keeps us coming back for more.
But when the match fails, signaling that our mental-model hypothesis has been falsified���an all-too frequent occurrence in my experience���we react to this disparity, this ���surprise���, in one of several ways. This too is the root of experiences of development.
The type of reaction which arises depends upon our confidence in which of the two should carry more weight. Do we trust what we expect? Or do we trust what we sense? Most importantly for our purposes here, how do we explain the discrepancies away? Let us enumerate the coping strategies here, for I propose that it is within these responses, conscious or unconscious, that many of the varieties of development experience lie.I have tried to arrange these in order of magnitude, but clearly such an effort is very subjective.
FirstlyAnd most mundanely. , we can doubt that we effectively mediated our mental models to the outside world, to the code. Did we make a typo? Did we forget a setting? Did we press the wrong button?
Secondly we can maintain our belief in the creation-solution mental-model we mentally constructed, but doubt that it is either a) a valid fit for the-thing-we-are-trying-to-create-slash-the-problem-we-are-trying-to-solve, or b) encodeable in our chosen language. Is our mental-model valid, but unsuitable for this situation?
Thirdly we can doubt���Doubt��� could just as easily be replaced with the word ���question���, or with ���become confused about���. The only differences are in how conscious we are of this feeling. ���Doubt��� will do for now. the integrity of the thing we constructed in our heads: the creation-solution mental-modelSee my previous post, ���Models, all the way down���, for the classifications and definitions of ���creation-solution��� and ���substrate��� mental models. , assembled in our quest to make / solve. Did we make an error in our mental model construction as it pertains to our situation?
Fourthly we can doubt our substrate mental models - the ones representing our languages, libraries/frameworks and tool chain.Again, see my previous post ���Models, all the way down��� for discussion of this. Here we likely go back to the documentation, and update one or more of these substrate mental models as a consequence. Do we misunderstand the tools we are working with?
Finally we can doubt what we are experiencing, and act as if it���s not happening. This can directly lead to us being blind to things, not seeing them when they are there, because to ���see��� is to understand, even in a most basic way.
How frequently this mis-match occurs is clearly important. But what strikes me as more worthy of investigation is how different people, at different times, and in different contexts experience, and consequently (re)act / respond, both intellectually and emotionally to all these outcomes, both predicted, and surprising. It also seems significant to look into instances when the distinction between one possible coping strategy and another is unclear, or when it is too clear.
I intend to go into this in future posts, but for now I will stop here and leave you to ponder your experiences in this area. Does my formulation make sense? Is it gibberish? I know for one that it represents at most one half of the experience of development. The other half? That���ll be the subject of a later post: the experience itself of mentally constructing our cloud-castles. But first I���ve al least one cogitation on other aspects to share. Next up, the difference levels of expertise make.
P.S. I���d love to know what you think. If you have any thoughts or any other kind of feedback please reach out to me on Twitter. Thanks!
March 8, 2021
Introduction Part 2: Models, All the Way Down
In part one of this introduction I put forward the idea that the predictions arising from mental models play an integral role in software development. Unlike that post which was intentionally formal, this one derives far more from introspection and observation. I want to use this relative freedom to share some musings.
Specifically I want to think about what these models might be made from, and given that, consider the dynamics which we experience when manipulating these thought-units.
More prosaically, I also want to introduce terminology (which I���ll flag as side-notes) which I hope will make subsequent posts on the experiences arising from all these comprehensible.
In order to speculate like this, I need to go bring in one more collection of concepts which, despite not appearing so at the time, have continued to resonate with me, and feel relevant for us here.
Back in 2016 I was trying to learn algebra. It was long overdue. It was also hard. Really hard. And once again, someone from the Java Posse Roundup came to my aid.
During a conversation about my travails (it probably took place in a queue for coffee) mentioned a book by Barbara Oakley which he���d found useful: ���A Mind for Numbers: How to Excel at Math and Science���.
I approached it with caution. I rarely have the cheek to say out loud how much I distain these ���pop-Psychology��� booksI have a real Psychology degree forchirstssakes!#$%^*!! , but I also frequently give them a try, and that���s what I did here, admitting to myself that anything which could give me even a toe-hold on the seemingly impossible cliff-face of mathematics would be of benefit.
It worked.
Before writing this post, I went back to the book and realised how much it contains. However I only want to bring out two elements here: the limited size of working memory, and a technique to tackle this called ���chunking���.
Working memory first. It���s an enduring concept in cognitive psychology, having survived at least from the days of my degree1993-97 up to the present. It���s a concept of mental functioning usefully conceived of as a workbench. This workbench is where you do your conscious, front-of-mind thinking and decision-making. This ���thinking��� can function simply to keep information available, such as when you are trying to remember a phone number someone just told you, but it can also be more active and combinatorial, such as when you are trying to solve a maths problem.
It doesn���t seem like too big a leap to suggest that working memory is also the locus of active (thinking) work in software development. Whether that���s trying to piece elements together to create our solution, thinking to solve a problem, figuring out which language construct to use, or even remedying the surprise asrising from the aforementioned mis-match between expectation and sensed reality.
Now a key and inescapable fact about working memory is that it has a limited capacity. Oakley characterises it as being capable of containing a finite number of ���chunks��� as she terms them. She makes it clear that to have a ���mind for numbers���And, by implication, a mind tuned for performing any complicated mental-manipulation task you need to be able to work effectively within this restricted space and clever chunk-fu is one way to do it. This ���chunking��� approach entails consciously, coherently and solidly forming shorthand chunk thought-units which can, while only taking up one of those precious chunk-slots, actually represent concepts of significant depth and detail.
Clearly when engaged in complex and conceptual brain work, ���chunks��� are incredibly valuable. But what are they exactly? Oakley offers us more detail:
���Chunks are pieces of information that are bound together through meaning. You can take the letters ���p���, ���o��� and ���p��� and bind them together into one conceptual, easy-to-remember chunk, the word ���pop���. Underneath that simple ���pop��� chunk is a symphony of neurons that have learned to trill in tune with one another. The complex neural activity that ties together our simplifying abstract chunks of thought - wether those pertain to acronyms, ideas, or concepts - are the basis of much of science, literature, and art.���
(A Mind for Numbers, Oakley, pp.53)
She goes on:
���Chunking the information you deal with helps your brain run more efficiently. Once you chunk an idea or a concept, you don���t need to remember all the little, underlying details; you���ve got the main idea���the chunk���and that���s enough.���
(A Mind for Numbers, Oakey, pp.53-54)
I won���t labour the point any further. I hope all this feels super-familiar. Firstly as a description of daily development experience (which may or may not have a basis in neuropsychogical fact) but also secondly as a concept with exciting echoes and resonances for what I shared in the previous post, with regards to Friston���s ideas of mental models and their arising hypotheses.
Given all this, and given the fact that it frequently feels during development as if we are trying to wrestle a number of thought-elements within a finite mental space, how might we use the concepts I���ve lifted from Oakley to gain some insight?
The key question arises: ���what might be the nature of the thought-units from which we build our mental-model castles?��� I propose that these too are mental models; but more specific, more fundamental ones. My supposition is that, in order to fashion higher-order cloud castles, we manipulate smaller blocks of mental model, blocks which represent concepts we already have. It is, I propose, mental models all the way down. And it is these mental model blocks which are the things taking up the chunk-slots in working memory. Let me unpack this a little.
In part one of this introduction I conjectured that when I am developing software I am forming a model in my mind which I am continually conveying to the outside world in the form of code.
In performing this act of creativity I deploy the raw elements of what I know (or more precisely, what I expect) of my chosen programming language. I am using these pre-existing blocks of mental model, which comprise my idea of how the programming language and it���s constructs work ��� let���s call them the ���substrate���This is your first terminology alert models ��� and from them construct my castle ��� let���s call that the ���creation-solution���This is your second terminology alert model.
From this it quickly becomes clear that these concepts of ���substrate��� and ���creation-solution��� model are relative. From one perspective a mental model is the former (i.e. a LinkedList collection is to me a substrate model when I���m flat-mapping my way to a sum), but from another perspective it is the latterWe all know the core libraries we use for example are simply pre-formed building blocks created for us by others (i.e. a creation-solution model when I���m implementing a quantum-safe collections library). This does not only between developers. It also happens within the same individual, depending on the viewpoint we hold at any given point in time during development. For example, one moment I���m creating a component or a microservice, the next moment I���m calling it.
Given that the difference here is only one of perspective, and given my conjecture, both models can be expected to work inside our heads in exactly the same way: embodying a set of expectations of how the real-world (i.e. code) things they represent are structured and will function.
This fits brilliantly into Oakley���s chunks, because chunks are pieces of information formed from smaller-level pieces of information, bound together by meaning. Or put another way, mental models, comprising of smaller-level mental models. We, in short, construct mental models out of other mental models.
It���s models all the way down.
Before I close I must acknowledge the inevitable question: how do we work with (or against) these elements (and perhaps adopt other techniques from Oakley, knowingly or unknowningly) in our acts of development? Having thought about this for some time now, and having spoken to, and watched others working with code, I am convinced that this area is one where a great deal of individual difference lies. Consequently I believe this will be a fruitful one to investigate, compare / contrast, and discuss, in the main body of this effort. My first steps into this form the topic of my next post: ���The Experience of Development - Chapter 1: The Fine Balance���
P.S. I���d love to know what you think about all this. If you have any thoughts or any other kind of feedback please reach out to me on Twitter. Thanks!
March 7, 2021
Introduction Part 1: A Conjecture
This is part one of a two-part introduction. Taken together, my intention is to describe a means of looking at the experience of development. It is my hope that they provide an explanation for how it feels to write code.
This part is more formal - scientific even - but I have no empirical proof, only personal experience and subjective anecdote. Despite this I intend it to be taken seriously, and so have attempted to lay it out in an appropriate style. The second part, in the following post, is on shakier ground, and so is written far more conversationally.
It is my conjecture that the experience of development, that is to say, how it feels to write code, arises entirely from our responses to convergences of, and differences between, two things: on the one hand our mental models of our code, and on the other, our perceptions arising from the code as it physically exists, outside our heads, on computers.
I believe that such a view of our individual and collective experiences with code is a fruitful one, and intend to share both personal insights, and those of others, derived from looking through this lens. However, before I share posts related to the wealth of perspectives and explanations made possible by this thesis, I want to share why I have come to this conclusion; I want to introduce the ideas from an area of neuroscience pioneered by Karl Friston, and I want to travel there via one of the great writers on the task of programming, Fred Brooks:
The programmer, like the poet, works only slightly removed from pure thought-stuff. [They] build castles in the air, from air, creating by exertion of the imagination.
Fred Brooks, The Mythical Man-Month
The fact this decades-old quote still resonates today is because it feels so true to us practitioners. I also intend to take licence from it to talk about the experience of development. There is a problem however; the act of development is conceived by a great many of us practitioners as a highly rational act. Is it possible to look through a lens which allows us to cater for this coldly logical aspect without losing our goal? the goal of describing a way to also address the softer, more unconscious, more esoteric side?
I am going to make such an attempt; an attempt which considers these imagination-castles (less prosaically admittedly) as elaborate, self-consciously-constructed mental models. Might it prove fruitful? Friston���s neuroscience here provides not only a useful perspective on what mental models might actually be, but also brings in the other aspect under consideration, their place in human experience.
The first step is to consider his take on mental models and their function as described by Hohwy in his book ���The Predictive Mind���:
The brain is a sophisticated hypothesis-testing mechanism, which is constantly involved in minimizing the error of its predictions of the sensory input it receives from the world. [���] The hypothesis [is] generated on the basis of our model of the world, and the sensory deliverances coming from the world.
Jakob Hohwy, The Predictive Mind, pp. 1-2
Let���s flip these statements round so they take us in the right direction. Friston���s theory posits that our neural states represent how we ���believe��� the world to be, and that we are constantly comparing the predictions these ���mental��� models make against the sense-data the world provides.Note that here I���m putting more of an experiential spin on things for my purposes. Friston at this point is far more concerned with the neurological.
With this in place we can now bring in a second element from Friston (again summarised by Hohwy) that brings us another step closer to our conjecture. When our expectations fail to match up with the world-as-sensed we experience surprise, and when we are surprised we respond:
[P]rior beliefs [are] harnessed in an internal model to generate predictions of the sensory input and then revise the model���s prediction (hypothesis), or change the input, to minimise the prediction error [���]. This [���] adds [an] all-important predictive element. [���] If all this is done in a reasonably optimal fashion, the resulting hypothesis must then be close to the real world���s state of affairs, at least as signalled in the sensory input: nothing is a surprise.
Jakob Hohwy, The Predictive Mind, pp. 44
The concept of ���surprise��� (or ���surprisal���) here is important. Hohwy describes its function:
Surprisal is a measure of how surprising it would be to observe the system in question in certain conditions, or having a certain sensory input. It is clear that this can only be assessed relative to its normal state, the state we are most likely to find it in.
Jakob Hohwy, The Predictive Mind, pp. 84
Once again we can re-assemble this in terms as they relate to the conjecture. Our internal mental models can be considered as our beliefs of what should be, and the comparison of these model-beliefs with sensory feedback from the as-perceived outside world lead to a psychological response - characterised as degrees of ���surprise��� depending upon how close the model-hypothesis is to the sense-data.
We���re almost there, but these descriptions still feel so cold. Despite what has been presented, a reader could still confidently argue that we have not presented enough evidence of a place for recognisable feelings, for psychological experiences. Hohwy offers us a final step:
The prediction-error minimization idea [���] (Friston 2010) has extreme explanatory scope. [���] Given this [���] it seems reasonable to anticipate, even if only briefly, what it will have to say about further, deeper aspects of mentality.
Jakob Hohwy, The Predictive Mind, pp. 241
He elaborates:
In some sense, emotion arises as a kind of perceptual inference on our own internal states. [���] Emotions then arise as interoceptive [the sense of the internal state of the body, both conscious and non-conscious] prediction error is explained away. [���] We should expect that sometimes it is our expectations for arousal or interoceptive precision that determine where we end up emotionally.
Jakob Hohwy, The Predictive Mind, pp. 242-3
Let us bring this one last time into our sphere. Hohwy proposesI think with some justification that the consideration of the low-level neurological drive to minimise prediction-error in our mental models, as manifested by the surprise arising, can be taken broadly and explains many human phenomena. One of these is the wholly-internal concept of feelings, of emotion, specifically as means to ���explain away��� this generic ���surprise��� which arises from experiencing prediction errors.
We now have enough to summarise: if Brooks is right, when we develop software, we construct mental castles, or more matter-of-factly, elaborate mental models. These castles are represented neurologically as a set of predictions of how they will manifest in the outside world. In accepting this, we also acknowledge that when castles are expressed ��� conveyed somehow ��� to the outside world and into code - that realm outside our minds - they must compile, the tests must pass, the executables must run, and they must achieve the goals we���ve set for them, in as many circumstances as we can conceive. They must, in short, meet our model���s predictions; they must operate as we expect them to.
It is my conjecture therefore that the mental models we have created and subsequently mediated to the outside world in the form of code must match our reality because we expect them to; and when they don���t, we are surprised and it is our individual, nuanced, situation-dependent grappling with this surprise which gives rise to our equally individual experiences of development.
In my next post, ���The Experience of Development - Introduction Part 2: Models, All the Way Down��� I introduce the other important element we need in place before we look through this lens: a deeper consideration of the nature of our mental models; of what they are composed, and how we manipulate these thought-units.
P.S. I���d love to know what you think. If you have any thoughts or any other kind of feedback please reach out to me on Twitter. Thanks!
March 6, 2021
Preface
Image: Mt Crested Butte, Colorado, USA, taken by the author
I first thought about the experience of development in February 2014, shortly after watching Joe Sondow lead an afternoon hackathon session at the then-Java Posse Roundup Open Spaces in Crested Butte, Colorado.
Joe was taking us through the Twitter Bootstrap framework. When we all started, Joe with his MacBook plugged into Bruce Eckel���s flat-screen, the rest of us watching from the various sofas and easy-chairs dotted around the room, none of us knew much about it.
As he went about his process of deciphering how Bootstrap was constructed and how it worked, Joe narrated his thought process. Some of us pitched in now and then with suggestions, offering tips when he���d missed something, but largely we were sitting there, inside Joe���s head, seeing his IDE as if through his eyes. I was transfixed. There were so many differences from how I approached the act of development; so many differences from how I experienced it.
The conscious realisation of this only struck me later. While it had been unfolding in front of my eyes I had simply wondered at how Joe reasoned about code; how he had a command of the substrate parts - raw JavaScript and CSS, as well as his tools and what he could rely on them for; how he expressed his mental state as code in his IDE (and equally how he commented things out to see if the extent of what he thought he���d turned off was met by reality).
It was probably a day later (most likely prompted by yet another session which made me realise how massively different my mental conception of things were from others) that questions began to arise. Questions such as ���is there a right way to think about the act of development?���, ���how many ways do people think about development?��� and the one which stuck with me far more than any other: ���what is it like when people experience the act of development?���
Over the years that followed this question never went away. I tried to grasp it myself at first, and kept an Evernote notebook full of glipses of people using emotive language with reference to code. I then found Kent Beck���s awe-inspiring preafce to his classic ���Test Driven Development: By Example���Someone needs to draw attention to the great prefaces and afterwords of Software books. There are a lot out there, and we miss out when we skip past them to get to the ���real book��� and I could relate to it completely. He talked about discomfort, unease, fear, pleasure, and joy. But he was largely alone. Year on year I kept wondering if this was really something to investigate further, but I could never figure out how. I couldn���t even figure out how to make sense of my own feelings on the subject.
But slowly things began to come together. Once again the fecund minds of the attendees at the Java Posse Roundup (now re-styled the ���Winter Tech Forum���) was the source of virtually all my inspiration. Week after week came the externalised thought processes of people thinking out loud on the #code slack channel (shout outs to Jack, Chris, Joey, Harrison, Jeremy, DJ, Wesley, Gordon, Justin, Tom, Marshall, Drew, J, Chris and Matt). Then there was Bruce and James��� in-depth discussions on the act of language design and writing for those of ���beginners mind��� on the ���Happy Path Programming��� podcast.
But best of all was the interviews that various attendees (Chris, Wesley, DJ, Dmitry, Harrison and Megan) allowed me to stumble through, asking them about their experiences of writing code. I learned so much yet again. I had raw data points all over the place, and data points that were crisp and clear too - I am blessed that everyone I���ve met out at that self-organised conference in a church hall deep in the Colorado Rockies is so eloquent and self-aware.
However , there was still something missing. I didn���t feel that confidence which I thought I���d feel before I took this to the world. To conquer my doubt it needed something to bring it together. I needed a lens to look through. I needed a hypothesis to test. I needed a something like a theory which to interrogate, to prove or invalidate.
Then one day, it all suddenly came together. I was catching up on the #code channel where Jack was describing a problem in Go, or Rust, or something like that. That wasn���t in itself unusual; it���s what the #code channel is for; but how he described his experience resonated like a bell. What he articulated was his sense of surprise when how he���d expected something to work had been very different from how it actually worked. In that instant all the elements which had up to then been sloshing around my mind suddenly tessellated.
The unifying element, the structure around which all this formed, had been explained to me years before in Denver Airport by Julie ���yakticus��� Amundsen, was Karl Friston���s idea of our mental models serving to predict the world, our experience of ���surprise��� when these model���s predictions fail, and the various ways we respond to reduce the impact of this experience. Yet again, I���d had the concept in front of me for years, and yet again it had come from conversations with a Roundup attendee.
The posts which follow this one are the result of this realisation. My hope is that not only will they be coherent, but also useful, and perhaps even enjoyable. The next two posts form the introduction: the first of these (Introduction Part 1: A Conjecture) goes into my idea in far more detail. The second (Introduction Part 2: Models, All the Way Down) adds some related thoughts I���ve had, which, though less formal, still form an important background for the investigations, hypothesis testings, and resulting musings which follow, the first of which takes the concepts from the introduction and attempts to form with them a broad categorisation of the types of development experience, both based on personal, and non-personal reports and observations.
P.S. I���d love to know what you think. If you have any thoughts or any other kind of feedback please reach out to me on Twitter. Thanks!
