Announcing a new book: “The Programmer’s Way”

A decade and change after “The Art of Unix Programming”, I’ve decided to do another book. Actually, I have more than just an intention and some notes; I’ve been working hard on it over the last five days it and have 41 Kwords of rough-cut manuscript ready.

No, I didn’t write all that from scratch in five days! I’m a pretty fast writer but I’m not that fast. What I’ve done is comb through everything I’ve written about software engineering and its practice since around 2003, throw it together into an anthology format, and begun adding updates and bridging material.

It’s grouped into episodes under thematic parts. Long-time A&D regulars will recognize many of the episodes. Other parts are IRC transcripts, a paper on engineering zero-defect software I wrote in 2012 that needed an update, things I wrote for the NTPsec blog, and so forth. For each episode I’m taking time to reflect on lessons learned since I wrote the material it was based on and folding in those lessons.

The book has a theme: like “The Art of Unix Programming” it’s about the importance of right mindset. Knowing tools and techniques only gets you to the threshold of expertise. At higher levels it’s about a way of engaging the world, of perceiving, of carving problems at their joints. The material is less focused on the Unix tradition, though, because I already wrote that book. This one has a broader reach.

I’m experimenting with new ways to promote and monetize the work. Portions of the draft in progress will be posted on Patreon; you can become a Bronze supporter to get early access. I shipped the first episode there this morning.

I haven’t shopped for a publisher yet. I don’t expect to have any trouble finding one, as my previous books have all been exceptional sellers for their categories and had long legs to boot – my habit of actually writing most of a book before I sell it helps, too. But I might go indie this time – we’ll see, I consider all options to be open. Not having a contract means I don’t have to be in a hurry; it will ship when it’s right, not before.

Here are the present section intros. These are subject to change.

The Rectification of Names

I like to find common tactics and traps in programming that don’t have names and name them. I don’t only do this because it’s fun. When you have named a thing you give your brain permission to reason about it as a conceptual unit. Bad jargon obfuscates, map hiding territory; good jargon reveals, aiding reflection on and and improvement of your practice.

Thus, the episodes in this section are exercises in noticing features of the challenges in software engineering and giving them useful names.

Chop Wood; Draw Water

Zen Buddhism has a tradition of teaching stories in which an eager novice beseeches a master for the way to enlightenment, only to be confused when the master directs him to “Chop wood; draw water” or some other similarly mundane thing.

The master knows something the student doesn’t. Right mindset, in Zen or programming, is best and most deeply achieved when it comes out of practice rather than lofty abstractions.

This collection of episodes is about practice.

Theoria from Poesis

Engineers may learn their most enduring lessons from what Aristotle called “poiesis” (making), but chopping wood and drawing water as a way of learning has its limits, too.

Western intellectualism has a abstraction-centric bias that tends to put “pure” theory before practice. But at its best, Aristotelian “theoria” works in the other direction; it compresses regularities observed in large ranges of examples into generative principles that are easier to communicate.

This collection of episodes is about principles.

The Rise and Decline of Programming Languages

Whole-systems engineering, when you get good at it, goes beyond being entirely or even mostly about technical optimizations. Every artifact we make is situated in a context of human action that widens out to the economics of its use, the sociology of its users, and the entirety of what Austrian economists call “praxeology”, the science of purposeful human behavior under scarcity, cost, and limited-information constraints.

This isn’t just abstract theory for me. When I wrote my papers on open-source development, they were exactly praxeology – they weren’t about any specific software technology or objective but about the context of human action within which technology is worked. An increase in praxeological understanding of technology can reframe it, leading to tremendous increases in human productivity and satisfaction, not so much because of changes in our tools but because of changes in the way we grasp them.

I’m also fascinated by the challenges of programming-language design. The first four episodes in this section came out of my attempt to understand long-term tends in the rise and fall of computer languages in a praxeological way. The fifth, on the waning of Python, is a report from the trenches on what I had to do to cope with the kind of problem those previous essays were about.

The Dialogues

Programming is not a skill readily taught in a classroom setting – not above its lowest and most mechanical level, anyway. What our schools produce is all too often incompetence that is incompletely remedied by field experience.

The reason goes straight back to the duality of tools and mindset. While use of tools can in fact be learned in a classroom setting, teaching right mindset is a great deal more difficult. I’ve known a handful of gifted teachers who made a respectable effort, but they were the exception rather than the rule.

The mindset part is best taught by apprenticeship – that is, a close relationship between an individual teacher and an individual student in which the student practices imitating the skills and mental habits of the master until that careful memesis induces a near-replica of the master’s mindset to form in the student.

These episodes are polished from communications with people who were formally or occasionally apprenticed to me. The resemblance to Zen stories or Socratic dialogues is intentional; these are powerful, time-tested models of teaching right mindset that I adopted quite consciously and seem to have served well.

Because the community of commenters around A&D has influenced and in some cases motivated the development of this material, I’m opening the comment thread to suggestions of what else I should write about.