Eric S. Raymond's Blog, page 12

Sneaky sifu tricks

My wing chun sifu played a classic martial-arts-master trick on me tonight.

We’re having a test. I’m not testing, not in my mind. I do the expected things, which in my case includes being a sparring opponent for whatever senior students are testing. Because I’m strong and aggressive, skilled with knife and long blade as well as empty hand, sifu likes to throw me at people to challenge them and elicit best performance. I’m OK with that.

The most interesting fight is knives with Lee, a teenager testing for 9th level. I’m 8th, nominally at the same skill level he is going in. But Lee is still a kid who sometimes falls off his mental focus and gets frustrated. He has speed, height, and reach on me, but I have more brute strength, more knife experience, and a better developed warrior mind (I’ve been training since before Lee was born). Usually this means the advantage is mine.

Tonight, though, Lee does a smart thing. He processes the fact that (more or less as usual) I’m chopping him into chutney at medium range. Instead of losing his focus or retreating he closes and grabs me with his off hand. This is so unlike previous performance that he gets inside my OODA loop – by the time I counter he has scored several solid strikes.

This is good; Lee’s tactical repertoire is broadening and he figured a way to use his superior speed against me effectively. Sifu deserves to know this because it bears directly on Lee’s growth as a student (yes, I think like an instructor, which not news to anyone at the kwoon) but I’m not sure he could see it; I was directly between sifu and Lee when Lee busted his move.

So when the test is over I go to the table sifu is sitting behind, all set to tell him how well Lee did. Before I can open my mouth he looks at me and announces “You’re a level 9 now”.

*blink*

A few minutes later I find out that everybody knew I was being tested except me. Sifu had told them what he intended to do the previous week when I had to skip a class for work-related reasons.

Well, no fscking wonder he’s been vague about when he intended me to test next. It’s a classic trick – test a student when he doesn’t know he’s under evaluation. I’d do it to either (a) spot a student I suspected was being lazy if I know what his peak performance is like, or (b) reduce the pressure on a student who I think might be prone to choke in a formal test. I don’t think sifu is likely to put me in either of those bins, so he probably had some subtler reason in mind. He’ll tell me if he wants me to know.

I’m now the highest-ranking student at the school who isn’t a Black Sash (beginning master, like a 1st Dan in Japanese systems) or higher (we have one Red Sash, 2nd-dan equivalent). I guess I can expect to qualify for my sash within six months or so. Not bad for a 60-year old guy with spastic palsy! I feel I have some reason to be proud.

UPSes compared to guard dogs

Somebody using the handle JeffyPoooh (yes, ‘ooo’) left a comment on The Register that is an only slightly exaggerated summary of the reasons for UPSide:

You’re concerned about your family’s safety. So you get a guard dog. The dog costs a fortune. It immediately poops on the floor. Then it chews off the entire left side of your Bang and Olufson. It bites the postman’s fingers. It then sleeps through an actual burglary. And finally it eats one of your children.

This is the UPS experience: If they’re not preoccupied with smoldering their lead acid batteries, then they’re busy buzzing and arcing. Then they blow an internal fuse on the output, and your Great American Novel is suddenly lost, again, for the third time. Then there’s an actually power failure (Yay!), so they turn on their patented 387 volt offset square wave, and your PC is instantly corrupted. Meanwhile battery acid squirts out onto the ceiling, again. Then, while you’re out trying to buy a replacement PC, the UPS catches fire and burns your house down.

I’d happily pay $800 to not have one.

How to get started on the UPSide project

The current state of play is: We have a high-level system design and a map of the behavior states. We have a capacity target (300W for 15 mins) and a peak-continuous-load spec (400W) We know we’re going to build a double-conversion design and we’re considering a couple of alternative topologies. We pretty much know the external-interface specs (some details may change).

I’m expecting both my prototype copy of the forebrain Unix SBC (an Olimex LIME2) and the interface contract for the high-power subsystem to land on my desk tomorrow.

Interest in this project continues to be huge. Another company wants in as of this morning. The volume of feature requests is high enough that I’m buckling under the editing load.

The rest of this post is instructions to potential contributors about how to get on board.

1, Get an ID on GitLab. Tell me what it is so I can add you to the project group.

2. If you have a feature request, please Don’t post it on this blog. Add it to the “General feature request thread” on the tracker.

3. Read the wiki. Read the tracker issues. I try to keep both pruned so the volume is not overwhelming. Read the Rejected Ideas page on the wiki, too.

4. Read the design documents in the project wiki. The important one is the transaction design; the I2C message inventory will change, but the basic state diagram probably won’t.

5. Participate in the design discussion. This takes place in tracker threads.

6. When we’re ready to breadboard a prototype, throw some parts money in the tip jar we don’t have yet. If you must contribute before then the PayPal blogbutton works fine.

7. Prototype builds will probably go down at PA Makerspace in Phoenixville, PA. If you are within driving distance and a competent electrics tech, consider joining us for a build.

8. Once we have a full design with a PC board and enclosure: if you have a shop facilities for it, try to replicate the build. We’ll know we have the build recipe debugged when other people can do it.

9. If your favorite hardware feature request doesn’t appear in the version 1 prototype, relax, We may think it’s a good idea but be holding off till v2 out of a desire to keep v1 simple and launch fast.

10. If your favorite software feature request doesn’t appear in the version 1 prototype, pitch in and make it happen. A Unix SBC is not a difficult programming environment – the OS on this one is a Debian port.

After step 10 and a couple of design iterations the future becomes less clear. maybe try to get it into volume manufacturing through a partnership with an established vendor.

Stop logging in local time!

Inertia is a powerful force. The computing world retains a lot of practices that are odd little dysfunctional relics of past stages of its technology. The one I’m here to talk about today looks like this:

Mar 6 15:11:07 snark postfix/qmgr[3927]: 0422513A6C53: removed

That’s a log message hot’n’fresh from my /var/log/mail.log file. It’s entirely typical of traditional log formats on Unix systems, and these things offend the bejeezus out of me every time I see them. Now let me show you how this would look in a sane universe:

2018-03-06T15:11:07Z snark.thyrsus.com postfix/qmgr[3927]: 0422513A6C53: removed

Logging events in local time (and with only local hostnames, but that’s not the subject of today’s rant) is a dysfunctional remnant of the time before wide-area networks. It means that log timestamps aren’t directly comparable across hosts in different time zones. This is death on diagnostics for a large class of network-transaction bugs.

Actually it can mean a lot worse than that, even locally. An A&D regular who wishes to remain nameless recently told me of once having to help troubleshoot a medical-records system at a major hospital. It was unusable – they had to plan around this and retreat to a paper backup system – for two hours a year. Those two hours were just adjacent to daylight-saving-time changes. Yes, that’s right, stamp collisions due to logging in local time crashed their multi-megabuck investment.

Another place logging and displaying in local time is a bad mistake is in distributed version-control systems. I’ve never seen a case in which it was not more important to know the relative time of a sequence of commits than to know the, er, “absolute” local time of any of them. And, of course, committers may be scattered across multiple timezones. Thus, the way to reduce cognitive friction on people browsing the history is to refer all commit timestamps to common timebase.

Yes, git does get this wrong. Git timestamps are stored in UTC but displayed in local time in git log and elsewhere. To accomplish this git has to keep a local time zone offset with each date, a pointless “feature” that often causes me chronic problems too tedious to explain when I do repository conversions.

You know who got this right first? Military and civil aviation. Long before Internet traffic routinely crossed timezones, airplanes did. Requiring pilots and ground controllers to constantly track everybody’s timezones and do conversions on the fly would be confusing and dangerous, so…Zulu time for everybody. Loss-of-life risk is lower where we play (except maybe at that hospital?) but the underlying logic for ditching local time is the same.

EDIT: It has been pointed out to me that radio and telegraph operators faced similar situational stresses as far back as the mid-1800s. It’s not clear, however, how soon GMT became on-air standard time after it was formalized in 1847.

So next time you have to choose a time stamp format, cut the crap and go straight to RFC3339 (with the T in the middle, thank you). It has many advantages. It’s unambiguous, compact, compares correctly no matter where it came from, constant length, sorts lexicographically the same as its time order, and parses out of text as a single token that is easy to distinguish from anything but an RC3339 timestamp (that’s why you want to leave the T in the middle).

(But in case you’re tempted to think about Zulu for all purposes…bad idea. See also In defense of calendrical irregularity.)

And now, to conclude this public-service announcement, a filk I composed for the occasion. Take the tune from this and superimpose these lyrics:

Baby you'll come knockin' on my firewall
Just as I'm dealin' with a system stall
I said yeah, well, what'm I supposed to do?
I don't need no cracker gettin' in too.

NANOG says they have some trouble in town
Now you're shutting some daemons down

Stop logging in...
Stop logging in...
Stop logging in local time.

It's hard to know where the intruders came from
It's hard to know just what we've lost
This doesn't have to be the big net meltdown
This doesn't have to be anything at all.

I know you really want to tell me good-bye
I know you really want to run your own show

Baby you could never look me in the eye
Yeah you buckle with the weight of the words

Stop logging in...
Stop logging in...
Stop logging in local time.

There's people running 'round loose in the world
Ain't got nothing better to do
Than make a meal of some P.F.Y.
You need someone looking after you

I know you really want to tell me good-bye
I know you really want to run your own show

Baby you could never look me in the eye
Yeah you buckle with the weight of the words

Stop logging in...
Stop logging in...
Stop logging in local time.

Stop logging in local time!

All of his complexion…

Andrew Klavan has a thoughtful essay out called A Nation of Iagos. In it, he comments on William Shakespeare’s depiction of Jews in a way I think is general insightful, but includes what I think is one serious mistake about the scene from The Merchant of Venice in which the (black) Prince of Morocco woos Portia.

He chooses poorly, fails Shylock’s test, and as he leaves Portia mutters “May all of his complexion choose me so.”, which Klavan reads as a racist dismissal. I winced.

I tried to leave a comment on the essay only to find when clicking “Post” that it required a login on the accursed Facebook, with which I will have no truck.

Here it is:

Others mentioned it before I got here, but I want to reinforce the point that Klavan’s reading of Portia’s muttered remark is unsubtle – and, I think, wrong.

I read that and I think “Bill, you magnificent bastard!” The Bard of Avon has constructed that whole scene to make it clear that the Prince’s plea not to be judged by his race is not falling on an unwilling ear. Portia is a good person. When she finds him wanting, we are ready to hear Portia reject him not for the color of his skin but for the content of his swaggering, overbearing character. It’s both the logical and dramatically correct conclusion of the scene.

Instead she…drops a racist clanger? No. No. Shakespeare is subtler than that. He uses “complexion” in a way that holds the audience in tension between what one night call its physical and psychological meanings [ed: in Elizabethan English “complexion” could mean a person’s character or psychological presentation]. Burton Dow is quite right to point out that both meanings were live in Shakespeare’s English. This ambiguity cannot be an accidental choice, not from a wordsmith with the Bard’s sensitivity to nuances of vocabulary.

Yes, Shakespeare is prodding his audience. He’s challenging their language and their prejudices, not in the angry evangelistic way of a modern SJW but in the way a man of what fifty years later would begin to be called a liberal inclination would hold up a wry mirror to the tragedy and comedy of human life.

Why the Culture books were bad SF

There’s a lot of buzz about Iain Banks’s Culture universe lately, what with Elon Musk naming his drone ships in Banksian style and a TV series in the works.

I enjoyed the Culture books too, but they were a guilty pleasure for me because in a fundamental way they are bad SF.

They’re bad SF because the Culture’s economics is impossible. That ship hits a rock called “Hayek’s Calculation Problem” and sinks – even superintelligent Minds can’t make central planning work, because without price signals and elicited preferences you can’t know where to allocate resources. What you get is accelerating malinvestment to collapse.

This is what happened to the Soviet Union in the late 1980s. Hayek predicted it fifty years in advance. Huge factories in Siberia destroyed wealth by producing trucks nobody needed from resources that would have been better spent on other things – but nobody could know that because there weren’t any price signals. Eventually the SU wore out its pre-Communist infrastructure, fell down, went boom.

The problem is epistemic and fundamental – can’t be solved by good intentions or piling on computational capacity. An SF writer is every bit as obligated to know what won’t work in economics as he is not to make elementary blunders about chemistry and physics. The concept of “deadweight loss” matters as much as “entropy”.

Banks’s lifelong friend and fellow Trotskyite Ken McLeod actually managed not to flunk this. In a long and revealing interview about the genesis of one of his early series (the “October Revolution” books IIRC) he once revealed that for years he read free-market economics on the know-your-enemy principle, then woke up one day realizing he couldn’t refute them. Subsequently his books took a decidedly libertarian turn. This demonstrates that Marxists can clean up their shit; alas, Banks never made it that far.

The Culture books also implied, though they never explicitly committed, a different fallacy. I often run into it in talking with people who want to defend the plausibility of the Culture. It’s the belief that superabundance is just a matter of being smart enough, that there are no fundamentally scarce resources.

Alas, no. What kills that dream is thermodynamic inefficiency of conversion. Even at the extreme where you have both nanotech and elemental transmutation, your limit will be the capacity to dissipate waste heat from your fabrication engines.

Banks wasn’t alone in this one. Lots of inferior SF writers make this mistake about nanotech by itself. They write as though it’s fairy dust you can sprinkle on scarcity problems and make them go away. Nope, nope, nope. Where’s the energy to drive your fabricators coming from? What about your feedstocks?

(And if you don’t understand those questions…what the hell are you doing writing SF? In that case all you’re actually good for is crapping out a Harlequin or something – stop bothering the adults.)

SF done right needs to understand and respect these limits. If you’re going to bust any of them, they count against the traditional one-McGuffin quota (you know: one strong counterfactual per story, but FTL doesn’t count) and you need to justify it to your readers lest you have a plausibility hole in your universe that destroys any possibility of suspension of disbelief.

EDIT: I should also have mentioned scarcity of human attention. Try to eliminate that and see where it takes you…

upside wants a firmware dev

The UPSide project, announced here two weeks ago, has come together with amazing speed.

We now have:

* A hardware lead – A&D regular Eric Baskin – with thirty years of experience as a power and signals engineer. He is so superbly qualified for this gig that my grin when I think about it makes my face hurt.

* A high-level system design (about which more below) that promises to be extremely capable, scalable, flexible, and debuggable.

* A really sharp dev group. Half a dozen experts have shown up to help spec this thing. critique te design docs, and explain EE things to ignorant me.

* Industry participation! We have a friendly observer who’s the lead software architect for one of the major UPS vendors.

* A makerspace near me where the owner recruited himself onto the project and is looking forward to donating bench time and skilled hands to the hardware build.

All this helpfulness almost – but not quite – fills in my deficits as a designer/implementer. I don’t really know from hardware design, so I’m attacking the problem with the modularity and information-hiding principles I know from software.

Here is how the design looks:

An I2C bus that ties together a “forebrain” which is a Unix SBC, almost certainly at this point an Olimrx LIME2, with a “midbrain” that is an Arduino-class microcontroller.

The midbrain is mechanism – a simple state machine whose job is to control the high-power subsystem (inverters, battery, AC input and output). Policy decisions and stuff like battery state modeling will live in the forebrain. It will also run the USB and Ethernet interfaces, and host the development environment for the firmware

The forebrain will talk to a 20×4 LCD panel over I2C, and various other controls like alarm mute and self-test buttons via GPIO pins.

I’ve actually written the spec for the I2C bus messages already. And here’s your cute hack for the day…

I realized early on that one of the first things I needed to do was draw a state/action diagram for the system so I could pin down its behavior in response to any given transition in its environment (mains power up, mains power down, battery dwell limit approaching, those sorts of things). So I reached for one of my favorite tools, a graph-drawing DSL called dot.

Only when I write the first version of the graph, I found the dot markup cluttered and repetitive. So I wrote a couple of cpp macros named “state” and “action” that expand to dot markup, and expressed the graph as a sequence of macro calls.

Then I blinked, looked again, and realized…hey, I could compile these calls to C source code for a state machine! And now it is done – I can already generate the tricky part of the application logic for the midbrain directly from the state/action diagram. (The action functions are stubs but the control flow is all there.)

(If the fact that I just solved a design problem by writing a DSL to generate code in another DSL and provably correct equivalent C application logic seems weird to you, you must be be new here. This is how I think all the time. It is obedience to the Unix wisdom: never hand-hack code you can generate from a higher-level description.)

This, however, does not solve the entire firmware problem by any means. The midbrain’s going to need system logic to do things like receive and send I2C messages, poll A2D converters from sensors watching the mains and battery voltage, and so forth.

Accordingly, we need a firmware developer. I’ll learn how to do this if nobody steps up (which is why I said “wants” in the post title) but the whole process will doubtless go faster and more smoothly if we have someone with experience. So:

WANTED: One firmware hacker. Must be familiar with AVR-class microcontrollers and the Linux toolchains for them. Experience with I2C and low-level programming of USB endpoints would be a plus. Perks of the job include getting one of the first UPSides made, your name in lights, and working with a dev crew that is impressive even by my elevated standards.

EDIT: Well, that didn’t take long. A&D regular Jay Maynard has signed on.

How elites are blind about immigration

I had been thinking about posting about immigration recently, because some facts on the ground have caused me to move away from a pure laissez-faire position on it. A few minutes ago I wrote a long comment on G+ that I realized says a lot of what I wanted to. This is a slightly revised and expanded version of that comment.

I am asked, by another member of the educated white elite, why we shouldn’t simply end border enforcement entirely rather than buid a wall or tolerate Joe Arpaio’s squalid detention camps.

Both here and in Europe there’s been a significant spike in communicable diseases that can be traced back to low immunization rates in what Trump may or may not have called “shithole” countries.

Crime is a real issue. Legal immigrants have a slightly higher criminal propensity than the native born (the difference is small enough that its significance is disputed) but illegals’ propensity is much higher, to the point that 22% of all incarcerees are illegals (that’s 92% of all jailed immigrants).

But the elephant in the room is the impact of illegal immigration on social trust.

Diversity erodes social trust, trust being that extremely valuable form of social capital that enables people to make handshake deals, leave their doors unlocked, and trust institutions to treat them fairly. Sociologist Robert Putnam was so shocked to discover this that he sat on his results for seven years before publishing. In diverse communities trust drops not only between ethnolinguistic groups but within them. It’s insidious and very harmful – low-trust societies are bad, bad places to live.

The U.S. has a proud tradition of assimilating legal immigrants into a high-trust society, but it succeeds in this by making them non-diverse – teaching them to assimilate folk values and blend in. Putnam’s work suggests strongly that without the ability to rate-limit immigration to be within some as yet undetermined maximum, the harm from erosion of trust would exceed the benefits of immigration.

We are probably above the optimal legal immigration rate – the highest compatible with avoiding net decrease in social trust over time – already (later in this post it should become obvious why I believe this). There is little doubt that we would greatly exceed it without immigration controls.

Anyway, even if ending border enforcement were a good idea (and I conclude that it is not, despite my libertarian reflexes) it’s a political nonstarter in the U.S. Trump got elected by appealing to sentiment against illegals, and beneath that is a phenomenon one might call Putnam backlash; everywhere outside a few blue-state enclaves, Americans sense the erosion of social trust and have connected it to illegal immigration.

If you run around saying “We should end border enforcement”, enough people to form a blocking coalition are going to hear that as “He wants the U.S. to sit on its hands as erosion of social trust degrades it into a shithole.” Of course most of them don’t have this intellectually analyzed – it’s a more a gut feeling. But no less powerful for that, especially since the problem is real.

Do you want more Trump? Because that is how you get more Trump – or possibly someone worse. I don’t think there is actually a large cohort of Americans willing to sign on to full-throated 19th-century-style nativism yet, and I’m glad of that. But that’s where the next turn of the screw takes us.

We can only save the positive benefits of immigration by controlling it. And by growing some freaking humility about our biases. It’s easy for elite whites like you and me to see only the upside of immigration (cool restaurants, interesting music, exotically pretty girls, lower price levels due to labor cost push on the things we buy, getting to feel virtuous about our inclusivity); immigration seldom has any obvious downside for us unless we roll snake-eyes and get killed by MS-13 or something.

We tend to miss the fact that if you’re a native-born unskilled laborer or minority or legal immigrant the cost-benefit ratio looks very different and not favorable at all. Loose labor markets are good to us, but sure as hell not to our poorer compatriots. A little more compassion and a little less class-blindness on our part would be an improvement.

(My comment ended here. Had I continued addressing my interlocutor directly I would have added the following…)

One of the major forces currently poisoning our politics is a breakdown in trust between people like you and me – the cognitive elites – and the rest of America. Deplorables. Flyover country. Brexit, and Trump’s election, slapped me upside the head. I’ve been forced to confront some uncomfortable truths.

They think we’ve betrayed and abandoned them for a mess of virtue signaling and glib ideologizing. On the left: identity politics, PC, and open borders justified on multiculturalist grounds. On the right: free trade and open borders justified on laissez-faire principle.

They have a point. I’m seeing that now.

I mean, I might still think free trade is a good idea and have lots of arguments for it. But my arguments don’t mean fuck-all to a Rust-Belt steelworker who’s watched his livelihood get exported and the community around him wither and has nothing left but a cheap high on opioids. Nor to an unskilled black or legal-immigrant urbanite who can’t get a job because the restaurants can hire illegals for cheaper.

We owe these people more than we have given them. What we owe can’t mainly be paid in money. It’s compassion; a fair hearing. Respect. Not dismissing them as trash or troglodytes or racists because they don’t love the brave new globalized world that gives us options but – too often – closes off theirs.

I don’t have easy solutions to these problems. But is it too much to ask that people like you and me should stop being arrogant assholes about them?

UPDATE: I’m sure I’ll be asked how I reconcile border controls with my libertarian principles. It’s a fair question – before Putnam I wouldn’t have tried, or even wanted to. Now I think in these terms: regardless of how you feel about government, high trust is a valuable kind of property for a society to have, and an ethically correct thing for it to defend.

If you blow up the Constitution, you’ll regret it

Predictably, the Stoneman Douglas High School shooting has triggered some talk on the left – and in the mainstream media, but I repeat myself – of repealing the Second Amendment.

I am therefore resharing a blog post I wrote some time back on why repealing 2A would not abolish the right to bear arms, only open the way to the U.S. government massively violating that right. Rights are not granted by the Constitution, they are recognized by it. This is black-letter law.

Thus, repeal of any right enumerated in the Constitution is not possible without abrogating the Constitutional covenant – destroying the legal and moral foundations of our system. The ten in the Bill of Rights are especially tripwires on an explosive that would bring the whole thing down. And of all these, the First and Second are especially sensitive. Approach them at your peril.

I will now add a very sober and practical warning: If the Constitution is abrogated by a “repeal” of 2A, it will be revolution time – millions of armed Americans will regard it as their moral duty to rise up and kill those who threw it in the trash. I will be one of them.

Left-liberals, you do not want this. I’m a tolerant libertarian, but many of the revolutionaries I’d be fighting alongside would be simpler and harder men, full of faith and hatred. If that revolution comes, you will lose and the political aftermath is likely to be dominated by people so right-wing that I myself would fear for the outcome.

You should fear it much more than I. Back away from those tripwires; you are risking doom. Ethnic cleansing? Theocracy? Anti-LGBT pogroms? Systematic extermination of cultural Marxists? In a peaceful, Constitutional America these horrors will not be. If you blow up the Constitution, they might.

In the face of uncertainty, buy options.

Yesterday I posted about how the streetlight effect pulls us towards bad choices in systems engineering. Today I’m going to discuss a different angle on the same class of challenges, one which focuses less on cognitive bias and more on game theory and risk management.

In the face of uncertainty, buy options. This is a good rule whether you’re doing whole-system design, playing boardgames, or deciding whether and when to carry a gun.

A useful way to sort the decision challenges we face is into situations of high uncertainty versus low uncertainty. These call for vary different adaptations. In a situation of low uncertainty there is a single optimal choice; your effort should go into determining what it is and then executing it as hard and fast as possible. Unless uncertainty rises during execution (for example because you discover you made a serious mistake in your problem analysis), deviation from plan is most likely to be a mistake. Buying options is wasteful.

In a situation of high uncertainty you don’t know what your best choice is up front; there’s a broad range of possible ones that might be optimal, and there may be choices you can’t yet see. In this situation, what you need to do is enable yourself to collect on as many of the options as you can identify and afford to buy. Your hope is to be able to narrow the range of conditions you need to cope with as you learn more.

This dichotomy is so fundamental that it has moral consequences. If, in any role other than military in a war zone, you are ever carrying a gun because you have a have a high-certainty expectation that you will use it against humans, your life choices have probably gone very badly wrong. On the other hand, carrying a gun as a hedge against uncertainty – for example, if you need to visit someone in a dicey enough part of town that you might have to defend yourself – makes both practical and moral sense.

In yesterday’s example, I described using a Unix SBC rather than an Arduino-class microcontroller as a way to counter-bias against our tendency to underestimate and underweight software-development costs we can’t estimate crisply. It is that; it is also a way to buy options as a hedge against uncertainty about what how the whole system should behave. Often we don’t actually know this until field testing. When we get it wrong, the pain of correcting is in direct proportion to the cost of changing the whole system’s behavior; it can be lowered if the controller is chosen for flexibility and low development costs.

This is how “overkill” can save your butt. Suppose you’re right that an Arduino-class chip is sufficient for every scenario you imagine in your planning phase; it may still be the case that you’ll be mugged by a field reality you didn’t anticipate. If you don’t have good judgement about how to hedge against this possibility, your designs will have more than your share of expensive failures.

The hardest part of this lesson to learn is that an early choice to buy hedges against uncertainty does not retrospectively turn into a mistake if you get lucky and everything goes as originally planned. You have to sum over all possible worlds; whether the choice to carry a gun or deploy a more flexible controller was wise depends only on the accuracy of your ex ante risk evaluation, not on whether you actually got attacked or the firmware Arduino-class controller is actually good enough on the first spin.

(Yes, I know. Firearms ethics and systems engineering in the same blog post. If you don’t find this amusing, go read something stupid.)