Eric S. Raymond's Blog, page 16

Fuzzbombing: abort() calls for great justice!

The Colossal Cave Adventure restoration is pretty much done now. One thing we’re still working on is getting test coverage of the last few corners in the code. Because when you’re up to 99.7% the temptation to push for that last 0.3% is really strong even if the attempt is objectively fairly pointless.

What’s more interesting is the technique one of our guys came up with for getting us above about 85% coverage. After that point it started to get quite difficult to hand-craft test logs to go to the places in the code that still hadn’t been exercised.

But NHorus, aka Petr Vorpaev, is expert at fuzz testing; we’ve been using American Fuzzy Lop, a well-designed, well-documented, and tasteful tool that I highly recommend. And he had an idea.

Want to get a test log that hits a particular line? Insert an abort() call right after it and rebuild. Then unleash the fuzzer. If you’ve fed it a good test corpus that gets somewhere near your target, it will probably not take long for the fuzzer to random-walk into your abort() call and record that log.

Then watch your termination times. For a while we’d generally get a result within hours, but we eventually hit a break after which the fuzzer would run for days without result. That knee in the curve is your clue that the fuzzer has done everything it can.

I dub this technique “fuzzbombing”. I think it will generalize well.

Managing modafinil

For the last year or so I have been deliberately experimenting with a psychoactive, nootropic drug.

You have to know me personally (much better than most of my blog audience does) to realize what a surprising admission this is. I’ve been a non-smoking teetotaller since I was old enough to form the decision. I went through college in the 1970s, the heyday of the drug culture, without so much as toking a joint. I have been open with my friends about having near enough relatives with substance-abuse problems that I suspect I have a genetic predisposition to those that I am very wary of triggering. And I have made my disgust at the idea of being controlled by a substance extremely plain.

Nevertheless, I have good reasons for the experiment. The drug, modafinil (perhaps better known by the trade name Provigil) has a number of curious and interesting properties. I’m writing about it because while factual material on effects, toxicology, studies and so forth is easy to find, I have yet to see useful written advice about why and how to use the drug covering any but the narrowest medical applications.

Before I continue, a caveat that may save both your butt and mine. In the U.S., modafinil is a Schedule IV restricted drug, illegal to use without a prescription. I use it legally. I do not – repeat, do not – advise anyone to use modafinil illegally. I judge the legal restriction is absurd – there are lots of over-the counter drugs that are far more dangerous (ibuprofen will do as an example) – but the law is the law and the drug cops can flatten you without a thought.

Another caveat: Your mileage may vary. This is a field report from one user that is consistent with the clinical studies and other large-scale evidence, but reactions to drugs can be highly idiosyncratic. Proceed with caution and skepticism and self-monitor carefully. You only have one neurochemistry and you won’t like what happens if you break it.

The first thing I should cover is why I get to use this drug legally. Many of you know that I have congenital spastic palsy. Our first weird fact about modafinil is that it is the only drug – indeed, the only medical intervention of any kind – that has improved the motor control of CP patients in controlled trials. This is a more interesting effect than drugs like Baclofen that are merely muscle relaxants.

Yes, modafinil helps me walk better – that’s why I can get it prescribed. I’m not actually very impaired as a normal thing, but it is of medically significant use in the circumstances when I am worst off – mainly, when I’m fatigued or under stress. Modafinil greatly reduces the likelihood that when I am tired or strained I will become extremely clumsy, barge into things, break them, and hurt myself.

Nobody knows how this works. (This is a phrase you will hear a lot in descriptions of modafinil effects.) CP is thought to be actual damage to the brain’s motor control circuitry. For a drug to be able to counteract or override that is bizarre – like observing a broken watch to work better when you pour butter into it – and nobody has any plausible theory about it.

Eppure, si muove. It really works. I have no explanation.

Modafinil has a much better known role as a wakefulness and pro-concentration drug. While on it, you will feel much less need or desire to sleep, and your ability to single-focus concentrate will go way, way up. Furthermore, there is no crash when the drug effect dissipates. And clinical studies indicate the drug has very low toxicity and addictive potential.

Nobody knows how this works. Pretty much all other wakefulness drugs exact a heavy price – they’re toxic, addictive, psychotogenic, derange your sleep cycle, and known or suspected to have baleful long-term effects on your neurochemistry even if you don’t have one of those short-term bad things land on you.

How do we know the drug is benign? Because there is actually decades of field experience with modafinil (and adrafinil, which is what it turns into when you metabolize it) in various military organizations. These drugs were first seriously investigated as go pills for pilots and SpecOps personnel on long-duration missions. They proved an effective (and unexpectedly non-injurious) replacement for dexedrine and other amphetamines.

This isn’t to say the military experience didn’t reveal any problems at all. I am reliably informed by a source close to the field military that in a few cases heavy modafinil users developed a kind of manic Superman syndrome – neglecting self-maintainance, going without sleep for months, overdriving themselves and their units into psychological and physical collapse. Later in this report I will suggest some lessons to be drawn from these incidents.

The things I read about the military experience were a major factor in convincing me to try modafinil. Nothing less than decades of toxic/addictive/psychotogenic effects not showing up at a level above statistical noise would have done for me, personally; I’m extreeeemely protective of my gray matter. I didn’t know of the Superman-syndrome outliers when I began using it; had I known, I would have found it obvious that this is not a problem with modafinil itself but with stupid people interpreting the drug as a license to be stupider.

I should also note that there have been a very few (as in about 5 cases per million) reports of a very nasty necrotic skin disorder called “Stevens-Johnson syndrome” being brought on by modafinil. But this also occurs as an equally rare side effect of a wide range of other drugs, and the incidence pattern suggests to me that the victims have a rare karotype that responds badly to all of them. Just as stupid people gonna stupid, fragile people gonna fragile. This means less than meets the eye.

(After writing the above I learned that my conjecture is correct. There’s a list of SNPs associated with susceptibility with Stevens-Johnson. The source also lists classes of other drugs that are likely to trigger it: antibiotics, analgesics, cough and cold medication, NSAIDs, antiepileptics, antigout drugs, cocaine and phenytoin.)

There have been an even tinier collection of reports (as in, countable on the fingers of one hand) of psychotic breaks in people introduced modafinil with no previous history of mental illness. Again, the incidence pattern creates a strong suspicion that these were fragile people whose neurotransmitter balance could have been messed with by any number of stimuli, and that modafinil was an accidental but not essential cause here.

Earlier I said “you will feel much less need or desire to sleep”, and that is true. However, modafinil does not actually abolish physical fatigue. What it does is (a) reduce your sleep need by about 2/3rds, sometimes more, and (b) sever the link from physical fatigue to drowsiness, distraction, and brain-fog. If you pay attention to your body while on the drug, you will notice that (after a longer time than usual) it is getting sore and clumsy from physical fatigue – but that will present as a sort of neutral mechanical fact that affects nothing about your mental state; your mind will stay clear, sharp, and focused until the dose dissipates.

Now to the last big thing about modafinil: there is clinical evidence of significant increases in IQ while on the drug. To what extent this can be separated from the large boost in ability to single-focus is not clear, and one 2005 study found a boost effect that decreased with increasing IQ. However, recent studies and a 2016 meta-analysis suggest a stronger and more consistent effect than did earlier ones, with significant gains in both executive function and learning capacity. But nobody knows how this works.

I can’t say as much about this from personal experience as I’d like to, because I don’t know how I’d tell if my IQ were elevated. It’s certainly not something one can notice as easily as “Hey! I can really concentrate.” Also, if it’s really true that the effect decreases with increasing baseline IQ, I’d be poorly positioned to notice it.

However: there is practical field evidence that backs up the more positive studies. I am reliably informed that demand for modafinil from STEM students at top universities and people in cognitively-demanding jobs has created a large underground around the drug. If this is true, the drug cops must be practicing benign neglect; to date, modafinil-related criminal charges in the U.S. can be counted on the fingers of one hand and all seem to have been a sort of decorative garnish on more serious indictments.

(Of course, this cannot be relied on to continue. Moral panics have been ginned up on even slighter causes in the past and doubtless will be again.)

I will also say that I notice some differences in my affect while on the drug that are at least consistent with it jacking up my IQ. It makes me feel calm, cerebral, and in control – the exact opposite of the jittery, volatile effect from caffeine or (I’m told) other conventional stimulants. Emotions aren’t gone but they’re a little damped, a little muted. Except, interestingly enough, for my sense of humor; that is fully operational or possibly even enhanced.

Related point: I find the onset bump when the adrafinil crosses my blood-brain barrier quite noticeable.

(I’ve heard that one of the commoner street names for modafinil is “zombie”. That makes a lot of sense if you think about how that slight damping of emotional swings is going to register to a person who lives in their emotions most of the time and barely even knows what “calm and cerebral” is like.)

Again, I’m probably not the best person to report on changes in affect. While doing web searches related to this topic I discovered several articles on the theme “ZOMG makes me a different person!”. That’s not my experience at all – rather, modafinil makes me more like me. It chemically pegs my affect to the same place I go naturally when I’m at the top of my game.

And am I actually more productive? Oh hell yes. I can tell by the amount of code and text I get written while on modafinil. Many users report productivity outside their normal range; I don’t get that, but I do get consistent performance at or near my normal peak level for as long as I’m on it.

You may be thinking modafinil sounds too good to be true. You have company; everybody who knows anything about drugs that mess with neurotransmitter balance has the same reaction when they learn the facts. Nobody knows how this works.

Before getting to management strategies, I will report another thing: Some but not all modafinil users develop a tolerance to the drug and require increasing doses to collect the effects they want. Odds of developing tolerance seem, unsurprisingly, to increase with frequency of use.

For the rest of this report, I am going to assume that you are either a U.S. citizen with a narrowly valid medical reason to use modafinil (such as spasticity or narcolepsy) and a legal prescription, or you live in a non-U.S. jurisdiction that does not restrict the drug, or you have other means of legal access. I will further assume that you want to maximize the nootropic and other benefits of modafinil while minimizing the risks.

Let’s inventory the risks:

First: Stevens-Johnson syndrome.

Second: Lifestyle dependency. While the clinical studies suggest very low potential for either physical or psychological addiction, you don’t want to go anywhere near subtler, functional versions of addiction either.

Third: Acquired tolerance requiring increasing doses.

Fourth: Self-damage through ignoring physiological cues partially suppressed by the drug (manic-Superman syndrome would be the extreme example of this).

Now the mitigation methods:

Against Stevens-Johnson, don’t have a fragile karotype. OK, there’s not much you can do to prevent that. So learn the symptoms of Stevens-Johnson syndrome and if you think they’re developing, stop taking modafinil and see your doctor immediately. One bit of good news is that you only have to pass this gate once – if they’re not triggered the first time you take it, they’re not going to be second and later times.

To avoid lifestyle dependency, plan your modafinil use around specific, non-recurring, slightly unusual challenges. Like: you need to not be clumsy and fatigued for a particular martial-arts test. Or, you need to put two or three working days’ worth days of peak effort into a project all at once.

To avoid tolerance buildup, don’t use it often. I seem to have a steady rate of about 1 200mg dose a month. I’m most likely to use it to handle unusual events where I want to be functioning at peak and perhaps expect to get less sleep than normal – SF conventions, for example. I will, as mentioned, also take it before a kung fu test for prophylaxis against palsy effects,

To avoid self-damage, self-monitor. In particular, stay aware of your physical fatigue level. Sometimes when I’m on modafinil and my muscles start getting fatigue-sore after hour 20 or so, I take a hot shower and a short nap to make the muscles happier even though my brain doesn’t need the rest yet.

That’s it, really. The last part, self-monitoring, is I think the most important. The drug will expand the envelope of what you can do; take those gains but treat yourself gently – no need to push the expanded envelope to collapse.

Modafinil is actually a mix of two enantiomers, only one of which is active. Once I use up my last few doses, I will be switching to a variant called armodafinil that is just the purified active enantiomer. I’m told it has a gentler onset and a longer dwell time,

A use for which I can certify it is combating fatigue on long drives. The effect I have seen in this application is so dramatic and benign compared to riskier drugs like amphetamines that I think this is in itself a sufficient argument for making modafinil and its variants over-the-counter drugs rather than prescription – they would would rapidly displace much more harmful substances and probably significantly decrease highway fatalities.

The drug also has much to recommend it for medical personnel, search & rescue people, police, and anyone else who has to work odd shifts under potential stress. The calming, anti-jitter effect is significant here and an improvement over large doses of caffeine, which promotes wakefulness without being particularly pro-concentration.

Finally, of course, there’s flow-state maintenance for programmers. Frankly, I don’t understand why steady use of modafinil is not already so dirt-common among people who code for a living that everyone takes it for granted. The pro-concentration effect is hugely helpful for productivity, and after a year of use I have experienced no downside at all, not even the jitters and wakefulness I would expect from deploying caffeine for similar purpose.

Nevertheless, I’m still wary of taking it more often, because I don’t want to develop that lifestyle dependency. On the other hand, I’ve seen a reason I might want to relax about this more as I get older. A recent study out of Italy suggests that modafinil improves centrality of neural function in elderly people, in effect mitigating or even partly reversing the effects of physical senescence on the brain.

Er, so, anti-senescence on top of everything else? Seems way too good to be true. But the positive results keep rolling in. I shall continue experimenting, self-monitoring, and perhaps occasionally reporting on it here.

Gift vs. reputation in hacker culture

A G+ follower pointed me at Note on Homesteading the Noosphere by Martin Sústrik. He concludes saying this:

In short: Labeling open source communities as gift cultures is not helpful. It just muddles the understanding of what’s actually going on. However, given that they are not exchange economies either, they probably deserve a name of their own, say, “reputation culture”.

I’m going to start by saying that I wish I’d seen a lot more criticism this intelligent. It bothers me that in 20 years nobody seems to have refuted or seriously improved on my theories – I see this as a problem, both for the study of hacker culture and in the field of anthropology.

That said, I think Sústrik gets a couple things wrong here. And don’t want them to obscure the large thing he’s gotten right.

First (possible) mistake: I have not observed that, as a matter of language, the term “gift culture” is as hard-edged and specific as he thinks it is. There’s a way we could both be right, though – it might be that terminology has shifted since I wrote HtN. Possibly this came about as part as the revival of interest in the concept that I seem to have stimulated.

But: one piece of evidence that anthropologists are still using “gift culture” in the inclusive sense Sústrik criticizes me for enmploying is that Sústrik himself feels, at the end of the article, that he needs to propose a contrasting term rather than citing one that is already established.

This so far is all about map rather than territory. As a General Semanticist I know better than to get over-invested in it.

Here’s the territory issue: Sústrik is not quite right about expectations of direct reciprocal exchange not being a shaping force. True enough that they aren’t salient at the macrolevel the way they were among the Kwakaka’wakwe. But if I download a piece of open source, and it’s useful to me, and I find a bug in it, I do indeed feel a reciprocal obligation to the project owner (not just an attenuated feeling about the culture in general) to gin up a fix patch if it is at all within my capability to do so – an obligation that rises in proportion to the value of his/her gift.

I should also point out that the cultures Sústrik think are paradigmatic for his strict sense of “gift culture” are mixed in the other direction. There is certainly an element of generalized reputation-seeking in the way individual Kwakaka’wakwe discharged their debts. There, and in the New Guinea Highlands, the “big man” is seen to have high status by virtue of his generosity – he overpays, on the material level, to buy reputation.

In the terms Sústrik wants to use, open-source culture is reputation-driven at both macro-level and microlevel, and also sometimes driven by gift reciprocation in his strict sense at microlevel. The macro-level reputation-seeking and micro-level gift reciprocity feed and reinforce each other.

This brings me to the large thing that Sústrik gets right. I think his distinction between “gift” and “reputation” cultures is fruitful – both testable and predictively useful. While I’m still skeptical about it being in general use among anthropologists, I rather hope I’m mistaken about that – better if it were.

Yes, real-world cultures are probably never pure examples of one or the other. But differentiating the mechanisms – and observing that the Kwakaka’wakwe and hacker culture are near opposing ends of the spectrum in how they combine – that is certainly worthwhile.

As a minor point, Sústrik is also quite right about reciprocal licenses being a red herring in this discussion. But I think he has the reason for their irrelevance mostly wrong. The important fact is they’re not mainly intended to regulate in-group behavior; they’re mainly a lever on the behavior of outsiders coming into contact with the hacker culture.

(It was actually my wife Cathy – a pretty sharp-eyed observer herself, and not coincidentally a lawyer – who brought this to my attention.)

Bottom line, however, is that this was high-quality criticism that got its most central point right. In fact, if I were writing HtN today I would use – and argue for – Sústrik’s distinction myself.

Open Adventure 1.1, and some thoughts on software preservation

Open Adventure 1.1 has shipped. There are a lot more changes under the hood than are readily apparent. In fact there have been no changes in gameplay at all, and only minor changes to the UI (reversible with the -o oldstyle switch).

We (Jason Ninneman, Per Vorpaev, Aaron Traas, Peje Nilsson and I) could have taken the approach of changing the original rather ugly C code (mechanically translated from FORTRAN) as little as possible, simply packaging it for compilation and release in a modern environment.

I elected not to do that, one reason being that I think we honor hacker tradition better by bring the code forward as a dynamic, living artifact that invites being hacked on than museumizing it as a static one. There’s also the fact that the extreme obscurity of the code made it difficult to appreciate what a work of genius Adventure actually was. (The code we inherited had over 350 gotos in it – rather hard to see past those.)

So we’ve taken a different path. We’ve translated the code into (almost) fully idiomatic C (but not trying to introduce pointer idioms; that should make translation to future languages easier). We’ve replaced the rather cryptic custom text database file that used to define the dungeon with a YAML document that is orders of magnitude easier to read and modify. We haven’t hesitated to use technology that wasn’t even a gleam in anyone’s eye when Adventure originated – the YAML is compiled to C structures at build time by a Python script.

The effect (we hope) is Adventure as it would have been written if Crowther & Woods had had today’s tools to do it – the same vision and design logic, expressed in modern coding idioms. Worth doing, because there are still some things to be learned from this design.

Probably the single cleverest thing in it – which pretty much has to go back to Crowther, Woods couldn’t have bolted it on afterwards – is the way movement in the dungeon is handled. The dungeon’s topology is expressed by a kind of pseudocode broadly resembling the microcode found underneath a lot of processor architectures; movement consists of dispatching to the sequence of opcodes corresponding to the current room and figuring out which one to fire depending not only on the motion verb the user entered but also on conditionals in the pseudocode that can test for the presence or absence of objects and their state.

It was hard to fully understand and appreciate this this before, because the code was a spaghetti tangle in what looks today like a shockingly primitive style. The abstraction of the dungeon topology into a declarative specification that – in effect – loads microcode into the game engine was a thing you could half-see, but the impact was blunted by the unreadability of both the code and the specification format. Lifting the specification to YAML was like polishing a rough diamond, revealing beauty and brilliance.

And that’s before we even get to Adventure considered as a work of communicative art. It’s had so many successful descendants – like, every dungeon-crawling game ever, and every text adventure ever – that it’s difficult to see with fresh eyes. But if you make the effort, it is astonishing how mature the wry, quirkily humorous, slightly surrealistic style of this very first game seems. The authors weren’t fumbling for an idiom that would be greatly improved by later artists more sure of themselves; instead, they achieved a consistent and (at the time, unique) style that would be closely emulated by pretty much everyone who followed them in text adventures, and not much improved on as style, even though the technology of the game engines improved by leaps and bounds.

I don’t know how they did it, and the authors would probably not be able to explain if we asked. But I think it is damned impressive how well this game has aged – the code may have needed a refresh, but the design still shines. I’m proud to have helped restore it, and hope I have brought it to a state where it can be forward-ported to future languages for as long as programming is a living art.

Open Adventure ships

Colossal Cave Adventure, that venerable classic, is back and better than ever! The page for downloads is here.

The game is fully playable. It would be astonishing if it were otherwise, since it has been stable since 1995. One minor cosmetic change a lot of people used to the non-mainline variants will appreciate is that there’s a command prompt.

The most substantial thing Jason Ninneman and I have done to the code itself is that the binary is now completely self-contained, with no need for it to refer to an external adventure.text file. That file is now compiled to C structures that are linked to the rest of the game at build time.

The other major thing we’ve done is that the distribution now includes a pretty comprehensive regression-test suite. We do coverage analysis to verify that it visits most of the code. This clears the way to do serious code cleanup (gotos and stale FORTRAN idioms must die!) while being sure we’re not breaking things,

We gratefully acknowledge the support of the game’s original authors, Don Woods and Will Crowther.

Request for WordPress help

I need a bit of hands-on time with someone who knows how to troubleshoot WordPress installations.

I recently had to upgrade my WordPress installation due to an exploit that inserted a malicious URL in one of my widgets. Since then, my spam filter has not been operating correctly. I am not sure whether Akismet is working very slowly or not working at all, but the net result is that I am having to approve every post by hand.

There is a suspicious thing visible from my admin account. It looks as though Akismet is installed twice.

I suspect the fix for this is something simple, but I don’t know what it is. Can anybody help?

Correction to: “Set the WABAC machine…”

Henry Spencer, upon reading my previous post, had this to say by email:

I won’t argue with Eric about the significance of that little event, but I do have to interject a few corrections about the historical details.

For this, I’ve got a couple of advantages over him. First, being somewhat of a packrat, I still have my notes from conferences 30+ years ago! (Not in digital form, alas.) And second, being the one who saved most of what we have of Usenet’s early days, I also have some relevant old Usenet postings to consult.

The precise date was 19 January 1984, in the second morning session of the Uniforum conference (a joint meeting of the Usenix Association and the /usr/group trade association). Kathleen Hemenway of Bell Labs gave a talk on work she and Helene Armitage had done: “A proposed syntax standard for UNIX system commands”. This was basically an attempt to codify and slightly tighten the rules already implemented several years earlier by AT&T’s getopt() library function.

As you might gather from that wording, getopt() wasn’t actually new then: it had existed within Bell for some time, but hadn’t made it out in any widely-available Unix release. In late 1981, I’d seen a Bell-internal Unix manual page describing it. I wanted it, and I couldn’t have it, so I wrote my own. It was indeed quite helpful, and on 11 Jan. 1982, I posted it to Usenet newsgroup net.sources. By the time of that conference, a fair number of people were using it.

Ms. Hemenway’s talk was mostly about the syntax issues, but she did mention that there were plans for enhancements to getopt() to help support the new spec. And during Q&A, she was indeed evasive about availability of the enhanced code — probably nobody had made any decisions about that. So I got up and said that I would upgrade my public-domain implementation to match any enhancements AT&T made. Didn’t seem like a big deal; to be honest, I was a bit surprised that it got cheers and applause. I guess a lot of people hadn’t yet grasped that we *had a choice* about this.

No…no, we hadn’t. And my having got the date wrong explains the particularly high anxiety about the Bell breakup; the consent decree would have taken effect not nine months previously but just eight days before.

Ever since, on the occasions that I remembered this before returning to a busy life, the second or third thought in my mind was always that I ought to find Henry and let him know what an influence he had on me in that moment.

Henry Spencer, you set an example that day for all of us, and we cheered you not for promising to write a few lines of code but because we heard the call behind the promise. The details grew dim in my memory, but the power of that example struck me again each time I recalled it. Value your craft and pursue excellence in it; share what you create; never fear to go up against a monopolist; be loyal to your peers in the work. This is how a hacker – how any kind of maker, really – does his duty to the future.

A lot of us have been trying to live out that lesson ever since. Thank you, Henry. I think the world owes you more than it knows for that inspiration.

Set the WABAC to 1984: Henry Spencer getopt, and the roots of open source

I excavated a bit of hacker history from old memories today. Not dead history either, but an important beginning of some large good things.

Here’s how it happened. I got email from a person requesting me to identify a source for the following allegedly famous quote: “All operating systems eventually turn into Unix”.

I told him that I’d never heard that quote as written but that it reminded me of a line by Henry Spencer: “Those who do not understand Unix are condemned to reinvent it, poorly.” This is of course a take on a well-known aphorism by Santayana.

I googled Henry Spencer’s name and some keywords to make sure I had the quote correct. The top hit, from which I verified the quote, was Henry’s Wikipedia page. I read it and something caught my eye – the assertion that Henry wrote his well-known and widely distributed regex package around 1987. And suddenly I remembered something.

Years ago I wrote Eminent Domains: The First Time I Changed History in which I mentioned Henry Spencer’s presence at the conference where I persuaded the NWG not to drop the functional domains in favor of a strict geographic system.

What I just remembered today is something else that happened at that conference that may have been as important, though in a less obvious way. It also pins the date to 1984, because I remember now that anxiety about what AT&T’s recent commercialization of Unix meant to us was very much in the air at that conference (IIRC the big announcement had happened in January or February). We were all worried that this meant the Bell Labs source code would no longer be shared with universities and research labs…as was indeed to become the case.

To fully understand this story you need to know that Henry was already a hugely respected figure in the tiny, not yet very conscious hacker culture of the time (there were, at a guess, somewhere around 500 of us then in the U.S. and Canada – maybe 700 worldwide). I knew him from USENET; I’d seen some of his code, in the relatively small amounts we passed around back then.

It also matters that the GNU Manifesto hadn’t been written yet – wouldn’t be until the following year, though I had some idea what was coming because Richard Stallman had discussed his plans with me at Boskone in February. (That was when I suggested to him that the flagship project for the FSF ought to be Emacs. Duh!) Stallman’s concept of “Free Software” was not yet part of the hacker zeitgeist,

So in October 1984 I was in a crowd of people watching a presentation by a woman from Bell Labs describing the then-new getopt(3) library, written by AT&T as a way to regularize the processing of command-line arguments in C programs. The custom up to then had been to write ad-hoc code for each application, leading to some odd and unpredictable variations in behavior.

Everybody thought this was a fine idea, and several people asked questions probing whether AT&T was going to let anyone else use the getopt code they had written. These questions related to the general anxiety about Unix source code distributions drying up.

Frustration mounted as the woman gave evasive answers which seemed to add up to “No, we refuse to commit to allowing general access to this code.” Which seemed to confirm everyone’s worst fears about what was going to happen to Unix source code access in general.

At which point Henry Spencer stands up and says (not in these exact words) “I will write and share a conforming implementation.” – and got a cheer from the assembled.

If you’re thinking “That’s not a big deal, we do this sort of thing all the time,” my actual point is that in October 1984 this was indeed a big deal. It took an actual imaginative leap for Henry Spencer to, in effect, say “Screw AT&T and its legalisms and evasions, if they’re going to cut off source access we hackers are gonna do it for ourselves.”

Yes, we had DECUS tapes and the Unix sources newsgroups on Unix already; Henry was building on that. But he got an actual cheer exactly because he was pushing forward, exposing the possibility of doing not just small projects and demos and quirky little tools but at competing with the likes of AT&T itself at software production.

Of course RMS was, as I was then one of the very few to already know, thinking along the same lines. But RMS’s was a more personal, ideological vision. Henry didn’t have any grand plan to save the world; he was just being a hacker, seeing where the solution to the problem posed by AT&T’s source-code lockdown had to lie, pointing it out just a bit sooner than anyone else, and putting his own skin and considerable reputtion in the game.

So yeah. In retrospect I think this was a historically pivotal moment. A respected elder of our tiny tribe (so tiny that probably a substantial fraction of it was in the room listening) declared his independence from what AT&T chose to do about restricting its code. And they took that possibility home with them.

I’m put in mind of the historian Oswald Spengler’s notion that cultures are born at a moment of what Nietzsche called “transvaluation of all values”. Arguably this was one, rather like the one I semi-accidentally triggered in the late 1990s, at which the hacker culture woke up some – became aware of the possibilities implied by its existing folkways.

And there’s a more direct and personal one I had half-forgotten. I was a young, unknown programmer then – just 27, still figuring out what I wanted. I watched Henry make that promise. I heard the cheer, and felt the change in the air as culturally, we realized what the solution to AT&T fscking us over had to be.

And I thought “I want to be like that guy.”

The Adventure begins again

Colossal Cave Adventure was the origin of many things; the text adventure game, the dungeon-crawling D&D (computer) game, the MOO, the roguelike genre. Computer gaming as we know it would not exist without ADVENT (as it was known in its original PDP-10 incarnation).

Long ago, you might have played this game. Or maybe you’ve just heard stories about it, or vaguely know that “xyzzy” is a magic word, or have heard people say “You are in a maze of twisty little passages, all alike”,

Though there’s a C port of the original 1977 game in the BSD game package, and the original FORTRAN sources could be found if you knew where to dig, Crowther & Woods’s final version – Adventure 2.5 from 1995 – has never been packaged for modern systems and distributed under an open-source license.

Until now, that is.

With the approval of its authors, I bring you Open Adventure. And with it some thoughts about what it means to be respectful of an important historical artifact when it happens to be software.

This is code that fully deserves to be in any museum of the great artifacts of hacker history. But there’s a very basic question about an artifact like this: should a museum preserve it in a static form as close to the original as possible, or is it more in the right spirit to encourage the folk process to continue improving the code?

Modern version control makes this question easier; you can have it both ways, keeping a pristine archival version in the history and improving it. Anyway, I think the answer to the general question is clear; if heritage code like this is relevant at all, it’s as a living and functional artifact. We respect our history and the hackers of the past best by carrying on their work and their playfulness.

I’ve actually dealt with this question before, which is why Super Star Trek has both interface improvements and a -t option that invokes the original TTY interface in all its brutal simplicity. Open Adventure will have a similar option.

When Willie Crowther wrote the very first version in 1976 he was also writing firmware for the earliest generation of routers on the operational ARPANET – he was one of the most ingenious and capable programmers of his time. The code makes a fascinating study in how to push the limits of the primitive tools then available to him.

It’s very strange to the modern eye just to see a simulation like ADVENT written with no analogue even of C structures, let alone objects. But the FORTRAN Crowther wrote in didn’t have such things; game state was all global variables, a “feature” preserved in the mechanically translated C of 2.5. (Which, alas, is extremely ugly code full of gotos.)

It also looks odd, 40 years after the fact, to see the amount of code complexity devoted to space/time optimization so that (for example) you don’t have to re-parse the text master of the dungeon-defining database on every startup. That’s what you had to do then, when a room-filling minicomputer cranked many fewer instructions per second than the controller in your microwave oven.

Despite all the energy Crowther and Woods had to spend fighting ancient constraints, ADVENT was a tremendous imaginative leap; there had been nothing like it before, and no text adventure that followed it would be innovative to quite the same degree.

Thus, one of my goals moving forward will be to make the logical design of the dungeon simulation easier to understand for a modern reader. This will probably entail something parallel to what was done in the BSD Games port – gradually massaging the code into more idiomatic C. This reflects my judgment that it’s more important to make Crowther & Wood’s algorithmic thinking more visible than it is to, in any sense, preserve the details of their code.

It’s actually more difficult to decide how much to change their data structures. In idiomatic C one would collect all the saveable game state into a structure rather than leaving it as a mess of global variables that have to be saved and restored one at a time. And a lot of the strange bit-packing hacks in the database representation could probably be done away with today. But would doing that sort of thing obscure their achievement too much?

Probably not – and the baseline version will still be visible in the repository, after all. Likely what I’ll end up doing (if I can allocate the time) is cleaning up stuff like that and leaving comments explaining what it looked like in the FORTRAN original.

I don’t plan on any gameplay changes yet. Any such things will be reversible by the switch to restore historic behavior.

One thing I definitely plan is to modify the code so it can be regression-tested by capturing command logs, replaying them to the game and checking that the output is as expected – testing a full walkthrough should be possible. Matter of fact that needs to happen before anything else so that the correctness of changes can be verified.

This is not a strategy that would likely have occurred to anyone (well, maybe outside of a handful of people at Bell Labs) in 1977. Finding it thinkable depends on a mindset created by Unix pipes and redirection that was only gestating then.

Replay also busts one of Crowther & Woods’s original goals – thwarting attempts to get around obstacles in the game with any investment less than quitting and playing again. Towards this end they deliberately obscured the encoding of save files to make point tweaks of the game state difficult.

That, however, is a hack that fails the Kerckhoffs’s Law test – it’s useless or near useless in an environment where the “adversary” has the source code. When ADVENT (so called because PDP-10 filenames were limited to six characters of uppercase) was written, that was a pretty safe assumption, but today it will never be safe again. No harm, then, in dropping their obscurifying measures.

Source code obscurity is a “feature” from 1977 I miss less than even FORTRAN…

The advent of ADVENT

A marvellous thing has just occurred.

Colossal Cave Adventure, the original progenitor of the D&D-like dungeon-crawling game genre from 1977 and fondly remembered as ADVENT by those of us who played it on PDP-10s, is one of the major artifacts of hacker history.

The earliest version by Crowther and Woods (sometimes known as 350-point Adenture) was ported to C by Jim Gillogly in ’77 just after it first shipped. That has been part of the bsd-games collection forever.

What I have have just received Crowther & Wood’s encouragement to polish up and ship under a modern open-source license is not the Gillogly port; it’s Crowther & Woods’s last version from 1995. It has 18 years of work in it that the Gillogly version doesn’t.

I feel rather as though I’d been given a priceless Old Master painting to restore and display. Behooves me to be careful stripping off the oxidized varnish.