How to write narrative documentation

The following is a very lightly edited version of email I wrote to my apprentice Ian Bruene after he wrote documentation for his new Kommandant project that was, alas, as awful as I generally expect from programmers. I’m not training Ian for mere coding competence; he’s too talented for that and anyway I have higher standards. This is my way of insisting that he do documentation well – and it was he who suggested it would make a good blog post.

Here’s how to write narrative documentation for a thing like Kommandant.

You want to do this well because people will pre-judge the quality of your software by how clearly you can write about it. They’re right to do so; explaining it clearly is their best warrant that your thinking was not muddy and confused when you wrote it.

In fact, writing good documentation is an excellent way to ensure that you really understand the problem space you’re in, and to throw light into corners of your software where defects might lurk. Do not underestimate the power of this effect! Often enough to matter, it will save you from serious embarrassment.

Doing this right needn’t be difficult. The quality of your documentation is like the quality of your code – less a result of how much effort you put in than it is of having the right mental habits to begin with.

One important mental habit is to not be terrified by the blank sheet of paper. One of my goals in giving you a procedure to follow is so when you start something like this you can do it on automatic pilot. That will make it easier to deal when you have to write the more difficult, more project-specific parts. Over time you will evolve the procedure to suit your tastes, and that’s fine. It’s meant to a springboard, not a straitjacket.

First choose a title. Make it simple and direct. Use the name of the product in it. The phrase “how to” may occur. Examples: “Writing Applications with Kommandant”, “Kommandant Client how-to”.

Then write an introductory paragraph that explains why the product exists. A good section title might be “Introduction” or “Motivation”. It is often good for the first, topic sentence to read something like “Foo is a library for doing Bar”. If the product is derived from or modeled on another piece of software, nod in that direction. If there are examples of similar products your userbase is likely to know about, mention them.

The reader should exit the introduction with a clear sense of how the software will help him and why he cares.

Next, a section on theory of operation. This explains the problem your software solves in more detail, and describes the strategy it uses. Here is where you want to sketch the relationship between the Kommandant core and the class that implements the user’s commands. Don’t get too far into the API weeds here, but *do* establish terminology – like “core”, “helper” and the different categories of user-defined hooks.

Be clear about why things are done as they are. Sometime this is best done by nodding at alternatives. Like: “We could have required the user to explicitly register command handlers, but by introspecting on the client class we both simplify the problem and eliminate a potential source of defects. Python Cmd set a good example here.”

The user should exit the “Theory of operation” section grasping the concepts he needs to understand a detailed description of the UI/API.

The next two sections should be a detailed API description and a set of simple worked examples with explanation. Which order to do this in depends on your product. For something as simple as Kommandant, examples first probably works better. On the other hand, if the examples are necessarily so complex that they’re hard to read without having seen the API description first, they go second.

(If you find yourself in the latter situation, consider it as a clue that your design may be overcomplicated and need a rethink.)

The reason it’s good to put worked examples first if you can is that expository arc in a document like this should always try to supply *motivation* at each step before requiring the reader to do more mental work. The examples are motivation for grappling with the API description.

(Notice that my explanation has followed the same kind of expository arc I’m describing…)

Because you’ll have an API reference nearby there is little need to annotate the examples heavily. Focus on details the reader might find surprising or a little tricky, like the difference between a basic-mode application and one that calls readline.

Your API section should walk through each call, ideally in the order they would occur in a typical application. You’re trying to do two things here: fill in details and tell a story about how an application uses your library

Next section: tricks and traps. Anticipate what might confuse your user about the API and address it directly. A good example in Kommandant is the fact that if you want to customize your prompt, it’s not enough to do it in the PreCmd hook; you have to do in PreLoop too, or the very first prompt won’t be what you expect. Explain why each gotcha is the way it is.

Next section: Credits and connections. Who contributed to this? Who sponsored it? What was its proof-of-concept application?

Finally, a brief revision history. Tie the document to versions of your software. Summarize changes. A user who is revisiting this document should be able to look here and tell what parts to read for the updates.

Once you have written a document like this, a one-stop shop for what users need to know, it attracts updates from them the way your software attracts patches. *You* will find it easier to keep in sync with your software than if its information were scattered across multiple files.

Now read and think about this document

http://www.catb.org/gpsd/client-howto...

as an example of the form. You don’t need to write anything so long or elaborate, because Kommandant’s problem domain is simpler.

When you get this right, the result will feel a little like art; this is not an accident. I’ve been using words like “story” and “narrative arc” for a reason. By engaging the human brain in the way it wants to be engaged, you lower the friction cost for your users of acquiring the information they need. This is a functional virtue and they will love you for it, giving your software long legs.

You have an advantage here. You have a sharp sense of humor that pervades what you write and say. Let that work for you.

Now go do it right; make me proud.

Some time later Ian popped up on IRC saying

* | ianbruene has enough teeth into HOWTO to start enjoying the process

To which I replied:

esr | ianbruene: *Excellent.* That was part of my evil plan.

Which of course it was. You’ll never write decent documentation if you think of it as a deadly chore. You need to learn how to take pride in your documentation and enjoy the process so your users will, too.