Improving docs and measuring quality, at STC Summit
This week I’m attending STC Summit 2019, the annual conference of the Society for Technical Communication (STC). I’m blogging my notes from the sessions that I attend. Thanks and all credit go to the speakers. Any mistakes are my own.
Early on Monday morning, Barbara Giammona presented a session called Measuring and Improving the Quality and Completeness of Your Documentation.
The idea for Barbara’s session came from a request from her boss to figure out ways to measure the effectiveness of documentation. She shared some of the resulting initiatives. She emphasized that measuring docs is tricky, and the initiatives are a work in progress.
Barbara covered the following topics:
Content
Processes
Productivity
Working with partners
Content improvement: Partnering with customer support
Barbara’s team talked to the global customer support team about improving the content of the manuals, partnering with them in a more deliberate way than before.
Some of the feedback was fairly obvious, in retrospect. Things like removing the number of notes and appendices by moving the content into the main doc, breaking up long sentences, and shortening titles. The customer support team also suggested sending high-impact docs for shared review.
One surprising request was to increase the number of acronyms, as they make the content more concise.
The measurement is that the number of customer-reported issues related to the docs goes down within 1, 2, and 3 years.
Content improvement: Hearing from customers
Barbara has made a deliberate effort to get in touch with customers directly. When customers come on site, for example to do beta tests of the product, Barbara makes an effort to meet them. She sets up review sessions, walks around at customer conferences asking people for feedback on the docs.
The measurement is improved customer satisfaction, year on year. To get these results, the teams uses a customer satisfaction survey.
Content improvement: Refactoring of much-used and visible docs
In the documentation plan, the team includes the time and effort needed for improvements to the docs. Tasks include:
Improving and adding/removing graphics
Breaking up long chapters
Simplifying language
Moving installation instructions into one place
Review FAQs for potential inclusion into the docs
Another important goal is to take customer feedback into account when designing new docs.
The measurement is fewer complaints from the support and delivery team.
Process improvement: Length of review cycles
Barbara talked about performing a root cause analysis for the problematically short reviews. Review cycles were too short, and doc tasks were being compressed into shorted periods.
The root cause analysis looks very thorough. People wrote Post-it notes about potential causes, which the team then grouped and analysed. The team used the “five whys” technique to ask up to five questions about why a problem arose. This technique helps find the root cause. Barbara walked us through the process in detail with examples.
The measurement is improved experience of reviews in new projects. Barbara joked that the reviews should no longer make you feel suicidal.
Process improvement: Productivity
The topic here is resource availability and the ability to estimate workload. It’s difficult to estimate the effort required for each project. The technique is to gather data from completed projects (hours spent) and use those for future estimates.
The measurement is accurate cost estimates and not having to scramble to find resources.
Process improvement: Partnerships/outsourcing
Barbara talked about working with a team outside the company to complete documentation tasks. At first the relationship failed due to lack of skill in the partner team.
The technique to solve this problem was to visit the partner team in person and provide training sessions. Training includes things like effective writing and how to with subject matter experts.
The measurement is to see whether the partner team sets targets and delivers on par with the other teams, and whether the US leads are writing less and leading more.
In conclusion
Thanks to Barbara for an enjoyable and thorough talk about techniques for improving our documentation.
