An Elegant Puzzle: Systems of Engineering Management

by Will Larson

2.4.2 Systems survive one magnitude of growth

.h3

If your company is designing systems to last one order of magnitude and is doubling every six months, then you’ll have to re-implement every system twice every three years.

2.4.3 Ways to manage entropy

Finally, the one thing that I’ve found at companies with very few interruptions and have observed almost nowhere else: really great, consistently available documentation.

When at all possible, build systems with sufficient isolation that you can allow most actions to go forward. And when they do occasionally fail, make sure that they fail with a limited blast radius.

2.4.4 Closing thoughts

.h3

2.5 Where to stash your organizational risk?

.h2

What I’ve found most successful is to identify a few areas to improve, ensure you’re making progress on those, and give yourself permission to do the rest poorly.

2.6 Succession planning

.h2

consider your work from several different angles:

Take a look at your calendar and write down your role in meetings.

For each of the individuals you support, in which areas are your skills and actions most complementary to theirs? How do you help them? What do they rely on you for?

Filter the gaps down to two lists:

.c1

The first should cover the easiest gaps to close.

.c2

The latter will be the riskiest gaps. These are the areas where you’re uniquely valuable to the company,

.c3

Chapter 3 Tools

.h1

3.1 Introduction to systems thinking

.h2

If you really want a solid grasp on systems thinking fundamentals, you should read Thinking in Systems: A Primer3 by Donella H. Meadows,

.rl

3.1.1 Stocks and flows

.h3

Big changes appear to happen in a moment, but if you look closely underneath the big change, there is usually a slow accumulation of small changes.

Changes to stocks are called flows. These can be either inflows or outflows.

an information link. This indicates that the value of a stock is a factor in the size of a flow.

keep in mind that every flow is a rate, whereas every stock is a quantity.

3.1.2 Developer velocity

.h3

Accelerate: The Science of Lean Software and DevOp, by Nicole Forsgren, Gene Kim, and Jez Humble,4 I’ve spent a lot of time pondering the authors’ definition of velocity.

.rl

Delivery lead time is the time from the creation of code to its use in production.

Deployment frequency is how often you deploy code.

Change fail rate is how frequently ...

Time to restore service is the time spent recover...

Pull requests are converted into ready commits based on our code review rate.

3.1.3 Model away

.h3

Stella5 is the gold standard, but the price is quite steep,

.c1

The best cheap alternative that I’ve found is Insight Maker,

.c2

3.2 Product management: exploration, selection, validation

.h2

Product management is an iterative elimination tournament, with each round consisting of problem discovery, problem selection, and solution validation.

Write a customer letter. Write the launch announcement that you would send after finishing the solution. Are you able to write something exciting, useful, and real?

A mild caveat: it’s better to rely on people you have some connection to instead of on conference talks and such, since there is a surprisingly large amount of misinformation out there.

A strategy recommends specific actions that address a given challenge’s constraints.

A structure that I’ve found extremely effective13 is described in Good Strategy/Bad Strategy by Richard Rumelt,14 and has three sections: diagnosis, policies, and actions.

Before you’ve even finished reading a great diagnosis, you’ll often have identified several good candidate approaches. That’s the power of a well-defined problem statement, and why it’s an important foundational element for your strategy.

If strategies describe the harsh trade-offs necessary to overcome a particular challenge, then visions describe a future in which those trade-offs are no longer mutually exclusive.

Visions should be detailed, but the details are used to illustrate the dream vividly, not to prescriptively constrain its possibilities.

Bad goals are indistinguishable from numbers. “Our p50 build time will be below two seconds,” or “We’ll finish eight large projects.” You’ll know a goal is just a number when you read it and aren’t sure if it’s ambitious or whether it matters.

Good goals are a composition of four specific kinds of numbers: A target states where you want to reach. A baseline identifies where you are today. A trend describes the current velocity. A time frame sets bounds for the change.

Baseline metrics are useful for narrowing the solution space that you explore in order to accomplish your investment goals. They are also useful for identifying when you should pause pursuing your goals and instead invest in platform quality.

.spark

Although your baselines will often be about preserving a current property, you can also decide to accept some degradation before you want to trigger reprioritization.

After you’ve evolved the design, the next step is to embed into the most challenging one or two teams, and work side by side with those teams to build, evolve, and migrate to the new system. Don’t start with the easiest migrations, which can lead to a false sense of security.

Many folks start migrations by generating tracking tickets for teams to implement, but it’s better to slow down and build tooling to programmatically migrate the easy 90 percent.

The best migration tools are incremental and reversible:

Answer the question you want to be asked. If someone asks a very difficult or challenging question, reframe it into one that you’re comfortable answering. Don’t accept a question’s implicit framing, but instead take the opportunity to frame it yourself.