An Elegant Puzzle: Systems of Engineering Management

by Will Larson

While David’s cloak is increasingly suspicious draped from the shoulders of Silicon Valley’s Goliaths, the vast majority of technology companies are well-meaning chrysalises that hope to one day birth a successful business, and they are guided by managers who are learning to lead, one unexpected lesson at a time.

There are few things peaceful about managing in rapidly growing companies, but I’ve never found anywhere better to learn and to grow.

Maintained consistently and changed sparingly, nothing else will help you scale more.

This re: good organizational design

The hard bit is keeping your courage up when circumstances get challenging.

Always

avoid creating a two-tiered class system of innovators and maintainers.

Adding new individuals to a team disrupts that team’s gelling process, so I’ve found it much easier to have rapid growth periods for any given team, followed by consolidation/gelling periods during which the team gels. The organization will never stop growing, but each team will.

Fundamentally, I believe that sustained productivity comes from high-performing teams, and that disassembling a high-performing team leads to a significant loss of productivity, even if the members are fully retained. In this worldview, high-performing teams are sacred, and I’m quite hesitant to disassemble them.

Teams take a long time to gel. When a group has been working together for a few years, they understand each other and know how to set each other up for success in a truly remarkable way.

Sure OK but I don’t think I’ve ever seen a given team composition be stable for longer than about a year

if it’s a choice of moving people rapidly or shifting scope rapidly, I’ve found that the latter is more effective and less disruptive.

When I started at Uber, we were almost 1,000 employees and were doubling the headcount every six months. An old-timer summarized their experience as: “We’re growing so quickly that every six months we’re a new company.” A bystander quickly added a corollary: “Which means our process is always six months behind our head count.”

The cardinality of supported systems increases over time as you add teams, and as “trivial” systems go from unsupported afterthoughts to focal points for entire teams as the systems reach scaling plateaus (things like Apache Kafka, mail delivery, Redis, etc.).

SAME

A similar variant of this is ad hoc meeting requests. The best tool that I’ve found for this is to block out a few large chunks of time each week to focus.

Finally, the one thing that I’ve found at companies with very few interruptions and have observed almost nowhere else: really great, consistently available documentation. It’s probably even harder to bootstrap documentation into a non-documenting company than it is to bootstrap unit tests into a non-testing company, but the best solution to frequent interruptions I’ve seen is a culture of documentation, documentation reading, and a documentation search that actually works.

There are a non-zero number of companies that do internal documentation well, but I’m less sure if there are a non-zero number of companies with more than 20 engineers that do this well.

Finally, a related antipattern is the gatekeeper pattern. Having humans who perform gatekeeping activities creates very odd social dynamics, and is rarely a great use of a human’s time. When at all possible, build systems with sufficient isolation that you can allow most actions to go forward. And when they do occasionally fail, make sure that they fail with a limited blast radius.

If you keep a to-do list, look at the categories of the work you’ve completed over the past six months, as well as the stuff you’ve been wanting to do but keep putting off.

Product management is an iterative elimination tournament, with each round consisting of problem discovery, problem selection, and solution validation.

Strategies are grounded documents which explain the trade-offs and actions that will be taken to address a specific challenge. Visions are aspirational documents that enable individuals who don’t work closely together to make decisions that fit together cleanly.

I’ve found that the hardest part of writing a good strategy is pretty mundane. You must be honest about the constraints that are making the challenge difficult, which almost always include people and organizational aspects that are uncomfortable to acknowledge.

No extent of artistry can solve a problem that you’re unwilling to admit.

If strategies describe the harsh trade-offs necessary to overcome a particular challenge, then visions describe a future in which those trade-offs are no longer mutually exclusive.

prefer usefulness over consistency.

Investments describe a future state that you want to reach, and baselines describe aspects of the present that you want to preserve.

The most common way to use goals is during a planning process. By agreeing on the mix of investment and baseline goals for each team, you’re able to set clear expectations for a team while still giving them full ownership of how they’ll satisfy the constraints.

Effective de-risking is essential, because each team who endorses a migration is making a bet on you that you’re going to get this damn thing done, and not leave them with a migration to an abandoned system that they have to revert to.

With all of this in mind, take an hour and write up as many goals as you can for what you’d like to accomplish in the next one to five years. Then prioritize the list, pick a few that you’d like to focus on for the next three to six months, and share it with your manager at your next one-on-one.

It’s okay to approach change gradually.

God this is so true across so many situations.

This is just one example of the dynamics that play out across many dimensions when you’re considering introducing a new authority, and the most useful framework I’ve found for thinking through this involves positive and negative freedoms. A positive freedom is the freedom to do something, for example the freedom to pick a programming language you prefer. A negative freedom is the freedom from things happening to you, for example the freedom not to be obligated to support additional programming languages, even if others would greatly prefer them.

Communication is company-specific. Every company has different communication styles and patterns.

Prepare a lot; practice a little.

I’ve always preferred learning in private. Got something difficult? Sure, leave me alone for a few hours and I can probably figure it out. If you want me to figure it out with you watching, I’m not even sure how to start.

:SAME:

At a sufficiently high rate of change, policy is indistinguishable from exception.

“With the right people, any process works, and with the wrong people, no process works.”

Give as much as you can sustainably give, and draw the line there.

It is extremely hard to consistently do the basics well in these circumstances, because you simply won’t have enough time to do them well. You’ll have to get comfortable doing as well as time constraints allow, and sometimes that will lead to being mediocre at things you’re passionate about.

There is a lot less competition for hard work.

unsettling period when you lose access to what used to make you happy—partnering directly with a team—and haven’t found new sources of self-worth in your work.

The Team Geek apple harvesting analogy?

The basis: an inclusive organization is one in which individuals have access to opportunity and membership. Opportunity is having access to professional success and development. Membership is participating as a version of themselves that they feel comfortable with.

It’s a strange tragedy that we hold ourselves accountable for building healthy, functional teams, and yet are so rarely on them ourselves.

RE: Managers

Wielding this distinction, “freedom” is neither inherently good nor inherently just, and descends into the murky gray that already embroils everything else in our lives. Each positive freedom we enforce strips away a negative freedom, and each negative freedom we guarantee eliminates a corresponding positive freedom. This sad state of affairs is often referred to as the Paradox of Positive Liberty.

A few closing tangents. First, Tom DeMarco’s Slack13 has an excellent suggestion for a good starting state between positive and negative freedoms for engineering teams: generally follow the standard operating procedure (i.e., keep doing what you’re already doing, the way you’re doing it), but always change exactly one thing for each new project.

Working at a company isn’t a single continuous experience. Rather it’s a mix of stable eras and periods of rapid change that bridge between eras.

Transitions are opportunities to raise the floor by building competency in new skills, and in stable periods you can raise the ceiling by developing mastery in the skills that the new era values.

Almost every unkind interviewer I’ve worked with has been either suffering from interview burnout after doing many interviews per week for many months or has been busy with other work to the extent that they have started to view interviews as a burden rather than a contribution.