An Elegant Puzzle: Systems of Engineering Management

by Will Larson

Mistake team size for impact. Managing a larger team is not a better job, it’s a different job. It also won’t make you important or make you happier. It’s hard to unlearn a fixation on team size, but if you can, it’ll change your career for the better.

Mistake title for impact. Titles are arbitrary social constructs that only make sense in the context they’re given. Titles don’t translate across companies meaningfully, and they’re a deeply flawed way to judge yourself or others. Don’t let them become your goal.

Broadly, there are three types of engineering management jobs: Manager: you manage a team directly, Director: you manage a team of managers, VP: you manage an organization.

The basis: an inclusive organization is one in which individuals have access to opportunity and membership. Opportunity is having access to professional success and development. Membership is participating as a version of themselves that they feel comfortable with.

The metrics that have been useful for me: Retention is the most important measure of availability of opportunity, although it’s also a very lagging indicator. This should be the first thing you’re paying attention to, but you must recognize that it’s slow to show change. Usage rate is how often folks get picked in project selection.2 The number of distinct team members picked to lead critical projects is a particularly interesting measure. Level distribution is useful, in particular comparing cohorts of folks with different backgrounds. People want role models for themselves in senior roles at the company where they work, which is why looking at the representation of underrepresented minorities and women is only a start. You also want to look in each role and level of seniority. Time at level is how long team members wait between promotions. How does this compare across cohorts? Something to watch for is that this number is also greatly influenced by initial level at hiring. For example, time at level might look equivalent because some cohorts are being under-leveled at time of hire.

Retention is once again the golden measure, and once again a long-trailing indicator. Referral rate by cohort provides insight into which individuals feel comfortable asking their friends and previous coworkers to join the company. Attendance rates for recurring events and team lunches provide some insight into whether folks feel comfortable with those groups. The quantity and completion rate of coffee chats are automatically measured with Donut.

Define the project’s scope and goals in a short document. Particularly important are identifying: Time commitment. People need to decide if they must ask permission from their managers. Requirements to apply. If there are no requirements, say so explicitly; otherwise, a lot of folks will assume that there are, and will opt out. Selection criteria. If multiple individuals apply, how will you select the project leader between them? Announce the project to a public email list, at an all-hands, over Slack, or however your company does persistent communication;

Previously, you would have just sent a ping to a favored individual and they’d have been off and running, but now you have to run a slower and more deliberate process. Increasingly, though, I believe this is the most important change in my approach to leadership over the past few years.

Partnership. Have they been effective partners to their peers, and to the team that they’ve managed? Execution. Can they support the team in operational excellence? Vision. Can they present a compelling, energizing vision of the future state of their team and its scope? Strategy. Can they identify the necessary steps to transform the present into their vision? Spoken and written communication. Can they convey complicated topics in both written and verbal communication? Can they do all this while being engaging and tuning the level of detail to their audience? Stakeholder management. Can they make others, especially executives, feel heard? Can they make these stakeholders feel confident that they’ll address any concerns?

A 90-day plan. The applicant writes a 90-day plan of how they’d transition into the role, and what they would focus on. They emphasize specific tactics, time management, and where they’d put their attention. This is also a great opportunity to understand their diagnosis of the current situation. Provide written feedback to them on their plan. Have them incorporate that feedback into their plan. This is an opportunity to try out working together in the new role. Vision/strategy document. The applicant writes a combined vision/strategy document. It outlines where the new team will be in two to three years, and how they’ll steer the team to get there. Provide written feedback on the document. Have them incorporate that feedback. Vision/strategy presentation. Have the applicant present their vision/strategy document to a group of three to four peers. Have the peers ask questions, and see how the applicant responds to this feedback. Executive presentation. Have the applicant present their strategy document, one-on-one, with an executive. In particular, test for their ability to adapt communication to different stakeholders.

we can carefully ramp toward negative freedom and away from positive freedom. This is one of our essential tools for facilitating and prolonging success. Further down the road, if the structure loses its luster, the economy shifts around us, or entropy’s endless march throws a wrench into the machinery, then once again we shift toward positive freedoms, which gives the organization a greater chance to successfully adapt to its new circumstances.

don’t treat growth as a foregone conclusion. Growth only comes from change, and that is something you can influence.

Be kind to the candidate. Ensure that all interviewers agree on the role’s requirements. Understand the signal your interview is checking for (and how to search that signal out). Come to your interview prepared to interview. Deliberately express interest in candidates. Create feedback loops for interviewers and the loop’s designer. Instrument and optimize as you would any conversion funnel.

Prepared presentations on a topic. Instead of asking the candidate to explain some architecture on the spur of the moment, give them a warning before the interview that you’ll ask them to talk about a given topic for 30 minutes, which is a closer approximation of what they’d be doing on the job. Debugging or extending an existing codebase on a laptop (ideally on their laptop). This is much more akin to the day-to-day work of development than writing an algorithm on the board. A great problem can involve algorithmic components without coming across as a pointless algorithmic question. (One company I spoke with had me implement a full-stack auto-suggest feature for a search inbox box, which required implementing a prefix tree, but the interviewer avoided framing it as yet another algo question.) Giving demos of an existing product or feature (ideally the one they’d be working on.) This task helps them learn more about your product and get a sense of whether they have interest around what you’re doing, and it helps you get a sense of how they deliver feedback and criticism. Roleplays (operating off a script that describes the situation.) This option can be pretty effective if you can get the interviewers to buy into it, allowing you to get the candidate to create more realistic behavior (architecting a system together, giving feedback on poor performance, running a client meeting, and so on).

the single clearest indicator of strong recruiting organizations is a close, respectful partnership between the recruiting and engineering functions. Spending some time cold sourcing is a great way to build empathy for the challenges that recruiters face on a daily basis, and it’s also an excellent opportunity to learn from the recruiters you partner with! We’ve been doing weekly cold sourcing meetings as a partnership between our engineering managers and engineering recruiters, and it’s been a great forum for learning, empathy-building, and, of course, hiring.

Onboard. How long does it take new hires to get up to speed? It’s pretty tricky to measure this, but since you’re trying to reason about the population rather than individuals, it’s okay to be a bit messy. Pick a productivity metric, maybe commits per week, and see how long it takes for new hires to reach P40 productivity. That’s a sufficiently good measure for understanding how long it takes folks to ramp up.

Promote. How long does it take individuals to get promoted after they’re hired? This is useful for understanding whether employees have access to opportunity8 within your organization.

Career level. For every role, a given level will be established as the career level, and most individuals are not expected to progress beyond that level. Over time, this often leads to career level clustering, with the normal distribution centered on the career level, as opposed to the typical goal of the distribution centering at mid-level.

Whenever possible, prefer tests that have a candidate show their strengths, avoiding formats in which they tell you about it. For example, we took an interview in which folks described their experience building a healthy team, and replaced it with an interview in which folks are given results from an organization health survey and asked to identify issues and propose how they’d address them.

Try to avoid design by committee. These almost always lead to incremental change. Prefer a working group of one or two people that is then tested against a bunch of candidates for feedback!

Don’t hire for potential. Hiring for potential is a major vector for bias, and you should try to avoid it. If you do decide to include potential, then spend time developing an objective rubric for potential, and ensure that the signals it indexes on are consistently discoverable.

https://lethain.com/durably-excellent-teams/