More on this book
Community
Kindle Notes & Highlights
by
Gene Kim
She’s pretty sure she’s missed a step in the Kübler-Ross grief cycle, but at the moment she can’t think of which one.
pulling up the file again, this time in a binary file editor. She stares at her screen, stunned that she missed it. A BOM is an invisible first character that some programs put in a CSV file to indicate whether it’s big-endian or little-endian.
The first thing she would have done right away is write some automated tests to ensure that all input files are correctly formed before they allow them to corrupt their production database, and that the correct number of rows are actually in the file.
Watermelon projects? Green on the outside, but red on the inside?
one fully loaded engineer works about two thousand hours per year—that’s forty hours per week, fifty-two weeks per year, if they don’t take any vacation.
“It’s an archaic word, resurrected by Sensei Rich Hickey. ‘Complect’ means to turn something simple into something complex. “In tightly coupled and complected systems, it’s nearly impossible to change anything, because you can’t just change one area of the code, you must change one hundred, or even a thousand, areas of the code.
“Sensei Hickey would say, ‘Think of four strands of yarn that hang independently—that’s a simple system. Now take those same four strands of yarn and braid them together. Now you’ve complected them.’ Both configurations of yarn could fulfill the same engineering goal, but one is dramatically easier to change than the other. In the simple system, you can change one string independently without having to touch the others. Which is very good.” Erik laughs, “However, in the complected system, when you want to make a change to one strand of yarn, you are forced to change the other three strands
...more
bad?” Erik laughs. “There are many definitions, but my favorite is how it was originally defined by Ward Cunningham in 2003. He said, ‘technical debt is what you feel the next time you want to make a change.’ There are many things that people call technical debt, but it usually refers to things we need to clean up, or where we need to create or restore simplicity, so that that we can quickly, confidently, and safely make changes to the system. “Sometimes it’s a build and test system that doesn’t give fast feedback to developers, or when it stops working entirely,” he continues. “Sometimes it’s
...more
called the Three Ways and the Four Types of Work.”
“There are Five Ideals,”
the First Ideal of Locality and Simplicity. We need to design things so that we have locality in our systems and the organizations that build them. And we need simplicity in everything we do. The last place we want complexity is internally, whether it’s in our code, in our organization, or in our processes. The external world is complex enough, so it would be intolerable if we allow it in things we can actually control! We must make it easy to do our work.”
“The Second Ideal is Focus, Flow, and Joy. It’s all about how our daily work feels. Is our work marked by boredom and waiting for other people to get things done on our behalf? Do we blindly work on small pieces of the whole, only seeing the outcomes of our work during a deployment when everything blows up, leading to firefighting, punishment, and burnout? Or do we work in small batches, ideally single-piece flow, getting fast and continual feedback on our work? These are the conditions that allow for focus and flow, challenge, learning, discovery, mastering our domain, and even joy.”
The Third Ideal is Improvement of Daily Work. Reflect upon what the Toyota Andon cord teaches us about how we must elevate improvement of daily work over daily work itself. The Fourth Ideal is Psychological Safety, where we make it safe to talk about problems, because solving problems requires prevention, which requires honesty, and honesty requires the absence of fear. In manufacturing, psychological safety is just as important as physical safety. And finally, the Fifth Ideal is Customer Focus, where we ruthlessly question whether something actually matters to our customers, as in, are they
...more
The First Ideal—Locality and Simplicity The Second Ideal—Focus, Flow, and Joy
The Third Ideal—Improvement of Daily Work The Fourth Ideal—Psychological Safety The Fifth Ideal—Customer Focus
Every developer uses a common build environment. Every developer is supported by a continuous build and integration system. Everyone can run their code in production-like environments. Automated test suites are built to replace manual testing, liberating QA people to do higher value work. Architecture is decoupled to liberate feature teams, so developers can deliver value independently. All the data that teams need is put in easily consumed APIs …
A century ago, when mass production revolutionized industry, the role of the leader was to design and decompose the work and to verify that it was performed correctly by armies of interchangeable workers,
who were paid to use their hands, not their heads. Work was atomized, standardized, and optimized. And workers had little ability to improve the system they worked within. “Which is strange, isn’t it?” Erik muses. “Innovation and learning occur at the edges, not the core. Problems must be solved on the front-lines, where daily work is performed by the world’s foremost experts who confront those problems most often.
“The opposite of the Third Ideal is someone who values process compliance and TWWADI,” he says with a big smile. “You know, ‘The Way We’ve Always Done It.’ It’s the huge library of rules and regulations, processes and procedures, approvals and stage gates, with new rules being added all the time to prevent the latest disaster from happening again. “You may recognize them as rigid project plans, inflexible procurement processes, powerful architecture review boards, infrequent release schedules, lengthy approval processes, strict separation of duties … “Each adds to the coordination cost for
...more
“You know the joke—what’s the plural of ‘developer’?” says Maxine. “A ‘merge conflict.’”
Some are just using the Docker images on their laptops, some are using environments in vagrant, Git, or terraform configurations, simulating both Dev and test environments.
This is one of the great things about using Docker containers, Maxine thinks. Containers are immutable, unable to be changed after they’re created, so if it works in Dev, it will almost certainly work in Test.
Immutability is another concept from functional programming that is making the world a more predictable and safer place,
“Every developer knows that in the next interval, they need to write automated tests as they write the feature, not afterward.
The only rule is that you can’t say ‘I should have done X’ or ‘If I had known about that, I would have done Y.’ Hindsight is always perfect. In crises, we never actually know what’s reallyl going on, and we need to prepare for a future where we have an equally imperfect understanding of the world.”
“But remember, we’re doing this so that we can be better prepared for the next crisis, when we will be equally ignorant of entirely new things that are just as important and will be just as obvious in hindsight
“Just what does it take to build great products? And how can we as developers help?” “Where do I start?” Maggie says. “It usually begins with understanding who our customers are, both current and desired. Then we typically segment the customer base, so we know what set of problems each faces. Once we know that, we can understand which of those problems we want to solve, based on market size, ease of reaching them, and so forth. Once we know that, we think about pricing and packaging, offer development, and more strategic issues, such as the overall profitability of the product portfolio and
...more
Tuckman phases of teams, going through form, storm, norm, and perform.
proposing to build a Spark-like big data and compute platform, fed by an entirely new event-streaming bus, modeled closely to what the tech giants all have built to solve their data problems at scale. It would allow hundreds, even thousands, of CPU cores to be thrown at the computations, allowing analyses that currently take days or weeks to be done in minutes or hours. Maxine is familiar with these techniques. Their use exploded after the famous 2004 Google Map/Reduce research paper was published, which described the techniques Google used to massively parallelize the indexing of the entire
...more
Over the last week, Brent and his team have been stress-testing the entire system, routinely causing parts of the technology landscape to blow up. And then in a blameless post-mortem, they’d all work together to figure out how to fix things so that they’ll survive the actual launch. The results of these “Chaos Engineering” exercises resulted in some surprising things breaking.
“Like most businesses these days, we use the Net Promoter Score to rate customer satisfaction. We ask our customers, on a scale of 0 to 10, how likely it is that they would recommend our stores to their friends. The 9s and 10s are promoters, the 7s and 8s are neutrals, and the rest are detractors. To compute the score, we subtract the percentage of detractors from the percentage of promoters. An NPS score of thirty is considered good, and above fifty is great.
“A great question,” Erik says, standing. “The concepts of Horizons 1, 2, and 3 were popularized by Sensei Dr. Geoffrey Moore, who is most famous for his book Crossing the Chasm, which introduced the customer adoption curve into modern business planning. He observed that customer adoption is a Gaussian distribution curve, naming them the innovators, early adopters, early majority, late majority, and laggards. And yet, as brilliant as this was, I think he will be best-known for the Four Zones, which help us better organize ourselves to win in all Three Horizons.
Horizon 2 lines of business are so important, because they represent the future of the company. They may introduce the company’s capabilities to new customers, adjacent markets, or with different business models. These endeavors may not be profitable, but this is where we find higher-growth areas. It is from here that enterprising leaders create the next generation of Horizon 1 businesses.
“You may have guessed that Horizon 2 efforts come from Horizon 3, where the focus is on velocity of learning and having a broad pool of ideas to explore,” he says. “Here, the name of the game is to prototype ideas and to answer as quickly as possible the three questions of market risk, technical risk, and business model risk: Does the idea solve a real customer need? Is it technically feasible? And is there a financially feasible engine of growth? If the answer is no to any of them, it’s time to pivot or kill the idea.
“Horizon 1 thrives on process and consistency, on rules and compliance, and on bureaucracies, which create extraordinary resilience. These are the mechanisms that allow greatness to be consistently delivered over decades. “In contrast, in Horizon 3, you must go fast, you must be constantly experimenting, and you must be allowed to break all the rules and processes governing Horizon 1,”
Sensei Dr. Steven Spear observed that for each day you can get to market faster, you can often capture upwards of millions of dollars of additional revenue. If you’re first to market, you will capture fifty percent of the revenue that the entire product category will ever yield. Second place will capture twenty-five percent, and third place will get fifteen percent. For any later entrants, it will surely have been a complete waste of time and money.
thinking of the Hoare principle: “There are two ways to write code: write code so simple there are obviously no bugs in it, or write code so complex that there are no obvious bugs in it.”
Google Map/Reduce paper
“Long term, we don’t want to manage our dependencies, we want to eliminate them,” she says. “That’s the system of work and the company architecture we need to create, which means fewer project managers.
“Last time we met, I mentioned Sensei Geoffrey Moore’s Three Horizons, but I didn’t have time to explain his concept of Core versus Context, which are what the Four Zones are about,” Erik says. “Sensei Moore observed that many businesses understand the Three Horizons but are still unable to properly invest in the next generation of innovation. In other words, they underinvest in Core, because they are being controlled by Context. “Cores are the central competencies of the organization. These are things that customers are willing to pay for and what investors reward,” he says. “Context is
...more
“A hundred years ago, most large factories had a CPO—a chief power officer—who ran the electricity generation processes. It was one of the most important roles in manufacturing, because no electricity, no production. It was a Core process,” he says. “But that role has disappeared entirely. Electricity has become infrastructure that you buy from a utility company. It is interchangeable, and you choose suppliers primarily on price. There is rarely a competitive advantage to generating your own power. It is now merely Context, no longer Core. You don’t want to be the organization that has a large
...more
“As Sensei Clay Christiansen once stated, one keeps what is ‘not good enough’ and outsources what is ‘more than good enough,’” he says. “Why did you choose to outsource your cafeteria point-of-sale system?”
simplicity enables effectiveness, and that complexity conspires against it.
At first, each department manager divided their people into three categories: critical, desired, and RIF. Of course, only a few people ended up on the third list. Seeing the three names, it was clear that managers were using this as an opportunity to get rid of people that should have been eased out long ago.
“HIPPO effect” (or Highest Paid Person’s Opinion),
They used a technique called Wardley Maps to better localize what parts of various value chains were commodities and should be outsourced, which should be purchased, and which should be kept in-house because they created durable, competitive advantage. They used this exercise to methodically disposition their technology stacks, given the business context.
It is now at the center of Project Shamu, forming the foundation of a massive architectural change that will eventually touch almost every back-end service and API across the entire company. Maxine knows that this is one of the most important technology initiatives in the company, because it solves something that has been bothering her for over a year. In the first Unicorn mini-launch, the transportation options service took
down the entire order funnel. It was just one of twenty-three deeply nested API calls made whenever someone checked for product availability. Even after a year, this problem remained unfixed. It was simply too expensive to shore up all twenty-three APIs to be a Tier 1 service—an SLA that requires five-nines of uptime, guaranteed response within ten milliseconds, and all sorts of other things that cost tons of money. What’s always bothered her was why twenty-three API calls were needed in the first place, why they had to respond within milliseconds, and why they had to be so expensive to run.
...more
“It’s no longer twenty-three API calls that all must be available and respond quickly. Instead, it’s just one API call that takes a product ID and zip code, and returns the shipping options and delivery times without having to compute anything,” she would say.
FAANGs: the Facebooks, Apples, Amazons, Netflixes, and Googles,”
