WTF?: What's the Future and Why It's Up to Us

by Tim O'Reilly

And that gets me to the third characteristic of true unicorns: They create value. Not just financial value, but real-world value for society.

History tells us technology kills professions, but does not kill jobs.

We are increasingly facing what planners call “wicked problems”—problems that are “difficult or impossible to solve because of incomplete, contradictory, and changing requirements that are often difficult to recognize.”

In the 1980s, though, the idea that “greed is good” took hold in the United States and we turned away from prosperity. We accepted the idea that what was good for financial markets was good for everyone and structured our economy to drive stock prices ever higher, convincing ourselves that “the market” of stocks, bonds, and derivatives was the same as Adam Smith’s market of real goods and services exchanged by ordinary people. We hollowed out the real economy, putting people out of work and capping their wages in service to corporate profits that went to a smaller and smaller slice of society.

How can a business create more value for society than it captures for itself?

PCH International, a design and logistics company for the electronics industry, from offices in Shenzhen, San Francisco, and Ireland. PCH provides direct-to-consumer just-in-time manufacturing for consumer electronics vendors, allowing them to take orders online that are shipped directly from China.

This is a central pattern of the Internet age: More freedom leads to more growth.

Google and Amazon are both fiercely committed to creating value for one side of the marketplace—users—and justify their actions to themselves on that basis. But as they replace more and more of the supplier side of the network with their own services, they risk weakening the marketplace as a whole. After all, someone else invented and invested in those products or services that they are copying. This is why antitrust law can’t just use lower costs for consumers as its primary benchmark, rather than the overall level of competition in the market. Lower costs are only one outcome of competition. Innovation withers when only one party can afford to innovate, or when there’s only one place to bring new products to market. The mental map used by regulators shapes their decisions, and thus the future.

Simple, decentralized systems work better at generating new possibilities than centralized, complex systems because they are able to evolve more quickly. Each decentralized component within the overall framework of simple rules is able to seek out its own fitness function. Those components that work better reproduce and spread; those that don’t die off.

“Lock-in” comes because others depend on the benefit from your services, not because you’re completely in control.

That was followed up by a slide titled “History Lesson,” which ended on the talk’s punch line: “A platform strategy beats an application strategy every time!”

Basically what we decided to do is build a [set of APIs] between those two layers so that you could just do coarse-grained coordination between those two groups.” (That is, “small pieces loosely joined.”) This is important: Amazon Web Services was the answer to a problem in organizational design.

Amazon Chief Technology Officer Werner Vogels described it in a 2006 blog post: “Services do not only represent a software structure but also the organizational structure. The services have a strong ownership model, which combined with the small team size is intended to make it very easy to innovate. In some sense you can see these services as small startups within the walls of a bigger company. Each of these services require a strong focus on who their customers are, regardless whether they are externally or internally.”

Work is done by small teams. (Amazon famously describes these as “two-pizza teams,” that is, teams small enough to be fed by two pizzas.) These teams work independently, starting with a high-level description of what they are trying to accomplish. Any project at Amazon is designed via a “working backwards” process. That is, the company, famous for its focus on the customer, starts with a press release that describes what the finished product does and why. (If it’s an internal-only service or product, the “customer” might be another internal team.) Then they write a “Frequently Asked Questions” document. They create mock-ups and other ways of defining the customer experience. They go so far as to write an actual user manual, describing how to use the product. Only then is the actual product green-lighted. Development is still iterative, informed by additional data from actual users as the product is built and tested, but the promise of the final product is where everything starts.

Of course, writing the press release, or providing the picture that shows you the result of following the recipe, is only one part of what it takes to build a promise-centered organization. You have to work backward from the promise to the customer to the promises that each part of the organization needs to make to each other in order to fulfill it. The small teams are also a part of this approach. As is the design of a single, clearly-defined “fitness function” for each team (the one thing it promises to deliver, that can be measured and continuously improved).

There’s a bit of a paradox here. Jeff was really asking for more effective, close communication within teams, coupled with highly structured communication between teams, mirroring the highly structured communication that makes it possible for modern Internet applications to work so well. He was arguing against the kind of backdoor communication that leads to messy workarounds that, over time, break under their own weight.

Promise theory, as Burgess outlines it, is a framework for understanding how independent actors make promises to each other—the essence of that highly structured communication. Those actors can be software modules promising to respond in a certain way to an API call, or small teams promising to deliver a particular result. Burgess writes: “Imagine a set of principles that could help you to understand how parts combine to become a whole, and how each part sees the whole from its own perspective. If such principles were any good, it shouldn’t matter whether we’re talking about humans in a team, birds in a flock, computers in a datacenter, or cogs in a Swiss watch. A theory of cooperation ought to be pretty universal, so we could apply it both to technology and to the workplace.”

Kim explained to me that “writing the press release first is a mechanism to make customer obsession concrete.” As are two-pizza teams producing services with hardened APIs. “Amazon does a better job of creating these kinds of mechanisms for its corporate values than any other company I’ve seen,” Kim added. “And it starts from first principles (values) more than other companies as well.”

I still remember the wonder with which Mark Lucovsky, who’d been a senior engineering leader at Microsoft, described how different his process became when he moved to Google. “I make a change and roll it out live to millions of people at once.” Mark was describing a profound transformation in how software development works in the age of the cloud. There are no more gold masters. Today software has become a process of constant, more or less incremental improvements. From the point of view of the company offering an online service, software has gone from being a thing to a process, and ultimately, a series of business workflows. The design of those workflows has to be optimized not just for the creators of the software but for the people who will keep them running day-to-day.

As I spoke to the staff at Amazon, I reminded them that the application wasn’t just software, but contained an ever-changing river of content produced by their network of suppliers, enhanced by reviews, ratings, and other contributions from their vast network of customers. Those inputs were then formatted, curated, and extended by their own staff in the form of editorial reviews, designs, and programming. And that dynamic river of content was managed, day in and day out, by all of the people who worked for Amazon. I remember saying “All of you—programmers, designers, writers, product managers, product buyers, customer service reps—are inside the application.”

“In the future, being a developer on someone’s platform will mean being hosted on their infrastructure.” For example, she talked about the competitive advantage she was developing by locating her data centers where power was cheap.

But DevOps also brings higher reliability and better responsiveness to customers. Gene Kim characterizes what happens in a high-performance DevOps organization: “Instead of upstream Development groups causing chaos for those in the downstream work centers (e.g., QA, IT operations, and Infosec), Development is spending twenty percent of its time helping ensure that work flows smoothly through the entire value stream, speeding up automated tests, improving deployment infrastructure, and ensuring that all applications create useful production telemetry.” He echoes the theme that it isn’t just a technical but an organizational practice: “Everyone in the value stream shares a culture that not only values each other’s time and contributions but also relentlessly injects pressure into the system of work to enable organizational learning and improvement.”

These programs are workers, and the programmers who create them are their managers. Each day, these “managers” take in feedback about their workers’ performance, as measured in real-time data from the marketplace, and if necessary, they give feedback to the workers in the form of minor tweaks and updates to the program or the algorithm.

While this paper was focused on language translation, it summed up the approach that has been essential to the success of Google’s core search service. Its insight, that “simple models and a lot of data trump more elaborate models based on less data,” has been fundamental to progress in field after field, and is at the heart of many Silicon Valley companies. It is even more central to the latest breakthroughs in artificial intelligence.

There’s an entirely new field of regulatory technology, or RegTech, that uses software tools and open data for regulatory monitoring, reporting, and compliance.

Labor advocates point out that the new on-demand jobs have no guaranteed wages, and hold them in stark contrast to the steady jobs of the 1950s and 1960s manufacturing economy that we now look back to as a golden age of the middle class. Yet if we are going to get the future right, we have to start with an accurate picture of the present, and understand why those jobs are growing increasingly rare. Outsourcing is the new corporate norm. That goes way beyond offshoring to low-wage countries. Even for service jobs within the United States, companies use “outsourcing” to pay workers less and provide fewer benefits. Think your hotel housekeeper works for Hyatt or Westin? Chances are good they work for Hospitality Staffing Solutions. Think those Amazon warehouse workers who pack your holiday gifts work for Amazon? Think again. It’s likely Integrity Staffing Solutions. This allows companies to pay rich benefits and wages to a core of highly valued workers, while treating others as disposable components. Perhaps most perniciously, many of the low-wage jobs on offer today not only fail to pay a living wage, but they provide only part-time work.

While Buffett believes that companies are spending the money on buybacks because they don’t see opportunities for productive capital investment, Fink points out that for long-term growth and sustainability, companies have to invest in R&D “and, critically, employee development and long-term financial well-being.” Rejecting the idea that companies or the economy can prosper solely by boosting short-term returns to shareholders, he continues: “The events of the past year have only reinforced how critical the well-being of a company’s employees is to its long-term success.”

Mistaking what is good for financial markets for what is good for jobs, wages, and the lives of actual people is a fatal flaw in so many of the economic choices business leaders, policy makers, and politicians make.

It isn’t Wall Street per se that is becoming hostile to humanity. It is the master algorithm of shareholder capitalism, whose fitness function both motivates and coerces companies to pursue short-term profit above all else. What are humans in that system but a cost to be eliminated?

As I remember the talk, its central argument went something like this: “I’m a successful capitalist, but I’m tired of hearing that people like me create jobs. There’s only one thing that creates jobs, and that’s customers. And we’ve been screwing workers so long that they can no longer afford to be our customers.”

These companies, often sold for billions of dollars, are valued not based on some multiple of their sales, profits, or cash flows, but for expectations of what they might become, promoted like fake news in a market of attention. This effect is central to understanding the hypnotic allure of financialization.

A company that has been financialized—that is, is valued in supermoney—has a huge advantage over companies that are operating solely in the market of real goods and services.

If you have access to supermoney, you can operate for years at a loss. This is one reason—not just the superior customer benefits and economic efficiencies of their technology or business model—that Internet companies can disrupt older, less highly valued companies.

As we saw in Chapter 11, stock options, which have played such a large role in Silicon Valley wealth, have also become a key part of the problem of income inequality. Even though Silicon Valley companies are actually better than many other companies at distributing the gains because they offer options to virtually every employee, those options are still overwhelmingly weighted toward founders and top management, with each lower rank of workers typically receiving a full order of magnitude less in value. This may or may not be appropriate, based on actual contribution to the business, but the net-net is that a huge proportion of the productivity gains we’ve seen in the past decades have increasingly gone to a small group of managers rather than to all workers. And when the market gets excited, those people get paid in a currency that appreciates at a rate far in excess of anything possible in the real economy.

Another effect of stock-based compensation is that it requires companies to keep growing, encouraging them to aim for complete market dominance and value capture. As long as employees are paid in stock, even founders with voting control over their companies still face pressure from “the market” to keep their earnings rising and their stock price increasing.

The amount of money spent on stock-based compensation has for too long not been properly reported by many technology companies. Amazon and Facebook only began reporting their stock-based compensation as part of their regular financials using Generally Accepted Accounting Principles (GAAP) rather than via special “non-GAAP” reporting in their quarterly reports in the first quarter of 2016. Twitter, less profitable than the others, still doesn’t do so because it would show that far from having a profit, it is actually still operating at a loss when stock-based compensation is taken into account.

Meanwhile, venture investors have come to have much the same risk profile as investment bankers, where gains are private but losses are socialized. Venture capitalists typically get paid a percentage of the fund as an annual management fee—usually 2%—so a VC firm with a billion-dollar fund will have taken out $200 million in proceeds over the ten-year life of the fund even if the firm loses money for its limited partners (the investors who put up the vast majority of its capital). Put another way, the $58.8 billion in venture capital investment in 2015 paid out nearly $1.2 billion to venture capitalists that year whether or not their investments ever succeed. This encourages VCs to raise larger funds and to deploy ever-larger amounts of capital even though evidence shows that smaller funds typically deliver better results.

Sharing rather than hoarding knowledge can also be a powerful lever for competitive advantage. Companies too often assume that the best way to increase their share of the gains from innovation is to keep it proprietary. Yet as the open source pioneers of Linux and the Internet taught us, knowledge compounds when it is shared.

Keynes didn’t think so in 1930, and I don’t think so now. “We are suffering just now from a bad attack of economic pessimism,” he wrote. “It is common to hear people say that the epoch of enormous economic progress which characterised the nineteenth century is over; that the rapid improvement in the standard of life is now going to slow down; that a decline in prosperity is more likely than an improvement in the decade which lies ahead of us. I believe that this is a wildly mistaken interpretation of what is happening to us. We are suffering, not from the rheumatics of old age, but from the growing-pains of over-rapid changes, from the painfulness of readjustment between one economic period and another” (italics mine).

The chorus of doubt about the jobless future sounds remarkably similar to the one that warned of the death of the software industry due to open source software. Clayton Christensen’s Law of Conservation of Attractive Profits holds true here too. When one thing becomes commoditized, something else becomes valuable. We must ask ourselves what will become valuable as today’s tasks become commoditized.

It was language that was our greatest invention, the ability to pass fire from mind to mind. In periods where knowledge is embraced and widely shared, society advances and becomes richer. When knowledge is hoarded or disregarded, society becomes poorer.

One key to understanding the future is to realize that as prior knowledge is embedded into tools, a different kind of knowledge is required to use it, and yet another to take it further. Learning is an essential next step with each leap forward in augmentation.

That was certainly true of me. I studied Greek and Latin in college. Everything I learned about computers, I learned on the job. The knowledge I learned in college was useless to me. The habits of mind that I formed were what mattered, the foundational skills of study, and particularly the ability to recognize patterns. The struggle to parse complex Greek texts that were, quite frankly, beyond my skill in the language was great preparation when I took on the challenge of documenting programs written in programming languages that at first I barely understood. It is not just knowledge that we have to teach, it is the ability to learn. To learn constantly. Over the course of my career, learning itself has been the most important part of my ongoing work.

Jeff continued with four tips for staving off Day 2: “customer obsession, a skeptical view of proxies, the eager adoption of external trends, and high-velocity decision making.” Customer obsession is the key to the WTF? of delight: “Even when they don’t yet know it,” Jeff wrote, “customers want something better, and your desire to delight customers will drive you to invent on their behalf.” Whether you’re in business or public policy, don’t settle for rehashing tired solutions. Keep looking for that positive astonishment that means you’ve accomplished something wonderful for the people you serve. Jeff continued: “Staying in Day 1 requires you to experiment patiently, accept failures, plant seeds, protect saplings, and double down when you see customer delight.”

Even without doing a scenario-planning exercise, asking yourself “What happens if this goes on?” is a great way to prepare for the future—and to spot entrepreneurial opportunities.