Tools and Weapons: The Promise and the Peril of the Digital Age

by Brad Smith

It’s important that this work builds on existing ethical and human rights traditions. I’ve been impressed by the US military’s deep and long-standing focus on ethical decision-making. This hasn’t freed the armed forces from ethical lapses or, at times, enormous mistakes, but, as I learned from individuals ranging from senior generals to a West Point cadet, one can’t graduate from an American military academy without taking a course in ethics.29 The same is not yet true for computer science majors at many American universities.

We’ll also need to see more focus on ethics in computer and data science courses themselves. This might take the form of a focused course, or it might become an element in almost every course. Or both.

“Could we see a Hippocratic oath for coders like we have for doctors?” We joined others in suggesting that such an oath could make sense.

Ultimately, a global conversation about ethical principles for artificial intelligence will require an even bigger tent. There will need to be seats at the table not only for technologists, governments, NGOs, and educators, but for philosophers and representatives of the world’s many religions. The need for this global conversation brought us to one of the last places I’d expect to talk about technology: the Vatican.

When we pulled up to the Vatican, we were met by a beaming Monsignor Vincenzo Paglia, a snowy-headed, jovial archbishop of the Italian Catholic Church. The author of numerous books, he leads the Vatican’s work addressing a variety of ethical issues, including new challenges relating to artificial intelligence. Microsoft and the Vatican had decided to cosponsor a doctoral dissertation award to explore the intersection between this emerging technology and long-standing ethical questions.

The two books together illustrate how science and technology can connect or collide with matters of faith, religion, and philosophy. As with inventions like the printing press and the telescope, it’s impossible to imagine that AI will leave these fields untouched. The question is how to promote a thoughtful, respectful, and inclusive global conversation.

The Pope then reminded me of what Einstein had said after the Second World War: “I know not with what weapons World War III will be fought, but World War IV will be fought with sticks and stones.”32 Einstein’s point was that technology, specifically nuclear technology, had progressed to the point where it could annihilate everything else.

As we left the meeting, Pope Francis shook my hand with his right hand and held my wrist with his left. “Keep your humanity,” he urged.

As we think about the future of artificial intelligence, it’s good ...

More than fifteen years later, this approach to law enforcement happily seems far-fetched. But today, one aspect of Minority Report seems to be on track to arrive much earlier than 2054. As Cruise is on the run, he walks into the Gap. The retailer has technology that recognizes each entering customer and immediately starts displaying on a kiosk the images of clothes it believes the customer will like. Some people might find the offers attractive. Others might find them annoying or even creepy. In short, entering a store becomes a bit like we sometimes feel after browsing the web and then turning to our social media feed only to find new ads promoting what we just viewed.

What we were experiencing reflected this view and went even further. In the tech sector, some employees wanted to play an active role in shaping their companies’ decisions and engagement on the issues of the day. Perhaps not surprisingly, this view was more pronounced at a time when people had less trust in governments. Employees were looking to another institution they hoped might do the right thing and have some influence on public outcomes.

Facial recognition, like so many AI-based technologies, improves with larger quantities of data. This creates an incentive to do as many early deals as possible and hence the risk of a commercial race to the bottom, with tech companies forced to choose between social responsibility and market success.

We believed that legislation could address three key issues—the risk of bias, privacy, and the protection of democratic freedoms.

Our point was not that new regulations should prohibit all such technology. To the contrary, we are among the companies working to help stores responsibly use technology to improve the shopping experience.

The need for government leadership does not absolve technology companies of our own ethical responsibilities. Facial recognition should be developed and used in a manner consistent with broadly held societal values.

There is no crystal ball. If anything, it’s too easy for a tech leader to lay claim as a great “futurist” and offer seemingly confident and even grandiose predictions about what the world will look like in a decade or two. People will undoubtedly listen, and the good news, if you pursue this approach, is that few are likely to remember a decade from now what you’ve said. And even if you’re completely wrong, there’s plenty of time for a course correction.

The first is to understand what AI can and cannot do well and appreciate how this will impact jobs and work. To state the obvious, AI will most readily replace jobs that involve functions that it can perform well. It makes sense to consider the recent advances that have enabled AI to understand human speech, recognize images, translate languages, and reach new conclusions based on an ability to discern patterns. If a large part of a job involves tasks that can be completed by AI—and faster—then that job is probably at risk of being replaced by a computer.

In an important way, this is the other side of the coin. There are certain tasks that AI likely won’t perform well. Many of these involve soft skills such as collaboration with other people, which will remain fundamental in organizations large and small. As Rick recognized, this often requires meetings (hopefully well-planned ones). AI similarly is unlikely to excel in providing the empathy required of nurses, counselors, teachers, and therapists. Each of these individuals will likely use AI for some tasks, but it seems unlikely that it could replace their work entirely.

One such opportunity arose in the spring of 2017 when I visited Microsoft’s subsidiary in the United Kingdom and we hosted a visit by Prime Minister Theresa May. As I stood next to Cindy Rose, the CEO of our UK business, we both held our breath a bit as we watched a young apprentice place a HoloLens headset on the prime minister’s head. We exhaled when the prime minister moved briskly through an augmented-reality demonstration of how the device could be used to identify faults in sophisticated machinery. (As it turned out, the HoloLens was far easier to master than devising a negotiating strategy for Brexit.)

After the demo, Prime Minister May took off the headset and turned to our apprentice to ask him about his job. He replied proudly, “I’m an envisioning adviser. I help customers envision how they can take new technology like augmented reality and use it inside their company.” “An envisioning adviser,” the prime minister repeated. “That’s a job I’ve never heard of.”

As was the case for generations past, there will be days when we feel the need for an updated dictionary to understand what people are describing.

When he said that AI would soon replace human interpreters, he paused for a moment, realized what he had said, and turned to the interpreter. “Sorry,” he said. The interpreter didn’t skip a beat. “Don’t worry,” she calmly replied. “Someone from IBM told me the same thing twenty years ago, and I’m still here.”

If anything, people tend to be too optimistic, predicting that change will arrive faster than is the case, but as Bill Gates has famously remarked, “We always overestimate the change that will occur in the next two years and underestimate the change that will occur in the next ten.”15

This all leads to an interesting question. When New Yorkers saw the first automobile roll down the street in the nation’s financial capital, how many predicted that the invention would lead to the creation of new jobs in the financial sector? The route from the combustion engine to consumer credit was indirect and unfolded over time, bolstered in no small part by other intervening inventions and business processes such as Henry Ford’s assembly line, which made possible the mass production and thus the cheaper and broader availability of automobiles.

While other American companies are present in China, only Apple with its iPhone has enjoyed success in the country on a level that is comparable to its leadership in the rest of the world. In recent years, Apple has earned three times as much revenue as Intel, which is the number two US tech company in China.6 When it comes to profits, the situation is likely starker. Apple may well generate more profits within China than the rest of the American tech sector put together. It’s a notable accomplishment but also a challenge for the company, given China’s large contribution to Apple’s global profitability. As we’ve found at Microsoft over time and more globally with products like Windows and Office, anytime you’re dependent on a particular source for a large share of revenue or profitability, it makes it difficult to contemplate changes in that area. It also explains why Apple’s leaders are such frequent visitors to Beijing.

One example is a product called XiaoIce, a female AI-based social chatbot designed to have conversations with teenagers and people in their early twenties.8 The chatbot seems to have filled a social need in China, with users typically spending fifteen to twenty minutes talking with XiaoIce about their day, problems, hopes, and dreams. Perhaps she fills a need in a society where children don’t have siblings? This social chatbot has grown to serve more than six hundred million users, and her capabilities are growing, including AI-based applications to compose poems and songs. XiaoIce has become a celebrity of sorts; she has made a guest appearance as a television weather forecaster and regularly hosts TV and radio programs.9

Much more notable has been the triumph of Chinese services like Alibaba over Amazon in e-commerce, Tencent’s WeChat over American services in messaging, and Baidu over Google in search. In important and well-documented respects, these services innovated to meet Chinese tastes in ways that their American counterparts did not.

As Master Xianxin explained to us, the monastery had devoted itself to merging Buddhist teachings and traditions with the modern world. He was a graduate of the Beijing University of Technology. Yes, a Buddhist monk with a computer science degree. He showed off thousands of volumes of ancient Buddhist literature the temple was digitizing with the help of AI. The master went on to share how the monks were using machine-based translation techniques to share their work in sixteen languages with people around the world. Modern technology was advancing some of the world’s most ancient teachings.

Tech companies have worked together and with the global human rights community to encourage adherence to global principles relating to privacy and free expression. But on some days these principles receive a less global endorsement than when they were endorsed by the world’s governments, including the Chinese, shortly after the end of World War II.

The US government understandably reacted strongly not only to incidents such as the hacking of the OPM but also to reports that Chinese hardware manufacturer Huawei had built routers that enabled the Chinese government to monitor communications by customers who used them.18 The shoe then was on the other foot when the Snowden disclosures included a photo of US personnel tampering with Cisco routers to achieve the same thing.19 Both companies have been working since—with less than complete success—to recover their reputations in the other’s market.

There are also some technology scenarios that clearly raise human rights concerns while others do not. Facial-recognition services and citizen and consumer data stored in the cloud are two that do.

In most respects, this limitation is the strongest in the United States. Consider the fact that President Xi’s education included reading American authors from Alexander Hamilton to Ernest Hemingway. How many American politicians have read comparable Chinese authors? With more than twenty-five hundred years of rich history, the problem is not a lack of supply but a shortage of interest. As history has demonstrated repeatedly, if the United States is going to navigate global challenges, it will need leaders who understand the world.

As we met with members of Congress in Washington, DC, a few senators mentioned that they had read the advance galleys sent to them for a new book called AI Superpowers. Its author, Kai-Fu Lee, is a former executive at Apple, Microsoft, and Google. Born in Taiwan, he is now a leading venture capitalist based in Beijing. His argument is sobering. He asserts that “the AI world order will combine winner-take-all economics with an unprecedented concentration of wealth in the hands of a few companies in China and the United States.”1 As he puts it, “other countries will be left to pick up the scraps.”2 What’s the basis for this view? Mostly it comes down to the power of data. The argument is that the firm that gains the most users will gain the most data, and because data is rocket fuel for AI, its AI product will become stronger as a result. With a stronger AI product, the firm will attract even more users and hence more data. The cycle will continue with returns to scale, so that eventually this firm will crowd out everyone else in the market. According to Kai-Fu, “AI naturally gravitates toward monopolies . . . once a company has jumped out to an early lead, this kind of ongoing repeating cycle can turn that lead into an insurmountable barrier to entry for other firms.”3

The concept is a common one in information technology markets. It’s referred to as “network effects.” It has long been true in the development of applications for an operating system, for example. Once an operating system is in a leadership position, everyone wants to develop apps for it. While a new operating system might emerge with superior features, it’s difficult to persuade app developers to consider it. We benefited from this phenomenon in the 1990s with Windows and then hit the barrier on the other side twenty years later, competing against the iPhone and Android with our Windows Phone. Any new social media platform that wants to take on Facebook encounters the same problem today. It’s part of what defeated Google Plus.

While the Hutch has access to important collections of health record data that help it pursue AI-based cancer research, in no way does it possess the world’s largest data sets. Like most organizations and companies, if the Hutch is to continue to lead into the future, it must compete without actually owning all the data that it will need.

As Satya put it during one of our Friday senior leadership team meetings at Microsoft, data is probably “the world’s most renewable resource.”

Trunnell spoke about his vision for a data commons that would enable multiple cancer research institutes to share their data in new ways.

His vision would bring together several organizations to pool their data in partnership with a tech company.

My enthusiasm grew as I listened to his presentation. In many ways, the challenge was like many others we had learned about and even experienced ourselves. As Trunnell described his plans, it reminded me of the evolution of software development. In the early days of Microsoft’s history, developers protected their source code as a trade secret, and most tech companies and other organizations developed their code by themselves. But open source had revolutionized the creation and use of software. Increasingly software developers were publishing their code under a variety of open-source models that allowed...

Those lessons were several: Don’t go as deep as the Clinton team had gone in building a data operation from the ground up. Instead use one of the major commercial tech platforms and focus on building on top of it.

Build with a broader federated ecosystem that brings together as many partners as possible to contribute and share data, as the RNC had done. Use this approach to focus resources on differentiated capabilities that can run on top of a commercial platform, like those developed by Parscale. And never assume that your algorithms are as good as you believe. Instead test and refine them constantly.

In addressing all these issues, the open-data movement can take a page from open-source trends for software. At first that effort was hampered by questions about license rights. But over time standard open-source licenses emerged. We can expect similar efforts for data.

As we keep working to bring more technology to humanity, we also need to bring more humanity into technology.

It’s unrealistic to expect the pace of technological change to slow. But it’s not too much to ask that we do more to manage this change. In contrast to prior technology eras and inventions such as the railroad, telephone, automobile, and television, digital technology has progressed for several decades with remarkably little regulation—or even self-regulation. It’s time to recognize that this hands-off attitude needs to give way to a more activist approach that addresses evolving challenges in a more assertive way.

A more active approach doesn’t mean that everything should be left to governments and regulation. That would be as shortsighted and unsuccessful as asking governments to do nothing at all. To the contrary, this needs to start with individual companies and with more collaborative work across the tech sector.

There were days when some engineers argued that we should instead keep on fighting. At times, I almost felt that they were calling my courage into question. While there were times when we needed to stand our ground, there were many moments when I argued that it took more bravery to compromise than it did to keep fighting. And it took persistence as well. The quest for common ground often led to negotiations that ended in impasse and failure before we could come back together and reach an agreement. We needed to develop the ability to fail gracefully, complimenting the other side even when things fell apart so we could preserve the ability to work through the hard problems again when the right moment arrived. It almost always did.

These concerns are important. But given the role that technology now plays in the world, it’s equally dangerous for a tech company to move faster than the speed of thought, or simply to fail to think at all about the broader implications of its services or products. One thesis of this book is that it is more than possible for companies to succeed while doing more to address their societal responsibilities. As Satya is quick to point out when such issues arise, we need to move fast but with some guardrails on our technology. An ability to anticipate issues and define a principled approach to address them is more likely to keep “the car on the road” as it gains speed. It helps avoid at least some of the public controversies and potential reputational damage that can force executives to spend more time on issues other than product development and user growth.

For one, there’s a strong case for governments to innovate in the regulatory space in a way that’s like innovation in the tech sector itself. Instead of waiting for every issue to mature, governments can act more quickly and incrementally with limited initial regulatory steps—and then learn and take stock from the resulting experience. In other words, take the concept of a “minimum viable product” and consider the type of approach we advocated for AI and facial recognition, described in chapter twelve. We readily recognized that just as for a new business or software product, the first regulatory step would not be the last, but we believed it was wiser for governments to take a series of more limited steps more quickly.

Is this an approach that can work in certain areas for technology regulation? If so, it could become a new regulatory tool for our time. If governments can adopt limited rules, learn from the experience, and subsequently use this learning to add new regulatory provisions much as companies add new features to products, it could put laws on a path to move faster.

Having worked in the technology sector for more than a quarter century, I realize that the products are complex. But so are contemporary commercial airplanes, automobiles, skyscrapers, pharmaceuticals, and even food products. You don’t hear any serious suggestion that the Federal Aviation Administration should leave aircraft unregulated because they are too complicated for people in government to understand.