More on this book
Community
Kindle Notes & Highlights
Unique among security strategies (at least until now), nuclear deterrence rests on a series ...
This highlight has been truncated due to consecutive passage length restrictions.
The Soviets reportedly explored the use of a system designed to be capable, once switched on by human users, of detecting an incoming nuclear strike and disseminating launch orders for a counterattack without further human intervention — an
nuclear war between major powers would involve irreversible decisions and unique risks for victor, vanquished, and bystanders alike.
Nuclear weapons presented policy makers with two persistent related riddles: how to define superiority and how to limit inferiority.
When a participant in the system enhances its power disproportionately over others, the system will attempt to adjust — either through the organization of countervailing force or the accommodation of a new reality.
This transformation includes so‑called cyber weapons, a class of weapons involving dual-use civilian capabilities so that their status as weapons is ambiguous.
Traditionally, parties to a conflict had no difficulty recognizing that a clash had occurred, or recognizing who the belligerents were. Opponents calculated rivals’ capabilities and assessed the speed with which their arsenals could be deployed.
The comparatively low cost of cyber capabilities and operations, and the relative deniability that some cyber operations may provide, has encouraged some states to use semiautonomous actors to perform cyber functions.
The most effective attacks have usually been those that occur (often without immediate recognition or formal acknowledgment) below the threshold of traditional definitions of armed conflict.
How does one develop a strategy — offensive or defensive — for something that perceives aspects of the environment that humans may not,
some cases, exceed the pace or range of human thought?
An algorithm knows only its instructions and objectives, not morale or doubt.
AI can also be used defensively, locating and repairing flaws before they are exploited. But since the attacker can choose the target, AI gives the party on offense an inherent if not insuperable advantage.
In contrast to the field of nuclear weapons, no widely shared proscription and no clear concept of deterrence (or of degrees of escalation) attend such uses of AI.
A commercial innovation by one society could be adapted for security or information-warfare purposes by another.
In cases where systems and capabilities are altered easily and relatively undetectably by a change in computer code, each major government may assume that its adversaries are willing to take strategically sensitive AI research, development, and deployment one step further than what they have publicly acknowledged or even privately pledged.
Likewise, smaller nations that do not possess nuclear weapons and have limited conventional weapons capability have the capacity to wield outsize influence by investing in leading-edge AI and cyber arsenals.
For nations, disconnection could become the ultimate form of defense.
Such limitations will have only limited meaning if they are adopted only unilaterally — by one nation or a small group of nations. Governments
advanced countries should explore the challenges of mutual restraint supported by enforceable verification.
Three qualities have traditionally facilitated the separation of military and civilian domains: technological differentiation, concentrated control, and magnitude of effect.
The railroads that delivered goods to market were the same that delivered soldiers to battle — but they had no destructive potential. Nuclear technologies are often dual-use and may generate tremendous destructive capacity, but their complicated infrastructure enables relatively secure governmental control. A hunting rifle may be in widespread use and possess both military and civilian applications, but its limited capacity prevents its wielder from inflicting destruction on a strategic level.
If a state has the means, it may elect to respond nearly simultaneously, before the attack can occur fully, constructing an AI‑enabled system to scan for attacks and empowering it to counterattack.
the compulsion to act first may overwhelm the need to act wisely — as
Unlike nuclear weapons, AIs are hard to track: once trained, they may be copied easily and run on relatively small machines. And detecting their presence or verifying their absence is difficult or impossible with the present technology.
But neither restricting AIs’ underlying capabilities nor restricting their number would be wholly compatible with the technology’s widespread civilian use and continual evolution.
The dilemma posed by AI‑related weapons technology is that keeping up research and development is essential for national survival;
without it we will lose commercial competitiveness and relevance.
Each society that is advancing the frontiers of AI should aim to convene a body at a national level to consider the defense and security aspects of AI and bridge the perspectives of the varied sectors that will shape AI’s creation and deployment.
This body should be entrusted with two functions: to ensure competitiveness with the rest of the world and, concurrently, to coordinate research on how to prevent or at least limit unwanted escalation or crisis.
Especially adversaries should endeavor to agree on a mechanism to ensure that decisions that may prove irrevocable are made at a pace conducive to human thought and deliberation — and survival.21
What human qualities will this age celebrate?
AI may prompt in humans the feeling of being tangential to the primary process governing a situation.
for the scholar who is told the most likely answer by an AI model before his or her research has begun in earnest, the experience may prove efficient but not always fulfilling.
Ultimately, individuals and societies will have to make up their minds which aspects of life to reserve for human intelligence and which to turn over to AI or human‑AI collaboration.
Optimizing the distribution of resources and increasing the accuracy of decision making is good for society, but for the individual, meaning is more often derived from autonomy and the ability to explain outcomes on the basis of some set of actions and principles.
Inventions such as the mechanical spinning machine displaced laborers and inspired the rise of the Luddites, members of a political movement who sought to ban — or, failing that, to sabotage — new technologies to preserve their old ways of life. The industrialization of agriculture sparked mass migration to the cities. Globalization altered manufacturing and supply chains, and both prompted changes, even unrest, before many societies ultimately absorbed the changes for their overall betterment.
Just as a divide exists today between “digital natives” and prior generations, so, too, will a divide emerge between “AI natives” and the people who precede them.
Over time, individuals may come to prefer their digital assistants over humans, for humans will be less intuitive of their preferences and more “disagreeable” (if only because humans have personalities and desires not keyed to other individuals).
How will the omnipresent companionship of a machine, which does not feel or experience human emotion (but may mimic it), affect a child’s perception of the world and his or her socialization?
How will it shape imagination? How will it change the nature of play? How will it alter the process of making friends or fitting in?
As long as people desire such explanation, the age of AI will disappoint the majority of people who do not understand the technology’s processes and mechanisms.
Societies need to build the intellectual and psychological infrastructure to engage with AI and exercise its unique intelligence to benefit humans as much as possible.
In this process, order and legitimacy are linked: order without legitimacy is mere force.
also requires the protection of human speech from AI distortion.
Does one classify an AI dialogue between two public figures who never met as misinformation, entertainment, or political inquiry — or does the answer depend on the context or on the participants?
These include efforts in academia, such as the MIT initiative to address the future of work,6 and efforts in government, such as the National Security Commission on Artificial Intelligence.
Most experience was lived, and most knowledge was transmitted orally.
Nor does it have awareness or reflective capabilities. It is a human creation, reflecting human-designed processes on human-created machines. Yet in some instances, at awesome scale and speed, it produces results approximating those that have, until now, only been reached through human reason.
We may comfort ourselves by repeating that AI is artificial, that it has not or cannot match our conscious experience of reality. But when we encounter some of AI’s achievements — logical feats, technical breakthroughs, strategic insights, and sophisticated management of large, complex systems — it is evident that we are in the presence of another experience of reality by another sophisticated entity.
