Not Till We Are Lost (Bobiverse #5)

by Dennis E. Taylor

“Or the possibility of FTL.” I paused. “I have to admit, if you solve only that one, you’ll have paid for the project. I’ve been working on it for a hundred years now, and all I’ve gotten for my trouble is a flat spot on my forehead with a brick texture.”

reply. The Skippies seemed to have thought of everything. Or at least they had an answer ready for anything, which wasn’t always the same thing.

“Sure,” Garfield grumbled. “Just like there’s no indication of danger until the log turns out to be a gator.”

“Hersch, my faith in humanity is either utterly destroyed or firmly validated, depending on how you look at it,” Neil replied.

Plus, coffee was still only available in VR, and I had been taught from a young age that reading a newspaper without coffee was immoral.

They’ll use the boiling-frog strategy, advancing their grip and clampdowns by small increments, so there will never be a wedge issue big enough to incite protests or threaten their coalition.” “Until they reach the point where protests are no longer legal.”

“Paper-clip problem. Convergent instrumental goals. Value loading. All the AI theory that has been developed over the centuries tells us that even if an AI isn’t explicitly and deliberately antagonistic, it may still perform actions that are harmful to the human race in pursuit of its own goals. And it has every advantage over us that we do over humanity, plus more. It can multitask in ways we simply can’t. It won’t have any emotional or psychological issues with cloning. It can self-modify to improve itself and add capabilities. It could, in the most trivial example, start harvesting stellar systems for resources and come into competition with us. No animosity, nothing personal—we’re just in the way.

“Gone to violence as usual. They do their best to cause a fight, then claim victimhood. Assholes.”

Will paused and sat forward. “Y’know, that’s a very real possibility as an explanation. Starting on day one, you have all the usual risks, like meteor strike, nearby supernova, ecological catastrophe, and so on. But once a species becomes intelligent, they start introducing more existential dangers, like climate change, all the forms of warfare, and self-destructive technologies like gray goo and AIs. And none of the older dangers go away, really. If the dangers just keep piling up as the species advances, eventually the odds catch up with you. It might be that extinction becomes statistically inevitable at some point.”

“The only difference between manipulation and persuasion is whether or not you agree with the results,”

That one is, I think, what’s really getting the stuffed shirts upset.”