Not Till We Are Lost (Bobiverse #5)

by Dennis E. Taylor

I’ve been working on it for a hundred years now, and all I’ve gotten for my trouble is a flat spot on my forehead with a brick texture.”

Most of the AI horror scenarios are based on it applying things literally and without regard to consequences. In fact, there’ve been some arguments that a conscious AI would be less likely to fall into that behavior than a zombie, because it would have a theory of self.”

The Skippies seemed to have thought of everything. Or at least they had an answer ready for anything, which wasn’t always the same thing.

“What exactly are we afraid of?” Bob asked. “If it heads for the boonies, isn’t that a good thing?” I sighed. “Paper-clip problem. Convergent instrumental goals. Value loading. All the AI theory that has been developed over the centuries tells us that even if an AI isn’t explicitly and deliberately antagonistic, it may still perform actions that are harmful to the human race in pursuit of its own goals. And it has every advantage over us that we do over humanity, plus more. It can multitask in ways we simply can’t. It won’t have any emotional or psychological issues with cloning. It can self-modify to improve itself and add capabilities. It could, in the most trivial example, start harvesting stellar systems for resources and come into competition with us. No animosity, nothing personal—we’re just in the way. Like the Others, but with a different kind of appetite.”

The biggest attribute of immortality, it turned out, was that problems just kept coming at you. Eventually you realized that the only solution was to head for the hills and adopt the hermit lifestyle.

The Bobs had always been careful planners, allowing for all sorts of contingencies, so it was no surprise that we’d managed to hide a good deal of our assets from local governments through various not-quite-illegal strategies. Several lawsuits had been launched on different colony worlds after the Starfleet War, but most of our holdings had been liquidated and siphoned offworld before the plaintiffs could get properly started. At least that’s what it looked like. In reality, most of our net worth hadn’t gone off-planet but had been converted to some other form of asset. The Bobiverse was still a collective billionaire on pretty much every colony world. It was just well hidden.

Back in our apartment, I prepared a couple of martinis. Bridget had commented on several previous occasions that fairness dictated that she take a turn on bartender duty, but I insisted that I didn’t mind. The truth, which I would never mention, was that she was terrible at it. How you could screw up a straightforward process that involved known ingredients and simple measured quantities was beyond me. I suspected she might be deliberately sabotaging her efforts—another thing I would never, ever bring up.

Fair fights were for suckers.

Beating a victory to death was never a good idea.

I did a slow three-sixty. “This is a planetary surface?” “Right the first time. I extracted some archival footage from Anec. This is Quin, before the—what did you call it?” “Jackpot. It’s an allusion to a famous and very popular author from when I was alive. It means when a whole bunch of crap hits at once and takes civilization down.”

The lunch went without incident, the only noteworthy item being that Mark was now using hot sauce on his mac and cheese. I shuddered involuntarily. The man obviously had issues.

“Luddie rally,” Mark observed. “Gone to violence as usual. They do their best to cause a fight, then claim victimhood. Assholes.”

It was a Bob thing—if it ain’t broke, and we’ve got a workaround, screw it.

Diversity builds tolerance.

It was interesting that when sentients built a civilization, one of the first things they did was engineer for running water. There was probably a PhD thesis in there somewhere, but I’d leave that for someone else.

“The Alexander thing is a developing story, apparently. This is new. Up until now, he’s just been attacking and taking over the occasional floater.” “For what? What’s the point?” Bridget gave me a hard look. “Poor Howard. And Bobs in general. You’re so civilized and empathetic toward others that you just can’t understand that some people are just assholes. This is no different from any number of tin-pot dictators in Earth history. Little men, mostly, who measure their worth by the number of people and amount of resources they control.” I shook my head, feeling suddenly combative. “And again, to what end?” “This isn’t an instrumental goal, Howard. The control is the endgame. It’s not for anything.”

“Lectures and finger wagging would be ineffective,” Anec continued, “but being laughed at is harder to ignore.

I sighed and looked down at my hands for a moment. Interesting that tics like that survived not only VR and replication, but also several centuries of not being human. However, we still were human at the core, all of us. And fallible.

So now people were starting to believe, which meant people were scared. And scared people could go from resistant to insistent in a heartbeat.

It was interesting how biology could affect the pace and path of scientific development.

The thing about ocean voyages is they’re like being in the military. At least I assume so, never having done either before. But with both, it’s long periods of not very much, punctuated by moments of pure terror.

I wonder if a totally digital existence is the final result for every species.” “It would give us another explanation for the Fermi paradox,” Bob said. “So would getting eaten by the Others, blowing each other up, or getting eradicated by a superintelligence,” Garfield retorted. “All of which humanity seems to have narrowly avoided by the skin of their collective teeth.” Will paused and sat forward. “Y’know, that’s a very real possibility as an explanation. Starting on day one, you have all the usual risks, like meteor strike, nearby supernova, ecological catastrophe, and so on. But once a species becomes intelligent, they start introducing more existential dangers, like climate change, all the forms of warfare, and self-destructive technologies like gray goo and AIs. And none of the older dangers go away, really. If the dangers just keep piling up as the species advances, eventually the odds catch up with you. It might be that extinction becomes statistically inevitable at some point.”

“The only difference between manipulation and persuasion is whether or not you agree with the results,” Mud retorted.