Superintelligence: Paths, Dangers, Strategies in Science and Inquiry

Apr 10, 2015 Rusty rated it liked it

Shelves: prose, non-fiction, science

A confluence of events surround the reading of this book that left me in a pretty weird place. First, this shit is scary. The whole topic of this book, really, is a list of things that could go wrong with a superintelligent AI, then a list of things we could do in the near future to help make our future masters look upon us favorably, like we might might a pet worm that got us through some tough times. Of course, if the pet worm got uppity and wanted something from us we might not be willing to A confluence of events surround the reading of this book that left me in a pretty weird place. First, this shit is scary. The whole topic of this book, really, is a list of things that could go wrong with a superintelligent AI, then a list of things we could do in the near future to help make our future masters look upon us favorably, like we might might a pet worm that got us through some tough times. Of course, if the pet worm got uppity and wanted something from us we might not be willing to give... like, say, $20, then we might decide that worm would make better fish bait than pet.

Same with the machines, sure, now they are dumb and do what we say. But what happens when they're smarter than we are... at everything. It doesn't matter if they're conscious or not, sentient or not, at that point they might decide that since the program they were given to run was to calculate the digits of pi, then the meat sacks that keep running around and poking at it, giving it commands and whatnot, would be better if converted to something it could use to help it in it's mission.

So our screams would be meaningless to it as we're disassembled into our constituent atoms and converted into computronium. Hell, all the biomatter on earth might be thusly converted... ah, what the heck, might as well convert all the baryonic matter in the observable universe into something it can use to help it compute those stupid digits of pi out until the heat death of the universe begins to break down it's ability to function.

So, that would suck.

Nick wants to explain things we can do to help mitigate some of those dangers.

While having nightmares about that scenario, I was catching up on my Ghengis Khan history, and somewhere in there (considering how recent this is, you'd think I'd remember more) one of his advisors/sons/lieutenants was questioning if it was really necessary to slaughter every inhabitant of the city they'd just taken and was mocked by his peers for being a woman who weeps over nothing.

I should really look that up. I can remember almost nothing else about that. Seriously, it hasn't been that long. I'm getting old.

So it's possible for us, soft, squishy, meat bags to be so sociopathic as to mock anyone that doesn't want to kill every other human in sight. How much more likely is it that a machine might be even more relentless in it's slaughter of us. At least the Mongols made a point of killing people. A machine intelligence might just do it almost as an afterthought, never considering us as sentient beings in the first place.

In the end, this book feels a lot longer than it should be. And it uses big-boy words way more often than it should. I don't mean curse words either, I mean it had me running to the dictionary so often I started to think the author was using that vocabulary to piss me off.

The topic deserves a lot of more thinking by me, and I'm glad there are several books on the topic. I'll read some more. But this one is written in such a way that you kind of have to dig in and work to get through. Too bad. Given the subject, it should have been much easier of a read.

...more

1 like · Like · see review

Jul 07, 2016 Leon Lahoud rated it it was amazing

“The AI does not hate you, nor does it love you, but you are made out of atoms which it can use for something else.” Eliezer Yudkowsky

AI is a very hot topic today, mainly because of the revolution in pattern recognition which comes under the heading of Deep Learning. Hollywood conditioned us to think of Artificial intelligence as a dangerous technology that will bring forth the end of human civilization. If you are a fan of these ideas, then Nick Bostrom’s Superintelligence will not disappoint. ...more

Like · see review