Goodreads helps you keep track of books you want to read.

Start by marking “Facing the Intelligence Explosion” as Want to Read:

Other editions

Enlarge cover

Rate this book

Clear rating

1 of 5 stars 2 of 5 stars 3 of 5 stars 4 of 5 stars 5 of 5 stars

Open Preview

Facing the Intelligence Explosion

Luke Muehlhauser

3.88 · Rating details · 170 ratings · 20 reviews

Sometime this century, machines will surpass human levels of intelligence and ability. This event—the “intelligence explosion”—will be the most important event in our history, and navigating it wisely will be the most important thing we can ever do.

Luminaries from Alan Turing and I. J. Good to Bill Joy and Stephen Hawking have warned us about this. Why do I think Hawking a ...more

Get A Copy

Or buy for

Kindle Edition, 91 pages

Published February 1st 2013 by Machine Intelligence Research Institute (first published 2013)

More Details... edit details

Friend Reviews

To see what your friends thought of this book, please sign up.

Reader Q&A

To ask other readers questions about Facing the Intelligence Explosion, please sign up.

Be the first to ask a question about Facing the Intelligence Explosion

Lists with This Book

This book is not yet featured on Listopia. Add this book to your favorite list »

Community Reviews

Showing 1-30

3.88 ·

Rating details

· 170 ratings · 20 reviews

Filter

Sort order

Dec 20, 2013 Miles rated it liked it

This is a useful but brief primer on critical thinking as it relates to the future of technology and the risks/potentials of creating superintelligent AIs. It's the kind of text I wish I'd read when I was first becoming interested in these elements of modern life, instead of after I'd already begun to dig into the popular literature surrounding the transhuman and futurist movements. Muehlhauser seems like a very smart guy, and I appreciate how his tiny book works very hard to make some pretty co This is a useful but brief primer on critical thinking as it relates to the future of technology and the risks/potentials of creating superintelligent AIs. It's the kind of text I wish I'd read when I was first becoming interested in these elements of modern life, instead of after I'd already begun to dig into the popular literature surrounding the transhuman and futurist movements. Muehlhauser seems like a very smart guy, and I appreciate how his tiny book works very hard to make some pretty complex ideas available to laypeople. However, the text also suffers from some instances of oversimplification.

One example of this is the way Muehlhauser frames the problem of using instrumental rationality to achieve desirable human goals. He repeatedly invokes the concept of "winning" in order to describe an instance of successfully reaching a goal. He probably does this in an attempt to make his arguments accessible, but in this case with the unfortunate side-effect of bringing in some unhelpful connotations that accompany of the idea of victory, especially as it exists in our modern American, ultra-capitalist culture. For someone who has obviously read up on the power of psychological framing, I'm surprised that Muehlhauser didn't realize that the notion of "winning," in almost all instances, automatically generates the underlying assumption that someone (or something) has simultaneously "lost." It's perpetuating the vicious myth that all aspects of life and progress are zero-sum games, rather than working to expose the truth that zero-sum games are largely a human construct. My problem with this isn't that competition and succession aren't a natural part of biological (and technological) development, because they are. Rather, it's that Muehlhauser fails to also acknowledge that cooperation and symbiosis occupy an equally important seat at the evolutionary table. He focuses too much energy on proposing ways that we can "control" AI and "make it safe" rather than proposing that we try to create AIs that won't want a world in which human values and needs are rendered entirely obsolete. So while I believe that Muehlhauser is very much committed to precisely this sort of outcome, I think this book can leave readers with the feeling that either we "win" or AIs do, without much room for collaboration or middle ground. Frustratingly, this whole issue could have been avoided by including a brief yet nuanced discussion of how the concept of "winning" isn't very useful and reframing the AI challenge in terms of prosperous, symbiotic commerce between man and machine (positive-sum instead of zero-sum). The book's last chapter leads me to believe that this is exactly what Muehlhauser is trying to argue for, but the message didn't come across clearly enough in my opinion.

While I can't blame Muehlhauser for the invention of the concept, I have to say that I really hate the central role that "utility functions" are coming to play in discussions about the future of humanity. This idea always leads me down the road of thinking that technology enthusiasts want to live in a world in which every element of sentient experience can be readily quantified. So we start slapping numbers on all our habits and activities, assigning them values that supposedly communicate their "utility" in terms of how they contribute or detract from the achievment of various goals. I won't argue that such a paradigm is physically or statistically impossible, but I will argue that I find it personally detestable. I don't want to live in a world where I'm constantly being reminded that I'm "wasting" my time doing things I enjoy simply because they might not contribute to what "goals" I am personally trying to reach, or that my community has chosen (either with our without my consent). That kind of life just seems exhausting and experientially impoverished. So I accept that utility functions are useful within a very limited scope, but I don't want them to play a dominant role in human life because I think that would cause us to lose sight of the raw value of habitual, enjoyable experiences, which are necessarily individualized and nuanced in ways that I think will always, at least in some way, defy quantification.

I don't want my criticisms to indicate that I think this is a bad or poorly conceived book. In the final analysis, I have to applaud Muehlhauser's slim text for convincing me thoroughly that we are not ready to give birth to AI with any assurance that it won't wipe us out with terrifying expedience. I found Muehlhauser's arguments concerning the fragility of human value systems and the difficulty of programming AIs to share human values extremely convincing and more than a bit distressing. The fact that our ability to create powerful AIs is vastly outstripping our ability to control their influence or render human life "necessary" rather than simply exploitable in the eyes of technological superintelligence is probably the second most important problem humanity faces in the 21st century, next to the threat of climate change (Muehlhauser often qualifies his arguments with the phrase "if technological progress is allowed to continue," which I interpret as an implicit acknowledgement of the possibility that humans might simply wipe ourselves out before strong AI comes into existence).

This is a fun, easy, and informative read. Highly recommended for anyone just getting interested in these topics, but probably not for folks already steeped in futurist thought. ...more

flag 4 likes · Like · see review

Apr 21, 2013 Harsh Pareek rated it it was amazing

Excellent introductory book to read if you haven't seen anything from Less Wrong. The book generously links to the original articles if you want to read more. In fact, a third of the book is just quotes and paragraphs from other articles and posts.

This is a review book and does not introduce any new ideas -- it does however try to collate and clarify many existing ideas. Why is an "Intelligence Explosion" likely? What do we even mean by "intelligence"? Are there limits to human intelligence? Are ...more

flag 2 likes · Like · see review

Jun 18, 2013 Mark Gomer rated it really liked it

Shelves: subject-philosophy, 2013-r, subject-compsci, library-r, library-read

This book does an excellent job of explaining the importance of the AI issue to laypeople. It doesn't leave anything out that I think it needs to have and it doesn't have anything I think it should have left out.

All in all, this is my new weapon of choice for convincing people that 1) AI is coming and 2) it can either be the best thing ever or the worst thing ever.

flag 2 likes · Like · see review

Feb 23, 2015 Eric rated it it was amazing

Shelves: philosophy, technology

This is one of the most exciting (and slightly terrifying) books I've ever read. Normally, I'm not one for much futurism as it has a pretty terrible track-record of being right, but this isn't that. It's a great, and undoubtedly a very accurate look into how artificial intelligence will surpass human intelligence, and what we can do about it to get the best possible result for ourselves. Because honestly, you know what? I like staying alive. And now I really want to do a lot more reading on the ...more

flag Like · see review

Feb 18, 2014 Gabriel rated it really liked it

This is an important subject treated (very) quickly and lightly here. The informal style serves to undermine the severity of the topic, namely that artificial intelligence has the very real possibility of being the next doomsday invention that could well kill us all. Instead, the tone gives you more of the sense that you're talking to a bright and a bit edgy person at a very interesting party.

All that said, this book is brief and snappy and absolutely worth reading. Some will get distracted by t ...more

flag Like · see review

Nov 15, 2013 Girish rated it really liked it

This book is short: read it. It's a personal and fantastic exploration of machine superintelligence for the layman. The introduction is nice and human, it helps you feel the importance of the claims in here.

Let the ideas germinate and you'll achieve some greater degree of self-abnegation. If anything, the value you get per page is immense.

I took off one star because the way some of the evidence cited can bias you, and this is a book largely about de-biasing. For example, "the computers that do ...more

flag Like · see review

Apr 29, 2014 Niels Bergervoet rated it it was amazing

The book does a real good job explaining why AI and super human computer intelligence concern us all and are a subject to take seriously. The author gives a good overview of the dangers and opportunities of AI and why we should act on it. The book is well written and easy to read, also for a laymen like me. His arguments are well constructed and he extensivly explains his methods and why he artived at this viewpoint.

flag Like · see review

Sep 07, 2015 Timo rated it really liked it

Shelves: usefwl-books

Good book on a very important topic. Short and easy to read. It uses the "outside view" to persuasively argue the likelihood of an intelligence explosion, and that the outcome of such an explosion depends on whether we're able to ensure that future intelligence carries our own values.

A bonus is that the first parts of the book is a great introduction to rationality. A necessity if one wants to introduce the absurd topic of AI risk to laypeople.

flag Like · see review

Apr 14, 2013 Viktor rated it it was amazing

Even if you think the possibility of "seed AI" or even Artificial general intelligence unlikely, Muehlhauser makes an excellent case for 1) why you might be mistaken, and 2) even if it would be unlikely, it's definitely something that should be taken seriously given the possibility of both extremely good and bad outcomes.

flag Like · see review

Sep 09, 2014 Boris Vidolov rated it it was ok

I expected a deeper book with breathtaking research. As I work in the software technology sector, all points of the book are simply trivial and well known. The writing style and the idea exposure was also very primitive, as if the target audience is 7 graders. If this is the first time you hear about artificial intelligence, you may still enjoy it.

flag Like · see review

Sep 18, 2013 Joshua Goller rated it it was amazing

This is the first book I have finished in less than a day, and I think the first complete AI book I have ever read. Everyone, from AI expert to total neophyte, should read this book - it is frightening but motivating, simple but thought provoking, and short. It lays the major groundwork to AI research and will point the reader in all the right directions.

flag Like · see review

Jun 20, 2013 Terra Bosart rated it it was amazing

This is an excellent primer on the though process one should engage in, regarding the serious application of safety measures toward artificial intelligence. The author is quite generous with their references, which can lead the reader into more detailed reading about the generalities presented.

flag Like · see review

Sep 25, 2013 Pete rated it really liked it

A fun look at the current state of IT learning. Worth the time if any of this trans-humanism stuff is interesting to you. It is always helpful to not get to sucked into the inevitability of this stuff.

flag Like · see review

Oct 29, 2013 Kristoffer rated it it was amazing

At the very least, this book will provide you with pointers to explode your own intelligence. Which will become necessary in facing the intelligence explosion, "the most important thing we can ever do", and hence this is a most important book.

flag Like · see review

Mar 01, 2014 Katie rated it really liked it

Good overview, and a quick read. I find the link-heavy nature of it (in ebook form) distracting, but I may go back and make better use of that as a resource.

flag Like · see review