More on this book
Community
Kindle Notes & Highlights
In contrast, you retrieve information from your brain similarly to how you retrieve it from a search engine: you specify a piece of the information or something related to it, and it pops
Such memory systems are called auto-associative, since they recall by association rather than by address.
Soviet Phobos 1 mission failed on September 2, 1988. This was the heaviest interplanetary spacecraft ever launched, with the spectacular goal of deploying a lander on Mars’
moon Phobos—all thwarted when a missing hyphen caused the “end-of-mission” command to be sent to the spacecraft while it was en route to Mars, shutting down all of its systems.12
“Might makes right” arguments to the effect that stronger is always better have largely fallen from grace these days, being widely associated with fascism.
Slow doesn’t necessarily mean boring: if future life lives in a simulated world, its subjectively experienced flow of time need not have anything to do with the glacial pace at which the simulation is being run in the outside world, so the prospects of infinite computation could translate into subjective immortality for simulated
Life 3.0 a quick read on futures Super intelligent life and how it get shaped by itself.
life forms.
Putting together everything we’ve explored in this chapter tells us that maximally efficient power plants and computers would enable superintelligent life to perform a mind-boggling amount of computation.
Powering your thirteen-watt brain for a hundred years requires the energy in about half a milligram of matter—less than in a typical grain of sugar.
intelligent entities naturally organize themselves into power hierarchies in Nash equilibrium, where any entity would be worse off if they altered their strategy. The better the communication and transportation technology gets, the larger these hierarchies can grow.
Charles Darwin elegantly explained why: since the most efficient copiers outcompete and dominate the others, before long any random life form you look at will be highly optimized for the goal of replication.
The subgoal to procreate was implemented as a desire for sex rather than as a desire to become a sperm/egg donor, even though the latter can produce more babies with less effort.
So far, most of what we build exhibits only goal-oriented design, not goal-oriented behavior: a highway doesn’t behave; it merely sits there. However, the most economical explanation for its existence is that it was designed to accomplish a goal, so even such passive technology is making our Universe more goal-oriented.
Teleology is the explanation of things in terms of their purposes rather than their causes, so we can summarize the first part of this chapter by saying that our Universe keeps getting more teleological.
In the ancient Greek legend, King Midas asked that everything he touched turn to gold, but was disappointed when this prevented him from eating and even more so when he inadvertently turned his daughter to gold. In the stories where a genie grants three wishes, there are many variants for the first two wishes, but the third wish is almost always the same: “Please undo the first two wishes, because that’s not what I really wanted.”
orthogonality thesis: that the ultimate goals of a system can be independent of its intelligence. By definition, intelligence is simply the ability to accomplish complex goals, regardless of what these goals are, so the orthogonality thesis sounds quite reasonable. After all, people can be intelligent and kind or intelligent and cruel, and intelligence can be used for the goal of making scientific discoveries, creating beautiful art, helping people or planning terrorist attacks.8
As Yuval Noah Harari puts it in his book Homo Deus:4 “If any scientist wants to argue that subjective experiences are irrelevant, their challenge is to explain why torture or rape are wrong without reference to any subjective experience.” Without such reference, it’s all just a bunch of elementary particles moving around according to the laws of physics—and what’s wrong with that? What’s
Indeed, unconscious information processing appears not only to be possible, but also to be more the rule than the exception. Evidence suggests that of the roughly 107 bits of information that enter our brain each second from our sensory organs, we can be aware only of a tiny fraction, with estimates ranging from 10 to 50 bits.7 This suggests that the information processing that we’re consciously aware of is merely the tip of the iceberg.
NCC researchers have carefully measured how long, and Christof Koch’s summary is that it takes about a quarter of a second from when light enters your eye from a complex object until you consciously perceive seeing it as what it is.13 This means that if you’re driving down a highway at fifty-five miles per hour and suddenly see a squirrel a few meters in front of you, it’s too late for you to do anything about it, because you’ve already run over it!
In summary, your consiousness lives in the past, with Christof Koch estimating that it lags behind the outside world by about a quarter second. Intriguingly, you can often react to things faster than you can become conscious of them, which proves that the information processing in charge of your most rapid reactions must be unconscious.
it’s only the structure of the information processing that matters, not the structure of the matter doing the information processing. In other words, consciousness is substrate-independent twice over!
if we discover a peaceful extraterrestrial civilization far more advanced than us in science, art and everything else we care about, this presumably wouldn’t prevent people from continuing to experience meaning and purpose in their lives. We could retain our families, friends and broader communities, and all activities that give us meaning and purpose, hopefully having lost nothing but arrogance.
The saddest aspect of life right now is that science gathers knowledge faster than society gathers wisdom.
