Blindsight (Firefall, #1)

by Peter Watts

I grew up and I got along. I learned to fit in. I observed, recorded, derived the algorithms and mimicked appropriate behaviors.

Meme management is so much subtler; the rose-tinted refraction of perceived reality, the contagious fear of threatening alternatives. There have always been those tasked with the rotation of informational topologies, but throughout most of history they had little to do with increasing its clarity.

The most altruistic and sustainable philosophies fail before the brute brain stem imperative of self-interest.

Computers bootstrap their own offspring, grow so wise and incomprehensible that their communiqués assume the hallmarks of dementia: unfocused and irrelevant to the barely intelligent creatures left behind.

presently a parasite on the Body Economic thanks to genes and machines on the cutting edge.”

Explaining the Incomprehensible to the Indifferent.

They’re just tweaks, mainly. Changing taste in music or cuisine, you know, optimizing mate compatibility. It’s all completely reversible.” “There aren’t drugs for that?” “Nah. Too much developmental variation between brains; our targeting is really fine-scale. But it’s not all microsurgery and fried synapses, you know. You’d be surprised how much rewiring can be done noninvasively. You can start all sorts of cascades just by playing certain sounds in the right order, or showing images with the right balance of geometry and emotion.”

LEADERS ARE VISIONARIES WITH A POORLY DEVELOPED SENSE OF FEAR AND NO CONCEPT OF THE ODDS AGAINST THEM. —Robert Jarvik

THEY KNOW THE WORDS BUT NOT THE MUSIC.

“How can you possibly tell the rest of us what your bleeding edge is up to if you don’t understand it yourself?” Chelsea demanded back when things were good between us. Before she got to know me. I shrugged. “It’s not my job to understand them. If I could, they wouldn’t be very bleeding edge in the first place. I’m just a, you know, a conduit.” “Yeah, but how can you translate something if you don’t understand it?” A common cry, outside the field. People simply can’t accept that patterns carry their own intelligence, quite apart from the semantic content that clings to their surfaces; if you manipulate the topology correctly, that content just comes along for the ride. “You ever hear of the Chinese Room?” I asked.

there’s no such thing as a one-size-fits-all survival strategy. Something that aces against one player blows goats against a different one. So the best you can do is mix up your strategies based on the odds. It’s a weighted dice roll and it gives you the best mean payoff over the whole game, but you’re bound to crap out and choose the wrong strategy at least some of the time.

We all come into the story halfway through, we all catch up as best we can, and we’re all gonna die before it ends.”

I specialize in processing informational topologies.” “Without understanding their content.”

Change one more synapse and I might turn into someone else.” “That’s ridiculous and you know it. Or every experience you had would turn you into a different person.” I thought about that. “Maybe it does.”

memories aren’t historical archives. They’re—improvisations, really. A lot of the stuff you associate with a particular event might be factually wrong, no matter how clearly you remember it. The brain has a funny habit of building composites. Inserting details after the fact. But that’s not to say your memories aren’t true, okay? They’re an honest reflection of how you saw the world, and every one of them went into shaping how you see it. But they’re not photographs. More like impressionist paintings.

The easily impressed might have even called it mind reading. It wasn’t, though. It was all just feedback and correlation. It doesn’t take a telepath to turn one set of patterns into another.

Predators run for their dinner, it goes. Prey run for their lives. The moral is supposed to be that on average, the hunted escape the hunters because they’re more motivated.

our conscious selves saw only the simulation in our heads, an interpretation of reality, endlessly refreshed by input from the senses.

Earlier generations had worshipped malign and capricious spirits. Ours put its faith in an ordered universe.

PROBLEMS CANNOT BE SOLVED AT THE SAME LEVEL OF AWARENESS THAT CREATED THEM. —Albert Einstein

The system called Robert Paglino sat quietly,

Maybe they’re just biomechanical machines.” “That’s what life is, Keeton. That’s what you are.” Another hit of nicotine, another storm of numbers, another sample. “Life isn’t either/or. It’s a matter of degree.”

“Now I wonder if you ever actually meant any of it.” “That’s not fair.” “No.” She pursed her lips. “No, it isn’t. That’s not really what I’m trying to say. I guess … it’s not so much that you don’t mean any of it. It’s more like you don’t know what any of it means.”

This is how you communicate with a fellow intelligence: You hurt it, and keep on hurting it, until you can distinguish the speech from the screams.

Bad eyes are only one bane of clear vision: bad assumptions can be just as blinding,

“Vision’s mostly a lie anyway,” he continued. “We don’t really see anything except a few hi-res degrees where the eye focuses. Everything else is just peripheral blur, just … light and motion. Motion draws the focus. And your eyes jiggle all the time, did you know that, Keeton? Saccades, they’re called. Blurs the image, the movement’s way too fast for the brain to integrate so your eye just … shuts down between pauses. It only grabs these isolated freeze-frames, but your brain edits out the blanks and stitches an … an illusion of continuity into your head.” He turned to face me. “And you know what’s really amazing? If something only moves during the gaps, your brain just … ignores it. It’s invisible.”

Brains are survival engines, not truth detectors. If self-deception promotes fitness, the brain lies. Stops noticing—irrelevant things. Truth never matters. Only fitness. By now you don’t experience the world as it exists at all. You experience a simulation built from assumptions. Shortcuts. Lies. Whole species is agnosiac by default.

Do you want to know what consciousness is for? Do you want to know the only real purpose it serves? Training wheels. You can’t see both aspects of the Necker cube at once, so it lets you focus on one and dismiss the other. That’s a pretty half-assed way to parse reality. You’re always better off looking at more than one side of anything. Go on, try. Defocus. It’s the next logical step. Oh, but you can’t. There’s something in the way. And it’s fighting back.

Evolution has no foresight. Complex machinery develops its own agendas. Brains—cheat. Feedback loops evolve to promote stable heartbeats and then stumble upon the temptation of rhythm and music. The rush evoked by fractal imagery, the algorithms used for habitat selection, metastasize into art. Thrills that once had to be earned in increments of fitness can now be had from pointless introspection. Aesthetics rise unbidden from a trillion dopamine receptors, and the system moves beyond modeling the organism. It begins to model the very process of modeling. It consumes evermore computational resources, bogs itself down with endless recursion and irrelevant simulations. Like the parasitic DNA that accretes in every natural genome, it persists and proliferates and produces nothing but itself. Metaprocesses bloom like cancer, and awaken, and call themselves I.

The system weakens, slows. It takes so much longer now to perceive—to assess the input, mull it over, decide in the manner of cognitive beings. But when the flash flood crosses your path, when the lion leaps at you from the grasses, advanced self-awareness is an unaffordable indulgence. The brain stem does its best. It sees the danger, hijacks the body, reacts a hundred times faster than that fat old man sitting in the CEO’s office upstairs; but every generation it gets harder to work around this—this creaking neurological bureaucracy. I wastes energy and processing power, self-obsesses to the point of psychosis. Scramblers have no need of it, scramblers are more parsimonious. With simpler biochemistries, with smaller brains—deprived of tools, of their ship, even of parts of their own metabolism—they think rings around you. They hide their language in plain sight, even when you know what they’re saying. They turn your own cognition against itself. They travel between the stars. This is what intelligence can do, unhampered by self-awareness. I is not the working mind, you see. For Amanda Bates to say “I do not exist” would be nonsense; but when the processes beneath say the same thing, they are merely reporting that the parasites have died. They are only saying that they are free.

IF THE HUMAN BRAIN WERE SO SIMPLE THAT WE COULD UNDERSTAND IT, WE WOULD BE SO SIMPLE THAT WE COULDN’T.

“You have a naïve understanding of evolutionary processes. There’s no such thing as survival of the fittest. Survival of the most adequate, maybe. It doesn’t matter whether a solution’s optimal. All that matters is whether it beats the alternatives.”

A smart automaton would blend in. It would observe those around it, mimic their behavior, act just like everyone else. All the while completely unaware of what it was doing. Unaware even of its own existence.” “Why would it bother? What would motivate it?” “As long as you pull your hand away from an open flame, who cares whether you do it because it hurts or because some feedback algorithm says Withdraw if heat flux exceeds critical T? Natural selection doesn’t care about motives. If impersonating something increases fitness, then nature will select good impersonators over bad ones.

Wegner thought it was an executive summary. Penrose heard it in the singing of caged electrons. Norretranders said it was a fraud; Kazim called it leakage from a parallel universe. Metzinger wouldn’t even admit it existed. The AIs claimed to have worked it out, then announced they couldn’t explain it to us. Gödel was right after all: No system can fully understand itself.

All those theories, all those drug dreams and experiments and models trying to prove what consciousness was: none to explain what it was good for. None needed: obviously, consciousness makes us what we are. It lets us see the beauty and the ugliness. It elevates us into the exalted realm of the spiritual. Oh, a few outsiders—Dawkins, Keogh, the occasional writer of hackwork fiction who barely achieved obscurity—wondered briefly at the why of it: why not soft computers, and no more? Why should nonsentient systems be inherently inferior? But they never really raised their voices above the crowd. The value of what we are was too trivially self-evident to ever call into serious question.

NOT UNTIL WE ARE LOST DO WE BEGIN TO UNDERSTAND OURSELVES. —Henry David Thoreau

Imagine you have intellect but no insight, agendas but no awareness. Your circuitry hums with strategies for survival and persistence, flexible, intelligent, even technological—but no other circuitry monitors it. You can think of anything, yet are conscious of nothing.