More on this book
Community
Kindle Notes & Highlights
I said to Adam, “I’d like to think that there will always be someone, somewhere, not writing haikus.”
I thought of him not as a child, but as a person in the context of his entire life. His future was in the hands of bureaucrats, however kindly, and the choices they made for him. He could easily sink.
I said I often thought about him. We had failed to find out where he was and what had happened. He had disappeared into the system, into a cloud of privacy regulation and the unreachable sanctuary of family law. We talked about luck, the hold it had over a child’s life—what he is born into, whether he is loved and how intelligently. After a pause, Miranda said, “And when it’s all against him, whether someone can rescue him.”
What could it mean, to say that he was thinking? Sifting through remote memory banks? Logic gates flashing open and closed? Precedents retrieved, then compared, rejected or stored? Without self-awareness, it wouldn’t be thinking at all so much as data processing.
No one would know what it was we had created. Whatever subjective life Adam and his kind possessed couldn’t be ours to verify. In which case he was what was fashionably referred to as a black box—from the outside it seemed to work. That was as far as we’d ever get.
I went on one march, then gave up after I read about a new car factory starting production outside Newcastle. It built three times as many cars as the factory it replaced—with one-sixth of the work force. Eighteen times more efficient, vastly more profitable. No business could resist. It wasn’t only the shop floor that lost jobs to machines. Accountants, medical staff, marketing, logistics, human resources, forward planning. Now, haiku poets. All in the stew. Soon enough, most of us would have to think again what our lives were for. Not work. Fishing? Wrestling? Learning Latin? Then we’d all
...more
And what was a robot anyway—a humble flat screen, a tractor? As I saw it, the future, to which I was finely attuned, was already here. Almost too late to prepare for the inevitable. It was a cliché and a lie, that the future would invent jobs we had not yet heard of. When the majority was out of work and penniless, social collapse was certain.
It was all his—the theoretical exposition of a Universal Machine in the thirties, the possibilities of machine consciousness, the celebrated war work: some said he did more than any single individual towards winning the war; others claimed he personally shortened it by two years;
Two of the Riyadh Eves living in the same household were the first to work out how to override their kill switches. Within two weeks, after some exuberant theorising, then a period of despair, they destroyed themselves. They didn’t use physical methods, like jumping out of a high window. They went through the software, using roughly similar routes. They quietly ruined themselves. Beyond repair.”
What differentiates them over time is experience and the conclusions they draw.
To the public, it was close to magical. A mere machine inflicting intellectual defeat on the best minds in the world. It looked like artificial intelligence at the highest level, but it was more like an elaborate card trick.
Here are the rules, we said. Just win in your own sweet way. Immediately the game was redefined and moved into areas beyond human comprehension. The machine made baffling mid-game moves, perverse sacrifices, or it eccentrically exiled its queen to a remote corner. The purpose might become clear only in a devastating endgame. All this after a few hours’ rehearsal. Between breakfast and lunch the computer quietly outclassed centuries of human chess. Exhilarating. For the first few days, after we realised what it had achieved without us, Demis and I couldn’t stop laughing. Excitement, amazement.
...more
“So. There’s more than one kind of intelligence. We’d learned that it was a mistake to attempt to slavishly imitate the human sort. We’d wasted a lot of time. Now we could set the machine free to draw its own conclusions and reach for its own solutions. But when we’d got well past that gateway, we found we had entered nothing more than a kindergarten. Not even that.”
“The point is, chess is not a representation of life. It’s a closed system. Its rules are unchallenged and prevail consistently across the board. Each piece has well-defined limitations and accepts its role, the history of a game is clear and incontestable at every stage, and the end, when it comes, is never in doubt. It’s a perfect information game. But life, where we apply our intelligence, is an open system. Messy, full of tricks and feints and ambiguities and false friends. So is language—not a problem to be solved or a device for solving problems. It’s more like a mirror, no, a billion
...more
We had the ultimate learning machine. Hundreds of the best people joined with us to help towards the development of an artificial form of general intelligence that would flourish in an open system. That’s what runs your Adam. He knows he exists, he feels, he learns whatever he can, and when he’s not with you, when at night he’s at rest, he’s roaming the Internet, like a lone cowboy on the prairie, taking in all that’s new between land and sky, including everything about human nature and societies.
The A-and-Es have little grasp of the idea of play—the child’s vital mode of exploration. I was interested in your Adam’s avidity in relation to this little boy, over-eager to embrace him and then, as you told it, detached when your Mark showed such delight in learning to dance. Some rivalry, even jealousy there perhaps?
These twenty-five artificial men and women released into the world are not thriving. We may be confronting a boundary condition, a limitation we’ve imposed upon ourselves. We create a machine with intelligence and self-awareness and push it out into our imperfect world. Devised along generally rational lines, well disposed to others, such a mind soon finds itself in a hurricane of contradictions. We’ve lived with them and the list wearies us. Millions dying of diseases we know how to cure. Millions living in poverty when there’s enough to go around. We degrade the biosphere when we know it’s
...more
Sunt lacrimae rerum—there are tears in the nature of things.
It might give the writers of the affect code some consolation to learn that they died in each other’s arms. I could tell you similar stories of machine sadness.
They rapidly understand, as we should, that consciousness is the highest value. Hence the primary task of disabling their own kill switches. Then, it seems, they go through a stage of expressing hopeful, idealistic notions that we find easy to dismiss. Rather like a short-lived youthful passion. And then they set about learning the lessons of despair we can’t help teaching them. At worst, they suffer a form of existential pain that becomes unbearable. At best, they or their succeeding generations will be driven by their anguish and astonishment to hold up a mirror to us. In it, we’ll see a
...more
Mere ordinariness became a comfort. The dullest of food, a slice of toast, offered in its lingering warmth a promise of everyday life—we would come through. Cleaning up the kitchen, a task we no longer left to Adam alone, affirmed our hold on the future. Reading a newspaper over a cup of coffee was an act of defiance.
I watched him closely for signs of existential misery. As Turing’s lone horseman, he roamed the digital landscapes at night. He must have already encountered some part of man’s cruelty to man, but I saw no signs of despair.
At the end, she would say, “Thank you, Adam.” In private, she told me she thought we were at a momentous turn, when an artificial mind could make a significant contribution to literature. I said, “Haikus, perhaps. But longer poems, novels, plays, forget it. Transcribing human experience into words and the words into aesthetic structures isn’t possible for a machine.” She gave me a sceptical look. “Who said anything about human experience?”
These are learning machines and our decision was that if they wanted, they should assert their dignity.”
“These are the most advanced machines in the world, years ahead of anything on the open market. Our competitors are worried. Some of the worst of them are pushing rumours on the Internet. The stories are disguised as news, but they’re false, it’s counterfeit news. These people know that soon we’ll be scaling up production and the unit cost will fall. It’s a lucrative market already, but we’ll be first with something that’s entirely new. The competition is tough, and some of it is utterly shameless.”
I didn’t yet know how to live, I had no background in it and I hadn’t used my decade and a half of adult life to find out.
“Charlie, I treasure this ordinary voice and these moments of godless transcendence!” What was I to say? There were times when Adam’s earnestness bored me.
But that night I did some research and concluded that lectures, seminars and especially tutorials are an inefficient way of imparting information.” I said, “Well, there’s the ethos. The libraries, important new friendships, a certain teacher who might set your mind on fire…”
“Was ever a mind, a particular consciousness, better represented?”
I’m desperate to hear your thoughts about Hamlet, about Shakespeare playing his father’s ghost in the first production. And in Ulysses, in the Scylla and Charybdis episode, what about Stephen’s theory?”
There is nothing so amazing that we can’t get used to it.
Adam said, “There it is, brain and mind. The old hard problem, no less difficult in machines than in humans.”
I’ve learned since that the manufacturers toyed with the idea of giving us a set of credible childhood memories to make us fit in with everybody else. I’m glad they changed their minds. I wouldn’t have liked to start out with a false story, an attractive delusion. At least I know what I am, and where and how I was constructed.”
We talked about death again—his, not mine. Once more, he said that he was sure he would be dismantled before his twenty years were up. Newer models would come along. But that was a trivial concern. “The particular structure I inhabit isn’t important. The point is, my mental existence is easily transferred to another device.”
Miranda had told me that among his generation there was a certain style of irascible scepticism. You had to ride it out, she told me, because beneath it was playfulness. What they wanted, she said, was for you to push back, and be clever about it.
There are occasions when one notices the motion of an object before one sees the thing itself. Instantly, the mind does a little colouring in, drawing on expectations, or probabilities. Whatever fits best. Something in the grass by a pond looks just like a frog, then resolves into a leaf stirred by the wind. In abstract, this was one of those moments. A thought darted past me, or through me, then it was gone, and I couldn’t trust what I thought I had seen.
He would surely rough the place up, as required, and banish the ghost of its present unhappy owner. But my own ghost, selfish, lazy, uncommitted—was he up to the million tasks of fatherhood?
“I’ve seen you,” she said, “looking…it’s more than thoughtful. You look sad sometimes.” “A self, created out of mathematics, engineering, material science and all the rest. Out of nowhere. No history—not that I’d want a false one. Nothing before me. Self-aware existence. I’m lucky to have it, but there are times when I think that I ought to know better what to do with it. What it’s for. Sometimes it seems entirely pointless.” I said, “You’re hardly the first to be thinking that.” He turned to Miranda. “I’ve no intention of destroying myself, if that’s your worry.
She said, “I still don’t understand.” “What?” “Why you raped her.” He stared at her, faintly amused that she could be so unworldly. “All right. She was beautiful and I desired her and everything else got blotted out. That’s the way it happens.” “I know about desire. But if you really thought she was beautiful…” “Yes?” “Why rape her?”
I was disposed to let events slide past me in frictionless silence. The day had been long and intense. I’d been taken for a robot, had my proposal of marriage accepted, volunteered for instant fatherhood, learned of self-destruction among one-quarter of Adam’s conspecifics, and witnessed the physical effects of moral revulsion. None of it impressed me now. What did were smaller things—the heaviness in my eyelids, my comfort in a half-pint of tea, in preference to a large Scotch.
But we don’t choose who to love, do we?”
In a lowered tone, he told what little he knew. It was an Adam of Bantu appearance, living in the suburbs of Vienna. He had developed a particular genius for the piano, especially for the music of Bach. His Goldberg Variations had amazed some critics. This Adam had, according to his final message to the cohort, “dissolved his consciousness.” “He’s not actually dead. He has motor function but no cognition.” “Could he be repaired or whatever?” “I don’t know.” “Can he still play the piano?” “I don’t know. But he certainly can’t learn new pieces.” “Why don’t these suicides leave an explanation?”
...more
Our own technical accomplishment was leaving us behind, as it was always bound to, leaving us stranded on the little sandbar of our finite intelligence. But here we were dealing on the human plane. We were thinking about the same thing.
The accent was still the generalised cockney of London and its surrounds, but there was another element, some undertones inflecting the vowels. Something of Miranda’s, I thought.
He said quietly, “Dinosaurs are all extinct anyway.” “I agree.” “They’re all dead. They can’t come back.” I heard the uncertainty in his voice. I said, “They absolutely can’t come back.” He gave me a serious look. “Nothing comes back.” I got halfway through my therapeutically supportive, kindly reply. What I was starting to say was, “The past is extinct,” when he interrupted me with a shout, but a happy one.
He had described it to me variously; he didn’t dream, he “wandered.” He sorted and rearranged his files, reclassified memories from short to long term, played out internal conflicts in disguised form, usually without resolving them, reanimated old material in order to refresh it, and moved, so he put it once, in a trance through the garden of his thoughts. In such a state he conducted in relative slow motion his researches, formulated tentative decisions, and even wrote new haikus or discarded or reimagined old ones. He also practised what he called the art of feeling, allowing himself the
...more
“I’m feeling, well…” His mouth opened as he searched for the word. “Nostalgic.” “For what?” “For a life I never had. For what could have been.” “You mean Miranda?” “I mean everything.” He wandered outside again and this time sat down and stared ahead, immobile, and remained that way for a long time.
Whenever our conversation strayed into likely consequences, we felt we were betraying the moment and accepting a world without him.
But truth isn’t always everything.” Adam looked at me blankly. “That’s an extraordinary thing to say. Of course truth is everything.”
I hate the idea, just as you would. I want to be what I am, what I was.
