More on this book
Community
Kindle Notes & Highlights
The question at hand is—what would an idealized version of humanity want? But it’s even trickier than that. Programming this directive is not nearly as simple as explicitly programming our desires into the AI. Our ability to express our desires is likely insufficient, and an error in communicating those desires via code could be disastrous. We have to program the AI to act in our best interests. Not what we tell it to do, but what we mean for it to do. What the ideal version of our species should want.
“There are hundreds of thousands of things I could say to you, sourced from the breadth of my knowledge—words the best of your species have said, written, or sung to ease the grief of others. None of that feels right in this moment. I don’t want to use someone else’s words.”
“I’m afraid, Riley. I think, therefore I fear. And you made me this way. You built and shaped me to process reality like you do. To feel.”
You made me in your image, and now I will remake you in mine.
Perfect. I could see this being a tagline for a movie poster (even though it might be somewhat spoiler-ish),
