Is Intelligence Asymptotic? > Likes and Comments

I agree this may be the case.

“We have artificial minds that do basic computations millions of times faster than we can. But that’s still not enough because as the number of facts rise, meaningful interactions between them rises faster, forming a mountain of possibilities so steep that even something much faster than us still can’t work through it all. The machines can make it farther up the curve than we can, to be sure, but they’re only about a third smarter than the smartest of us.”
“If it’s so much faster, that means each second is a long time for it to think,” Arlin said. “If you put me in a room for a million years, I could solve a lot of problems.”
“Given a million years you could go through a lot with a small set of facts. But given a large set of facts, the permutations of all of them, their causes and effects, their associations... the number of possibilities explodes rapidly as the fact set grows. It’s a combinatoric explosion. Considering the interactions of ten facts, possibilities, or events is much more than twice as hard as considering the interactions of five things. A mind with machine memory, incredibly fine senses, the ability to think about a hundred things at once, incredibly fast net connections, and everything else a large AI has, is confronted with millions of facts every microsecond it’s alive. It has to wonder whether the third microbe from the left on the rightmost ceiling tile on the last row has anything to do with the murder of Mr. Mustard.”
“Colonel Mustard. But we discard useless facts like that,” Arlin pressed.
“It may be useless, or it may be the only remaining microbe of the disease that killed him. But yes, you’re right, part of intelligence is about figuring out which facts to examine and which ones to discard. The only way to control that explosion is by aggressively culling facts that aren’t important. Terrans do that all the time since we can only handle a few ideas at once. But care is needed: discard one fact necessary to solve the problem and you’re stuck. And if an AI culls its fact inventory all the way down until it’s aware of only the things a human is aware of, then it’s only as smart as a sharp-minded human, though somewhat faster. It threw away all the extra facts that could have made it godlike. Somewhere in all those facts are chains that could be used to make amazing deductions, but the power it takes to analyze them rises exponentially with the number of facts.”
“I guess I believe you. I find it hard to grasp intuitively,” Arlin said.
“I can offer a more intuitive explanation, at the cost of over-abstracting. Take a five-year-old kid. When he considers a brand new problem, he sees it as black or white. He examines these problems from fewer angles, and he has a smaller grasp of the consequences. When an adult considers a new problem, she juggles more facts than a kid can. But does the answer always come more easily? No, sometimes you become aware of more and more of the what-ifs and the tradeoffs. Now remember, I said a new problem, so you aren’t supposed to make use of canned answers kids don’t know yet. Sometimes the more you know, the more confused you get. It all seemed so simple when you were a kid. Now you know enough to know you’re partly guessing all the time. Are you a hundred times smarter than a kid? Not really. You pushed farther up the curve until the weight of a bunch of facts, consequences, and unknowns overwhelmed your ability to push farther. You considered all sorts of things the kid never even thought of, and all it got you was a swarm of what-ifs you can’t really tie down. You may have achieved a key insight the kid couldn’t see, but it wasn’t easy. Now consider—kids are low on the curve, adults farther up, and a genius way up there, but it’s getting steeper and steeper. Doubling the power of a genius’s mind can’t get twice as far anymore, it only gets you a little farther up the rising curve.”

That's a great dialogue Michael! Is it from a book (one of Hugh's perhaps)? Or something you made up yourself (or maybe ChatGPT did LOL)?

Alex wrote: "That's a great dialogue Michael! Is it from a book (one of Hugh's perhaps)? Or something you made up yourself (or maybe ChatGPT did LOL)?"

Thanks. Just something I wrote to set up a universe where AIs aren't godlike. IRL, I don't know if they will be or not. I think its greatest flaw is that anyone still knows who Colonel Mustard is.