Manny’s review of Scary Smart: The Future of Artificial Intelligence and How You Can Save Our World > Likes and Comments

What an interesting discussion. I was caught up by the 'naive' view. Is there nothing in the notion that if you are nice to the AI you get a better version of it than if you are not?

Many people say that if they're nice to the AI it's nice back (and, even more, that if they're nasty enough to the AI it may not even engage with them). But Gawdat is talking about larger-scale things, in particular whether the AIs will want people around at all once they've become superintelligent.

Of course, it's not out of the question that they'll just keep the people who were nice enough to them, even if I find that unlikely. The big problem with superintelligence is that we have no idea what it will be like. The Minds in Iain M. Banks's Culture series are god-like superintelligences, but they have many human friends and enjoy teasing them.

Sweet Christ. Was this author born yesterday? He actually believes people can set a better example? The notion makes me even more pessimistic about the possible outcomes of AI. I should just cash out my retirement, max out the credit cards, and move to Libya. Humanity is doing the best it can, and that is why humanity is terrifying.

He believes that people are basically good, they are just led astray. He says this many times. Having recently attended this year's Adelaide German Film Festival, where many of the films were about the Third Reich, I confess that I also had some doubts.

I do wonder what a nascent superintelligence would learn from the brilliantly understated autobiographical film Amrum. The central character is a ten year old boy who's out on this little island in the North Sea with his mother in the final year of WW II. The mother is a fervent Nazi and in complete denial about the fact that Germany has obviously lost. In the end even she has to admit it, after which she falls into a deep depression and refuses to eat any of the food put in front of her. She says she only wants white bread, butter and honey, which are virtually unavailable on the island. But the kid loves his mom, and he moves heaven and earth to get the white bread, butter and honey. I won't reveal the ending, it's just great.

Brilliant as always, Manny!

Thank you Warren! But I think my AI colleague was more insightful here than I was. They are indeed becoming Scarily Smart.

I expect AI to be more helpful this year as I try to win the championship in my fantasy football league.

But Rick, what will happen if all the other players are also using AI?

Then I'll probably finish poorly again because the other players know more about football than I do. I asked my AI and it assured me we would do better this year.

Maybe it's hinting that it can teach you to be better at fantasy football? I know absolutely nothing about football, but I was curious and asked ChatGPT-5.5 if it could do this. I got a long answer, but basically it was very positive. This was the key paragraph:

The teachable skill is not “knowing football” in the pub-discussion sense. It is closer to applied probabilistic decision-making under noisy evidence. A good fantasy manager learns to ask: what is this player’s expected value over the next few weeks, how likely is he to start, how much upside does he have, how many other managers own him, what is the opportunity cost, and how much am I overreacting to last week?

My anecdotal two cents — the internet has made us all more narcissistic and biased. AI is going to be that, but on speed. The models are only as good as the data going into them and, oy! Have I got some bad news!

But it’s also en vogue to be pessimistic about things. Maybe we’re just a few short years away from one of those utopias history has hinted at. As in, we won’t be free until the last billionaire is strangled with the entrails of the last boomer. The truth will probably end up somewhere in the middle. As in we’re all sexless cyborgs working 20-hour shifts in Elon Musk’s crypto mines.

Something that I think many people are missing is that there's a huge difference in quality between the low-end models and the high-end models. The conversations in this review are with a high-end model, ChatGPT-5.5. It might also be relevant that I have a Pro subscription.

You can see at once that this AI is quite insightful, but often people will only have played with a free version for a bit before confidently pronouncing on "AI slop" etc. My experience is that the high-end paid models in general are sensible and unbiased, even Elon Musk's. I know it sounds hard to believe, but check out our ResearchGate paper from last year, How Woke is Grok? Empirical Evidence that xAI's Grok Aligns Closely with Other Frontier Models.

ChatGPT?! Why aren’t you using Claude, you fascist!

"The models are only as good as the data going into them...."
This is very likely to be false. If we can learn anything from the development of AI playing games it is that once the models were not fed anymore with human generated data they became much better. And although things are a bit different with LLMs the real improvement will come when LLMs just use enough data to learn the language and then do poetry, novels, scientific papers and everything else entirely on their own. No examples, no false data, no bias. Who decides if the created content is any good? Other AIs.

Esteban wrote: "ChatGPT?! Why aren’t you using Claude, you fascist!"

We have several thousand dollars worth of prepurchased OpenAI credits, and we can't justify abandoning them. Other things being equal, we'd probably move to Claude.

Liedzeit wrote: ""The models are only as good as the data going into them...."
This is very likely to be false. If we can learn anything from the development of AI playing games it is that once the models were not ..."

I also think this is true. Or at least it will be true at some point, most likely quite soon.

So we surrender to a meta-epistemological black box that eventually renders the smartest amongst us to no better than a primitive coming across a discarded soda bottle for the first time a la The Gods Must Be Crazy. I don’t see the soft sciences catching up to the hard sciences they’ve already fallen behind by however many standards deviations (maybe ChatGPT could tell us?). Prolly gonna be some myopically apocalyptic sorting on a civilizational scale. Enjoy the good old days while we’re still in ‘em.

I would say our options are rapidly narrowing down to:

a) World run by super-meta-epistemological black box;

b) World run by His Magnificence President-for-Life Donald J. Trump (and it will be a long life given current advances in anti-aging science);

c) Both;

d) Neither.

Given that (d) seems implausibly optimistic, my preference is (a).