A New ChatGPT Scam

I’ve mentioned before (perhaps to ad nauseum) that I don’t really like generative AI. Overall, I think it’s as bad for the mind as meth is for the body.

So today I’m going to share about an exciting new scam for writers courtesy of ChatGPT.

Lately I’ve been getting a lot of extremely detailed emails praising my books. That’s nice, except the emails are always written like a corporate memo and maintain numerous detailed references to stuff that happened in a specific book. If I email back, suddenly I’ll get an elaborate pitch for various book marketing services.

Scammy book promoters selling dubious marketing services, alas, are nothing new in the indie publishing space. But the twist with this one is the scammer feeds the text of your book into ChatGPT (or possibly Claude), which then generates both a summary and a letter praising the book, including the specific details and plot points.

So it’s kind of like very personalized spear phishing.

Now, I’m old and cynical (in indie author terms, I’m almost but not quite primeval), so there’s no way I’m signing up for random book marketing services from a stranger on the Internet.

But! New writers very much want validation more than anything else in the world. That’s why you occasionally see a social media storm about a newish writer who gets a bad review, flips their lid, and then endures a pile-on as they become Internet Main Character for a day. So I suspect new writers would be very vulnerable to a letter flattering their book, and will end up spending money on “book marketing services” that provide no real benefit.

That said, I will admit this is a clever scam. I suspect that generative AI like ChatGPT is essentially a Narcissus Machine – it spits out whatever you want to see or hear, so you sit there gazing at essentially the reflection of your own mind like Narcissus enraptured by his reflection in the pool. (This is likely why you occasionally encounter people who think that they’ve taught sentience to ChatGPT, or mentally ill people who think that they’ve attained higher consciousness and ChatGPT is telling them the secrets of the cosmos.) So rather than using the Narcissus Machine on himself, the scammer is weaponizing the Narcissus Machine at someone else in hopes of selling the target dubious marketing services.

Clever. Morally bankrupt, but clever. Unfortunately, I suspect this model of scamming can be applied to more lucrative fields of fraud. Some high-level medical or tech executive is going to get talked into giving up information by a scammer creating a ChatGPT girlfriend for him.

So, I suppose there’s no grand conclusion here, but two points.

1.) Be very wary of any email you get.

2.) I keep hearing how generative AI will lead to amazing scientific advances and the curing of chronic diseases. What we actually seem to have gotten is raised power bills, mass layoffs, badly coded applications, questionable image memes, and a supercharged tool for scammers. So I remain dubious about the benefits of generative AI, mainly since they have yet to materialize.

-JM