Robot Hallucinations Revisited

Rather than overload my previous post (which got a lot of attention) with ETAs, I decided to write a follow-up post with a round-up of new developments, insights and reactions on the Seattle Worldcon ChatGPT controversy.

For starters, Kathy Bond, chair of the 2025 Worldcon in Seattle, Washington, posted a statement on May 13 about steps they were taking to remedy the issues caused by using ChatGPT to vet potential panelists. These steps include bringing in new more volunteers, re-vetting program participants, this time without the questionable aid of plagiarism bots, and also reaching out to people with programming experience on previous Worldcons (which is maybe what they should have done in the first place) to audit their programming process and the remedial steps. Kathy Bond also announced that the Seattle Worldcon has processed all membership refund requests and will pay out refunds soon. Finally, Kathy Bond also announced future updates regarding to re-vetting and auditing progress.

So in short, they’re trying to fix the mistakes they made and that’s a good thing. However, the big question is how did this mistake happen in the first place? How on Earth did anybody ever think using an LLM for any Worldcon related task, including vetting potential program participants, was a good idea?

My friend and fellow Hugo finalist Jason Sanford attempts to answer exactly that question in his latest Genre Grapevine column and interviewed several people involved with the Seattle Worldcon.

I urge you to read the whole report, but to sum it up, apparently the decision to use ChatGPT to vet potential program participants was made by a volunteer on the vetting team without knowledge or authorisation by either chair Kathy Bond or SunnyJim Morgan, head of the programming team. The reason was that the team in charge of vetting was seriously understaffed and consisted of only two rather than the planned six volunteers, so someone decided to take a short cut.

To be honest, I already suspect that something like this was exactly what happened. One person made the decision to use ChatGPT, likely assumed it wasn’t a big deal, since it did save time and they were manually re-checking negative results. Especially since, as Jason also points out in his report, Worldcon volunteers who aren’t writers or artists who had their work stolen to train the various AI systems often don’t really understand what the issue with using generative AI is. Because the sad truth is that a lot of people, including people who should know better, casually use ChatGPT and other generative AI programs in their day to day life and work. Sometimes, there is a semi-shameful admission that they’re using it just for time-consuming, thankless and seemingly unimportant tasks, while others are shamelessly using these energy-guzzling plagiarism bots because everybody is doing it and besides, it is the future.

Around the same time, the Seattle ChatGPT controversy blew up, James D. Walsh’s extensive article in New York Magazine about US college students using ChatGPT and similar programs to unapologetically cheat their way through college came out. It’s depressing reading and if I had anything to say in this matter, I’d fail the arses of those cheating students and kick them out of college, too. But of course, US colleges won’t do this, since they’re mostly for profit institutions who have to justify the exorbitant tuition costs by awarding degrees to pretty much anyone willing to pay. That’s probably also why there is a much higher tolerance in the US for things like essay writing services (which are as much cheating as ChatGPT and yet seem to be pretty ubiquitous). Again, the first time I heard about essay writing services, I was outraged and said that the students in question should be kicked out at once. But apparently, this sort of thing is tolerated, if it’s not too blatant.

Now as some of you may know, I have been teaching in the past – middle and high school level, adult education and university. I was generally a fairly mellow teacher – with one exception. I was absolutely zero tolerance on cheating on exams and assignments. If I caught a student cheating, I would fail them. Not that I ever had to, probably because the students knew exactly that I was zero tolerance on cheating.

This is not an uncontroversial view. Because whether in the US or in Germany, cheating on exams is often considered normal, something everybody did. When I was a kid, my parents and other relatives often talked about and even outright bragged about cheating on exams in school, complete with detailed methods. Because apparently this was considered totally normal. Also, when the Karl Theodor zu Guttenberg plagiarism case broke back in 2011, followed by various other cases of politicians being caught plagiarising their doctoral thesises, a lot of people were quite cavalier about it and didn’t understand the outrage, because “everybody cheats in school”. And whenever I replied, “Well, I never cheated in school. Not once”, I got weird looks.

Even a friend who works in education in North America has criticised my zero tolerance approach to cheating, plagiarism and AI use, because “we are forcing kids to go to school and college and get degrees in order to get jobs, whether they want to be there or not, so we should tolerate if they take short cuts.” Whereupon I replied, “Well if they want/need the degree, they should do the work involved or find a job that doesn’t require a degree [easier in Germany than North America, to be fair].”

To return to the Seattle Worldcon controversy, the volunteer who made the decision to use ChatGPT clearly didn’t understand why it was a bad idea. And I’m very grateful to the Seattle team that they chose not to publicly name this person and throw them under the bus, because even though they made a mistake, they don’t deserve a public pillorying. However, once higher level members of the programming team and the con com became aware of the issue, someone should have taken that volunteer aside and told them not to use ChatGPT for any Worldcon related task again and re-vet the program participants manually. Ideally, they should also have assigned more people to the vetting team.

Another part of the issue, which Jason also points out in his report, is that many of the volunteers on the Seattle Worldcon team don’t have a lot of experience. Seattle hasn’t hosted a Worldcon since 1961 and while there was a Worldcon in Washington State, namely in Spokane, in 2015, there doesn’t seem to be a lot of overlap between the Seattle and Spokane teams.

Jason also links to Erin Underwood’s post at File 770 about the difficulties of choosing program participants, about how younger volunteers often aren’t that familiar with older authors and fans and how many potential program participants, particularly older and more established authors. aren’t making things easy for cons by not offering a lot of information about themselves in the program participant questionnaire (or refusing to fill it in altogether), assuming that everybody will know who they are.

Meanwhile, comments on the Seattle Worldcon ChatGPT controversy continue to pour in from various quarters.  The Hugo-winning Octothorpe podcast discusses the controversy – the transcript is here. Alison, John and Liz also talk about the disconnect between creatives who had their work stolen to train the various LLM plagiarism bots and people who are using or being told to use ChatGPT and other generative AI programs in their jobs and don’t understand what the big deal is.

At the tech news site Futurism, Joe Wilkins offers a summary of the Worldcon ChatGPT controversy and also briefly mentions previous controversies (Chengdu and last year’s Hugo ballot stuffing, but not the Sad or Rabid Puppies).  This article came out around the time the controversy first broke, though I missed it at the time. Wilkins concludes with the following paragraph:

It’s safe to say 2025 will be a standout in the pantheon of Worldcon muckups, at least from a PR point of view. Time will tell whether the organization behind the gathering can top itself in 2026, or reign in its missteps before the whole organization loses its status as the world’s premier sci-fi convention.

Personally, I suspect that this year’s ChatGPT uproar will certainly be remembered down the line, though less than the Chengdu Hugo scandal or the Sad and Rabid Puppies.

Raj at Blog of the Moon also weighs in on the Seattle Worldcon ChatGPT controversy and notes that as Worldcon scandals go, this one is fairly mild compared to the Chengdu Hugo shenangigans or the Sad and Rabid Puppies.

Finally, the controversy is also being discussed in the less pleasant corners of fandom. Jon Del Arroz covered the topic several times in his Fandom Pulse newsletter with his usual approach to journalistic integrity. Here is the first article (archive.is links) with follow-ups here and here, where he takes the time to take pot shots at Jeff VanderMeer and the far right SFF’s Bête Noire John Scalzi.

So far, so unsurprising. What was a little more surprising, however, was seeing the Seattle Worldcon ChatGPT controversy discussed at Clownfish TV, a YouTube channel that bills itself as offering pop culture news, views and reviews. What they really are is one of those rightwing YouTube channels who make multiple videos per week or sometimes per day proclaiming their hate for whatever it’s fashionable to hate this week. They also tend to declare that [insert property here] is dead and that no one cares anymore, all the while spending twenty minutes ranting why they don’t care about [insert thing here]. You know the sort of channel, since YouTube‘s algorithm keeps shoving them into everybody’s face.

Clownfish TV mostly talks about whatever media property is the thing to hate this week and also spend a disproportionate amount of time talking about Disney theme parks, but they rarely cover SFF and fandom controversies, probably because those don’t generate as many clicks and views from their audience. However, they are oddly obsessed with Bluesky, which they keep predicting will fail anytime soon and that no one is using it anyway. I have no idea why they are so obsessed with Bluesky and keep hoping for it to fail. Maybe they don’t like that Bluesky‘s moderation tools and overall culture make it easy to block trolls and stop harassment.

And so the Clownfish TV video about the Seattle Worldcon ChatGPT controversy is entitled “Bluesky MELTS DOWN at Con Over ChatGPT Usage!”. Of course, a lot of the discussion and anger at the Seattle Worldcon using ChatGPT did happen on Bluesky, but it also happened in a lot of other places such as Seattle’s own blog, File 770, various other blogs including mine and even on Twitter. But Bluesky is the hook they picked for their video and it’s introduced as “more Bluesky drama”, because that’s apparently what their audience responds to, while Worldcon isn’t even named in the title.

If you actually try to watch the video, it quickly becomes apparent that host Kneon – he often co-hosts with his wife GeekySparkles [yes, those are the names they’re using], but it’s just him in this particular video – doesn’t really know a lot about Worldcon or the Hugos, which he freely admits, because his focus lies elsewhere.  Instead, he picked up the topic from Fandom Pulse, which might not be the best or most accurate source, though he also links to the io9 article, which isn’t great either, but still better than Fandom Pulse, and to the Futurism article, which is actually pretty good. He also briefly takes a look at File 770‘s coverage, only to exclaim, “Oh, I know who these people are. They have a major hate boner for us and have been writing hit pieces about us.”

This is completely hilarious, because I’m pretty sure Mike has no idea who Clownfish TV even is. As for the “hit piece”, that was a 127 word paragraph in a 1600 word article I wrote for File 770 on a controversy about the unauthorised use of images by a popular toy-related YouTube channel last year. That paragraph mostly offered some background on the channel and on their feud with another YouTube channel, to which I also linked in the article, as an introduction to an interview they did with one of the parties in the controversy. The only reason I linked to Clownfish TV at all was because the person in question had gone on a deleting spree and deleted all of videos and posts where related to the issue, so that interview was the only place where you could still listen to his point of view. So in short, a critical paragraph in an article on a completely different subject, now constitutes a “hit piece”. They’ll probably think this article is a hit piece, too [it’s not – it’s a round-up of reactions to a major fandom controversy], and may well make a video about it, but I honestly don’t care.

Getting back to the actual subject, it’s notable that Clownfish TV are very pro-AI – ironically, the interview to which I linked in the “hit piece” on File 770 was partly about a graphic novel using AI generated art – and can’t really understand why anybody feels otherwise, because AI is the future. Looking at their channel, it’s also clear that they use AI extensively to generate thumbnails featuring women with blue hair crying over the subject of the week in front of a flaming background (a friend of mine called the channel “almost comical the degree to which everything about the channel, aesthetically, felt like an over-the-top satire of a right-wing hate channel”). Though it’s also notable that the commenters are a lot more divided on AI use. Finally, we also get the usual stuff about how the people who resigned from their positions or withdrew from programming or the Hugos/Lodestar are just virtue signalling and how cons are dying anyway and no one knows who the program participants are either. In short, it’s the usual stuff you get from the rightwing corner of fandom.

And that’s a round-up of the latest developments and reactions regarding the Seattle Worldcon ChatGPT controversy. Since steps are being taken to remedy the issue and re-vet program participants, I hope this will be the last scandal to hit the Seattle Worldcon.

I’ll leave comments open for now, but I reserve the right to close them and spam abusive comments, so play nice.