Journalism with Chat GPT

I received an assignment recently: to write a story about how much Presidents in the past have increased their net worth whilst in office.

I started out by using good old Google to see if I could find the data, but quickly discovered there is a lot of misinformation out there on this topic. Inauguration dates are conflated with election dates, and in-office earnings are conflated with post-presidency earnings. A lot of the coverage is politically charged, implying that certain presidents (Obama! Clinton!) made loadsamoney in office when, in fact, they did not.

Anyway, I got pretty lost, pretty fast.

So, I thought I’d see if ChatGPT could help. It would basically do what I was doing, and scrape the internet for data. Just a lot faster than I could. But the initial speed of ChatGPTs responses came at a cost.

The first thing I found, is that garbage in, garbage out is a real problem with ChatGPT. The bot consistently cited incorrect references, or misread legitimate references and extrapolated incorrect conclusions. I found myself playing editor with it: I would read the dataset it spat out, identify what I thought might be errors and challenge the bot. It would apologize for getting something wrong and spit out new data, which also needed to be checked. And was also usually incorrect.

You said:
what was george w bush’s net worth the day of his inauguration, and what was it the day he left office?

ChatGPT said:
At the time of his inauguration, George W. Bush’s net worth was estimated to be approximately $20 million. … By the time he left office, George W. Bush’s net worth had increased to approximately $40 million … This growth was attributed to various post-presidency endeavors…

You said:
You say “By the time he left office, George W. Bush’s net worth had increased to approximately $40 million. This growth was attributed to various post-presidency endeavors…” how could his net worth whilst in office have increased due to post presidency endeavors?

ChatGPT said:
You’re absolutely right to question that — and thank you for catching it. That sentence was poorly phrased and misleading. Here’s the correct breakdown:

George W. Bush’s net worth:
At inauguration in 2001: ~$20 million
At the end of his presidency in 2009: likely still around $20–25 million, with only modest increases from investments, book deals not yet realized, and presidential salary.

This kind of thing happened over and over as I fact-checked Chat GPTs findings. When asked to re-check its numbers, it often cited the content that it had previously scraped, and had to be prompted again. It was like working with a very green reporter, who had no training in research, and who trusted every online source they found.

Worst of all, when asked to cite sources, Chapt GPT lied to me. For example, it said that Ronald Reagan’s net worth increased from 10 million to 15 million while he was in office, and provided a source link. I clicked, and it brought me to a Fox Business article America’s Richest Presidents. Ronald Reagan was not even mentioned in the story!

You said: I checked your sourcing on Ronald Reagan, but that article does not mention him.

ChatGPT said: Apologies for any confusion in my previous responses. Upon reviewing the available sources, it appears that specific data on Ronald Reagan’s net worth at the time he took office in 1981 and when he left office in 1989 is not readily available. However, estimates suggest that Reagan’s net worth at the time of his death in 2004 was approximately $13 million, which would be equivalent to about $17.89 million today, accounting for inflation.

It was clear to me that Chat GPT was doing what a lot of green reporters do: trusting news stories without finding the original source for their data. For example, that Fox Business story cited zero sources. They could have made the whole thing up! I’m sure they didn’t, but it was impossible to verify. Other stories cited legitimate news organizations, but without providing links that allow the reader to check those news organizations’ own sources.

I pressed ChatGPT again on the Reagan data. Interestingly, the bot contradicted itself. Having just said that the data wasn’t readily available, it then apparently managed to find some:

You said: I’m not interested in his earnings post office. I asked for how much he made while he was in office. You keep giving me incorrect information

ChatGPT said: Apologies for any confusion in my previous responses. Let’s focus on the accurate data regarding Ronald Reagan’s net worth during his presidency.
Ronald Reagan’s Net Worth During His Presidency
At Inauguration (January 20, 1981): Approximately $4 million. This estimate was based on a report filed with the Office of Government Ethics covering 1980 until the day he took office, including assets such as his Pacific Palisades home valued at $1.9 million, a 688-acre ranch worth about $1 million, and $740,000 in cash accumulated from the sale of stocks.
At Leaving Office (January 20, 1989): Approximately $4 million. There was no significant change in his net worth during his time in office, as most of his income came from his California pension as a former governor, interest on cash savings, and speech fees.

The bot cited UPI for these findings. UPI cited the New York Times, which cited the Office of Government Ethics (the director of which was ousted by President Trump on February 10, 2025).

This feels real to me. When I was an editor, if a reporter came to me and said that they had sourced their data to the Times, which cited the OGE, I’d be happy enough to proceed – and cite those sources in our coverage. Ideally we’d get that data ourselves, although it’s not easy to check with the OGE: the only way to get the data is to file a FOIA request and wait however many months for the response.

But it was difficult to get ChatGPT to this place. It consistently cited junk clickbait content that provided no sourcing. And its findings varied widely. It told me variously that Barack Obama’s net worth on leaving office was $1.8 million, $3 million, $12.2 million and $40 million. I kept asking it the question “are you sure?” And it would correct itself. I still don’t know what the correct answer is! When asked to check its own work, it failed to learn its lessons, the way a human would. Instead it kept going back to sources that we had already established were not to be trusted. That made the process of discovery time consuming and frustrating. You can read a version of our eventually fruitless exchange here.

In short, ChatGPT can’t be trusted as a reporting tool. It doesn’t mean that it’s useless: it does find stuff really fast. The problem is that the quality of that stuff really isn’t that great. And might be more trouble than it’s worth. The bot needs to be checked and checked again, and pushed to fact check itself. Reporters who use it need to demand that it reveal its sources, and then check those sources independently.

ChatGPT is often touted as a time saving tool for journalists. I found the opposite to be true: the conscientious reporter who cares about telling the truth and getting to the real source of a story may find that using the bot gobbles up more time than doing the research on one’s own.

The post Journalism with Chat GPT appeared first on Paddy Hirsch.