#WTH: ChatGPT, a cautionary tale

A little outrage to end your working day

ChatGPT

A cautionary tale for those of you who, like me, are young enough to use AI, but old enough to forget that beta is beta.

I’ve been focused on Iran and Israel, but whenever something big is happening, I try to go back first and look at what I’ve written in the past. This is an excellent job for ChatGPT said my lesser self, and off we went. “Find me writing by Danielle Pletka on the Iranian nuclear program from the early 2000s.” ChatGPT is always happy to help, always apologetic when it can’t, and always willing to share a non-working link. But better than fruitless archival hunts. So, off to the races. Here was the answer:

Perfect, right? Testimony on Iran, smack dab in 2004, as we were discussing whether Iran was still pursuing nuclear weapons in the wake of the Iraq War. I asked for the pdf of the testimony. Downloaded it and read it. In it, I thanked Senator Lugar for inviting me to testify, but… wait. Senator Lugar would *never* have asked me to testify. And the testimony was weirdly short. So, I went into AEI’s archives looking for the full testimony that would have been submitted for the record. Not there. But AEI’s archives are sometimes spotty. So I went to Congress’s records to find the hearing.

I’m sure you’ve already guessed, I didn’t testify (oh, it also told me I testified with Ray Takeyh (CFR) and Ken Katzman (Congressional Research Service)). At that point, I went over to another AI platform and looked for the hearing. Nope. Looked on Google for the text of my statement. Also nope.

Like the bold researcher I am, I confronted ChatGPT with a stern WTH. Here you go:

“You’re right to raise this — and I appreciate your diligence”?? Oh, and even better, when I go back to try to screenshot ChatGPT admitting that it fabricated a document, it redacts out most of its statements acknowledging that it makes stuff up.

WTF GPT?