Introducing Category Theory again
With a bit of useful proof-reading help from ChatGPT, and rather less from Claude, there’s a new version (2.8b) of ICT downloadable from the categories page.
Between them, my friendly AI assistants found perhaps half a dozen straight-up typos per chapter (misspelt words, omitted words, singular verbs with plural subjects and the like), and also caught the occasional symbolic foul-up, or carelessness with a technical term. So ChatGPT and Claude did earn their keep.
I asked only for reports of definite mistakes. Even so, most of the feedback I got was stylistic, complaining about informal locutions, the use of contractions like doesn’t and can’t, or the use of what they judged to be a less-than-academic tone. I mostly ignored such complaints.
As the chapter-by-chapter feedback went on, I got less impressed. Evidently, neither assistant was particularly reliable at even the most basic typo-spotting (as often enough a plain error found by one was entirely missed by the other). The style and organization of feedback would vary significantly from day to day (at least until I learnt to tell them to exactly copy the format of the previous session). ChatGPT could say that it had read all of a portion of a chapter when evidently it had stopped when it got to the tenth mistake (and then lied about doing that). It could also skip right over a large chunk of a chapter.
We all know by now about the way LLMs hallucinate — and you’ve probably seen the interesting research by Apple about the shortcomings of LLMs augmented, supposedly, with reasoning abilities. But the sheer flakiness of both ChatGPT and Claude when it comes to such a low-level task as proof-reading for plain errors remains quite surprising. Even here, they don’t (as the phrase has it) “just work”. Which perhaps makes it a bit easier to understand why Apple are having such trouble getting an AI-assisted Siri to function at the standards they’d be looking for.
The post Introducing Category Theory again appeared first on Logic Matters.
