And You Will Know Us By The Interminable Appendices

Yes, things have been very quiet on here this month, partly because I’ve been trying to focus on making progress with my book and partly because I’ve been spending most of the rest of the time lugging hardcore from the bottom of the garden – and it’s a long garden, with too many steps to use a wheelbarrow for the whole stretch – and slowly hacking down the grass in the orchard where we’ve avoided mowing since last autumn, which yielded the wonderful result of a wild orchid appearing out of nowhere but is now bloody hard work. This all continues, so, apart from my ‘One Day Per Month’ journal entry at the end of the month, I wouldn’t expect any blog posts until September.

But I have been struck, amidst the ongoing collapse of Twitter and the slow migration to BlueSky (yes, I’m now over there as @nevillemorley.bsky.social, as well as on Mastodon as @NevilleMorley@historians.social, and I really must update various things to this effect when I have a moment…), by various interesting ‘Here we go again!’ teaching prep posts and discussions, as the new academic year starts to loom. And it occurred to me that I meant to post a copy of the guidance I’ve introduced into all my module handbooks on the use of LLMs and Generative ‘AI’, in case anyone finds them useful. It conforms to the general University guidance, which is words to the effect of ‘Check with your individual module director, we can’t stop you using it but if you submit AI-generated material as your own work we can come down on you like a ton of bricks’, but attempts to be slighter more friendly – not least because the dividing line between ‘AI-generated content’ and ‘content based in some part on AI-generated material’ seems to me tendentious in the extreme…

You can, if you wish, make use of ‘generative AI’ (e.g. Chat-GPT) in the development of your assessment, but this needs to be properly acknowledged; submitting AI-generated content as your own work constitutes academic misconduct. You do not need to acknowledge the use of AI tools for spell-checking or grammar correction, but any other usage, including the generation of background material for research or as a starting-point for your discussion, must be explicitly noted in your submitted work: at the end of the essay, before the bibliography, you must include a statement about which tools you have used and for what purpose, and after the bibliography you must provide a full description, including the prompts used and the original output. These statements do not count towards the essay word count. See the university’s guidance at https://libguides.exeter.ac.uk/referencing/generativeai.

Caution is strongly advised; gAI will at best give you a plausible-looking summary of information available on the internet, but its outputs are driven by probability models of what word is most likely to follow a given word, not by any criteria of truth or plausibility, and it is notorious for generating completely fictional references to scholarship. It is not in any way an authoritative source, and all its statements need to be critically evaluated. Where it may be useful is in generating material against which you can test your own critical skills, identifying flaws and errors.

A couple of comments. Firstly, this is very much a holding measure, as I have every hope that by the end of the coming academic year, having completed my project on how to develop the assessment of historical skills in the age of LLMs, I’ll have a much better idea how to respond; I can’t change the assessment of these modules, so this is an attempt at holding some sort of line, and trying to bring it all out into the open rather than issuing a blanket ban and (probably) just pushing the students who might think of using Chat-GPT into doing it secretly and probably badly.

Secondly, I am very conscious that this creates a whole load of new requirements on top of the usual things expected of students in presenting their work. I am simply following the guidelines in demanding information about prompts, output etc. – but can’t help wondering whether this is actually, secretly, intended as a deterrent, as it will just look like too much trouble. (It is also potentially a load more stuff for me to read, and I don’t know how useful this supporting material will actually be – but it cannot be as much work as trawling through a bit of coursework trying to establish whether the quotes and references are genuine or not). Again, I firmly expect that I’ll be in a position to develop this next year, thinking about how far (whether) Chat-GPT and the like can be actually useful rather than an annoying shortcut, and how far critical skills in LLM-wrangling can realistically be incorporated into assessment.

Thirdly, yes, I have taken the opportunity to flag up the deeply problematic and unreliable nature of gAI output, and will be very happy to mention stuff about its environmental impact, social costs, inbuilt biases etc. given the opportunity…