Manny’s review of Why Can’t AI Coding Assistants Be Authors of Academic Papers? > Likes and Comments
29 likes · Like
AI co-authorship has kept me employed. Now I only have to "architect the documentation house," not architect and actually build it. I spent decades doing both jobs. I'm more than happy to just do one job.
MountainAshleah wrote: "AI co-authorship has kept me employed. Now I only have to "architect the documentation house," not architect and actually build it. I spent decades doing both jobs. I'm more than happy to just do o..."
Exactly. In the software world, it's now taken for granted that AIs will do most of the writing: they're obviously very good at it, better than almost all people and certainly far quicker. The same arguments apply in academia. But academics are obliged to play by different rules, for reasons which in fact are anything but clear. This is the paradox we examine in the long version of this paper, Vibe Research.
I don't see that academia can continue much longer with this stonewalling strategy; nothing works any more, in particular the reviewing system. I have stopped submitting papers to human-reviewed venues and now use AiraXiv and ResearchGate.
I worked as a ghost writer and did get no credit. And I did not care. I just did my job. – But in principle I agree. AI should get credit.
I have often thought I'd like to read a carefully researched book on ghost writers. I must look around. I've been particularly curious about the trashy French series Brigade Mondaine, obviously ghost-written by multiple people. My strong feeling when reading them is that the ghosts don't all feel the same way about the job. Some actually seem to be enjoying it. Some are just turning in workmanlike text for the money. Some look like they hate it and are doing it more or less against their will. I always wish I knew more.
I think I don't agree, mainly because AI is just a tool. We don't credit photoshop for running macros. I do believe in disclosing that AI was used. But not crediting a piece of software at the same level as a human being, because AI is not sentient and it can't have "ideas" of its own.
Mila wrote: "I think I don't agree, mainly because AI is just a tool. We don't credit photoshop for running macros. I do believe in disclosing that AI was used. But not crediting a piece of software at the same..."
This is a common objection, which I address in §2 of the paper. In fact, it's now clear that AIs can come up with complex and subtle ideas of their own. The most impressive and highest-profile examples are in mathematics, but just in terms of numbers I think the largest contributions are most likely in software engineering.
I think the problem with AI authors is that they can be used to deflect responsibility for any mistakes in the content of the paper. Authorship is not just a measure of ‘who did what and how much of it’, it is also a statement that the authors believe the work and are willing to defend it. An AI can contribute, but it can't reply to a referee, issue a correction, or put its reputation on the line, so AI authorship dilutes this responsibility. If a human author presents shit work and can't back it up, they are rightly ignored by the academy, but if an AI does, should the field ignore all outputs by that AI, across every version and every user prompting it? In reality you'd ignore the work of the co-authors, which tells you where the accountability actually lies. So what's the point of listing the AI as an author, rather than disclosing its use in detail in the acknowledgments or a tools/methods section, like any other software?
Chris wrote: "I think the problem with AI authors is that they can be used to deflect responsibility for any mistakes in the content of the paper. Authorship is not just a measure of ‘who did what and how much o..."
In fact, one of the key arguments in this paper (and even more in Vibe Research, the longer paper it refers to) is that high-end AIs can take intellectual responsibility for work, reply to referee questions, etc. We have gone as far as setting things up so that it's possible to log in to our platform and ask questions about Vibe Research and some other papers.
Caveat: this costs money for the AI time, about USD 1.00 per question. But if you're curious, then create an account, mail me, and I'll give you enough credit that you can ask a few questions.
Hmm, I’m not sure a Q&A interface is the same as taking intellectual responsibility for the outputs, but it’s an interesting idea. How do you ensure that the AI has a consistent context window from one conversation to the next? Or that the underlying ‘high-end’ AI model is not having its weights or pre-prompt quietly updated in the background? Essentially, how do you ensure that you get a consistent, reliable narrator? I find that subtle changes in the questions I ask can change the output I get from an LLM - would I be able to convince an LLM with the right argument to retract a paper in one chat, and rigorously defend it in another, just from tone, and not any changes in the substance of my argument?
These are excellent questions. Unfortunately, in my experience, they are equally relevant to human authors.
Considering it further, what one should ideally do is investigate the issue empirically: carry out the kinds of experiments you suggest with both AIs and humans, then evaluate the consistency of the answers, to the extent feasible using a double-blind methodology. But it would be a lot of work.
Informally, my immediate feelings, based on many years of interaction with human researchers and about a year of interaction with AI coding assistants, are as follows:
1. There is a lot of variance, but there is less variance with AIs than with humans.
2. The best humans are still somewhat better than the best AIs.
3. Average humans are somewhat worse than the best AIs.
4. The largest advantage the AIs have is that they're always available to answer questions. Even if a good human researcher is in theory able to answer more accurately than an AI, this isn't much use if they can't be bothered to reply to your questions, or give you a couple of perfunctory lines after a month.
For my first ten years writing book reviews, I was on my own. Unlike a reviewer at, say, the New York Times, where a review will pass through the hands of a team of editors who will check for accuracy, spelling, and grammar, as well as offering editorial suggestions. For these past four years I've used ChatGPT as my editor -- writing my review and then asking Chat to provide a spell check and editorial suggestions regarding grammar. I feel entirely justified in calling this work my own. All writers need a good editor.
Glenn wrote: "For my first ten years writing book reviews, I was on my own. Unlike a reviewer at, say, the New York Times, where a review will pass through the hands of a team of editors who will check for accur..."
I'm certainly not suggesting that all uses of an AI in the writing process mean it should be listed as an author. Sometimes, as you say, it's more appropriate to call it an editor. I'm just saying it can fill the (co-)author role too.
Manny wrote: "Glenn wrote: "For my first ten years writing book reviews, I was on my own. Unlike a reviewer at, say, the New York Times, where a review will pass through the hands of a team of editors who will c..."
Spot-on, Manny. I understood your meaning. I simply wanted to clarify for those who want to dismiss AI. As we mentioned previously, when used intelligently, AI can serve as a learned friend always at the ready to help.
back to top
date
newest »
newest »
message 1:
by
MountainAshleah
(new)
Oct 22, 2026 04:46AM
AI co-authorship has kept me employed. Now I only have to "architect the documentation house," not architect and actually build it. I spent decades doing both jobs. I'm more than happy to just do one job.
reply
|
flag
MountainAshleah wrote: "AI co-authorship has kept me employed. Now I only have to "architect the documentation house," not architect and actually build it. I spent decades doing both jobs. I'm more than happy to just do o..."Exactly. In the software world, it's now taken for granted that AIs will do most of the writing: they're obviously very good at it, better than almost all people and certainly far quicker. The same arguments apply in academia. But academics are obliged to play by different rules, for reasons which in fact are anything but clear. This is the paradox we examine in the long version of this paper, Vibe Research.
I don't see that academia can continue much longer with this stonewalling strategy; nothing works any more, in particular the reviewing system. I have stopped submitting papers to human-reviewed venues and now use AiraXiv and ResearchGate.
I worked as a ghost writer and did get no credit. And I did not care. I just did my job. – But in principle I agree. AI should get credit.
I have often thought I'd like to read a carefully researched book on ghost writers. I must look around. I've been particularly curious about the trashy French series Brigade Mondaine, obviously ghost-written by multiple people. My strong feeling when reading them is that the ghosts don't all feel the same way about the job. Some actually seem to be enjoying it. Some are just turning in workmanlike text for the money. Some look like they hate it and are doing it more or less against their will. I always wish I knew more.
I think I don't agree, mainly because AI is just a tool. We don't credit photoshop for running macros. I do believe in disclosing that AI was used. But not crediting a piece of software at the same level as a human being, because AI is not sentient and it can't have "ideas" of its own.
Mila wrote: "I think I don't agree, mainly because AI is just a tool. We don't credit photoshop for running macros. I do believe in disclosing that AI was used. But not crediting a piece of software at the same..."This is a common objection, which I address in §2 of the paper. In fact, it's now clear that AIs can come up with complex and subtle ideas of their own. The most impressive and highest-profile examples are in mathematics, but just in terms of numbers I think the largest contributions are most likely in software engineering.
I think the problem with AI authors is that they can be used to deflect responsibility for any mistakes in the content of the paper. Authorship is not just a measure of ‘who did what and how much of it’, it is also a statement that the authors believe the work and are willing to defend it. An AI can contribute, but it can't reply to a referee, issue a correction, or put its reputation on the line, so AI authorship dilutes this responsibility. If a human author presents shit work and can't back it up, they are rightly ignored by the academy, but if an AI does, should the field ignore all outputs by that AI, across every version and every user prompting it? In reality you'd ignore the work of the co-authors, which tells you where the accountability actually lies. So what's the point of listing the AI as an author, rather than disclosing its use in detail in the acknowledgments or a tools/methods section, like any other software?
Chris wrote: "I think the problem with AI authors is that they can be used to deflect responsibility for any mistakes in the content of the paper. Authorship is not just a measure of ‘who did what and how much o..."In fact, one of the key arguments in this paper (and even more in Vibe Research, the longer paper it refers to) is that high-end AIs can take intellectual responsibility for work, reply to referee questions, etc. We have gone as far as setting things up so that it's possible to log in to our platform and ask questions about Vibe Research and some other papers.
Caveat: this costs money for the AI time, about USD 1.00 per question. But if you're curious, then create an account, mail me, and I'll give you enough credit that you can ask a few questions.
Hmm, I’m not sure a Q&A interface is the same as taking intellectual responsibility for the outputs, but it’s an interesting idea. How do you ensure that the AI has a consistent context window from one conversation to the next? Or that the underlying ‘high-end’ AI model is not having its weights or pre-prompt quietly updated in the background? Essentially, how do you ensure that you get a consistent, reliable narrator? I find that subtle changes in the questions I ask can change the output I get from an LLM - would I be able to convince an LLM with the right argument to retract a paper in one chat, and rigorously defend it in another, just from tone, and not any changes in the substance of my argument?
These are excellent questions. Unfortunately, in my experience, they are equally relevant to human authors.
Considering it further, what one should ideally do is investigate the issue empirically: carry out the kinds of experiments you suggest with both AIs and humans, then evaluate the consistency of the answers, to the extent feasible using a double-blind methodology. But it would be a lot of work.Informally, my immediate feelings, based on many years of interaction with human researchers and about a year of interaction with AI coding assistants, are as follows:
1. There is a lot of variance, but there is less variance with AIs than with humans.
2. The best humans are still somewhat better than the best AIs.
3. Average humans are somewhat worse than the best AIs.
4. The largest advantage the AIs have is that they're always available to answer questions. Even if a good human researcher is in theory able to answer more accurately than an AI, this isn't much use if they can't be bothered to reply to your questions, or give you a couple of perfunctory lines after a month.
For my first ten years writing book reviews, I was on my own. Unlike a reviewer at, say, the New York Times, where a review will pass through the hands of a team of editors who will check for accuracy, spelling, and grammar, as well as offering editorial suggestions. For these past four years I've used ChatGPT as my editor -- writing my review and then asking Chat to provide a spell check and editorial suggestions regarding grammar. I feel entirely justified in calling this work my own. All writers need a good editor.
Glenn wrote: "For my first ten years writing book reviews, I was on my own. Unlike a reviewer at, say, the New York Times, where a review will pass through the hands of a team of editors who will check for accur..."I'm certainly not suggesting that all uses of an AI in the writing process mean it should be listed as an author. Sometimes, as you say, it's more appropriate to call it an editor. I'm just saying it can fill the (co-)author role too.
Manny wrote: "Glenn wrote: "For my first ten years writing book reviews, I was on my own. Unlike a reviewer at, say, the New York Times, where a review will pass through the hands of a team of editors who will c..."Spot-on, Manny. I understood your meaning. I simply wanted to clarify for those who want to dismiss AI. As we mentioned previously, when used intelligently, AI can serve as a learned friend always at the ready to help.