The Detection of ChatGPT’s Textual Crumb Trails is an Unsustainable Solution to Imperfect Detection Methods

1 Ethical Limitations of Using ChatGPT, and Associated Risks in Academic Publishing

OpenAI’s large language model (LLM) Generative Pre-trained Transformer (ChatGPT) received a very positive welcome in academia, aspects that are not covered herein because they are numerous and self-explanatory, but it has also been the subject of some criticism, as is focused in this article. ChatGPT is a disruptive technology because it upends established norms regarding human-based innovation (Sardana, Fagan, & Wright, 2023) for four reasons. First, ChatGPT is an artificial intelligence (AI)-driven tool that is, through training, only able to recreate newly phrased ideas from a finite body of already-existing information that humans initially created; i.e., it is not a source of originality or intelligence per se, in the humanistic interpretation of these terms (Floridi, 2023). Second, older versions like ChatGPT-3 might not perform remarkably well, including the introduction of errors, such as to references (Walters & Wilder, 2023), known colloquially as “hallucinations” (Alkaissi & McFarlane, 2023; Azamfirei, Kudchadkar, & Fackler, 2023). Third, some consider these errors as “fabrications and falsifications” (Emsley, 2023). Finally, humans with innovative thinking may feel threatened by ChatGPT’s “originality,” specifically the fear of their human skills being substituted by AI. In an academic setting, scholars who decide to rely exclusively on their own human endeavor may feel threatened by competing scholars who rely on this technology, or fail to declare its use transparently, and thus feign originality.

Such concerns and threats were captured quite rapidly by the academic community following the use, starting in late 2022, of ChatGPT-3, leading publishers and globalist organizations dedicated to ethics policy, like the Committee on Publication Ethics COPE (COPE, 2023) and the International Committee of Medical Journal Editors (ICMJE) (ICMJE, 2024), to quickly implement policies and guidelines that did not ban the use of ChatGPT in academic articles, but instead placed limitations, such as the illegitimate authorship of ChatGPT (Nazarovets & Teixeira da Silva, 2024) or of those who claimed to be authors of text that was “created” by this LLM. Instead, COPE and the ICMJE opted for policies that require authors to declare and acknowledge the use of AI/LLMs, indicate the exact text derived from it, and specify in the methodology how it was used (Teixeira da Silva & Tsigaris, 2023; Teixeira da Silva, 2023a).

However, close scrutiny of the COPE “position” statement and the ICMJE recommendations reveals statements that carry limited practical weight. This is because, despite the ethical guidance and advice to over 14,100 member journals and publishers in the case of COPE (COPE, 2024), there is a distinct flaw in those guidelines: they fail to indicate the agent of responsibility when it comes to detecting the use of AI/LLMs or text derived from their use (e.g., in the case of ChatGPT). It has been fervently argued that the responsibility of detection lies squarely on the shoulders of journal editors (Kaebnick et al., 2023; Teixeira da Silva, 2023b). Even if there is currently no effective method to detect ChatGPT-derived text, this does not absolve editors (or publishers) of their responsibility to detect it. Due to the absence of efficient detection methods, yet aware of the risks of the undeclared use of AI/LLMs, the publishing industry is still unclear on how to deal with this threat to originality, or how to detect ChatGPT-derived text. In the meantime, it is likely that a body of unscrupulous authors who do not respect publishing ethics, or who may be willing to go to any length to cheat as a way to advance their careers, might employ ChatGPT opaquely to write or edit text in their articles, not being able to complete this task effectively on their own (Kendall & Teixeira da Silva, 2024). Absent the ability to effectively detect such text, it may be premature to claim that the use of AI/LLMs will liberalize academia and diversify participation. Consequently, its use in diversity, equity and inclusion policies (Abdelhalim, Anazodo, Gali, & Robson, 2024) is cautioned.

Since scientific writing is a fundamental aspect of academic publishing, the latter is experiencing a unique crisis of trust because of AI/LLMs because the globalist ethical leadership (ICMJE, COPE, editors, etc.) is acting reactively rather than preemptively. Thus, by the time a solution is found, LLM-derived text may have already populated and tainted hundreds or even thousands of articles, despite having gone through peer review and editorial screening, because peer reviewers and editors were not provided any tools by publishers to detect LLM-derived text. When efficient detection methods are eventually found, authors who violated established codes of conduct (e.g., in journals claiming to follow the ethical guiding principles of COPE and the ICMJE) will need to be held ethically accountable.

2 Two Primers to Detect the Potential Undeclared Use of ChatGPT

This article highlights an unsustainable detection method to compensate for gaps in the detection of LLM-derived text, specifically the detection of ChatGPT’s textual “crumbs”, an issue that received some media attention (Conroy, 2023; Retraction Watch, 2023). This method is considered minimalistic because it only captures instances where authors may have been careless in removing traces of ChatGPT-derived textual output from their article’s text. In other words, this method does not provide a reliable, replicable, or even long-term solution, merely only one way to detect possible ChatGPT-derived text, and thus serves as a way to identify authors who might not have been sufficiently careful, honest, or transparent about their use of ChatGPT or other AI/LLMs. However, since evidence must be irrefutable to avoid the false accusation of scholars (Gorichanaz, 2023), this places a heavy burden of responsibility on the management agents of the publishing industry (journal editors, copyeditors, and publishers) to devise or use effective detection tools.

Authors who might not be satisfied with ChatGPT’s output and would like it to regenerate another or alternative response might press the “Regenerate response” button (Baumgartner, 2023). However, this may result in different outputs (Beutel, Geerits, & Kielstein, 2023). The existence of this text (i.e., “regenerate response”) serves as the first primer – in the context of this article, primer means possible detection method – to detect ChatGPT-derived text in articles. The second primer, “as an AI language model”, is typically part of ChatGPT’s response when it explains the limitations of what information it can offer. In the case of both primers, authors’ failure to delete such ChatGPT-derived text leaves this residual trace in their articles. However, as noted above, this is not an absolute certainty. Only a confession by authors would serve as absolute confirmation. Using PubPeer as the source of information, Table 1 lists several examples of open-access articles in which the two primers were detected.

The vast majority of the 46 examples in Table 1 are in journals that claim to be peer-reviewed, but 13/46 (28%) of cases were in preprints. To hold authors to equal ethical standards, preprints need to be treated in the same manner as peer-reviewed articles (Teixeira da Silva, 2022).

What is the likelihood that such text was generated by ChatGPT? Since the wording in both primers is very context-specific, it is unlikely that an author writing about a thematically specific topic (e.g., agriculture) would use these primers’ terms or expressions. The second clue that suggests that such text may represent vestigial ChatGPT-derived text is their random placement in odd locations of the text.

3 Ethical Repercussions and Epistemic Limitations

Editors of journals whose articles carry these primers need to question the authors regarding the use of ChatGPT, in much the same way that any ethical investigation regarding the integrity of a article would take place. Where authors confess their undeclared use of AI/LLMs, confirming that the primers served their purpose in detecting the undeclared use of ChatGPT, is this an ethical breach? At a basic level, there is the issue of a false claim, i.e., claiming originality when in fact, “originality” was derived from the assistance of an LLM. The second ethical argument is that such authors have failed to declare the use of an LLM (ChatGPT in this case); i.e., they have hidden a methodological truth, namely their reliance on an LLM to help generate text. Based on these arguments, journals claiming to follow COPE guidelines or ICMJE recommendations would then need to consider, depending on the level of reliance on ChatGPT, whether a correction or retraction is merited.

Due to the absence of a confession of the undeclared use of AI/LLMs by authors, do journal editors currently have at their disposal robust tools to verify whether text is de facto LLM/ChatGPT-derived? How will editors determine whether the cases in Table 1 were due to the undeclared use of ChatGPT? To avoid false accusations and thus potentially reputation-damaging litigious resolutions, the for-profit publishing industry should be tasked with seeking a reliable tool to detect LLM-derived text.

4 Limitations and Future Research Hypotheses

This article, as well as the primer-based method of detection, has some limitations. Despite the presence of either primer in articles listed in Table 1, absence of the confession by authors of their undeclared use of ChatGPT, resulting in text in their articles that was not written or edited by humans, and absence of irrefutable proof of the volume of text that was derived from ChatGPT, it cannot be stated with absolute certainty that the authors are guilty of unethical behavior or that they are in violation of established codes of conduct. Consequently, until reliable and robust methods are devised that can effectively detect LLM-derived text, such cases may remain in an ethically indefinite unresolved state. The cases in Table 1 relied exclusively on searches using PubPeer, focusing exclusively on open access to allow for public and open verification. However, those cases likely represent only a fraction of a larger pool of articles whose authors may have employed LLMs, so a more extensive analysis would be needed using scientific databases (Scopus, Web of Science, and PubMed) and Google Scholar to appreciate if these two ChatGPT textual primers exist in indexed and grey literature. Additional primers that might represent residual ChatGPT-derived text also need to be investigated.