Ghostwriter Sally McGraw recently experienced the frustration of having her professional work misidentified as machine-generated. While submitting a writing sample for a book proposal, she received an email stating that multiple artificial intelligence detection tools had flagged her text. Despite having written the content herself over a span of more than 20 years, McGraw felt a surge of anger and confusion at the dismissal of her craft. She described the sensation as feeling heat rise from her chin to her head, a visceral reaction to the unexpected accusation.
Although the client ultimately hired a different writer, citing experience and fit as the primary reasons, McGraw believes the detector results played a role in losing the opportunity to write the full proposal. The client confirmed to NPR that she had screened the work but maintained that the AI flags were not the sole determining factor. McGraw was compensated for her samples but remains concerned about the impact such screenings have on her career prospects.
This incident highlights a broader trend in the publishing industry. As editors face a surge in AI-generated content, many are turning to automated detection tools to manage the increasing volume of submissions. However, these tools are not infallible. They frequently produce false positives, flagging human-written work as AI-generated, leaving writers with little recourse to prove their authorship. McGraw noted that while everyone acknowledges the screeners are not perfect, the process still creates significant stress for professionals.
Neil Clarke, publisher of the science fiction magazine Clarkesworld, has observed a dramatic increase in submissions since large language models became widely available. In 2021, the publication received approximately 1,100 submissions per month. By February 2023, that number had doubled, prompting Clarke to temporarily close the submission portal. He estimates his own workload has increased by about 30 percent, introducing an element of mistrust into the editing process.
Research suggests that different AI detectors can reach conflicting conclusions about the same piece of writing. A study published in the International Journal for Educational Integrity in June compared the accuracy of four popular detectors using 160 academic papers. The study categorized the papers into human-written, AI-written, mixed, and human-rewritten AI text. The results showed that while some tools performed better than others, none were perfectly accurate. Pangram, for instance, was found to be the most consistently accurate in identifying combined or rewritten AI text.
Experts point to specific linguistic patterns that may trigger these false alarms. Chris Callison-Burch, a professor at the University of Pennsylvania, explained that detectors often assess how predictable a passage is to a large language model. Text that resembles training data or follows common structural patterns, such as lists of three items or specific word choices like 'delve,' can be flagged. Callison-Burch noted that these are also common habits among human writers, meaning a single stylistic choice is rarely enough to confirm AI usage.
Janelle Shane, a science blogger, encountered a striking false positive in late 2022 when testing excerpts from her 2019 book. One detector reported a 98.7 percent confidence that the passage was machine-generated, despite her having written it herself. This experience underscores the difficulty writers face in proving their work is original when the tools used to judge it are based on statistical probabilities rather than definitive proof.
The Authors Guild has described the current environment as reminiscent of witch hunts, where writers worry about false accusations and the challenge of defending their reputation. A false positive can jeopardize a publishing contract, while a false negative can erode reader trust if AI-generated work slips through. Clarke advises against turning suspicions into accusations, suggesting that submissions should end in rejection rather than an indictment of the writer's integrity.
For McGraw, the fear of future accusations has already altered her writing style. She now avoids certain words and sentence constructions, and even omits commas she previously used, hoping to create enough inconsistency to prove her humanity. She misses the em dash, a punctuation mark she learned in journalism school, but sees the irony in trying to make her prose look less like the human writing that trained the machines. She remains braced for the next high-stakes assignment where the outcome could make or break her career.
