Could We Become So Good at Detecting AI That We Start Finding It Before It Existed?

Something interesting happened today.

I had been reading about new requirements intended to make AI-generated material easier to identify. At first, that sounded reassuring.

Transparency matters. Provenance matters. If an image or piece of writing has been generated by artificial intelligence, surely having a way to establish where it came from is a good thing. Then I encountered the other side of the story.

Examples are circulating of AI detectors identifying supposed AI characteristics in writing created long before generative AI existed. Even classic literature is being put through detectors and producing surprising results.

That raises a different question.

What exactly are we detecting?

A provenance marker deliberately embedded into AI-generated content is evidence about its origin. A detector examining finished prose is making an inference. Those aren’t the same thing. And the distinction becomes particularly important for writers.

Authors have used editing tools for years. Spellcheckers. Grammar checkers. Editorial software. Grammarly. Increasingly, some of those tools themselves contain generative AI.

A human might write every original thought, every argument and every paragraph - then allow software to polish the grammar or restructure a sentence.

At what point does human writing become AI writing? And who gets to decide?

The problem becomes even stranger when polished human prose can resemble the statistical patterns a detector associates with artificial intelligence. If a detector can look at something written before ChatGPT existed and decide that it appears AI-generated, the score cannot logically establish authorship.

It establishes resemblance.  That is a very different claim. Perhaps this is where provenance becomes more valuable than detection. And this is no longer merely a theoretical problem.

In publishing, questions about AI involvement are already beginning to affect real careers and commercial decisions. A recent multimillion-dollar book deal reportedly collapsed after concerns were raised about possible AI use, despite the manuscript previously attracting significant interest from publishers. The case itself remains disputed, which is precisely the point.

Once suspicion of AI becomes consequential, establishing what actually happened matters far more than whether a piece of prose simply looks like AI. And if historical human writing can also trigger AI detectors, we should be particularly careful about turning statistical suspicion into evidence of authorship.

Instead of asking a machine to examine the finished artefact and guess:

Did AI make this? 

Perhaps the better question is: Can we show how this was made?

That means drafts. Version histories. Source material. Human decisions. AI contributions. Editorial intervention. And, increasingly, machine-readable provenance attached at creation.

For my own work, I have deliberately used the words co-authored with AI where that accurately describes the process. Not because every sentence belongs equally to human and machine. But because pretending AI wasn’t in the room would be less truthful than acknowledging that it was.

The irony is rather lovely. For years, I have talked about keeping the receipts. Now technology is beginning to do the same. But perhaps we need to be very careful not to confuse a receipt with a guess.

And somewhere beyond the writer and publisher sit the lawmakers, lawyers and insurers, who will eventually need something rather more concrete than “the detector thought so.”

TFP: Could We Become So Good at Detecting AI That We Start Finding It Before It Existed?

TRUE:

AI provenance technologies can provide information about how digital content originated.

FALSE:

An AI-detector percentage is proof that AI wrote something. Even detector providers acknowledge false positives and limitations. 

PIN IT:

As human writing, AI assistance and automated editing increasingly overlap, our definition of authorship may have to become more sophisticated than simply asking whether AI touched the words.

Reader reflection

If you wrote the idea, shaped the argument, challenged the suggestions and made every final editorial decision - but AI helped you find the words, who wrote it?

And perhaps the even better TFP question: Are we trying to detect AI or establish provenance?

Not to preach – Just enough food for thought…