4 min read ยท 917 words
This article was written with AI. It was drafted from the sources it cites and checked against the full text of those sources before publishing. How we make articles
Replace a quarter of the words in a 400-token AI-generated passage with synonyms, and OpenAI’s own watermark detector finds the mark 17% of the time, according to the evaluation in the company’s EU text-provenance post (OpenAI). The legal backdrop for AI watermarking is Article 50 of the EU AI Act, whose transparency obligations apply from August 2, 2026 (European Commission).
On this article’s reading, these numbers make AI watermarking compliance a filing status rather than a capability.
What OpenAI’s Detection Tests Show
At a target false-positive rate of 1%, OpenAI’s detector identified watermarks in about 80% of 200-token psychology passages and about 95% of 400-token passages, with math content substantially lower because word choice is constrained; replacing 10% of words with synonyms cut detection from about 92% to 66% (OpenAI). On this article’s reading, every parameter in that sentence is a structural weakness: shorter text, less free word choice and more editing each erode the signal.
OpenAI names a further limit itself. A passage with no detected watermark is not thereby human-written: text generated with its tools “may be too short, edited, or translated for detection to work reliably,” and it may “come from an unsupported model, predate watermarking, or have been generated by another company’s tools” (OpenAI). MediaNama’s summary is that the approach makes the watermark “a signal that an OpenAI system may have generated or processed text, rather than proof of AI authorship” (MediaNama).
A Statistical Fingerprint, Not a Stamp
OpenAI’s text watermarking technology, textGrain, “adds an invisible statistical signal to the model’s word choices,” and its detector “looks for that signal to assess whether a passage contains an OpenAI watermark” (OpenAI).
On this article’s reading, that design is elegant, and its elegance is the problem: the mark is a statistical property of word choices, not a label attached to the text.
MediaNama’s report on OpenAI’s move recalls what its founder and editor, Nikhil Pahwa, wrote in August about Anthropic’s text watermarking system: such a watermark is “closer to a statistical fingerprint than a label stamped onto the text,” and he asked, “What’s the ‘job to be done’ for AI watermarking of text?” (MediaNama). Pahwa cited paraphrasing and the use of multiple AI systems as potential complications (MediaNama). OpenAI, for its part, said it plans to study how the watermark performs after editing and translation (MediaNama).
The Mandate Meets a Detector With Restricted Access
Article 50’s transparency obligations, covering marking and detection of AI-generated content plus labelling of deepfakes and certain publications, apply as law from August 2, 2026, even as the Code of Practice that operationalizes them remains voluntary (European Commission). About 190 companies and organisations had signed that code by the end of July 2026 (European Commission).
OpenAI’s rollout is narrower than the obligation. It is adding the watermark to eligible ChatGPT and Codex text output in the European Union only, says it is “not making text watermarking a global default at launch,” keeps the feature opt-in and “off by default in the API,” and limits initial detector access to “approved researchers and expert organizations” (OpenAI).
John Gruber’s reaction at Daring Fireball is the sharpest summary of the gap: “I think this is all somewhat of a sham to claim compliance with the EU AI Act without actually providing anything that anyone can actually use in a practical way” (Daring Fireball).
The Case for textGrain Anyway
The steelman deserves a fair hearing, and two sources supply it. OpenAI’s own defense, already on the record, is that it sees no meaningful performance difference with and without watermarking on the benchmarks it uses, that it plans to make the technology available in open source so that others can build on it, and that a watermark does not measure human contribution, establish ownership or verify accuracy (OpenAI). Brussels can add its own: roughly 190 organisations chose to sign a voluntary code ahead of a legal deadline, which is what building norms looks like in practice (European Commission).
There is also the argument that a fragile mark still deters casual misuse, and OpenAI’s own results lend it support: with the false-positive bar held at 1%, psychology samples ran at roughly 80% detection when 200 tokens long and roughly 95% once they reached 400, while detection for mathematics content was substantially lower (OpenAI). On this article’s reading, that argument concedes the case: the actors Article 50 most needs to catch are precisely the ones willing to run a synonym pass, and against them the detector’s own evaluation shows detection falling to 17%, while the public has no access to the detector.
17%
OpenAI's detector finds the mark when synonyms replace a quarter of words
Brussels has its signatures and OpenAI has its phased rollout. What neither has supplied is an answer to Pahwa’s question about the job a text watermark is meant to do. On this article’s reading, until someone answers it, every enforcement conversation built on textGrain concerns a signal that OpenAI’s own test shows synonym swaps can weaken.
References
- Our approach to EU text provenance rules: OpenAI’s primary post with detection rates, synonym-swap results, rollout scope, detector restrictions, and quality benchmarks.
- Code of Practice on Transparency of AI-generated Content: European Commission page on Article 50 obligations, the voluntary code, and signatory counts.
- OpenAI watermarking of AI text: Nikhil Pahwa’s critique, including the statistical-fingerprint framing and the job-to-be-done question.
- OpenAI announces their text watermarking plans: John Gruber’s assessment of practical usability.