OpenAI has developed a tool capable of identifying students who use ChatGPT to complete their assignments. However, The Wall Street Journal reports that the company is currently debating whether to release it. In a statement to TechCrunch, an OpenAI spokesperson confirmed the research into the text watermarking method mentioned in the Journal’s article. They emphasized that the company is adopting a “deliberate approach” due to the “complexities involved and its likely impact on the broader ecosystem beyond OpenAI.”
The spokesperson elaborated, explaining, “While the text watermarking method we’re developing shows technical promise, we are carefully considering significant risks, such as the ease with which bad actors might bypass it and the potential negative impact on non-English speakers, as we explore other options.”
This approach differs from previous attempts to detect AI-generated text, which have generally been ineffective. OpenAI itself discontinued its previous AI text detector last year because of its “low rate of accuracy.”
The text watermarking method focuses exclusively on detecting writing from ChatGPT, not from other AI models. It works by making subtle changes to how ChatGPT selects words, essentially embedding an invisible watermark in the text that can be detected by a specialized tool.
Following the Journal’s article, OpenAI updated a May blog post regarding its research on detecting AI-generated content. The update states that text watermarking has shown “high accuracy and effectiveness against localized tampering, such as paraphrasing,” but it is “less robust against globalized tampering,” such as using translation systems, rewording with another generative model, or inserting special characters between words and then removing them.
Consequently, OpenAI acknowledges that this method is “trivial to circumvent by bad actors.” The update also reiterates concerns about non-English speakers, stating that text watermarking could “stigmatize the use of AI as a valuable writing tool for non-native English speakers.”
In summary, while OpenAI’s text watermarking technology shows promise in accurately detecting AI-generated content from ChatGPT, the company is carefully considering the broader implications and potential risks before deciding on its release. The goal is to balance technological advancement with fairness and accessibility for all users.
