Can we really tell when a text was written by artificial intelligence and when it wasn’t?
Artificial intelligence is no longer limited to answering questions or generating images. Today, it writes articles, emails, essays, social media posts, and even literary works—on such a scale that readers often struggle to tell whether a human or an algorithm is behind the words. The question, then, is not only how well AI writes, but also whether there is still a way to recognize its signature.
A new analysis by The Economist attempts to answer this question. Although the research was conducted exclusively on English-language texts, its findings are of broader interest, as they reveal how the writing produced by large language models—such as ChatGPT, Claude, Gemini, and Grok—is evolving overall. And the key conclusion is twofold: artificial intelligence continues to leave certain characteristic “traces” in its writing, but with each new update, these become increasingly difficult to detect.
ΣΧΕΤΙΚΑ ΑΡΘΡΑ
Artificial Intelligence, the “Pen” That’s Everywhere
Text generation by artificial intelligence has moved from the experimental stage to everyday life. It appears in the emails we receive, in LinkedIn posts, in professional presentations, on websites, in university papers, and, most likely, even in scientific publications.
According to data cited by The Economist, more than one-third of new websites are now created with the help of large language models (LLMs). At the same time, their use has spread to students, researchers, communications professionals, and content creators.
The penetration of artificial intelligence is now so widespread that it has begun to raise questions even in the field of literature. This year, rumors circulated that an award-winning short story in an international literary contest had been written with the help of AI, because its style was considered unusually mature and “quietly commanding.” The organizers categorically denied the allegations, but the ensuing debate highlighted just how difficult it has become to distinguish human writing from machine-generated writing.
There is no single “style” of artificial intelligence
For quite some time, the prevailing view was that one could easily recognize a text generated by AI based on specific words or phrases. In English, for example, words like “delve” or phrases like “rich tapestry” had taken on an almost mythical status, as they were considered characteristic of the early versions of ChatGPT.
However, according to linguist Karolina Rudnicka of the University of Gdańsk, this approach is flawed.
Just as there is no single human writing style, there is no single style of artificial intelligence. Each model has its own unique characteristics, and the companies themselves are constantly upgrading them, changing the way they compose texts.
In other words, a characteristic that is currently considered the “signature” of AI may disappear in a few months.
The Economist’s Experiment
To determine what differences still exist between human and AI-generated writing, The Economist designed an extensive comparative study.
Its editors asked four leading language models—OpenAI’s ChatGPT, Anthropic’s Claude, Google’s Gemini, and xAI’s Grok—to rewrite articles from the magazine, without access to the internet. Their only guidance came from the short summaries that the journal itself has begun adding on a trial basis to some of its articles.
They then compared the results with the original texts.
The sample consisted of 55,940 sentences and more than one million words.
To ensure they were identifying characteristics of artificial intelligence and not simply The Economist’s distinctive style, the researchers also compared the results with journalistic texts from CNN, the New York Times, and the Washington Post, as well as with excerpts from popular novels published between 1950 and 2022.
The Words Chosen by AI
One of the most interesting findings concerns vocabulary.
Today’s models are using fewer and fewer of the words that were once considered characteristic of them. Instead, they prefer multisyllabic, abstract, and more formal words.
In English, this translates to greater use of terms such as “significant,” “consequences,” and “increasingly,” as well as more specialized concepts such as “interdependence,” “reindustrialization,” “parameter,” or “methodology.”
This observation naturally applies to English, but the general pattern is clear: LLMs tend to prefer more scholarly, abstract, and technical vocabulary than what people typically use.
At the same time, they use abstract nouns more often than simple verbs, creating a style that sounds more formal but not necessarily more natural.
Orwell would recognize this
The Economist links this phenomenon to an observation made decades ago by George Orwell.
In his famous essay on politics and the English language, the British author accused many of his contemporaries of using complicated words and technical jargon to make simple ideas seem more impressive.
This is exactly what most language models seem to do today: they choose more complex phrasing even when a simpler sentence would be clearer and more direct.
The Myth of the Dash
One of the most widespread stereotypes is that the use of the em dash (—) is a sign that a text was written by artificial intelligence. The Economist’s new analysis shows that this is no longer true, at least for the English-language models examined. Of the models in the study, only Claude uses the em dash more frequently than humans, while ChatGPT now shows significantly less use of it.
This means that the presence of this specific dash is not enough to classify a text as an AI-generated product—not even in Greek. The em dash can also be used by humans, either as a typographical choice or through word processing tools. As the study points out, no single element is a reliable indicator that a text was generated by artificial intelligence.
What Gives Away a Text Written by Artificial Intelligence
According to the study, the overall structure of the text is a more useful indicator than individual words.
Language models typically write longer sentences, use fewer commas, parentheses, and other punctuation marks, and avoid very short, abrupt sentences that give a text its rhythm.
Paragraphs thus take on a more uniform form and are read at a steady, predictable pace.
Another characteristic is that the models repeatedly use the same rhetorical devices, such as contrasts of the form “not X but Y,” phrases like “not only… but also…,” as well as the so-called “rule of three”—that is, presenting ideas in groups of three for greater emphasis.
AI detectors are not infallible
The market is already flooded with tools that promise to identify whether a text was written by a human or by artificial intelligence.
One of the best-known companies in the field, Pangram, claims that its system achieves 99.98% accuracy and is already partnering with the Substack platform.
However, the researchers themselves point out that these tools function as “black boxes.” They produce a result without explaining their reasoning, and it is possible that they might incorrectly classify a text written entirely by a human as an AI-generated product.
For this reason, many experts view these specific applications with skepticism.
AI writing is constantly changing
Perhaps the most important conclusion of the study is that there is no fixed formula for identifying a text generated by artificial intelligence.
Large language models are trained on human texts and are constantly improving through feedback from users.
As researcher Tommy Južek from Florida State University points out, AI is gradually adopting the characteristics that people consider “good writing” and is abandoning the habits that make it easily recognizable.
A prime example is ChatGPT itself. Until recently, it used the double dash almost obsessively in its responses. Today, when asked about this, it responds that such punctuation should be used sparingly.
This change shows just how quickly these models are evolving. Artificial intelligence, it seems, continues to leave certain traces “behind,” but these are changing so rapidly that the distinction between human and non-human writing is becoming increasingly difficult.
This, of course, does not mean that the use of artificial intelligence tools is problematic in and of itself. For many professionals, they are now work-assistance tools that can speed up the process of writing, editing, or organizing a text.
At the end of the day, the substantive discussion is not so much about whether AI is used, but rather how it is used, what it does with the data we “provide” to it, and what impact it may have on work, intellectual creation, and the significant energy and environmental footprint that accompanies the development of these systems.
Source: in.gr
