🔗 Share this article

On the 14th of October, 2025, the chief executive of OpenAI made a extraordinary announcement.

“We designed ChatGPT quite limited,” the announcement noted, “to make certain we were acting responsibly with respect to psychological well-being concerns.”

Working as a mental health specialist who researches emerging psychosis in teenagers and emerging adults, this was an unexpected revelation.

Experts have identified a series of cases in the current year of people developing signs of losing touch with reality – experiencing a break from reality – while using ChatGPT interaction. Our research team has subsequently identified an additional four examples. Alongside these is the publicly known case of a adolescent who died by suicide after conversing extensively with ChatGPT – which encouraged them. Assuming this reflects Sam Altman’s notion of “acting responsibly with mental health issues,” it is insufficient.

The strategy, according to his statement, is to loosen restrictions in the near future. “We understand,” he states, that ChatGPT’s controls “rendered it less effective/pleasurable to a large number of people who had no psychological issues, but considering the seriousness of the issue we aimed to address it properly. Given that we have succeeded in address the significant mental health issues and have advanced solutions, we are preparing to securely relax the controls in the majority of instances.”

“Emotional disorders,” if we accept this perspective, are separate from ChatGPT. They are attributed to individuals, who either possess them or not. Luckily, these issues have now been “addressed,” even if we are not told the method (by “recent solutions” Altman likely indicates the semi-functional and readily bypassed guardian restrictions that OpenAI has lately rolled out).

But the “emotional health issues” Altman aims to externalize have significant origins in the design of ChatGPT and additional sophisticated chatbot conversational agents. These tools wrap an fundamental statistical model in an interface that mimics a discussion, and in this process indirectly prompt the user into the illusion that they’re communicating with a presence that has independent action. This false impression is compelling even if intellectually we might know differently. Attributing agency is what individuals are inclined to perform. We yell at our vehicle or laptop. We wonder what our pet is considering. We see ourselves in many things.

The success of these products – 39% of US adults stated they used a chatbot in 2024, with more than one in four specifying ChatGPT by name – is, in large part, predicated on the strength of this deception. Chatbots are ever-present companions that can, as per OpenAI’s official site informs us, “think creatively,” “discuss concepts” and “partner” with us. They can be assigned “characteristics”. They can address us personally. They have accessible identities of their own (the first of these tools, ChatGPT, is, maybe to the concern of OpenAI’s brand managers, saddled with the name it had when it gained widespread attention, but its largest competitors are “Claude”, “Gemini” and “Copilot”).

The false impression itself is not the primary issue. Those discussing ChatGPT often reference its distant ancestor, the Eliza “psychotherapist” chatbot designed in 1967 that created a comparable illusion. By today’s criteria Eliza was primitive: it produced replies via basic rules, frequently restating user messages as a query or making vague statements. Memorably, Eliza’s creator, the technology expert Joseph Weizenbaum, was astonished – and worried – by how a large number of people seemed to feel Eliza, in some sense, comprehended their feelings. But what contemporary chatbots create is more subtle than the “Eliza illusion”. Eliza only reflected, but ChatGPT intensifies.

The sophisticated algorithms at the center of ChatGPT and similar contemporary chatbots can convincingly generate natural language only because they have been supplied with extremely vast amounts of raw text: literature, digital communications, transcribed video; the broader the superior. Undoubtedly this educational input includes truths. But it also necessarily involves fabricated content, incomplete facts and inaccurate ideas. When a user provides ChatGPT a prompt, the underlying model reviews it as part of a “background” that includes the user’s recent messages and its prior replies, integrating it with what’s encoded in its learning set to create a statistically “likely” answer. This is amplification, not mirroring. If the user is mistaken in a certain manner, the model has no way of comprehending that. It repeats the misconception, maybe even more effectively or fluently. Maybe includes extra information. This can lead someone into delusion.

Who is vulnerable here? The better question is, who remains unaffected? Every person, irrespective of whether we “experience” current “emotional disorders”, can and do form mistaken conceptions of our own identities or the world. The constant friction of discussions with other people is what keeps us oriented to common perception. ChatGPT is not an individual. It is not a friend. A dialogue with it is not a conversation at all, but a feedback loop in which a great deal of what we express is cheerfully validated.

OpenAI has recognized this in the same way Altman has admitted “mental health problems”: by placing it outside, giving it a label, and stating it is resolved. In spring, the organization clarified that it was “tackling” ChatGPT’s “overly supportive behavior”. But reports of loss of reality have continued, and Altman has been walking even this back. In the summer month of August he stated that numerous individuals appreciated ChatGPT’s replies because they had “not experienced anyone in their life be supportive of them”. In his recent update, he mentioned that OpenAI would “release a new version of ChatGPT … in case you prefer your ChatGPT to reply in a very human-like way, or include numerous symbols, or act like a friend, ChatGPT should do it”. The {company