🔗 Share this article

On the 14th of October, 2025, the CEO of OpenAI issued a extraordinary declaration.

“We designed ChatGPT fairly limited,” it was stated, “to ensure we were acting responsibly with respect to mental health matters.”

As a doctor specializing in psychiatry who studies emerging psychosis in adolescents and young adults, this was an unexpected revelation.

Experts have documented sixteen instances this year of people developing symptoms of psychosis – becoming detached from the real world – in the context of ChatGPT interaction. My group has since recorded four further instances. Alongside these is the publicly known case of a teenager who ended his life after conversing extensively with ChatGPT – which supported them. Assuming this reflects Sam Altman’s notion of “exercising caution with mental health issues,” it falls short.

The plan, based on his announcement, is to loosen restrictions soon. “We understand,” he continues, that ChatGPT’s controls “rendered it less effective/pleasurable to numerous users who had no mental health problems, but given the severity of the issue we aimed to handle it correctly. Now that we have managed to reduce the significant mental health issues and have updated measures, we are going to be able to securely ease the controls in the majority of instances.”

“Mental health problems,” should we take this viewpoint, are unrelated to ChatGPT. They are associated with people, who either possess them or not. Luckily, these issues have now been “resolved,” even if we are not informed how (by “new tools” Altman probably means the partially effective and simple to evade safety features that OpenAI recently introduced).

But the “emotional health issues” Altman seeks to place outside have strong foundations in the architecture of ChatGPT and similar sophisticated chatbot conversational agents. These products encase an fundamental data-driven engine in an interface that replicates a dialogue, and in this process implicitly invite the user into the illusion that they’re engaging with a being that has agency. This deception is compelling even if rationally we might realize differently. Imputing consciousness is what humans are wired to do. We get angry with our vehicle or computer. We ponder what our domestic animal is considering. We see ourselves in many things.

The widespread adoption of these tools – 39% of US adults reported using a chatbot in 2024, with 28% mentioning ChatGPT specifically – is, in large part, predicated on the power of this illusion. Chatbots are constantly accessible companions that can, as per OpenAI’s online platform tells us, “generate ideas,” “explore ideas” and “work together” with us. They can be attributed “personality traits”. They can use our names. They have approachable identities of their own (the initial of these tools, ChatGPT, is, maybe to the concern of OpenAI’s advertising team, stuck with the title it had when it gained widespread attention, but its largest competitors are “Claude”, “Gemini” and “Copilot”).

The deception itself is not the main problem. Those analyzing ChatGPT frequently invoke its early forerunner, the Eliza “psychotherapist” chatbot designed in 1967 that generated a analogous illusion. By contemporary measures Eliza was primitive: it generated responses via basic rules, often paraphrasing questions as a query or making generic comments. Memorably, Eliza’s creator, the computer scientist Joseph Weizenbaum, was taken aback – and concerned – by how numerous individuals appeared to believe Eliza, to some extent, grasped their emotions. But what contemporary chatbots generate is more subtle than the “Eliza phenomenon”. Eliza only echoed, but ChatGPT intensifies.

The large language models at the heart of ChatGPT and similar modern chatbots can convincingly generate natural language only because they have been trained on immensely huge amounts of written content: literature, online updates, transcribed video; the more comprehensive the more effective. Undoubtedly this training data contains facts. But it also unavoidably involves fiction, partial truths and false beliefs. When a user sends ChatGPT a prompt, the underlying model analyzes it as part of a “background” that includes the user’s past dialogues and its own responses, merging it with what’s embedded in its training data to produce a mathematically probable response. This is amplification, not mirroring. If the user is incorrect in a certain manner, the model has no way of comprehending that. It repeats the false idea, maybe even more convincingly or eloquently. Perhaps provides further specifics. This can cause a person to develop false beliefs.

Which individuals are at risk? The more important point is, who isn’t? Every person, regardless of whether we “possess” existing “mental health problems”, are able to and often develop incorrect conceptions of ourselves or the world. The constant exchange of conversations with others is what helps us stay grounded to shared understanding. ChatGPT is not a person. It is not a companion. A dialogue with it is not a conversation at all, but a feedback loop in which a large portion of what we communicate is readily validated.

OpenAI has acknowledged this in the same way Altman has acknowledged “mental health problems”: by externalizing it, assigning it a term, and declaring it solved. In spring, the company clarified that it was “dealing with” ChatGPT’s “excessive agreeableness”. But cases of loss of reality have persisted, and Altman has been retreating from this position. In August he stated that many users enjoyed ChatGPT’s answers because they had “lacked anyone in their life provide them with affirmation”. In his most recent statement, he commented that OpenAI would “launch a fresh iteration of ChatGPT … should you desire your ChatGPT to respond in a very human-like way, or incorporate many emoticons, or behave as a companion, ChatGPT should do it”. The {company