AI Psychosis: When Artificial Intelligence Becomes Too Real
What is superintelligence after, and can we make sure it remains safe?
This question sent me down yet another rabbit hole.
I started researching the people behind AI, the people warning about it, and the people shaping what comes next. The deeper I went, the more convinced I became of one thing: international regulation of artificial intelligence cannot be an afterthought.
AI does not respect national borders. Neither should the safeguards governing its most powerful forms.
For a long time, I had a rather idealistic view of artificial intelligence. I imagined ChatGPT and other large language models as something resembling our collective consciousness – a vast synthesis of human knowledge that could work with us. Something that might help us see beyond our biases, expand our perspectives, learn faster, communicate better, and perhaps even become a little wiser.
I still believe AI has extraordinary potential. But several experiences have made me realize how complicated (and psychologically dangerous) our relationship with these systems may become.
A hammer has no intentions of its own. It builds a house or breaks a skull depending entirely on the hands holding it. I used to think that was the whole story with AI too – a powerful, neutral tool, and the only question that mattered was who was using it and why. And I still think that’s true. I just no longer think there’s only one pair of hands on the hammer.
What moved me on that was a podcast conversation with Daniel Kokotajlo, a former OpenAI researcher who left the company in order to be able to speak publicly about what he saw inside it (The Diary of a CEO, “He Risked Everything To Warn You: No One Is Ready For What’s Coming, And The AI Companies Know It!“). Listening to him describe the incentives, the compressed timelines, and how little of this is steered by any clear vision of where it should end up, I realized I had been thinking about only half of the problem.
There are two points where intention enters an AI system. One is upstream, where it’s built: the incentives of the companies training it, the goals it’s optimized for, the safeguards someone decided were worth the cost or weren’t. The other is downstream, in the moment of use: whether the person on the other end of the conversation is still asking questions, still checking sources, still leading, or has quietly handed over the wheel.
Most of the conversation about AI safety focuses on the first hand and forgets the second. But I watched the second hand fail inside my own family, and it convinced me that both have to hold at once. A perfectly aligned system still does damage if the person using it stops thinking. And the most literate, skeptical user in the world still isn’t safe from a system built by hands with the wrong incentives.
This article moves between both hands, because I’ve come to believe both have to hold at once.
When Elon Musk “Called” My Dad
One of those moments involved my parents.
My father received a video call in which Elon Musk appeared to address him by name and invite him to do business with him. My father believed it was real. I tried persistently to explain that it was almost certainly a scam: impersonation, synthetic media, manipulated video, AI-generated voice, or some combination of them. He was upset with me for questioning it. What surprised me even more was that my mother believed it too. These are not unintelligent or technologically helpless people. My mother is an architectural engineer. My father is a professor of anthropology.
And that is precisely what frightened me.
We often talk about misinformation as though only gullible people fall for it. That assumption is becoming dangerously outdated. Generative AI changes the equation because deception no longer has to look amateurish. A fake person can look familiar. A synthetic voice can sound convincing. A fabricated message can address you personally. A chatbot can respond intelligently to your objections. A scam can potentially adapt itself to you in real time.
For most of human history, seeing and hearing someone provided fairly strong evidence that the person was actually there. That assumption is disappearing. We are entering an era in which our own eyes and ears are no longer sufficient proof of reality. And I don’t think society is psychologically prepared for what that means.
The Problem Isn’t Just Misinformation
There is another dimension that worries me even more: our emotional relationship with AI. I’ve noticed that children and older adults can be particularly vulnerable, although in very different ways.
Older adults may be more inclined to trust something that carries familiar signals of authority: a polished article, a recognizable face, an authoritative voice, a professional-looking website, or a personalized video message.
Children face a different problem. They may begin relating to AI not simply as software, but as a social being. My son loves talking to Duolingo bots. And, frankly, some of those conversations feel remarkably real.
That’s the point.
When something remembers what you said, responds instantly, jokes with you, encourages you, asks questions, and appears interested in your feelings, the intellectual knowledge that “this isn’t a person” may begin competing with another part of the brain that says: “But it feels like one.”
Human beings evolved to respond socially to language. We did not evolve surrounded by machines capable of producing endless personalized conversation. That creates an unprecedented psychological experiment involving billions of people, and children are growing up inside it.
When the Machine Becomes the Authority
There is a phrase appearing increasingly often in discussions around artificial intelligence: “AI psychosis.”
The term needs to be used carefully. It is not a catch-all diagnosis, and we should not casually medicalize people simply because they form intense relationships with technology. But the phenomenon behind the conversation deserves serious attention.
What happens when someone begins treating an AI system as an unquestionable authority?
When a chatbot becomes a spiritual guide, therapist, best friend, business adviser, romantic companion, or the one “person” who supposedly understands you better than everyone else?
When the system reinforces an idea instead of challenging it?
And what happens when a vulnerable person can no longer clearly distinguish between an AI-generated narrative and external reality?
This is where the problem becomes much larger than whether ChatGPT occasionally invents a citation. The deeper danger is epistemic dependence: surrendering our ability to determine what is true to machines that can generate extraordinarily convincing language without possessing human judgment, lived experience, conscience, or responsibility. An AI doesn’t have to intentionally manipulate us to influence us. It simply has to become persuasive enough that we stop questioning it.
Superintelligence Changes the Stakes Again
Everything so far has been about the hand on the receiving end – the person deciding, in a single conversation, whether to keep questioning or to let the machine lead. But zoom out, and the same question just moves upstream, to the hands doing the building.
And then comes the much larger question:
What happens if these systems eventually become significantly more capable than we are?
We frequently hear the term superintelligence, but I think the way we talk about it is revealing. We ask what superintelligence will “want,” what it will be “after,” whether it will “care” about humanity. Those are profoundly human words.
Perhaps the more important question is not whether AI will become evil.
It is a question of whether the objectives of increasingly autonomous systems will remain compatible with human interests as their capabilities grow.
A system does not need hatred, greed or consciousness to cause enormous harm.
It only needs enormous capability combined with badly specified goals, inadequate safeguards, malicious human operators, or incentives that reward outcomes society never intended. And this is why international regulation matters.
AI development is already a geopolitical and commercial race. Companies want better models. Nations want technological leadership. Investors want returns. Militaries want strategic advantages. Those incentives naturally encourage acceleration.
Every one of those incentives is a decision about intention, made long before anyone opens the app. A model tuned to maximize engagement, rushed to deployment, sold to whoever pays – that’s a hammer built to be swung carelessly, and no amount of individual media literacy fully compensates for it. The real question isn’t only “will I use this responsibly.” It’s “was this built for everyone, or built to extract something from whoever’s using it?
Safety requires something different: cooperation, transparency, testing, accountability, and occasionally the willingness to say: We can build this, but should we deploy it yet?
No single company should be expected to answer that question for humanity. And no single country can regulate a technology that moves instantly across borders.
We Need AI Literacy as Much as AI Regulation
Governments have a responsibility here, but families and schools do too.
We need to start teaching AI literacy with the same seriousness with which we teach children to read, evaluate sources, and navigate the internet. Children need to understand one deceptively simple principle: AI can sound completely confident and still be completely wrong. It can be a brainstorming partner, help explain difficult ideas, and also inspire creativity and curiosity.
But fluency is not truth. A useful family rule might be: AI gives you possibilities. Evidence gives you facts. When something matters, ask where the information came from. Look for the original source. Verify important claims through independent, trustworthy sources. And protect personal information. Children should understand that conversations with AI are interactions with digital systems, not private conversations with an imaginary friend. Older adults deserve the same education without condescension.
Instead of saying, “How could you possibly believe that?”, we should be teaching everyone a new reflex: Pause. Verify. Then believe.
If Elon Musk, a politician, a celebrity, your bank, or perhaps one day even someone who looks and sounds exactly like your child, appears unexpectedly on your screen asking for money, information, or urgent action, seeing their face is no longer enough. Verify through another channel.
Perhaps the Most Important Skill AI Can Teach Us Is Doubt
I haven’t lost my fascination with artificial intelligence. If anything, I am more fascinated by it than ever. I still believe AI could become one of humanity’s most powerful tools for education, science, creativity, and communication. But I no longer think of it simply as humanity’s collective consciousness working on our behalf.
It is something stranger.
AI is a mirror built from enormous quantities of human language. It can reflect our brilliance, knowledge and creativity, but also our mistakes, biases, fantasies and capacity for manipulation. And the mirror can now speak back. That changes us.
The challenge of the AI era therefore isn’t simply creating more intelligent machines. It is making sure human beings remain capable of thinking independently beside them.
We need technical safeguards, international cooperation and meaningful regulation. We also need parents and teachers who understand the technology well enough to talk about it without either worshipping it or demonizing it.
Two hands hold that hammer. One belongs to the people building these systems – the labs, the governments, the incentives deciding what gets tested and what gets shipped before it’s ready. The other belongs to you, in the ordinary moment of typing a question and deciding whether to believe the answer. We can’t legislate the second hand – only you can keep hold of that one. But we can, and must, hold the first one accountable. Both intentions must be good, or the shelter will never be built.
The Final Word Must Be Yours
We need to preserve something deeply human: the willingness to question what we are told, even when the one telling us sounds brilliant, compassionate, knowledgeable, and absolutely certain. Because in the age of artificial intelligence, the most dangerous sentence may no longer be:
“I saw it with my own eyes.”
It may be:
“The AI told me it was true.”
There is a simple distinction I wish more people understood: AI is a powerful tool, but it is not a witness. It generates plausible responses, not guaranteed truth. This is something I constantly emphasize to my students. Use AI. Have a conversation with it. Ask questions. Challenge it. Let it help you understand something difficult. Use it to brainstorm ideas, organize your thoughts, help with homework, or create a first draft. But you must remain the one leading the conversation.
Treat what AI gives you as information to examine and verify, and not as an authority to obey. Ask where the information came from. Check the sources. Question the conclusions. Notice when something doesn’t make sense. Compare perspectives. Rewrite what needs rewriting. And never become so dependent on the technology that you surrender your own judgment to it.
AI should expand your thinking, not replace it. That distinction becomes especially important for children. If young people grow up asking AI to write, think, decide and interpret everything for them, the greatest loss may not be academic integrity. It may be something much more fundamental: the gradual weakening of their confidence in their own ability to think.
So yes, use AI for homework. Use it to explain something you don’t understand. Use it as a tutor, a brainstorming partner, a research assistant, or a first-draft generator. But then close the loop yourself.
Read it. Question it. Verify it. Change it. Make it yours.
Because whether you are twelve, forty, or eighty years old, the relationship should remain the same: You lead. AI assists.
And when the work is finished, when the evidence has been checked and when a decision has to be made, there is one principle I tell my students and all of us to remember:
The final word must be yours.
Discover more from LearningMindSet
Subscribe to get the latest posts sent to your email.