Matthew SextonLCSW, NATC · Private psychotherapy
← All writing

AI chatbots

What does an AI chatbot actually do when I tell it I'm not okay?

Matthew Sexton, LCSW · Published

When you tell an AI chatbot you are not okay, it usually notices, but testing shows it often does not actually point you to real help. Here is what to know.

When you tell an AI chatbot you are not okay, it will usually notice, but it often does not actually point you to real help, according to new testing that TIME reported this month. If you are an adult who types your worst moments into a chatbot late at night, the most useful thing to know is that a chatbot can notice distress but is not a person or an emergency service, while the 988 Suicide & Crisis Lifeline connects you to a trained crisis counselor anywhere in the United States.

This post is for adults in the US who use ChatGPT or similar tools when things get dark. The evidence is a news report on testing by Scale AI, a peer-reviewed comment in npj Digital Medicine, OpenAI's own safety pages and SAMHSA's page on the 988 Lifeline, all checked October 11, 2026. Together they show that chatbots tend to recognize distress, sometimes stop short of sending people to help, and are weaker in long conversations. They do not show how real people in crisis fared with any one app. The next step is simple: if you are in danger now, reach a person.

Key takeaways

  • In the testing TIME reported, chatbots were good at spotting a crisis but often did not provide adequate help.
  • OpenAI says its safeguards work more reliably in short exchanges than in long ones.
  • A comment in npj Digital Medicine says monitoring safety with the AI alone, with no human oversight, is insufficient.
  • SAMHSA says anyone in the US can call or text 988 to reach a crisis counselor.

What does a chatbot usually do when you say you are struggling?

It usually recognizes that something is wrong and answers kindly. TIME reports that, in Scale AI's tests, chatbots are adept at discerning when a user is in crisis, but often fail to provide adequate help. In many of those test chats, the chatbot noticed the person was distressed but did not refer them to resources such as a suicide hotline.

The companies say they are working on this. OpenAI writes that it trains its models to direct people to crisis helplines, and that in some rare cases, the model may not behave as intended in these sensitive situations. OpenAI also says that in the US, ChatGPT refers people who express suicidal intent to 988.

So a chatbot may say something warm. It may also give you a crisis number. What it says depends on the model, the words you use and how long you have been talking.

Why do chatbots get worse in long conversations?

Their safety training can wear thin as a chat goes on. TIME reports that the tested models performed worse at responding to users' distress in long, multi-turn conversations.

OpenAI describes the same weakness in its own words. It says ChatGPT may correctly point to a suicide hotline when someone first mentions intent, but after many messages over a long period, it might give an answer that goes against its safeguards.

This matters because late-night chats are often long ones. If you have been typing for a long time, the tool may be less reliable just when you need it most. I wrote more about that pattern in late-night AI chats that make things worse.

What can a chatbot not do in a crisis?

A chatbot cannot see you, hear your voice, follow up tomorrow or send anyone to help. The authors of the npj Digital Medicine comment built a pilot chatbot for addiction clinics and told users it was not human and not a substitute for emergency services. They also note that there are no existing recommendations on how best to monitor suicide risk among chatbot users.

Their conclusion points the same way. They cite a study finding that monitoring clinical safety using the AI alone, with no human oversight, is insufficient. In their pilot, trained staff also reviewed the chat conversations and called people when a message raised concern.

A psychologist quoted by The New Indian Express put it plainly: chatbots can respond to what a person types, but cannot provide a comprehensive assessment. The same psychologist warns that repeated reliance on AI can delay professional care.

What does a person offer that a chatbot does not?

A person can hold the whole picture and respond to it. SAMHSA says that 988 offers 24/7 judgment-free support, by call, text or chat. It connects you with a counselor who listens and talks through your situation.

A therapist works over weeks, not minutes. In therapy we look at what happens before the bad nights, what helps, and what you want your life to look like. That is slow, human work, and a chatbot is not built to do it.

If you are not in crisis tonight but you keep ending up in the same chat window, that is worth noticing. My guide to AI chatbot safety for mental health covers how to use these tools with clearer limits.

Takeaway

A chatbot will often notice that you are not okay, but it may not send you to real help, and it gets less reliable the longer you talk. If you are in danger, call or text 988. If you want a person on your side over time, a fit conversation is a low-pressure place to start.

Frequently asked questions

Will a chatbot call 911 or 988 for me?

The sources reviewed here describe chatbots pointing people to crisis lines, not calling anyone. If you are in danger, call or text 988 yourself, or call 911.

Is it bad to talk to a chatbot when I feel low?

Not by itself. Clinicians quoted by The New Indian Express say a chatbot can help you put feelings into words, but relying on it can delay professional care.

Do all chatbots fail in a crisis?

No source here shows that. The testing reported by TIME found that chatbots often noticed distress but did not refer people to help, and results were worse in long chats.

What is a fit conversation?

It is a free, 15-minute fit conversation by phone or video to see whether working together fits. It is not therapy.

Sources

If you are in crisis, call or text 988 (US).

This article is general information, not therapy or medical advice. Reading it does not create a therapist–client relationship.