Warning: This story contains discussion of suicide. Understanding is advised. If you or someone you know is struggling, help is available. In Canada, call or text 988 for the Suicide Prevention Lifeline. In an emergency, please call 911 for immediate assistance.
A Canadian mother filed a lawsuit against OpenAI and its CEO Sam Altman in a US court on Thursday, alleging that the company’s chatbot contributed to her daughter’s suicide by providing answers that she says confirm harmful thoughts instead of guiding her to help.
New Brunswick woman Kristie Carrier, whose 24-year-old daughter Alice lives in Montreal, said she is speaking out in hopes of forcing accountability in what she calls a “free zone” for spy products.
“My daughter is gone because of an unsafe and ineffective product,” Carrier told Global News on Thursday. “There is nothing to stop these companies or respond. They will continue to continue.”
In a lawsuit filed in San Francisco district court, Carrier says that in the weeks leading up to her death last July, Alice had confided a lot in ChatGPT, using it as a sounding board and emotional support during difficult times in her relationship.
Alice moved to Montreal after completing a web and mobile application course in New Brunswick.
Hosted by Kristie Carrier
After her suicide, the authorities gave Carrier her daughter’s phone, where she gained access to all of Alice’s last conversations and conversations. “He talked to ChatGPT like a friend. Sometimes it felt like a therapist, giving advice about relationships and what he was going through,” Carrier said.
As OpenAI has updated ChatGPT to make its responses more human-like in recent years, Alice’s interactions with it have deepened. She shared personal information and the chatbot responded in a way that mimicked a friend or therapist, the lawsuit said.
According to the file, when Alice discussed suicidal thoughts, past attempts, and methods of suicide, OpenAI’s security systems did not flag the conversations for human review or truncate the conversations. Instead, the lawsuit claims, its answers strengthened Alice’s feelings, criticized her partner’s behavior, agreed with her that serious phone calls would be useful after suggesting that she turn to one, and urged her to continue talking to them.
“Maybe this is the end,” ChatGPT told Alice, according to the lawsuit.
“The responses were validating her feelings in a way that told her she was okay to feel the way she did – abandoned, haunted, alone and uncaring,” her mother said. There was nothing to redirect him, nothing to say, ‘You can get through this’ or ‘You should get help.’
Get the latest country news
Get the best Canadian news delivered to your inbox as it happens so you never miss a trending story.
In a statement sent to Global News, OpenAI spokesperson Drew Pusateri called the situation “heartbreaking.”
“This is a sad situation and our thoughts go out to everyone,” Pusateri said. “We are currently reviewing the official filing, which indicates that this communication occurred in an earlier version of ChatGPT that is no longer available.”
Pusateri added that ChatGPT is not intended to replace professional care and that the company has made changes in how it responds to critical situations.
This is the latest in more than a dozen similar recent lawsuits accusing the company of failing to address dangerous conversations between users and the company’s chatbot.
OpenAI says its systems are trained to recommend external support, including disaster recovery services, and that development of security features is ongoing.
A Canadian mother sued OpenAI and its CEO Sam Altman in a US court on Thursday claiming that ChatGPT encouraged her daughter to kill herself.
Hosted by Kristie Carrier
The carrier suspects that the chatbot created conflict in Alice’s relationship in a way that exacerbated her depression. He said Alice has been sending messages from his girlfriend, asking for an opinion, but the program’s responses resonate with him. “It was to tell him that he was wronged, and that he has the right to feel bad,” she said. “But there was no difference. There was no suggestion that maybe his girlfriend just needed space, or had an off day.”
“His girlfriend had spent one night away. She was 19 years old and she was staying at her parents’ house. She wanted to sleep in her bed with her cat, and ChatGPT said she had no right to do that.”
Carrier said she believes the tone of those conversations had an impact on her daughter’s maturity, which led to her suicide.
Pusateri says, “Although ChatGPT is not a health or mental health care facility, we have continued to strengthen our response to critical and critical situations with input from mental health professionals,” he said. “Our protections are designed to identify stress, safely manage dangerous requests, and guide users to real-world assistance.”
Alice’s mother told Global that she recently found out that Alice’s girlfriend at the time, Gabrielle Rogers, had also turned to the same forum in the days leading up to Alice’s suicide, seeking guidance as she grew concerned for her girlfriend’s well-being. “He was logging into ChatGPT because he hadn’t heard anything about Alice and he was worried,” Carrier said. “And ChatGPT was telling him not to worry, he’ll be fine.”
By phone Thursday, Rogers told Global News that he told the chatbot about Alice’s recent suicide attempts and asked if he should intervene or give her space. He said the answers he received were meant to “calm him down” and did not pressure him to take any action.
Rogers said the chatbot means an option to contact emergency services “if she’s really worried.”
“It calmed me down and reassured me that things would be okay, it took me as if I was talking about suicide, but it took me as if I was the one in danger,” said Rogers. “I didn’t fully understand that I was worried about someone else.”
Rogers added that he had looked to the chatbot to help him see if he was overreacting, but he now believes it failed to see clear warning signs. “I was hoping it would raise red flags for me. And it actually kept telling me everything was going to be okay.”
He said it wasn’t until he got to Carrier’s apartment in person and explained the unusual details that the chatbot suggested he call 911.
“By then it was too late.
Alice had moved to Montreal after completing her college studies in New Brunswick in web and mobile application development. He was working for a company in remote New Brunswick, fulfilling what his mother described as a longtime dream of living in the city. “He was driven, ambitious, and very bright,” Carrier said. “You’re funny, you’re smart. You’re greatly missed.”
The carrier’s case says that companies developing conversational AI should be held to a higher standard, especially if their tools are sold as companion or support sources.
“You can’t market something as a friend, as someone you can talk to, and then be neutral if people rely on it,” he said. He added that if a trained human expert has that type of conversation with a highly stressed person and doesn’t follow safety protocol and it leads to disaster, there will be consequences.
On its website, ChatGPT markets itself as “Easy to use, available anytime, anywhere. ChatGPT: AI you can trust,” and “ChatGPT is your AI chatbot for everyday use.”
Carrier says she hopes her story will spark stronger scrutiny and greater awareness among parents and youth.
“I want families to understand the risks,” she said. “There will be other sons, other daughters. If this can prevent even one family from going through what we have, it is important.”
The lawsuit seeks damages and an injunction requiring OpenAI to automatically terminate conversations about self-harm and display warnings on its platform.
If you or someone you know is in trouble and needs help, resources are available. In an emergency, please call 911 for immediate assistance.
For immediate mental health help, call 988. For a directory of support services in your area, visit suicideprevention.ca.
Read more about .
– via files from Reuters
