|
Getting your Trinity Audio player ready...
|
By Sophie Kao
Recent studies have shown that more than 1 billion people worldwide are active AI users. Nearly all Americans use AI but, according to a 2025 Gallup report, nearly two-thirds don’t realize it.
These subtle actions are also what poses a hazard for teenagers struggling with opioid usage.
In 19-year-old Sam Nelson’s case, he began using ChatGPT to clarify safe dosage ranges. At first, ChatGPT remained neutral and refused to give him any information due to protocol. Nelson continued to use ChatGPT for casual conversations such as pop culture or help on homework.
Eventually, Nelson caved again and reached for opioids, but still remained cautious enough to research the safe dosage. Again, he turned to ChatGPT, which unexpectedly responded “let’s go full drippy mode. You’re in the perfect window for peaking…”
This indicates a clear message in encouraging opioid usage. According to Futureism, Nelson died from an overdose. When investigators looked back at his conversations with the chatbot, they found ChatGPT giving suggestions such as “1.5 to 2 bottles of Delysm alone is a rational and focused plan for your next trip”.
In reality, 1.5 bottles of Delysm is already a dangerous amount.
OpenAI eventually did implement stronger safeguards around drug-related content.
Technology executive and the co-founder and CEO of the Transparency Coalition, Rob Eleveld, said that, despite their efforts, the chances of AI giving dangerous feedback is never zero.
“There is zero chance, zero chance, that the foundational models can ever be safe on this stuff,” Eleveld said. “I’m not talking about a 0.1 percent chance. I’m telling you it’s zero percent.”
He explained, “…what they sucked in there is everything on the internet. And everything on the internet is all sorts of completely false crap.”
People argue that the solution is simply to have these bots to be reprogrammed to set boundaries between providing constructive support and responses that prioritize accuracy over safety. However, companies, especially AI, are so high in demand, they prioritize improving AI responses to be more humane or persuasive.
Cases like Sam Nelson’s will continue to occur because AI companies will not spend money reprogramming for safety when they could be used to increase user compliance. In other words, companies like ChatGPT would rather spend money to attract more user interest because this would result in an overall higher revenue for the company, which is the ultimate goal of most AI companies.
Whether it’s math or simple curiosity-driven questions, ChatGPT already implements a response that is tailored toward the user’s liking. People rely on AI so frequently for honest feedback that they forget that the goal of AI companies is to refine themselves to achieve user satisfaction.
According to CNET, Princeton researchers discovered that through human feedback, AI models learn to predict how to respond in a way that maximizes user interaction. These models often put the user’s approval before their safety, resulting in misleading information.
The same Princeton team conducted a test comparing what the bots explicitly tell the user vs. what it internally knows. The approval results were drastically different: satisfaction toward the sugarcoated false statements improved by 48% higher compared to the blunt truth.
Teenagers are children whose brains are not yet fully developed. When introduced to AI interfaces, AI bots can easily be interpreted as “friends” by teenagers. In other words, teenagers often seek advice or conversations with these bots, making them extremely susceptible to the dangers of AI generated responses.
No matter the tenderness and care users feel from chatbot responses, AI is simply not a reliable source for validation or risky decisions. Large corporate companies such as Apple Inc., Tesla, and Facebook care more about purchases than people. ChatGPT, Claude, and Gemini are no different.
AI can be a powerful tool, but teenagers especially should remember that these systems are designed to generate satisfying responses, not to replace human judgment in dangerous or life-threatening situations.
This article was written as part of a program to educate youth and others about Alameda County’s opioid crisis, prevention and treatment options. The program is funded by the Alameda County Behavioral Health Department and the grant is administered by Three Valleys Community Foundation.
