Newsom signs bill regulating AI-powered chatbots
California Gov. Gavin Newsom (D) signed a bill Monday that puts new guardrails around how artificially intelligent chatbots interact with children and deal with issues of suicide and self-harm.
SB 243, approved by the state Legislature in mid-September, would require developers of “companion chatbots” to create protocols that prevent their models from producing content about suicidal ideation, suicide or self-harm and direct users to crisis services if necessary.
It also requires chatbots to issue “clear and conspicuous” notices that they have been artificially created if someone could be misled into thinking they are interacting with another human.
When interacting with children, chatbots must issue reminders every three hours that they are not human. Developers are also required to create systems that prevent their chatbots from producing sexually explicit content in conversations with minors.
“Emerging technology like chatbots and social media can inspire, educate and connect — but without real guardrails, technology can also exploit, mislead and endanger our children,” Newsom said in a statement.
He added: “We have seen some truly horrific and tragic examples of young people harmed by unregulated technology, and we will not stand idly by while companies continue without the necessary boundaries and accountability.” “We can continue to lead in AI and technology, but we must do so responsibly – protecting our children every step of the way. Our children’s safety is not for sale.”
The family of a California teen filed a lawsuit against OpenAI in late August, alleging that ChatGPT encouraged their 16-year-old son to commit suicide. The father, Matthew Ren, testified before a Senate committee last month, along with other parents who accused chatbots of leading their children to suicide or self-harm.
OpenAI on Monday praised the signing of the chatbot bill as “an important step forward when it comes to AI safety standards.”
“By setting clear guardrails, California is helping shape a more responsible approach to developing and deploying AI across the country,” a company spokesperson said in a statement.
Growing concerns about how AI-powered chatbots interact with children prompted the Federal Trade Commission (FTC) to launch an investigation into the issue, requesting information from several leading technology companies.
Sens. Josh Hawley (R-Mo.) and Dick Durbin (D-Ill.) also introduced legislation late last month that would classify AI-powered chatbots as products in order to allow affected users to file liability claims.
California’s measure is the latest of several AI and technology-related bills Newsom has signed into law this session. On Monday, he also approved measures requiring warning labels on social media platforms and age verification through operating systems and app stores.
In late September, he also signed SB 53, which requires sophisticated artificial intelligence models to deploy frameworks detailing how to assess and mitigate catastrophic risks.
Updated at 6:50 PM EST
