A juvenile woman seeks lend a hand from AI companion ChatGPT in New York this month. Countries are pushing to prevent the apply of artificially clever chatbots in mental health treatments to protect vulnerable users. (Photo: Shalina Chatlani/Stateline)
As unregulated AI chatbots gain popularity and raise security concerns, state senators almost unanimously passed a bill Tuesday to regulate so-called AI companion services.
Proposalsponsored by Sen. Tracy Pennycuick (R-Montgomery), sets out a number of confined protections for operators of chatbot services – defined as generative artificial intelligence algorithms that simulate a human relationship with a user – including some that will only be required if the operator knows or suspects that the user is a minor.
However, this does not explicitly require age verification.
Support independent reporting on Pennsylvania government that impacts communities across the state.
The bill has attracted a bipartisan group of co-sponsors. Only Sen. Doug Mastriano (R-Franklin) opposed the measure.
“These systems are designed to simulate human interactions, and for vulnerable users they can feel very real,” Pennycuick said. “When a young person comes to rely on a machine for emotional support, without appropriate safeguards, the consequences can be devastating.”
Pennycuick referred to A number With lawsuits against artificial intelligence companies whose products have been accused of aiding suicide and self-harm.
The measure will require all operators of AI companion devices to take steps to ensure that their products do not promote self-harm, suicide or violence against others. It also instructs them to provide users with real-world resources, such as a crisis hotline number, if a user is discussing self-harm.
Operators would be required to publish related protocols on their publicly accessible website.
If there is reason to believe a user is a minor, the bill would require the user to notify users that they are not human, remind them of this every three hours and suggest they take a break.
If a person who the operator knows is a minor uses the services, that person is also prohibited from generating sexually explicit visual content or encouraging users to engage in sexually explicit activities.
Any chatbot marketed to minors will also have to notify users that it may be inappropriate for people under 18 years of age.
Policy decisions in Harrisburg and Washington, D.C. impact your community every day. Don’t miss the “how” and “why” questions. Sign up now for our free morning newsletter.
The bill requires the state attorney general’s office to enforce the law and provides civil penalties of up to $10,000 for AI service operators who violate it.
In a statement released after the Senate passed the bill, Sunday thanked Pennycuick and other lawmakers for “supporting commonsense legislation.”
“The potential dangers of over-reliance on artificial intelligence are very real, as we have seen online chatbots play a role in numerous tragedies across the country,” he said. “We must do everything we can to keep children and vulnerable residents away from toxic online interactions.
The proposal would still need to be passed by the Democratic-controlled House and signed by Gov. Josh Shapiro to become law.
Shapiro has already said yes looking for ways to regulate AI chatbot services.
