Companion chatbots: crisis interruption pauses.
What changed between versions
New definition of 'human moderator' as a natural person who is an employee or agent of an operator, responsible for reviewing credible crisis expressions and determining the course of action on behalf of the operator.
Legislative finding (e) was modified to add 'accompanied by human review' to the description of a structured pause as a proportional safety intervention tool, reflecting the new human moderator requirement.
The core response mechanism was restructured. Previously, the chatbot itself detected crisis expressions and initiated a 20-minute pause after escalation. Now, operators must implement a monitoring system, issue a warning on first detection (notifying the user that a second detection within 72 hours will trigger a pause), and only then initiate a crisis interruption pause on the second detection. The pause no longer has a fixed 20-minute duration; instead it continues until a human moderator reviews the chat.
Operators must adopt and make publicly available a policy governing their protocol for identifying and responding to credible crisis expressions, including policies on terminating pauses, suspending or canceling user accounts, and notifying appropriate contacts or authorities.
A crisis interruption pause cannot be terminated until a human moderator has reviewed the credible crisis expression in context and determined the appropriate course of action. The moderator must document the basis for their decision.
New Section 22587.4 provides that violations of the chapter are enforceable pursuant to Section 22605 (California's Unfair Competition Law enforcement mechanism).
Reports to the Office of Suicide Prevention must not contain any personal or identifying information of a user or other individual, adding a privacy safeguard to the annual reporting requirement.
The fiscal committee designation changed from 'no' to 'yes,' indicating the amendment introduces state fiscal impact that was not present in the introduced version.