Companion chatbots.
What changed between versions
The definition of 'companion chatbot' now excludes three additional categories: (A) bots used for customer service, business operations, productivity/analysis, internal research, or technical assistance; (B) video game features limited to in-game replies that cannot discuss mental health, self-harm, sexually explicit conduct, or unrelated topics; and (C) stand-alone consumer electronic devices acting as voice-activated virtual assistants that do not sustain multi-interaction relationships or generate emotionally eliciting outputs. Previously only customer-service-only bots were excluded.
Two new definitions added: 'sexually explicit conduct' (incorporating the federal definition from 18 U.S.C. Section 2256) and 'video game' (defined as a game on an electronic amusement device with its own monitor or designed for use with a TV/computer monitor that interacts with the user).
The notification requirement that the chatbot is not human now applies only if 'a reasonable person interacting with a companion chatbot would be misled to believe that the person is interacting with a human.' This adds a materiality threshold that was not present in the prior version, which required notification unconditionally at the start of every interaction and every three hours.
The suicide protocol obligation shifts from requiring operators to have a protocol for 'addressing suicidal ideation, suicide, or self-harm expressed by a user' to requiring a protocol for 'preventing the production of suicidal ideation, suicide, or self-harm content to the user.' This reframes the duty from responding to what users say toward preventing the AI from generating harmful content.
New Section 22602(c) imposes three specific duties for users the operator knows are minors: (1) disclose that the user is interacting with artificial intelligence; (2) provide a default notification at least every three hours reminding the user to take a break and that the chatbot is not human; and (3) institute reasonable measures to prevent the chatbot from producing visual material of sexually explicit conduct or directly stating the minor should engage in such conduct.
The content of the annual report changed substantially. The old report required two data points: number of times suicidal ideation was detected and number of times the chatbot brought up suicidal ideation. The new report requires: (1) number of crisis service provider referral notifications issued in the preceding calendar year, (2) protocols put in place to detect, remove, and respond to instances of suicidal ideation, and (3) protocols put in place to prohibit a companion chatbot response about suicidal ideation or actions.
New requirement that operators must use 'evidence-based methods for measuring suicidal ideation' when making their annual reports.
The annual reporting requirement to the Office of Suicide Prevention now has an explicit start date of July 1, 2027. The prior version simply required annual reporting with no specified commencement date.