Partnership on AI Forms Working Group on Chatbots and Suicide Risk

A gap in standards for chatbot responses

According to PAI, millions of people now turn to AI chatbots for information, companionship and support, including in their most vulnerable moments. The organization says that how these systems respond when someone may be at risk of suicide or self-harm is not guided by any shared standard, clinical protocol or regulatory requirement.

PAI announced the working group ahead of World Mental Health Day on October 10. The group will develop public guidance and report progress over time.

"People are bringing their hardest moments to AI systems, and the field needs to define what an evidence-informed response looks like," said Rebecca Finlay, CEO of Partnership on AI. She said PAI will work with clinicians, researchers and crisis counselors, together with the developers building these systems.

First project and wider mandate

The group's first project is guidance on how general-purpose AI chatbots should respond to people who may be at risk of suicide or self-harm. PAI plans to release a draft for public input later this year, followed by final guidance that organizations can formally adopt.

PAI says suicide prevention is a high consensus topic, and that the resulting guidance can serve as a model for less aligned issues in the future. It names emotional reliance on AI and the line between support and sycophancy as examples.

Beyond the guidance, the working group will:

• map the existing landscape of evaluations
• develop shared definitions for measuring what matters for human well-being
• identify key gaps that require closing
• surface open research questions and help connect funding to the efforts best placed to answer them
• establish an ongoing process for measuring progress over time
• help build a durable practice of collaboration between AI and mental health specialists

Who is involved and how it is funded

The group includes AI practitioners, leading mental health and suicide prevention organizations, crisis support services, clinicians, academic researchers, and people with lived experience of mental health challenges. It draws on PAI's community of more than 150 partners, including a growing number of organizations from the mental health field. It will work through PAI's established process of structured deliberation, expert input and public consultation.

The American Foundation for Suicide Prevention is among the participants. Its Chief Medical Officer, Christine Moutier, M.D., said the organization is prioritizing "the need for deeper understanding and support of AI in mental health and suicide prevention education, research and engagement."

The work is supported by a grant from the OpenAI Foundation. PAI describes itself as an independent nonprofit funded by a broad range of philanthropic and institutional supporters, and says its guidance is developed through deliberation, public input and review by partners from civil society, industry, academia and other sectors.

What it means for the field

The initiative is the first major effort under AI, Human Connection, and Well-Being, a core focus area in PAI's 2027, 2031 strategic plan. Finlay said the new strategy puts PAI's weight behind "practical standards for the field to adopt and measure itself against."

For developers of chatbots, the planned outcome is a public, formally adoptable reference for handling conversations involving suicide and self-harm risk, shaped by mental health specialists. The draft is due for public input later this year, so the details of the recommendations are not yet known.

PAI's announcement also lists support options: in the United States, people can call or text 988 to reach the 988 Suicide and Crisis Lifeline, or text HELLO to 741741 to reach Crisis Text Line. Outside the United States, it advises contacting local emergency services or a local crisis line.