CDT Report: AI Firms' Safeguards Against Non-Consensual Intimate Imagery Fall Short

What the report examines

The CDT report looks at how foundation models, the large statistical models behind most generative AI applications, can be used to create non-consensual intimate images (NCII). It uses the term AIG-NCII for AI-generated cases. According to the report, such content can be produced by prompting a company's online model with a person's name or a clothed photo, by asking a model for advice on finding "undressing" or "nudification" services, or by getting a model to write code that can be used to create the imagery.

The report also describes a route through third-party developers, who build specialized systems on top of existing models. For closed models, the original developer generally can oversee how the model is adapted. Open-weight models are different: according to CDT, they allow deeper technical changes, including removing interventions meant to restrict harmful outputs, often at little cost and without the original developer's oversight.

The authors base their analysis on public documentation, academic literature, interviews with foundation model companies under the Chatham House Rule, and interviews with civil society groups working on human rights, sexual violence, sex education and sexual content.

Key findings on policies and technical measures

CDT says many companies focus their policies and technical measures on sexual content in general, rather than the narrower problem of using someone's likeness in sexual content without consent. Policies also categorize AIG-NCII differently: as sexual violence, as sexual content, as unauthorized use of likeness, or a mix. According to the report, existing policies often conflate AIG-NCII with lawful sexual content.

That imprecision is made worse, the report argues, by technical interventions that rely on inherently unpredictable AI-based systems to interpret and enforce the policies. CDT says this can mean both too little mitigation of AIG-NCII and infringement on lawful expression.

The report also stresses that foundation models can serve non-abusive purposes. Examples it gives include a person generating images of themselves in lingerie they are considering buying, changing an identifying feature such as a tattoo for privacy, or creating content with a long-distance romantic partner. It adds that NCII can even arise unintentionally, for instance when a prompt for a generic unclothed person produces an image that could be mistaken for a public figure.

Gaps in testing and coverage

CDT finds that technical interventions are mainly applied at the product level, not the model level. As a result, products built on top of a foundation model may lack the protections found in the model provider's own products. The report calls this a particular concern for open-weight models, which it says perpetrators with little-to-no technical skill can easily adapt for AIG-NCII.

Companies report using several technical interventions, but according to CDT they fail to rigorously test how well these work against AIG-NCII. The report adds that academic research documenting fundamental flaws in such interventions, together with civil society reports of rising AIG-NCII incidents, raises significant questions about their effectiveness.

What CDT calls for

The report calls for multi-stakeholder engagement to define a testing standard. Such a standard would measure how far an intervention, a foundation model or a downstream product has mitigated AIG-NCII from a technical standpoint, and set what test results would qualify an intervention as effective or a product as sufficiently "safe."

CDT also says the issue goes beyond engineering. It describes AIG-NCII as influenced by financial incentives, platform design, cultural norms, legal regimes and educational gaps. Technical interventions with tested efficacy are one important way to respond, the report says, but action that shapes the broader financial, social and legal structures that incentivize sexual abuse is also needed.

The authors note that some companies are mentioned more often than others because they publish more public information, not necessarily because of their practices.