CDT urges Oversight Board: LLMs should assist content moderators, not replace them

Why CDT sees a need for human oversight

According to CDT, the Board's forthcoming guidance comes at a good time. As companies begin to adopt these systems, the guidance could help them make more accurate moderation decisions at scale, make decisions more explainable, and even triage user reports, while mitigating risks.

CDT points to several incidents to argue that automated tools need human oversight. Earlier this month, Meta cited "automated protections" for the temporary removal of the No Kings Instagram account, which is run by the coalition behind U.S.-wide pro-democracy protests. CDT says this incident might not have involved an LLM, but it shows the need for safeguards against machine-driven mistakes, as the Oversight Board noted after the incident.

The organization also cites YouTube, which moved the bulk of its content moderation to automated processes during the COVID quarantine period. According to CDT, the service then saw a spike in appeals, and 50% of content appeals following the change resulted in a reinstatement of content. On TikTok, a former moderator warned that AI-powered processes could cause a great deal of harm to users without adequate human review and oversight.

Risks for marginalized speakers and algorithmic monoculture

CDT's submission identifies specific shortcomings of LLM-based moderation for user speech. It highlights users outside the United States, speech by political and religious minorities, and other marginalized speakers as particularly affected.

The group also raises the risk of what is often called algorithmic monoculture. If the same or similar models moderate content across multiple user-generated content services, the same individuals and members of the same marginalized groups could face adverse outcomes on several services at once.

According to CDT, these limitations are most serious in high-risk cases, such as content that calls for violence against elected officials or depicts real-world conflicts. For that reason, the organization says human review remains essential even as moderation processes incorporate LLMs. It argues that LLM-based systems can help if they are given to trust and safety practitioners and even contract content moderators to help triage content, not replace them entirely.

CDT's recommendations for trust and safety teams

Building on the work of trust and safety practitioners, CDT recommends that trust and safety teams take the following steps:

• Use LLM-based tools to enhance, not replace, existing trust and safety processes.
• Conduct human rights impact assessments on a chosen tool before implementation, to understand whether it poses novel risks to human rights or amplifies existing ones, and how to mitigate them.
• Require third-party providers of LLM-based moderation tools to disclose key performance indicators such as precision, recall and false positive rates across every language the tool is said to work in.
• Develop and apply clear methods to assess tool performance before deployment, including efficacy and drift over time, with indicators on precision and recall across content types, risk categories and languages.
• Explain any adverse moderation decision made by an LLM-based tool to the user, use explainability tools while noting their limitations, and allow the user to appeal.
• Disclose which languages the tool was fine-tuned and tested in, and using which benchmarks, if at all.
• Include LLM-based moderation in transparency reports, for example the number of decisions made by an LLM, how many were appealed and how many appeals succeeded.

What it means for platforms and users

The comments are an input to the Oversight Board's work, not a binding rule. If the Board's guidance reflects these points, companies using LLMs for moderation could face expectations to test tools by language, publish performance data and give users explanations and appeal routes. For users, particularly those writing in languages other than English or belonging to minority groups, CDT's position is that human review and transparency are the main protection against automated errors.