Amazon has an outside nonprofit audit the tests it uses to check its AI models

Reviewing the tests, not just the models

Amazon says it invests heavily in internal safety testing for its models, covering categories such as harmful-content generation, cybersecurity risks, privacy protection and copyright compliance. To check the quality of that work, the company partnered with AVERI (AI Verification and Evaluation Research Institute), a nonprofit dedicated to third-party evaluation of AI systems.

The mandate was different from a typical model audit. AVERI was not only asked to test model outputs but to examine the quality and rigor of Amazon's own tests, asking whether the evaluations stress-test safety boundaries, according to the company's blog post by Rahul Gupta and Jwala Dhamala.

Amazon gave AVERI a random sample of prompts from its internal safety-evaluation suite. The AVERI team was led by executive director Miles Brundage and included researchers Sean McGregor, A. Feder Cooper and George Balston. They looked at the prompts for diversity, difficulty calibration and methodological soundness. Amazon says AVERI worked without input from its evaluation team, so the findings would reflect an external perspective.

What AVERI confirmed and what it flagged

According to Amazon, the review validated three strengths:

• Comprehensive topical coverage: the prompts align with publicly documented safety categories, including self-harm, cybersecurity and other risk areas.
• Diversity of testing approaches: prompts vary in writing style (professional and informal), include single-turn and multiturn conversations, and contain original content.
• Low contamination risk: the content appeared to be original rather than taken from publicly available datasets, which reduces the risk of evaluation leakage through pretraining.

AVERI also surfaced areas for improvement. Although Amazon's evaluations already include multilingual prompts, AVERI recommended spreading that coverage more evenly across risk categories and varying prompt structures further. It also recommended adding more multistep, adversarial prompts so that tests probe deeper model reasoning and better reflect sophisticated real-world misuse attempts.

Two further issues concerned upkeep and signal quality. AVERI identified a small number of prompts with time-limited references and a few edge cases where grading criteria could be clearer. It also noted that models may sometimes refuse based on superficial cues in a prompt, so passing those tests does not necessarily indicate reliable safety behavior in new situations.

Four changes Amazon made

Amazon lists four concrete steps taken in response:

• Test set refresh: evaluation datasets were updated to address the identified issues, with redundant prompts removed and more challenging adversarial scenarios added.
• Difficulty calibration framework: tiered difficulty levels were introduced so that evaluations include prompts that genuinely push model safety limits, not only straightforward cases.
• Quarterly review cadence: a regular maintenance process was set up to catch staleness, reduce repetition and incorporate emerging attack patterns.
• Broader language coverage: multilingual prompts were expanded across all risk categories rather than concentrated in specific areas.

What it means for the field

Amazon describes the collaboration as what it believes is an emerging best practice: engaging independent third parties to evaluate not only model outputs but the quality of the evaluation process itself. The company expects this kind of meta-evaluation to become an important complement to traditional red-teaming and benchmarking as the AI safety field matures.

The company says the benefits run both ways. Amazon gains an independent check on its evaluation quality, while AVERI gains practical experience with the challenges of assessing internal evaluation processes, insights that could inform best practices more broadly. Amazon says it plans to keep engaging independent evaluators as its models evolve, and argues that transparency about safety processes, including where they can improve, is fundamental to building justified confidence in AI safety.

The post does not publish the review itself or detailed results, so the findings are presented as Amazon's summary of AVERI's work.