Amazon says autonomous AI agents built its AWS Emulator

An AWS emulator built without human code review

AWS Emulator is designed to run on a user’s machine for building and testing AWS applications. Amazon says its team did not write or review any of the code, relying instead on a system that limits what the agents can change and checks their work before it is merged.

The project began as a weekend experiment. Amazon gave Kiro a prompt to write tests based on the AWS Command Line Interface, make those tests pass against real AWS, and then build an in-memory implementation that passed the same tests. After 15 minutes, Kiro had produced a first version of Amazon Simple Queue Service. Amazon says the team refined the prompt and had Kiro build the rest of AWS over the weekend.

The company says the initial result had gaps, but was useful enough for its team to begin testing software with it the following Monday. Turning the experiment into a product required additional engineering work, particularly the guardrails used to keep agents within defined boundaries.

Three agent teams divide the work

AWS Emulator uses three groups of agents, each assigned a bounded role. The first manages a fidelity test suite based on AWS API Models and checks every test against the real AWS cloud before submitting it. Amazon says the suite contains tens of thousands of tests across AWS and continues to grow as agents discover additional behavior.

The second group replays those tests against the emulator, finds divergences and files issues. Agents are instructed to reason about failures and group problems that may share a root cause. The third group receives individual issues, investigates them and changes the code to address them.

Before a change can be merged, it passes a gate that checks whether it compiles, follows coding standards, is secure and has not reduced fidelity. Failed checks send the change back for rework. Amazon says hundreds of agents work concurrently and that thousands of issues have been fixed each day.

Code and test history are available on GitHub

Once changes reach the mainline codebase, Amazon’s release pipeline builds and retests them before packaging the emulator as a downloadable binary. The company says the code is published to GitHub at the end of that process, along with the fidelity tests.

The repository includes every issue filed by the agents. Fixed issues are stored in issues/things-that-now-work, while unresolved items are in issues/things-that-do-not-work. Amazon says the source code is available under a Creative Commons CC0 license and is continually updated.

For users, the release offers a local way to build and test AWS applications while exposing the testing and issue history behind the software. For developers working on autonomous coding systems, Amazon’s account also presents AWS Emulator as a case study in using narrowly defined tasks, repeatable tests and merge gates to control agent-generated code. Amazon says future posts will cover its multi-agent workflow, fidelity measurement, quality and security processes.