Scale AI's AgentEnv evals can now run on Vercel Sandbox
AgentEnv, Scale AI's framework for testing agents against tasks and scoring their results, now runs on Vercel Sandbox. According to Vercel, the provider ships in AgentEnv 0.9.1334 and later.
- AgentEnv, Scale AI's framework for testing agents and scoring results, now runs on Vercel Sandbox.
- The Vercel provider ships in AgentEnv 0.9.1334 and later.
- The agent and its task environment run in separate Sandboxes.
- An example lets users inspect the task's score and the agent's recorded actions.
- The run command uses the existing Vercel CLI login and requires a team slug and an existing project name.
What Vercel has added
Vercel announced in its changelog that AgentEnv, the framework Scale AI built for testing agents against tasks and scoring their results, can now use Vercel Sandbox as its execution environment. The support comes as a Vercel provider included in AgentEnv version 0.9.1334 and later.
The provider takes a specific approach to isolation. According to Vercel, it runs the agent and its task environment in separate Sandboxes. That means the agent being evaluated and the environment it is working against do not share a single runtime.
How to run an evaluation
Vercel points developers to an AgentEnv example that runs an evaluation on Vercel Sandbox. With it, users can inspect the task's score and the agent's recorded actions, which covers both the outcome of the test and the steps the agent took to get there.
The process starts with the setup guide. Once that is complete, the command is run from the cloned repository. It uses the user's existing Vercel CLI login:
- Command: ./vercel-eval --scope YOUR_TEAM --project YOUR_PROJECT
- YOUR_TEAM: replace with your Vercel team slug.
- YOUR_PROJECT: replace with an existing project name.
Because the command relies on an existing project, a project must already be set up on the Vercel account before the evaluation is run.
What it means for developers
For teams already using Vercel, the change offers a way to run AgentEnv evaluations on Vercel Sandbox through their existing CLI login and project setup. The separation of agent and task environment into two Sandboxes is the main design detail Vercel highlights.
The announcement is brief and does not include performance figures, pricing or availability details beyond the version number. Developers who want more information can consult the Sandbox ecosystem documentation that Vercel links from the changelog entry.
Vercel: Run Scale AI’s AgentEnv evals on Vercel Sandbox
