Novita Sandbox Integrates With Miles for Agentic RL Training
Novita Sandbox is now a listed sandbox provider in Miles, RadixArk’s open-source framework for large-scale reinforcement learning and model post-training. The integration supports Harbor trials and a Terminal-Bench-2 OpenEnv recipe, with provider selection controlled by an environment variable.
- Novita Sandbox is now listed as a provider in Miles.
- The integration supports Harbor trials and Terminal-Bench-2 OpenEnv episodes.
- Users select Novita with OPENENV_SANDBOX_BACKEND or HARBOR_ENV_TYPE.
- OpenEnv uses deterministic per-task templates and per-episode sandboxes.
- Sandbox TTL, keepalive, concurrency limits and retries support rollout operations.
A provider switch inside Miles
The integration is available on Miles’ main branch and covers two agentic environment recipes: Harbor trials and the Terminal-Bench-2 OpenEnv recipe. Users can switch the sandbox provider while continuing to use the Miles training command, without running a separate provider service.
For OpenEnv, setting OPENENV_SANDBOX_BACKEND=novita creates each episode’s sandbox on Novita in the region specified by NOVITA_DOMAIN. The quickstart requires a working Miles installation, a Novita account and API key, plus a Terminal-Bench-2 task checkout. The Novita extra installs novita-sandbox>=2.0.7.
Why isolated environments matter for RL agents
Miles separates rollout generation, policy updates and environment execution. Rollouts run on SGLang, while policy updates run on Megatron-LM, with FSDP2 and torchtitan available as alternatives. The environment layer is where an agent runs commands, examines files, observes results and chooses its next action.
Agentic episodes can install packages, edit repositories, start servers or leave processes running. If episodes share a filesystem or background process, later rollouts may begin from a different state. Novita’s role is to provide the execution environment, while Miles continues coordinating rollouts, rewards and policy updates. Task environments, agent processes, command execution and verification run inside a Novita Sandbox.
Harbor and OpenEnv use separate connectors
The Harbor recipe runs trials from the Miles rollout worker while hosting the task environment on Novita. It uses the harbor-miles-v0.20.0 branch and the harbor[novita] extra. Setting HARBOR_ENV_TYPE=novita selects Novita for the trials. The session server address must be reachable from the sandboxes rather than being a loopback address on the trainer.
The OpenEnv connector pairs Novita with Terminal-Bench-2. It creates one named sandbox template per task and one sandbox per episode. Template names are derived from the task ID and a digest of the build recipe, including the base image, build commands, resources, build user and start command. Changing the recipe therefore produces a new template name instead of silently reusing a stale image.
Lifecycle controls support longer rollout jobs
At the start of an OpenEnv episode, the sandbox launches the environment server and waits for its health endpoint before handing control to the agent. When the episode ends, the sandbox is killed. Per-episode sandboxes keep modified files and running processes from leaking into later episodes.
The integration also includes a time-to-live mechanism, keepalive behavior and bounded concurrency. OPENENV_NOVITA_SANDBOX_TTL_S defaults to 1800 seconds, and a keepalive thread re-arms the TTL while the creating process is alive. If that process is hard-killed, the sandbox is reclaimed within the TTL. Configurable concurrency limits and retries for Novita API rate-limit responses are intended to support larger rollout jobs.
For users running Miles from main, the integration provides remote isolated environments for Harbor tasks and Terminal-Bench-2 episodes without requiring local Docker capacity for concurrent trials. Novita says its sandbox is supplying the execution layer while the Miles recipes retain responsibility for the surrounding training workflow.
