RAI Institute Releases Exploy to Ease Robot Policy Deployment

October 7, 2026, 16:21Source: RAI Institute

The RAI Institute has introduced Exploy, an open-source library that exports a reinforcement learning policy together with its full environment logic as one self-contained computational graph. According to the institute, the tool is meant to shrink the effort of moving policies from simulation onto real robots.

Key points

The RAI Institute released Exploy, an open-source library that exports environment logic and neural network policies as one computational graph.
Exploy traces PyTorch operations and packs the pipeline and metadata into a single .onnx file run by a C++ controller built on ONNX Runtime.
It has built-in adapters for IsaacLab and MjLab, supports architectures such as MLPs, LSTMs, CNNs, Transformers and diffusion policies, and offers a ROS 2 package.
Evaluation tools check step-by-step numerical equivalence between the PyTorch policy and the exported ONNX artifact before hardware runs.
The institute says it has used Exploy on Roadrunner, Spot, UMV, Unitree G1 and Atlas.

The deployment problem Exploy targets

Reinforcement learning has become a popular method for agile locomotion and manipulation in robotics, but the RAI Institute says moving a simulation-trained policy to physical hardware remains a major bottleneck. In its blog post, the institute argues that the sim-to-real gap comes not only from out-of-distribution observations or physics model mismatches, but also from software implementation discrepancies, bugs and code complexity.

Training usually happens in Python with RL frameworks, while real-time control loops run in high performance environments. Bridging the two without silent bugs or weeks of rewriting infrastructure is, in the institute's words, a persistent engineering challenge.

Common workflows export only the network weights in an open format such as ONNX. That makes the weights portable, but developers still have to manually rewrite observation functions, sensor processing and action post-processing. The RAI Institute lists three pain points:

Numerical discrepancies: small differences in math or floating-point tolerances between Python and C++ can cause unexpected and dangerous robot behavior.
Engineering friction: recreating environment logic for each iteration is error-prone, labor-intensive and hard to audit.
Parallel codebases: simulation environments and control stacks diverge over time, which makes sharing policies or replicating research harder.

How Exploy works

Exploy, short for export and deploy, traces PyTorch operations directly from the source environment. Rather than exporting only weights, it exports the entire control pipeline, mapping raw sensor measurements to outputs that range from low-level actuator commands to high-level velocity targets. The library was developed mainly with reinforcement learning in mind, though the institute says its framework is generalizable.

It has two main parts:

Exporter: traces observation generation, the neural network forward pass and action processing in PyTorch. It registers inputs, outputs, persistent memory for recurrent models such as LSTMs and RNNs, and custom metadata such as control rates, joint stiffness and damping gains, through a unified context manager.
Controller: a lightweight C++ runtime built around ONNX Runtime. It automatically discovers ONNX tensor inputs and outputs and wires them to physical hardware state interfaces.

Because the full graph and configuration metadata sit in a single .onnx file, one C++ deployment stack can load and run policies across different tasks and robots without manual code changes, according to the institute.

On compatibility, Exploy has built-in adapters for IsaacLab and MjLab, plus an extensible interface for custom PyTorch-based frameworks. Since it builds on ONNX, it supports any operation ONNX supports, including MLPs, LSTM and GRU recurrent networks, CNNs, Transformers and diffusion policies. For hardware integration it offers plain CMake exportable libraries and a native ROS 2 package.

Field use and what it means for roboticists

The RAI Institute says its research groups already use Exploy on a range of platforms. Examples include Roadrunner, a custom multimodal robot that transitions between Segway balancing, bicycle driving and stepping modes with a unified RL policy. Exploy handles deployment of its unified control and get-up policies zero-shot from simulation to hardware. Boston Dynamics' Spot has used depth-based and recurrent policies through Exploy for high obstacles and aggressive parkour terrain.

The library is also used with the UMV and to power autonomous trail racing on RAI's custom autonomous bike platform. The institute adds that it has been deployed on the Unitree G1 humanoid and Boston Dynamics' Atlas for autonomous navigation, blind walking and rough terrain traversal with locomotion policies.

For teams working on sim-to-real transfer, the institute's pitch is less manual C++ re-implementation of observation and action pipelines, a smaller control codebase and shorter iteration time. Exploy also includes evaluation tools to verify step-by-step numerical equivalence between the original PyTorch policy and the exported ONNX artifact before any code runs on physical hardware. The code and instructions for exporting environments or integrating the C++ controller are available in the Exploy GitHub repository.