RAI Institute's Spot rolls a tire outdoors using only onboard cameras

From a lab setup to full onboard execution

In earlier work, the RAI Institute showed Spot uprighting and rolling a car tire. The tire weighs 15 kg, more than Spot's arm can lift alone (11 kg peak, 5 kg continuous), so the robot had to coordinate its legs, body and arm. That controller combined reinforcement learning with sampling-based trajectory optimization.

The earlier system worked only in the lab because it depended on three things that were not on the robot: motion-capture cameras in the ceiling that located the tire, a desktop workstation running the controllers, and a script that read the motion-capture state and decided when to switch skills.

According to the institute, the new system handles all of this itself. The robot finds the tire with its own depth cameras, which lets it work outdoors and in previously unseen environments. It can roll, upright and drag the tire and recover from unexpected perturbations. In the demonstration, a person simply gives the robot a goal location, and the robot performs the whole sequence, including standing the tire up if it tips over on the way.

Every skill is a neural network

The control stack has two layers. A locomotion policy trained with reinforcement learning keeps the robot balanced and walking. A skill policy decides where the body should go and what the arm should do. The skill layer is what changed.

The previous version sampled dozens of possible actions and simulated their futures across multiple CPU threads to upright, drag and stack the tire, while reinforcement learning handled rolling. That worked, the institute says, but needed a powerful desktop computer and an engineered script to mix the techniques. Now each skill is a reinforcement learning policy trained in simulation with the tire's exact pose and then transferred to use the robot's cameras. Running a skill on the robot takes a single neural network pass.

• A skill policy outputs ten numbers at 50 Hz: three adjustments to base velocity and seven to the arm's joint targets.
• The adjustments accumulate, so zero holds the current target and a sustained push builds sustained force on the tire.
• All skills share the same action space and reward template, which rewards progress toward the goal, getting the gripper onto the tire and positioning the body, and penalizes jerky tire motion.
• Tire mass, friction and size are randomized in training so the policy cannot rely on one exact tire.

With 16,000 to 64,000 simulated robots training in parallel on a scene, a single skill trains in a few hours on one desktop GPU, according to the institute.

Crossfade RL: teaching the policy to see

A policy trained with state information always knows where the tire is, because the simulator tells it. The usual way to move to cameras is to freeze the trained policy as a teacher and train a camera-based student to imitate it. The institute says this transferred some tasks such as uprighting and dragging, but more dynamic motions like re-orienting the tire during rolling worked poorly.

Its answer is a pipeline that keeps reinforcement learning running throughout training while privileged inputs are gradually replaced with more realistic perceptive observations. It has three stages, each training an encoder that learns to output the same feature vector as the previous one from less privileged information:

• Encoding object position from exact poses: the exact tire pose is encoded into a latent vector, used as the ground-truth comparison for the next stage.
• Learning to handle intermittent visibility: exact poses are given only when the camera sees the tire, otherwise the input is zeroed out, so the encoder learns robust behavior when the tire is out of view.
• Translating from poses to images: the input is depth images, so the network uses only information available from vision sensors.

As each stage converges, the skill policy relies less on the previous encoder until the new one fully replaces it. The institute calls the method Crossfade RL, because the policy keeps learning with reinforcement learning while distillation helps newly introduced components.

What it means

The result moves a whole-body manipulation system out of a motion-capture lab and onto a robot that carries everything it needs. Replacing the CPU-heavy planner and switching script with neural network skills simplifies the system, and the institute says it lets Spot operate in environments it has not seen before. The source describes the work for a single object, a tire, and does not give performance figures or deployment plans.