RoboGym — play a game, train a robot

The problem

AI learned to see and write from the internet. But there's no internet of physical actions — so robots still can't reliably pick up a cup.

“There is no such treasure trove of robot data… large amounts of data need to be collected with that particular robot, for that particular application.”

— Physical Intelligence (π₀), 2024 · a lab now valued ~$5.6B on exactly this problem

Leading labs build this dataset by paying teleoperators to puppet robots one task at a time — slow, expensive, and narrow. The thing they can't simply buy is breadth: thousands of different people solving thousands of tasks thousands of ways. That diversity is exactly what a game collects best.

How it works

Playing the game is the data collection.

01 — PLAY

↦

You solve tasks

Steer a robot arm to reach, sort, and stack objects against the clock. Combos, ranks, a global leaderboard. It's a real game.

02 — RECORD

⦿

We capture every move

Each frame logs joint angles, end-effector position, the objects, and your gripper action — a complete state→action trajectory.

03 — TRAIN

⟁

Robots learn from it

Sessions export as LeRobot-compatible episodes — the same format real robot policies train on. Your play becomes their lessons.

The payoff

Teach it. Then watch it play itself.

This is the whole point of Physical AI, running in your browser. After you demonstrate, RoboGym trains an imitation-learning policy from your moves — and the arm starts solving tasks on its own. Because it learns from the pooled data, everyone's play makes every robot better. That's the data flywheel — and you can watch it spin.

You demonstrate → it learns a policy → the arm acts on its own → more play makes it better

Real data, not a toy

Every solve is a labeled training episode.

No mock-ups. The game emits structured demonstration data in a schema that drops straight into a Hugging Face / LeRobot pipeline. We're honest about what it is: high-diversity data for co-training and demo-amplification — not standalone robot training. That's exactly the axis the research shows moves the needle.

episode_0247.json

// one recorded frame
{
  "t": 2.40,
  "observation": {
    "joint_angles": [-0.94, 1.21],
    "ee": [412, 268],
    "object": [395, 301],
    "goal": [430, 300],
    "gripper": 1
  },
  "action": {
    "target_ee": [410, 260],
    "gripper": 1
  }
}

Why now

Everyone's building the robots. Almost no one's building the open data layer.

Think Foldit, but for teaching robots to use their hands — a game that generates the breadth of demonstrations labs can't buy, pooled into an open dataset on the format the field already runs on. The wedge is fun; the moat compounds with every player.

$27.6B

into robotics & Physical AI in 2025 — up from $13.7B the year before.

16–56%

policy-success lift from gamified demo data (RoboCade, Stanford 2025) — our benchmark to beat.

~16,000

open LeRobot datasets on Hugging Face — median just 10 episodes. The ecosystem is starving.

Play a game. Train a robot.