What Simulation Cannot Teach a Robot

Every robotics team I have worked with has a version of the same story. The policy hits 98% success in simulation on Friday. On Monday it goes onto the real arm, and within an hour it has knocked a part off the fixture, stalled a gripper finger, and once, memorably, tried to grasp a shadow.

Nobody did anything wrong. The simulator was good. The training was sound. The robot simply lived in a slightly different universe from the one it learned in, and the differences mattered.

That gap has a name, the reality gap, and closing it has become one of the most productive research areas in robotics. It is worth understanding what the tools actually do, what they demonstrably fix, and what they leave for the engineer standing next to the cell.

The simulators we lean on

Three tools show up in almost every modern robotics stack.

MuJoCo (Multi-Joint dynamics with Contact) was introduced by Emanuel Todorov, Tom Erez and Yuval Tassa at IROS 2012. It was designed for model-based control: generalized coordinates, a velocity-stepping contact formulation, and fast enough that the original paper reports around 400,000 dynamics evaluations per second for an 18-DOF humanoid with six active contacts on a 12-core machine. DeepMind acquired it in 2021 and it is now free and open source.

Gazebo is the workhorse of the ROS world. It grew out of the Player Project and is now maintained by Open Robotics. If you are still on Gazebo Classic, note that it reached end-of-life in January 2025; ROS 2 Jazzy expects the modern, lettered Gazebo releases.

NVIDIA Isaac Sim is built on Omniverse and leans hard into photorealistic rendering and synthetic data. It can randomize lighting, colour and object placement and export labelled data in formats like COCO and KITTI, which makes it popular for training perception models.

All three are excellent. All three are also approximations, and the approximations are not evenly distributed. Rigid-body kinematics are nearly perfect. Contact is not.

What the reality gap is made of

When people say "the sim doesn't match", they usually mean one of four things.

Contact and friction. Real contact is a messy, compliant, slightly sticky event that depends on surface finish, contamination and load history. Most simulators model it with a friction cone and a stiffness or constraint solver. Grasping a machined aluminium block and grasping the same block with a film of cutting fluid on it are different problems, and only one of them is in your sim.

Actuation. Simulated motors produce the torque you ask for. Real ones have backlash, gear compliance, current limits and thermal derating. Tendon-driven hands are worse.

Latency. A real control loop has camera exposure time, USB transfer, inference time, network hops and controller cycle time. A policy trained at a clean 50 Hz with zero delay can oscillate badly when its observations arrive 40 ms stale.

Wear. This is the one people forget. The robot you characterised in January is not the robot you have in June. Grippers pick up grime, cables stiffen, joints loosen. A simulator has a fixed model of a robot that never ages.

Domain randomization: making the sim deliberately wrong

The most influential idea for crossing the gap is counterintuitive. Instead of making the simulator more accurate, make it vary.

Tobin et al. (2017) trained an object detector purely on simulated images with random, deliberately non-realistic textures, lighting and camera positions. The real-world detector localised objects to within 1.5 cm and was used for grasping in clutter. The argument is simple: if the network has seen thousands of strange worlds, the real one looks like just another variation.

Peng et al. (2017) applied the same idea to dynamics, randomising masses, friction, damping and latency during training of a pushing policy on a robot arm. Their policies transferred without any real-world training and stayed robust to significant calibration error.

The Dactyl result, read carefully

OpenAI's 2018 paper "Learning Dexterous In-Hand Manipulation" is the canonical success story. A Shadow Dexterous Hand learned to reorient a block to arbitrary target orientations using a policy trained entirely in MuJoCo with heavy randomization of physics and appearance.

The headline numbers are impressive. Experience was generated on 6,144 CPU cores with an 8-GPU optimizer. Learning the task without randomization took about 3 years of simulated experience; learning it under full randomization took about 100 years, roughly 50 hours of wall-clock time.

The details are more useful to a practitioner than the headline:

In simulation the block policy achieved a median of 50 consecutive successful rotations. On the physical hand the median was 13. The authors state plainly that randomization and calibration narrow the reality gap but do not remove it.
The hand is tendon-driven with substantial backlash, while the MuJoCo model applied torque directly to the joints. The team added a hand-written backlash model, action delays, action noise, simulated motion-capture dropouts and small random forces on the object.
The wrist pitch joint was the most common source of hardware breakage. A policy trained with that joint locked transferred better.
The authors list robot breakages as one of the key challenges of the work. Repairs took time and often changed the system enough that results had to be collected at different times.

Read that list again. Every item is a judgment call made by a person who was looking at the physical robot: which unmodeled effect matters, which joint to lock, when a repaired hand is "the same robot" for evaluation purposes. The learning algorithm did not make any of those decisions.

Why the shipping decision stays on the floor

Suppose your simulator is perfect for today's robot. You still have to answer questions that are not in the reward function.

What does failure cost here? Dropping a foam block costs nothing. Dropping a 4 kg casting onto a conveyor belt that a person reaches into costs a great deal. The same 2% failure rate is acceptable in one cell and unacceptable in the other.

What is the distribution on this line, this month? The sim was randomized over friction coefficients between, say, 0.5 and 1.2. Did anyone measure the new supplier's parts? Has the cell started running a coolant that changes surface friction?

How does it fail? A policy that fails by stopping is very different from one that fails by accelerating. Simulation success rates rarely tell you which you have.

What has changed since the last validation? A gripper pad swap, a camera remount 3 mm lower, a firmware update on the drive. Any one of them can move you out of the distribution you trained on.

The engineers who answer these questions well do it with unglamorous practices:

Shadow mode first. Run the policy with its outputs logged but not executed, alongside the existing controller, and compare decisions for a full shift.
Speed and force envelopes. Ship at 30% speed with tight torque limits, then open them up in steps with explicit sign-off at each step.
Real-world holdout sets. Keep a box of physical "nasty" parts: the warped one, the oily one, the one with a burr. Every candidate policy runs through the box.
Re-measure the robot. Periodically identify friction, backlash and latency on the actual hardware, and check them against the randomization ranges you trained on.
A named owner. Someone signs the release. If nobody is willing to sign, the policy is not ready.

What more capable AI changes, and what it does not

Better models, better simulators and better world models will keep shrinking the gap. Tools that automatically tune simulator parameters from real rollouts already exist, and they will get better.

What they will not do is decide how much risk a particular plant, with particular people walking past it, is willing to carry. They will not notice that the night shift has started stacking parts two-high because the outbound conveyor is backed up. They will not be accountable when the arm drops something.

Simulation is the best practice field robotics has ever had. It is not the match. The person who decides when a policy goes live needs to have stood next to the real cell, watched it fail, and understood why.

Sources

MuJoCo: A physics engine for model-based control (Todorov, Erez, Tassa, IROS 2012) — original MuJoCo paper; generalized coordinates, velocity-stepping contact, benchmark figures.
MuJoCo project site — current free and open-source engine maintained by Google DeepMind.
Gazebo Classic End-of-Life (Open Robotics Discourse, Feb 2024) — Classic EOL in January 2025 and the need for modern Gazebo with ROS 2 Jazzy.
NVIDIA Isaac Sim — built on Omniverse; scene randomization and synthetic data export (COCO, KITTI).
Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World (Tobin et al., 2017) — sim-only detector accurate to 1.5 cm, used for grasping in clutter.
Sim-to-Real Transfer of Robotic Control with Dynamics Randomization (Peng et al., 2017) — dynamics randomization for a pushing task; robust to calibration error.
Learning Dexterous In-Hand Manipulation (OpenAI, 2018) — sim-trained Shadow Hand policy; 50 vs 13 median rotations, backlash model, hardware breakage.