Nvidia presented eight simulation-to-real robotics papers at the International Conference on Robotics and Automation, part of 28 papers the company had accepted at the event. The work spans multi-arm scheduling, cross-body navigation, grasping, precise assembly, and vision-language-action models, and the throughline is that policies trained entirely in simulation are landing on physical hardware with success rates that approach human-in-the-loop methods. The most striking number: a pipeline called PEEK delivered a 41x real-world accuracy improvement for a policy trained purely in simulation.
The headline navigation result comes from COMPASS, a framework that builds a baseline policy with imitation learning and then uses residual reinforcement learning inside Nvidia's Isaac Lab to specialize for different robot bodies. No real-world robot data is used at any training stage. COMPASS posted a 4.5x improvement in average success rate over an imitation-learning baseline and roughly 80% success across 20 real-world navigation trials on autonomous mobile robots and humanoids.
On the manipulation side, Grasp-MPC adaptively corrects a robot's motion as it closes on an object instead of executing a fixed plan. The team generated 2 million simulated trajectories across 8,000 objects using the GraspGen dataset and cuRobo motion planning, then trained on both successful and failed runs. The policy reached around 75% real-world success on grasping novel objects in cluttered tabletops and shelves, against a 41% baseline.
Key facts
- 01Nvidia had 28 papers accepted at ICRA, eight of them focused on simulation-to-real transfer across the robotics stack.
- 02The COMPASS navigation framework hit roughly 80% success across 20 real-world trials and a 4.5x improvement over an imitation-learning baseline.
- 03Grasp-MPC reached about 75% grasping success on real robots versus a 41% baseline, trained on 2 million simulated trajectories across 8,000 objects.
- 04The PEEK pipeline produced a 41x real-world accuracy gain for a sim-only trained policy, and 2-3.5x gains for larger VLA models.
- 05Nvidia's Physical AI Dataset has surpassed 15 million downloads, making it the largest open dataset for physical AI development.
Deformable Cluster Manipulation tackles a stranger problem: grabbing a tangle rather than a discrete object. The motivating task is clearing tree branches that have grown over a power line, where the robot uses its whole arm to wrap around and sweep the cluster aside. Nvidia built a tree generator using biological growth equations to produce synthetic trees, trained the policy across thousands of them in Isaac, and deployed it to real branches zero-shot.
ScheduleStream addresses parallelism on the other end of the stack. Traditional robot scheduling handles multi-arm tasks sequentially; ScheduleStream runs the computations on GPUs so multiple arms can plan and operate at once. Nvidia reports a 3x speedup across multi-arm planning scenarios on hardware including the Jetson edge platform, with code published on GitHub.
Two papers focus on precise assembly, the kind of work where a simulator's rounded edges and idealized sensors stop matching reality. SPARR splits the job: a policy trained in Isaac Lab learns the general assembly strategy, then a second layer on the real hardware corrects for whatever the simulator missed, using only the robot's own camera and no human demonstrations. SPARR improves success rates by 38% and cuts cycle time by about 30% versus zero-shot sim-to-real baselines, and on unseen NIST assembly tasks it improves success by nearly 75%.
Refinery extends that approach to multi-step assembly, where finishing step one in the wrong position makes step two impossible. By training across hundreds of simulated assembly scenarios and modeling how success varies with initial conditions, Refinery hit 91% simulation success and a nearly 11% mean improvement over baselines, with policies that can be chained for long sequences.
PEEK is the language-model-flavored result. A vision-language model reads the task instruction, highlights the relevant objects and a movement path in the camera image, and fades out the rest, so the underlying policy acts on an annotated view instead of the raw cluttered scene. For a sim-only policy, PEEK produced the 41x real-world accuracy gain; for larger VLA models and smaller policies, gains ranged from 2x to 3.5x. Because it operates at the image level, it plugs into any camera-based policy without modification.
SEAL, a collaboration with Carnegie Mellon, the University of Utah, and the University of Sydney, addresses a failure mode in long-horizon tasks: the model reasons correctly about what to do, then executes something different. SEAL fixes this at runtime without retraining by generating several candidate action sequences, simulating where each would lead, and picking the one that matches the model's stated plan. The method delivers up to 15% accuracy gains and stays robust against rephrased instructions, swapped objects, and shifted camera angles.
The caveats are familiar. Sim-to-real success rates around 75-80% are strong research results but well short of what industrial deployment requires, and most of these papers benchmark on constrained task suites rather than open-ended factory or household work. None of the eight papers claims a general-purpose robot policy, and several rely on hardware-specific finetuning at deployment. The branches-on-power-lines and pharmaceutical-lab scenarios remain demonstrations, not commercial products.
The broader signal is infrastructure. Nvidia's Physical AI Dataset has crossed 15 million downloads, the Isaac GR00T X Embodiment Sim dataset is among the most-downloaded in robotics, and nearly 50 accepted ICRA papers from labs including Carnegie Mellon, ETH Zurich, MIT, and UT Austin reference Nvidia tooling. The company is building the same kind of moat in robotics simulation that CUDA built in deep learning training: not the best individual robot, but the substrate that every serious robotics lab runs on. If embodied AI follows the language-model pattern, the platform that owns simulation owns the next decade of robot deployment economics.
Working on something we should cover, or seeing a story we missed? Send leads, documents, or feedback to hello@aichatdaily.com. For sensitive tips, see our secure tips page for Signal and PGP options.
Spotted an error? Email hello@aichatdaily.com with the URL and the issue, or read our full corrections policy.
