Plans Are Not InterventionsMeasuring and repairing plan leakage in driving world models

Self-driving systems increasingly use learned world models to judge candidate plans and train policies. This requires predicting what happens if the ego executes a plan, yet a model trained on driving logs learns what happened when drivers chose that plan. These differ because drivers react to traffic: slowing down in the logs usually means the car ahead slowed down, so the model learns to infer other cars' motion from the plan. We call this the plan leak and measure it by changing only the plan and checking whether cars that cannot react move.

In OmniNWM, a 4D driving world model, a plan that drives into the car ahead, even one stopped at a red light, moves that car away by as much as the ego advances, while parked cars barely move, so the crash never appears. Policies trained in a leaky model crash up to 2.8 times as often, and large leaks make planners brake unnecessarily. The cause is how the model sees the plan, not missing data: training on counterfactual plans barely helps, letting each step see only the ego's past removes the leak, and averaging over past-only plans repairs frozen models, bringing OmniNWM's crash back in 16 of 18 scenes.

A planner needs to know what happens if it executes a plan, p(future | history, do(plan)). A model trained on logs learns p(future | history, plan). The two differ whenever the plan carries information about the rest of the world, and in driving it always does: a driver slows down because the car ahead slows down. So when a planner proposes to speed up, a plan-conditioned model moves the car ahead out of the way.

We measure this with one number. Give the model the logged plan and a counterfactual plan that differs only in the ego's motion, and read how far the car ahead moves in the generated future per metre the plan moves the ego further. That slope is the plan leak, β. Within seconds a real car ahead does not react to the car behind it, so the right answer is 0. Parked cars, which no plan can affect, are the control.

OmniNWM generates panoramic video with metric depth and semantics, conditioned on the ego's camera pose in every frame. We ran its released weights on 18 held-out nuScenes scenes with a real car up to 25 m ahead in the ego lane, and read the generated car from OmniNWM's own depth and semantics. The counterfactual plan adds the smallest constant acceleration that drives the ego at least 1 m into the real car. All tests were preregistered.

The generated ego does move: visual odometry on the generated frames recovers 86% of the commanded motion. The leak is linear (slope 0.94 over braking, mild and crashing plans) and works both ways. Braking, well within the training distribution, pulls the car ahead towards the ego in all eight scenes where the ego moves. A real car ahead reacts to neither plan. OmniNWM's released collision rule alarms on half of the safe logged plans.

We ran the same test on four more released world models. Vista, another video model, also keeps the gap, but executes under half of the plan change, so it counts as supporting evidence only. The occupancy models SparseWorld-TC and DOME leak near 1, I²-World about half. Parked cars stay below 0.1 in all three.

Let D_e and D_a be the future displacements of the ego and of another agent. A model fitted to logs learns, to first order, the slope β = Cov(D_a, D_e | h) / Var(D_e | h). When the ego follows a car, its motion is caused by the lead's, so β > 0, although the true, interventional slope is zero. How much a model absorbs depends on which plan steps it sees when it generates the next moment.

The causal model, whose world at t + 1 sees the ego only up to t, does not leak (|β| ≤ 0.13 under two plan families) yet keeps the real reaction of a car following the ego. Accuracy rewards the leak: logged-plan error is lowest for the whole-plan model. More counterfactual data does not remove it either. Fine-tuning SparseWorld-TC on counterfactual plans barely changes its leak, from 0.83 to 0.77, or 0.73 with fully random plans.

Extra ego motion closes the imagined gap by only 1 − β, so near β = 1 crash checks pass every plan. On 653 held-out decisions, trusting SparseWorld-TC's or DOME's verdict does not beat ignoring it; physics is best. The leak also shapes what is learned and planned inside the model.

In closed loop, a planner replans every 0.5 s and keeps plans whose predicted gap to every car ahead stays above a margin. To vary only the leak, it judges with the causal prediction plus k times (whole plan − causal). Harm grows with k. In the simulator the whole-plan model brakes hard four to eight times as often as the causal one, and 72% of these brakes are unnecessary. On real nuScenes traffic (log replay, 150 scenes) near-misses rise 1.7- to 3.3-fold at k = 3 for all seeds.

Frozen models can be repaired at test time. g-computation steps a one-step model forward, shows the candidate plan only up to the present, and averages over past-only continuations; it removes the leak and the policy cost of the next-0.5 s model. A whole-plan video model can be repaired for detection: query OmniNWM with a plan that reveals only the past, so no car can respond to the crash, and view that world from the candidate plan's cameras using OmniNWM's own metric depth.