Beckmann World Models

Direct Terminal-Map Learning for
One-Step Action-Conditioned Prediction

Adam LeeBerkeley Shobhit AgarwalStanford JP SuhIndependent Researcher
Conditional Beckmann terminal-map conservation, source supervision, and motion and temporal losses
BWM combines terminal-map transport training with direct supervision of predictions from noise. Inference uses one call per RGB chunk or latent frame. Geometry is schematic; the noisy thumbnails illustrate input mixtures rather than observed autonomous-flow states.
Training the predicted future: changed pixels, temporal differences, frozen spatial features and detached generated history
Motion-weighted, temporal-difference and frozen spatial-feature losses supervise the directly predicted future. Recovery training uses detached generated histories with the recorded actions and future target. These signals act during training; deployment keeps one generator call per future chunk.

Source anchoring: measured endpoint error

The inherited stabilized map’s pure-noise error is 27 times its boundary error on the fixed Can probe. Direct source supervision reduces this gap.

Measured endpoint error across inputs, with shared-scale RGB error maps for one fixed sample and a single recorded reference.
Panel a: one-call MSE on 90 Can windows with four shared source draws. Panel b: one fixed crop and source draw, shown as RGB error maps computed from saved 8-bit outputs. All tiles share the same square-root colour scale. The motion-aware parent predates spatial supervision.

Qualitative results

PushTSpatial feature supervision

Recorded framesBWM

Robomimic CanSpatial feature supervision

Recorded framesBWM

Bridge-V2Recovery features · 500 updates

RT-1Recovery features · 1,000 updates

Selected examples from the evaluated models. RGB: four observed and 60 predicted frames. Native: eight-frame autoregressive chunks, each starting from four recorded frames; VAE-reconstructed references.

PushT
Model t = 4 t = 23 t = 43 t = 63
Ground truth PushT, Ground truth, frame 4 PushT, Ground truth, frame 23 PushT, Ground truth, frame 43 PushT, Ground truth, frame 63
DriftWorld PushT, DriftWorld, frame 4 PushT, DriftWorld, frame 23 PushT, DriftWorld, frame 43 PushT, DriftWorld, frame 63
MSE U-Net PushT, MSE U-Net, frame 4 PushT, MSE U-Net, frame 23 PushT, MSE U-Net, frame 43 PushT, MSE U-Net, frame 63
GPC PushT, GPC, frame 4 PushT, GPC, frame 23 PushT, GPC, frame 43 PushT, GPC, frame 63
AVDC PushT, AVDC, frame 4 PushT, AVDC, frame 23 PushT, AVDC, frame 43 PushT, AVDC, frame 63
BWM + spatial PushT, BWM + spatial, frame 4 PushT, BWM + spatial, frame 23 PushT, BWM + spatial, frame 43 PushT, BWM + spatial, frame 63
Robomimic Can
Model t = 4 t = 23 t = 43 t = 63
Ground truth Robomimic Can, Ground truth, frame 4 Robomimic Can, Ground truth, frame 23 Robomimic Can, Ground truth, frame 43 Robomimic Can, Ground truth, frame 63
DriftWorld Robomimic Can, DriftWorld, frame 4 Robomimic Can, DriftWorld, frame 23 Robomimic Can, DriftWorld, frame 43 Robomimic Can, DriftWorld, frame 63
MSE U-Net Robomimic Can, MSE U-Net, frame 4 Robomimic Can, MSE U-Net, frame 23 Robomimic Can, MSE U-Net, frame 43 Robomimic Can, MSE U-Net, frame 63
GPC Robomimic Can, GPC, frame 4 Robomimic Can, GPC, frame 23 Robomimic Can, GPC, frame 43 Robomimic Can, GPC, frame 63
AVDC Robomimic Can, AVDC, frame 4 Robomimic Can, AVDC, frame 23 Robomimic Can, AVDC, frame 43 Robomimic Can, AVDC, frame 63
BWM + spatial Robomimic Can, BWM + spatial, frame 4 Robomimic Can, BWM + spatial, frame 23 Robomimic Can, BWM + spatial, frame 43 Robomimic Can, BWM + spatial, frame 63

Quantitative results

RGB prediction

ModelPushTRobomimic Can
MSE ↓Motion MSE ↓LPIPS ↓MSE ↓Motion MSE ↓LPIPS ↓
MSE U-Net 0.03030.40080.08040.00510.05440.0104
DriftWorld 0.03140.40930.07900.01040.07690.0682
GPC 0.06301.42410.16090.02760.31030.0589
AVDC 0.06841.18400.18600.04000.26440.1194
BWM + spatial 0.00980.12470.02800.00220.02420.0056

388 PushT and 30 Can episodes. Four observed and 60 autoregressive predictions at 96 × 96 resolution.

A separate cumulative training-cost study

The source-anchored Can checkpoint reaches LPIPS 0.0225 after 2.64 cumulative training hours; local DriftWorld reaches 0.0682 after 10.19 hours. This earlier-stage study includes inherited training and is separate from the final 30-epoch spatial accuracy comparison above.

Existing cumulative Can training-time measurements

PushT policy rollouts and planning

World modelPlanning IoU ↑Policy correlation ↑Policy IoU error ↓
DriftWorld5-epoch baseline0.7090.8570.031
MSE U-Net5-epoch baseline0.6950.9150.012
GPC diffusion5-epoch baseline0.6150.3660.513
BWM + spatialSpatial epoch 300.7210.9280.010

Planning: GPC-RANK on 50 fixed evaluation seeds. Policy rollouts: seven frozen policies × 300 paired trials in the world model and simulator; correlation is Pearson r and error is mean absolute IoU difference.

Native video prediction

ModelBridge-V2RT-1Calls per frame
SSIM ↑PSNR ↑LPIPS ↓SSIM ↑PSNR ↑LPIPS ↓
Matched subset · 2,078 Bridge-V2 and 2,041 RT-1 training episodes
DriftWorld Population drifting objective.81321.919.104.81422.419.1291
MSE U-Net.81322.439.166.81523.026.1791
GPC.64219.932.335.69420.692.2943
AVDC.74619.595.159.74920.269.175100
BWM spatial .79622.130.155.80223.147.1661

215 Bridge-V2 and 198 RT-1 validation episodes. Eight-frame autoregressive chunks, each starting from four recorded frames. Metrics use VAE-reconstructed references.

Native video prediction · full-data refinement

The new recipe adds feature supervision to recovery predictions while retaining one model call per frame. These runs use the full training data and are reported separately from the matched-subset comparison above.

The added DINO and latent-feature drifting terms also supervise predictions made from detached generated histories. They act during training; the generator still uses one call per latent frame. The comparison below holds the parent checkpoint and added update count fixed within each dataset.

Matched RT-1 and Bridge-V2 LPIPS and FVD improvements from recovery-feature refinement
Matched continuations from the same parent checkpoint: RT-1 uses 1,000 added updates and Bridge-V2 500. Equal updates do not mean equal compute; PSNR decreases slightly.
ModelSSIM ↑PSNR ↑LPIPS ↓FID ↓FVD ↓
RT-1 · 86,634 training episodes · 198 validation episodes
BWM population Starting model · epoch 700.88427.3540.09219.77167.50
BWM population Original objective continued0.88427.3410.09320.56172.89
BWM + recovery features0.88627.2710.0618.05104.91

The two RT-1 continuations start from the same model and use the same added-update budget. All rows use the validation protocol above and one model call per frame. Recovery feature supervision lowers LPIPS, FID and FVD, with a small PSNR decrease; contact and motion errors remain.

Citation

Download .bib
@misc{lee2026beckmannworld,
  title = {Beckmann World Models: Direct Terminal-Map Learning for One-Step Action-Conditioned Prediction},
  author = {Lee, Adam and Agarwal, Shobhit and Suh, June Pyo},
  year = {2026},
  eprint = {2610.26903},
  archivePrefix = {arXiv},
  primaryClass = {cs.CV},
  url = {https://arxiv.org/abs/2610.26903}
}