GeoWAM: Visual Geometry World Action Models for Autonomous Driving

Yiren Lu^1,2, Xin Ye¹ , Jiaming Liu¹, Philip Jacobson¹, Jin Yao¹, Yi-chung Chen¹, Liam Merino¹, Dhruva Dixith Kurra¹, Min Cai¹, Tom Lampo¹, Yu Yin² , Danhua Guo¹, Burhan Yaman¹^†

¹Uber AV Labs
²Case Western Reserve University
Corresponding authors
^†denotes project lead

Paper arXiv

Abstract

Video world models predict how pixels evolve, leaving 3D transformations implicit; geometry world models predict future 3D structure, making the transformations explicit.

Video and geometry world models represent scene dynamics differently. Given the same current observation, a video world model predicts how pixel values evolve over time, and the underlying 3D transformations remain implicit in these pixel changes and are therefore difficult to recover. In contrast, a geometry world model predicts future 3D structure, whose evolution explicitly exposes the underlying spatial transformations and provides a representation naturally aligned with motion planning.

World action models (WAMs) have recently gained increasing attention as a framework for jointly modeling scene evolution and ego actions in autonomous driving. Most existing WAMs learn scene dynamics in pixel space by combining a video-generation backbone for future-observation prediction with an action head for ego-trajectory prediction. Pixels, however, provide only an indirect representation of these dynamics: they entangle geometry and motion with appearance, texture, and illumination, forcing the model to infer three-dimensional transformations from two-dimensional observations. We argue that geometry, represented by point clouds, offers a more natural state space for driving because it explicitly captures spatial structure and the rigid and non-rigid transformations that govern scene evolution while directly aligning with the space in which driving actions are executed. Building on this insight, we introduce GeoWAM, a visual geometry world action model for autonomous driving. Rather than predicting future images, GeoWAM is pretrained to forecast future scene geometry, yielding representations that jointly encode spatial structure and temporal evolution. A geometry-conditioned action head then leverages these learned geometric dynamics to predict future ego trajectories. Extensive open-loop and closed-loop evaluations show that visual geometry world modeling yields substantially stronger driving policies than image-based alternatives, establishing future-geometry prediction as an effective pretraining objective for autonomous driving.

Method Overview

GeoWAM architecture: a geometry encoder builds a history memory of geometry and ego tokens, a future geometry decoder forecasts future point maps, and a future pose decoder feeds a trajectory head.

Stage 1 — Visual geometry world model. A geometry encoder maps the multiview image history into multi-level geometry tokens, which describe the spatial structure seen from each camera, and ego tokens, which carry ego-motion context. Together they form a history memory. A future geometry decoder starts from learned geometry queries that carry temporal, view, and 2D positional embeddings; causal temporal self-attention models how each spatial location evolves across the future steps, and cross-attention to the history memory retrieves the relevant observed context. A shared geometry head decodes the resulting features into a dense point map and a per-pixel confidence map for every future step. Supervision comes from cosine feature alignment against targets produced by the same encoder on the future frames (detached, and never given to the forecasting branch or used at inference), together with a dense point-map loss combining Euclidean regression, confidence-aware regression, and multi-scale surface-normal consistency, applied to both the future steps and the current frame.

Stage 2 — Geometry-conditioned action head. GeoWAM adopts an inverse-dynamics-like formulation: rather than regressing a trajectory directly, learned future ego queries cross-attend to both the history memory and the predicted future geometry to produce future ego tokens that encode the ego motion compatible with the forecast scene evolution. A stop-gradient on the predicted geometry keeps the trajectory loss from reshaping the pretrained forecasting capability. The action head refines the historical ego-token sequence with a causal temporal transformer, conditions a learnable trajectory query on the predicted future ego tokens, and regresses a single trajectory directly — with no trajectory anchors, mode classification, or iterative sampling.

Future Geometry Prediction

We convert each predicted future point map to ray depth and report absolute relative error (Abs Rel, lower is better) and threshold accuracy δ < 1.25 (higher is better), at horizons from one to four seconds and averaged over all eight predicted frames. Video world models generate future RGB frames, so we reconstruct their geometry with DVGT to give every method a common geometric output.

Method / Horizon	Abs Rel ↓					δ < 1.25 ↑
Method / Horizon	1s	2s	3s	4s	mean	1s	2s	3s	4s	mean
Epona + DVGT	0.229	0.263	0.292	0.310	0.274	0.732	0.677	0.620	0.589	0.655
Cosmos 3 + DVGT	0.300	0.376	0.405	0.422	0.376	0.588	0.513	0.464	0.447	0.503
VGGT-World	0.272	0.329	0.342	0.357	0.325	0.612	0.553	0.513	0.497	0.544
GeoWAM (ours)	0.228	0.245	0.256	0.297	0.257	0.708	0.769	0.746	0.703	0.754

Future geometry prediction at different horizons. Bold and underlined values indicate the best and second-best results. GeoWAM attains the lowest Abs Rel at every horizon, improving the aggregate mean from 0.274 for the strongest baseline to 0.257, and lifts the mean δ < 1.25 from 0.655 to 0.754.

Planning on NAVSIM v2 `navtest`

We finetune GeoWAM on the NAVSIM navtrain split and evaluate with the Extended Predictive Driver Model Score (EPDMS), which aggregates no at-fault collision (NC), drivable-area compliance (DAC), driving-direction compliance (DDC), traffic-light compliance (TLC), ego progress (EP), time-to-collision (TTC), lane keeping (LK), history comfort (HC), and extended comfort (EC). All metrics use the official human-penalty protocol; higher is better.

Method	NC ↑	DAC ↑	DDC ↑	TLC ↑	EP ↑	TTC ↑	LK ↑	HC ↑	EC ↑	EPDMS ↑
Transfuser	96.9	89.9	97.8	99.7	87.1	95.4	92.7	98.3	87.2	84.0
Hydra-MDP++	97.2	97.5	99.4	99.6	83.1	96.5	94.4	98.2	70.9	81.4
DriveSuprim	97.5	96.5	99.4	99.6	88.4	96.6	95.5	98.3	77.0	83.1
ARTEMIS	98.3	95.1	98.6	99.8	81.5	97.4	96.5	98.3	–	83.1
DiffusionDrive	98.2	96.2	99.5	99.8	87.4	97.3	96.9	98.4	87.7	88.2
WoTE	98.5	96.8	98.8	99.8	86.1	97.9	95.5	98.3	82.9	87.7
DriveVLA-W0	98.4	95.2	99.4	99.9	86.6	97.9	97.8	98.3	82.7	86.9
PWM	98.8	95.9	99.4	99.9	86.4	98.4	97.6	98.3	85.3	88.2
DriveLaW	98.7	96.9	99.6	99.8	87.5	98.3	97.6	98.4	77.4	88.6
DVGT-2	98.7	97.9	99.7	99.9	87.9	98.0	98.2	98.2	77.0	89.6
EponaV2	98.5	97.4	99.5	99.9	87.9	98.1	97.7	98.2	77.4	88.9
GeoWAM (ours)	98.7	97.7	99.7	99.9	87.0	98.1	97.9	98.3	86.8	90.2

Results on the NAVSIM v2 navtest split. GeoWAM reaches an EPDMS of 90.2, improving on its DVGT-2 initialization by 0.6 points and giving the best overall score in the table, while matching the best DDC and TLC and staying competitive on the remaining safety and progress components.

Two-Stage Planning on `navhard`

The navhard benchmark approximates closed-loop evaluation with scenes reconstructed by 3D Gaussian Splatting: after the planner predicts a trajectory, a new observation is rendered from the resulting ego pose and fed back for the next planning step, so planning errors propagate into subsequent observations. Stage 1 (S1) evaluates the original scenes and Stage 2 (S2) the synthetic reactive scenes.

Method	Stage	NC ↑	DAC ↑	DDC ↑	TLC ↑	EP ↑	TTC ↑	LK ↑	HC ↑	EC ↑	EPDMS ↑
CV	S1	88.8	42.8	70.6	99.3	77.5	87.3	78.6	97.1	60.4	11.4
CV	S2	83.2	59.1	76.5	98.0	71.3	81.1	47.9	97.1	61.9	11.4
Ego MLP	S1	93.2	55.7	86.6	99.3	81.2	92.2	83.5	97.5	77.7	14.1
Ego MLP	S2	77.2	51.9	74.4	98.2	77.1	75.0	40.8	97.8	79.8	14.1
LTF	S1	96.2	79.5	99.1	99.5	84.1	95.1	94.2	97.5	79.1	25.1
LTF	S2	77.7	70.2	84.2	98.0	85.1	75.6	45.4	95.7	75.9	25.1
DriveVLA-W0	S1	96.8	83.3	99.0	99.6	84.6	95.3	96.4	97.6	78.2	24.4
DriveVLA-W0	S2	76.8	64.3	79.9	98.3	89.2	75.0	46.8	95.8	53.1	24.4
DriveLaW	S1	97.3	89.1	99.2	99.6	84.3	97.1	96.2	97.8	67.6	30.6
DriveLaW	S2	82.5	67.6	83.5	98.1	84.8	78.5	45.8	96.4	57.3	30.6
DVGT-2	S1	97.2	91.3	98.4	99.8	84.8	95.5	95.5	97.5	71.4	31.7
DVGT-2	S2	77.8	73.8	81.3	98.3	91.5	73.2	48.0	83.9	45.1	31.7
LTFv6^†	S1	96.5	86.6	99.2	99.5	84.4	95.1	94.4	97.7	76.4	31.9
LTFv6^†	S2	79.8	75.5	86.2	97.8	89.5	76.0	50.0	95.2	66.7	31.9
NavFormer^†	S1	96.2	92.4	95.7	99.6	83.8	96.0	94.7	96.4	60.9	34.1
NavFormer^†	S2	85.7	81.0	83.5	97.6	90.1	82.4	48.2	94.9	48.4	34.1
EponaV2^†	S1	97.3	90.7	99.4	100.0	83.3	97.3	97.3	97.6	60.9	36.1
EponaV2^†	S2	83.6	78.0	88.0	98.9	86.0	80.3	50.1	96.1	52.0	36.1
GeoWAM (ours)	S1	97.7	91.5	99.1	99.8	83.8	95.8	96.0	97.8	79.0	36.6
GeoWAM (ours)	S2	80.4	76.3	87.3	98.7	88.9	76.2	49.9	94.0	56.0	36.6

navhard leaderboard. Methods trained with reinforcement learning or PDMS-score supervision are shown in gray and marked with ^†; among the remaining methods, bold and underlined values indicate the best and second-best results. GeoWAM reaches an EPDMS of 36.6, outperforming all baselines including those trained with score supervision.

Qualitative Results

Aggregated future geometry predictions with predicted ego poses for a left turn, driving straight, and a right turn.

For three representative maneuvers — turning left, driving straight, and turning right — we aggregate the predicted geometry from all future time steps into a single visualization, with bounding boxes marking the predicted ego poses at successive steps. Across all three, GeoWAM preserves coherent scene structure over the prediction horizon and reconstructs environmental elements such as trees and poles as well as fine-grained road markings. In the left-turn case, another vehicle follows the ego vehicle through the turn in the predicted future geometry, showing that GeoWAM captures the dynamics of surrounding agents and not only ego motion. In the straight-driving case, the predicted trajectory steers around a vehicle along the roadside, showing that the forecast geometry provides actionable spatial context for planning.

Citation

If you find our work helpful, please consider cite us:

@misc{lu2026geowam,
      title={GeoWAM: Visual Geometry World Action Models for Autonomous Driving},
      author={Yiren Lu and Xin Ye and Jiaming Liu and Philip Jacobson and Jin Yao and Yi-chung Chen and Liam Merino and Dhruva Dixith Kurra and Min Cai and Tom Lampo and Yu Yin and Danhua Guo and Burhan Yaman},
      year={2026},
      eprint={2608.23486},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2608.23486},
}