Anqi Li*
Zhiyong Wang*
Jiazhao Zhang*
Minghan Li
Yunpeng Qi
Zhibo Chen
Zhizheng Zhang†
He Wang†
Peking University Galbot USTC BAAI
This repository is the official implementation of ICRA 2026 contributed paper UrbanVLA: A Vision-Language-Action Model for Urban Micromobility. It contains the data generation and post-processing pipeline, and benchmark evaluation pipeline. In the future, we will also release the model architecture and training code, as well as the checkpoint. Please stay tuned!
We plan to further implement the following items:
- Sekai data postprocessing instructions.
- UrbanVLA architecture and training code.
- UrbanVLA model checkpoint.
First, clone the repository and setup the python environment:
git clone https://github.com/GalaxyGeneralRobotics/UrbanVLA.git
cd UrbanVLA
conda create -n urbanvla python=3.9
conda activate urbanvla
# First, install metaurban dependencies.
pip install -e .
conda install pybind11 -c conda-forge
cd metaurban/orca_algo && rm -rf build
bash compile.sh && cd ../..
pip install stable_baselines3 imitation tensorboard wandb scikit-image pyyaml gdown
# Ensure image on cuda
pip install cupy-cuda12x # also cupy-cuda11x, depending on cuda env.
pip install PyOpenGL PyOpenGL_accelerate pycuda panda3d cuda-python
# For UrbanVLA model inference
pip install transformers==4.42.4 timm==0.6.13 fairscale==0.4.13 decord==0.6.0 accelerate==0.34.2UrbanVLA is distributed as a source repository rather than a standalone wheel.
The ORCA extension is compiled locally by the commands above. Renderer code and
resources under metaurban/render_pipeline are included in Git; pull_asset
downloads the static-object and pedestrian asset packs used by the simulator.
Then, pull assets, verify installation and headless environment, and add current folder to your python path.
# Pull metaurban assets
python -m metaurban.pull_asset
# Verify installation
python -m metaurban.tests.test_env.profile_metaurban
# Verify headless environment
python -m metaurban.tests.test_env.verify_headless_env
export PYTHONPATH=$PYTHONPATH:~/path/to/UrbanVLA- pycuda installation failure:
src/cpp/cuda.hpp:14:10: fatal error: cuda.h: No such file or directorySolution:export PATH=/usr/local/cuda/bin:$PATH export CUDA_ROOT=/usr/local/cuda
- You can also refer to MetaUrban for guidance on installation.
A unified CLI dispatches to the A* collector or the RL expert collector. The
new --view_mode flag selects between the historical 4-camera panorama and a
single front-view layout suitable for single-view VLA backbones:
python scripts/collect.py \
--policy astar --env dynamic --view_mode front \
--gpu 0 \
--start_seed 0 --goal 100 --start_episode 0 --name my_urbanvla_dataUse --policy rl for the PPO expert; it loads
benchmark/policy/weights/metaurban_ppo/pretrained_policy_576k.zip by default,
or pass --expert_policy_path to override it.
Two-pass pipelines that compute the strict 99th-percentile of future-8-step
relative motion (dx, dy, dyaw) across the whole dataset, print those
three constants, and normalize all supervision by them:
# A* trajectories
python -m data_postprocess.metaurban.astar.post_process \
--dataset_dir ./mydata/my_urbanvla_data \
--name my_urbanvla_data \
--start 1 --end 3001
# RL-expert trajectories (run sample_frames + clean_images first)
python -m data_postprocess.metaurban.rl_expert.sample_frames --dataset_dir ./mydata/my_urbanvla_data
python -m data_postprocess.metaurban.rl_expert.clean_images --dataset_dir ./mydata/my_urbanvla_data
python -m data_postprocess.metaurban.rl_expert.post_process --dataset_dir ./mydata/my_urbanvla_data --name my_urbanvla_data
# Sekai prompts
python -m data_postprocess.sekai.cache_noisy_traj --traj_in <...>.json --traj_out <...>.json
python -m data_postprocess.sekai.get_correct_traj --sekai_in <...>.json --traj_in <...>.json --output <...>.jsonThe [norm] banner emitted at the end of post-processing looks like:
==============================================================================
[norm] A* post-processing: 99th-percentile of future-8-step relative motion
[norm] NORM_SCALE_X = 4.231087
[norm] NORM_SCALE_Y = 1.872430
[norm] NORM_SCALE_THETA = 0.873198
[norm] Copy these values into your NavigationPolicy subclass.
==============================================================================
Important. The eval pipeline multiplies every model output by the policy's
class-attribute NORM_SCALE_X / Y / THETA. Those constants MUST equal the
numbers printed above for the dataset the model was trained on; mismatched
scales silently corrupt eval metrics.
Smoke-test the PointNav pipeline without model weights using the DummyAgent:
MAX_EPISODES=1 NO_VISUALIZATION=1 bash benchmark/eval.sh pointnav-test 0Run the bundled MetaUrban PPO expert:
POLICY=metaurban_ppo NO_VISUALIZATION=1 bash benchmark/eval.sh pointnav-test 0Each full configuration contains 1000 test or 100 unseen episodes in total,
balanced across X/C/S. Set SUBSET=0.1 for exactly 100 test or 10 unseen
episodes.
Aggregate results into a Markdown table:
python benchmark/aggregate_results.py \
--save_tag metaurban_ppo --config pointnav-test --wp 2Full details for the four PointNav/SocialNav configurations across X/C/S are in
evaluation.md.
Subclass NavigationPolicy and register it. The eval pipeline does the rest.
# benchmark/policy/my_model.py (the module name matches --policy)
import numpy as np
from . import NavigationPolicy, register_policy
@register_policy("my_model")
class MyPolicy(NavigationPolicy):
requires_panorama = False # set True for 4-camera input
NORM_SCALE_X = 4.231 # match post-processing's printed values
NORM_SCALE_Y = 1.872
NORM_SCALE_THETA = 0.873
def __init__(self, model_path: str, **_):
self.model = ... # load your weights
def reset(self): ...
def predict(self, observations, info):
rgb = observations["front"] # (H, W, 3) uint8
# info has keys: traj_to_go, next_action, duration, ego_state
return [[dx, dy, dtheta] for ...] # 8 normalized local-frame stepsRun with --policy my_model; the module is imported lazily.
UrbanVLA/
├── benchmark/ evaluation pipeline + per-policy adapters
│ ├── policy/ NavigationPolicy ABC, dummy, metaurban_ppo
│ ├── metaurban_eval.py
│ ├── eval.sh, kill_metaurban_eval.sh
│ └── aggregate_results.py
├── scripts/ data collection (astar / rl) + 2D occupancy
│ ├── collect.py unified CLI dispatcher
│ ├── astar_collect.py
│ ├── rl_expert_collect.py
│ ├── occupancy/
│ │ └── occupancy_map_2d.py
│ ├── visualization/
│ │ └── image_utils.py
│ └── _collect_utils.py shared camera capture (panorama/front)
├── data_postprocess/ post-processing pipelines
│ ├── common/ traj_noise + 99th-percentile normalization + tests
│ ├── metaurban/{astar,rl_expert}/
│ └── sekai/
├── metaurban/ simulator + packaged engine configuration
├── config/ YAML configs for data collection / evaluation
├── setup.py
├── evaluation.md benchmark spec + how to run + aggregator
├── LICENSE
└── assets/ teaser, pipeline diagram, benchmark table, setup
If you find our work helpful, please consider citing the following BibTeX entry.
@inproceedings{li2025urbanvla,
title = {UrbanVLA: A Vision-Language-Action Model for Urban Micromobility},
author = {Li, Anqi and Wang, Zhiyong and Zhang, Jiazhao and Li, Minghan
and Qi, Yunpeng and Chen, Zhibo and Zhang, Zhizheng and Wang, He},
booktitle = {IEEE International Conference on Robotics and Automation (ICRA)},
year = {2026}
}This repository is released under the Apache License 2.0 — see
LICENSE. Third-party components retain their respective upstream
licenses where included.
- This repo is built on top of MetaUrban.
- A* path-planning modules contributed by the MM-Nav team (arXiv:2510.03142).