REFINE-DP

REFINE-DP is a full pipeline for learning whole-body humanoid manipulation policies — from VR teleoperation in simulation, to diffusion policy pre-training, to task-specific reinforcement fine-tuning, to real robot deployment.

We demonstrate the approach on the Booster T1 humanoid (29-DOF) across tasks including door pushing, box pick-and-place, and long-horizon stair navigation with object carrying.

📄 Paper | Project Page

---

Overview

Training a humanoid to perform whole-body manipulation is hard — it requires coordinating locomotion and arm control simultaneously over long horizons. REFINE-DP tackles this with a three-stage approach:

1. Teleoperation & Data Collection — Collect demonstrations in Isaac Sim using a Meta Quest headset and XRoboToolkit. A heuristic planner can also generate data autonomously.
2. Diffusion Policy Pre-training — Pre-train a transformer-based diffusion policy on the collected demonstrations across diverse tasks.
3. DPPO Fine-tuning — Fine-tune the pre-trained policy in simulation using Diffusion Policy Optimization (DPPO), a PPO-based RL algorithm tailored for diffusion models.

The result is a policy that can be deployed directly to a real T1 robot.

Supported Tasks

Task	Description
T1-Data-Door-v0	Push open a door while walking through
T1-Data-Box-v0	Pick up and place a box
T1-Data-Box-Long-v0	Long-horizon box manipulation
T1-Data-BoxStair-v0	Carry a box up stairs

---

Installation

See diffusion_humanoid/README_DP_Isaac.md

---

Usage

Step 1 — Collect Data

Collect teleoperated demonstrations (requires Meta Quest + XRoboToolkit), or use the autonomous heuristic planner:

# With Meta Quest VR controller
python t1_dp_tasks/scripts/teleop_full_body.py \
    --task T1-Data-Door-v0 \
    --leg_task Isaac-Foot-Track-T1-Velocity-To-Foot-Track-Play-v0 \
    --leg_checkpoint <path/to/loco_ckpt.pt> \
    --arm_task FK-Tracking-T1-Play-v0 \
    --arm_checkpoint <path/to/arm_ckpt.pt> \
    --num_envs 1 \
    --total_episodes_to_collect 20 \
    --enable_cameras

# With autonomous heuristic planner (no VR needed)
python t1_dp_tasks/scripts/teleop_full_body.py \
    --task T1-Data-Door-v0 \
    --autonomous \
    ...

Collected episodes are saved in HDF5 format. To combine multiple datasets, use diffusion_humanoid/data_cleaning_visualization_zarr_preparation/convert_hdf5_to_zarr.ipynb.

Step 2 — Pre-train

cd diffusion_humanoid
python scripts/train_t1.py \
    --config-name=train_diffusion_transformer_lowdim_workspace.yaml \
    task=t1_task2_fullbody

Task configs live in diffusion_humanoid/diffusion_policy/config_files/task/. Update the dataset path in your config before running.

Step 3 — Fine-tune

cd diffusion_humanoid
python scripts/ft.py --config-name=ft_T1_task2

Update ft_T1_task2.yaml with your pre-trained checkpoint path and low-level policy checkpoints before running.

Step 4 — Evaluate

cd diffusion_humanoid
python scripts/eval_t1.py --config-name=eval_T1_box

---

Repository Structure

REFINE-DP/
├── IsaacLab/                    # Copy of Isaac Lab
├── diffusion_humanoid/
│   ├── scripts/                 # Train, fine-tune, evaluate entry points
│   ├── diffusion_policy/        # Diffusion model architectures and training workspaces
│   │   └── config_files/        # Hydra YAML configs for all tasks
│   ├── data_cleaning_visualization_zarr_preparation/  # Dataset preprocessing notebooks
│   └── sim2real/                # T1 deployment bridges
└── t1_dp_tasks/
    ├── scripts/                 # Teleoperation, replay, visualization scripts
    └── source/t1_dp_locomotion_manager_based/
        └── tasks/manager_based/ # Isaac Lab task definitions per scenario

---

Real Robot Deployment

Sim2real bridges are provided for:

• Booster T1 via booster_robotics_sdk

We recommend validating in sim-to-sim (MuJoCo) before deploying to hardware. See diffusion_humanoid/sim2real/ for deployment scripts.

---

Citation

If you find this work useful, please cite:

@misc{gu2026refinedpdiffusionpolicyfinetuning,
      title={REFINE-DP: Diffusion Policy Fine-tuning for Humanoid Loco-manipulation via Reinforcement Learning}, 
      author={Zhaoyuan Gu and Yipu Chen and Zimeng Chai and Alfred Cueva and Thong Nguyen and Yifan Wu and Huishu Xue and Minji Kim and Isaac Legene and Fukang Liu and Matthew Kim and Ayan Barula and Yongxin Chen and Ye Zhao},
      year={2026},
      url={https://arxiv.org/abs/2603.13707}, 
}

---

Acknowledgements

This work builds on IsaacLab, Diffusion Policy, and DPPO.