Skip to content

Reference

Commands

Command What it does
urdf2vla doctor [--offline] Check the machine: GPU driver, Docker and the NVIDIA Container Toolkit, images and containers, disk, NVIDIA's asset server. Changes nothing; each problem comes with its fix.
urdf2vla init DIR --urdf URDF Create a project for a URDF. --from examples/<robot>/project.yaml starts from an example instead.
urdf2vla <skill> --project DIR [options] Run one step (see below). urdf2vla <skill> --help lists its options.
urdf2vla status --project DIR Each step's last result and what to run next.
urdf2vla answer --project DIR key=value … Record answers to questions in answers.yaml. Values are YAML: 'robot.default_pose={j1: 0.0, j2: 0.3}'.
urdf2vla show --project DIR <name> Print a generated file, e.g. robot_metadata, robot_config.
urdf2vla diagnose --project DIR Match the last failed step's log against known problems.
urdf2vla view --project DIR [--port 8080] The browser viewer. Needs .[view].
urdf2vla teleop-keys [--port 50555] Send terminal key presses to collect_dataset --device socket.
urdf2vla skills [--json] The steps with their inputs, outputs and prerequisites (machine-readable with --json).

Steps (skills)

Step What it does
inspect_robot Reads the URDF: links and joints, the arm, the gripper (including mimic joints), hand-link candidates, mesh files. Asks about anything ambiguous.
configure_robot Builds the robot configuration from the URDF and your answers, and writes a normalized URDF (package:// resolved, empty root link removed).
configure_task Chooses the objects and the container (task.objects, task.container) and places the table, the spawn area and the bin from the arm's reach and shoulder height.
configure_camera Places the cameras: DROID layout by default (wrist, left, right), or front / side presets, or explicit cameras.
import_robot Converts the URDF to USD (attaching a SimReady gripper if configured) and runs the robot checks. Proposes an IK starting pose if the hand doesn't point down.
launch_sim Builds the pick-and-place environment with Isaac Lab Arena, saves camera snapshots, checks cameras, scene and a test grasp. --livestream keeps it running for a WebRTC client.
configure_vla_interface Defines the VLA's inputs (camera images, arm joints + gripper value) and outputs (joint targets + gripper command).
setup_teleoperation Configures hand-space control through inverse kinematics and checks the hand moves as commanded.
collect_dataset Records successful demonstrations (--device scripted, keyboard or socket) to HDF5 and checks them.
generate_mimic Optional, instead of collect_dataset: records a few scripted source demonstrations, has Isaac Lab Mimic generate --trials new episodes from them, and records the successful ones with images (one object and a container).
convert_dataset Converts the recording to LeRobot v2.1 with GR00T N1.7's modality files and checks it with GR00T's loader.
train_vla Writes and checks a GR00T N1.7 fine-tuning job; trains if the GPU has 44 GB or more, otherwise packages the job.
evaluate_vla Runs episodes with a GR00T inference server (--model) or a dataset replay (--replay) and reports the success rate.

Where each step runs, what it needs first, what it reads and writes, and its options (host = your machine, sim = the simulation container, gr00t = the GR00T container):

Skill Runs in Needs Reads Writes Options
inspect_robot host — project.yaml: robot.urdf generated/robot_metadata.yaml —
configure_robot host inspect_robot generated/robot_metadata.yaml
answers.yaml
project.yaml: robot.*
generated/robot_config.yaml
build/robot.urdf
—
configure_task host configure_robot generated/robot_config.yaml
project.yaml: task.*
generated/task.yaml —
configure_camera host configure_task generated/robot_config.yaml
generated/task.yaml
project.yaml: cameras
answers.yaml: cameras
generated/cameras.yaml --preset
import_robot sim configure_robot generated/robot_config.yaml
build/robot.urdf
build/usd/robot.usd
build/import_report.yaml
--force
--timeout
launch_sim sim import_robot
configure_camera
generated/robot_config.yaml
generated/cameras.yaml
generated/task.yaml
build/usd/robot.usd
snapshots/*.png
build/launch_report.yaml
--steps
--livestream
--timeout
configure_vla_interface host launch_sim generated/robot_config.yaml
generated/cameras.yaml
generated/task.yaml
generated/vla_interface.yaml --vla-type
setup_teleoperation sim configure_vla_interface generated/robot_config.yaml
generated/vla_interface.yaml
generated/teleop.yaml
build/teleop_report.yaml
--timeout
collect_dataset sim setup_teleoperation generated/teleop.yaml
generated/vla_interface.yaml
datasets/<name>.hdf5
build/collect_report.yaml
--device
--source
--num-demos
--max-attempts
--dataset
--seed
--livestream
--timeout
convert_dataset gr00t collect_dataset datasets/<name>.hdf5 datasets/<name>_gr00t/ or datasets/<name>_droid_gr00t/
build/convert_report.yaml or build/convert_droid_report.yaml
--dataset
--force
--layout
--exterior
--timeout
train_vla gr00t convert_dataset datasets/<name>_gr00t/ training/<job>/train.sh
training/<job>.tar.gz or training/<job>/checkpoints/
--dataset
--job
--max-steps
--batch-size
--save-steps
--save-total-limit
--base-model
--check-only
--force-local
--timeout
evaluate_vla host setup_teleoperation --model <checkpoint> or --replay datasets/<name>_gr00t/
generated/vla_interface.yaml
build/eval_report.yaml
snapshots/eval_*.png
build/eval_videos/*.mp4 (--video)
--model
--replay
--dataset
--episodes
--execution-horizon
--port
--seed
--timeout
--droid
--exterior
--video
generate_mimic host setup_teleoperation generated/teleop.yaml
generated/vla_interface.yaml
datasets/<name>_mimic_src.hdf5
datasets/<name>_mimic_gen.hdf5
datasets/<name>.hdf5
--source-demos
--source-device
--trials
--dataset
--seed
--timeout

The result of every step

Every step prints one JSON object and saves it in logs/latest_<step>.json:

Field Meaning
status ok, needs_input or error (exit code 0, 2 or 1)
next The step(s) to run next
questions With needs_input: key, prompt, candidates, suggestion, reason
inferred Values derived from the URDF that you may want to confirm
warnings Things worth knowing that didn't stop the step
errors Why the step failed
outputs Files written (relative to the project)
data The step's main results, including checks
log The full log file

project.yaml

name: myrobot

robot:
  urdf: /path/to/robot.urdf            # required
  package_paths: [../my_ws/src]        # extra folders to resolve package:// paths
  end_effector: tool0                  # the hand link (otherwise asked if ambiguous)
  control_joints: [joint1, joint2]     # arm joints (otherwise detected)
  gripper_joints: [finger_l, finger_r] # finger joints (otherwise detected)
  gripper_open_position: upper         # fingers open at the lower or upper joint limit
  default_pose: {joint1: 0.0}          # starting pose [rad or m]
  effort_limits: {"joint[1-6]": 50.0}  # N·m; joint-name regex → value, overrides the URDF
  base_pose: {pos: [0, 0, 0], rot_xyzw: [0, 0, 0, 1]}
  gripper: {simready: robotiq_2f_85}   # attach a gripper from the Isaac Sim asset library (robotiq_2f_85, ufactory_xarm); + mount_link to choose the frame on an arm without a gripper (optional)

task:
  objects: [apple, lime, pear, color_cube]   # one per episode; see the object list in the "Bring your own robot" guide
  container: klt_bin                   # or plastic_tray, or "disc" (a flat green target)

cameras:                               # presets: wrist, left, right, front, side — or explicit
  - preset: wrist
  - preset: left
  - preset: right

overrides:                             # same structure as the files in generated/
  robot: {}                            #   generated/robot_config.yaml
  task: {}                             #   generated/task.yaml
  cameras: {}                          #   generated/cameras.yaml
  teleop: {}                           #   generated/teleop.yaml

Frequently used overrides:

Key Default Effect
overrides.robot.asset.solver_position_iterations 32 (64 with a SimReady gripper) Contact accuracy; raise when fingers sink into objects
overrides.task.instruction "put the {object} in the {target}" The instruction; {object} and {target} become the episode's object and the container
overrides.task.spawn_x / .spawn_y from the arm's reach Where an episode puts its object
overrides.task.target.pos beside the spawn area Where the bin stands
overrides.task.success.container_margin 0.01 m How far inside the bin's rim the object must end up
overrides.task.episode_length_s 30 s Episode timeout
overrides.task.render_fix false Render once more per step if a camera freezes
overrides.teleop.pos_step / .rot_step 0.008 m / 0.04 rad Keyboard step size per control step
overrides.teleop.grasp_height_offset 0 m Scripted grasp height relative to the object's center

Project folder

projects/<name>/
  project.yaml      your settings            answers.yaml   your answers
  generated/        robot_metadata, robot_config, task, cameras, vla_interface, teleop (do not edit)
  build/            robot.urdf, usd/, meshes/, *_report.yaml, eval_videos/
  snapshots/        camera images from launch_sim, collect_dataset and evaluate_vla
  datasets/         <name>.hdf5 recordings, <name>_gr00t/ converted data
  training/         <job>/train.sh, <job>.tar.gz, <job>/checkpoints/
  logs/             one log and one JSON result per run; latest_<step>.json
  state.json        the last status of every step

runtime.yaml

Where the steps run. The defaults match docker/build.sh and docker/run.sh.

Key Default Meaning
mode docker docker, or local to run in the current Python environment (inside an Isaac Lab installation)
container / image urdf2vla-sim / urdf2vla/isaaclab_arena:b9e37c5 The simulation container
python …/isaaclab.sh -p How Python is started in that container
mounts the repository at /workspace/urdf2vla Host folders visible in the containers
env — Extra environment variables
gr00t.container / gr00t.image urdf2vla-gr00t / urdf2vla/gr00t:23ace64 The GR00T container

Checks

Step Checks
import_robot articulation_root, all_joints_driven, mimic_constraints_off, joint_name_match, bodies_exist, no_nan, default_pose_tracking, within_limits, gripper_motion:<gripper>, joint_limits_hold:<gripper>
launch_sim camera_alive:<camera>, camera_not_black:<camera>, camera_mount:<camera>, no_nan, and per object: object_on_table:<object>, asset_size:<object>, grasp_hold:<gripper>:<object>, grasp_opening:<gripper>:<object> (box-shaped objects); asset_size:<container>
setup_teleoperation ik_direction:<arm>:+x, ik_direction:<arm>:+z, ik_direction:<arm>:-y, action_label:<arm>
collect_dataset camera_frames_updated, first_frame_fresh, label_replay
convert_dataset gr00t_loader, video_matches:<camera>
train_vla finetune_args_valid
evaluate_vla camera_frames_updated

What a failure means and what to change: Troubleshooting.