Skip to content
Projects

VladimirYugay/CalfVO has 89 GitHub stars and 1 forks. GitHot measured 0 stars in 24 hours. Last checked 2026-11-15 16:57 UTC. Last push 2026-10-12.

How these counts are measured

Measured star growth

in 24h
0
in 7 days
Update delayed
in 15 days
Update delayed
in 1 month
Update delayed

Net star changes over overlapping periods, not daily totals. Missing measurements are left blank. Stars measure attention, not software quality.

About

Monocular Visual Odometry without Calibration or Test-time Optimization

Creative & Media

README.md

Original

CalfVO: Monocular Visual Odometry without Calibration or Test-time Optimization

Vladimir Yugay* · Duy-Kien Nguyen* · Theo Gevers · Cees G. M. Snoek · Martin R. Oswald

*Equal contribution  ·  University of Amsterdam

Project Page | arXiv

⚙️ Setting Things Up

Clone the repo:

git clone https://github.com/VladimirYugay/CalfVO.git

Make sure that nvcc --version matches nvidia-smi version. The optional RoPE2D CUDA kernel is compiled against it.

Run the following commands to set up the environment

conda create -n calfvo python=3.10
conda activate calfvo

Install pytorch. We trained and evaluated with the CUDA 12.1 build and have not tried the other two:

# CUDA 11.8
pip install torch==2.5.1 torchvision==0.20.1 --index-url https://download.pytorch.org/whl/cu118
# CUDA 12.1  (tested)
pip install torch==2.5.1 torchvision==0.20.1 --index-url https://download.pytorch.org/whl/cu121
# CUDA 12.4
pip install torch==2.5.1 torchvision==0.20.1 --index-url https://download.pytorch.org/whl/cu124

Install FFmpeg, capped below version 8 (torchcodec only supports FFmpeg 4-7):

conda install -c conda-forge "ffmpeg<8"

Skipping this fails at the first import torchcodec with Could not load libtorchcodec.

Install other dependencies:

pip install -r requirements.txt

Fetch CroCo and DUSt3R. This clones naver/croco at 743ee71a and naver/dust3r at c9e9336a, then patches one file in croco so it can be imported as a package. --with-curope also builds the CUDA RoPE2D kernel. Without it CroCo uses a pure PyTorch RoPE that gives the same answer and runs slower.

bash scripts/setup_dependencies.sh --with-curope

We tested our code on A100-SXM4-40GB GPUs with Ubuntu 22.

🔨 Running CalfVO

Getting the Data Every dataset is stored the same way, one directory per sequence holding an all-intra video.mp4, a poses.npz of camera-to-world matrices, and an intrinsics.npz. Frame i of the video and pose i are the same instant.

Every script below lives in scripts/data/.

Corpus Role Getting the raw data Preparing it
TartanAir v2 train (45) process_tartanv2.py fetches it from HuggingFace process_tartanv2.py
TartanAir v1 test process_tartanv1.py fetches it from HuggingFace process_tartanv1.py
ScanNet train (40) + test download_scannet.py, after accepting the ScanNet terms of use process_scannet.py
TUM RGB-D test download_tum.sh process_tum.py
EuRoC MAV test download_euroc.sh process_euroc.py
KITTI odometry train 00–08 (20) + test 09–10 manual, the odometry grayscale/colour set from cvlibs.net process_kitti.py
Virtual KITTI 2 train (20) manual, the vkitti_2.0.3_rgb and vkitti_2.0.3_textgt tarballs from Naver Labs process_vkitti.py

The two TartanAir scripts need no separate download. They pull one zip at a time from HuggingFace, convert it, and delete it before moving on, so you need room for one archive rather than the whole corpus.

Training
torchrun --nnodes=1 --nproc_per_node=16 train.py \
    --config_path configs/calfvo_croco.yaml

The schedule and the data mixture live in the config, and the paper explains the choices. Runs log to wandb. To keep them on disk instead, set logging.offline: True in the config or start the run with WANDB_MODE=offline.

Checkpoints Two trained models live on HuggingFace, one per backbone.

Config Checkpoint Encoder Licence
configs/calfvo_croco.yaml calfvo_croco.pth CroCo v2, as distributed with DUSt3R CC BY-NC-SA 4.0
configs/calfvo_dinov2.yaml calfvo_dinov2.pth DINOv2 ViT-L/14 Apache-2.0

The paper reports calfvo_croco.pth. Its encoder weights are non-commercial, so use calfvo_dinov2.pth if that matters to you. It is the weaker of the two on most metrics.

Evaluation

Scoring our released model takes nothing but the config. The checkpoint for its backbone is fetched from HuggingFace and cached, so there is no download step.

torchrun --nnodes=1 --nproc_per_node=16 test.py \
    --config_path configs/calfvo_croco.yaml

This scores every sequence in test_datasets and writes per-scene metrics and trajectories, plus one summary per corpus.

Give --checkpoint_path a run directory to score a model you trained yourself, and it reads that run's own config:

torchrun --nnodes=1 --nproc_per_node=16 test.py \
    --checkpoint_path /path/to/output/calfvo/calfvo_croco
Demo Point it at any video and it writes the trajectory. Nothing needs preparing, and it does not need ground truth:
python scripts/run_vo_demo.py \
    --config configs/calfvo_croco.yaml \
    --video /path/to/clip.mp4 \
    --output_dir output/demo/vo

The checkpoint comes from HuggingFace here too, the released one for the config's backbone. Pass --checkpoint to run a model of your own instead.

It writes the trajectory where the visualiser can read it, and prints the command to view it:

python scripts/visualize_trajectory.py \
    --trajectory_dir output/demo/vo \
    --video /path/to/clip.mp4 --output_file out/clip.rrd

That needs pip install rerun-sdk, which the training environment does not include.

Training on Your Own Data CalfVO is a neural network, not a foundation model and not an optimisation pipeline. It predicts what it was trained on, so on a camera or a motion unlike the corpora above it is better to train on your own recordings.

Store your sequences the way the scripts in scripts/data/ store theirs, one directory per sequence with a video.mp4 and a poses.npz, then add an entry for it under training_datasets.

  • Frames go into a video to limit the number of files.
  • They are stored as 288 px squares, which is the crop training reads anyway, so less is decoded per frame.
  • Every frame is a keyframe. Training reads windows from random places in a sequence, and in an ordinary video each seek decodes forward from the last keyframe.

Set pose_stats to match your data. Translations are divided by these before the loss sees them. The shipped numbers do not come from any one corpus, they were picked to work across all of them. If your motion has a different scale, take the mean and the standard deviation of your own frame to frame translations and use those.

Results may differ slightly between runs because training samples data at random.

📌 Citation

If you find our paper and code useful, please cite us:

@misc{yugay2026calfvo,
      title={Monocular Visual Odometry without Calibration or Test-time Optimization},
      author={Vladimir Yugay and Duy-Kien Nguyen and Theo Gevers and Cees G. M. Snoek and Martin R. Oswald},
      year={2026},
      eprint={2510.03348},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2510.03348},
}