daniel-code/TubeViT
daniel-code/TubeViT has 95 GitHub stars and 9 forks. GitHot measured 0 stars in 24 hours. Last checked 2026-11-18 22:40 UTC. Last push 2026-07-15.
計測したスター増加
- 24時間
- 0
- 7日間
- 履歴を収集中
- 15日間
- 履歴を収集中
- 1か月
- 履歴を収集中
期間ごとの純増数です。期間は重複しており、日別の値ではありません。未計測の値は空欄です。スター数は注目度の指標であり、品質の保証ではありません。
概要
An unofficial implementation of TubeViT in "Rethinking Video ViTs: Sparse Video Tubes for Joint Image and Video Learning"
機械学習README.md
オリジナルTubeViT
An unofficial implementation of TubeViT in "Rethinking Video ViTs: Sparse Video Tubes for Joint Image and Video Learning"
Spec.
- Fixed Positional embedding
- Sparse Tube Construction
- Multi-Tube
- Interpolated Kernels
- Space To Depth
- config of tubes
- pipeline
- training
- evaluating
- inference
Usage
This project is based on torch==1.13.1 and pytorch-lightning
Setup
Install requirements
pip install -r requirements.txtDownload UFC101 dataset
Convert ViT pre-trained weight
Use convert_vit_weight.py to convert torch ViT pre-trained weight to TubeVit.
python scripts/convert_vit_weight.py --help ✔
Usage: convert_vit_weight.py [OPTIONS]
Options:
-nc, --num-classes INTEGER num of classes of dataset.
-f, --frames-per-clip INTEGER frame per clip.
-v, --video-size <INTEGER INTEGER>...
frame per clip.
-o, --output-path PATH output model weight name.
--help Show this message and exit.
Example
Convert ImageNet pre-trained weight to UCF101. --num-classes is 101 by default.
python scripts/convert_vit_weight.py
Train
Current train.py only train on pytorch UCF101 dataset.
Change the dataset if needed.
--dataset-root and --annotation-path is based
on torchvision.datasets.UCF101
python scripts/train.py --help
Usage: train.py [OPTIONS]
Options:
-r, --dataset-root PATH path to dataset. [required]
-a, --annotation-path PATH path to dataset. [required]
-nc, --num-classes INTEGER num of classes of dataset.
-b, --batch-size INTEGER batch size.
-f, --frames-per-clip INTEGER frame per clip.
-v, --video-size <INTEGER INTEGER>...
frame per clip.
--max-epochs INTEGER max epochs.
--num-workers INTEGER
--fast-dev-run
--seed INTEGER random seed.
--preview-video Show input video
--help Show this message and exit.
Examples
python scripts/train.py -r path/to/dataset -a path/to/annotation
Evaluation
python scripts/evaluate.py --help
Usage: evaluate.py [OPTIONS]
Options:
-r, --dataset-root PATH path to dataset. [required]
-m, --model-path PATH path to model weight. [required]
-a, --annotation-path PATH path to dataset. [required]
--label-path PATH path to classInd.txt. [required]
-nc, --num-classes INTEGER num of classes of dataset.
-b, --batch-size INTEGER batch size.
-f, --frames-per-clip INTEGER frame per clip.
-v, --video-size <INTEGER INTEGER>...
frame per clip.
--num-workers INTEGER
--seed INTEGER random seed.
--verbose Show input video
--help Show this message and exit.
Examples
python scripts/evaluate.py -r path/to/dataset -a path/to/annotation
Model Architecture

Positional embedding
