daniel-code/TubeViT
daniel-code/TubeViT has 95 GitHub stars and 9 forks. GitHot measured 0 stars in 24 hours. Last checked 2026-11-18 22:40 UTC. Last push 2026-07-15.
측정된 스타 증가
- in 24h
- 0
- in 7 days
- 기록 수집 중
- in 15 days
- 기록 수집 중
- in 1 month
- 기록 수집 중
서로 겹치는 기간의 순증가량이며 일별 합계가 아닙니다. 측정값이 없으면 비워 둡니다. 스타는 관심도를 나타내며 품질을 보장하지 않습니다.
소개
An unofficial implementation of TubeViT in "Rethinking Video ViTs: Sparse Video Tubes for Joint Image and Video Learning"
머신러닝README.md
OriginalTubeViT
An unofficial implementation of TubeViT in "Rethinking Video ViTs: Sparse Video Tubes for Joint Image and Video Learning"
Spec.
- Fixed Positional embedding
- Sparse Tube Construction
- Multi-Tube
- Interpolated Kernels
- Space To Depth
- config of tubes
- pipeline
- training
- evaluating
- inference
Usage
This project is based on torch==1.13.1 and pytorch-lightning
Setup
Install requirements
pip install -r requirements.txtDownload UFC101 dataset
Convert ViT pre-trained weight
Use convert_vit_weight.py to convert torch ViT pre-trained weight to TubeVit.
python scripts/convert_vit_weight.py --help ✔
Usage: convert_vit_weight.py [OPTIONS]
Options:
-nc, --num-classes INTEGER num of classes of dataset.
-f, --frames-per-clip INTEGER frame per clip.
-v, --video-size <INTEGER INTEGER>...
frame per clip.
-o, --output-path PATH output model weight name.
--help Show this message and exit.
Example
Convert ImageNet pre-trained weight to UCF101. --num-classes is 101 by default.
python scripts/convert_vit_weight.py
Train
Current train.py only train on pytorch UCF101 dataset.
Change the dataset if needed.
--dataset-root and --annotation-path is based
on torchvision.datasets.UCF101
python scripts/train.py --help
Usage: train.py [OPTIONS]
Options:
-r, --dataset-root PATH path to dataset. [required]
-a, --annotation-path PATH path to dataset. [required]
-nc, --num-classes INTEGER num of classes of dataset.
-b, --batch-size INTEGER batch size.
-f, --frames-per-clip INTEGER frame per clip.
-v, --video-size <INTEGER INTEGER>...
frame per clip.
--max-epochs INTEGER max epochs.
--num-workers INTEGER
--fast-dev-run
--seed INTEGER random seed.
--preview-video Show input video
--help Show this message and exit.
Examples
python scripts/train.py -r path/to/dataset -a path/to/annotation
Evaluation
python scripts/evaluate.py --help
Usage: evaluate.py [OPTIONS]
Options:
-r, --dataset-root PATH path to dataset. [required]
-m, --model-path PATH path to model weight. [required]
-a, --annotation-path PATH path to dataset. [required]
--label-path PATH path to classInd.txt. [required]
-nc, --num-classes INTEGER num of classes of dataset.
-b, --batch-size INTEGER batch size.
-f, --frames-per-clip INTEGER frame per clip.
-v, --video-size <INTEGER INTEGER>...
frame per clip.
--num-workers INTEGER
--seed INTEGER random seed.
--verbose Show input video
--help Show this message and exit.
Examples
python scripts/evaluate.py -r path/to/dataset -a path/to/annotation
Model Architecture

Positional embedding
