daniel-code/TubeViT
daniel-code/TubeViT has 95 GitHub stars and 9 forks. GitHot measured 0 stars in 24 hours. Last checked 2026-11-18 22:40 UTC. Last push 2026-07-15.
Crecimiento de estrellas medido
- in 24h
- 0
- in 7 days
- Recopilando historial
- in 15 days
- Recopilando historial
- in 1 month
- Recopilando historial
Cambios netos en períodos que se solapan, no totales diarios. Los datos no disponibles se dejan en blanco. Las estrellas indican interés, no calidad.
Acerca de
An unofficial implementation of TubeViT in "Rethinking Video ViTs: Sparse Video Tubes for Joint Image and Video Learning"
Aprendizaje automáticoREADME.md
OriginalTubeViT
An unofficial implementation of TubeViT in "Rethinking Video ViTs: Sparse Video Tubes for Joint Image and Video Learning"
Spec.
- Fixed Positional embedding
- Sparse Tube Construction
- Multi-Tube
- Interpolated Kernels
- Space To Depth
- config of tubes
- pipeline
- training
- evaluating
- inference
Usage
This project is based on torch==1.13.1 and pytorch-lightning
Setup
Install requirements
pip install -r requirements.txtDownload UFC101 dataset
Convert ViT pre-trained weight
Use convert_vit_weight.py to convert torch ViT pre-trained weight to TubeVit.
python scripts/convert_vit_weight.py --help ✔
Usage: convert_vit_weight.py [OPTIONS]
Options:
-nc, --num-classes INTEGER num of classes of dataset.
-f, --frames-per-clip INTEGER frame per clip.
-v, --video-size <INTEGER INTEGER>...
frame per clip.
-o, --output-path PATH output model weight name.
--help Show this message and exit.
Example
Convert ImageNet pre-trained weight to UCF101. --num-classes is 101 by default.
python scripts/convert_vit_weight.py
Train
Current train.py only train on pytorch UCF101 dataset.
Change the dataset if needed.
--dataset-root and --annotation-path is based
on torchvision.datasets.UCF101
python scripts/train.py --help
Usage: train.py [OPTIONS]
Options:
-r, --dataset-root PATH path to dataset. [required]
-a, --annotation-path PATH path to dataset. [required]
-nc, --num-classes INTEGER num of classes of dataset.
-b, --batch-size INTEGER batch size.
-f, --frames-per-clip INTEGER frame per clip.
-v, --video-size <INTEGER INTEGER>...
frame per clip.
--max-epochs INTEGER max epochs.
--num-workers INTEGER
--fast-dev-run
--seed INTEGER random seed.
--preview-video Show input video
--help Show this message and exit.
Examples
python scripts/train.py -r path/to/dataset -a path/to/annotation
Evaluation
python scripts/evaluate.py --help
Usage: evaluate.py [OPTIONS]
Options:
-r, --dataset-root PATH path to dataset. [required]
-m, --model-path PATH path to model weight. [required]
-a, --annotation-path PATH path to dataset. [required]
--label-path PATH path to classInd.txt. [required]
-nc, --num-classes INTEGER num of classes of dataset.
-b, --batch-size INTEGER batch size.
-f, --frames-per-clip INTEGER frame per clip.
-v, --video-size <INTEGER INTEGER>...
frame per clip.
--num-workers INTEGER
--seed INTEGER random seed.
--verbose Show input video
--help Show this message and exit.
Examples
python scripts/evaluate.py -r path/to/dataset -a path/to/annotation
Model Architecture

Positional embedding
