Skip to content
Projekte

gruai/koifish has 29 GitHub stars and 0 forks. GitHot measured 0 stars in 24 hours, 0 in 7 days, and 0 in 15 days. Last checked 2026-11-20 08:25 UTC. Last push 2026-10-31.

So werden diese Zahlen gemessen

What you can do

  • Support both Masked diffusion language models (MDLMs) and AR models
  • Hybrid 16/8/4/2/1 bit training
  • Training Qwen3-0.6B on single 4090
  • Evolutionary optimization of experts

Gemessener Sternzuwachs

in 24h
0
in 7 days
0
in 15 days
0
in 1 month
Verlauf wird gesammelt

Nettoänderungen in überlappenden Zeiträumen, keine Tagessummen. Fehlende Messwerte bleiben leer. Sterne zeigen Interesse, nicht Qualität.

Über uns

Sparse & quantized LLM training/inference/CPT/SFT/DPO

Maschinelles Lernen

README.md

Original

Koifish

Sparse & quantized LLM training/inference/CPT/SFT/DPO.

Koifish is a lightweight & powerfule framework in C++/cu, which needs much less resource than other. It needs only sveral hours to fintue 4~32B model on single 4090. It needs only one day to train ~2B model on single 4090 as shown in the following table.

Model Parameter Loss(Baseline) GPU Memory Total Time (Training+Evaluating) Throughput Log
GPT2-124M 124M 3.293(3.425) ~6.8G ~8 hours ~140k/s log
GPT2-774M 774M 3.146(3.00) ~15G ~18 hours ~70k/s log
GPT2-1558M 1558M 3.04(2.83) ~23G ~30 hours ~50k/s log

Note

  1. The baseline results are from GPT2. Our training only used 5B tokens of FineWeb.
  2. The evaluating time depends on the frequency of testing and the sampling ratio(We use only ~10% randomly sampled tokens to reduce total time).

Features

Minimum dependencies

The inference module (cpu version) does not require any third-party libraries.

Training module needs more:

  • cudnn(would removed in future version)
  • GPUs that support FP8 computation (Compute Capability 8.9 or higher) with more than 16GB+ VRAM
  • CUDA Toolkit (12.5+)

Download & Build

# sudo apt-get -y install libcudnn9-dev-cuda-12  # maybe need this to install CUDNN
# sudo apt-get install libicu-dev
# export CPATH=/usr/local/cuda/include:$CPATH        # maybe need this to export CPATH
git clone --recurse-submodules https://github.com/gruai/koifish
cd koifish
mkdir build && cd build && cmake ..
make clean && make -j$(nproc)

Tutorial

  1. Training of QWen3_0.6B on single 4090
  2. Training of GPT2_1558M_ on single 4090
  3. Training of GPT2(774M/124M) on single 3090

Techniques/Tricks

Working plan

  • Diffusion Language Model
  • High accuracy 1-bit training/inference algorithm

Citation

Please use the following bibtex entry:

@article{chen2025eoe,
  title={EOE: Evolutionary Optimization of Experts for Training Language Models},
  author={Chen, Yingshi},
  journal={arXiv preprint arXiv:2509.24436},
  year={2025}
}

Contributing

  • Any help with managing issues, PRs and projects is very appreciated!

Acknowledgements

  • Thanks very much for the highly instructive work of qwen600, calm, llm.c & ggml.
  • Thanks very much for the support of Professor TianLan.

More

History

QQ group: 167653113