Oneflow-Inc/libai
Oneflow-Inc/libai has 404 GitHub stars and 57 forks. GitHot measured 0 stars in 24 hours and 0 in 7 days. Last checked 2026-11-20 22:04 UTC. Last push 2025-07-31.
Crecimiento de estrellas medido
- in 24h
- 0
- in 7 days
- 0
- in 15 days
- Recopilando historial
- in 1 month
- Recopilando historial
Cambios netos en períodos que se solapan, no totales diarios. Los datos no disponibles se dejan en blanco. Las estrellas indican interés, no calidad.
Acerca de
LiBai(李白): A Toolbox for Large-Scale Distributed Parallel Training
Aprendizaje automáticoREADME.md
OriginalLiBai
Introduction
English | 简体中文
LiBai is a large-scale open-source model training toolbox based on OneFlow. The main branch works with OneFlow 0.7.0.
Highlights
Support a collection of parallel training components
LiBai provides multiple parallelisms such as Data Parallelism, Tensor Parallelism, and Pipeline Parallelism. It's also extensible for other new parallelisms.
Varied training techniques
LiBai provides many out-of-the-box training techniques such as Distributed Training, Mixed Precision Training, Activation Checkpointing, Recomputation, Gradient Accumulation, and Zero Redundancy Optimizer(ZeRO).
Support for both CV and NLP tasks
LiBai has predefined data process for both CV and NLP datasets such as CIFAR, ImageNet, and BERT Dataset.
Easy to use
LiBai's components are designed to be modular for easier usage as follows:
- LazyConfig system for more flexible syntax and no predefined structures
- Friendly trainer and engine
- Used as a library to support building research projects on it. See projects/ for some projects that are built based on LiBai
High Efficiency
Installation
See Installation instructions.
Getting Started
See Quick Run for the basic usage of LiBai.
Documentation
See LiBai's documentation for full API documentation and tutorials.
ChangeLog
Beta 0.3.0 was released in 03/11/2024, the general changes in 0.3.0 version are as follows:
Features:
- Support mock transformers, see Mock transformers
- Support lm-evaluation-harness for model evaluation
- User Experience Optimization
New Supported Models:
- These models are natively supported by libai
| Models | 2D(tp+pp) Inference | 3D Parallel Training |
|---|---|---|
| BLOOM | ✔ | - |
| ChatGLM | ✔ | ✔ |
| Couplets | ✔ | ✔ |
| DALLE2 | ✔ | - |
| Llama2 | ✔ | ✔ |
| MAE | ✔ | ✔ |
| Stable_Diffusion | - | - |
New Mock Models:
- These models are extended and implemented by libai through mocking transformers.
| Models | Tensor Parallel | Pipeline Parallel |
|---|---|---|
| BLOOM | ✔ | - |
| GPT2 | ✔ | - |
| LLAMA | ✔ | - |
| LLAMA2 | ✔ | - |
| Baichuan | ✔ | - |
| OPT | ✔ | - |
See changelog for details and release history.
Contributing
We appreciate all contributions to improve LiBai. See CONTRIBUTING for the contributing guideline.
License
This project is released under the Apache 2.0 license.
Citation
If you find this project useful for your research, consider cite:
@misc{of2021libai,
author = {Xingyu Liao and Peng Cheng and Tianhe Ren and Depeng Liang and
Kai Dang and Yi Wang and Xiaoyu Xu},
title = {LiBai},
howpublished = {\url{https://github.com/Oneflow-Inc/libai}},
year = {2021}
}
Join the WeChat group
