facebookresearch/fairscale
facebookresearch/fairscale has 3,402 GitHub stars and 292 forks. GitHot measured 0 stars in 24 hours, -2 in 7 days, and -2 in 15 days. Last checked 2026-11-21 15:13 UTC. Last push 2025-04-26.
측정된 스타 증가
- in 24h
- 0
- in 7 days
- -2
- in 15 days
- -2
- in 1 month
- 기록 수집 중
서로 겹치는 기간의 순증가량이며 일별 합계가 아닙니다. 측정값이 없으면 비워 둡니다. 스타는 관심도를 나타내며 품질을 보장하지 않습니다.
소개
고성능 및 대규모 학습을 위한 PyTorch 확장 기능입니다.
머신러닝README.md
Original
Description
FairScale is a PyTorch extension library for high performance and large scale training. This library extends basic PyTorch capabilities while adding new SOTA scaling techniques. FairScale makes available the latest distributed training techniques in the form of composable modules and easy to use APIs. These APIs are a fundamental part of a researcher's toolbox as they attempt to scale models with limited resources.
FairScale was designed with the following values in mind:
Usability - Users should be able to understand and use FairScale APIs with minimum cognitive overload.
Modularity - Users should be able to combine multiple FairScale APIs as part of their training loop seamlessly.
Performance - FairScale APIs provide the best performance in terms of scaling and efficiency.
Watch Introductory Video
Installation
To install FairScale, please see the following instructions. You should be able to install a package with pip or conda, or build directly from source.
Getting Started
The full documentation contains instructions for getting started, deep dives and tutorials about the various FairScale APIs.
FSDP
FullyShardedDataParallel (FSDP) is the recommended method for scaling to large NN models. This library has been upstreamed to PyTorch. The version of FSDP here is for historical references as well as for experimenting with new and crazy ideas in research of scaling techniques. Please see the following blog for how to use FairScale FSDP and how does it work.
Testing
We use circleci to test FairScale with the following PyTorch versions (with CUDA 11.2):
- the latest stable release (e.g. 1.10.0)
- the latest LTS release (e.g. 1.8.1)
- a recent nightly release (e.g. 1.11.0.dev20211101+cu111)
Please create an issue if you are having trouble with installation.
Contributors
We welcome contributions! Please see the CONTRIBUTING instructions for how you can contribute to FairScale.
License
FairScale is licensed under the BSD-3-Clause License.
fairscale.nn.pipe is forked from torchgpipe, Copyright 2019, Kakao Brain, licensed under Apache License.
fairscale.nn.model_parallel is forked from Megatron-LM, Copyright 2020, NVIDIA CORPORATION, licensed under Apache License.
fairscale.optim.adascale is forked from AdaptDL, Copyright 2020, Petuum, Inc., licensed under Apache License.
fairscale.nn.misc.flatten_params_wrapper is forked from PyTorch-Reparam-Module, Copyright 2018, Tongzhou Wang, licensed under MIT License.
Citing FairScale
If you use FairScale in your publication, please cite it by using the following BibTeX entry.
@Misc{FairScale2021,
author = {{FairScale authors}},
title = {FairScale: A general purpose modular PyTorch library for high performance and large scale training},
howpublished = {\url{https://github.com/facebookresearch/fairscale}},
year = {2021}
}
