johnma2006/mamba-minimal
johnma2006/mamba-minimal has 2,972 GitHub stars and 218 forks. GitHot measured +1 stars in 24 hours, +3 in 7 days, and +3 in 15 days. Last checked 2026-11-22 08:58 UTC. Last push 2024-03-08.
計測したスター増加
- 24時間
- +1
- 7日間
- +3
- 15日間
- +3
- 1か月
- 履歴を収集中
期間ごとの純増数です。期間は重複しており、日別の値ではありません。未計測の値は空欄です。スター数は注目度の指標であり、品質の保証ではありません。
概要
PyTorchの1ファイルで実装した、シンプルでミニマルなMamba SSM実装。
機械学習README.md
オリジナルmamba-minimal
Simple, minimal implementation of Mamba in one file of PyTorch.
Featuring:
- Equivalent numerical output as official implementation for both forward and backward pass
- Simplified, readable, annotated code
Does NOT include:
- Speed. The official implementation is heavily optimized, and these optimizations are core contributions of the Mamba paper. I kept most implementations simple for readability.
- Proper parameter initialization (though this could be added without sacrificing readability)
Demo
See demo.ipynb for examples of prompt completions.
from model import Mamba
from transformers import AutoTokenizer
model = Mamba.from_pretrained('state-spaces/mamba-370m')
tokenizer = AutoTokenizer.from_pretrained('EleutherAI/gpt-neox-20b')
generate(model, tokenizer, 'Mamba is the')
Mamba is the world's longest venomous snake with an estimated length of over 150 m. With such a large size and a venomous bite, Mamba kills by stabbing the victim (which is more painful and less effective than a single stab of the bite)
150 meters... 🫢 scary!
References
The Mamba architecture was introduced in Mamba: Linear-Time Sequence Modeling with Selective State Spaces by Albert Gu and Tri Dao.
The official implementation is here: https://github.com/state-spaces/mamba/tree/main
関連プロジェクトを探す
読み込み中…