johnma2006/mamba-minimal
johnma2006/mamba-minimal has 2,971 GitHub stars and 218 forks. GitHot measured 0 stars in 24 hours, +2 in 7 days, and +2 in 15 days. Last checked 2026-11-19 15:55 UTC. Last push 2024-03-08.
Gemessener Sternzuwachs
- in 24h
- 0
- in 7 days
- +2
- in 15 days
- +2
- in 1 month
- Verlauf wird gesammelt
Nettoänderungen in überlappenden Zeiträumen, keine Tagessummen. Fehlende Messwerte bleiben leer. Sterne zeigen Interesse, nicht Qualität.
Über uns
Simple, minimal implementation of the Mamba SSM in one file of PyTorch.
Maschinelles LernenREADME.md
Originalmamba-minimal
Simple, minimal implementation of Mamba in one file of PyTorch.
Featuring:
- Equivalent numerical output as official implementation for both forward and backward pass
- Simplified, readable, annotated code
Does NOT include:
- Speed. The official implementation is heavily optimized, and these optimizations are core contributions of the Mamba paper. I kept most implementations simple for readability.
- Proper parameter initialization (though this could be added without sacrificing readability)
Demo
See demo.ipynb for examples of prompt completions.
from model import Mamba
from transformers import AutoTokenizer
model = Mamba.from_pretrained('state-spaces/mamba-370m')
tokenizer = AutoTokenizer.from_pretrained('EleutherAI/gpt-neox-20b')
generate(model, tokenizer, 'Mamba is the')
Mamba is the world's longest venomous snake with an estimated length of over 150 m. With such a large size and a venomous bite, Mamba kills by stabbing the victim (which is more painful and less effective than a single stab of the bite)
150 meters... 🫢 scary!
References
The Mamba architecture was introduced in Mamba: Linear-Time Sequence Modeling with Selective State Spaces by Albert Gu and Tri Dao.
The official implementation is here: https://github.com/state-spaces/mamba/tree/main