Skip to content
プロジェクト

johnma2006/mamba-minimal has 2,972 GitHub stars and 218 forks. GitHot measured +1 stars in 24 hours, +3 in 7 days, and +3 in 15 days. Last checked 2026-11-22 08:58 UTC. Last push 2024-03-08.

この数値の測り方

計測したスター増加

24時間
+1
7日間
+3
15日間
+3
1か月
履歴を収集中

期間ごとの純増数です。期間は重複しており、日別の値ではありません。未計測の値は空欄です。スター数は注目度の指標であり、品質の保証ではありません。

概要

PyTorchの1ファイルで実装した、シンプルでミニマルなMamba SSM実装。

機械学習

mamba-minimal

Simple, minimal implementation of Mamba in one file of PyTorch.

Featuring:

  • Equivalent numerical output as official implementation for both forward and backward pass
  • Simplified, readable, annotated code

Does NOT include:

  • Speed. The official implementation is heavily optimized, and these optimizations are core contributions of the Mamba paper. I kept most implementations simple for readability.
  • Proper parameter initialization (though this could be added without sacrificing readability)

Demo

See demo.ipynb for examples of prompt completions.

from model import Mamba
from transformers import AutoTokenizer

model = Mamba.from_pretrained('state-spaces/mamba-370m')
tokenizer = AutoTokenizer.from_pretrained('EleutherAI/gpt-neox-20b')

generate(model, tokenizer, 'Mamba is the')

Mamba is the world's longest venomous snake with an estimated length of over 150 m. With such a large size and a venomous bite, Mamba kills by stabbing the victim (which is more painful and less effective than a single stab of the bite)

150 meters... 🫢 scary!

References

The Mamba architecture was introduced in Mamba: Linear-Time Sequence Modeling with Selective State Spaces by Albert Gu and Tri Dao.

The official implementation is here: https://github.com/state-spaces/mamba/tree/main

関連プロジェクトを探す

読み込み中…