English | 中文

🔥 Pyre Code

Implement the internals of modern AI systems from scratch — Transformers, vLLM, TRL, and beyond.

Read the paper, then write the code. No GPU required.

---

🧠 What is Pyre Code?

76 problems. You write the implementation, a local grading service runs the tests, you see what broke. That's it.

The problems cover what's actually inside Transformers, vLLM, TRL, diffusion models, and GNNs — attention variants, training tricks, inference kernels, alignment algorithms, graph neural networks. No GPU needed.

Who is this for?

• Preparing for ML interviews — practice implementing core components under test, not just reading about them
• Learning by building — if you learn best by writing code rather than watching lectures, this is your gym
• Deepening your understanding — you've used nn.MultiheadAttention, now write it yourself

Features

• Browser editor — Monaco with Python syntax highlighting, no IDE setup
• Instant feedback — submit and see pass/fail per test case in seconds
• Reference solutions — compare after your own attempt
• Progress tracking — solved count and attempt history, persisted across sessions
• AI Help — optional AI-powered hints via any OpenAI-compatible API (configure in .env or per-user in the UI)
• Fully local — nothing leaves your machine (unless you opt into AI Help)

Tech Stack

Layer	Technology
Frontend	Next.js + Monaco Editor + Tailwind CSS
Backend	FastAPI grading service
Judge Engine	torch_judge — executes and validates submissions
Storage	SQLite (progress tracking)

---

📢 News

• [2026/04/20] New GNN learning path — 8 problems covering GCN, GAT, GIN, MPNN, GraphSAGE, link prediction, and graph autoencoders. 🔥
• [2026/04/20] New UI redesign with OKLch color system, dark mode, and Geist typography — classic design still available via toggle. 🔥
• [2026/04/13] Submission history — review all your past attempts per problem.
• [2026/04/10] AI Help — optional AI-powered hints via any OpenAI-compatible API. 🔥
• [2026/04/10] Print output capture — print() statements now show in test results.
• [2026/04/09] 68 problems covering Transformers, vLLM, TRL, diffusion models, and more. 🔥
• [2026/04/09] Initial release of Pyre Code 🎉

---

🚀 Getting Started

Prerequisites

• Python 3.11+
• Node.js 18+

Installation

Option A — one-liner (recommended)

macOS / Linux:

git clone https://github.com/whwangovo/pyre-code.git
cd pyre-code
./setup.sh
npm run dev

Windows (PowerShell):

git clone https://github.com/whwangovo/pyre-code.git
cd pyre-code
.\setup.ps1
npm run dev

Windows (CMD):

git clone https://github.com/whwangovo/pyre-code.git
cd pyre-code
setup.bat
npm run dev

The setup script automatically creates a .venv Python environment (prefers uv, falls back to python -m venv), installs all dependencies, then prints the start command.

When .venv exists, npm run dev prefers that project-local Python automatically. If .venv is missing, it falls back to the current shell's python.

Option B — conda

git clone https://github.com/whwangovo/pyre-code.git
cd pyre-code
conda create -n pyre python=3.11 -y && conda activate pyre
pip install -e ".[dev]"
npm install
npm run dev   # run with conda env activated

Option C — manual (venv)

git clone https://github.com/whwangovo/pyre-code.git
cd pyre-code

# create a Python env — pick one:
uv venv --python 3.11 .venv && source .venv/bin/activate && uv pip install -e ".[dev]"
# or: python3 -m venv .venv && source .venv/bin/activate && pip install -e ".[dev]"
# Windows: python -m venv .venv && .venv\Scripts\activate && pip install -e ".[dev]"

npm install
npm run dev

Either way, once running:

• Grading service → http://localhost:8000
• Web app → http://localhost:3000

Option D — Docker

git clone https://github.com/whwangovo/pyre-code.git
cd pyre-code
docker compose up --build

Open http://localhost:3000. Progress is persisted in a Docker volume. Run docker compose down -v to reset.

AI Help (optional)

To enable server-side AI hints, copy web/.env.example to web/.env and fill in:

AI_HELP_BASE_URL=https://api.openai.com/v1
AI_HELP_API_KEY=sk-...
AI_HELP_MODEL=gpt-4o-mini

Any OpenAI-compatible endpoint works (OpenAI, Anthropic via proxy, Ollama, etc.). Users can also configure their own API key in the UI if no server-side config is set.

---

📋 Problem Set

76 problems organized by category:

Category	Problems
Fundamentals	ReLU, Softmax, GELU, SwiGLU, Dropout, Embedding, Linear, Kaiming Init, Linear Regression
Normalization	LayerNorm, BatchNorm, RMSNorm
Attention	Scaled Dot-Product, Multi-Head, Causal, Cross, GQA, Sliding Window, Linear, Flash, Differential, MLA
Position Encoding	Sinusoidal PE, RoPE, ALiBi, NTK-aware RoPE
Architecture	SwiGLU MLP, GPT-2 Block, ViT Patch, ViT Block, Conv2D, Max Pool, Depthwise Conv, MoE, MoE Load Balance
Training	Adam, Cosine LR, Gradient Clipping, Gradient Accumulation, Mixed Precision, Activation Checkpointing
Distributed	Tensor Parallel, FSDP, Ring Attention
Inference	KV Cache, Top-k Sampling, Beam Search, Speculative Decoding, BPE, INT8 Quantization, Paged Attention
Loss & Alignment	Cross Entropy, Label Smoothing, Focal Loss, Contrastive Loss, DPO, GRPO, PPO, Reward Model
Diffusion & DiT	Noise Schedule, DDIM Step, Flow Matching, adaLN-Zero
Adaptation	LoRA, QLoRA
Reasoning	MCTS, Multi-Token Prediction
SSM	Mamba SSM
Graph Neural Networks	GCN, Graph Readout, GAT, GIN, MPNN, GraphSAGE, Link Prediction, Graph Autoencoder

Learning Paths

Pick one based on what you're working toward:

Path	Problems	Description
Transformer Internals	12	Activations → Normalization → Attention → GPT-2 Block
Attention & Position Encoding	13	Every attention variant + RoPE, ALiBi, NTK-RoPE
Train a GPT from Scratch	15	Embeddings → architecture → loss → optimizer → training tricks
Inference & Distributed Training	9	KV cache, quantization, sampling, tensor parallel, FSDP
Alignment & Agent Reasoning	6	Reward model → DPO → GRPO → PPO → MCTS
Vision Transformer Pipeline	7	Conv → patch embedding → ViT block
Diffusion Models & DiT	5	Noise schedule → DDIM → flow matching → adaLN-Zero
LLM Frontier Architectures	7	GQA, Differential Attention, MLA, MoE, Multi-Token Prediction
Graph Neural Networks	8	GCN → GAT → GIN → MPNN → GraphSAGE → Link Prediction → GAE

Not sure where to start?

Fundamentals ──→ Transformer Internals ──→ Train a GPT from Scratch
                       │                          │
                       ▼                          ▼
              Attention & Position       Inference & Distributed
                       │                          │
                       ▼                          ▼
              LLM Frontier Archs         Alignment & Reasoning
                       │
               ┌───────┼───────┐
               ▼       ▼       ▼
     Vision Trans.  Diffusion  Graph Neural Networks

---

⚙️ Configuration

Variable	Default	Description
GRADING_SERVICE_URL	http://localhost:8000	Grading service URL
DB_PATH	./data/pyre.db	SQLite database for progress tracking

Set in web/.env.local to override.

---

📁 Project Structure

pyre/
├── web/                  # Next.js frontend
│   ├── src/app/          # Pages and API routes
│   ├── src/components/   # UI components
│   └── src/lib/          # Utilities, problem data
├── grading_service/      # FastAPI backend
├── torch_judge/          # Judge engine (problem definitions + test runner)
└── package.json          # Dev scripts (runs frontend + backend concurrently)

---

🤝 Contributing

Contributions are welcome! Here are some ways you can help:

• Submit a new problem — open a PR with the problem definition and test cases in torch_judge/
• Report a bug — open an issue with steps to reproduce
• Fix a bug — fork, fix, and submit a PR
• Improve docs — typos, clarifications, translations

Please open an issue first for larger changes so we can discuss the approach.

---

⭐ Star History

---

🙏 Acknowledgements

Problem set and judge engine based on TorchCode by duoan, licensed under MIT.

---

📄 License

Distributed under the MIT License. See LICENSE for more information.