⭐ If you find this repository helpful, please consider giving it a ⭐ here on GitHub (click the star button in the top right corner)
It's a quick way to show support for this openly available code. ⭐

Transformers - The definitive Guide
This is the corresponding code for the book Transformers - The definitive Guide
The book can be found here
TOC
- Chapter 1 From First Principles to State-of-the-Art Transformers
- Chapter 2 Transformers for Time Series
- Chapter 3 Transformers for Vision Tasks
- Chapter 4 Transformers for Image Generation
- Chapter 5 Transformers for Video Generation
- Chapter 6 Transformers for Audio Tasks
- Chapter 7 Reinforcement Learning Transformers
- Chapter 8 Transformers for Planing, Reasoning and Coding
- Chapter 9 AI Agents for Complex Tasks
- Chapter 10 Optimizing Transformer for Problem Solving
- Chapter 11 Deploying transformer models
- Chapter 12 Where to go next
Instructions and Navigation
All of the code is organized into folders. Each folder starts with CH followed by the chapter number. For example, CH01.
The notebooks are then organized as follows: ch01_attention_mechanism_variations.ipynb, where ch01 indicates the chapter
and attention_mechanism_variations what is done in the notebook.
Repo structure
├── LICENSE
├── README.md <- The top-level README for developers using this project.
├── CH01 <- Per chapter folder with Jupyter notebooks.
├── [name].ipynb <- Jupyter notebooks with naming as mentioned above.
├── CH02 <- Per chapter folder with Jupyter notebooks.
... <- Same structure for all chapters.
├── utils <- Custom classes and functions and utility functions.
├── resources <- Some miscellaneous resources.
Virtual Envrionment
The provided bash script create_env.sh automates the process of creating a Python virtual environment using either conda or pipenv,
installing the required packages from a requirements.txt file. To use the script run bash create_env.sh in your
terminal on Microsoft Windows (with WSL), Apple macOS, or Linux operating systems.
NOTE: A virtual environment is not necessary for the notebooks in this repository, as they are designed to be run on a cloud service with GPU support. Therefore, the provided instructions for creating a virtual environment are more for reference and general guidance than a strict requirement. Running the Notebooks
Every notebook can be opened and run on Google Colab directly from the links below. Just click the Open In Colab badge next to the notebook you want to run.
Chapter 1 — From First Principles to State-of-the-Art Transformers
| Notebook |
Colab |
| Attention Mechanism Variations |
 |
| Embeddings |
 |
| Perplexity |
 |
Chapter 2 — Transformers for Time Series
| Notebook |
Colab |
| Chronos |
 |
| PatchTST Hyperparameters (IBM, 10 Days Ahead, 32 Context Window) |
 |
| Time Series Fine-Tuning (PyTorch) |
 |
Chapter 3 — Transformers for Vision Tasks
| Notebook |
Colab |
| Fine-tune SAM with W&B + Optuna |
 |
| ViT Embeddings & Tokens |
 |
| Image Classification |
 |
| Segment Videos with SAM 2 |
 |
Chapter 4 — Transformers for Image Generation
| Notebook |
Colab |
| DiT |
 |
| KV Compression |
 |
| PixArt-Σ XL Inference |
 |
| Quantize Text-to-Image Models |
 |
Chapter 5 — Transformers for Video Generation
| Notebook |
Colab |
| LTX |
 |
| Latte |
 |
| Tora |
 |
Chapter 6 — Transformers for Audio Tasks
| Notebook |
Colab |
| Kimi-Audio Meeting Transcription |
 |
| Qwen2-Audio Audio Tasks |
 |
| SAM Audio |
 |
| Music Generation |
 |
| Waveforms & Spectrogram Plots |
 |
Chapter 7 — Reinforcement Learning Transformers
| Notebook |
Colab |
| STORM |
 |
| Decision Transformer |
 |
| Replay Buffer (Decision Transformer) |
 |
Chapter 8 — Transformers for Planning, Reasoning and Coding
| Notebook |
Colab |
| Qwen3 |
 |
| Rethink MCTS |
 |
| TreeQuest |
 |
| Kimi K2 Instruct |
 |
Chapter 9 — AI Agents for Complex Tasks
| Notebook |
Colab |
| LangGraph Multi-turn Conversation |
 |
| Market Research Team (LangGraph Multi-Agent) |
 |
Chapter 10 — Optimizing Transformers for Problem Solving
| Notebook |
Colab |
| AdaptThink |
 |
| ART·E with LangGraph |
 |
| rLLM |
 |
Chapter 11 — Deploying Transformer Models
| Notebook |
Colab |
| LLM Performance Evaluation |
 |
| Advanced LoRA Fine-Tuning |
 |
| LangGraph Code Interpreter |
 |
Chapter 12 — Where to Go Next
| Notebook |
Colab |
| SAM 3 Agent |
 |
Each notebook is connected with this Github repo, meaning by running a notebook, it will automatically clone the repo, so you can easily access all resources outside the notebook.
Like customs functions and classes as well as utility functions to automatically install the requirements per chapter:
!git clone https://github.com/Nicolepcx/transformers-the-definitive-guide
current_path = %pwd
if '/transformers-the-definitive-guide' in current_path:
new_path = current_path + '/utils'
else:
new_path = current_path + '/transformers-the-definitive-guide/utils'
%cd $new_path
NOTE: You need to run the notebooks with a GPU.