Hello LLM

Build a Small Model, Understand Large Models

20 lessons · 23 videos · Train in your browser

简体中文

From one neuron to a model that generates text

An original course by 好好用 AI / DavidDingXu. Follow one 92,352-parameter language model through neurons, gradients, attention, pretraining, generation and supervised fine-tuning.

No prior Python or deep learning knowledge is required to start with the diagrams and browser experiments. All code runs on a CPU. Each lesson includes diagrams, code and exercises. Companion videos have Chinese audio and captions; PDF and EPUB downloads are Chinese editions.

Try it: Edit and run a neuron · Train the model in your browser

Course contents

Start with the introduction or opening lesson.

Make a numerical network learn

Lesson	The question you will answer	Video · Chinese
01	Neurons: What Does a Model Calculate Before It Writes the Next Character?	6:19
02	Vectors and Matrices: Do Two Outputs Require Two Copies of the Code?	5:13
03	Nonlinearity: Why Can Multiply-and-Add Not Express Four Simple Rules?	4:30
04	Gradients: Why Can an Update in the Right Direction Make Things Worse?	5:14
05	Backpropagation: Must We Test All 9 Parameters One by One?	5:44
06	The Training Loop: What Happens to Four Predictions After 500 Updates?	5:31

Turn text into training examples

Lesson	The question you will answer	Video · Chinese
07	Embedding: How Does Text Become a Network Input?	4:23
08	Softmax: How Do Scores Become Next-Character Probabilities?	4:41
09	Training Targets: How Does a Sentence Become a Learnable Task?	4:32

Read context and assemble the model

Lesson	The question you will answer	Video · Chinese
10	Attention: How Does Predicting the Next Character Read Earlier Text?	4:37
11	Causal Masks: Why Is Parallel Training Not Peeking at the Answer?	4:09
12	Multi-Head Attention: Where Does Each Head Read?	4:16
13	RoPE: How Does the Model Know the Order?	4:58
14	SwiGLU: What Happens After Reading the Prefix?	4:01
15	Residual Connections and RMSNorm: Why Keep a Direct Path?	4:26
16	The Complete Model: Follow the Character “欢” Through Every Step	4:42

Pretrain, generate and fine-tune

Lesson	The question you will answer	Video · Chinese
17	Pretraining: Why Can New Questions Get Worse as Training Questions Improve?	5:39
18	Generation: How Many Choices Does One Sentence Take?	4:50
19	Supervised Fine-Tuning: Why Can It Continue Text but Still Fail to Answer?	6:21
20	The Full Workflow: Connect Pretraining, Fine-Tuning, and Generation	6:29

Finish with closing thoughts. Runnable entries are in the chapter and code index.

Resources

Run the code · Video course · Experiment records

Contribute

Questions, corrections and translation reviews are welcome: open an issue or read the contribution guide. Local reading and website maintenance are covered in the build guide. Star the project if it helps you learn.

Maintained by DavidDingXu / 好好用 AI.

License

Software: PolyForm Noncommercial 1.0.0. Original text, figures, videos and data: CC BY-NC 4.0. Both restrict commercial use; see LICENSING.md.

Links

LINUX DO