luhengshiwo/LLMForEverybody

     

Learning LLM is all you need.

中文 | English | Русский

👉 Visit LearnLLM.AI | Start learning large language models here

LearnLLM.AI Highlights

Curated LLM interview question bank: practical questions from fundamentals to frontier topics, built to help you prepare for job interviews and catch career opportunities.

Structured paper reading: starting from the foundational 2017 Transformer paper, the course follows the evolution of the field with a clear knowledge structure, suitable for developers with different backgrounds who want to improve step by step.

Exclusive promo code

We have prepared a limited-time promo code for GitHub users: GITHUB50. Hope to keep learning and growing together with you on LearnLLM.AI!

Companion video lessons, continuously updated:

👉 Watch on bilibili

👉 Watch on YouTube

If you have any questions, feel free to contact us at any time.

Happy Learning!

The LearnLLM.AI team

---

Selected LLM Papers

Time	Paper	Introduction	Video	Start learning
2017-06-12	Transformer	Introduced self-attention and the Transformer architecture
2018-06-11	GPT-1	A generative Transformer with pre-training plus fine-tuning
2018-10-11	BERT	Bidirectional encoder: MLM plus NSP
2019-02-14	GPT-2	Large-scale unsupervised text generation
2019-10-23	T5	A unified text-to-text framework
2020-05-28	GPT-3	Large models and few-shot learning ability
2020-10	ViT	Brought the Transformer backbone into vision
2021-02	ViLT	A minimal vision-language pre-training architecture
2021-02	CLIP	Zero-shot visual learning through natural language supervision
2021-02	DALL·E 1	The beginning of autoregressive text-to-image generation
2021-07-07	CodeX	A GPT-family model for code generation
2021-12	Stable Diffusion	Latent diffusion models pushed open text-to-image generation forward
2022-02-08	AlphaCode	A competitive programming level code generation system
2022-03-04	InstructGPT	Human feedback alignment and instruction tuning
2022-04	DALL·E 2	High-fidelity text-to-image generation based on CLIP Latents
2022-12	Whisper	A foundation model for speech recognition with large-scale weak supervision
2023-02-27	LLaMA-1	An efficient open pre-trained base model
2023-04	LLaVA	An important starting point for open multimodal instruction tuning
2023-07-18	LLaMA-2	An upgraded LLaMA release with commercial use allowed
2023-08	Qwen-VL	An early vision-language base model from Qwen
2023-09-28	Qwen 1	The first-generation Qwen base model
2023-10-10	Mistral 7B	An efficient open 7B-class model
2023-12	LVM	A route toward large vision models with pure visual autoregressive modeling
2024-02	Mixtral 8x7B	A representative open sparse MoE model
2024-03	Gemma 1	The first release of Google's lightweight open model family
2024-05	DeepSeek-V2	An efficient MoE language model balancing quality and inference cost
2024-06	ChatGLM	A Chinese model family evolving from GLM-130B to GLM-4
2024-07	Llama 3	Meta's new generation of open flagship models
2024-07	Gemma 2	Continued quality improvements for practical open model sizes
2025-03	Gemma 3	Native multimodality and 128K context in Gemma
Continuously updated...

Expand/collapse

The Road to AGI

Expand/collapse

Contents

• 🐳Prologue: the road to AGI
• 🐱Chapter 1: LLM Pre-Training
  • 🐼Architecture
  • 🐹Optimizer
  • 🐰Activation functions
  • 🐭Attention
  • 🐯Positional encoding
  • 🐨Tokenizer
  • 🐻Parallel strategies
  • 🐷LLM training frameworks
• 🐶Chapter 2: LLM deployment and inference
• 🐯Chapter 3: LLM fine-tuning
• 🐻Chapter 4: LLM quantization
• 🐼Chapter 5: GPU and LLM parallelism
• 🐨Chapter 6: Prompt Engineering
• 🦁Chapter 7: Agent
  • 🐷RAG
• 🐘Chapter 8: Enterprise LLM adoption
• 🐰Chapter 9: LLM evaluation metrics
• 🐷Chapter 10: Current topics
• 🦁Chapter 11: Mathematics

Prologue: the road to AGI

⬆ Back to contents

Annual LLM paper reviews

2017: Transformer arrived, and everything started here

2018: GPT and BERT, pre-training split into two tracks

2019: models started getting bigger, GPT-2 and T5

2020: GPT-3 arrived, what did 175 billion parameters actually bring?

2021: not just text, CLIP taught models to see images

2022: making models more obedient, InstructGPT and RLHF

2023: after LLaMA was released, open models started catching up

2024: open models started recalculating the cost of training and inference

What is Scaling Law, the thing everyone keeps talking about?

Intelligence emergence and the origin of AGI

What is perplexity?

Pre-Training Llama-3.1 405B, how much compute does the extra-large version need?

Chapter 1: LLM Pre-Training

⬆ Back to contents

Architecture

Understand why Transformer uses LayerNorm instead of BatchNorm in 10 minutes

Mixture of Experts, MoE, explained excerpt

The easiest way to understand Mamba, Chinese translation

What is a multimodal LLM in 10 minutes

Optimizer

The most complete neural network optimizer overview

Neural network optimizers (1): overview

Neural network optimizers (2): SGD

Neural network optimizers (3): Momentum

Neural network optimizers (4): ASGD

Neural network optimizers (5): Rprop

Neural network optimizers (6): AdaGrad

Neural network optimizers (7): AdaDelta

Neural network optimizers (8): RMSprop

Neural network optimizers (9): Adam

Neural network optimizers (10): Nadam

Neural network optimizers (11): AdamW

Neural network optimizers (12): RAdam

Activation functions

Why large language models use SwiGLU as the activation function

Neural network activation functions (1): overview

Neural network activation functions (2): Sigmoid, Softmax, and Tanh

Neural network activation functions (3): ReLU and its variants

Neural network activation functions (4): ELU and its variant SELU

Neural network activation functions (5): gating family, GLU, Swish, and SwiGLU

Neural network activation functions (6): GELU and Mish

Attention

What math do you need to understand FlashAttention? The final problem from China's gaokao math exam

What changed in FlashAttention v2 compared with v1?

Why Multi-Query-Attention and Group-Query-Attention appeared

DeepSeek MLA explained in one article

Positional encoding

What is Position-Encoding in LLMs?

Applications of complex functions in LLM positional encoding

The most beautiful mathematical formula: Euler's formula

From the beauty of Euler's formula to rotary position encoding RoPE

Tokenizer

The most complete LLM tokenizer overview

Understanding LLM tokenizers (1)

Understanding LLM tokenizers (2)

Understanding LLM tokenizers (3)

Understanding LLM tokenizers (4)

Understanding LLM tokenizers (5)

Understanding LLM tokenizers (6)

Parallel strategies

LLM parallel strategies, Chinese translation

LLM distributed training parallelism (1): overview

LLM distributed training parallelism (2): data parallelism

LLM distributed training parallelism (3): pipeline parallelism

LLM distributed training parallelism (4): tensor parallelism

LLM distributed training parallelism (5): hybrid parallelism

LLM training frameworks

LLM training frameworks (1): overview

LLM training frameworks (2): FSDP

LLM training frameworks (3): DeepSpeed

LLM training frameworks (4): Megatron-LM

LLM training frameworks (5): Accelerate

Chapter 2: LLM deployment and inference

⬆ Back to contents

Deploy a private LLM locally in 10 minutes

Model deployment without asking anyone: the ultimate performance estimation formulas from TTFT to Throughput

Why LLM output tokens cost more than input tokens

How to evaluate LLM output speed: what is the difference between first-token latency and following-token latency?

What is the difference between latency and throughput in LLMs?

How vLLM uses PagedAttention to serve LLMs easily, quickly, and cheaply, Chinese translation

DevOps, AIOps, MLOps, LLMOps: what do these Ops mean?

LLM inference frameworks (1): overview

LLM inference frameworks (2): vLLM

LLM inference frameworks (3): Text generation inference (TGI)

LLM inference frameworks (4): TensorRT-LLM

LLM inference frameworks (5): Ollama

Chapter 3: LLM fine-tuning

⬆ Back to contents

Learn to wrap, not really, Llama-3 in 10 minutes, beginner friendly

Parameter-efficient fine-tuning (PEFT), LoRA fine-tuning, and more

LLM fine-tuning with Soft prompts (1): overview

LLM fine-tuning with Soft prompts (2): Prompt Tuning

LLM fine-tuning with Soft prompts (3): Prefix-Tuning

LLM fine-tuning with Soft prompts (4): P-Tuning

LLM fine-tuning with Soft prompts (5): Multitask prompt tuning

LLM fine-tuning with Adapters (1): overview

LLM fine-tuning with Adapters (2): LoRA

LLM fine-tuning with Adapters (3): QLoRA

LLM fine-tuning with Adapters (4): AdaLoRA

LLM fine-tuning frameworks (1): overview

LLM fine-tuning frameworks (2): Huggingface-PEFT

LLM fine-tuning frameworks (3): Llama-Factory

Chapter 4: LLM quantization

⬆ Back to contents

Understand LLM quantization in 10 minutes

Three levels of understanding LLM quantization

Chapter 5: GPU and LLM parallelism

⬆ Back to contents

GPU knowledge everyone can understand in the AGI era

How GPU parallelism in Transformer differs from earlier NLP algorithms

Three factors in LLM deployment: VRAM, compute, and communication

Chapter 6: Prompt Engineering

⬆ Back to contents

Can past tense jailbreak an LLM? A guide to LLM security and attacks

A long guide to Prompt Engineering: unlocking the power of LLMs

CoT chain of thought, ToT tree of thought, GoT graph of thought: what are they?

Chapter 7: Agent

⬆ Back to contents

How to design agent architecture: follow OpenAI or Anthropic?

MCP: basic concepts, quick start, and internal principles

LLM application adoption guide: application categories (1)

LLM application adoption: architecture design (2)

LLM application adoption: Text-2-SQL (3)

Develop LLMs or use LLMs

Agent design paradigms and common frameworks

LangChain to the left, Coze to the right

What is a sandbox: concept and three implementation approaches

RAG

Vector databases embrace LLMs

RAG architecture with Knowledge Graph

GraphRAG: unlocking LLM retrieval over narrative private data, Chinese translation

Practical guide to enterprise RAG adoption

How to do multimodal RAG in 10 minutes

Chapter 8: Enterprise LLM adoption

⬆ Back to contents

The end of CRUD-ETL engineers: from NL2SQL to ChatBI

LLM adoption challenge: hallucinations

LLM adoption challenge: output uncertainty

LLM adoption challenge: structured output

New job opportunities emerging around LLM applications: Red-teaming

The LLM repeater problem

Chapter 9: LLM evaluation metrics

What evaluation metrics do LLMs have?

LLM performance evaluation: Needle In A Haystack

Evaluation metrics: LLM performance test for counting stars

Chapter 10: Current topics

⬆ Back to contents

Why is Llama 3.1 405B so large?

Is 9.11 greater than 9.9? How did the LLM fail again?

The Korean "Nth Room" case reappears through DeepFake: technology and countermeasures

How I passed the advanced Ruankao system architect design exam in the second half of 2022

Using Exploit and Explore to solve the "what should we eat?" problem

Chapter 11: Mathematics

⬆ Back to contents

Linear algebra

Mathematics for AI and LLM from scratch: linear algebra (1)

Mathematics for AI and LLM from scratch: linear algebra (2)

Mathematics for AI and LLM from scratch: linear algebra (3)

Calculus

Mathematics for AI and LLM from scratch: calculus (1)

Mathematics for AI and LLM from scratch: calculus (2)

Probability and statistics

Mathematics for AI and LLM from scratch: probability and statistics (1), Bayes' theorem and probability distributions

Mathematics for AI and LLM from scratch: probability and statistics (2), ways to describe probability distributions

Mathematics for AI and LLM from scratch: probability and statistics (3), Central Limit Theorem

---

🌐 Visit LearnLLM.AI | Start learning large language models here