Track 02: Transformer Architecture

Every model in this curriculum — GPT, BERT, Llama, Claude, whichever chat product you use daily — is a transformer. This track builds the architecture up from the linear algebra you actually need (dot products, nothing heavier), computes self-attention by hand on a tiny example, and ends with you implementing a working mini-transformer in PyTorch. No research-paper notation assumed.

Modules

1. The Problem Transformers Solve
2. The Math You Actually Need: Vectors, Matrices, Dot Products
3. Embeddings as Lookup Tables
4. What Attention Actually Computes
5. Queries, Keys and Values
6. Self-Attention Step by Step, By Hand
7. Scaled Dot-Product Attention
8. Attention Masks and Causal Masking
9. Multi-Head Attention
10. Why Positional Information Is Needed
11. Sinusoidal Positional Encoding
12. Rotary Position Embeddings (RoPE)
13. The Feed-Forward Network
14. Residual Connections and Layer Normalization
15. The Complete Transformer Block
16. Stacking Blocks Into a Model
17. Encoder-Only vs. Decoder-Only vs. Encoder-Decoder
18. Implementing a Mini Transformer in PyTorch
19. Capstone Project