# Track 03: Landmark Models — BERT, GPT & Mixture-of-Experts

Track 02 built the transformer block in the abstract. This track shows
the two directions the field split it into — BERT's masked,
bidirectional encoder for *understanding* text, and GPT's causal,
decoder-only generation for *producing* it — why the generative path won
for the chat-and-agents era, and how Mixture-of-Experts architectures
scale parameter count without scaling compute proportionally.

## Modules

1. The Pretraining Paradigm
2. BERT: Masked Language Modeling
3. BERT Architecture Deep Dive
4. Fine-Tuning BERT for Classification
5. BERT Variants: RoBERTa, DistilBERT, DeBERTa
6. Sentence Transformers and Bi-Encoders
7. GPT: Causal Language Modeling
8. GPT-1 and GPT-2
9. GPT-3 and In-Context Learning
10. GPT-4 and Beyond
11. Why Decoder-Only Won
12. Llama and the Open-Weight Lineage
13. Mixture of Experts: The Core Idea
14. MoE Routing and Load Balancing
15. Mixtral and MoE in Practice
16. How to Read a Model Card
17. Capstone Project