# Track 05: GPUs & GenAI Hardware

Why none of this runs practically on a CPU, what a GPU is actually doing
differently at the hardware level, why VRAM rather than raw compute is
the bottleneck you hit first, and how to calculate — before you rent
anything — whether a given model will fit on a given card. Ends with the
economics: which GPU, cloud or local, and what it costs.

## Modules

1. Why GPUs, Not CPUs
2. GPU Architecture for Software Engineers
3. CUDA Cores vs. Tensor Cores
4. Memory Hierarchy and Bandwidth
5. VRAM: The Real Constraint
6. Calculating a Model's Memory Requirements
7. Numeric Formats: FP32, FP16, BF16
8. Mixed Precision Training
9. FlashAttention and Memory-Efficient Kernels
10. Data Parallelism
11. Model and Tensor Parallelism
12. Pipeline Parallelism and ZeRO
13. Choosing GPUs: Consumer vs. Datacenter
14. Cloud GPU Economics
15. Running Models on CPU and Apple Silicon
16. Capstone Project