How language models work

Understand what a token is, how a language model predicts text, and how the transformer architecture works.

Developers who use LLMs and want to know what is inside. Beginner · GenAI · 52 modules · about 7 h of reading

Before you start

  • Basic programming
  • High-school maths is enough

What you will cover

  • Tokens and embeddings
  • The transformer architecture
  • How models are trained

The curriculum

  1. GenAI Foundations

  2. Tokens & Language Modeling

  3. Transformer Architecture

Build with the OpenAI API · All learning paths