How models are trained

You don't need the math, but knowing the three-stage recipe behind every major model explains almost every quirk you'll encounter at work — from why models sound so agreeable to why they have a knowledge cutoff.

Stage 1 — Pre-training: read (almost) everything

The model ingests a huge slice of public text — books, articles, websites, code — and plays the next-word prediction game trillions of times. This is where capability comes from, and it explains three things you'll notice: models know public information but not your company's; their knowledge stops at a cutoff date; and they absorbed both the brilliance and the biases of human writing (Module 4 deals with that).

Stage 2 — Fine-tuning: learn to be an assistant

A raw pre-trained model just continues text — ask it a question and it might reply with more questions, because that's what internet forums look like. So it's trained further on curated examples of the behavior we want: question → helpful answer, instruction → compliant execution. This is why the tool behaves like an assistant instead of an autocomplete toy.

Stage 3 — Feedback: learn what people prefer

Humans (and increasingly, AI judges) rate candidate answers, and the model is nudged toward the preferred ones. This makes output dramatically more useful — and creates the one side effect you must manage: agreeableness. Models are optimized toward answers people rate highly, and people rate agreement highly. Push back on a correct answer and the model may fold. State a wrong assumption confidently and it may play along.