Pre-Training and Post-Training in AI Models

Al Engineering 101 with Chip Huyen (Nvidia, Stanford, Netflix)

Topic 2 of 7

Published Oct 23, 2025 · Clip 7:08–15:21 (8:13)

Chip explains pre-training as encoding statistical language patterns to predict next tokens, using examples like Sherlock Holmes' frequency analysis. Post-training, including fine-tuning, adjusts models on specific data for tasks, often via supervised fine-tuning or distillation from larger models. She notes most users rely on pre-trained models, with post-training transforming raw models into usable ones, and highlights the shift toward post-training for differentiation as pre-training data plateaus.