#post-training
9 articles
RLHF, Reward Models, and Constitutional AI
How reinforcement learning from human feedback actually works: the reward model, PPO loop, RLAIF, and Constitutional AI — the baseline before DPO shortcuts.
Reasoning Distillation: Teaching Small Models to Think
How DeepSeek-R1 distillation lets a 7B model out-reason a 32B baseline — the mechanics, the learnability gap, and what breaks in production.
How reasoning models are trained: RLVR, GRPO, and the post-training stack
The post-training recipe behind o1 and DeepSeek-R1: how RLVR and GRPO replaced RLHF for reasoning, with real training mechanics and failure modes.
Synthetic Data Engineering: Pipelines, Quality, and Model Collapse Risks
How to build production-grade synthetic data pipelines for LLM fine-tuning—Magpie, Evol-Instruct, quality filters—and why model collapse is a real but avoidable failure.
Preference Optimization: DPO, SimPO, ORPO, and the Post-RLHF Stack
How DPO, SimPO, ORPO, and GRPO replaced the classic RLHF pipeline — what each method removes, when to use which, and the 2025 reasoning-alignment frontier.
LoRA, QLoRA, and the PEFT Landscape
How low-rank adapters let you fine-tune a 70B model on a single GPU, with the rank, alpha, and target-module settings that actually matter in production.
SFT and Instruction Tuning: Data Is the Job
Supervised fine-tuning fundamentals with an honest look at why 1,000 curated examples reliably beat 100,000 scraped ones, and the training recipes that actually work.
What Are Reasoning Models? Chain-of-Thought, o1, and the Inference-Time Shift
How o1-style models differ from standard LLMs: internal chain-of-thought via RL, not prompted CoT, and why that changes your prompting, cost model, and architecture.
Fine-Tuning: The Decision Framework
When to fine-tune, when to stay with RAG or prompting, and how LoRA, DPO, and distillation actually work — with the cost math to make the call.