#fine-tuning
11 articles
RLHF, Reward Models, and Constitutional AI
How reinforcement learning from human feedback actually works: the reward model, PPO loop, RLAIF, and Constitutional AI — the baseline before DPO shortcuts.
RAG evaluation, failure modes, and when to use long context or fine-tuning instead
Measure a RAG pipeline with RAGAS, spot the failure modes that kill production quality, and decide between RAG, long context, or fine-tuning.
The data flywheel: closing the LLM feedback loop in production
How to run a data flywheel in production: capture implicit and explicit user signals, curate them into fine-tuning data, and compound model quality every cycle.
Reasoning Distillation: Teaching Small Models to Think
How DeepSeek-R1 distillation lets a 7B model out-reason a 32B baseline — the mechanics, the learnability gap, and what breaks in production.
How reasoning models are trained: RLVR, GRPO, and the post-training stack
The post-training recipe behind o1 and DeepSeek-R1: how RLVR and GRPO replaced RLHF for reasoning, with real training mechanics and failure modes.
Synthetic Data Engineering: Pipelines, Quality, and Model Collapse Risks
How to build production-grade synthetic data pipelines for LLM fine-tuning—Magpie, Evol-Instruct, quality filters—and why model collapse is a real but avoidable failure.
Preference Optimization: DPO, SimPO, ORPO, and the Post-RLHF Stack
How DPO, SimPO, ORPO, and GRPO replaced the classic RLHF pipeline — what each method removes, when to use which, and the 2025 reasoning-alignment frontier.
LoRA, QLoRA, and the PEFT Landscape
How low-rank adapters let you fine-tune a 70B model on a single GPU, with the rank, alpha, and target-module settings that actually matter in production.
SFT and Instruction Tuning: Data Is the Job
Supervised fine-tuning fundamentals with an honest look at why 1,000 curated examples reliably beat 100,000 scraped ones, and the training recipes that actually work.
The Decision Ladder: Fine-Tuning vs RAG vs Prompting
A concrete decision framework for when prompting, RAG, or fine-tuning pays off — grounded in real cost, latency, and data trade-offs, not vendor marketing.
Fine-Tuning: The Decision Framework
When to fine-tune, when to stay with RAG or prompting, and how LoRA, DPO, and distillation actually work — with the cost math to make the call.