#fundamentals
20 articles
Reading LLM Benchmarks Without Getting Fooled
LLM leaderboard scores are routinely inflated by contamination and benchmark saturation. Learn how to read them skeptically — and what to do instead.
RLHF, Reward Models, and Constitutional AI
How reinforcement learning from human feedback actually works: the reward model, PPO loop, RLAIF, and Constitutional AI — the baseline before DPO shortcuts.
Why Models Hallucinate: A Mechanistic View
Why LLMs hallucinate is not a mystery — it is a structural consequence of next-token prediction, probabilistic decoding, and a missing verification loop.
The AI Engineer's Toolkit: Python, uv, and a Sane Dev Setup
Set up a Python AI engineering environment with uv, proper API key hygiene, and the minimal stack to start calling frontier models without the usual friction.
Decoding and Sampling: Temperature, Top-P, Min-P, and When to Use Each
How temperature, top-p, top-k, and min-p reshape the next-token probability distribution — and which setting to reach for in production vs creative tasks.
Capability limits and failure modes every AI engineer must know
Hallucination, lost-in-the-middle, injection, and benchmark overfitting are structural properties of LLMs — and they shape how you design systems.
Open vs. closed models: a decision framework, not a religion
A practical framework for choosing between open-weight and closed frontier models — covering cost math, data governance, performance gaps, and operational burden as of mid-2026.
The Transformer Block: Residual Stream, MLP, and What Each Part Learns
How attention and MLP sublayers share a residual stream, what factual knowledge each stores, and why this view changes how you debug and fine-tune LLMs.
Modalities Explained: From Text-Only to Omni Models
What modalities are, how frontier models encode images, audio, and video into transformer attention, and how to pick the right model for your pipeline in 2026.
Self-Attention: The Mechanism That Reads Every Token Against Every Other
How self-attention computes dynamic, context-dependent relevance weights between all token pairs, why it scales quadratically, and what that costs you in production.
The model landscape in 2025-2026: closed frontier, open-weight, and reasoning
GPT-5, Claude Opus 4, Gemini 3.1, Llama 4, DeepSeek-R1: what the three competitive tiers actually mean for engineers picking models in production.
JSON Mode vs Structured Outputs vs Tool Calling: What Actually Differs
Three APIs that all return JSON but with very different guarantees. Understand when each breaks, what they cost, and which to reach for in production.
Embeddings and positional encoding: how tokens become geometry
How token embedding tables and positional encodings work, why RoPE dominates 2025 models, and what context-window advertising actually hides from you.
AI Engineer vs ML Engineer: what each role actually owns
AI Engineer is a distinct role from ML Engineer and AI Researcher. Learn the skill boundaries, daily work, and career tradeoffs between each in 2025-2026.
Tool and function calling: the LLM's hands
How LLMs invoke external tools via structured JSON — the protocol, the round-trip mechanics, schema design basics, and the failure modes that bite in production.
What Are Embeddings? Geometry, Meaning, and the Math That Makes Them Work
How embedding models project text into vectors, what semantic neighborhoods mean in practice, and where the math breaks down in production RAG.
Tokenization and BPE: Why your model cannot spell strawberry
How BPE tokenization works, why it makes English cheap and Arabic expensive, and why LLMs fail at counting letters and doing arithmetic.
What generative AI actually does: the next-token prediction loop
Strip away the hype: GenAI is autoregressive next-token prediction over a learned probability distribution. Understanding that loop explains both its power and its failure modes.
The Prediction Loop: What Generative AI Actually Does
GenAI is one loop repeated: predict the next token. Understanding that loop explains the power, the cost model, and every major failure mode — including why models can't count R's.
The Model Landscape: Choosing Without Getting Played
A decision framework for picking between closed frontier models, open-weight, and reasoning tiers — plus benchmark literacy so leaderboard scores stop misleading you.