What Are Embeddings? Geometry, Meaning, and the Math That Makes Them Work

Your RAG pipeline passed every demo. A month after launch, users report that a search for "no early termination fee" returns a document titled "Early Termination Fees Explained." The similarity score was 0.91. Nothing errored. That is what happens when you bolt a retrieval system onto geometry you don't understand.

What an embedding actually is

A discrete symbol — a token ID, a word, a document — has no natural geometric meaning. The number 21,639 (the token ID for " refund" in GPT-4's cl100k_base vocabulary) has no inherent relationship to 71,111 (" reimbursement"). An embedding model's job is to learn a mapping from these arbitrary IDs to points in a continuous vector space where the geometry encodes something useful: proximity means semantic similarity.

Formally, an embedding model is a function $f: \mathcal{X} \to \mathbb{R}^d$ where $\mathcal{X}$ is the input domain (sentences, documents, images) and $d$ is the number of dimensions. For text-embedding-3-large, $d = 3072$. For most open-source bi-encoders, $d$ is 768 or 1024.

The training signal is contrastive: given a (query, positive passage, hard negative) triplet, the model is penalized when the query is closer to the negative than to the positive. After billions of such examples, the resulting space has structure. "Customer refund policy" lands near "money-back guarantee" because both appeared as positives for similar training queries. "Astrophysics paper" lands far from both.

Semantic neighborhoods form organically from training signal, not from any explicit rule about synonymy. The viz below shows the pipeline that vector travels through — from raw text to an indexed, retrievable coordinate.

{ "type": "rag-pipeline", "title": "From raw text to retrievable vector", "query": "policy" }

Static vs contextual: why Word2Vec is dead for retrieval

The first generation of embeddings — Word2Vec, GloVe, FastText — produced one fixed vector per word. "Bank" got a single point in space, averaging over its river-bank and financial-institution senses. For retrieval this is a serious problem: a query about financial banking returns documents about riverbanks and vice versa, with no way to distinguish based on context.

Contextual embeddings, introduced with BERT in 2018, run the full input through a transformer and produce representations where each token is influenced by every other token in the sequence. "Bank" in "the river bank was flooded" has a different representation than "bank" in "the bank account was overdrawn" — different values in every dimension.

For production retrieval, the architecture that matters is the bi-encoder: two separate encoder passes (one for the query at query time, one for each document at indexing time) that each produce a single pooled vector. A cross-encoder is more accurate — it processes query and document together, capturing their interaction — but cannot be pre-indexed, which means it must run at query time against every candidate. The standard production pattern uses a bi-encoder for ANN retrieval over millions of documents, then optionally a cross-encoder as a reranker over the top 50-100 candidates.

flowchart LR
    subgraph Indexing["Indexing (offline)"]
        D["Documents"] --> BI_D["Bi-encoder"]
        BI_D --> IDX[("Vector index")]
    end
    subgraph Query["Query time (online)"]
        Q["Query"] --> BI_Q["Same bi-encoder"]
        BI_Q --> ANN["ANN search → top-k"]
        ANN --> RERANK["Cross-encoder reranker\n(optional, top-50 only)"]
        RERANK --> RESULT["Final ranked list"]
    end
    IDX --> ANN
    style BI_D fill:#0e7490,color:#fff
    style BI_Q fill:#0e7490,color:#fff
    style IDX fill:#0e7490,color:#fff
    style RERANK fill:#00e5ff,color:#0a0a0f

The geometry: what high-dimensional space actually does

Three things happen in high-dimensional vector spaces that are not intuitive from 3D experience.

Semantic neighborhoods form. Words and phrases cluster by topic, style, and function. In a well-trained 1536-dim space, all the legal terms cluster together, all the medical dosage terms cluster together, and (critically) they cluster away from each other. A nearest-neighbor query finds things that are similar in meaning, not just in vocabulary.

Analogical structure holds, up to a point. The famous king - man + woman ≈ queen relationship from Word2Vec extends into modern embeddings. Vector arithmetic on sentence embeddings is noisier but still real: you can find semantic transformations by moving in consistent directions through the space. This property is what makes embeddings composable — you can encode attributes and entities separately and combine them.

The curse of dimensionality is real but misunderstood. At very high dimensions, all pairs of random vectors become nearly equidistant. This sounds like it would break everything, but in practice embedding models are trained to concentrate meaning into a small fraction of the available dimensions — they do not use the full space uniformly. The practical consequence is that you need a lot of data and a lot of parameters to fill high-dimensional spaces usefully. A 3072-dim model trained on too little data will be worse than a 768-dim model trained well. Dimension count is not quality.

Matryoshka embeddings: truncate at query time

Matryoshka Representation Learning (MRL), published in 2022 and now standard in OpenAI's text-embedding-3 series and Cohere Embed v4, trains the model with an additional loss: the first $m$ dimensions of any vector should be semantically meaningful on their own, for all values of $m$ in a set like {64, 128, 256, 512, 1024, 2048, 3072}.

The practical consequence: you embed once at full dimensionality during ingestion and then truncate to a smaller slice at query time. The retrieval quality degrades gracefully.

Matryoshka truncation precision (text-embedding-3-large on MTEB)

Full 3072 dims → MTEB avg recall ~64.6
Truncated 1536 dims → ~64.1  (-0.8%)
Truncated 512 dims  → ~62.3  (-3.6%)
Truncated 256 dims  → ~61.0  (-5.5%)
Truncated 64 dims   → ~54.9  (-14.8%)

Storage at 1M vectors (float32):
3072 dims → 11.7 GB
256 dims  → 0.98 GB  (12× reduction)

For most RAG applications, truncating to 512 or 768 dimensions costs under 4% recall and cuts RAM requirements for the vector index significantly. If you are running on managed vector storage where you pay per stored dimension, the economics can be compelling. The catch: precision loss compounds when queries are already borderline-relevant — the aggregate 3-6% loss at 256-512 dims hides per-query losses that can be double that for hard cases. Measure on your own data distribution before committing.

What embeddings cannot represent

This is the section most tutorials skip, and it is the section that explains most production retrieval failures.

Negation. "No early termination fee" and "early termination fee" produce cosine-similar vectors — typically 0.85-0.95 — because the transformer encodes the presence of lexical content, not its logical relationship. A query for "conditions without medication" will retrieve documents about medication. There is no clean fix at the embedding layer. You must layer in a keyword filter that catches negation terms in the original query, or switch to a cross-encoder that can reason about negation in context.

Temporal ordering. "A caused B" and "B caused A" are nearly identical in embedding space. Causal and temporal structure is represented poorly unless the model was explicitly fine-tuned for it (rare). This matters for legal and medical retrieval where sequence matters.

Entity disambiguation. "Apple stock" and "Apple fruit" are better separated in contextual models than in Word2Vec, but the disambiguation is imperfect and degrades on long tail entities. A product named "Titan" may not be distinguishable from the Titan moon or Titan cameras without metadata filtering.

Precise numeric comparison. "Stores up to 2TB" and "stores up to 4TB" can be very close in embedding space. Numeric constraints are a retrieval anti-pattern at the vector layer; filter on a metadata field instead.

These are not bugs to be fixed with a better model. They are structural properties of the dot-product retrieval paradigm. The hybrid search approach that combines dense vectors with sparse BM25 recovers some ground on lexical precision, but it doesn't solve negation.

How modern embedding models are trained

The training recipe for a production-quality sentence embedding model has three phases.

Phase 1: Large-scale contrastive pre-training. The model is initialized from a pretrained language model (something in the BERT/RoBERTa/LLaMA family) and then trained on hundreds of millions of (query, positive, hard negative) triplets using InfoNCE loss. Hard negatives — passages that look lexically similar but are semantically wrong — are the key ingredient; the model learns to ignore surface-level word overlap and focus on meaning.

Phase 2: Domain-specific fine-tuning. For commercial embedding APIs, this often happens implicitly — the training set covers many domains. For open-source models used in specialized applications (code, biomedical, legal), fine-tuning on domain-specific triplets is common and often necessary. The 2026 MTEB leaderboard shows Qwen3-Embedding-8B at the top of overall rankings, but domain-specific tasks can flip the order dramatically.

Phase 3: MRL training (for Matryoshka models). A second loss term trains the prefix of the embedding to be informative on its own, enabling the truncation behavior described above.

flowchart TD
    LM["Pretrained language model\n(BERT / LLaMA base)"]
    subgraph Training["Contrastive training"]
        TP["(Query, Positive, Hard Negative) triplets\nhundreds of millions"]
        LOSS["InfoNCE loss:\npull query toward positive\npush away from negatives"]
    end
    LM --> Training
    Training --> BI["Bi-encoder\n(query tower + doc tower)"]
    BI --> MRL["+ MRL loss for Matryoshka\n(prefix must be informative)"]
    MRL --> FINAL["Production embedding model"]
    style LM fill:#0e7490,color:#fff
    style FINAL fill:#0e7490,color:#fff
    style LOSS fill:#00e5ff,color:#0a0a0f

Dimensionality in practice: the numbers

The choice of dimension affects three things: storage, the RAM footprint of your ANN index, and retrieval quality.

Index RAM estimates (HNSW, float32)
Model dims × bytes per float × vectors × HNSW overhead (~1.5-2x)

text-embedding-3-small (1536 dims), 10M vectors:
→ 1536 × 4 × 10M × 1.5 = ~92 GB

text-embedding-3-large (3072 dims), 10M vectors:
→ 3072 × 4 × 10M × 1.5 = ~184 GB

Matryoshka truncated to 256 dims, 10M vectors:
→ 256 × 4 × 10M × 1.5 = ~15 GB

At $3/GB-month for cloud RAM (illustrative):
→ $276/month vs $45/month for the same corpus

For 10M vectors at full 3072 dims, HNSW requires roughly 184 GB of RAM — memory-optimized-instance territory (an r6i.8xlarge tops out at 256 GB) just for the index, before you've provisioned anything for the model serving it. Matryoshka truncation to 256 dims cuts that to 15 GB. The recall difference is real but smaller than the cost difference; whether it matters depends on your application. Truncation is also not the only compression lever: int8 scalar quantization cuts float32 storage 4x and binary quantization 32x, often at better recall-per-byte than dropping dimensions — especially with a rescoring pass over full-precision vectors for the top candidates. The ANN indexes article covers the memory math and quantization tradeoffs in more depth for different index types.

The pooling question

Transformer models produce one vector per token. An embedding model must convert that sequence of token vectors into a single vector for the whole input. The pooling strategy matters and is almost never mentioned in tutorials.

Mean pooling averages all token vectors (often excluding padding tokens). This is the most common approach and works well for sentence-level semantic similarity. Most modern sentence transformers use this.

CLS pooling uses the vector for the special [CLS] token prepended to every input. BERT-style models were originally trained this way for classification tasks. For retrieval tasks, mean pooling usually outperforms CLS pooling by a few percentage points.

Last-token pooling uses the final token's representation. Decoder-only models (the GPT family, LLaMA) don't have a CLS token, so embedding models built on them typically use the last token's representation or mean-pool the final few tokens.

The pooling strategy is baked into the model and must match how you generate vectors at query time. If you use a library like sentence-transformers, this is handled for you. If you call the raw transformer API yourself, getting it wrong produces nonsense embeddings with no error thrown.

Pooling is not the only silent mismatch. Many open models are trained asymmetrically and expect an instruction prefix: E5 and BGE want "query: " prepended to search queries and "passage: " to documents, and Qwen3-Embedding expects a task instruction ahead of the query. Skip the prefix at query time and the model still returns a perfectly valid-looking vector — recall just quietly drops several points, with nothing in the logs to tell you why. Read the model card before you embed a single document; the prefix convention is as much a part of the model's contract as its dimensionality.

What breaks

Embedding model version changes invalidate the entire index. When OpenAI deprecated text-embedding-ada-002 and introduced text-embedding-3-small, the two models produce vectors in completely different spaces. A cosine similarity between an ada-002 document vector and a text-embedding-3-small query vector is meaningless. If you upgraded your query path without re-indexing documents, recall dropped to near-random and your application started confidently returning wrong results. Always store the model name and version with each vector in your metadata. Always treat a model version upgrade as a full re-ingestion job.

Embedding long documents into a single vector loses structure. A 3000-token legal document embedded into one vector averages over hundreds of distinct clauses. A query about indemnification matches the document's vector with moderate similarity, but the relevant clause might score higher than the document as a whole — or might be washed out. This is why chunking granularity matters more than model choice: retrieval quality is fundamentally bounded by whether your indexed units match your query units. The document ingestion pipelines article covers chunking strategies in detail.

Semantic drift across languages. Multilingual models cover 100+ languages but quality is uneven. Low-resource languages (Swahili, Tagalog) sit in sparser regions of the training distribution, so nearest-neighbor retrieval in those languages is less reliable. If your application is multilingual, verify recall on each target language independently rather than assuming the English MTEB score transfers.

Out-of-distribution queries surface a hard boundary. Embedding models are trained on text that looks like natural language and existing documents. A query like "SQL injection payload XOR 1=1" or a regulatory code "CFR 21 Part 11" may not embed meaningfully near the relevant compliance documents because neither appeared in training at meaningful frequency. MTEB scores measure in-distribution retrieval; your use case may not be in-distribution. This is where hybrid search and exact keyword filtering recover precision.

The version mismatch failure is the most operationally dangerous because it produces high confidence scores on wrong results — cosine similarity between vectors from incompatible spaces is not zero, it's just meaningless noise that looks like real signal.

flowchart TD
    subgraph Index["Index (built with ada-002)"]
        D1["doc vector\n[ada-002 space]"]
        D2["doc vector\n[ada-002 space]"]
    end
    subgraph Query["Query (upgraded to text-3-small)"]
        Q["query vector\n[text-3-small space]"]
    end
    Q -->|"cosine sim = 0.74\n(meaningless)"| D1
    Q -->|"cosine sim = 0.71\n(meaningless)"| D2
    WARN["No error thrown\nTop-k returns wrong docs\nwith high confidence scores"]
    D1 --> WARN
    D2 --> WARN
    style Q fill:#ff2e88,color:#fff
    style WARN fill:#00e5ff,color:#0a0a0f
    style D1 fill:#0e7490,color:#fff
    style D2 fill:#0e7490,color:#fff

The embedding as a coordinate in a shared space

The single most important mental model: an embedding is a coordinate. The query and every candidate document are points in the same space. Distance between points is similarity. The embedding model is the coordinate system — and it is shared between queries and documents. If the query and a document were embedded with the same model, the coordinate system is consistent and distance is meaningful. If they were embedded with different models, or with the same model but different versions, you are comparing coordinates from two different maps. No amount of clever indexing or reranking will fix that.

This is why the choice of which similarity metric to use — cosine, dot product, L2 — must match what the model was trained with. You can read the full treatment in the similarity metrics article, but the short version: for normalized vectors (which most modern models produce), cosine similarity and dot product are mathematically identical and dot product is faster. OpenAI's text-3 series outputs unit-normalized vectors, which is why all three metrics give the same ranking.

When to use embeddings vs alternatives

Embeddings are the right tool when you need to match meaning across paraphrase, synonym variation, and natural language variation at scale. They are not the right tool when:

• You need exact keyword matching for product codes, SKUs, or regulatory references — use BM25 or a keyword filter.
• Your queries hinge on negation or precise numeric constraints — filter on structured metadata fields.
• You have fewer than 10,000 documents — cosine similarity over a numpy matrix without an ANN index costs under 10ms and requires no infrastructure.
• Your content is highly structured (spreadsheets, database rows) — text-to-SQL or structured query tools are more precise than semantic retrieval.

The practical answer for most production RAG systems is embeddings plus keyword search in a hybrid configuration, with metadata filters as a hard pre-filter before the ANN search. That combination recovers the lexical precision that dense retrieval trades away. The choosing an embedding model article covers the current model options in 2026; the vector database landscape article covers where to store and query them at scale.

The decision in practice

If you are building a retrieval system in 2026, the baseline setup that loses least often is:

• text-embedding-3-small for most English RAG applications, or Gemini Embedding at $0.008/1M tokens if cost is the binding constraint (2.5x cheaper than 3-small's $0.02/1M, 16x cheaper than 3-large, at similar quality). For mixed text-and-image corpora in a single vector space, Cohere Embed v4 is the most mature API option as of mid-2026; Gemini Embedding also supports multimodal input.
• 1536 or 768 dimensions as a starting point. Only scale to 3072 if you have a measurable recall gap on your specific data.
• Chunk at the paragraph or semantic section level, not at fixed byte boundaries. The model is not the bottleneck; your chunking is.
• Store the model name and version in every vector's metadata. You will need it when you upgrade.
• Never mix model versions in the same index. If you upgrade the embedding model, re-embed everything.

The geometry works. The failure modes are operational, not algorithmic. Treat the embedding space as a coordinate system, know its limits, and you will spend your pager time on the downstream LLM, not the retrieval layer.