Choosing an embedding model in 2026: OpenAI, Cohere, Gemini, Voyage, and open-source

The first sign that something was wrong came from the customer support logs. Semantic search was finding documents about a "neural network architecture" paper when users queried "building electrical wiring inspection." The embeddings were from a general-purpose model trained on web text, and "architecture" sat close to "building" in that vector space. The fix was not prompt engineering. It was switching models.

Embedding model choice is upstream of everything else in a retrieval pipeline. A bad model produces bad vectors, and bad vectors produce bad recall, and no amount of index tuning or reranking saves you when the query vector and the document vector don't land in the same semantic neighborhood to begin with. This is a harder mistake to diagnose than a crashed server because the system keeps running and returning results — they're just the wrong ones.

In 2026 there are more embedding models than at any previous point, spanning three or four price tiers, multiple modalities, and a range of embedding dimensions. The decision has gotten harder to make well. This article is the framework.

What the MTEB leaderboard actually tells you

The Massive Text Embedding Benchmark (MTEB) is the standard reference for embedding model quality. It covers tasks across retrieval, classification, clustering, semantic textual similarity, and more, aggregated into a single score. As of mid-2026, Qwen3-Embedding-8B leads at 70.6, ahead of all API-based commercial models.

That number is real. And it's also not enough information to make a decision.

MTEB's retrieval benchmarks are general-domain English tasks — Wikipedia passages, news articles, diverse web content. If you're building a legal document retrieval system, a code search engine, or a Spanish-language customer support bot, your actual recall numbers will diverge from MTEB in ways that are model-specific and hard to predict. The model that ranks third on MTEB overall might rank first on your domain. The reverse is equally common.

The right workflow: take your top three or four candidate models, embed 1,000-2,000 representative documents from your actual corpus, write 50-100 representative queries with known relevant documents, and measure recall@10 directly. This takes an afternoon and saves months of grief.

from openai import OpenAI
import numpy as np

client = OpenAI()

def recall_at_k(model: str, queries: list[dict], k: int = 10) -> float:
    """
    queries: [{"query": str, "relevant_doc_ids": list[str]}]
    Returns recall@k across the query set.
    """
    # Embed the corpus once
    corpus_texts = [doc["text"] for doc in corpus]  # your document list
    corpus_resp = client.embeddings.create(model=model, input=corpus_texts)
    corpus_vecs = np.array([e.embedding for e in corpus_resp.data])
    # L2-normalize for cosine similarity via dot product
    norms = np.linalg.norm(corpus_vecs, axis=1, keepdims=True)
    corpus_vecs_norm = corpus_vecs / norms

    hits = 0
    for q in queries:
        q_resp = client.embeddings.create(model=model, input=[q["query"]])
        q_vec = np.array(q_resp.data[0].embedding)
        q_vec_norm = q_vec / np.linalg.norm(q_vec)
        scores = corpus_vecs_norm @ q_vec_norm
        top_k_ids = [corpus[i]["id"] for i in np.argsort(scores)[::-1][:k]]
        if any(doc_id in top_k_ids for doc_id in q["relevant_doc_ids"]):
            hits += 1

    return hits / len(queries)

Run this across models before you commit. The cost of 1,000 queries × a few candidate models is under $1. The cost of migrating a production system after choosing wrong is re-embedding a multi-million document corpus plus regression testing.

The 2026 model map

OpenAI: the safe default

text-embedding-3-large produces 3072-dimensional vectors with Matryoshka support for truncation to 256, 512, or 1024 dims. It scores in the mid-60s on MTEB and is the safe default for teams already using the OpenAI API — one provider, one billing relationship, consistent latency via the same infrastructure handling your LLM calls.

text-embedding-3-small is 1536 dims at ~6× lower cost. The recall difference is real but modest on typical RAG workloads — on average about 5-8 percentage points on retrieval tasks. If your budget is the constraint and you're not at very high precision thresholds, small is worth evaluating.

At $0.13/million tokens for 3-large, OpenAI sits toward the expensive end of API pricing. The ecosystem integration (direct support in LangChain, LlamaIndex, every major RAG framework) is worth something, but at scale it becomes the reason to look at alternatives.

Cohere: the multimodal option

Cohere Embed v4 (released April 2025) is the most mature production option for embedding text and images in a single vector space — Gemini Embedding accepts both modalities too, but Cohere's single endpoint and unified pricing are operationally simpler for mixed corpora. This matters when your corpus contains PDFs with diagrams, product catalogs with images, or slide decks — content where the visual and textual information both carry retrieval signal.

The model accepts four input types: raw text, base64-encoded images, PDF pages (passed as images), and URL references to images. It produces Matryoshka-compatible embeddings at 256, 512, 1024, and 1536 dims. Pricing is $0.10/million tokens for text, with image input priced by the number of images rather than tokens.

import cohere
import base64

co = cohere.ClientV2()

img_b64 = base64.b64encode(open("product_diagram.png", "rb").read()).decode()

# Embed text and an image into the same vector space (v2 API)
response = co.embed(
    model="embed-v4.0",
    input_type="search_document",
    embedding_types=["float"],
    output_dimension=512,  # Matryoshka truncation
    inputs=[
        {"content": [{"type": "text", "text": "return policy for damaged goods"}]},
        {"content": [{"type": "image_url",
                      "image_url": {"url": f"data:image/png;base64,{img_b64}"}}]},
    ],
)

text_embedding = response.embeddings.float_[0]
image_embedding = response.embeddings.float_[1]
# Both live in the same 512-dim vector space

If you don't have mixed media content, Cohere's text-only quality is competitive but not clearly superior to OpenAI or Gemini at the same price point. The multimodal capability is the differentiator, not the text-only MTEB score.

Gemini: the cost argument

Gemini Embedding at $0.008/million tokens is the cheapest credible API option at mid-2026 prices. At that rate, 5B tokens/month costs $40 instead of the $650 OpenAI text-embedding-3-large would charge. Neither number will bankrupt anyone — the 16x ratio starts to matter when you re-embed a hundred-million-document corpus or run tens of billions of tokens a month.

The MTEB scores are competitive with OpenAI in the mid-60s range. Gemini Embedding has native multilingual support and handles longer documents without chunking penalties as aggressively as some models. The API latency is generally comparable to OpenAI.

The risks: Google has a history of deprecating models with relatively short notice, and the Gemini Embedding model line is newer than OpenAI's, so the production track record is shorter. If you're building a system that will run for 2+ years, factor in re-embedding costs when the model is deprecated. Price in an extra 30-40% for model migration contingency.

For high-volume workloads where the retrieval domain is close to general English text and you don't need multimodal, Gemini is the economic argument that's hard to ignore.

Voyage: the domain specialist

Voyage AI focuses on domain-optimized models: voyage-code-3 for code search, voyage-law-2 for legal retrieval, voyage-finance-2 for financial documents. These are not general-purpose models competing on aggregate MTEB — they're purpose-built for contexts where the vocabulary, syntax, and semantic relationships are sufficiently domain-specific that a general model underperforms.

Voyage models are priced comparably to Cohere for standard input and tend to outperform general models on their target domains by 5-15 percentage points on relevant benchmarks. If you're building a code search tool or a legal document system, evaluate voyage-code-3 or voyage-law-2 specifically.

The downside: narrower ecosystem support, smaller team behind the API, and fewer integration examples in the wild.

Open-source: Qwen3, BGE-M3, nomic

The open-source picture in 2026 is genuinely competitive at the top end.

Qwen3-Embedding-8B tops the MTEB leaderboard at 70.6 as of mid-2026. It's an 8B parameter model — you need a GPU with at least 16GB VRAM to run it, and inference is slower than a small dedicated embedding model. The quality is real, but "top of MTEB" with an 8B model is not the same operation as calling an API endpoint.

BGE-M3 from BAAI is the practical open-source choice for most teams: multilingual (100+ languages), supports dense, sparse, and multi-vector retrieval in the same model pass, and runs on a single A10G or equivalent. Throughput with sentence-transformers and batch size 64 is on the order of 30,000 tokens/second on an A10G — illustrative; measure on your own hardware and sequence lengths, since long documents cut this sharply.

nomic-embed-text-v2 is a strong option for teams that want Apache 2.0 licensing with no restrictions. It scores well on MTEB for its parameter count and has been widely adopted for on-device use cases.

from sentence_transformers import SentenceTransformer
import torch

model = SentenceTransformer("BAAI/bge-m3", device="cuda")

# BGE-M3 supports dense, sparse, and colbert-style multi-vector
# For standard single-vector retrieval:
embeddings = model.encode(
    ["your documents here"],
    batch_size=64,
    normalize_embeddings=True,  # for cosine via dot product
    show_progress_bar=True,
)

Matryoshka truncation: the storage trade-off

Matryoshka Representation Learning (MRL) is a training technique where a model is optimized simultaneously for multiple embedding sizes. The first 256 dimensions of a 3072-dim vector carry more information than a random 256-dim slice — they're the most information-dense prefix. This lets you truncate without re-embedding.

The numbers for OpenAI text-embedding-3-large, per OpenAI's own benchmarks:

Dimensions	Storage per 1M vecs	Relative MTEB performance
3072 (full)	12 GB	100% (baseline)
1536	6 GB	~99.4%
512	2 GB	~97.6%
256	1 GB	~96.5%

The loss figures above are aggregate averages from OpenAI's benchmarks. On easy queries with clear semantic signal, you won't notice any difference. On borderline queries — ambiguous terms, domain-specific jargon, very long documents — the loss compounds and can run 2× the average. Before committing to a truncated dimension, run your evaluation benchmark at the target size on your actual query distribution, not aggregate benchmark numbers.

import numpy as np
from openai import OpenAI

client = OpenAI()

response = client.embeddings.create(
    model="text-embedding-3-large",
    input=["your text here"],
    # API-level truncation via 'dimensions' returns re-normalized vectors.
    # Truncating post-hoc (embedding[:512]) works too, but the prefix is
    # no longer unit-length — re-normalize it yourself before similarity.
    dimensions=512,
)

embedding_512 = response.data[0].embedding  # 512-dim float list

{"type": "rag-pipeline", "title": "Embedding in the retrieval pipeline", "query": "policy"}

Matching the metric to the model

This is where teams quietly destroy recall without raising a single error. See the similarity metrics article for the full treatment, but the quick version:

• OpenAI text-embedding-3 models: vectors come back unit-normalized, so cosine and dot product produce identical rankings and L2 is rank-equivalent — use whichever your database computes fastest. One caveat: a truncated Matryoshka prefix is no longer unit-length. Re-normalize after truncating, or the equivalence quietly breaks.
• Cohere Embed v4: cosine similarity as primary metric.
• BGE-M3: normalize first, then dot product (equivalent to cosine on normalized vectors — faster).
• Gemini Embedding: cosine similarity.

The failure mode is mixing un-normalized vectors with dot product similarity. If you forgot to normalize and you're using dot product, vectors with high magnitude (common words, frequent tokens) dominate rankings regardless of semantic relevance. The results look plausible because the model is still producing reasonable vectors — they're just being ranked incorrectly.

flowchart LR
    A[Raw embedding vectors] --> B{Normalized?}
    B -->|Yes| C["Dot product = Cosine\n(use either — dot is faster)"]
    B -->|No| D{Which metric?}
    D -->|Cosine| E[Correct — angle only]
    D -->|Dot product| F["Magnitude dominates rankings\nSilently wrong results"]
    D -->|L2| G[Sensitive to magnitude\nCheck model guidance]
    style F fill:#ff2e88,color:#fff
    style E fill:#15803d,color:#fff
    style C fill:#15803d,color:#fff

Always check the model's documentation for the recommended metric and use exactly that in your vector database configuration. Qdrant, Weaviate, Pinecone, and Milvus all let you specify the distance function at collection creation — set it once and verify it's right before loading data. Changing it after the fact requires re-indexing.

Queries are not documents: input modes and instruction prefixes

The second silent recall killer sits right next to the metric mismatch. Most 2026 embedding models embed queries and documents differently, and if you push both through the document path, nothing errors — recall just quietly sags.

• Cohere Embed v4: input_type="search_query" for queries, "search_document" for corpus text. The snippet above uses search_document because it embeds corpus content; the query side of the same system must use search_query.
• Gemini Embedding: task_type=RETRIEVAL_QUERY versus RETRIEVAL_DOCUMENT.
• Qwen3-Embedding: expects an instruction prefix on the query side ("Given a web search query, retrieve relevant passages...") and plain text for documents — the model card reports a measurable quality drop when queries go in bare.
• BGE-M3: no instruction needed on either side, but earlier BGE v1.5 models did require the "Represent this sentence..." query prefix. Check the card for whichever version you deploy.
• OpenAI text-embedding-3: symmetric — one input path for both, nothing to get wrong here.

The concrete failure: you embed the corpus with search_document, then embed queries with search_document too because that was the mode already baked into the helper function. Everything runs, similarity scores look plausible, and recall@10 sits well below where it should be. Bake the query/document split into your embedding wrapper's signature — a mode argument with no default — so it physically can't be forgotten.

Latency: the embed call on your critical path

Corpus embedding is a batch job you run overnight. The query embed is different: it sits on the critical path of every single RAG request, in front of the vector search, the reranker, and the LLM call.

Illustrative numbers as of mid-2026 — these drift with region and provider load, so measure your own: hosted embedding APIs return a single short query in roughly 50-150 ms at p50, with p99 excursions of 300-500 ms or worse during provider load spikes. OpenAI, Cohere, and Gemini all sit in the same band, close enough that latency rarely decides between them. A self-hosted model in the same VPC answers in 5-15 ms at p50, which is the strongest latency argument for self-hosting: it removes a third-party network hop from every user request, and it removes the provider's p99 tail from your p99 tail.

If you stay on an API, two mitigations. Cache query embeddings — production traffic repeats itself, and support-style workloads commonly see 20-40% hit rates. And use batch endpoints for corpus work: OpenAI's Batch API halves embedding cost in exchange for asynchronous completion, which is exactly the trade you want for offline re-embedding jobs and exactly the wrong one for the query path.

The self-hosting calculus

API models and self-hosted models are not in competition — they're different trade-offs. Here's the actual math.

Self-hosting cost estimate: BGE-M3 on A10G (24GB VRAM)
────────────────────────────────────────────────────────
Instance cost (AWS g5.2xlarge): ~$1.01/hr = ~$730/mo on-demand
                                 ~$0.50/hr = ~$360/mo reserved (1yr)

Throughput: BGE-M3 batched ≈ 30,000 tokens/sec (illustrative)
Monthly capacity (24hr/day): 30,000 × 3,600 × 24 × 30 ≈ 78B tokens/mo

Effective cost at full utilization: $360/mo ÷ 78,000M ≈ $0.005/1M tokens
────────────────────────────────────────────────────────
Break-even vs API pricing (reserved instance, $360/mo):
  vs OpenAI text-embedding-3-large ($0.13/1M):  ~2.8B tokens/mo
  vs Cohere Embed v4 ($0.10/1M):                ~3.6B tokens/mo
  vs Gemini Embedding ($0.008/1M):              ~45B tokens/mo

The crossover point varies by model and instance type, but the shape is clear: against OpenAI or Cohere pricing, self-hosting starts winning in the low billions of tokens per month; against Gemini, not until tens of billions. That reframes the decision — for most teams, self-hosting is a data-residency, latency, or fine-tuning decision, not a cost play. The hidden costs of self-hosting are model updates, inference server maintenance, GPU availability, and the initial engineering time to stand up a proper serving layer. Add 20-30% to the infrastructure cost for operational overhead.

Data residency is a separate argument entirely. If you're operating in a regulated industry and can't send content to external APIs, self-hosting is not optional regardless of the cost calculation.

The practical self-hosting stack in 2026: vLLM for GPU-batched inference (yes, it handles embedding models, not just generative ones), or text-embeddings-inference from Hugging Face for a purpose-built embedding server with dynamic batching. Both expose an OpenAI-compatible /v1/embeddings endpoint, so switching between your self-hosted model and an API model is a one-line config change.

What breaks

Model version changes invalidate cached embeddings. When OpenAI deprecated text-embedding-ada-002 and replaced it with text-embedding-3-*, the vectors were no longer comparable. Documents embedded with the old model can't be searched with queries embedded with the new model — they're in different geometric spaces. If you upgrade your embedding model, you must re-embed your entire corpus. At 100M documents this is non-trivial. Version-pin your embedding model explicitly and treat a model upgrade as a schema migration with full corpus re-processing.

MTEB ranking reverses on domain-specific data. The model that scores highest on MTEB general benchmarks can rank 3rd or 4th on legal, biomedical, or code-specific retrieval. A 3-4 point MTEB advantage on general benchmarks has been observed to flip to a 10-point disadvantage on specialized corpora. Never pick a model based on aggregate MTEB without domain evaluation.

Matryoshka truncation loss compounds on hard queries. The 2-3% aggregate precision loss looks acceptable. But aggregate means some queries lose 0% and some lose 8-10%. The queries that lose the most are the ones where the model's full representational power was barely sufficient in the first place — borderline relevance, ambiguous terms, short queries against long documents. Before truncating, evaluate specifically on your hard cases.

Embedding the wrong granularity is the most underrated failure. The chunking article covers this in depth, but the embedding model is only as good as what you feed it. A 3072-dim model embedding 4,000-token chunks with irrelevant preamble will underperform a 1536-dim model embedding clean 256-token passages. Chunk quality dominates model quality for RAG retrieval.

High-cardinality filtered search degrades recall silently. When you combine embedding search with metadata filters — user_id = X, date > Y, department = Z — the effective search space shrinks. If you're using HNSW and the filter is highly selective, the graph traversal can no longer find enough valid neighbors and recall drops below 50% with no error or warning. This is an index problem, not an embedding problem, but it shows up as embedding model "failures." See the ANN indexes article and the metadata filtering article for the mechanics.

sequenceDiagram
    participant App as Application
    participant VDB as Vector DB
    participant Idx as HNSW Index

    App->>VDB: query vec + filter: user_id=42
    VDB->>Idx: ANN search top-50
    Idx-->>VDB: 50 candidates (ignoring filter)
    VDB->>VDB: apply user_id=42 filter
    VDB-->>App: 2 results (48 filtered out)
    Note over App,VDB: Silent recall collapse — only 2 results,<br/>but relevant docs for user 42 exist

    App->>VDB: same query, pre-filter enabled
    VDB->>Idx: ANN search within user_id=42 subspace
    Note over Idx: Breaks HNSW graph links —<br/>recall drops 40-60% on high-selectivity filters
    Idx-->>VDB: 10 results from restricted subgraph
    VDB-->>App: 10 results (better but still degraded)

Comparison table

Model	MTEB (mid-2026)	Dims	Price/1M tokens	Multimodal	MRL
OpenAI text-embedding-3-large	~65	3072	$0.13	No	Yes
OpenAI text-embedding-3-small	~62	1536	$0.020	No	Yes
Cohere Embed v4	~64	1536	$0.10	Yes (text+image)	Yes
Gemini Embedding	~65	3072	$0.008	Yes (text+image)	Yes
Voyage voyage-3-large	~66	1024	$0.06	No	No
Voyage voyage-code-3	top for code	1024	$0.06	No	No
Qwen3-Embedding-8B	70.6	4096	N/A (self-host)	No	Yes
BGE-M3	~68	1024	N/A (self-host)	No	No
nomic-embed-text-v2	~64	768	N/A (self-host)	No	Yes

Prices are illustrative and volatile — verify with provider pricing pages before making budget decisions. MTEB scores shift as new model versions land and benchmark tasks are added.

The decision in practice

Start with the question that eliminates the most options fastest.

Mixed text and image content in the corpus? Cohere Embed v4 is your model. Gemini Embedding also accepts text and images, but Cohere's single endpoint and unified pricing make it the more mature choice for mixed corpora today.

Data residency, fine-tuning, or volume in the billions of tokens per month? Self-host. BGE-M3 if you need multilingual, nomic-embed-text-v2 if you need clean Apache 2.0 licensing, Qwen3-Embedding-8B if you need maximum MTEB quality and have the GPU capacity. Use vLLM or text-embeddings-inference with dynamic batching.

General English retrieval on an API? Run the Gemini Embedding vs OpenAI cost comparison against your volume, then run the recall comparison against your domain. If Gemini recall is within 2-3 points of OpenAI on your data, the 16x cost difference is usually the deciding factor. If your domain is close to general web text, Gemini often matches or beats OpenAI on recall because of its multilingual pretraining breadth.

Specialized domain (code, legal, finance)? Evaluate Voyage's domain-specific models before defaulting to a general model. A 10-point recall improvement on your exact workload is worth more than any aggregate leaderboard position.

Need Matryoshka? Use OpenAI text-embedding-3-large, Cohere Embed v4, Gemini Embedding, or Qwen3-Embedding. Set the dimension at the lowest value where your recall evaluation shows acceptable performance. For most RAG workloads, 512 dims is the practical sweet spot — 6x storage savings, under 3% recall loss.

Whatever you choose, pin the exact model version in your embedding pipeline config, store it alongside the vectors in your metadata, and write a re-embedding job before you need it. The next model deprecation is coming, and you want to be the team that handles it in a planned migration rather than an emergency.

For the downstream retrieval mechanics — how your vector database indexes these embeddings, which ANN algorithm to use, and how to add sparse keyword search on top — see ANN Indexes Under the Hood, Vector Database Landscape 2026, and Hybrid Search: Fusing Dense Vectors and Sparse BM25.