Multi-Tenancy — Silo, Pool, and the Noisy Neighbor Problem

The support ticket that ends careers is short. "Why does our dashboard show a company called Meridian Health in the account picker? We are not Meridian Health." Nothing is down. Every graph is green. And you are now drafting an email to two customers explaining that, for a window of time you cannot yet bound, one of them could see the other's data.

Every company that sells software to more than one customer has already made the decision that makes this bug possible. Should each customer get their own copy of the system, or should they all share one? A copy per customer is easy to reason about — Meridian physically cannot appear in anyone else's account picker if Meridian has its own database. It is also linearly expensive: a hundred customers means a hundred deployments to patch, monitor, migrate, and pay for. Sharing one system is dramatically cheaper, which is why almost everything you use — Slack, Notion, Salesforce, your bank's SaaS vendors — is multi-tenant: one deployment, many customers, each customer a tenant.

The price of sharing is that isolation stops being a property of the infrastructure and becomes a property of your code. This article is about what that costs and how to pay it deliberately.

A tenant is a boundary, not a user

A tenant is the customer organization: the Slack workspace, the GitHub org, the Shopify store. A user is a person inside one. The distinction matters because the two are different units of almost everything:

• Isolation — tenant A's rows must never reach tenant B, but users within a tenant are expected to see shared data. That inner problem is authorization, and it has its own machinery — see designing an authorization system. Tenancy is the outer wall.
• Billing and quotas — plans, seat counts, and rate limits attach to the tenant, not the person.
• Configuration — feature flags, SSO settings, data retention policies, custom domains: per tenant.
• Lifecycle — tenants onboard, upgrade, churn, and demand their data exported and deleted. Users just come and go.

One person can belong to many tenants — your email address in two Slack workspaces is two memberships with two identities, and nothing you do in one is visible in the other. In B2C products the tenant often collapses to the individual account, which is why B2C engineers sometimes think multi-tenancy doesn't apply to them. It does; their tenants are just very small and very numerous.

The useful mental move is to treat the tenant as the unit you would want to move, restore, throttle, or delete as one thing. Every architectural question that follows — where data lives, who shares compute, how far a failure spreads — is a question about that boundary.

The spectrum: silo, bridge, pool

AWS's SaaS practice popularized three names for the options, and they are worth adopting because they end arguments quickly. A silo gives a tenant dedicated infrastructure. A pool shares everything, with tenancy enforced by data. A bridge mixes the two — most commonly a shared, stateless application tier in front of per-tenant data stores.

At the data tier, the spectrum has three practical stops:

flowchart LR
    subgraph G1["Pool — shared tables"]
    direction TB
    PD[("one schema\nevery row tagged tenant_id")]
    PN["cheapest, elastic\nisolation is a WHERE clause"]
    end
    subgraph G2["Bridge — schema per tenant"]
    direction TB
    BD[("acme.invoices\nbolt.invoices\ncato.invoices")]
    BN["clean on paper\ncatalog bloat + N× DDL at scale"]
    end
    subgraph G3["Silo — database per tenant"]
    direction TB
    SD[("acme-db, bolt-db, ...")]
    SN["strongest isolation\ncost and ops scale with N"]
    end
    style PD fill:#15803d,color:#fff
    style BD fill:#b45309,color:#fff
    style SD fill:#0e7490,color:#fff

	Shared tables (pool)	Schema per tenant (bridge)	Database per tenant (silo)
Isolation enforced by	WHERE tenant_id = ? + RLS	Schema grants, search_path	Physical separation
Cost at 1,000 tenants	One cluster	One overloaded catalog	~10× the pooled cluster
One schema migration	1 run	1,000 DDL runs	1,000 orchestrated runs
Per-tenant restore	Row surgery, painful	pg_dump one schema	Trivial — restore the DB
Noisy neighbor risk	Highest	Shared buffer pool anyway	Contained
Compliance / residency story	Hardest to tell	Middling	Easiest
Tenant onboarding	INSERT a row	CREATE SCHEMA + migrate	Provision infrastructure

The cost gap deserves real numbers, because it is the entire reason pooled architectures win:

Data tier for 1,000 tenants (illustrative mid-2026 AWS list prices)

Database-per-tenant:
  smallest sensible RDS Postgres (db.t4g.micro + 20 GB gp3)  ≈ $15/mo
  × 1,000 tenants                                            ≈ $15k/mo ≈ $180k/yr
  ...while ~950 of those instances idle below 1% CPU,
  and every release ships 1,000 schema migrations

Pooled:
  one 3-node Postgres cluster (r6g.xlarge primary + 2 replicas) ≈ $1.5k/mo
  one migration, one backup policy, one pager target
  ≈ 10× cheaper — the margin story of SaaS in one line

Schema-per-tenant looks like the adult compromise and mostly isn't. At tens of tenants it's fine. At thousands, the database's own catalog becomes the problem: 5,000 schemas × 200 tables is a million catalog entries, pg_dump and migration tooling crawl, connection pooling fights search_path, and ORM migrations were never designed to run 5,000 times per deploy. Teams that pick it at four digits of tenants usually end up writing the blog post about leaving it. Salesforce, the most famous multi-tenant system in existence, runs the opposite extreme — everyone pooled in shared tables, with an OrgID on every row and a metadata layer instead of per-tenant schema changes.

The honest default: pool for the many, silo for the few. Slack serves every workspace from a shared application tier over Vitess-sharded MySQL. Notion pools workspaces across sharded Postgres. Both would also sell your bank a very different deployment story, because the silo tier is a product ("enterprise, dedicated instance") as much as an architecture.

Compute shares the same spectrum

The database gets all the attention, but every shared component makes the same choice. A stateless app fleet is naturally pooled — requests from all tenants interleave on the same processes. Background workers, caches, queues, and search indexes are where tenancy quietly matters:

• Cache: one Redis for everyone, or a keyspace prefix per tenant, or (silo tier) a cache per tenant. An unprefixed key is a leak waiting to happen — more below.
• Queues: one shared queue means one tenant's 50,000-message backfill delays everyone's password-reset email. Per-tenant or per-tier queues restore fairness — this is the same producer/consumer fairness problem covered in backpressure and flow control.
• Search: one shared index with a mandatory tenant filter term, or an index per large tenant. Filters forgotten in search are exactly as bad as filters forgotten in SQL.
• Third-party APIs: your Stripe or SES account has global rate limits; one tenant's burst can exhaust a quota that all tenants share. The noisy neighbor problem doesn't stop at your own infrastructure.

Silo-tier compute exists too — dedicated worker pools or even dedicated clusters for tenants that pay for it — but the stateless tier is usually the last thing worth dedicating, because it holds no data and scales elastically. Dedicate state first.

Keeping the pool honest

In a pooled schema, isolation is a promise your code makes on every single query. Three layers make the promise hard to break.

Layer 1: the column. Every tenant-owned table carries tenant_id, non-null, indexed as the leading column of most indexes ((tenant_id, created_at), not (created_at)). Composite foreign keys extend the discipline to relationships:

CREATE TABLE invoice_lines (
  tenant_id  uuid   NOT NULL,
  invoice_id bigint NOT NULL,
  amount_cents bigint NOT NULL,
  FOREIGN KEY (tenant_id, invoice_id)
    REFERENCES invoices (tenant_id, id)
);

That foreign key makes a cross-tenant reference unrepresentable — an invoice line cannot point at another tenant's invoice no matter what the application does.

Layer 2: the scoped repository. Application code never writes WHERE tenant_id = ? by hand; a repository or ORM scope injects it from ambient tenant context. Hand-written filters are fine until the 2 a.m. analytics query that joins four tables and forgets one.

Layer 3: row-level security, the backstop. Postgres RLS makes the database enforce what the application intends:

ALTER TABLE invoices ENABLE ROW LEVEL SECURITY;
ALTER TABLE invoices FORCE ROW LEVEL SECURITY;  -- applies to the table owner too

CREATE POLICY tenant_isolation ON invoices
  USING (tenant_id = current_setting('app.tenant_id', true)::uuid);

Each request runs inside a transaction that pins the tenant:

BEGIN;
SET LOCAL app.tenant_id = '7f9c1e2a-...';
SELECT * FROM invoices WHERE status = 'open';  -- policy appends the tenant filter
COMMIT;

SET LOCAL scopes the setting to the transaction, which is what makes this safe behind a transaction-mode connection pooler like PgBouncer — the setting cannot bleed into another tenant's request on the same connection. If the setting is missing, current_setting(..., true) returns NULL and the policy matches nothing: the query fails closed with zero rows instead of failing open with everyone's rows. (Zero rows is also a silent way to be wrong, so raise loudly in development when context is absent.) Overhead for a simple indexed equality policy is typically low single-digit percent; complex policies with subqueries can be far worse — measure yours. And know the escape hatches: superusers and roles with BYPASSRLS skip policies entirely, which is precisely what you want for admin tooling and precisely what you must keep out of the application's credentials.

For tenants with regulatory weight, add a fourth layer: a KMS encryption key per tenant (envelope encryption). It turns "delete this tenant's data everywhere, including backups" from an archaeology project into a key deletion — crypto-shredding — and it gives the enterprise sales deck a true sentence about tenant data being separately encrypted.

The noisy neighbor problem

Dedicated hardware gave every customer a fenced yard. The pool gives them a shared swimming lane, and some customers swim wide:

Shared Postgres, max_connections = 200, app pool = 150

"Export to CSV" runs 8 parallel queries, ~30 s each.
One enterprise customer bulk-triggers exports for 30 workspaces:
  30 exports × 8 queries = 240 connection slots demanded
  → pool exhausted; every tenant's 5 ms query now queues behind 30 s scans
  → product-wide p99: 80 ms → 30,000 ms. Every tenant. One customer.

The fix is not a bigger pool (they'll trigger 60 next quarter):
  per-tenant cap of 10 concurrent heavy queries + a separate export queue
  → that tenant's exports take longer; nobody else ever notices

flowchart LR
    TEN_A["Tenant A\nexport storm, 240 queries"] --> GATE{"Per-tenant\nconcurrency caps"}
    TEN_B["Tenant B\nnormal traffic"] --> GATE
    TEN_C["Tenant C\nnormal traffic"] --> GATE
    GATE -->|"A capped at 10, rest queued"| DBPOOL["Shared DB pool\n200 connections"]
    GATE -->|"B and C flow untouched"| DBPOOL
    style TEN_A fill:#ff2e88,color:#111
    style GATE fill:#ffaa00,color:#111
    style DBPOOL fill:#0e7490,color:#fff

Mitigation is a toolkit, and every tool is keyed by tenant ID:

• Rate limits at the edge — token buckets per tenant (per plan tier, really), enforced at the gateway. The algorithms are the ones from design a rate limiter; the tenancy twist is only the key.
• Concurrency caps on expensive operations: reports, exports, bulk imports, AI jobs. Queue the excess instead of rejecting it — the tenant sees "slower," not "broken."
• Fair scheduling for background work: round-robin or weighted-fair across tenant queues, so a 50,000-job backfill from one tenant interleaves with — rather than precedes — everyone else's two jobs.
• Tier segregation: free-tier traffic on one worker pool, enterprise on another. Fairness within a pool, insulation between pools.

There's a prerequisite hiding under all of these: per-tenant accounting. If your metrics can't answer "which tenant is generating this load," you will diagnose your next brownout by grepping logs during the incident. Tag request metrics, slow-query logs, and job runtimes with tenant ID from day one. (Watch metric cardinality at five digits of tenants — track the top-N tenants individually and the long tail in aggregate.) The same accounting later becomes cost attribution, which is how you discover that one design-partner tenant costs more to serve than your ten biggest contracts pay.

Blast radius: rings, cells, and shuffle sharding

Sharing couples failure. A bad deploy on a single-tenant estate breaks one customer; on a pooled system it breaks all of them at once, which turns a bug into an outage and an outage into a churn event. Three structures cut the coupling back down.

Deployment rings. Roll out by tenant cohort: internal tenants → free tier → paid → enterprise, with bake time between rings. The blast radius of a bad release becomes "the ring it reached," and your most sensitive customers sit in the last ring behind days of soak time.

Cells. Partition the whole stack — app fleet, database, cache — into self-contained cells of, say, 500 tenants each, with a thin routing layer directing each tenant to its cell. Shopify calls them pods and assigns every shop to one; a pod-wide failure caps at that pod's shops. Cells also bound data-tier growth: when a cell's database fills, you add a cell rather than resharding the world — the per-tenant version of the strategies in database sharding. The routing layer (tenant → cell) is the one genuinely global component, so it must be boring: a small, cached, replicated directory service that does nothing else.

Shuffle sharding. Within a shared fleet, don't give every tenant access to every worker — give each tenant a random small subset:

flowchart TD
    P["Poison tenant\nassigned w1 + w4"] --> W1
    P --> W4
    A["Tenant A\nassigned w2 + w5"] --> W2
    A --> W5
    B["Tenant B\nassigned w4 + w7"] --> W4
    B --> W7
    subgraph FLEET["Worker fleet — 8 nodes, C(8,2) = 28 possible pairs"]
    W1["w1 — down"]
    W2[w2]
    W3[w3]
    W4["w4 — down"]
    W5[w5]
    W6[w6]
    W7[w7]
    W8[w8]
    end
    style W1 fill:#dc2626,color:#fff
    style W4 fill:#dc2626,color:#fff
    style P fill:#ff2e88,color:#111

With 8 workers and 2 per tenant, there are C(8,2) = 28 distinct pairs. A poison tenant — one whose requests crash or saturate whatever serves them — takes down its own pair only. The chance that another tenant shares both workers is 1/28 ≈ 3.6%; most tenants (like Tenant B above) overlap on at most one node and degrade instead of dying, and retries route around the sick worker. Scale the numbers and the isolation gets absurdly good: 100 workers choosing 5 gives ~75 million combinations. AWS Route 53 runs this play with 2,048 virtual name servers, 4 per hosted zone, so a DDoS against one customer's domain leaves other customers' DNS standing. Tenant-to-subset assignment is just hashing — consistent hashing if subsets must survive fleet resizes.

Tenant context: the thread through everything

All of the machinery above consumes one input: which tenant is this? That fact is born in exactly one trustworthy place — the verified auth token — and must survive every hop after it.

sequenceDiagram
    autonumber
    participant U as Browser
    participant GW as API gateway
    participant SVC as Invoice service
    participant PG as Postgres with RLS
    U->>GW: GET /invoices with JWT claim org_id = acme
    GW->>SVC: request + verified tenant claim
    SVC->>PG: BEGIN then SET LOCAL app.tenant_id = 'acme'
    SVC->>PG: SELECT * FROM invoices WHERE status = 'open'
    Note over PG: policy appends tenant_id = 'acme'<br/>a forgotten filter returns zero rows, not a leak
    PG-->>SVC: Acme rows only
    SVC-->>U: 200 OK

In-process, ambient context beats parameter-threading — Node's AsyncLocalStorage, Python's contextvars, a request-scoped bean, goroutine context:

import { AsyncLocalStorage } from 'node:async_hooks';

const tenantCtx = new AsyncLocalStorage<{ tenantId: string }>();

app.use((req, _res, next) => {
  // From the *verified* JWT. Never from a header the client controls,
  // never from the request body, never from a query parameter.
  tenantCtx.run({ tenantId: req.auth.claims.org_id }, next);
});

export function currentTenant(): string {
  const ctx = tenantCtx.getStore();
  if (!ctx) throw new Error('no tenant context — refusing to touch data');
  return ctx.tenantId;
}

The repository layer, the cache client, the job enqueuer, and the logger all call currentTenant(). Two hops need special care, because ambient context does not cross them by itself:

• Queues. A job payload must carry tenantId explicitly, and the worker must rehydrate context before running. The classic leak is the ops script that replays a dead-letter queue under the wrong — or no — tenant.
• Caches. Every key gets the tenant prefix: acme:user:42:dashboard, never user:42:dashboard. In a shared cache, an unscoped key is a cross-tenant read primitive with sub-millisecond latency.

Logs and traces get the tenant tag too — not for isolation but for operability: "is this outage all tenants or one?" is the first question of every incident, and it should be a filter, not an investigation.

Operating a fleet of tenants

A few realities of tenancy only show up after launch.

The tenant directory is a real system. Somewhere a control plane maps tenant → shard/cell, plan tier, feature flags, encryption key, data-residency region, and lifecycle state. Everything consults it; it must be small, cached, and more available than anything behind it.

Migrations multiply. Pooled: one ALTER TABLE (kept online-safe, since it now rewrites every tenant's rows at once). Siloed: N migrations, some of which will fail mid-fleet — so you track a schema version per tenant and make the app tolerate one version of skew, because at any given moment some database in the fleet is mid-migration.

Residency is routing. "EU tenant data stays in the EU" means tenant → region is one more directory attribute, decided at signup, enforced at the routing layer — and the hardest part is the global surfaces (search, analytics, ML training sets) that quietly want to aggregate everything in one place.

Offboarding is a feature. Tenants leave. Contracts and regulators require export (a complete, machine-readable dump of one tenant's data — trivial in a silo, a real project in a pool) and deletion with proof. Per-tenant encryption keys turn the deletion story from "we scrubbed 14 tables and hope the backups age out" into "we destroyed the key."

What breaks

The missing filter. A hand-written reporting query joins four tables and scopes three of them. Code review misses it; the ORM wasn't involved; it ships. This is the argument for RLS in one sentence: the database catches the query your review didn't.

The unscoped cache key. In March 2023, a Redis client library bug caused canceled requests to receive responses belonging to other users — some ChatGPT users saw titles from other users' chat histories, and a few saw another subscriber's billing details. No SQL was harmed; the leak rode the shared cache and connection reuse. Tenant-prefix every key, and treat "shared connection + interleaved responses" as part of your isolation surface.

The job that lost its thread. A nightly reconciliation job reads a config table without tenant scoping "because it runs for all tenants anyway," then writes results keyed by a tenant_id variable that survived from the previous loop iteration. Cross-tenant writes are rarer than cross-tenant reads and strictly worse.

The whale. Pooling assumes tenants are small relative to the pool. Then one tenant grows to 100× the median — their workspace is 40% of a shard, their exports dominate the queue, their p99 is your p99. Every pooled architecture eventually grows a "graduate a tenant to a silo" runbook; build the tenant-move tool before you need it during an incident.

The enumerable ID. /invoices/48291 with a sequential global ID and an ownership check that only verifies the invoice exists — the oldest cross-tenant vulnerability on the web (IDOR). Scope the lookup (WHERE tenant_id = ? AND id = ?), and prefer non-guessable IDs so a bug in the check isn't also a directory listing.

The shared limit nobody owned. One tenant's email campaign consumes the platform's entire SES send quota; every tenant's password resets bounce. Inventory the quotas you share upstream — email, SMS, payment APIs, LLM tokens — and budget them per tenant like any other pooled resource.

The decision in practice

Start pooled: shared tables, tenant_id on every row, RLS from the first migration, per-tenant metrics and rate limits from day one. These cost almost nothing at 10 tenants and are excruciating to retrofit at 10,000 — Salesforce-scale pooling works because the discipline was never optional. Keep the tenant directory separate and boring. When one database stops scaling, shard or cell by tenant, and when sales brings you a hospital, a government, or a whale, graduate them to the silo tier — and charge for it, because isolation is a product feature that happens to be an architecture.

flowchart TD
    D1{"Regulated data or a contract\nthat demands physical isolation?"} -->|"those tenants"| S1["Silo tier\ndedicated DB, own keys, own region"]
    D1 -->|no| D2{"Tenant count?"}
    D2 -->|"tens, high-value"| S2["Database per tenant is affordable\ntake the easy restore + deletion wins"]
    D2 -->|"thousands"| D3{"Whales much bigger\nthan the median tenant?"}
    D3 -->|yes| S3["Pool + RLS for the long tail\ngraduate whales to silos"]
    D3 -->|no| S4["Pool + RLS everywhere\ncells when one DB stops scaling"]
    style S1 fill:#0e7490,color:#fff
    style S3 fill:#22c55e,color:#111
    style S4 fill:#22c55e,color:#111

The trap to avoid is the unprincipled middle: schema-per-tenant everywhere, or silos handed out ad hoc to whoever asked loudest, until you run a zoo of half-shared systems with the costs of both models and the guarantees of neither. Pick the pool as the default, the silo as the exception you charge for, and hold the line in between.

Things you should now be able to answer

• What exactly is a tenant, and why is it a different unit than a user?
• Walk through silo, bridge, and pool — what does each cost at 1,000 tenants, and what breaks first in each?
• A customer demands their data be provably isolated and deletable. Which layers deliver that in a pooled system?
• One tenant's export feature just took product-wide p99 from 80 ms to 30 s. What do you deploy this week, and what do you build this quarter?
• How does shuffle sharding reduce shared fate, and what do the numbers look like for 8 workers with 2 per tenant?

Further reading

• AWS Well-Architected Framework, SaaS Lens — the silo/bridge/pool vocabulary and tiering patterns
• Salesforce, The Force.com Multitenant Architecture whitepaper — the canonical pooled design: OrgID on every row, metadata instead of schema changes
• Colm MacCárthaigh / AWS Builders' Library, Workload isolation using shuffle-sharding — the combinatorics of blast radius, with the Route 53 story
• Notion engineering, Herding elephants: lessons learned from sharding Postgres (2021) and The Great Re-shard (2023) — pooled tenancy meeting physical limits
• Slack engineering, Scaling Datastores at Slack with Vitess — the migration from workspace-sharded MySQL to flexible shard keys behind a pooled application tier
• PostgreSQL documentation, Row Security Policies — the exact semantics of USING, FORCE, and BYPASSRLS
• OpenAI, March 20 ChatGPT outage postmortem — the cache-layer cross-user leak, in the vendor's own words