# Multi-Tenancy — Silo, Pool, and the Noisy Neighbor Problem > Should each customer get their own stack or share one? Tenant isolation models, noisy neighbors, blast radius, and how cross-tenant leaks actually happen. Canonical: https://ironclad.academy/system-design/articles/multi-tenancy-architecture | Difficulty: medium | Topics: Architecture, Isolation, Database, Scalability, Security Asked at: Salesforce, AWS, Slack, Notion, Shopify ## Key takeaways - Multi-tenancy is a spectrum, not a binary: pool (shared tables + tenant_id), bridge (shared app over per-tenant data), silo (dedicated stack) — and mature SaaS products run all three at once, tiered by plan. - Pooled storage at 1,000 tenants is roughly 10× cheaper than database-per-tenant, and you pay the difference in engineering discipline: every query, cache key, job, and search index must carry tenant context. - Postgres row-level security is the standard backstop for shared tables: one policy catches every query that forgets the tenant filter, at low single-digit percent overhead for an indexed equality check. - Noisy neighbors are structural in shared pools — per-tenant rate limits, concurrency caps, and fair queuing rebuild the fairness that dedicated hardware used to give you for free. - Shuffle sharding cuts shared fate combinatorially: assigning each tenant 2 of 8 workers means a poison tenant fully overlaps with only 1 in 28 other tenants. - Most cross-tenant leaks are not exotic attacks — they are one missing filter: an unscoped cache key, a search query without a tenant term, a background job that lost its context. > **In one line:** Multi-tenancy is one deployment serving every customer, with isolation rebuilt in software — a tenant ID threaded through every query, cache key, queue, and log line, and a deliberate per-tier choice between sharing (cheap) and dedicating (safe). **The problem.** A copy of the system per customer is easy to reason about and linearly expensive; one shared system is ~10× cheaper and couples every customer's fate to every other's. Once tenants share a database and a worker fleet, three things become your job that physics used to do for you: keeping their data apart, keeping one tenant's load from starving the rest, and keeping one bad deploy from hitting everyone at once. **The spectrum.** ```mermaid flowchart LR subgraph SILO["Silo — dedicated stack"] SA[Acme app] --> SD[(Acme DB)] SB[Bolt app] --> SE[(Bolt DB)] end subgraph BRIDGE["Bridge — shared app, split data"] BA[Shared app fleet] --> BD[(Acme schema)] BA --> BE[(Bolt schema)] end subgraph POOL["Pool — shared everything"] PA[Shared app fleet] --> PD[("One DB\nevery row tagged tenant_id")] end style SD fill:#0e7490,color:#fff style SE fill:#0e7490,color:#fff style PD fill:#15803d,color:#fff ``` **Key decisions.** | Question | Answer | | --- | --- | | Default data model? | Pooled tables + `tenant_id` on every row, Postgres RLS as the backstop | | When silo? | Compliance, data residency, whale tenants — or a contract that pays for it | | Noisy neighbor fix? | Per-tenant token buckets, concurrency caps, fair queuing — accounting first | | Blast radius control? | Deploy in tenant rings; cells of N tenants; shuffle sharding on shared fleets | | Where does tenant context come from? | The verified auth claim only — never a request parameter | | How do leaks actually happen? | A missing WHERE, an unscoped cache key, a job that lost its context | **If you have 60 seconds, say this.** "A tenant is the unit of isolation, billing, and configuration — a company, not a user. Data placement is a spectrum: pooled tables with a tenant_id column (cheapest, isolation is a WHERE clause), schema- or database-per-tenant (cleaner isolation, operations scale with N), or a full dedicated stack. Default to pooled with row-level security as the backstop, and carve out silos for regulated or whale tenants — most real products tier it. Sharing creates the noisy neighbor problem, so you add per-tenant quotas and fair queuing; it also widens blast radius, so you deploy in rings and partition tenants into cells, with shuffle sharding where the fleet is shared. And one tenant ID must flow from the auth token through every query, cache key, queue message, and log line — every famous cross-tenant leak is a place where that thread broke." The support ticket that ends careers is short. "Why does our dashboard show a company called Meridian Health in the account picker? We are not Meridian Health." Nothing is down. Every graph is green. And you are now drafting an email to two customers explaining that, for a window of time you cannot yet bound, one of them could see the other's data. Every company that sells software to more than one customer has already made the decision that makes this bug possible. Should each customer get their own copy of the system, or should they all share one? A copy per customer is easy to reason about — Meridian physically cannot appear in anyone else's account picker if Meridian has its own database. It is also linearly expensive: a hundred customers means a hundred deployments to patch, monitor, migrate, and pay for. Sharing one system is dramatically cheaper, which is why almost everything you use — Slack, Notion, Salesforce, your bank's SaaS vendors — is **multi-tenant**: one deployment, many customers, each customer a **tenant**. The price of sharing is that isolation stops being a property of the infrastructure and becomes a property of your code. This article is about what that costs and how to pay it deliberately. ## A tenant is a boundary, not a user A **tenant** is the customer organization: the Slack workspace, the GitHub org, the Shopify store. A **user** is a person inside one. The distinction matters because the two are different units of almost everything: - **Isolation** — tenant A's rows must never reach tenant B, but users *within* a tenant are expected to see shared data. That inner problem is authorization, and it has its own machinery — see [designing an authorization system](/system-design/articles/design-authorization-system). Tenancy is the outer wall. - **Billing and quotas** — plans, seat counts, and rate limits attach to the tenant, not the person. - **Configuration** — feature flags, SSO settings, data retention policies, custom domains: per tenant. - **Lifecycle** — tenants onboard, upgrade, churn, and demand their data exported and deleted. Users just come and go. One person can belong to many tenants — your email address in two Slack workspaces is two memberships with two identities, and nothing you do in one is visible in the other. In B2C products the tenant often collapses to the individual account, which is why B2C engineers sometimes think multi-tenancy doesn't apply to them. It does; their tenants are just very small and very numerous. The useful mental move is to treat the tenant as the unit you would want to move, restore, throttle, or delete *as one thing*. Every architectural question that follows — where data lives, who shares compute, how far a failure spreads — is a question about that boundary. ## The spectrum: silo, bridge, pool AWS's SaaS practice popularized three names for the options, and they are worth adopting because they end arguments quickly. A **silo** gives a tenant dedicated infrastructure. A **pool** shares everything, with tenancy enforced by data. A **bridge** mixes the two — most commonly a shared, stateless application tier in front of per-tenant data stores. At the data tier, the spectrum has three practical stops: ```mermaid flowchart LR subgraph G1["Pool — shared tables"] direction TB PD[("one schema\nevery row tagged tenant_id")] PN["cheapest, elastic\nisolation is a WHERE clause"] end subgraph G2["Bridge — schema per tenant"] direction TB BD[("acme.invoices\nbolt.invoices\ncato.invoices")] BN["clean on paper\ncatalog bloat + N× DDL at scale"] end subgraph G3["Silo — database per tenant"] direction TB SD[("acme-db, bolt-db, ...")] SN["strongest isolation\ncost and ops scale with N"] end style PD fill:#15803d,color:#fff style BD fill:#b45309,color:#fff style SD fill:#0e7490,color:#fff ``` | | Shared tables (pool) | Schema per tenant (bridge) | Database per tenant (silo) | | --- | --- | --- | --- | | Isolation enforced by | `WHERE tenant_id = ?` + RLS | Schema grants, `search_path` | Physical separation | | Cost at 1,000 tenants | One cluster | One overloaded catalog | ~10× the pooled cluster | | One schema migration | 1 run | 1,000 DDL runs | 1,000 orchestrated runs | | Per-tenant restore | Row surgery, painful | `pg_dump` one schema | Trivial — restore the DB | | Noisy neighbor risk | Highest | Shared buffer pool anyway | Contained | | Compliance / residency story | Hardest to tell | Middling | Easiest | | Tenant onboarding | `INSERT` a row | `CREATE SCHEMA` + migrate | Provision infrastructure | The cost gap deserves real numbers, because it is the entire reason pooled architectures win: ```text Data tier for 1,000 tenants (illustrative mid-2026 AWS list prices) Database-per-tenant: smallest sensible RDS Postgres (db.t4g.micro + 20 GB gp3) ≈ $15/mo × 1,000 tenants ≈ $15k/mo ≈ $180k/yr ...while ~950 of those instances idle below 1% CPU, and every release ships 1,000 schema migrations Pooled: one 3-node Postgres cluster (r6g.xlarge primary + 2 replicas) ≈ $1.5k/mo one migration, one backup policy, one pager target ≈ 10× cheaper — the margin story of SaaS in one line ``` Schema-per-tenant looks like the adult compromise and mostly isn't. At tens of tenants it's fine. At thousands, the database's own catalog becomes the problem: 5,000 schemas × 200 tables is a million catalog entries, `pg_dump` and migration tooling crawl, connection pooling fights `search_path`, and ORM migrations were never designed to run 5,000 times per deploy. Teams that pick it at four digits of tenants usually end up writing the blog post about leaving it. Salesforce, the most famous multi-tenant system in existence, runs the opposite extreme — everyone pooled in shared tables, with an `OrgID` on every row and a metadata layer instead of per-tenant schema changes. The honest default: **pool for the many, silo for the few**. Slack serves every workspace from a shared application tier over Vitess-sharded MySQL. Notion pools workspaces across sharded Postgres. Both would also sell your bank a very different deployment story, because the silo tier is a product ("enterprise, dedicated instance") as much as an architecture. ## Compute shares the same spectrum The database gets all the attention, but every shared component makes the same choice. A stateless app fleet is naturally pooled — requests from all tenants interleave on the same processes. Background workers, caches, queues, and search indexes are where tenancy quietly matters: - **Cache**: one Redis for everyone, or a keyspace prefix per tenant, or (silo tier) a cache per tenant. An unprefixed key is a leak waiting to happen — more below. - **Queues**: one shared queue means one tenant's 50,000-message backfill delays everyone's password-reset email. Per-tenant or per-tier queues restore fairness — this is the same producer/consumer fairness problem covered in [backpressure and flow control](/system-design/articles/backpressure-and-flow-control). - **Search**: one shared index with a mandatory tenant filter term, or an index per large tenant. Filters forgotten in search are exactly as bad as filters forgotten in SQL. - **Third-party APIs**: your Stripe or SES account has global rate limits; one tenant's burst can exhaust a quota that all tenants share. The noisy neighbor problem doesn't stop at your own infrastructure. Silo-tier compute exists too — dedicated worker pools or even dedicated clusters for tenants that pay for it — but the stateless tier is usually the last thing worth dedicating, because it holds no data and scales elastically. Dedicate state first. ## Keeping the pool honest In a pooled schema, isolation is a promise your code makes on every single query. Three layers make the promise hard to break. **Layer 1: the column.** Every tenant-owned table carries `tenant_id`, non-null, indexed as the leading column of most indexes (`(tenant_id, created_at)`, not `(created_at)`). Composite foreign keys extend the discipline to relationships: ```sql CREATE TABLE invoice_lines ( tenant_id uuid NOT NULL, invoice_id bigint NOT NULL, amount_cents bigint NOT NULL, FOREIGN KEY (tenant_id, invoice_id) REFERENCES invoices (tenant_id, id) ); ``` That foreign key makes a cross-tenant reference *unrepresentable* — an invoice line cannot point at another tenant's invoice no matter what the application does. **Layer 2: the scoped repository.** Application code never writes `WHERE tenant_id = ?` by hand; a repository or ORM scope injects it from ambient tenant context. Hand-written filters are fine until the 2 a.m. analytics query that joins four tables and forgets one. **Layer 3: row-level security, the backstop.** Postgres RLS makes the database enforce what the application intends: ```sql ALTER TABLE invoices ENABLE ROW LEVEL SECURITY; ALTER TABLE invoices FORCE ROW LEVEL SECURITY; -- applies to the table owner too CREATE POLICY tenant_isolation ON invoices USING (tenant_id = current_setting('app.tenant_id', true)::uuid); ``` Each request runs inside a transaction that pins the tenant: ```sql BEGIN; SET LOCAL app.tenant_id = '7f9c1e2a-...'; SELECT * FROM invoices WHERE status = 'open'; -- policy appends the tenant filter COMMIT; ``` `SET LOCAL` scopes the setting to the transaction, which is what makes this safe behind a transaction-mode connection pooler like PgBouncer — the setting cannot bleed into another tenant's request on the same connection. If the setting is missing, `current_setting(..., true)` returns NULL and the policy matches nothing: the query **fails closed** with zero rows instead of failing open with everyone's rows. (Zero rows is also a silent way to be wrong, so raise loudly in development when context is absent.) Overhead for a simple indexed equality policy is typically low single-digit percent; complex policies with subqueries can be far worse — measure yours. And know the escape hatches: superusers and roles with `BYPASSRLS` skip policies entirely, which is precisely what you want for admin tooling and precisely what you must keep out of the application's credentials. For tenants with regulatory weight, add a fourth layer: **a KMS encryption key per tenant** (envelope encryption). It turns "delete this tenant's data everywhere, including backups" from an archaeology project into a key deletion — crypto-shredding — and it gives the enterprise sales deck a true sentence about tenant data being separately encrypted. ## The noisy neighbor problem Dedicated hardware gave every customer a fenced yard. The pool gives them a shared swimming lane, and some customers swim wide: ```text Shared Postgres, max_connections = 200, app pool = 150 "Export to CSV" runs 8 parallel queries, ~30 s each. One enterprise customer bulk-triggers exports for 30 workspaces: 30 exports × 8 queries = 240 connection slots demanded → pool exhausted; every tenant's 5 ms query now queues behind 30 s scans → product-wide p99: 80 ms → 30,000 ms. Every tenant. One customer. The fix is not a bigger pool (they'll trigger 60 next quarter): per-tenant cap of 10 concurrent heavy queries + a separate export queue → that tenant's exports take longer; nobody else ever notices ``` ```mermaid flowchart LR TEN_A["Tenant A\nexport storm, 240 queries"] --> GATE{"Per-tenant\nconcurrency caps"} TEN_B["Tenant B\nnormal traffic"] --> GATE TEN_C["Tenant C\nnormal traffic"] --> GATE GATE -->|"A capped at 10, rest queued"| DBPOOL["Shared DB pool\n200 connections"] GATE -->|"B and C flow untouched"| DBPOOL style TEN_A fill:#ff2e88,color:#111 style GATE fill:#ffaa00,color:#111 style DBPOOL fill:#0e7490,color:#fff ``` Mitigation is a toolkit, and every tool is keyed by tenant ID: - **Rate limits at the edge** — token buckets per tenant (per plan tier, really), enforced at the gateway. The algorithms are the ones from [design a rate limiter](/system-design/articles/design-rate-limiter); the tenancy twist is only the key. - **Concurrency caps** on expensive operations: reports, exports, bulk imports, AI jobs. Queue the excess instead of rejecting it — the tenant sees "slower," not "broken." - **Fair scheduling** for background work: round-robin or weighted-fair across tenant queues, so a 50,000-job backfill from one tenant interleaves with — rather than precedes — everyone else's two jobs. - **Tier segregation**: free-tier traffic on one worker pool, enterprise on another. Fairness within a pool, insulation between pools. There's a prerequisite hiding under all of these: **per-tenant accounting**. If your metrics can't answer "which tenant is generating this load," you will diagnose your next brownout by grepping logs during the incident. Tag request metrics, slow-query logs, and job runtimes with tenant ID from day one. (Watch metric cardinality at five digits of tenants — track the top-N tenants individually and the long tail in aggregate.) The same accounting later becomes cost attribution, which is how you discover that one design-partner tenant costs more to serve than your ten biggest contracts pay. ## Blast radius: rings, cells, and shuffle sharding Sharing couples failure. A bad deploy on a single-tenant estate breaks one customer; on a pooled system it breaks all of them at once, which turns a bug into an outage and an outage into a churn event. Three structures cut the coupling back down. **Deployment rings.** Roll out by tenant cohort: internal tenants → free tier → paid → enterprise, with bake time between rings. The blast radius of a bad release becomes "the ring it reached," and your most sensitive customers sit in the last ring behind days of soak time. **Cells.** Partition the whole stack — app fleet, database, cache — into self-contained cells of, say, 500 tenants each, with a thin routing layer directing each tenant to its cell. Shopify calls them pods and assigns every shop to one; a pod-wide failure caps at that pod's shops. Cells also bound data-tier growth: when a cell's database fills, you add a cell rather than resharding the world — the per-tenant version of the strategies in [database sharding](/system-design/articles/database-sharding). The routing layer (tenant → cell) is the one genuinely global component, so it must be boring: a small, cached, replicated directory service that does nothing else. **Shuffle sharding.** Within a shared fleet, don't give every tenant access to every worker — give each tenant a random small subset: ```mermaid flowchart TD P["Poison tenant\nassigned w1 + w4"] --> W1 P --> W4 A["Tenant A\nassigned w2 + w5"] --> W2 A --> W5 B["Tenant B\nassigned w4 + w7"] --> W4 B --> W7 subgraph FLEET["Worker fleet — 8 nodes, C(8,2) = 28 possible pairs"] W1["w1 — down"] W2[w2] W3[w3] W4["w4 — down"] W5[w5] W6[w6] W7[w7] W8[w8] end style W1 fill:#dc2626,color:#fff style W4 fill:#dc2626,color:#fff style P fill:#ff2e88,color:#111 ``` With 8 workers and 2 per tenant, there are C(8,2) = 28 distinct pairs. A poison tenant — one whose requests crash or saturate whatever serves them — takes down its own pair only. The chance that another tenant shares *both* workers is 1/28 ≈ 3.6%; most tenants (like Tenant B above) overlap on at most one node and degrade instead of dying, and retries route around the sick worker. Scale the numbers and the isolation gets absurdly good: 100 workers choosing 5 gives ~75 million combinations. AWS Route 53 runs this play with 2,048 virtual name servers, 4 per hosted zone, so a DDoS against one customer's domain leaves other customers' DNS standing. Tenant-to-subset assignment is just hashing — [consistent hashing](/system-design/articles/consistent-hashing) if subsets must survive fleet resizes. ## Tenant context: the thread through everything All of the machinery above consumes one input: *which tenant is this?* That fact is born in exactly one trustworthy place — the verified auth token — and must survive every hop after it. ```mermaid sequenceDiagram autonumber participant U as Browser participant GW as API gateway participant SVC as Invoice service participant PG as Postgres with RLS U->>GW: GET /invoices with JWT claim org_id = acme GW->>SVC: request + verified tenant claim SVC->>PG: BEGIN then SET LOCAL app.tenant_id = 'acme' SVC->>PG: SELECT * FROM invoices WHERE status = 'open' Note over PG: policy appends tenant_id = 'acme'
a forgotten filter returns zero rows, not a leak PG-->>SVC: Acme rows only SVC-->>U: 200 OK ``` In-process, ambient context beats parameter-threading — Node's `AsyncLocalStorage`, Python's `contextvars`, a request-scoped bean, goroutine context: ```ts import { AsyncLocalStorage } from 'node:async_hooks'; const tenantCtx = new AsyncLocalStorage<{ tenantId: string }>(); app.use((req, _res, next) => { // From the *verified* JWT. Never from a header the client controls, // never from the request body, never from a query parameter. tenantCtx.run({ tenantId: req.auth.claims.org_id }, next); }); export function currentTenant(): string { const ctx = tenantCtx.getStore(); if (!ctx) throw new Error('no tenant context — refusing to touch data'); return ctx.tenantId; } ``` The repository layer, the cache client, the job enqueuer, and the logger all call `currentTenant()`. Two hops need special care, because ambient context does not cross them by itself: - **Queues.** A job payload must carry `tenantId` explicitly, and the worker must rehydrate context before running. The classic leak is the ops script that replays a dead-letter queue under the wrong — or no — tenant. - **Caches.** Every key gets the tenant prefix: `acme:user:42:dashboard`, never `user:42:dashboard`. In a [shared cache](/system-design/articles/design-distributed-cache), an unscoped key is a cross-tenant read primitive with sub-millisecond latency. Logs and traces get the tenant tag too — not for isolation but for operability: "is this outage all tenants or one?" is the first question of every incident, and it should be a filter, not an investigation. ## Operating a fleet of tenants A few realities of tenancy only show up after launch. **The tenant directory is a real system.** Somewhere a control plane maps tenant → shard/cell, plan tier, feature flags, encryption key, data-residency region, and lifecycle state. Everything consults it; it must be small, cached, and more available than anything behind it. **Migrations multiply.** Pooled: one `ALTER TABLE` (kept online-safe, since it now rewrites every tenant's rows at once). Siloed: N migrations, some of which will fail mid-fleet — so you track a schema version per tenant and make the app tolerate one version of skew, because at any given moment some database in the fleet is mid-migration. **Residency is routing.** "EU tenant data stays in the EU" means tenant → region is one more directory attribute, decided at signup, enforced at the routing layer — and the hardest part is the global surfaces (search, analytics, ML training sets) that quietly want to aggregate everything in one place. **Offboarding is a feature.** Tenants leave. Contracts and regulators require export (a complete, machine-readable dump of one tenant's data — trivial in a silo, a real project in a pool) and deletion with proof. Per-tenant encryption keys turn the deletion story from "we scrubbed 14 tables and hope the backups age out" into "we destroyed the key." ## What breaks **The missing filter.** A hand-written reporting query joins four tables and scopes three of them. Code review misses it; the ORM wasn't involved; it ships. This is the argument for RLS in one sentence: the database catches the query your review didn't. **The unscoped cache key.** In March 2023, a Redis client library bug caused canceled requests to receive responses belonging to other users — some ChatGPT users saw titles from other users' chat histories, and a few saw another subscriber's billing details. No SQL was harmed; the leak rode the shared cache and connection reuse. Tenant-prefix every key, and treat "shared connection + interleaved responses" as part of your isolation surface. **The job that lost its thread.** A nightly reconciliation job reads a config table without tenant scoping "because it runs for all tenants anyway," then writes results keyed by a `tenant_id` variable that survived from the previous loop iteration. Cross-tenant writes are rarer than cross-tenant reads and strictly worse. **The whale.** Pooling assumes tenants are small relative to the pool. Then one tenant grows to 100× the median — their workspace is 40% of a shard, their exports dominate the queue, their p99 *is* your p99. Every pooled architecture eventually grows a "graduate a tenant to a silo" runbook; build the tenant-move tool before you need it during an incident. **The enumerable ID.** `/invoices/48291` with a sequential global ID and an ownership check that only verifies the invoice *exists* — the oldest cross-tenant vulnerability on the web (IDOR). Scope the lookup (`WHERE tenant_id = ? AND id = ?`), and prefer non-guessable IDs so a bug in the check isn't also a directory listing. **The shared limit nobody owned.** One tenant's email campaign consumes the platform's entire SES send quota; every tenant's password resets bounce. Inventory the quotas you share upstream — email, SMS, payment APIs, LLM tokens — and budget them per tenant like any other pooled resource. ## The decision in practice Start pooled: shared tables, `tenant_id` on every row, RLS from the first migration, per-tenant metrics and rate limits from day one. These cost almost nothing at 10 tenants and are excruciating to retrofit at 10,000 — Salesforce-scale pooling works because the discipline was never optional. Keep the tenant directory separate and boring. When one database stops scaling, shard or cell by tenant, and when sales brings you a hospital, a government, or a whale, graduate them to the silo tier — and charge for it, because isolation is a product feature that happens to be an architecture. ```mermaid flowchart TD D1{"Regulated data or a contract\nthat demands physical isolation?"} -->|"those tenants"| S1["Silo tier\ndedicated DB, own keys, own region"] D1 -->|no| D2{"Tenant count?"} D2 -->|"tens, high-value"| S2["Database per tenant is affordable\ntake the easy restore + deletion wins"] D2 -->|"thousands"| D3{"Whales much bigger\nthan the median tenant?"} D3 -->|yes| S3["Pool + RLS for the long tail\ngraduate whales to silos"] D3 -->|no| S4["Pool + RLS everywhere\ncells when one DB stops scaling"] style S1 fill:#0e7490,color:#fff style S3 fill:#22c55e,color:#111 style S4 fill:#22c55e,color:#111 ``` The trap to avoid is the unprincipled middle: schema-per-tenant everywhere, or silos handed out ad hoc to whoever asked loudest, until you run a zoo of half-shared systems with the costs of both models and the guarantees of neither. Pick the pool as the default, the silo as the exception you charge for, and hold the line in between. ## Things you should now be able to answer - What exactly is a tenant, and why is it a different unit than a user? - Walk through silo, bridge, and pool — what does each cost at 1,000 tenants, and what breaks first in each? - A customer demands their data be provably isolated and deletable. Which layers deliver that in a pooled system? - One tenant's export feature just took product-wide p99 from 80 ms to 30 s. What do you deploy this week, and what do you build this quarter? - How does shuffle sharding reduce shared fate, and what do the numbers look like for 8 workers with 2 per tenant? ## Further reading - AWS Well-Architected Framework, *SaaS Lens* — the silo/bridge/pool vocabulary and tiering patterns - Salesforce, *The Force.com Multitenant Architecture* whitepaper — the canonical pooled design: OrgID on every row, metadata instead of schema changes - Colm MacCárthaigh / AWS Builders' Library, *Workload isolation using shuffle-sharding* — the combinatorics of blast radius, with the Route 53 story - Notion engineering, *Herding elephants: lessons learned from sharding Postgres* (2021) and *The Great Re-shard* (2023) — pooled tenancy meeting physical limits - Slack engineering, *Scaling Datastores at Slack with Vitess* — the migration from workspace-sharded MySQL to flexible shard keys behind a pooled application tier - PostgreSQL documentation, *Row Security Policies* — the exact semantics of `USING`, `FORCE`, and `BYPASSRLS` - OpenAI, *March 20 ChatGPT outage* postmortem — the cache-layer cross-user leak, in the vendor's own words ## Frequently asked questions Q: What is the difference between single-tenant and multi-tenant architecture? A: Single-tenant gives each customer a dedicated copy of the application and database; multi-tenant serves every customer from one shared deployment, with each row of data tagged by a tenant identifier. Single-tenant isolates failures and simplifies compliance but costs roughly linearly with customer count; multi-tenant amortizes infrastructure and operations across all customers, which is why nearly all SaaS at scale is multi-tenant. Q: What are the silo, bridge, and pool models in multi-tenant design? A: Silo gives each tenant dedicated infrastructure — its own database, sometimes its own full stack. Pool shares everything: one schema, one set of tables, every row tagged with a tenant_id. Bridge mixes the two, most commonly a shared application tier over per-tenant databases or schemas. Mature SaaS products usually run all three at once, tiered by plan: pooled for free and standard tiers, siloed for the largest or most regulated customers. Q: How do you prevent one tenant from reading another tenant's data in a shared database? A: Defense in depth: a tenant_id column on every table, a repository layer that injects the tenant filter into every query automatically, and Postgres row-level security as the backstop — a policy like USING (tenant_id = current_setting('app.tenant_id')::uuid) means a query that forgets the filter returns zero rows instead of another tenant's rows. Composite foreign keys that include tenant_id make cross-tenant references unrepresentable. A simple indexed equality policy typically costs low single-digit percent overhead. Q: What is the noisy neighbor problem and how do you mitigate it? A: The noisy neighbor problem is one tenant's heavy workload — a giant export, an API burst, a pathological query — degrading latency for every tenant sharing the same connection pools, workers, and caches. Mitigations are all per-tenant: token-bucket rate limits keyed by tenant ID, concurrency caps on expensive operations, weighted fair queuing for background jobs, and separate worker pools per pricing tier. Per-tenant accounting comes first: without it you cannot even identify which tenant is the noisy one. Q: When should you use database-per-tenant instead of a shared schema? A: Use database-per-tenant when contracts or regulation demand physical isolation (healthcare, government, data residency), when per-tenant backup, restore, and deletion must be trivial, or when tenants number in the tens with high revenue each. At around 1,000 tenants the economics invert: roughly $15/month per idle small instance is ~$180k/year (illustrative AWS list pricing) versus ~$1.5k/month for one pooled cluster — about 10× — and every schema migration becomes a 1,000-database orchestration project. Q: What is shuffle sharding and how does it shrink blast radius? A: Instead of assigning each tenant to one worker or one fixed shard, shuffle sharding assigns each tenant a random small subset of the fleet — say 2 workers out of 8, which gives C(8,2) = 28 distinct pairs. A poison tenant can only saturate its own pair; the probability that any other tenant shares both of those workers is 1/28 (~3.6%), so almost everyone keeps at least one healthy worker. AWS Route 53 uses this technique so a DDoS against one domain does not take out the name servers of other customers.