# Cascaded pipeline vs native speech-to-speech: when to use each > Chain STT, LLM, and TTS for control — or use one speech-native model for latency. The real tradeoffs, with numbers, and when each architecture wins. Canonical: https://ironclad.academy/ai-engineering/articles/cascaded-vs-native-speech-to-speech | Difficulty: medium | Topics: Voice, Multimodal, Latency, Architecture Covers: OpenAI, Google, ElevenLabs, Deepgram, LiveKit ## Key takeaways - Native speech-to-speech models preserve emotional prosody but sacrifice the intermediate text transcript — you cannot have both without a separate transcription side channel. - An optimized cascaded pipeline at 500–700ms TTFA is comparable to or faster than most native S2S models in production; native latency wins only against an unoptimized cascaded stack. - PSTN telephony degrades native S2S quality significantly — cascaded with telephony-optimized STT is the right call for phone numbers. - The cost gap between native S2S ($0.30/min) and a well-chosen cascaded stack ($0.04–0.085/min) compounds to $250,000+ per year at 100K minutes/month. - Hume EVI 3 is the only production model that natively tracks and generates emotional prosody — relevant for healthcare, mental health, and high-empathy use cases. - Choose native S2S for consumer apps where natural conversation feel matters more than cost or auditability; choose cascaded for enterprise, telephony, compliance-heavy, or high-volume deployments. > **In one line:** Native speech-to-speech models collapse the STT→LLM→TTS chain into one pass — better emotional coherence, but cascaded pipelines still win on cost, telephony, debuggability, and often latency. **The idea.** A cascaded voice pipeline is three separate models glued together: a speech-to-text (STT) model transcribes audio to text, a language model generates a text response, a TTS model synthesizes audio back. Each hand-off is a conversion between audio and text. Native speech-to-speech models — OpenAI gpt-realtime-1.5, Gemini 3.1 Flash Live, Amazon Nova Sonic, Hume EVI 3 — skip the text intermediate and process audio directly, emitting audio directly. The appeal is lower latency and richer vocal continuity. The tradeoffs are higher cost, no transcript, and worse behavior on telephone-grade audio. **Key points.** - Cascaded pipelines give you a text transcript at every hop — essential for compliance logging, CRM writes, tool calls, and mid-turn routing. - Native S2S models preserve tone, pacing, and emotional prosody because the model never collapses audio to text and back. - An optimized cascaded stack can reach 500–700ms TTFA including end-of-turn detection; most native S2S models benchmark at 800ms–3s TTFA, making "lower latency" context-dependent. - PSTN telephony (8 kHz, codec-compressed) degrades native S2S significantly; cascaded with telephony-optimized STT wins on phone channels. - Cost at scale: OpenAI gpt-realtime-1.5 at ~$0.30/min vs a typical cascaded stack at $0.04–0.085/min — a 4–7x gap. ``` Back-of-envelope — 100,000 minutes/month: Native S2S (OpenAI gpt-realtime-1.5): 100,000 min × $0.30/min = $30,000/month Cascaded (Deepgram Nova-3 + mid-tier LLM + Cartesia Sonic 4): STT: 100,000 min × $0.005 = $500 LLM: 100,000 min × $0.03 = $3,000 (varies by token density) TTS: 100,000 min × $0.028 = $2,800 Total ≈ $6,300/month → Native S2S costs ~4.8× more at 100K min/month. At 500K min/month the gap is $150,000 vs $31,500. ``` ```mermaid flowchart LR subgraph Cascaded["Cascaded Pipeline"] A1[Audio in] --> STT[STT model\nDeepgram / Whisper] STT -->|"text transcript"| LLM[LLM\nGPT-4o / Claude] LLM -->|"text response"| TTS[TTS model\nCartesia / ElevenLabs] TTS --> A2[Audio out] end subgraph Native["Native Speech-to-Speech"] B1[Audio in] --> S2S["S2S model\nOpenAI Realtime / Gemini Live"] S2S --> B2[Audio out] end style STT fill:#0e7490,color:#fff style LLM fill:#a855f7,color:#fff style TTS fill:#15803d,color:#fff style S2S fill:#00e5ff,color:#0a0a0f ``` **Key design decisions.** | Decision | Cascaded | Native S2S | |---|---|---| | Intermediate transcript | Available at every stage | None (or async side channel) | | Emotional prosody | Depends on TTS expressiveness | Preserved from input audio | | Component swappability | Swap STT/LLM/TTS independently | Single vendor, all-or-nothing | | Answer quality | Any frontier text LLM | Capped by the speech model's reasoning | | Telephony (8 kHz PSTN) | Cascaded with telephony STT wins | Quality degrades significantly | | Cost at scale | $0.04–0.085/min typical | $0.02–0.30/min | | Latency (optimized) | 500–700ms TTFA achievable | 780ms–3s current production range | | Tool calls and RAG | Text is natively available | Requires async transcript | **If you have 60 seconds, say this.** "Cascaded pipelines convert audio to text, run the LLM on text, then convert back to audio — three models, three stages, but a transcript at every hop. Native S2S models process audio-in to audio-out in one model, preserving vocal tone and emotion but losing the intermediate text. Cascaded still wins for telephony because PSTN degrades S2S quality, for compliance because you need a transcript, and often for cost — OpenAI Realtime runs $0.30/min vs $0.04–0.08/min for a well-tuned cascaded stack. Go native when the conversation feel matters more than auditability or cost: consumer voice assistants, empathy-heavy healthcare bots, or demos where 'it sounds like a person' is the product. For enterprise, telephony, or anything that touches a compliance team, stick to cascaded." The production voice agent you demoed last month sounded great. Then the legal team asked for call transcripts. Then the telephony integration broke. Then the billing team saw the API invoice. Welcome to the real decision between cascaded and native speech-to-speech architecture — not as a capability question, but as an engineering and economics problem. ## What the two architectures actually are A **cascaded voice pipeline** is a chain of three specialized models. An STT model — Deepgram Nova-3, AssemblyAI Universal-2, or Whisper — converts the incoming audio stream to a text transcript. That transcript goes into an LLM (GPT-4o, Claude Sonnet, Gemini Flash, whatever fits your quality and cost budget). The LLM's text response goes to a TTS model — Cartesia Sonic 4, ElevenLabs Flash v2.5, Deepgram Aura-2 — which synthesizes audio and streams it back. Text is the connective tissue throughout. A **native speech-to-speech model** collapses the three stages into one. Audio arrives, the model processes it through a unified multimodal transformer, and audio comes out. There is no text intermediate in the hot path. OpenAI's `gpt-realtime` (GA August 2025, updated to `gpt-realtime-1.5` in May 2026) works this way. So does Gemini 3.1 Flash Live (launched March 2026), Amazon Nova Sonic, and Hume EVI 3. The conceptual appeal of native S2S is obvious: fewer handoffs, no modality conversion losses, the model "hears" the user's tone rather than reading a transcript of it. But conceptual appeal and production performance are different things. Let's look at each axis where they diverge. ## Latency: the one claim that deserves scrutiny The most common assumption is that native S2S is faster. It can be — but the comparison depends heavily on how well-optimized your cascaded stack is. Take an unoptimized cascaded pipeline: STT waits for the user to finish speaking before transcribing (full utterance wait), then the LLM processes the full transcript, then TTS waits for the complete LLM response before synthesizing. In that sequential, non-streaming mode, you're looking at: ``` Unoptimized sequential cascaded (worst case): End-of-turn detection (silence VAD): ~500ms STT full-utterance processing: ~300ms LLM TTFT (no prefix caching): ~800ms LLM full response (~40 tok at 80 tok/s): ~500ms TTS full synthesis before playback: ~200ms Network round-trips: ~150ms Total TTFA: ~2,450ms ``` Against that baseline, any native S2S model looks fast. But that is not the right comparison. A properly optimized cascaded pipeline streams at every stage. Deepgram streams partial transcripts as the user speaks. The LLM receives partial input and begins generating before the user finishes. Cartesia Sonic 4 starts synthesizing the first sentence of the LLM's response before the rest arrives. Each stage overlaps with the next. The [voice pipeline end-to-end article](/ai-engineering/articles/voice-pipeline-stt-llm-tts) covers the streaming mechanics in detail; the short version is that this overlap, plus an aggressively tuned endpointer, cuts the sequential total by roughly three quarters. ``` Optimized streaming cascaded (achievable): End-of-turn detection (tuned VAD): ~250ms STT transcript finalization (streaming partials already fed the LLM): ~50ms LLM TTFT with prefix caching: ~200ms TTS first sentence playback: ~40ms (Cartesia Sonic 4) Network (regional routing, WebRTC): ~60ms Total TTFA: ~600ms ``` One line in that budget never appears in vendor marketing diagrams: end-of-turn detection. TTFA is measured from the moment the user stops speaking, and the pipeline has to *detect* that stop — a tuned silence threshold still costs ~200–300ms, and semantic endpointing adds ~100ms on top. No amount of streaming removes it. Budget for 500–700ms TTFA in practice, not the 400ms you get by summing only the model stages. [Interactive visualizer on the original page: voice-latency — Sequential vs streaming cascaded — where the TTFA goes] Now look at what native S2S models actually deliver in independent benchmarks from April 2026: | Model | TTFA (benchmarked) | Cost/min | Notes | |---|---|---|---| | xAI Grok Voice | ~780ms | Early access | Fastest native S2S benchmark | | OpenAI gpt-realtime-1.5 | ~820ms | ~$0.30 | GA, most production usage | | Amazon Nova Sonic | ~1,140ms | ~$0.02 | Lowest cost native option | | Gemini 3.1 Flash Live | ~2,980ms | Varies | 90+ languages, higher latency | A well-tuned cascaded stack at 500–700ms TTFA is comparable to or faster than every production-grade native S2S model in the current generation. The narrative that native S2S is "lower latency" is true only relative to a naive sequential cascaded pipeline. Relative to an optimized streaming pipeline, native S2S holds no latency advantage today. This will change. The models will get faster. But as of mid-2026, "go native for latency" is not yet correct advice. ## The transcript problem Here is where the choice becomes structural. Text is the cascaded pipeline's native byproduct — you have it at every stage. It flows from STT into the LLM, and the LLM's output text feeds the TTS. You can: - Log every turn for compliance or audit - Write to CRM in real time based on transcript content - Route to different tools based on detected intent in the transcript - Stream partial transcripts to a supervisor UI during a live call - Run a sentiment classifier on the text mid-turn Native S2S has no intermediate text. The model produces audio. OpenAI's Realtime API does return transcripts as a side channel, but they arrive after the audio — you cannot intercept a partial transcript mid-utterance to make a routing decision before the response has started playing. If your use case needs to look at the text content during the conversation to do anything (tool calling, compliance flagging, handoff to a human agent), you are fighting the architecture. This is a real constraint for enterprise deployments. If a voice agent is handling customer service calls that touch PCI-DSS data, you need transcripts. If an agent is writing call summaries to Salesforce, you need transcripts. If a healthcare bot needs to detect a user mentioning medication names for documentation, you need transcripts. The cascaded pipeline is not just easier for these use cases — it is the only option without adding a parallel transcription stream, which partially defeats the simplification that native S2S offers. ## Tool calls and RAG Language model tool calls require text. When a voice agent needs to look up an account balance, book an appointment, or query a knowledge base, it needs to emit a structured tool call and parse the result. In a cascaded pipeline, this is straightforward: the LLM is just an LLM, it can call tools exactly as it would in a text context, the tool result feeds back into the LLM's context, and the cycle continues. ```mermaid sequenceDiagram participant U as User (audio) participant STT as STT participant LLM as LLM participant TOOL as Tool / API participant TTS as TTS participant UA as User (audio out) U->>STT: "What's my account balance?" STT->>LLM: text transcript LLM->>TOOL: get_account_balance(user_id=...) TOOL-->>LLM: { balance: "$1,240.50" } LLM->>TTS: "Your balance is $1,240.50" TTS->>UA: synthesized audio ``` In native S2S, tool calls are handled through a text-based side channel even in models that support them. The audio model signals a tool call, the system extracts it, calls the API, returns the result as text, and the model continues. This works — OpenAI's Realtime API supports it — but it reintroduces a text intermediate specifically for tool calls, which limits the "purity" of the native approach. For agents that are mostly tool-calling (scheduling bots, CRM-integrated support agents, appointment booking), the cascaded pipeline is architecturally cleaner. ## Answer quality: whose brain is behind the voice The quietest argument for cascaded is also the strongest one: the LLM in the middle can be *any* text model. If your agent needs frontier-grade reasoning — policy-heavy support decisions, multi-constraint scheduling, precise tool sequences — you point the pipeline at the strongest text model you can afford, and when a better one ships you swap it in that afternoon. The voice layer doesn't care. Native S2S ties the conversation's intelligence to the speech model itself, and as of mid-2026 the speech-native variants measurably trail their text siblings on complex instruction-following, multi-step reasoning, and tool-call accuracy. That is not a permanent law, but it follows from how these models are built: post-training budget spent on sounding natural is budget not spent on being right, and the realtime constraint discourages the long deliberation that [reasoning models](/ai-engineering/articles/reasoning-models-tool-use-agents) use to earn their accuracy. If your agent's hardest problem is *what* to say rather than *how* to say it, cascaded wins before latency or cost ever enter the discussion. ## Telephony is a different problem PSTN telephone networks use 8 kHz audio with G.711 or G.729 codec compression. Web apps use 16 kHz Opus over WebRTC. The quality difference is significant, and native S2S models trained on high-quality audio take a noticeable hit on telephone-grade input. The practical evidence: telephony deployments that switched from cascaded to native S2S and back have found that cascaded pipelines with STT models specifically trained on telephone audio (Deepgram is the clearest example, with explicit telephony model variants) outperform native S2S on phone call quality. The emotional prosody advantage of native S2S is also largely moot over phone audio — the codec compression strips most of the prosodic information before it ever reaches the model. And the native S2S pricing does not discount for telephone audio quality. You still pay $0.30/min for gpt-realtime-1.5 whether the audio is pristine WebRTC or 8 kHz PSTN. The [voice pipeline article](/ai-engineering/articles/voice-pipeline-stt-llm-tts) has a full breakdown of WebRTC vs SIP transport choices. For telephony deployments specifically, the architectural verdict is simple: use cascaded, use a telephony-optimized STT model, and save the native S2S experiments for your web app. ## Where native S2S genuinely wins This is not a blanket argument for cascaded. Native S2S has a real advantage category, and it is not latency — it is **emotional and vocal continuity**. A cascaded pipeline destroys vocal context at every hand-off. The STT model converts the user's tired, anxious, or excited speech to neutral ASCII text. The LLM responds to that neutral text. The TTS model synthesizes a response with whatever expressiveness its defaults allow, with no connection to how the user actually sounded. The voice output is vocally decoupled from the voice input. Native S2S maintains the acoustic thread. The model "heard" the user's hesitations, their raised pitch, their trailing-off cadence. That information shapes how it responds. Hume EVI 3 takes this furthest: it is the only production model that explicitly tracks emotional prosody and generates responses calibrated to the user's detected emotional state. This is not a gimmick for consumer apps — it is meaningful for healthcare consultations, mental health support tools, and any use case where the quality of the interaction depends on emotional attunement. For a consumer voice assistant where "it feels like a real conversation" is the core product value, native S2S is worth the cost premium and the transcript constraint. For a scheduling bot handling appointment reminders at 50,000 calls per month, it is not. ```mermaid flowchart TD START[Voice agent project] --> PHONE{Phone / PSTN?} PHONE -->|Yes| CASCADE[Cascaded pipeline\nwith telephony STT] PHONE -->|No| COMPLY{Compliance logging\nor tool-heavy?} COMPLY -->|Yes| CASCADE COMPLY -->|No| VOLUME{Volume > 50K min/month?} VOLUME -->|Yes| COST{Can absorb\n$0.30/min?} COST -->|No| CASCADE COST -->|Yes| EMOTION{Emotional prosody\nor empathy-critical?} VOLUME -->|No| EMOTION EMOTION -->|Yes| NATIVE[Native S2S\nHume EVI 3 or OpenAI Realtime] EMOTION -->|No| FEEL{Conversation feel\nis the core product?} FEEL -->|Yes| NATIVE FEEL -->|No| CASCADE style CASCADE fill:#0e7490,color:#fff style NATIVE fill:#00e5ff,color:#0a0a0f ``` ## Cost math at scale The billing difference between architectures compounds quickly. Here is the calculation for a customer service voice agent at different volumes, using illustrative mid-2026 pricing: ``` Assumptions: Average call: 4 minutes LLM tokens: ~80 output/min; effective input ~8–12K tokens/min, because every turn resends the growing conversation history (fresh speech alone is only ~200 tokens/min) Cascaded: Deepgram Nova-3 + GPT-4o + Cartesia Sonic 4 Native: OpenAI gpt-realtime-1.5 --- 10,000 min/month (small deployment) --- Cascaded: STT 10,000 × $0.005 = $50 LLM 10,000 × $0.030 = $300 (history-heavy input dominates; prefix caching can cut this 2–4×) TTS 10,000 × $0.028 = $280 Total: $630/month Native S2S: 10,000 × $0.30 = $3,000/month Gap: $2,370/month — probably not the deciding factor at this volume. --- 100,000 min/month (mid-scale) --- Cascaded: ~$6,300/month Native S2S: ~$30,000/month Gap: $23,700/month → $284,400/year --- 500,000 min/month (large deployment) --- Cascaded: ~$31,500/month Native S2S: ~$150,000/month Gap: $118,500/month → $1,422,000/year ``` Below 10,000 minutes per month, the absolute cost difference is manageable and should not be the deciding factor. Above 50,000 minutes per month, it is the primary architectural concern. Above 200,000 minutes per month, native S2S requires a specific product justification that cost cannot override. Amazon Nova Sonic changes this calculation at the low end: at ~$0.02/min it is 15× cheaper than gpt-realtime-1.5, though it carries a ~1.14s TTFA vs the ~820ms of OpenAI's model. For use cases that can absorb the higher latency, Nova Sonic is the native S2S option that makes cost math work at larger volumes. ## What breaks ### The echo feedback spiral Without acoustic echo cancellation (AEC), the agent's outgoing audio leaks into the user's microphone through their speakerphone. The STT model transcribes the agent's own voice, the LLM receives its own output as user input, and the agent starts responding to itself. This happens in both architectures, but in cascaded it is more immediately visible because the transcript shows the problem. In native S2S, the model may silently incorporate its own audio output into its next response without an obvious text artifact. Twilio's server-side AEC works for good acoustics; it fails on cheap speakerphones and poor room conditions. If your deployment targets mobile users or call centers, AEC is a non-negotiable first-class concern, not an afterthought. ### Turn detection misfires on silence Both architectures depend on voice activity detection (VAD) to know when a user has finished speaking. Energy-based VAD fires on silence thresholds — it treats any pause as a turn boundary. Users who pause mid-thought, hesitate on a word, or use filler speech get cut off. In native S2S this produces an especially jarring result because the model's response is already emotionally calibrated to a half-finished input. OpenAI's Semantic VAD mode — which waits for semantic completeness rather than mere silence — significantly reduces false-positive interruptions but adds roughly 100ms to the turn detection latency. The [VAD and turn detection article](/ai-engineering/articles/vad-turn-detection-voice-ai) covers the mechanics and tradeoffs in detail. The point here: semantic VAD is important for both architectures, but it is especially important for native S2S where a premature turn-end produces a contextually inappropriate emotional response. ### Barge-in: cancelling a response already in flight The user interrupts mid-answer — the single most common event in a real voice conversation — and both architectures have to unwind work that is already streaming. In a cascaded pipeline that means cancelling three things at once: the in-flight LLM generation, the TTS synthesis it was feeding, and the audio sitting in the playback buffer. Then comes the part teams get wrong: reconciling conversation state. If the agent had generated three sentences but the user heard one and a half, the history must be truncated to what was actually played — track playback position, cut the assistant turn at that point — or the model will keep referring to things the user never heard. Native S2S handles interruption *detection* in-model, which is genuinely nicer: the model stops on its own instead of waiting for your cancellation plumbing. But the state-reconciliation problem does not go away. You still have to tell the server how much audio the user actually heard (OpenAI's Realtime API exposes a truncation call for exactly this), and getting the truncation point wrong is a known pain point — the model's memory of its own turn silently diverges from the user's. Frameworks help here; LiveKit Agents runs the cascaded cancellation loop for you. But test barge-in explicitly in either architecture, with a user who interrupts rudely and often. Production users do. ### Debugging without a transcript When a cascaded pipeline produces a bad response, you have a complete paper trail: the STT transcript, the LLM input, the LLM output, the latency at each stage. When a native S2S model produces a bad response, you have audio in and audio out. Reproducing the failure requires re-playing the audio; attributing it to input quality vs model behavior requires instrumentation that native S2S does not naturally provide. This is a real operational cost for teams that take support escalations or run conversation quality reviews. The OpenAI Realtime API's async transcript side channel helps, but it is not available mid-turn and is not a substitute for the per-stage observability that cascaded provides. ### Component lock-in A cascaded pipeline lets you swap STT, LLM, and TTS independently. If Deepgram's latency spikes, you re-route to AssemblyAI. If GPT-4o costs too much for your token density, you switch to Gemini Flash. If ElevenLabs releases a better voice for your brand, you swap it in. Each component evolves independently. Native S2S is a single vendor decision. If OpenAI degrades Realtime quality in a model update, you cannot swap the STT layer to compensate — you either accept the degradation or migrate the entire voice stack. For products where voice quality is a brand-level concern, this single-vendor risk is worth pricing in before you commit. ## Mixing approaches Nothing prevents a hybrid. A common production pattern: run native S2S for the main conversation flow (where emotional continuity matters), and spin up a cascaded STT side channel specifically for transcript-dependent operations. The side channel captures the user's speech in parallel and writes to your logging and CRM system. The native model handles the conversation feel. The cost of this hybrid is running two inference pipelines simultaneously and the added complexity of keeping them in sync. It is worth considering for use cases that need both emotional prosody and a compliance log — but be honest that you are paying for both architectures, not getting the advantages of each for free. ```mermaid sequenceDiagram participant U as User (audio) participant VAD as VAD participant S2S as Native S2S model participant STT as Parallel STT\n(compliance side channel) participant CRM as CRM / Audit log participant UA as User (audio out) U->>VAD: audio stream VAD->>S2S: speech segment VAD->>STT: same segment (parallel) S2S->>UA: audio response (emotional) STT->>CRM: text transcript (async) ``` ## The frameworks question The choice of voice agent framework interacts with the architecture choice. Vapi and Retell AI are managed platforms that abstract both cascaded and native S2S behind their APIs — useful for low-volume deployments and fast iteration. LiveKit Agents 1.5.x and Pipecat v1.0 give you full control over the pipeline and support both architectures, with better economics above 50,000 minutes per month. The [voice agent frameworks article](/ai-engineering/articles/voice-agent-frameworks-vapi-livekit-pipecat) has the decision matrix and cost thresholds. One practical note: LiveKit's native S2S integration routes audio directly to the model using WebRTC transport and handles the barge-in cancellation loop for you. Pipecat's compositional architecture lets you wire native S2S as one "stage" in a pipeline that includes a separate STT side channel — which is the hybrid pattern described above. Neither framework forces your hand on the cascaded vs native decision; they support both. ## The decision in practice If you are picking between these architectures today, answer four questions in order: **1. Is PSTN telephony in scope?** If yes, cascaded — unconditionally. Telephone-grade audio degrades native S2S quality while retaining its pricing. Use Deepgram Nova-3 with their telephony model variant. **2. Do you need a mid-turn transcript?** Compliance logging, CRM writes, tool-call routing based on live transcript content, supervisor monitoring — any of these requires cascaded, or a hybrid with a parallel STT stream. **3. What is your monthly minute volume?** Below 10,000 minutes, cost is not the deciding factor. Above 50,000 minutes, native S2S at $0.30/min requires a product justification that overrides the cost differential. Amazon Nova Sonic at $0.02/min is the exception that changes this threshold. **4. Is emotional prosody or conversation feel the core product value?** Healthcare, mental health, high-empathy consumer assistants — go native. Hume EVI 3 specifically if emotional attunement is the product. For a scheduling bot or FAQ agent, voice quality is important but "sounds like a real person" is not the product differentiator; cascaded with ElevenLabs Flash v2.5 or Cartesia Sonic 4 sounds excellent and costs a fraction of the price. Weigh this against answer quality: if the agent's value lives in getting hard answers right, cascaded lets you put a frontier text model behind the voice, while native S2S caps reasoning at what the speech model can do. The default recommendation for most enterprise and telephony deployments is cascaded with streaming enabled end-to-end, Deepgram for STT, a mid-tier frontier model for the LLM (with [prompt prefix caching](/ai-engineering/articles/prompt-caching-cost-optimization) enabled), and Cartesia Sonic 4 for TTS. That stack, properly optimized, hits 500–700ms TTFA — matching or beating most native S2S models at a fifth of the cost or less. Native S2S is the right answer for a specific product profile: consumer-facing, conversation-quality-is-the-product, emotional attunement matters, volume is manageable, and compliance teams are not in the room. As the models improve and costs come down, that profile will expand. But the engineering decision today requires being honest about where the production numbers actually land, not where the marketing says they should. For the latency budget breakdown and optimization techniques that apply to both architectures, see [latency budget engineering](/ai-engineering/articles/voice-ai-latency-budget-production). ## Frequently asked questions Q: What is the difference between a cascaded voice pipeline and a native speech-to-speech model? A: A cascaded pipeline converts audio to text (STT), passes text to an LLM, then converts the response text back to audio (TTS) — three separate components with text as the intermediate representation. A native speech-to-speech model (like OpenAI gpt-realtime-1.5 or Gemini 3.1 Flash Live) processes audio in and emits audio out in a single model, skipping the text intermediate entirely. Cascaded pipelines are more debuggable and modular; native models preserve vocal tone and emotion but lose the intermediate transcript. Q: Is native speech-to-speech actually faster than a cascaded pipeline? A: Not necessarily. OpenAI gpt-realtime-1.5 benchmarks at around 820ms time-to-first-audio (TTFA) in April 2026. An optimized cascaded pipeline — end-of-turn detection (~250ms) + streaming Deepgram Nova-3 finalization (~50ms) + LLM TTFT (~200ms) + Cartesia Sonic 4 (~40ms) — can reach 500–700ms TTFA when fully streaming. Native S2S eliminates the STT and TTS latency budgets but runs a heavier single model; the net result is often comparable latency, not a clear win for native. Q: Why should I not use native speech-to-speech for telephony? A: PSTN telephone networks deliver audio at 8 kHz with significant codec compression, G.711 or G.729 encoding, and background noise. Native S2S models are trained on higher-quality audio and degrade noticeably on telephony-grade input — accuracy drops, emotional modeling becomes unreliable, and the premium pricing (around $0.30/min for OpenAI Realtime vs $0.02/min for Amazon Nova Sonic or a self-hosted cascaded stack) remains. Cascaded pipelines with telephony-optimized STT like Deepgram Nova-3 outperform on phone calls. Q: How do I log or record conversations when using a native speech-to-speech model? A: Native S2S models do not produce a stable intermediate text transcript, which means compliance logging, CRM writes, and conversation summaries require a separate post-processing step. OpenAI Realtime API does return transcripts as a side channel, but they arrive after the fact and cannot be intercepted mid-turn for real-time routing. If your product needs deterministic mid-turn text access — for tool calls, sentiment tagging, or audit trails — a cascaded pipeline gives you that at every hop. Q: What is the cost difference between native S2S and a cascaded pipeline? A: As of mid-2026, OpenAI gpt-realtime-1.5 costs approximately $0.30/min of audio. A comparable cascaded stack — Deepgram Nova-3 at around $0.005/min, a mid-tier LLM at $0.01–0.05/min, and Cartesia Sonic 4 at $0.025–0.03/min — lands at $0.04–0.085/min. At 100,000 minutes per month, native S2S costs $30,000 vs $4,000–$8,500 for cascaded. Amazon Nova Sonic closes the gap at ~$0.02/min but with higher latency (~1.14s TTFA). Q: Which native speech-to-speech models are production-ready in 2026? A: As of mid-2026, four models are in production use: OpenAI gpt-realtime (GA August 2025; the current gpt-realtime-1.5 shipped May 2026 with a GPT-Realtime-Whisper transcript side channel), Gemini 3.1 Flash Live (GA March 2026, 90+ languages, higher latency at ~2.98s TTFA), Amazon Nova Sonic (~$0.02/min, lowest cost, ~1.14s TTFA), and Hume EVI 3 (the only model with native emotional prosody tracking, suited for healthcare and mental health use cases). xAI Grok Voice posted the fastest TTFA benchmarks (~0.78s) in April 2026 but remains early-access.