Voice Agent Frameworks: Vapi vs LiveKit vs Pipecat

The call lasted four seconds. The customer had asked a simple billing question. The agent said nothing for 1.4 seconds, then started answering — but the customer had already hung up assuming the call dropped. The transcript was perfect. The latency wasn't. The product team had shipped the voice agent on a managed platform with a stack they barely controlled, and they were now trying to debug something they didn't build.

This is the core tension in voice agent infrastructure: the faster you launch, the less you understand. The more control you take, the more you own. There are now three viable layers to build at, and which one you choose should be a deliberate business decision — not a default.

What the pipeline actually looks like

Before comparing frameworks, it helps to have the full architecture sharp. The voice pipeline is detailed elsewhere; the key point for framework selection is that every stage must stream into the next. Sequential processing — wait for full transcript, send to LLM, wait for full response, send to TTS — adds 800ms–2s on top of the component minimums. Streaming overlap is where the time goes:

sequenceDiagram
    participant U as User
    participant VAD as VAD
    participant STT as STT (Deepgram)
    participant LLM as LLM
    participant TTS as TTS (Cartesia)
    participant SPK as Speaker

    U->>STT: audio (still speaking)
    STT-->>LLM: partial transcripts (streaming)
    Note over LLM: inference starts before full transcript
    U->>VAD: speech ends
    VAD->>STT: endpoint detected, flush final transcript
    LLM-->>TTS: token stream
    Note over TTS: flush first sentence immediately
    TTS-->>SPK: audio bytes (first word)
    Note over SPK: user hears response at ~400ms

The gap between 400ms and 800ms in production is almost entirely orchestration quality — how well the framework handles partial transcripts, when it flushes TTS, and whether it cancels inflight requests correctly on interruption. That orchestration is exactly what Vapi, LiveKit, and Pipecat each handle differently.

Vapi: managed platform, fast start, real ceiling

Vapi is a fully managed voice agent platform billed per minute. You configure STT, LLM, and TTS providers through a dashboard or API, point a phone number at an assistant config, and you're live. Telephony works out of the box — Vapi handles PSTN ingress, SIP trunking, and call routing. Webhooks fire on call events. A client SDK lets you embed a voice widget in a web app.

The ease of launch is real. A competent engineer can ship a working outbound calling agent in an afternoon. VAD and turn detection are managed; interruption handling is built in. You don't configure a WebRTC server, you don't tune silence thresholds, you don't implement the barge-in cancel logic.

The ceiling is also real. Vapi's provider roster is curated — you get the providers they've integrated, at prices they negotiate. You cannot add a custom STT model, you cannot self-host the TTS, and you cannot inspect the exact audio path when a call sounds wrong. Debugging is through logs and the Vapi dashboard, not through your own traces. At scale, the per-minute cost becomes the problem.

Vapi pricing (illustrative, mid-2026):
  Base platform:         ~$0.05/min
  STT (Deepgram via Vapi): ~$0.007/min
  LLM (GPT-4o-mini via Vapi): ~$0.004/min
  TTS (Cartesia via Vapi): ~$0.035/min
  Estimated total:       ~$0.096–0.15/min depending on provider tier

Direct provider cost for same stack:
  Deepgram Nova-3:       $0.0059/min
  GPT-4o-mini (cached):  ~$0.0003/min
  Cartesia Sonic 4:      ~$0.025/min
  Total:                 ~$0.031/min

Vapi markup:             ~3–5x at this volume

At 5,000 minutes a month that markup is $300–600 — noise. At 100,000 minutes it's $6,500–12,000 monthly over direct cost. Vapi's value proposition is that it makes the orchestration engineering not your problem; the question is whether that engineering is worth more or less than the markup at your volume.

Retell AI occupies the same tier as Vapi with stronger outbound campaign tooling and call analytics — worth evaluating if your use case is sales or appointment-scheduling at scale.

LiveKit Agents: open infrastructure, real telephony, real upside

LiveKit is a WebRTC media server and SDK that went open-source in 2021 and has been steadily adding AI-native features. The Agents SDK reached 1.0 in April 2025 and 1.5.x in April 2026 — it is now the de facto self-hosting substrate for production voice agents.

The architecture is worth understanding because it's different from Pipecat. LiveKit thinks in rooms — WebRTC media sessions with one or more participants. An agent is a participant in a room. When a user connects (via web, mobile, or phone), they join a room; the agent joins the same room and begins processing their audio track. This model scales cleanly to multi-participant scenarios — conference calls, real-time translation, supervised coaching sessions — that Vapi's phone-call mental model doesn't address.

flowchart TD
    PHONE[Inbound phone call] -->|SIP trunk| LK_SIP[LiveKit SIP gateway]
    WEB[Web app user] -->|WebRTC| LK_ROOM[LiveKit Room]
    LK_SIP --> LK_ROOM
    LK_ROOM --> AGENT[Agent process]
    AGENT --> STT[Deepgram / Whisper]
    STT --> LLM[Any LLM]
    LLM --> TTS[Cartesia / ElevenLabs]
    TTS --> LK_ROOM
    LK_ROOM --> PHONE
    LK_ROOM --> WEB
    style LK_ROOM fill:#0e7490,color:#fff
    style AGENT fill:#15803d,color:#fff

Native SIP and Phone Numbers shipped in 2025. You provision a phone number directly in LiveKit, configure a SIP trunk, and inbound calls arrive as room participants. The Twilio bridge that most LiveKit deployments used in 2023–2024 is no longer necessary. This matters: every additional service in the call path adds 20–80ms of latency and another failure domain.

Adaptive interruption handling in 1.5.x is the most operationally significant feature. Early voice agents had a binary choice: use energy-based VAD (fires on any silence, cuts users off mid-thought) or use semantic VAD (waits for grammatical completeness, ~100ms slower but far fewer false positives). LiveKit's adaptive system adjusts sensitivity based on conversation history — if the agent has been cutting users off, it backs off; if the conversation is turn-taking cleanly, it stays responsive. Tuning this yourself in Pipecat is a multi-week problem.

MCP tool support in 1.5.x means LiveKit agents can call tools via the Model Context Protocol, making it straightforward to integrate with external APIs without writing custom tool-call plumbing.

The self-hosting story is Docker or Kubernetes. LiveKit Server is a Go binary with a small footprint; a single instance handles thousands of concurrent rooms. LiveKit Cloud exists if you want managed WebRTC infrastructure without running the server yourself — you still own the agent process and provider integrations, but LiveKit handles media routing and TURN servers.

A practical LiveKit Agents agent looks like this:

from livekit.agents import WorkerOptions, cli
from livekit.agents.voice import Agent, AgentSession
from livekit.plugins import deepgram, cartesia, openai as lk_openai
from livekit.plugins.turn_detector.multilingual import MultilingualModel

class BillingAgent(Agent):
    def __init__(self) -> None:
        super().__init__(
            instructions="""You are a billing support agent.
            Answer concisely. If the customer is asking about their balance,
            call get_account_balance before responding.""",
        )

    async def on_enter(self):
        self.session.say("Hi, this is billing support. How can I help?")

async def entrypoint(ctx):
    session = AgentSession(
        stt=deepgram.STT(model="nova-3"),
        llm=lk_openai.LLM(model="gpt-4o-mini"),
        tts=cartesia.TTS(model="sonic-4", voice="helpful-woman"),
        # LiveKit's semantic turn-detector model; adaptive
        # interruption handling arrives with it in 1.5.x
        turn_detection=MultilingualModel(),
    )
    await session.start(ctx.room, agent=BillingAgent())

if __name__ == "__main__":
    cli.run_app(WorkerOptions(entrypoint_fnc=entrypoint))

The provider swap surface is clean: replace deepgram.STT with assemblyai.STT or a local Whisper wrapper, swap cartesia.TTS for elevenlabs.TTS. LiveKit's plugin ecosystem covers most of the common providers with consistent interfaces.

Pipecat: composable pipeline, maximum control

Pipecat is an open-source Python framework by Daily.co that reached v1.0 in April 2026. Where LiveKit thinks in rooms and participants, Pipecat thinks in pipeline frames — typed messages that flow through a processing chain. Audio input becomes an AudioRawFrame; STT produces a TranscriptionFrame; the LLM emits TextFrame tokens; TTS converts those back to AudioRawFrame. You build a pipeline by composing frame processors.

import asyncio
import os

from pipecat.audio.vad.silero import SileroVADAnalyzer
from pipecat.pipeline.pipeline import Pipeline
from pipecat.pipeline.runner import PipelineRunner
from pipecat.pipeline.task import PipelineTask
from pipecat.processors.aggregators.openai_llm_context import OpenAILLMContext
from pipecat.services.cartesia import CartesiaTTSService
from pipecat.services.deepgram import DeepgramSTTService
from pipecat.services.openai import OpenAILLMService
from pipecat.transports.services.daily import DailyParams, DailyTransport

async def main():
    room_url = os.getenv("DAILY_ROOM_URL")
    token = os.getenv("DAILY_TOKEN")
    transport = DailyTransport(
        room_url, token, "Billing Agent",
        DailyParams(audio_out_enabled=True, vad_enabled=True,
                    vad_analyzer=SileroVADAnalyzer())
    )

    stt = DeepgramSTTService(api_key=os.getenv("DEEPGRAM_API_KEY"),
                              model="nova-3")
    llm = OpenAILLMService(api_key=os.getenv("OPENAI_API_KEY"),
                            model="gpt-4o-mini")
    tts = CartesiaTTSService(api_key=os.getenv("CARTESIA_API_KEY"),
                              voice_id="helpful-woman")
    context = OpenAILLMContext(messages=[
        {"role": "system", "content": "You are a billing support agent."}
    ])
    context_agg = llm.create_context_aggregator(context)

    pipeline = Pipeline([
        transport.input(),
        stt,
        context_agg.user(),
        llm,
        tts,
        transport.output(),
        context_agg.assistant(),
    ])

    runner = PipelineRunner()
    await runner.run(PipelineTask(pipeline))

if __name__ == "__main__":
    asyncio.run(main())

The composability is the feature. You can inject custom frame processors anywhere in the chain — a profanity filter between LLM and TTS, a compliance logger that writes every transcript frame to a data store, a sentiment classifier that adjusts TTS speaking rate. In LiveKit you'd add this logic inside the agent class or in middleware; in Pipecat it's literally a list insertion.

Pipecat currently ships integrations for most major providers — Deepgram, AssemblyAI, Whisper for STT; Cartesia, ElevenLabs, Deepgram Aura for TTS; OpenAI, Anthropic, Gemini for LLM. The transport layer supports Daily.co rooms, raw WebRTC via aiortc, local audio, and WebSocket. SIP/PSTN requires a separate SIP adapter or bridging through Daily or Twilio.

The catch: everything above the pipeline frame abstraction is yours to own. VAD configuration, interruption handling, latency tuning, the sequence of events when a user barge-ins mid-sentence — Pipecat ships defaults that work in demos and need adjustment in production. The framework doesn't have LiveKit's adaptive interruption or semantic turn detection built in (though you can wire in OpenAI's Realtime API as the LLM service to get semantic VAD from that side).

What breaks

Voice agent infrastructure has a specific failure profile. Knowing it upfront determines which framework is actually appropriate.

Echo cancellation. When the agent plays audio through a speaker and the user's microphone picks it up, the STT model transcribes the agent's own output and generates a response to itself — a feedback spiral. Server-side AEC (acoustic echo cancellation) works for headphones. For speakerphones and low-end hardware, it frequently fails. Vapi and LiveKit both run AEC in the WebRTC layer; with Pipecat you rely on the transport layer's AEC, which is Daily's when using DailyTransport. If you're targeting phone calls on cheap devices, test AEC explicitly — don't discover this failure mode in production.

Double-talk. User and agent speaking simultaneously is genuinely unsolved at scale. In published evaluations, word error rate on overlapping speech runs 4–5× worse than on clean audio. Most pipelines queue user speech rather than handling it in parallel, which produces walkie-talkie conversations. If your use case is a high-banter conversational experience (therapy, coaching), budget significant time to test this.

Barge-in cascade. Interruption handling has a hidden failure mode: if the cancel logic is slow, the LLM continues generating and the TTS continues buffering frames after the user has already started a new turn. The user hears a partial sentence from the old turn, then the new response. In Pipecat you need to flush the pipeline explicitly on interruption — LiveKit's 1.5.x adaptive handling manages this automatically. Test interruption at 100ms, 300ms, and 700ms into agent speech; behavior often differs dramatically.

Transcript quality on phone calls. PSTN delivers 8 kHz narrowband audio. Most STT benchmarks use 16 kHz or 44.1 kHz clean audio. Deepgram Nova-3's published WER is measured on studio-quality recordings; Coval's independent benchmarks show 15–30% WER degradation under real call-center noise. If you're building for telephony, test STT providers on your actual audio — not on their published numbers.

LLM prompt assumptions. Prompts tuned for chat interfaces behave poorly in voice. Voice transcripts include filler words, truncated sentences from barge-ins, and ambiguous partial phrases. A prompt that says "Answer in 2–3 sentences" produces six sentences when the model tries to hedge against an unclear question. Write voice-specific prompts with explicit length constraints and instructions for handling ambiguous input.

{ "type": "voice-latency", "title": "Voice pipeline latency: sequential vs streaming" }

The break-even math

The decision matrix reduces to a single number: your monthly minute volume.

Volume-based decision:

<10K min/month:
  Vapi or Retell. Engineering cost to self-host >> SaaS markup.
  Focus on agent quality, not infrastructure.

10K–50K min/month:
  Re-evaluate quarterly. Vapi still likely cheaper when engineering cost included.
  Start building the self-hosted spike now so the migration isn't rushed.

50K–200K min/month:
  Self-host on LiveKit Agents. 60–80% cost reduction typically covers
  the engineering investment within 2–4 months.
  
>200K min/month:
  Self-host on Pipecat or LiveKit with full provider negotiation.
  At this scale, negotiate direct contracts with Deepgram/Cartesia —
  volume pricing reduces STT cost from $0.0059/min to ~$0.003/min.

Cost at 200K min/month (illustrative):

Vapi:              $0.12/min × 200K = $24,000/month
LiveKit self-hosted:
  Deepgram (volume):  $0.003/min
  GPT-4o-mini cached: $0.0003/min
  Cartesia (volume):  $0.018/min
  SIP trunk / PSTN:   ~$0.007/min (carrier termination — Vapi's price includes this)
  Agent compute:      ~$0.002/min (workers running VAD + turn detection per call)
  LiveKit Cloud:      ~$0.001/min (participant minutes; running the Go server
                      yourself swaps this for fixed compute instead)
  Total: ~$0.031/min × 200K = $6,200/month
Annual savings: ~$214,000

That $214K/year figure is before accounting for the engineering cost of owning the stack. Budget 4–8 weeks of a senior engineer's time to get a production-grade self-hosted deployment correct — VAD tuning, SIP integration, monitoring, the long tail of phone call edge cases. The math still favors self-hosting at 200K minutes, but it's not free.

Framework comparison

flowchart LR
    START{Monthly minutes?} -->|"<50K"| VAPI[Vapi or Retell\nManaged SaaS]
    START -->|">50K, rooms/video"| LK[LiveKit Agents\nSelf-hosted infra]
    START -->|">50K, max control\nor compliance logging"| PC[Pipecat\nOpen-source pipeline]
    VAPI --> V_OUT[Fast launch\nBuilt-in telephony\nHigher per-min cost]
    LK --> LK_OUT[Native SIP + WebRTC\nAdaptive turn detection\nProvider-swappable]
    PC --> PC_OUT[Composable processors\nFull pipeline control\nOwn the VAD tuning]

Dimension	Vapi	LiveKit Agents 1.5.x	Pipecat v1.0
Telephony (PSTN)	Built-in	Native SIP (2025)	Manual adapter
WebRTC transport	Managed	Self-hosted or LiveKit Cloud	Daily.co or aiortc
VAD / turn detection	Managed	Adaptive (1.5.x)	Manual / Silero
Interruption handling	Built-in	Adaptive, automated	Manual pipeline cancel
Multi-participant rooms	No	Yes (core feature)	Limited
MCP tool support	No	Yes (1.5.x)	Via LLM tool calls
Provider choice	Curated list	Open	Open
Self-host	No	Yes	Yes
Debugging surface	Dashboard + logs	Your traces + LiveKit metrics	Full Python stack traces
Best fit	Fast launch, low volume	Self-hosted, rooms, telephony	Max control, compliance

Latency optimization that applies to all three

Whichever framework you choose, the top-ROI optimizations are the same — the frameworks expose different handles for them.

Stream STT partials into the LLM. Don't wait for the final transcript. Feed partial transcriptions into the LLM context as they arrive and let the model begin prefill while the user is still finishing. This saves 200–400ms of serial waiting. LiveKit Agents and Pipecat both support this; Vapi handles it internally.

Flush TTS at the first sentence boundary. The LLM doesn't need to finish its full response before TTS starts. Detect the first . or ? in the token stream and send that sentence to TTS immediately. Users hear the first word at sentence-end rather than response-end — typically saves 200–500ms. This is explicit in Pipecat (you write the boundary detection), semi-automatic in LiveKit (configured per TTS plugin), and opaque in Vapi.

Cache the LLM system prompt. Voice agent system prompts are long (persona, instructions, tool schemas) and identical across turns. Anthropic prefix caching and OpenAI prompt caching both cache static prefixes for a reduced token rate. For a 2,000-token system prompt with GPT-4o-mini, this is a ~$0.0004/turn savings — small per call, but at 100K minutes with ~8 turns per call that's meaningful. More importantly, cached prefills skip re-processing, reducing TTFT by 100–300ms.

Regional inference routing. Place your STT, LLM, and TTS calls in the same cloud region as your WebRTC server. A New York WebRTC session routing STT to US-East, LLM to US-West, and TTS back to US-East adds 60–150ms of unnecessary cross-region latency per turn. LiveKit Cloud handles regional affinity automatically; in Pipecat you configure it per provider.

See latency budget engineering for the full breakdown of every millisecond.

The decision in practice

The question "which framework?" is really three separate questions.

What is your volume? If you're below 50K minutes a month, Vapi or Retell. Full stop. Don't spend engineering weeks building telephony plumbing to save $200/month — that's a bad trade. Above 50K minutes, self-hosting becomes economically sensible.

Do you need multi-participant rooms? If your use case involves more than two participants — group calls, real-time translation overlaid on video meetings, supervisors monitoring agent calls — LiveKit is the correct choice. Its room model was designed for this; Vapi and Pipecat were not.

How much do you need to own the audio processing logic? Compliance use cases sometimes require complete control over every frame in the pipeline — transcription logging, PII redaction in the audio stream, custom VAD models trained on specific speaker demographics. That's Pipecat territory. If your need is a straightforward Q&A or task-completion agent on the phone, the adaptive turn detection in LiveKit Agents 1.5.x is a better engineering investment than rolling your own in Pipecat.

One pattern that works at scale: start with Vapi to validate the product, migrate to LiveKit Agents when the monthly bill crosses $3,000–5,000, and consider Pipecat only when you have specific control requirements that LiveKit doesn't satisfy. The voice pipeline architecture and cascaded vs native speech-to-speech articles cover the underlying component decisions that remain consistent regardless of framework.

What most teams get wrong is treating this as a technical decision when it's primarily a financial and organizational one. The frameworks aren't meaningfully different in output quality — a well-tuned Pipecat pipeline and a well-tuned LiveKit pipeline produce nearly identical audio experiences. The difference is in who owns the complexity and what that complexity costs.