Skip to main content
These guides connect a phone call to a provider-hosted voice agent pipeline: cascaded STT, LLM, and TTS stages running behind a single realtime endpoint. The provider orchestrates the stages. You configure each one, often including which LLM to use, without operating any of them yourself.
PipelineVoice Agent Pipeline
  1. Phone Call
  2. Plivo
    WebSocket
  3. Your ServerRelay
    WebSocket
  4. Voice Agent PipelineSTTLLMTTS
Your server sits between two WebSocket connections: Plivo streams the caller’s audio over one, and the provider’s realtime endpoint is on the other. The server relays audio between the two and translates each side’s message format. The pipeline itself runs entirely on the provider’s side.

​
Why a Hosted Pipeline

  • Control per stage: pair the LLM you want with the voice you want, and swap either independently
  • Provider-managed orchestration: turn-taking, interruption, and stage hand-offs are the provider’s job, not yours
  • Text in the loop: transcripts of both sides come for free, which makes logging, evaluation, guardrails, and compliance straightforward
  • Mature building blocks: the LLM stage is a standard text model, so instruction adherence and tool calling behave the way they do in text applications
Products in this category include Deepgram’s Voice Agent API, ElevenLabs Agents, and Speechmatics Flow.

​
Trade-Offs

  • Compared with speech-to-speech models: the cascade adds latency, one hop per stage, and tone and emotion in the caller’s voice are lost when audio is flattened to text between stages. If minimal latency and audio-native conversation matter most, see Speech-to-Speech Models.
  • Compared with orchestrating your own pipeline: whether with Pipecat or a fully custom pipeline built on the raw audio stream, self-orchestration lets you use any STT, LLM, or TTS service and own the turn-taking logic. A hosted pipeline limits you to the stages, providers, and behaviors its platform supports.
  • Vendor lock-in: your agent’s configuration, prompts, tools, and tuning live inside one provider’s platform, and its per-stage choices only span that provider’s partner catalog. Migrating later means rebuilding the agent, not just repointing an endpoint.

​
Other Ways to Build

Speech-to-Speech Models

A single audio-native model instead of a cascade, for the lowest latency

Pipecat

An open-source framework that manages the Plivo transport and lets you compose your own STT, LLM, and TTS pipeline in code