> ## Documentation Index
> Fetch the complete documentation index at: https://plivo.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Speech-to-Speech Model Integrations

> Connect phone calls directly to speech-to-speech models over WebSockets, with no framework and no intermediate text

These guides connect a phone call directly to a speech-to-speech model: a single model that listens and speaks natively, with no intermediate text step. One model handles the full conversation, including speech recognition, understanding, response generation, and voice synthesis, as well as the conversational mechanics a voice agent needs: voice activity detection, turn handling, and barge-in when the caller interrupts. Configuration is minimal. You pick a model, choose a voice, and write instructions.

<div className="sipflow sipflow--vertical">
  <div className="sipflow__head">
    <span>Pipeline</span>

    <span className="sipflow__head-right">
      <span className="sipflow__led" aria-hidden="true" />

      Speech-to-Speech
    </span>
  </div>

  <ol className="sipflow__body">
    <li>
      <div className="sipflow__node">
        <span className="sipflow__node-body">
          <span className="sipflow__title">Phone Call</span>
        </span>
      </div>

      <div className="sipflow__link sipflow__link--bi" aria-hidden="true">
        <span className="sipflow__arrow sipflow__arrow--back" />

        <span className="sipflow__rail" />

        <span className="sipflow__arrow" />
      </div>
    </li>

    <li>
      <div className="sipflow__node">
        <span className="sipflow__node-body">
          <span className="sipflow__title">Plivo</span>
        </span>
      </div>

      <div className="sipflow__link sipflow__link--bi">
        <span className="sipflow__arrow sipflow__arrow--back" aria-hidden="true" />

        <span className="sipflow__rail" aria-hidden="true" />

        <span className="sipflow__edge-label">WebSocket</span>

        <span className="sipflow__arrow" aria-hidden="true" />
      </div>
    </li>

    <li>
      <div className="sipflow__node">
        <span className="sipflow__node-body">
          <span className="sipflow__title">Your Server</span>
          <span className="sipflow__sub">Relay</span>
        </span>
      </div>

      <div className="sipflow__link sipflow__link--bi">
        <span className="sipflow__arrow sipflow__arrow--back" aria-hidden="true" />

        <span className="sipflow__rail" aria-hidden="true" />

        <span className="sipflow__edge-label">WebSocket</span>

        <span className="sipflow__arrow" aria-hidden="true" />
      </div>
    </li>

    <li>
      <div className="sipflow__node">
        <span className="sipflow__node-body">
          <span className="sipflow__title">Speech-to-Speech Model</span>
          <span className="sipflow__sub">Realtime API</span>
        </span>
      </div>
    </li>
  </ol>
</div>

Your server sits between two WebSocket connections: Plivo streams the caller's audio over one, and the model's realtime endpoint is on the other. The server relays audio between the two and translates each side's message format. In most cases the audio itself passes through untouched.

***

## Why Speech-to-Speech

* **Lowest latency**: one model turn instead of three chained stages
* **Natural conversations**: tone, emotion, and context survive because audio is never flattened to text
* **Simple architecture**: turn handling and interruption live inside the model, so your server stays small

***

## Trade-Offs

* **Observability**: there is no text between stages to inspect. You depend on the transcripts the model chooses to emit, which makes debugging, evaluation, and monitoring harder than with a pipeline that produces text at every hop.
* **Steerability**: instruction adherence in audio-native models is generally less mature than in text LLMs. A pipeline lets you use a proven text model whose prompting behavior you already understand.
* **Tool calling**: function calling from a speech-to-speech model can be less consistent than from a text LLM, which matters if your agent depends on reliable structured actions.
* **Component choice**: you adopt the model as a whole. You cannot swap in a different LLM or a specific voice vendor, and pricing is bundled rather than per stage.

If these trade-offs matter more than latency for your use case, you have three alternatives, in increasing order of control: [Voice Agent Pipelines](/docs/voice-agents/audio-streaming/integration-guides/voice-agent-pipelines/overview) are the fastest to adopt but tie you to one provider's stack; [Pipecat](/docs/voice-agents/audio-streaming/integration-guides/pipecat/overview) lets you compose the STT, LLM, and TTS services you choose within an open-source framework; and a fully custom pipeline, where you build the orchestration yourself on the raw [audio stream](/docs/voice-agents/audio-streaming/concepts/audio-streaming-guide), gives you complete ownership of every stage.

***

## Other Ways to Build

<CardGroup cols={2}>
  <Card title="Voice Agent Pipelines" icon="workflow" href="/docs/voice-agents/audio-streaming/integration-guides/voice-agent-pipelines/overview">
    Provider-hosted STT, LLM, and TTS stages behind one realtime endpoint, when you want control over each stage
  </Card>

  <Card title="Pipecat" icon="https://mintcdn.com/plivo/rP0r53Ttc3NtJi4l/images/pipecat-logo.svg?fit=max&auto=format&n=rP0r53Ttc3NtJi4l&q=85&s=a479c973bdfcce8625fa3050a825bff7" href="/docs/voice-agents/audio-streaming/integration-guides/pipecat/overview" width="24" height="24" data-path="images/pipecat-logo.svg">
    An open-source framework that manages the Plivo transport and composes services, including speech-to-speech models, in code
  </Card>
</CardGroup>
