> ## Documentation Index
> Fetch the complete documentation index at: https://mantle.rasa.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Voice agents

> Build voice agents that hold natural, real-time conversations, with streaming speech, fast turn-taking and interruptions handled for you.

A voice agent in Mantle is the same project as a chat agent, with a voice channel
switched on. Your skills, memory, tools, responses, and control levers do not
change. Mantle detects that a turn arrived on a voice channel and adapts how
the agent speaks, listens, and waits.

That means you build and test one agent, and it serves both channels. Voice
just adds a few things worth tuning.

## How a voice turn works

```mermaid theme={null}
flowchart LR
    A[Caller speaks] --> B[Speech recognition]
    B -->|transcript| C[Mantle turn]
    C -->|text, streamed| D[Speech synthesis]
    D --> E[Caller hears the reply]
```

1. **Speech recognition** turns the caller's audio into text and decides when
   they have finished speaking.
2. **Mantle runs the turn** exactly as it does for chat: it picks the skill,
   runs tools, enforces your constraints, and writes memory.
3. **Speech synthesis** turns the agent's reply into audio. Providers that
   support streaming text input can start speaking before the whole response
   is written. Custom engines must implement the
   [streaming input interface](https://mantle.rasa.com/reference/custom-speech-providers.md#custom-speech-synthesis)
   to do this.

The speech recognition and synthesis providers are model groups in
`integrations.yml`. A voice channel names a group on `asr` and `tts`, or you set
`voice.asr` and `voice.tts` in `agent.yml` for every voice channel. The channel
value wins. Recognition and synthesis are chosen separately, so you can mix
providers, for example Deepgram for recognition and Rime for synthesis. Your
skills do not change when you switch provider. See
[Speech providers](https://mantle.rasa.com/reference/speech-providers.md).

## What Mantle handles for you

| On a voice call | What happens | Where to change it |
| - | - | - |
| **Spoken style** | The model is told to speak in short sentences, ask one question at a time, avoid formatting, and say dates, amounts, URLs, and email addresses aloud | [`orchestrator.voice_rules`](https://mantle.rasa.com/reference/agent-yml#orchestrator) |
| **Pauses while tools run** | The model says a short acknowledgement before calling a tool | [`fillers.enabled` and `fillers.examples`](https://mantle.rasa.com/reference/agent-yml#fillers) |
| **Interruptions** | When the caller talks over the agent, playback stops. A tool that is already running finishes, then the agent answers what the caller said. On by default in the Inspector; turn it on for telephony channels. | [Interruptions](https://mantle.rasa.com/reference/integrations-yml.md#interruptions) |
| **Backchannels** | With interruptions on, short speech that is classified as an acknowledgement does not interrupt the agent | [Interruptions](https://mantle.rasa.com/reference/integrations-yml.md#interruptions) |
| **Silence** | After the silence timeout, the agent checks in. After three check-ins, it says goodbye and hangs up | [Silence timeout](https://mantle.rasa.com/reference/builtin-skills.md#silence-timeout) |
| **Buttons** | Response buttons are read aloud as a list of their titles | [Responses](https://mantle.rasa.com/concepts/responses.md#static-buttons) |
| **Call start** | The channel sends `/session_start` when the call connects, so a greeting skill can speak first | [Start a conversation](https://mantle.rasa.com/mantle/runtime-loop.md#start-a-conversation) |

None of this needs changes to your skills. The same skill keeps its text
behavior on chat channels.

## What you control

These are the decisions worth making for a voice agent. Each links to how.

* **Who speaks first.** Replace the silent `builtin_session_start` with a
  greeting, and add any recording notice there. See
  [Designing for voice](https://mantle.rasa.com/voice/designing-for-voice.md#speak-first-when-the-caller-expects-it).
* **How much the agent says at once.** Lists and long explanations need
  breaking up for a listener. See
  [Read lists in small pieces](https://mantle.rasa.com/voice/designing-for-voice.md#read-lists-in-small-pieces).
* **Exact wording for numbers.** Order numbers and references belong in
  [responses](https://mantle.rasa.com/concepts/responses.md), not in model-generated text.
* **Confirmation before side effects.** Use
  [`requires_confirmation`](https://mantle.rasa.com/build-guide/tool-constraints#explicit-confirmations)
  so the caller agrees out loud before anything irreversible.
* **Keypad input.** A collect step can accept digits from the phone keypad. See
  [DTMF collect steps](https://mantle.rasa.com/reference/ordered-block-steps.md#dtmf).
* **Ending the call.** An ordered-block `execute_tool: hangup` ends the call after an
  authored goodbye. See [Step kinds](https://mantle.rasa.com/reference/ordered-block-steps.md#step-kinds).
* **Voice and recognition.** Choose the speech providers, models, and voice per
  language. See [Speech providers](https://mantle.rasa.com/reference/speech-providers.md).

## Try it

The Inspector has a voice mode built in. With an ASR and TTS model enabled, run `rasa inspect` and select **Start voice conversation**. See
[Getting Started](https://mantle.rasa.com/getting-started.md#7-talk-to-your-agent).

## Learn from a complete community project

The [Rasa Community voice tutorial](https://rasa.community/library/tutorials/voice-ai-agent/)
builds a travel agent with references, tools, confirmations, and ordered steps.
Use it when you want to see several skills working together after completing
the [card replacement tutorial](https://mantle.rasa.com/voice/tutorial.md).
Check the tutorial's tested Rasa version and compatibility notes before adapting
its examples; the configuration reference in these docs describes this Mantle
version.

> [card] [Rasa Community voice tutorial](https://rasa.community/library/tutorials/voice-ai-agent/)
>
> Follow the travel-agent walkthrough on rasa.community.

> [card] [Runnable community agent](https://github.com/RasaHQ/rasa-community-resources/tree/4aa0c4419dc193fef7a969c12d59edcf720f2606/tutorials/rasa-voice-agent-tutorial)
>
> Open the companion project's setup instructions, skills, and dependency pins.

## Next steps

> [card] [Build a voice agent](https://mantle.rasa.com/voice/tutorial.md)
>
> A step-by-step tutorial, from first draft to phone-ready.

> [card] [Designing for voice](https://mantle.rasa.com/voice/designing-for-voice.md)
>
> Patterns for skills and responses that work well out loud.

> [card] [Connect a phone line](https://mantle.rasa.com/voice/telephony.md)
>
> Put the agent behind a real phone number.

> [card] [Test and monitor voice](https://mantle.rasa.com/voice/testing.md)
>
> Listen in the Inspector, then watch latency and interruptions in production.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.