Rime

Text-to-speech service implementations using Rime AI

Overview

Rime AI provides two TTS service implementations: RimeTTSService (WebSocket-based) with word-level timing and interruption support, and RimeHttpTTSService (HTTP-based) for simpler use cases. RimeTTSService is recommended for real-time interactive applications.

Rime TTS API Reference

Pipecat’s API methods for Rime TTS integration

Example Implementation

Complete example with word timestamps

Rime Documentation

Official Rime WebSocket and HTTP API documentation

Voice Models

Explore available voice models and features

Installation

To use Rime services, install the required dependencies:

uv add "pipecat-ai[rime]"

Prerequisites

Rime Account Setup

Before using Rime TTS services, you need:

1. Rime Account: Sign up at Rime AI
2. API Key: Generate an API key from your account dashboard
3. Voice Selection: Choose from available voice models

Required Environment Variables

• RIME_API_KEY: Your Rime API key for authentication

Configuration

RimeTTSService

api_key · str · required

Rime API key for authentication.

voice_id · str · required · deprecated

ID of the voice to use for synthesis. Deprecated in v0.0.105. Use settings=RimeTTSService.Settings(voice=...) instead.

url · str · default: "wss://users-ws.rime.ai/ws3"

Rime WebSocket API endpoint.

model · str · default: "coda" · deprecated

Model ID to use for synthesis. Deprecated in v0.0.105. Use settings=RimeTTSService.Settings(model=...) instead.

sample_rate · int · default: "None"

Output audio sample rate in Hz. When None, uses the pipeline’s configured sample rate.

text_aggregation_mode · TextAggregationMode · default: "TextAggregationMode.SENTENCE"

Controls how incoming text is aggregated before synthesis. SENTENCE (default) buffers text until sentence boundaries, producing more natural speech. TOKEN streams tokens directly for lower latency. Import from pipecat.services.tts_service.

aggregate_sentences · bool · default: "None"

Deprecated in v0.0.104. Use text_aggregation_mode instead.

params · InputParams · default: "None" · deprecated

Deprecated in v0.0.105. Use settings=RimeTTSService.Settings(...) instead.

settings · RimeTTSService.Settings · default: "None"

Runtime-configurable settings. See RimeTTSService Settings below.

RimeHttpTTSService

api_key · str · required

Rime API key for authentication.

voice_id · str · required · deprecated

ID of the voice to use for synthesis. Deprecated in v0.0.105. Use settings=RimeHttpTTSService.Settings(voice=...) instead.

aiohttp_session · aiohttp.ClientSession · required

An aiohttp session for HTTP requests.

model · str · default: "coda" · deprecated

Model ID to use for synthesis. Deprecated in v0.0.105. Use settings=RimeHttpTTSService.Settings(model=...) instead.

sample_rate · int · default: "None"

Output audio sample rate in Hz. When None, uses the pipeline’s configured sample rate.

params · InputParams · default: "None" · deprecated

Deprecated in v0.0.105. Use settings=RimeHttpTTSService.Settings(...) instead.

settings · RimeHttpTTSService.Settings · default: "None"

Runtime-configurable settings. See RimeTTSService Settings below.

RimeNonJsonTTSService

A non-JSON WebSocket service for models like Arcana that use plain text messages.

api_key · str · required

Rime API key for authentication.

voice_id · str · required · deprecated

ID of the voice to use for synthesis. Deprecated in v0.0.105. Use settings=RimeNonJsonTTSService.Settings(voice=...) instead.

url · str · default: "wss://users.rime.ai/ws"

Rime WebSocket API endpoint.

model · str · default: "arcana" · deprecated

Model ID to use for synthesis. Deprecated in v0.0.105. Use settings=RimeNonJsonTTSService.Settings(model=...) instead.

audio_format · str · default: "pcm"

Audio output format.

sample_rate · int · default: "None"

Output audio sample rate in Hz. When None, uses the pipeline’s configured sample rate.

text_aggregation_mode · TextAggregationMode · default: "TextAggregationMode.SENTENCE"

Controls how incoming text is aggregated before synthesis. SENTENCE (default) buffers text until sentence boundaries. TOKEN streams tokens directly for lower latency. Import from pipecat.services.tts_service.

aggregate_sentences · bool · default: "None" · deprecated

Deprecated in v0.0.104. Use text_aggregation_mode instead.

params · InputParams · default: "None" · deprecated

Deprecated in v0.0.105. Use settings=RimeNonJsonTTSService.Settings(...) instead.

settings · RimeNonJsonTTSService.Settings · default: "None"

Runtime-configurable settings. See RimeNonJsonTTSService Settings below.

RimeTTSService Settings

Runtime-configurable settings passed via the settings constructor argument using RimeTTSService.Settings(...). These can be updated mid-conversation with TTSUpdateSettingsFrame. See Service Settings for details.

Parameter	Type	Default	Description
model	str	None	Model identifier. (Inherited.)
voice	str	None	Voice identifier. (Inherited.)
language	Language | str	None	Language for synthesis. (Inherited.)
segment	str	NOT_GIVEN	Segment type for synthesis.
speedAlpha	float	NOT_GIVEN	Speed alpha parameter.
reduceLatency	bool	NOT_GIVEN	Whether to reduce latency.
pauseBetweenBrackets	bool	NOT_GIVEN	Pause between brackets.
phonemizeBetweenBrackets	bool	NOT_GIVEN	Phonemize between brackets.
noTextNormalization	bool	NOT_GIVEN	Disable text normalization.
saveOovs	bool	NOT_GIVEN	Save out-of-vocabulary words.
inlineSpeedAlpha	str	NOT_GIVEN	Inline speed alpha.
repetition_penalty	float	NOT_GIVEN	Token repetition penalty (arcana only, 1.0-2.0).
temperature	float	NOT_GIVEN	Sampling temperature (arcana only, 0.0-1.0).
top_p	float	NOT_GIVEN	Cumulative probability threshold (arcana only, 0.0-1.0).
timeScaleFactor	float	NOT_GIVEN	Audio playback speed factor (arcana, mistv3, and coda only). Values above 1.0 slow down the audio; values below 1.0 speed it up.

RimeNonJsonTTSService Settings

Runtime-configurable settings passed via the settings constructor argument using RimeNonJsonTTSService.Settings(...). These can be updated mid-conversation with TTSUpdateSettingsFrame. See Service Settings for details.

Parameter	Type	Default	Description
model	str	None	Model identifier. (Inherited.)
voice	str	None	Voice identifier. (Inherited.)
language	Language | str	None	Language for synthesis. (Inherited.)
segment	str	NOT_GIVEN	Segment type for synthesis.
repetition_penalty	float	NOT_GIVEN	Token repetition penalty.
temperature	float	NOT_GIVEN	Sampling temperature.
top_p	float	NOT_GIVEN	Cumulative probability threshold.

Usage

Basic Setup (WebSocket)

from pipecat.services.rime import RimeTTSService

tts = RimeTTSService(
    api_key=os.getenv("RIME_API_KEY"),
    settings=RimeTTSService.Settings(
        voice="cove",
    ),
)

With Customization (WebSocket)

from pipecat.services.rime import RimeTTSService
from pipecat.transcriptions.language import Language

tts = RimeTTSService(
    api_key=os.getenv("RIME_API_KEY"),
    settings=RimeTTSService.Settings(
        voice="cove",
        model="mistv2",
        language=Language.ES,
        speedAlpha=1.2,
        reduceLatency=True,
    ),
)

HTTP Service

import aiohttp
from pipecat.services.rime import RimeHttpTTSService

async with aiohttp.ClientSession() as session:
    tts = RimeHttpTTSService(
        api_key=os.getenv("RIME_API_KEY"),
        settings=RimeHttpTTSService.Settings(
            voice="cove",
        ),
        aiohttp_session=session,
    )

Non-JSON WebSocket (Arcana)

from pipecat.services.rime import RimeNonJsonTTSService

tts = RimeNonJsonTTSService(
    api_key=os.getenv("RIME_API_KEY"),
    settings=RimeNonJsonTTSService.Settings(
        voice="cove",
        model="arcana",
    ),
)

Customizing Speech

RimeTTSService provides a set of helper methods for implementing Rime-specific customizations, meant to be used as part of text transformers. These include methods for spelling out text, adjusting speech rate, and modifying pitch. See the Text Transformers for TTS section in the Text-to-Speech guide for usage examples.

RimeTTSService.SPELL(text: str) -> str

Static method that implements Rime’s spell function to spell out text character by character. Call this method on the class, not on an instance.

# Text transformers for TTS
# This will insert Rime's spell tags around the provided text.
async def spell_out_text(text: str, type: str) -> str:
    return RimeTTSService.SPELL(text)

tts = RimeTTSService(
    api_key=os.getenv("RIME_API_KEY"),
    text_transforms=[
        ("phone_number", spell_out_text),
    ],
)

RimeTTSService.PAUSE_TAG(seconds: float) -> str

Static method that implements Rime’s custom pause functionality to generate a properly formatted pause tag you can insert into the text. Call this method on the class, not on an instance.

# Text transformers for TTS
# This will insert a one second pause after questions.
async def pause_after_questions(text: str, type: str) -> str:
    if text.endswith("?"):
        return f"{text}{RimeTTSService.PAUSE_TAG(1.0)}"
    return text

tts = RimeTTSService(
    api_key=os.getenv("RIME_API_KEY"),
    text_transforms=[
        ("sentence", pause_after_questions), # Only apply to sentence aggregations
    ],
)

PRONOUNCE(self, text: str, word: str, phoneme: str) -> str:

Convenience method to support Rime’s custom pronunciations feature. It takes a word and its desired phoneme representation, returning the text with the provided word replaced by the appropriate phoneme tag.

# Text transformers for TTS
# This will a phoneme in place of the word "potato" to define how it
# should be pronounced.
async def maybe_say_potato_all_fancylike(text: str, type: str) -> str:
    if using_fancy_voice:
        return RimeTTSService.PRONOUNCE(text, "potato", "potato")
    else:
        return RimeTTSService.PRONOUNCE(text, "potato", "poteto")

tts = RimeTTSService(
    api_key=os.getenv("RIME_API_KEY"),
    text_transforms=[
        ("*", maybe_say_potato_all_fancylike), # Apply to all text
    ],
)

INLINE_SPEED(self, text: str, speed: float) -> str:

A convenience method to support Rime’s inline speed adjustment feature. It will wrap the provided text in the [] tags and add the provided speed to the inlineSpeedAlpha field in the request metadata.

# Text transformers for TTS
# This will make the word slow always be spoken more slowly.
async def slow_down_slow_words(text: str, type: str) -> str:
    return text.replace(
        "slow",
        RimeTTSService.INLINE_SPEED("slow", speed=0.5)
    )

tts = RimeTTSService(
    api_key=os.getenv("RIME_API_KEY"),
    text_transforms=[
        ("*", slow_down_slow_words), # Apply to all text
    ],
)

The InputParams / params= pattern is deprecated as of v0.0.105. Use Settings / settings= instead. See the Service Settings guide for migration details.

Notes

• Word-level timestamps: RimeTTSService provides word-level timing information, enabling synchronized text highlighting.
• WebSocket vs HTTP: The WebSocket service supports word-level timestamps, interruption handling, and maintains context across messages within a turn. The HTTP service is simpler but lacks these features.
• Non-JSON WebSocket: RimeNonJsonTTSService is for models like Arcana that use plain text messages instead of JSON. It does not support word-level timestamps.

Event Handlers

Rime WebSocket TTS services support the standard service connection events:

Event	Description
on_connected	Connected to Rime WebSocket
on_disconnected	Disconnected from Rime WebSocket
on_connection_error	WebSocket connection error occurred

@tts.event_handler("on_connected")
async def on_connected(service):
    print("Connected to Rime")