xAI has released a new Text to Speech API, designed to convert text into natural-sounding spoken audio. This API supports a range of expressive voices and allows for fine-grained control over speech delivery through inline speech tags. It also provides output formats from high-fidelity MP3 to telephony-optimized μ-law.
Key Points
- The Text to Speech API converts text into spoken audio with a single API call.
- It supports expressive voices and inline speech tags for delivery control.
- Output formats include high-fidelity MP3 and telephony-optimized μ-law.
- The Responses API is the preferred method for interacting with xAI models, allowing optional stateful interactions.
- Conversation history via the Responses API is stored for 30 days on xAI's servers.
- The Speech to Speech API enables real-time voice conversations over WebSocket.
- Speech to Speech API pricing starts at $0.05 per minute for audio or $3.00 per hour, plus $0.004 per text input message.
Context
According to xAI documentation, the Text to Speech API is part of a broader suite of model capabilities that includes text generation, image generation, and other voice functionalities like Speech to Speech and Custom Voices. The Responses API is highlighted as the primary interface for model interaction, offering stateful conversations where previous prompts and responses are saved on xAI's servers for 30 days. This allows users to continue interactions without resending full conversation histories. For Python users, an xAI SDK is available, utilizing gRPC for performance.
The Speech to Speech API, as detailed by xAI, supports real-time voice conversations via WebSocket. It includes function calling capabilities with features such as web search, X search, collections, MCP, and custom functions. Pricing for this API is based on audio duration and text input messages, with specific rates for audio and text events. xAI also offers "Collections," which allow users to upload and search through documents, integrating enterprise knowledge bases for applications like RAG. Collections provide persistent document storage with semantic search across multiple documents and support various UTF-8 encoded text file types, with special conversion and chunking techniques for certain MIME types.
Why It Matters
Builders can leverage the Text to Speech API for applications requiring natural-sounding audio output, while the Responses API streamlines conversational AI development by managing conversation state. The Speech to Speech API offers real-time voice interaction capabilities, and "Collections" provide a structured way to integrate proprietary data for enhanced model performance, impacting how developers manage and utilize conversational and retrieval-augmented generation systems.
What To Do
- Review the xAI documentation for the Text to Speech API to understand supported voices and output formats.
- Explore the Responses API documentation to understand its stateful interaction capabilities and the 30-day data retention policy.
- Compare the pricing structure of the Speech to Speech API for audio duration and text input messages.
- Investigate the "Collections" feature for integrating custom data and building RAG applications.
Keep Exploring
/techniques/output-schema /techniques/system-user-separation /techniques/multimodal-grounding /studio?pack=foundation /atlas/**grok**-family /atlas/image-models /techniques/function-calling /techniques/rag