xAI has released a Speech to Speech API designed for real-time voice conversations. This API operates over WebSocket and supports function calling, including capabilities such as web search, X search, collections, MCP, and custom functions. Billing for the service is calculated by the minute of audio processed and a flat fee per text input message.
Key Points
- The Speech to Speech API enables real-time voice conversations via WebSocket.
- The API supports function calling for web search, X search, collections, MCP, and custom functions.
- Audio processing is billed at $0.05 per minute (or $3.00 per hour).
- Text input messages are billed at $0.004 per message for each
conversation.item.createevent. - The service is hosted in the us-east-1 region.
function_call_outputitems and items withinput_audiooraudiocontent are not billed as text input messages.
Context
According to xAI, the Speech to Speech API charges based on the duration of audio sent or received, and for text events that are sent without accompanying audio. Each conversation.item.create event sent from a client is billed, with specific exceptions for server-requested tool results and audio-content items.
Why It Matters
This API provides developers with a tool for integrating real-time voice interactions into applications, offering a structured pricing model for both audio and text components of conversations. The inclusion of function calling expands the utility of these voice interactions.
What To Do
- Review the Speech to Speech API documentation for detailed integration instructions.
- Test the API's capabilities in the provided Playground environment.
- Note the pricing structure for audio and text inputs when planning application costs.
- Explore the API reference for WebSocket endpoint details.
Keep Exploring
/atlas/grok-family
