> ## Documentation Index > Fetch the complete documentation index at: https://docs.inceptionlabs.ai/llms.txt > Use this file to discover all available pages before exploring further. # Realtime Voice Assistant: Mercury 2.5 + ElevenLabs > Stream Mercury 2.5 in ElevenLabs for a low-latency voice assistant. This notebook demonstrates how to build a low-latency voice assistant using Mercury 2.5 for real-time intelligence combined with ElevenLabs' speech-to-text and text-to-speech models. We walk you through how to: 1. Convert text to speech using ElevenLabs TTS 2. Convert audio to text using ElevenLabs speech-to-text 3. Generate realtime responses with Mercury 2.5 4. Optimize latency using Mercury 2.5 streaming Directly via Google Colab or download the `.ipynb` to run it locally in Jupyter *** ## Dependencies Install packages and import the libraries required for the Mercury 2.5 SDK, ElevenLabs, and audio playback: ```python theme={null} %pip install \ "inceptionai" \ "elevenlabs>=2.15.0" \ "python-dotenv>=1.0.0" ``` ```python theme={null} import io import os import time import elevenlabs from dotenv import load_dotenv from inceptionai import Inception from IPython.display import Audio ``` ## API Keys Set up your API keys for both ElevenLabs and Inception Labs. 1. Create a `.env` in this directory 2. Add your API keys to the `.env`: * `INCEPTION_API_KEY` * `ELEVENLABS_API_KEY` The keys will be automatically loaded from the `.env` file. ```python theme={null} # Load environment variables from .env file load_dotenv() INCEPTION_API_KEY = os.getenv("INCEPTION_API_KEY") ELEVENLABS_API_KEY = os.getenv("ELEVENLABS_API_KEY") ``` ## Initialize Clients Create the ElevenLabs and Inception (Mercury 2.5) clients: ```python theme={null} assert INCEPTION_API_KEY is not None, ( "ERROR: INCEPTION_API_KEY not found. Please copy .env.example to .env and add your API keys." ) assert ELEVENLABS_API_KEY is not None, ( "ERROR: ELEVENLABS_API_KEY not found. Please copy .env.example to .env and add your API keys." ) mercury_client = Inception(api_key=INCEPTION_API_KEY) elevenlabs_client = elevenlabs.ElevenLabs( api_key=ELEVENLABS_API_KEY, base_url="https://api.elevenlabs.io" ) MERCURY_MODEL = "mercury-2.5" VOICE_ID = "21m00Tcm4TlvDq8ikWAM" # Rachel def mercury_completion(user_text, max_completion_tokens=1000, temperature=0.75): return mercury_client.chat.completions.create( model=MERCURY_MODEL, messages=[{"role": "user", "content": user_text}], max_completion_tokens=max_completion_tokens, temperature=temperature, reasoning_effort="low", ) def mercury_stream(user_text, max_completion_tokens=1000, temperature=0.75): stream = mercury_client.chat.completions.create( model=MERCURY_MODEL, messages=[{"role": "user", "content": user_text}], max_completion_tokens=max_completion_tokens, temperature=temperature, reasoning_effort="low", stream=True, ) for chunk in stream: if not chunk.choices: continue text = chunk.choices[0].delta.content if text: yield text ``` ## Create Input Audio We can simulate a user audio input using ElevenLabs text-to-speech model: ```python theme={null} audio = elevenlabs_client.text_to_speech.convert( voice_id=VOICE_ID, output_format="mp3_44100_128", model_id="eleven_v3", text="Hi, Mercury. ", ) audio_data = io.BytesIO() for chunk in audio: audio_data.write(chunk) Audio(audio_data.getvalue()) ``` ## Transcribe Speech into Text We then transcribe the audio input using ElevenLabs' speech-to-text model. We'll also measure the transcription latency: ```python theme={null} audio_data.seek(0) start_time = time.time() transcription = elevenlabs_client.speech_to_text.convert(file=audio_data, model_id="scribe_v1") end_time = time.time() transcription_time = end_time - start_time print(f"Transcribed text: {transcription.text}") print(f"Transcription time: {transcription_time:.2f} seconds") ``` ## Get a Response from Mercury 2.5 We send the transcribed text to Mercury 2.5 and measure the response time. We're using `mercury-2.5` for fast reasoning responses: ```python theme={null} start_time = time.time() completion = mercury_completion(transcription.text, max_completion_tokens=1000, temperature=0.75) mercury_response = completion.choices[0].message.content end_time = time.time() non_streaming_response_time = end_time - start_time print(mercury_response) print(f"\nResponse time: {non_streaming_response_time:.2f} seconds") ``` ## Real-time with Streaming Improve response latency by using Mercury 2.5 streaming. This allows us to receive the first tokens more quickly, reducing user-perceived latency: ```python theme={null} start_time = time.time() first_token_time = None mercury_full_response = "" for text in mercury_stream(transcription.text, max_completion_tokens=1000, temperature=0.75): mercury_full_response += text print(text, end="", flush=True) if first_token_time is None: first_token_time = time.time() streaming_time_to_first_token = first_token_time - start_time print( f"\n\nStreaming time to first token: {streaming_time_to_first_token:.2f} seconds - reducing response wait by {(non_streaming_response_time - streaming_time_to_first_token) * 100 / non_streaming_response_time:.2f}%" ) ``` We can also stream the text-to-speech response to reduce silence: ```python theme={null} start_time = time.time() first_audio_chunk_time = None audio_buffer = io.BytesIO() audio_generator = elevenlabs_client.text_to_speech.stream( voice_id=VOICE_ID, output_format="mp3_44100_128", text=mercury_full_response, model_id="eleven_turbo_v2_5", ) for chunk in audio_generator: if first_audio_chunk_time is None: first_audio_chunk_time = time.time() audio_buffer.write(chunk) streaming_tts_time_to_first_chunk = first_audio_chunk_time - start_time print(f"Streaming TTS time to first audio chunk: {streaming_tts_time_to_first_chunk:.2f} seconds") Audio(audio_buffer.getvalue()) ``` ## Streaming Mercury 2.5 Directly to TTS We've optimized Mercury 2.5 streaming and TTS separately. Now we can stream the model response and convert it to audio as soon as we have complete sentences, demarcated by `!`, `.`, `?`. ```python theme={null} import re sentence_pattern = re.compile(r"[.!?]+") sentence_buffer = "" audio_chunks = [] start_time = time.time() first_audio_time = None for text in mercury_stream(transcription.text, max_completion_tokens=1000, temperature=0.75): print(text, end="", flush=True) sentence_buffer += text if sentence_pattern.search(sentence_buffer): sentences = sentence_pattern.split(sentence_buffer) # Process all complete sentences (all but the last element) for i in range(len(sentences) - 1): complete_sentence = sentences[i].strip() if complete_sentence: audio_gen = elevenlabs_client.text_to_speech.stream( voice_id=VOICE_ID, output_format="mp3_44100_128", text=complete_sentence, model_id="eleven_turbo_v2_5", ) sentence_audio = io.BytesIO() for chunk in audio_gen: if first_audio_time is None: first_audio_time = time.time() sentence_audio.write(chunk) audio_chunks.append(sentence_audio.getvalue()) sentence_buffer = sentences[-1] if sentence_buffer.strip(): audio_gen = elevenlabs_client.text_to_speech.stream( voice_id=VOICE_ID, output_format="mp3_44100_128", text=sentence_buffer.strip(), model_id="eleven_turbo_v2_5", ) sentence_audio = io.BytesIO() for chunk in audio_gen: if first_audio_time is None: first_audio_time = time.time() sentence_audio.write(chunk) audio_chunks.append(sentence_audio.getvalue()) sentence_streaming_time_to_first_audio = first_audio_time - start_time print(f"\n\nTime to first audio: {sentence_streaming_time_to_first_audio:.2f} seconds") combined_pcm = b"".join(audio_chunks) Audio(combined_pcm) ``` In this sentence-by-sentence approach, each chunk is synthesized independently, so the audio can sound less natural than a fully context-aware streaming setup. To offload the voice loop to a managed platform, see how to set up Mercury 2.5 with the [ElevenLabs Agents platform](/integrations/elevenlabs). This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.