IndexTTS2: Bilibili's Open Source Zero-Shot TTS with Emotion Control and Multilingual Voice Cloning

Traditional TTS either needs per-speaker training data or produces flat, unemotional speech. IndexTTS2 solves three practical problems: it clones a voice from a short reference clip without fine-tuning; it controls emotion via an emotional audio prompt, an 8-dimensional vector, or natural language; and it offers pinyin-level control to reduce Chinese mispronunciations. The proposed duration-control mechanism also points toward future dubbing with lip sync.

Text-to-speech has moved from robotic reading to expressive generation. IndexTTS2 brings zero-shot voice cloning, emotion control, and multilingual synthesis together in one system backed by a major video platform. That makes it particularly relevant for video dubbing, content creation, and personalized voice assistants. The custom license limits some commercial uses, but the release still gives developers and researchers a state-of-the-art reference implementation that can run locally.

Strictly speaking, this is not open source in the OSI sense. The code is public, but the repository uses Bilibili's custom Model Use License Agreement (SPDX: NOASSERTION). It grants a worldwide, non-exclusive, royalty-free limited license, but with important restrictions: if your products/services have more than 100 million monthly active users or annual revenue above RMB 1 billion, you must obtain a separate written license from Bilibili; you may not use the model or derivative works to improve any AI model except for non-commercial AI models or Bilibili's IndexTTS2 itself. Model weights must also be downloaded separately from Hugging Face or ModelScope.

If you are not a developer, you can still use IndexTTS2 once someone installs it for you. It ships with a local WebUI: after running `uv run webui.py`, open http://127.0.0.1:7860, upload a reference audio, type your text, choose emotion and language, and generate speech. No external API account is needed. However, there is no one-click installer or public hosted demo, so non-technical users should wait for a community wrapper or an official online playground.

Deployment steps: 1) Prepare a machine with an NVIDIA GPU and CUDA Toolkit 12.8+; 2) Install git, git-lfs, and the uv package manager; 3) Clone the repository using HTTPS or SSH (note that the repository history was reset, so delete any old local clone first); 4) Run `uv sync --all-extras` in the project directory; 5) Download the IndexTTS-2 model weights from Hugging Face or ModelScope and place them in the expected path; 6) Start the WebUI with `uv run webui.py`; 7) Open http://127.0.0.1:7860. Optional acceleration flags include `--fp16`, `--accel`, and `--torch_compile`. Small auxiliary models are downloaded automatically on first run. This is not a beginner-friendly setup, but it is manageable for any team with GPU infrastructure.

According to the paper, IndexTTS2 achieves state-of-the-art results in word error rate, speaker similarity, and emotional fidelity. However, the current release does not enable the promised 'precise duration control', so exact lip-sync dubbing is not possible yet. In addition, enabling random sampling during inference reduces voice cloning fidelity. The system is best suited for emotionally expressive audiobooks, multilingual voiceovers, and low-latency voice cloning, not for phoneme-level editing tasks.

Overview

IndexTTS2 is an industrial-grade zero-shot text-to-speech system open-sourced by Bilibili. With a single reference audio, it can clone a speaker's voice and synthesize Chinese, English, and cross-lingual speech. The release includes the model weights and a paper, positioning it as a controllable and efficient zero-shot TTS system.

Key capabilities:

• **Zero-shot voice cloning**: Clone a voice from one reference clip without fine-tuning.
• **Emotion control**: Use an emotional audio prompt, an 8-dimensional emotion vector, or natural language descriptions to shape tone.
• **Pinyin-level Chinese control**: Specify pronunciation to reduce errors on polyphonic characters.
• **Multilingual / cross-lingual**: Chinese and English are demonstrated; cross-lingual synthesis preserves the source voice.
• **Duration control (not enabled in this release)**: The paper proposes a precise duration-control architecture, but the current version does not yet expose it.

What Problem Does It Solve?

Conventional TTS has three common pain points: voice cloning requires large amounts of training data, synthesized speech sounds flat, and Chinese polyphonic characters are often mispronounced. IndexTTS2 avoids fine-tuning with zero-shot cloning, adds expressiveness through emotion prompts, and improves Chinese accuracy with pinyin-level control. Ideal use cases include video dubbing, audiobooks, character voices, and multilingual content creation.

Open Source and License

This is a 'source-available' project with a custom license, not traditional open source. Key points:

• Small companies can use it commercially, but must comply with all terms.
• Products with more than 100 million MAU or annual revenue above RMB 1 billion require a separate written license from Bilibili.
• You may not use the model or derived works to improve other AI models (except for non-commercial AI models).
• Model weights are downloaded separately.

For commercial inquiries: indexspeech@bilibili.com

Deployment

The barrier is moderate: you need a GPU, CUDA 12.8+, uv, and manual weight download. Once the environment is ready, the WebUI is easy to use:

```bash git clone https://github.com/index-tts/index-tts.git cd index-tts uv sync --all-extras # Download IndexTTS-2 weights and place them in the required directory uv run webui.py ```

Then open http://127.0.0.1:7860.

Capability Ceiling

The biggest limitation is that precise duration control is not actually enabled in this release, so exact lip-sync dubbing is not possible. Also, random sampling during inference lowers cloning similarity. For high-quality, emotionally rich, multilingual speech, it is excellent; for phoneme-level timing manipulation, you will need to wait for a future release or use another tool.

Alternatives

• **Coqui TTS**: A mature TTS toolkit with XTTS v2 zero-shot cloning in 16+ languages and a permissive license, but maintenance has stalled.
• **Tortoise TTS**: A high-fidelity English TTS with Apache 2.0 and excellent prosody, but slow inference and no emotion/duration control.
• **Confucius4-TTS**: NetEase Youdao's LLM-based TTS supporting 14 languages, cross-lingual cloning, and emotion transfer with a friendly license.
• **Step-Audio-EditX**: A 3B-parameter audio editing model with fine-grained emotion/style control and Apache 2.0, but requiring 12GB+ GPU memory.

Conclusion

IndexTTS2 is one of the strongest zero-shot TTS systems available today. Its capability is impressive, but the custom license and GPU-heavy deployment make it less universal. If you have GPU resources and can comply with the license, it is a powerful addition to your content generation stack. If you are a large enterprise or need unrestricted commercial use, consider an alternative or contact Bilibili for a separate license.