Zero-shot text-to-speech for English and six Indian languages: Hindi, Marathi, Bengali, Punjabi, Gujarati and Odia.
pip install, a command-line tool, and sentence-by-sentence streaming.Unseen speakers only. Word error (English, Whisper large-v3) and character error (Indian languages, IndicConformer), in %; lower is better.
| Test set | Metric | DviTTS | DviTTS-Nano |
|---|---|---|---|
| Seed-TTS test-en | WER | 2.04 | 2.65 |
| CMU Arctic | WER | 1.45 | 2.74 |
| LibriTTS-R test | WER | 3.07 | 3.63 |
| LibriSpeech-PC test-clean | WER | 2.05 | — |
| IndicVoices Hindi | CER | 3.53 | 3.52 |
| FLEURS Hindi | CER | 4.52 | 5.16 |
| FLEURS Marathi | CER | 2.57 | 3.70 |
| FLEURS Bengali | CER | 2.69 | 3.54 |
| FLEURS Punjabi | CER | 5.23 | 5.54 |
| FLEURS Gujarati | CER | 3.76 | 4.54 |
| FLEURS Odia | CER | 3.11 | 3.75 |
DviTTS reads new text in the voice of a short prompt. The prompts are speakers the model never saw in training; each output is one take, not picked from several.
Text “It might rain this evening, so please take an umbrella when you leave the house.”
Prompt (LibriTTS-R test-clean, unseen speaker)
DviTTS output
Text “आज शाम बारिश हो सकती है, इसलिए घर से निकलते समय छाता साथ ले जाइए।”
Prompt (FLEURS, unseen speaker)
DviTTS output
Text “आज संध्याकाळी पाऊस पडण्याची शक्यता आहे, त्यामुळे घरातून निघताना छत्री सोबत घ्या.”
Prompt (FLEURS, unseen speaker)
DviTTS output
Text “আজ সন্ধ্যায় বৃষ্টি হতে পারে, তাই বাড়ি থেকে বেরোনোর সময় ছাতা সঙ্গে নিন।”
Prompt (FLEURS, unseen speaker)
DviTTS output
Text “ਅੱਜ ਸ਼ਾਮ ਨੂੰ ਮੀਂਹ ਪੈ ਸਕਦਾ ਹੈ, ਇਸ ਲਈ ਘਰੋਂ ਨਿਕਲਦੇ ਸਮੇਂ ਛਤਰੀ ਨਾਲ ਲੈ ਜਾਓ।”
Prompt (FLEURS, unseen speaker)
DviTTS output
Text “આજે સાંજે વરસાદ પડી શકે છે, તેથી ઘરેથી નીકળતી વખતે છત્રી સાથે લઈ જજો.”
Prompt (FLEURS, unseen speaker)
DviTTS output
Text “ଆଜି ସନ୍ଧ୍ୟାରେ ବର୍ଷା ହୋଇପାରେ, ତେଣୁ ଘରୁ ବାହାରିବା ବେଳେ ଛତା ସାଙ୍ଗରେ ନିଅନ୍ତୁ।”
Prompt (FLEURS, unseen speaker)
DviTTS output
Text “The meeting starts at 10:30 on Monday, so please be on time.”
Prompt (FLEURS, unseen speaker)
DviTTS output
The same sentence in each language, from both models. Voice names are labels, not the speakers’ names.
Text “Please call me back after the meeting ends.”
Emma female
DviTTS
DviTTS-Nano
Grace female
DviTTS
DviTTS-Nano
James male
DviTTS
DviTTS-Nano
Oliver male
DviTTS
DviTTS-Nano
Text “कल सुबह हम स्टेशन पर मिलेंगे।”
Ananya female
DviTTS
DviTTS-Nano
Kavya female
DviTTS
DviTTS-Nano
Arjun male
DviTTS
DviTTS-Nano
Rohan male
DviTTS
DviTTS-Nano
Text “आज संध्याकाळी पाऊस पडण्याची शक्यता आहे.”
Sneha female
DviTTS
DviTTS-Nano
Aditya male
DviTTS
DviTTS-Nano
Text “আজ আকাশ খুব পরিষ্কার।”
Ishita female
DviTTS
DviTTS-Nano
Arnab male
DviTTS
DviTTS-Nano
Text “ਅਸੀਂ ਕੱਲ੍ਹ ਸਕੂਲ ਜਾਵਾਂਗੇ।”
Jasleen female
DviTTS
DviTTS-Nano
Gurpreet male
DviTTS
DviTTS-Nano
Text “અમે આવતીકાલે બજારમાં જઈશું.”
Hetal female
DviTTS
DviTTS-Nano
Kunal male
DviTTS
DviTTS-Nano
Text “ଆଜି ପାଗ ବହୁତ ଭଲ ଅଛି।”
Subhashree female
DviTTS
DviTTS-Nano
Debasis male
DviTTS
DviTTS-Nano