Zero TTSVietnamese speech that starts talking in 70 milliseconds
ZeroTTS is a fast, natural, and efficient text-to-speech model built for real-time conversations.
Four takes, no cherry-picking of the easy cases
Every clip below is generated by the released model: conversation, long-form narration, code-switched text and a news read. The transcripts are underneath each one, so you can follow along and catch anything it gets wrong.
Two speakers, one take
--:--Casual dialogue, with the timing a punchline needs
Long-form storytelling
--:--The opening of Tấm Cám, read as narration with dialogue inside it
English inside Vietnamese
--:--Names, job words and a whole English sentence, no phoneme table involved
News read, numbers & abbreviations
--:--Product names, percentages and version numbers spoken the way a presenter would
A voice it has never heard, in a language that voice never spoke
ZeroTTS compresses a reference clip into latent vectors that carry identity but not language. So a Japanese speaker can read Vietnamese, and a Vietnamese speaker can read English, from a few seconds of reference audio and no transcript at all.
Japanese reference
--:--Six seconds, no transcript given
Reads Vietnamese
--:--Same timbre, tones intact
Vietnamese reference
--:--Two seconds, shortest useful clip
Reads English
--:--The English training half paying off
Fast or natural? You can have both
ZeroTTS exists for one job: real-time voice agents. An agent is judged on the gap between deciding what to say and the listener hearing the first syllable of it, and on whether what follows sounds like a person. Those two things have always pulled in opposite directions. They do not have to.
Large TTS models, the billion-parameter kind, are expressive enough to carry emotion and accent, and they need a GPU and seconds of compute per utterance to do it. The lightweight models that fit on a CPU buy their speed somewhere specific: they shrink or remove the part of the model that decides prosody, so the output lands on flat, evenly-spaced, monotone delivery. That is fine for reading out a train platform number. It is immediately wrong in a conversation, where a listener reads intent from timing and pitch before they finish parsing the words.
ZeroTTS is built to sit where nothing was: 202M parameters, first audio in about 70 milliseconds on a CPU, and the highest naturalness score of any open Vietnamese system we measured.
How a frame of speech is generated
ZeroTTS is an autoregressive language model over audio tokens. What makes it small and fast is where it refuses to spend sequence length.
Neural audio codec
The model never sees a waveform. Audio is tokenized by MOSS-Audio-Tokenizer-Nano, a causal codec running at 12.5 frames per second with 16 residual quantizers of 1024 entries each, reconstructing 48 kHz audio. One frame is 80 ms of speech described by 16 integers.
That low frame rate is what makes a CPU-affordable autoregressive decoder possible. We use the codec frozen, and only its decoder graphs ship with the release.
Generative model
Every codec language model runs into the same problem: a frame is 16 tokens, not one. Spend a sequence position on each and a 10-second utterance costs 2000 steps. The usual answers either split the work across two models or interleave the codebooks with a delay pattern, which buys back the length at the cost of latency.
ZeroTTS puts the two axes in two different transformers. A 9-layer global transformer steps along time and emits one hidden state per 80 ms frame. A small 4-layer local transformer takes that state and walks down the codebook stack, producing all 16 indices for that frame. Only the finished frame re-enters the global sequence, so the sequence the model attends over stays at one position per frame no matter how deep the codec is. That single decision is most of why this runs on a laptop.
Voice and text conditioning
Text is read by a 9-layer bidirectional encoder whose states never enter the autoregressive stream. The decoder reaches them by cross-attention instead. Two things follow: the cache that grows during generation holds only voice and audio positions, so per-frame cost does not depend on sentence length, and the text is encoded once per utterance rather than re-read 12.5 times a second. There is no grapheme-to-phoneme step anywhere, which is why the model can read words like ZeroTTS and 31/12/2025 with ease.
Voice arrives the same way, as a prefix rather than a prompt. The reference clip is encoded to codec frames, contextualized, and squeezed through a fixed set of learned queries, whether the clip was 3 seconds or 30, and with no transcript of it required.
Streaming runtime
What ships is three ONNX graphs with no PyTorch dependency, costing two operator calls per 80 ms of audio. Decoded frames go to a KV-cached codec decoder on a doubling chunk schedule: the first chunk is a single frame, so sound reaches the speaker as soon as physically possible, then chunks grow to 16 frames to amortize the per-call overhead once someone is already listening.
Model size breakdown
202.3M for synthesis. The voice encoder adds 22.4M and is only needed to create a voice, not to speak with one.
Measured on a benchmark we had to build first
The Vietnamese benchmarks in circulation are not hard enough to separate modern zero-shot systems. Most are audio paired with transcripts, which makes the obvious evaluation a copy-eval: the same clip is the reference voice and the ground truth. The model re-reads a sentence it was just handed, in a voice it is currently listening to. Everything scores well.
The hard cases are elsewhere: a voice carried into a language it never spoke, English embedded mid-sentence, text nobody normalized first, rare words and unfamiliar names. And the two things a listener judges first, whether it still sounds like the reference speaker and whether it sounds like a person, never appear in a word error rate.
So we released ZeroBench-TTS: 137 items across 59 held-out voices, in monolingual, code-switching, cross-lingual and adversarial-orthography subsets, with the reference clip never drawn from the sentence being scored. It reports speaker similarity and naturalness next to word error rate, and its scorer accepts every legitimate spoken reading of a written form while still counting a wrong tone as an error, because in Vietnamese the tone is the word. The scorer lives in the dataset repository, not ours, and the same code scored every system below.
| Metric | ZeroTTS | OmniVoice | XTTS-v2-vi | viXTTS |
|---|---|---|---|---|
Word error rate ↓ | 1.03% | 4.13% | 16.42% | 18.40% |
Naturalness (UTMOSv2) ↑ | 2.91 | 2.76 | 2.43 | 2.35 |
Speaker similarity ↑ | 0.936 | 0.950 | 0.940 | 0.935 |
Dead air per clip ↓ | 0.029s | 0.340s | 0.532s | 0.233s |
Real-time factor, CPU ↓ | 0.50× | 6.12× | 0.71× | 0.73× |
Parameters ↓ | 202M | 775M | 467M | 467M |
ZeroBench-TTS, raw-text condition. ↓ / ↑ marks whether lower or higher is better; bold is the best value in the row.
Naturalness is the row to read alongside the speed. Being the fastest system here would not be worth much if it were also the flattest, and it is the opposite: 2.91 UTMOSv2 is the highest of the four, and the dead-air figure is an order of magnitude better than anything else, which is the defect a listener notices first in a live agent.
The gap that decides whether a voice agent feels alive, though, is none of those. It is the wait before anything comes out of the speaker at all:
Time to the first audible sample
log scale · CPU · medium inputTwo honest caveats. ZeroTTS does not lead on speaker similarity, since the latent bottleneck trades some timbre detail for conditioning that costs no context, and carrying a foreign speaker's voice into Vietnamese is where that shows.
Everything is public
pip install zerotts
from zerotts import ZeroTTS
tts = ZeroTTS.from_pretrained("zeroweight-ai/ZeroTTS")
tts.synthesize("Xin chào, đây là ZeroTTS.", voice="maichi", out="hello.wav")One thing we did not release: the voice encoder that creates a voice from a reference clip. Shipped voices drop in as a small file with no code change, and withholding the encoder raises the cost of the obvious abuse, cloning a real person from a few seconds of their audio, without withholding the system's usefulness. If you build with ZeroTTS, tell your listeners the speech is synthetic wherever they might reasonably assume otherwise.
Frequently asked
Is ZeroTTS free to use commercially?+
The code and the released weights are open under the terms in the repository, including commercial use. The one thing the model card prohibits is impersonation: using ZeroTTS to attribute speech to a real person without their consent, or to produce audio meant to deceive a listener.
What hardware does it need?+
A CPU. ZeroTTS synthesizes at a 0.50× real-time factor on a laptop processor, meaning two seconds of speech per second of compute, and starts producing audio in about 70 milliseconds. No GPU is required at any point, and there is a browser build that runs entirely on the visitor’s machine.
Is it fast enough for a real-time voice agent?+
That is what it was built for. The number that matters for an agent is not throughput but the delay before the first sound, because everything after that streams while the listener is already listening. ZeroTTS starts speaking in 53 to 89 milliseconds depending on input length, against 2.5 to 52 seconds for the open Vietnamese alternatives, and it keeps ahead of real time on CPU from there.
Can it clone a voice from my own recording?+
Not from the public release. ZeroTTS ships ready-made voices and the runtime that plays them; the voice encoder that turns a reference clip into a voice is deliberately withheld, because one-shot cloning from a few seconds of audio is the part of this technology with a real misuse problem.
Does it handle English words mixed into Vietnamese?+
Yes, and that was a design goal rather than a side effect. ZeroTTS reads raw orthography with no grapheme-to-phoneme frontend, and 12,000 hours of English audio sit in the training corpus specifically so that embedded brand names, loanwords and whole English sentences come out pronounced rather than spelled out.
How was it evaluated?+
On ZeroBench-TTS, a public 137-item Vietnamese benchmark released alongside the model, with monolingual, code-switching, cross-lingual and adversarial-orthography subsets. Its scorer transcribes with two ASR systems and takes the lower error, and accepts every legitimate spoken reading of a written form, while still counting a wrong tone as an error.
Vietnamese voice, without the GPU bill
Take the open model and run it yourself, or build on the hosted platform.