How Does Text to Speech Work? From Spelling to Sound

How does text to speech work? In three steps. First, software cleans your text and decides how every word should be pronounced. Second, a neural model predicts the rhythm, pitch and timing of the sentence as a spectrogram. Third, a vocoder turns that spectrogram into an actual audio waveform you can hear. The whole chain runs in a few seconds for a 1,000-word article, which is why you can paste a link and press play.

What actually happens between the page and your ears?

Think of it as a relay with four runners. Text goes in. Normalized text passes to a pronunciation stage. Pronunciation passes to a prosody stage. Prosody passes to a waveform generator. Audio comes out.

Every modern engine, from the voice in your phone's accessibility menu to the narrators inside reading apps, follows some version of that relay. Some newer systems collapse two or three runners into one network, but the jobs still exist. They are just done inside a single model.

INPUT   → "Dr. Voss paid $4.50 on 5/21."
STAGE 1 → "Doctor Voss paid four dollars fifty cents on May twenty-first."
STAGE 2 → /ˈdɑːktɚ vɔs .../
STAGE 3 → pitch, duration, energy per phoneme
STAGE 4 → waveform, 22,050 to 48,000 samples per second

That is the entire trick. The rest of this piece is what each runner does, where each one fails, and what you can hear when it does.

Why does the first step, text normalization, matter so much?

Because written text is not speakable text. A page is full of numbers, dates, abbreviations, currency symbols, URLs, footnote markers and emoji. None of those have an obvious pronunciation.

Text normalization expands them. "5/21/24" becomes "May twenty-first, twenty twenty-four". "$4.50" becomes "four dollars fifty cents". "St." becomes "Street" or "Saint" depending on what sits around it. Older engines did this with hand-written rules. Newer ones mix rules with a small learned model for the ambiguous cases.

This is also where articles go wrong in ways you notice immediately. A product name like "iPhone 15 Pro Max" read as "fifteen" versus "one five" changes the sentence. A citation like "[12]" read aloud mid-paragraph breaks the thought. Good article-to-audio tools strip footnote markers, skip navigation text, and decide what to do with tables before the voice ever opens its mouth.

On a long read you will hear the difference in the first minute. If the narrator stumbles on a number, the front end failed, not the voice.

How does the system know how to pronounce each word?

English spelling is a poor guide to sound. "Though", "through", "tough" and "thought" share most of their letters and almost none of their sounds. So the engine needs a pronunciation stage, usually called grapheme-to-phoneme conversion, or G2P.

A phoneme is the smallest unit of sound that distinguishes words. English uses roughly 40 of them. The G2P step maps letters to phonemes in two ways. A pronunciation dictionary handles common words and known exceptions. A learned model guesses the rest, such as a surname it has never seen.

Then comes the hard part, homographs. "Read" in "I read it yesterday" and "I read every morning" is the same spelling with different sounds. "Bass" the fish and "bass" the instrument too. The model has to look at the sentence around the word, which is why modern engines use context-aware networks instead of simple lookups.

Where it still fails:

• Proper names, especially non-English ones, in a mostly English article.

• Technical jargon and acronyms, where "SQL" might be "sequel" or "S-Q-L".

• Brand names that deliberately break spelling rules.

Most engines let you add a custom pronunciation. Few readers ever do. If one word keeps tripping the narrator, that setting is worth finding.

Where do rhythm and intonation come from?

This is the stage that separates a flat voice from one you can listen to for forty minutes. It is called prosody: stress, pacing, pauses and pitch movement across a sentence.

Humans do it without thinking. We raise pitch at the end of a question, slow down before a comma, and lean on the word that carries the point. A TTS model has to predict all of that from text alone. For each phoneme it estimates three numbers: how long it lasts, how high the pitch sits, and how much energy it carries. It does this using the surrounding words, the punctuation, and the position of each sound within its syllable, word and phrase.

Punctuation is the strongest signal the model gets. A comma earns a short pause, a period a longer one, a paragraph break a longer one still. That is why a badly formatted article, with no paragraph breaks or with a wall of nested quotes, sounds rushed even through an excellent voice.

Here is a number worth keeping. A typical narrator speaks at about 150 words per minute. A 3,000-word essay is therefore around 20 minutes of audio. Most apps let you push playback to 1.5x or 2x, which turns that same essay into 13 or 10 minutes. Prosody quality is what decides whether 1.5x stays comfortable or collapses into mush.

What is a vocoder, and why do you need one?

After prosody planning, the model has a plan but no sound. In most neural systems the plan is a mel-spectrogram: a picture of how energy is spread across frequencies over time, usually at around 80 frames per second.

A spectrogram is not audio. It throws away the fine phase detail that makes a waveform a waveform. So a second model, the vocoder, has to reconstruct a waveform that matches it. That means generating tens of thousands of audio samples every second, each one a number describing air pressure at that instant.

WaveNet, published by DeepMind in 2016, was the model that made neural vocoding famous. It generated raw audio one sample at a time and sounded far more natural than anything before it. It was also slow. Later vocoders such as HiFi-GAN generate audio in parallel and run many times faster than real time, which is what makes instant playback possible on a phone.

Some systems skip the middle step. VITS, described in this 2021 paper, trains one network to go from phonemes straight to waveform. Fewer handoffs means fewer places for errors to pile up.

How did we get here from the talking toys of the 1980s?

Speech synthesis did not start with neural networks. It went through three eras, and you can still hear remnants of the first two in older screen readers.

Formant and rule-based synthesis. Engineers modeled the vocal tract with filters and tuned the parameters by hand. The sound was intelligible and unmistakably mechanical. This is the voice of early talking toys and the first generation of screen readers. Many blind users still keep one on a phone at high speed, because it is fast, predictable and never changes between releases.

Concatenative or unit-selection synthesis. Instead of modeling the voice, record a human speaker for dozens of hours, slice the audio into small units, and stitch the right ones together at runtime. It sounded far more human, but with audible seams. You could hear the joins on unusual words, and a voice could not change its style without a new recording session.

Neural synthesis. Today's approach. The model learns the whole mapping from a large set of recordings and generates audio rather than splicing it. Style, speed and emotion become controllable. Cloning a voice from a short sample became possible. And the seams mostly vanished.

One honest caveat. Neural voices are not perfect. They can hallucinate a word, drop a syllable, or flatten a sentence that needed emphasis. The failures are rarer and stranger than the old ones, but they exist. On long documents, I would rather have a slightly plainer voice that never glitches than a lovely one that fumbles every fiftieth sentence.

Which TTS tools show these stages in practice?

Different products make different bets on the relay. A few worth knowing, with the honest take on each.

Speechify leans on app polish and a large voice library. Strong on mobile, with imports from many sources. If you want the widest set of integrations, start here.

NaturalReader is the plainer, older name in the field. It is steady on documents and PDFs, and a sensible choice when your input is files rather than web articles.

ElevenLabs sits at the other end. It is a voice-generation platform first, with strong prosody and cloning, and it is built for producing audio you publish, not for working through a reading queue.

Voice Dream Reader is the one many low-vision readers already know. It lets you swap the underlying voice engine, tune pronunciations, and set speed in fine steps, which makes the pipeline above something you can actually adjust.

Skip the idea that one of them has "the best voice". What differs is the front end: how they clean an article, what they do with a table, how they handle your 5,000-word piece with nested quotes. That is where a reading workflow succeeds or quietly fails.

What does this change about how you listen?

Knowing the relay gives you three practical moves.

1. Clean the input. Strip ads, cookie banners and footnote clutter before you send an article to audio. Reader mode helps. The front end can only fix so much.

2. Match speed to prosody. If a voice sounds good at 1.0x but breaks at 1.5x, its duration model is the weak point. Try another voice before you give up on speed.

3. Fix the repeat offenders. If a name or acronym keeps going wrong, add a custom pronunciation once instead of enduring it for months.

Also, test with your worst article, not your best. A clean 800-word blog post proves nothing. A dense 4,000-word piece with numbers, quotes and a table is the real exam.

Before your next commute, queue one long read and one short one. Listen to the first two minutes of each at normal speed, then at 1.5x. You will hear the stages we just covered, and you will know which runner dropped the baton.