The half-second problem: why turn-taking will define the next generation of voice AI

Play a recording of a modern AI voice to someone and they'll often struggle to tell it apart from a person. Put that same person on a live call with a poorly tuned agent and they'll know within a few exchanges. The giveaway isn't the voice. It's the timing.

The agent answers a beat too late, or cuts in while you're still reading out your address. You say "sorry, no, the fourteenth" and it keeps talking. You say "mm-hm" to show you're listening and it stops dead, as if you'd asked a question. None of these are about what the agent says. They're about when.

People take turns astonishingly fast

Conversation researchers who have compared turn-taking across many languages keep finding the same thing: the typical gap between one person finishing and the next starting is around a fifth of a second. That's faster than anyone could hear a sentence end and then begin planning a reply. People are predicting where the other person's turn will end and preparing their answer while still listening.

That's the bar a voice agent is held to, whether anyone says so or not. Callers don't consciously time the gaps, but they feel them. A pause that's too long reads as confusion or a dropped line. A reply that starts too early reads as rudeness.

Silence doesn't mean someone has finished

The simplest way to decide when a caller is done is to wait for silence. That's roughly what basic voice activity detection does: it listens for speech, and when speech stops for long enough, it assumes the turn is over.

The trouble is that people pause all the time without finishing. They pause to find a card in their wallet, to remember a date, to read a long number in chunks. Set the silence threshold short and the agent interrupts them. Set it long and every exchange drags. There isn't a single number that works for both "what's your name?" and "what's your account number?"

Better systems use context. If the agent just asked for a ten-digit phone number and has heard seven, a pause almost certainly isn't the end. If it asked a yes-or-no question and heard "yes," it can reply immediately. Deciding the end of a turn from meaning, not just from silence, is one of the biggest quality jumps available in voice AI right now.

Interruptions are a feature, not an error

In a natural call, people interrupt to correct, to hurry things along, or because they've already heard enough. An agent needs to handle that gracefully, which the industry calls barge-in. When the caller starts speaking, the agent should stop, listen, and respond to what was actually said, not finish its sentence and then carry on as if nothing happened.

But not every sound is an interruption. Short acknowledgements like "okay," "right," or "mm-hm" are signals that the caller is following along. So is background noise from a kitchen, a truck cab, or a crying child. An agent that stops talking for every one of those feels skittish and slow. Telling a real interruption from a backchannel is subtle, and it's where a lot of tuning effort goes.

Every step of a reply costs time

A spoken reply involves several steps: turning the caller's audio into something the system understands, deciding what to say or do, and turning that decision back into speech. If each step waits for the previous one to finish completely, the delays stack up into an awkward silence.

The fix is to overlap the steps. Start understanding speech while it's still arriving. Start speaking the first words of a reply while the rest is still being produced. Keep slow work, like looking up a record, off the critical path, or cover it the way a person would, with a natural "let me check that for you." We go deeper on that in no dead air.

Speed isn't the whole story, though. An agent that answers instantly but gets the answer wrong is worse than one that takes a breath and gets it right. The goal is timing that feels natural, not timing that's as short as possible.

Where real-time voice is heading

• Models that listen and speak at once. Newer systems process audio directly rather than converting everything to text and back, which cuts delay and lets them pick up on tone, hesitation, and emphasis.
• Turn-taking from meaning. End-of-turn decisions will increasingly weigh what was asked and what's been said so far, not just how long the line has been quiet.
• Prosody that carries information. A rising tone for a question, a slower pace when reading back a number, a softer delivery when the news isn't good. These cues make calls easier to follow, not just more pleasant.
• Graceful handoffs to text. When a detail is hard to say aloud, the best move is often to stop talking about it and send a text. Mixing channels mid-call sidesteps the hardest timing problems entirely.

What this means if you're choosing a voice agent

Don't judge an agent by a scripted demo where the caller speaks in perfect, complete sentences. Test it the way your customers actually call:

• Read out a long number slowly, in chunks, with pauses.
• Correct yourself halfway through an answer.
• Spell a name the agent is likely to get wrong, and check it reads it back.
• Call from somewhere noisy.
• Ask for a person partway through.

Testing like this is built into how HiloVoice agents are created: you call your agent, try to trip it up, and adjust before a customer ever hears it. It's also why agents confirm details that matter, like spellings and phone numbers, as part of verification rather than hoping they heard right.