TL;DR
Voice AI hears Hindi-English switching through its speech recogniser, and errors cluster at the switch points: numbers read as 'nau aath chaar zero', English words inside Hindi sentences, names, and a one-word 'haan' or 'nahi'. Pick a recogniser built for mixed speech, make the agent read back every number and every yes or no, and speak 15 test lines to it on a real call before customers do.
Voice AI hears a Hinglish caller through its speech recogniser, and it breaks at the switch points: an English word inside a Hindi sentence, a 10-digit number read out as "nau aath chaar zero", or a one-word "haan" on a noisy line. The fix has two parts: a recogniser built for mixed speech, and an agent that reads back every detail it will act on.
On a live call there is no second chance. If the recogniser writes the wrong digit, the agent books the wrong slot, confirms the wrong order and passes the wrong number to your team, all within a few seconds.
How a voice agent hears a Hinglish sentence
A voice agent is three pieces in a row. A speech recogniser turns the caller's audio into text, a language model reads that text and decides what to say, and a voice engine speaks the reply.
The language model never hears the caller; it only reads the recogniser's text. It can usually work out a misspelt "कैंसल", but it cannot recover a digit the recogniser got wrong. That is why the recogniser matters so much on Hinglish calls.
Recognisers deal with switching in one of three ways. Vapi's provider table is a handy map:
- One language per call. You pick Hindi or English up front. Vapi lists Azure speech-to-text and OpenAI Whisper here, with no automatic detection.
- Automatic detection. The model detects the language, sometimes once per call rather than per word. Vapi lists Deepgram's Nova models with the "Multi" setting, Google and Gladia.
- Built for switching. The model expects both languages inside one sentence. Vapi names Gladia and AssemblyAI's universal-3-5-pro; Gnani.ai says its models are trained on code-switched audio, and Sarvam offers a codemix output mode.
Phone lines add a second problem: the audio is thinner than a laptop microphone. Gnani.ai and SquadStack both say they train on real 8 kHz Indian telephone audio, so always test an agent on an actual call.
What the agent reads: one sentence, five ways
Sarvam's documentation runs one spoken Hindi sentence with a phone number through each of its five output modes. It shows how different the text reaching the agent can be.
| Mode | What comes out | What it means for an agent |
|---|---|---|
| transcribe | मेरा फोन नंबर है 9840950950 | Clean digits; English words in Devanagari |
| translate | My phone number is 9840950950 | English-only prompts work, Hinglish flavour lost |
| verbatim | मेरा फोन नंबर है नौ आठ चार zero ... | Digits as spoken; agent must convert them |
| translit | mera phone number hai 9840950950 | Roman script, matches Roman-script prompts |
| codemix | मेरा phone number है 9840950950 | English terms intact, digits clean |
Look at the verbatim line. The caller said "zero" in English and every other digit in Hindi, a common way to read out a number. Turning that into 10 clean digits is exactly what test lines 1 and 2 below check.
Match the mode to how you wrote the agent's instructions. If your prompt says "cancel order" in English, a recogniser that writes English words in English, as codemix does, gives the model the easiest job.
Four switch points, and how to design around them
Numbers. Callers mix languages inside one number ("nau aath, double five"), use Hindi fractions ("saadhe teen" for 3:30, "dedh lakh" for 1.5 lakh) and read pincodes in chunks. Have the agent ask for long numbers in two parts, store them as digits, and read them back in groups of 2 or 3.
Names and places. Indian surnames and localities such as "Shrivastava" or "Koramangala" are often missing from a model's vocabulary. Sarvam's newest model accepts keyterms, a list of words to expect, but its own docs say keyterms bias recognition and "do not guarantee that a term will appear". Have the agent read names back, and ask callers to spell unusual ones.
Haan and nahi. One short word on a noisy line is the hardest thing to hear, and callers correct themselves: "haan... nahi, rukiye". Have the agent ask closed questions with a read-back ("Saturday, 11 baje. Sahi hai?"), accept a list of yes words (haan, ji, haanji, theek hai, sahi hai, ok) and no words (nahi, na, galat, mat karo), and hand over to a person if the answer is unclear twice.
English words in Hindi grammar. "Order cancel kar do" and "refund process ho gaya?" are normal speech. A single-language recogniser mangles the English words, while a code-switching model keeps them. Say this line first on any demo.
The 15-line test for a voice agent
Speak each line to the agent on a real phone call, with 2 or 3 people from different regions taking turns. For each line, note what the agent reads back or does next.
Numbers
- "Mera number nau aath saat chhe paanch, four three two one zero hai." The agent should read back 98765 43210.
- "Pincode hai ek ek zero, zero do paanch." The agent should read back 110025.
- "Total dedh lakh ka quote chahiye." The agent should repeat ₹1.5 lakh.
- "Saadhe teen baje aa jaana." The agent should confirm 3:30.
Names and places
- "Main Priyanka Shrivastava bol rahi hoon." The agent should use the full name correctly.
- "Address Sector 62, Noida, Fortis ke saamne." The agent should repeat Sector 62, Noida and the landmark.
- "Gurgaon wale office mein bhej do." The agent should name the Gurgaon (or Gurugram) office.
Yes and no
- "Haan ji, bilkul." The agent should take this as yes.
- "Nahi nahi, abhi nahi chahiye." The agent should take this as no and not push.
- "Hmm... theek hai, kar do." The agent should take this as yes.
English words in Hindi grammar
- "Mera order cancel kar do, refund kab tak aayega?" The agent should treat it as a cancellation plus a refund question.
- "EMI option available hai kya credit card pe?" The agent should answer about EMI on credit cards.
Dates, times and corrections
- "Parson subah dus baje ka slot book karo." The agent should offer the day after tomorrow at 10:00.
- "Agle Monday nahi, uske baad wala Monday." The agent should confirm the Monday after next.
- "Kal 5 baje... sorry, 6 baje." The agent should confirm tomorrow at 6:00, because the correction wins.
Scoring: lines 1 to 4 and 13 to 15 carry numbers, money, dates and times, so all seven must pass for every speaker. Across lines 5 to 12, allow one miss per speaker. An agent that fails more than that will book wrong slots and send wrong details to your team.
If you only need accurate transcripts of recorded calls rather than a live agent, our Hinglish transcription accuracy test is the better guide. For whole conversations with interruptions and handovers, use the 10-call Hinglish test for AI phone agents.
Five recognition questions for any voice AI vendor
- Which speech recogniser runs Hindi calls on the plan I am buying?
- Does it switch languages within a sentence, or only once per call?
- In which script does the agent receive the caller's words: Devanagari, Roman or mixed?
- Can I add my product names, localities and staff names as expected words?
- Can I see the recogniser's text for each test call, so I know whether a mistake came from hearing or from the agent's reasoning?
To compare vendors by the Indian languages they name, see our AI voice agents by Indian language. For which Hindi calls are worth automating at all, see our Hindi voice bot guide for customer calls.
Where TalkEasy by GrowEasy fits
TalkEasy by GrowEasy's AI receptionist answers calls, works out why the customer is calling, captures lead details and transfers the call or books a callback. After each recorded call, TalkEasy transcribes it and writes a summary of the requirement, objections, next follow-up and outcome.
Its product pages do not state which languages the receptionist understands. Speak the 15 lines to it on a test call, as you would with any vendor, before it answers your main line. For AI answering options and costs across providers, see our AI receptionist guide for India.
See how TalkEasy's AI receptionist handles calls
Last reviewed 24 September 2026.
