Replacing IVR With a Voice AI Agent: Costs, Limits, Payoff

If your contact center still greets callers with "press 1 for sales, press 2 for support", you are paying for a menu that most callers try to escape. Replacing that menu with a voice AI agent is now a practical engineering decision, not a science project: the speech models are open source, telephony costs fractions of a cent per minute, and the orchestration frameworks are free. The honest answer to whether you should do it is yes for structured, high-volume call types, no for everything else, and the economics only hold if you measure containment on your own call recordings before you spend anything.

This guide walks through what a voice AI agent actually is, what it costs with primary-source numbers, which calls to automate first, where the limits are, and a migration path that does not bet your contact center on a demo.

What a voice AI agent actually is

A traditional IVR is a decision tree. The caller presses keys or speaks single words, the system walks down branches, and anything the tree did not anticipate ends in "please hold". The IVR cannot answer a question; it can only route.

A voice AI agent is a streaming pipeline with four stages:

• Telephony carries the call in over SIP or a carrier API and streams audio both ways.
• Speech-to-text transcribes the caller's words as they speak, not after they finish.
• A language model decides what the caller wants, looks up answers in your systems through API calls, and writes a response.
• Text-to-speech speaks the response back, streaming, so the caller hears the start of the answer in under a second.

Every component of this pipeline is available as open source you can run yourself. OpenAI's Whisper is an MIT-licensed speech recognition model trained across 99 languages, and faster-whisper reimplements it on the CTranslate2 engine for up to four times the speed at the same accuracy. Kokoro is an Apache-licensed text-to-speech model with only 82 million parameters that produces natural voices at a fraction of the compute of cloud TTS. Orchestration frameworks like Pipecat and LiveKit Agents wire the stages together, handle interruptions, and connect to phone networks over SIP. For the reasoning step you can call a hosted model such as Claude, or run an open-weight model on your own GPU if calls must stay on your infrastructure.

For a broader tour of what these agents can do across a business, our complete guide to AI voice agents for business covers the use cases. This post stays on one decision: replacing the IVR.

Why the phone menu leaks value

The case against the menu tree is mechanical, not mystical.

Every caller with a request the tree did not anticipate either mashes zero, picks a wrong branch and gets transferred, or hangs up. Misrouted calls cost a second conversation. Abandoned calls cost the sale, the renewal, or the patient who does not reschedule. After hours, the tree does nothing at all and the call goes to voicemail that nobody acts on until morning.

Menu trees also cannot do work. The best they achieve is getting the caller to the right human. A voice agent can check a balance, confirm an appointment, or reset a password inside the call, which is where the real cost difference comes from: containment, not routing.

The cost math, with real numbers

Here is the part vendor blog posts obscure. All prices below are from the vendors' own pricing pages.

What the human side costs in platform fees alone. Genesys Cloud CX runs $75 per user per month for the voice tier, rising to $155 and $240 as you add workforce management and AI features, billed annually. Five9 lists its Digital bundle at $119 per seat per month and Core at $159, priced per concurrent user with a 50-seat minimum. So a 50-seat operation pays $3,750 to $7,950 a month in platform fees before a single salary, and Five9's bundled AI covers 3,000 minutes per seat with usage charges after that.

What the AI side costs. A worked example with round numbers: 30,000 calls a month averaging 4 minutes is 120,000 minutes of talk time.

• Telephony: Twilio charges $0.0085 per minute to receive calls on a local US number, plus $1.15 a month for the number, or $0.004 per minute if you bring your own carrier over SIP trunking. That is $480 to $1,020 a month at this volume.
• Compute: one rented GPU runs the whole open-source pipeline. On RunPod's community cloud, an RTX 4090 costs $0.74 an hour and an A100 costs $1.39 an hour. Run 24/7, that is roughly $540 to $1,015 a month. Average concurrency at this volume is under three simultaneous calls, so even the 4090 has headroom for peak hours; a smaller L4 at $0.49 an hour works if you quantize the models.
• Total: about $1,000 to $1,600 a month, or $0.009 to $0.017 per minute, excluding the one-time build. If you use a hosted language model API instead of a self-hosted one, add its per-token cost, which for short structured calls typically adds a cent or two per minute.

Put the two side by side. The platform fees for 50 human seats alone run two to five times the entire self-hosted AI stack, before any salary. That does not mean you fire 50 people; it means every structured call the AI contains is a call that never touches a seat you are paying $75 to $159 a month to license, plus the human sitting in it.

If you do not want to run GPUs, managed voice AI platforms charge per minute instead, and the break-even between the two models depends on your volume and your country. We break down the per-minute economics and the Kenya-specific numbers in what a voice AI agent costs to run in Kenya.

Which calls to automate first

Containment is where the money is, and containment is a property of the call type, not the model. Good first candidates share five traits: the call is short, the request is structured, the answer lives in a system the agent can query through an API, the emotional stakes are low, and the volume is high.

Calls that pass all five:

• Balance and status lookups: account balance, order status, ticket status, claim status.
• Appointment work: confirm, reschedule, cancel, reminders.
• Credential and access issues: password reset, PIN reset, unlock requests.
• Static information: branch hours, locations, required documents, payment due dates.
• Simple intake: opening a ticket, registering a complaint reference, capturing a callback request with details.

Calls that should stay human for now: complaints with any heat in them, disputes, cancellations and retention saves, bereavement and hardship cases, fraud reports, and anything where the caller needs judgment or empathy rather than a lookup. Automating those first is how voice AI projects get killed by their own pilot.

The limits, stated plainly

Accents and code-switching are real. Whisper covers 99 languages, but error rates climb with accents the training data underrepresents, and the hardest case in East Africa is the caller who switches between English and Swahili mid-sentence. This is testable: take fifty real recordings and transcribe them before you commit. Do not discover your word error rate in production.

Latency has a hard ceiling of caller patience. Past about a second of dead air after the caller stops speaking, the conversation feels broken. Every stage has to stream, and the agent has to handle barge-in, where the caller talks over it. This is an engineering discipline, not a default setting.

The model can say wrong things confidently. The mitigation is architectural: the agent answers from tool calls into your systems and a constrained intent set, not from open-ended generation. A balance is read from your core banking or CRM API, never invented.

No emotional intelligence. The agent will not notice the caller whose mother just died. Escalation triggers and a fast path to a human are not optional features.

A migration path in five steps

1. Listen before you build. Record four weeks of calls (Twilio recording costs $0.0025 per minute; most PBX systems record free), transcribe them, and cluster what callers actually ask for. That analysis gives you two things money cannot buy later: your real intent distribution and your baseline containment of zero. Our voice AI customer service implementation guide covers this discovery phase in more depth.
2. Pick one intent from the checklist above, ideally the highest-volume one with a clean API behind it.
3. Run in shadow mode. The agent listens to live calls and produces responses nobody hears. Compare its answers to what the human agents did. Two to three weeks of this tells you your true containment rate on your own audio before a single caller is affected.
4. Soft launch with an escape hatch. Route only the chosen intent to the agent, start with after-hours calls if you are cautious, keep "press 0" and the word "agent" as instant transfers, and review every failed conversation weekly.
5. Expand intent by intent. Retire a menu branch only after the agent covering it has held its containment rate for eight or more weeks.

If you want this built rather than described, a scoped custom build covering steps one through three is the honest starting scope: recordings analyzed, one intent live in shadow mode, and a containment report with your real numbers.

When replacing your IVR is the wrong choice

• Volume is low. Under roughly 5,000 calls a month, the fixed cost of building and maintaining the pipeline dominates. Use a managed per-minute platform, or keep the menu and spend the money on one more well-trained human.
• Your backend has no APIs. If checking an order means a human opening a green screen, the agent has nothing to call. Fix the systems first; the voice layer is the last 20 percent of the work.
• Your calls are mostly emotional or high-stakes. A complaints line, a fraud line, a hospital oncology desk: containment is not the goal there.
• You cannot record or process calls under your regulatory regime or consent framework. Sort that out with counsel before any pilot.

Common mistakes

No escape hatch. Every deployment needs "press 0" and a spoken "agent" keyword that transfer instantly, no questions asked. Trapping callers in an AI they cannot leave is the fastest way to generate complaints and regulator attention.

Automating complaints first. Teams reach for the highest-cost call type. High cost usually means high complexity. Start with the boring calls.

Buying AI as a seat add-on without doing the usage math. Bundled AI minutes run out; Five9's own pricing notes 3,000 AI minutes per seat then usage charges. Compare the add-on's marginal per-minute cost against the per-minute numbers above before signing.

Testing only in studio English. A demo recorded in a quiet room with an American accent tells you nothing about Nairobi matatu background noise or a caller switching to Swahili when frustrated. Test on your recordings.

No weekly transcript review. Containment decays as products, prices, and policies change. Someone has to read the failures every week, forever.

Silent deployment. Tell callers they are speaking with an automated agent, and check your recording-consent obligations in every jurisdiction you take calls from.

Frequently asked questions

How much does a voice AI agent cost per call? Self-hosted at meaningful volume, the marginal cost lands around $0.01 to $0.02 per minute including telephony and GPU, based on the Twilio and RunPod prices cited above. A 4-minute call costs five to eight cents. Managed platforms charge several times that per minute in exchange for not running infrastructure.

Will it understand Kenyan callers who mix English and Swahili? Sometimes, and you must verify rather than hope. Whisper was trained across 99 languages including Swahili, but mid-sentence language switching raises error rates. Test on fifty of your own recordings first, and design the fallback so a misunderstood caller reaches a human fast.

Can we keep our existing phone numbers? Yes. The agent sits behind your numbers via SIP trunking or call forwarding; Twilio's BYOC rate is $0.004 per minute in both directions. Callers notice nothing except a shorter path to an answer.

What containment rate should we expect? There is no honest universal number, and anyone who quotes one without listening to your calls is guessing. Containment depends on your intent mix and your backend APIs. Measure it in shadow mode on your own audio; that number is the only one that matters.

How long does a proper deployment take? Plan on eight to twelve weeks for the first intent done right: four weeks of recording and analysis, two to three weeks of shadow mode, then a soft launch with weekly review. Faster timelines skip the measurement that makes the project defensible.

Does this replace our agents? It absorbs structured volume, which is the part of the queue with the highest turnover anyway. Whether headcount shrinks depends on whether your call volume is growing; most teams redeploy people to the complex calls the AI escalates.

Where to go from here

The decision is not "AI or IVR". It is which of your call types are structured lookups wearing a phone call as a costume, and whether your systems can answer them through an API. Start with your own recordings, measure in shadow mode, and let your containment data make the case. If you want help with the plumbing, our business automation service wires voice agents into CRMs, scheduling, and back-office systems, and we build and run the self-hosted stack when data residency or per-minute economics demand it.

Related reading

• Voice Biometrics for Call Center Authentication: Costs and Limits
• AI Reconciliation for Fund Administration: Costs and Limits