AI Agent Error Recovery Strategies for Production Systems
Build self-healing AI agents that recover gracefully from errors. Learn fallback patterns, state management, circuit breakers, and rollback strategies that maintain system reliability.

An AI agent in production will fail: APIs rate-limit you, tools time out, models return malformed output, and users send inputs nobody planned for. What separates a production system from a demo is not avoiding these failures but recovering from them without losing the user's work or trust. This guide covers the recovery patterns that matter — retries with backoff, circuit breakers, fallback chains, state checkpoints, and graceful degradation — with working code and the failure modes each one actually addresses.
Why error recovery matters for AI agents
A traditional web request fails in familiar ways. An agent adds new failure modes because it chains model calls, tool calls, and external APIs into multi-step workflows where every step can fail independently.
The failures you will actually see fall into three groups:
External service failures
- Rate limits. The Claude API, like most model APIs, enforces organization-level limits with a token bucket algorithm and returns a 429 error with a
retry-afterheader when you exceed them (Claude rate limits documentation). Per-minute limits can be enforced over shorter windows, so short bursts trip them even when your average rate looks fine. - Timeouts and connection resets from tools, databases, and third-party APIs.
- Authentication failures from expired tokens or rotated credentials.
Model failures
- Context window overflow when a long conversation or large retrieved documents exceed the model's input limit.
- Malformed output: invalid JSON, truncated tool calls, or answers that ignore the requested format.
- Hallucinated tool arguments, or confident answers built on nothing.
Workflow failures
- A step midway through a ten-step workflow fails after earlier steps already changed state — sent the email, wrote the record, charged the card.
- Adversarial or malformed inputs that push the agent outside what it can handle.
Each group needs a different response. Retrying a hallucination does nothing. Restarting a half-finished workflow from scratch can duplicate side effects. The patterns below map to these groups, and the "when recovery is the wrong choice" section later covers the cases where the right move is to stop, not retry.
Core error recovery patterns
1. Retries with exponential backoff and jitter
Most transient failures — timeouts, connection resets, 429s, 5xx errors — clear on their own. Retrying immediately and aggressively makes things worse: Amazon's engineering guidance describes retries as "selfish," because a client that retries spends more of a struggling server's time and can turn a partial failure into a full outage (AWS Builders' Library: Timeouts, retries, and backoff with jitter). The fix is capped exponential backoff with jitter: wait longer between attempts, and randomize the wait so many clients don't retry in lockstep.
const retryWithBackoff = async (operation, options = {}) => {
const maxRetries = options.maxRetries || 3;
const baseDelay = options.baseDelay || 1000;
const maxDelay = options.maxDelay || 30000;
for (let attempt = 1; attempt <= maxRetries; attempt++) {
try {
return await operation();
} catch (error) {
if (!isRetryable(error)) throw error;
// Honor retry-after headers from the API when present
const retryAfter = error.headers?.['retry-after'];
if (retryAfter) {
await sleep(Number(retryAfter) * 1000);
continue;
}
if (attempt === maxRetries) {
throw new Error(`Failed after ${maxRetries} attempts: ${error.message}`);
}
const delay = Math.min(baseDelay * Math.pow(2, attempt - 1), maxDelay);
const jitter = Math.random() * delay; // "full jitter"
await sleep(jitter);
}
}
};
const isRetryable = (error) => {
if (error.status === 429 || error.status >= 500) return true;
return ['ETIMEDOUT', 'ECONNRESET', 'SERVICE_UNAVAILABLE'].includes(error.code);
};
Three details matter here. First, only retry retryable errors — a 401 or malformed request will fail forever. Second, when the API tells you how long to wait (as Claude's 429 responses do with retry-after), believe it. Third, cap both the delay and the total attempts, and treat hitting the cap as a real failure you handle upstream, not as something to paper over. The AWS Architecture Blog's exponential backoff and jitter analysis shows why jitter specifically prevents the correlated retry bursts that stall recovery. For a deeper treatment of retry policy design, see our AI agent error handling and retry strategies guide.
2. Circuit breakers
Retries handle transient failure. A circuit breaker handles a dependency that is genuinely down. Michael Nygard popularized the pattern in Release It! and Martin Fowler's write-up remains the reference: wrap the protected call, count failures, and once failures cross a threshold the breaker "trips" and subsequent calls fail fast without touching the sick dependency at all.
class CircuitBreaker {
constructor(service, options = {}) {
this.service = service;
this.failureThreshold = options.failureThreshold || 5;
this.resetTimeout = options.resetTimeout || 60000;
this.state = 'CLOSED'; // CLOSED, OPEN, HALF_OPEN
this.failures = 0;
this.lastFailureTime = null;
}
async execute(operation, ...args) {
if (this.state === 'OPEN') {
if (Date.now() - this.lastFailureTime > this.resetTimeout) {
this.state = 'HALF_OPEN'; // allow one trial call
} else {
throw new Error(`Circuit breaker OPEN for ${this.service}`);
}
}
try {
const result = await operation(...args);
this.failures = 0;
this.state = 'CLOSED';
return result;
} catch (error) {
this.failures++;
this.lastFailureTime = Date.now();
if (this.failures >= this.failureThreshold) {
this.state = 'OPEN';
logger.error(`Circuit breaker opened for ${this.service}`);
}
throw error;
}
}
}
Without a breaker, every call to a hung service ties up a worker while it waits for a timeout. Under load those waiting calls pile up until your agent runs out of resources too — one sick component topples the whole system. The breaker converts minutes of cascading failure into an instant, loggable error that your fallback logic can act on. Pair it with an alert: an open breaker is an operations event, not just a code path.
3. Hierarchical fallbacks
When the primary approach fails, have a cheaper or simpler one ready:
const executeWithFallback = async (query, context) => {
// Tier 1: primary model (most capable, most expensive)
try {
const result = await primaryModel.generate(query, context);
if (isValid(result)) return { result, tier: 'primary' };
} catch (error) {
logger.warn('Primary model failed', { error: error.message });
}
// Tier 2: smaller, faster model with a simplified query
try {
const result = await fastModel.generate(simplifyQuery(query), context);
if (isValid(result)) return { result, tier: 'secondary' };
} catch (error) {
logger.warn('Secondary model failed', { error: error.message });
}
// Tier 3: cached answer for a known question
const cached = await responseCache.get(normalizeQuery(query));
if (cached) return { result: cached, tier: 'cache' };
// Tier 4: graceful degradation with human escalation
return {
result: generateFallbackResponse(query),
tier: 'fallback',
requiresHumanEscalation: true
};
};
The validation check between tiers is the important part. A model call that returns garbage has not "succeeded"; falling through to the next tier on invalid output is what makes this a recovery pattern rather than a try/catch chain. Be honest downstream about which tier answered — a cached or degraded answer should be labeled as such to the user and in your logs.
4. State checkpoints and rollback
Multi-step agents need the equivalent of a save point. Checkpoint state before risky or irreversible operations so a failure does not force a full restart:
const executeWithCheckpoint = async (agent, operation) => {
const checkpoint = await agent.saveState();
try {
const result = await operation();
if (isValidResult(result)) {
await agent.commitState();
return result;
}
await agent.restoreState(checkpoint);
return retryWithModifiedApproach(agent, operation);
} catch (error) {
await agent.restoreState(checkpoint);
return handleError(error, { recovery: 'rollback', checkpointId: checkpoint.id });
}
};
You do not have to build this yourself. Agent frameworks have converged on checkpointing as a first-class feature: LangGraph's checkpointers, for example, save a snapshot of graph state at every step, let you restart a failed run from the last successful step, and even preserve the outputs of nodes that completed inside a failed step so they are not re-run on resume (LangGraph checkpointer documentation). Two lessons from that documentation apply regardless of framework:
- Use durable storage in production. In-memory savers lose every checkpoint when the process restarts; use a database-backed saver.
- Choose a durability mode deliberately. Writing checkpoints synchronously before each step costs latency; writing them asynchronously or only at exit is faster but loses more work in a crash. Match the mode to how expensive re-running a step is.
One hard limit: checkpoints restore your agent's state, not the world's. If step three sent an email and step four crashed, rolling back the agent's state does not un-send the email. For irreversible side effects, prefer designing the step to be idempotent or compensatable rather than relying on rollback.
5. Partial failure handling
When an agent runs several independent tasks, one failure should not void the rest:
const executeWithPartialFailure = async (tasks) => {
const results = await Promise.allSettled(tasks.map(t => t.execute()));
const succeeded = results.filter(r => r.status === 'fulfilled');
const failed = results.filter(r => r.status === 'rejected');
if (succeeded.length === 0) throw new Error('All tasks failed');
if (failed.length > 0) {
logger.warn(`Partial failure: ${failed.length}/${tasks.length} tasks failed`);
return {
status: 'partial_success',
data: succeeded.map(r => r.value),
failures: failed.map((r, i) => ({ task: tasks[i].name, error: r.reason?.message })),
message: `Completed ${succeeded.length} of ${tasks.length} operations`
};
}
return { status: 'complete_success', data: succeeded.map(r => r.value) };
};
Promise.allSettled instead of Promise.all is the whole trick: one rejection no longer aborts the others. The caller gets usable results plus a structured list of what failed, which feeds directly into a retry or escalation path.
6. Context window recovery
Long-running agents eventually overflow the model's context window. Handle it as a planned degradation, not a crash:
const handleContextOverflow = async (prompt, context, maxTokens) => {
if (estimateTokens(prompt) + estimateTokens(context) <= maxTokens) {
return await model.generate(prompt, context);
}
// Strategy 1: summarize older context, keep recent turns verbatim
const summarized = await summarizeContext(context, { targetTokens: maxTokens * 0.3 });
try {
return await model.generate(prompt, summarized);
} catch (error) {
if (error.code !== 'CONTEXT_LENGTH_EXCEEDED') throw error;
// Strategy 2: keep only the most recent context
const recent = truncateToRecent(context, { maxTokens: maxTokens * 0.2 });
return await model.generate(prompt, recent);
}
};
Summarize-then-truncate preserves more signal than truncation alone, but it costs an extra model call and can lose details the summary elided. For agents where old context carries obligations (an earlier instruction, a constraint the user stated), structured memory — extracting facts into a store rather than compressing the transcript — is more reliable than either.
7. Graceful degradation and escalation
When every automated path fails, the agent should still fail well: tell the user what happened, keep their context, and offer a route to a human.
const generateFallbackResponse = (query, error) => {
const messages = {
'RATE_LIMIT': 'I am hitting a usage limit right now. Here is a simpler answer while that clears...',
'TIMEOUT': 'That is taking longer than expected. Here is what I can tell you quickly...',
'CONTEXT_EXCEEDED': 'That is a complex question. Let me focus on the most important part...',
'SERVICE_UNAVAILABLE': 'I cannot reach one of my data sources. Here is what I can answer without it...'
};
return {
message: messages[error.code] || 'I hit a technical problem. Let me help another way...',
degradedResponse: generateSimplifiedResponse(query),
escalationOption: 'Would you like me to bring in a human colleague?'
};
};
A dead letter queue belongs next to this: operations that exhaust all recovery paths get queued with full context for manual review, and critical ones page whoever is on call. Silent failure is the worst outcome in production — worse than a visible error — because it is invisible to both the user and your team.
A worked recovery decision flow
Patterns are easy to list and hard to sequence. Here is the order a production agent should apply them when a step fails:
- Classify the error. Is it retryable (429, 5xx, timeout, connection reset) or permanent (401, 403, malformed request, validation failure)? Permanent errors skip to step 5.
- Retry with backoff and jitter, honoring any
retry-afterheader. Cap attempts at two to four. - Check the circuit breaker for the failing dependency. If the breaker opens, stop hammering the service and route around it.
- Fall to the next tier — a smaller model, a cached answer, a simplified query — with output validation at each tier.
- Restore from the last checkpoint if the failure corrupted workflow state, then retry the step with a modified approach.
- Degrade gracefully and escalate. Return a labeled partial answer, preserve the user's context, queue the operation in the dead letter queue, and offer human handoff.
Steps 1-3 take milliseconds to seconds. Step 6 is where user trust is won or lost, and it is the step most demos skip.
When recovery is the wrong choice
Recovery logic is not free, and sometimes the right response to failure is to stop:
- Never retry non-idempotent side effects blindly. Amazon's guidance is blunt: an API with side effects is not safe to retry unless it is idempotent. If "charge card" timed out, the charge may already have happened. Retry only against APIs with idempotency keys, or check state before repeating the operation.
- Do not retry permission and validation failures. A 403 will be a 403 forever. Fix credentials or the request, not the retry count.
- Do not let recovery hide a real outage. If your fallback tier serves 40% of traffic for an hour, your "recovery" is masking an incident. Alert on tier distribution, not just error counts.
- Escalate early on high-stakes actions. For money movement, legal filings, medical or HR actions, a single failed validation should route to a human rather than trigger creative self-correction.
- Do not return degraded output as if it were full-quality. A summarized or cached answer presented as fresh misleads users and every system downstream of them.
Common error recovery mistakes
Infinite retry loops. Without a hard cap, agents retry forever and burn budget. Every retry path needs a maximum attempt count and a terminal failure state.
Losing user data on failure. Never discard the user's input or conversation context because your system errored. Persist state before risky operations so recovery can resume where things left off.
Silent failures. An error that is not logged, metered, and (for critical paths) alerted on cannot be diagnosed or improved. Log the error, the recovery strategy attempted, and the outcome.
Retry storms. Many agents retrying a struggling service simultaneously multiply its load. Jitter, caps, and circuit breakers exist precisely to prevent this — the AWS Builders' Library piece above is the best short read on why.
No human escalation path. Automated recovery will eventually fail. If there is no route to a person, the user is stranded.
Measuring recovery
Recovery paths need the same observability as the happy path. Track at least:
- Error rate by type — model, tool, network, validation — so you know which pattern to invest in.
- Recovery success rate by strategy — which tiers and patterns actually rescue interactions.
- Fallback tier distribution — a rising share of cached or degraded answers is an early warning of an upstream problem.
- Dead letter queue depth and age — unresolved failures piling up means recovery is failing systematically.
- Escalation rate and time to human handoff — the user-facing cost of unrecoverable errors.
These metrics feed directly into your broader AI agent monitoring and observability setup; recovery events are among the most valuable signals an agent system produces, and our production resilience guide covers how to alert on them without drowning in noise.
Frequently asked questions
How many times should an AI agent retry a failed call?
Two to four attempts for transient errors, with exponential backoff and jitter between them. If the API sends a retry-after header, wait at least that long. More attempts rarely help and actively hurt when the downstream service is overloaded.
What is the difference between a retry and a fallback? A retry repeats the same operation hoping the failure was transient. A fallback abandons that approach and tries a different one — a smaller model, a cached answer, a simplified query. Retry first for transient errors; fall back when retries are exhausted or the error is a type retries cannot fix.
How do I stop an agent losing its progress when it crashes? Checkpoint its state after every meaningful step, using durable storage rather than process memory. Frameworks like LangGraph do this natively and can resume a run from the last successful step, including preserving the outputs of steps that completed before the crash.
Should every failure trigger automated recovery? No. Authentication failures, permission errors, and malformed requests should fail fast. Non-idempotent operations with side effects — payments, emails, record writes — should not be retried blindly. And high-stakes actions should escalate to a human after minimal automated attempts.
Can retries make an outage worse? Yes. Retries add load to a service that may be failing because it is already overloaded, and synchronized retries from many clients create bursts that delay recovery. This is why backoff, jitter, attempt caps, and circuit breakers are not optional extras.
Where to go from here
Error recovery is designed, not bolted on: the failure modes, retry policies, checkpoint strategy, and escalation paths have to be decided alongside the agent's tools and workflow, not after the first incident. This is core to how we build production systems in our Claude implementation engagements, and it is built into the always-on AI employees we run for clients. If you are hardening an agent for production, start with the decision flow above and instrument every step before you need it.
About AI Agents Plus Editorial
AI automation expert and thought leader in business transformation through artificial intelligence.



