AI Agent Error Handling and Retry Strategies: Production Guide 2026

AI agents fail in ways ordinary software does not: the model can return malformed tool calls, the API can throttle you mid-workflow, and a step can half-succeed and leave corrupted state behind. The fix is not "wrap everything in try/except" but a layered set of strategies: classify the error, retry only what is retryable with backoff and jitter, trip a circuit breaker when a dependency is down, and checkpoint multi-step work so a failure never means starting over. This guide walks through each layer with working Python and the provider-specific details that decide whether a retry helps or makes things worse.

Why AI agents fail differently from ordinary software

A web request either works or returns an error. An agent run can fail in five distinct ways, and each needs different handling:

• Transient infrastructure failures — timeouts, connection resets, rate limits. Usually retryable.
• Model output failures — malformed JSON, hallucinated tool names, invalid arguments. Retryable, but only with feedback, because repeating the identical prompt often reproduces the identical mistake.
• Partial execution — the agent completes steps 1 through 3 of 5 and dies. Retrying from scratch can duplicate side effects (two emails sent, two charges created).
• Cascading failures — one flaky dependency poisons every agent run until someone intervenes.
• Silent failures — the run "succeeds" but the output is wrong. No exception is ever raised, so no retry logic fires.

The practical consequence: a single generic retry loop is the wrong tool. You need a decision step first.

Classify the error before you retry

Every retry decision starts with one question: would calling this again, unchanged, plausibly succeed?

Using the Claude API as the concrete example, Anthropic's error documentation defines distinct classes:

• 429 rate_limit_error — your organization exceeded a per-minute limit. Retryable, and the response carries a retry-after header telling you exactly how long to wait.
• 429 from a spend cap — looks identical, but has no retry-after header and keeps failing until access resumes. Retrying it is pure waste; you must detect and alert instead.
• 529 overloaded_error — Anthropic's servers are at capacity across all users. Retryable with backoff, but if it persists, the answer is failing over to the same model on another provider (Bedrock or Vertex), not retrying harder.
• 500 api_error / 504 timeout_error — retryable with backoff. For long requests, the docs recommend the streaming API instead of a single blocking call.
• 400 invalid_request_error, 401, 403, 404 — not retryable. Fix the request, the key, or the permissions.

Worth knowing: the official Anthropic SDK already retries transient failures (connection errors, 429, 5xx) with exponential backoff twice by default, honoring retry-after. Your own retry logic sits on top of that, so configure max_retries deliberately rather than stacking two uncoordinated retry layers.

The same classification applies to tool calls your agent makes. A database unique-constraint violation will fail forever; a connection pool timeout will not.

Strategy 1: Exponential backoff with jitter

For transient failures, wait progressively longer between attempts, and randomize the wait. Both halves matter. Amazon's Builders' Library article on timeouts, retries, and backoff with jitter explains why: if many clients fail at once and all back off by the same schedule, they retry in lockstep and re-overload the recovering service. Jitter decorrelates the retry bursts.

import asyncio
import random

async def retry_with_backoff(func, max_attempts=5, base_delay=1.0, max_delay=30.0):
    for attempt in range(max_attempts):
        try:
            return await func()
        except RetryableError as e:
            if attempt == max_attempts - 1:
                raise
            # Honor the server's hint when it gives one
            delay = getattr(e, "retry_after", None) or min(
                base_delay * (2 ** attempt), max_delay
            )
            # Full jitter: sleep anywhere in [0, delay]
            await asyncio.sleep(random.uniform(0, delay))

Three details that separate this from a toy:

• Cap the attempts. AWS's Well-Architected guidance on controlling retry calls lists "retries without a maximum" as a top anti-pattern. Four to five attempts is a sane default for an LLM call; more just delays the inevitable failure.
• Retry only retryable errors. Catching Exception here means retrying 401s and schema bugs for 30 seconds before giving up.
• Set a timeout on every call. Without one, a hung connection holds its resources forever. Amazon's practice is to derive timeouts from the downstream service's latency percentiles, picking a value near p99.9 so roughly 0.1% of healthy calls false-trip.

Strategy 2: Circuit breakers for sick dependencies

Retries assume the dependency is mostly healthy and failing transiently. When a dependency is genuinely down, retries amplify the damage, which is exactly what the Builders' Library warns against. The circuit breaker, popularized by Michael Nygard's Release It and described in Martin Fowler's write-up, stops the bleeding: after a threshold of failures, calls fail immediately without touching the dependency, and after a cooldown the breaker lets one trial call through (half-open) to test recovery.

import time

class CircuitBreaker:
    def __init__(self, failure_threshold=5, reset_after=60):
        self.failure_threshold = failure_threshold
        self.reset_after = reset_after
        self.failures = 0
        self.opened_at = None

    async def call(self, func):
        if self.opened_at is not None:
            if time.time() - self.opened_at < self.reset_after:
                raise CircuitOpenError("dependency down, failing fast")
            # Half-open: let one trial call through
        try:
            result = await func()
        except Exception:
            self.failures += 1
            if self.failures >= self.failure_threshold:
                self.opened_at = time.time()
            raise
        # Success closes the circuit
        self.failures = 0
        self.opened_at = None
        return result

Two points from Fowler that are easy to miss. First, not every error should trip the breaker; a 400 from a malformed request is your bug, not the dependency's health. Second, when the breaker is open, your agent needs a defined answer: serve cached data, queue the task for later, or tell the user the feature is temporarily unavailable. "Fail fast with a plan" beats "hang for 30 seconds, then fail."

A production agent typically wraps each external dependency in its own breaker: one for the LLM provider, one for the search API, one for the CRM. That way a sick CRM does not take down agent runs that never needed it.

Strategy 3: Checkpointing for multi-step agents

Retrying a five-step agent run from step 1 is expensive and dangerous when earlier steps had side effects. Persist progress after each step so a crash resumes from the last good state:

async def run_with_checkpoints(agent_id, steps, store):
    state = store.load(agent_id) or {"completed": 0, "results": []}
    for i in range(state["completed"], len(steps)):
        result = await steps[i]()
        state["results"].append(result)
        state["completed"] = i + 1
        store.save(agent_id, state)  # persist BEFORE the next step
    return state["results"]

For this to be safe, each step must be idempotent or carry an idempotency key: "create customer with reference X" must not create a second customer if retried. The AWS guidance makes the same point for distributed systems generally — only retry side-effecting calls that were designed to be retried. Where a step cannot be made idempotent, record that it ran and require human review on resume.

Checkpoint state belongs in durable storage (Postgres, Redis with persistence, a queue), not in process memory. An always-on agent that loses its checkpoint on every restart has no real checkpointing. This is one of the reasons our AI Employees service runs agents on a persistent, monitored stack rather than as ephemeral scripts.

Strategy 4: Validation retries with self-correction

Model output failures need a different loop: retrying the identical prompt often reproduces the identical malformed JSON. Feed the validation error back so the model can correct itself:

async def validated_completion(llm, prompt, validate, max_attempts=3):
    for attempt in range(max_attempts):
        response = await llm.complete(prompt)
        ok, error = validate(response)
        if ok:
            return response
        prompt = (
            f"Your previous response failed validation.\n"
            f"Response:\n{response}\n\n"
            f"Validation error: {error}\n"
            f"Return a corrected response only."
        )
    raise ValueError("validation failed after max attempts")

Keep the validator cheap and deterministic: JSON parse plus a schema check, a regex for an ID format, a range check on a number. Validating with another LLM call doubles cost and introduces a second model that can also be wrong. And cap attempts at two or three — a model that cannot produce valid output with the error spelled out rarely succeeds on attempt six. If you are building tool-use integrations with MCP servers and Skills, this is the layer where most production bugs actually live; our Claude implementation work spends a disproportionate share of time here for exactly that reason.

A worked example: a resilient intake agent

Take an agent that reads an inbound email, extracts structured data, checks it against a CRM, and files a ticket. One run, four failure points. A defensible configuration:

• LLM extraction call — SDK max_retries=2 built in, wrapped in backoff with jitter capped at 4 attempts, 20-second per-call timeout. 429 honors retry-after; persistent 529 fails over to the same model on Bedrock.
• Output validation — schema check, two self-correction attempts, then the email routes to a human review queue instead of guessing.
• CRM lookup — circuit breaker with a 5-failure threshold and 60-second reset. When open, the agent files the ticket with a "CRM unverified" flag instead of blocking the whole intake pipeline.
• Ticket creation — idempotency key derived from the email's message ID, so a resume after a crash never double-files.
• The whole run — checkpointed per step, with a dead-letter queue catching anything that exhausts every layer, so a human sees exactly which step failed with what error.

Total added latency in the happy path: zero beyond one state write per step. Total cost in the failure path: bounded, visible, and recoverable.

Common mistakes

• Retrying everything. A 400 or 401 retried five times is five wasted calls and a slower error message.
• Ignoring retry-after. The server told you when to come back; retrying earlier is guaranteed to fail and burns your remaining quota.
• No jitter. A fleet of agents that all crash on the same rate limit will all retry on the same second and crash again.
• Stacking retry layers. SDK retries 2x, your wrapper retries 5x, the queue retries 3x: one bad call becomes 30 attempts against a struggling API.
• No timeout. A retry policy on top of an infinite wait is a policy for hanging forever, slowly.
• Silent DLQ. A dead-letter queue nobody reads is where tasks go to be forgotten. Alert on arrival.
• Retrying the spend-cap 429. As Anthropic's docs note, that error has no retry-after and never clears on its own. The fix is billing configuration and an alert, not backoff.

When retries are the wrong choice

Retries are for transient, load-independent failures. Do not retry when the failure is deterministic (bad request shape, missing permission, schema mismatch), when the system is failing because of load and your retries add more of it, or when the cost of a duplicate side effect exceeds the cost of a human handling it. In those cases the right tools are input validation, circuit breaking with a fallback, a queue for later processing, or an escalation path to a person. A well-built agent knows when to stop trying.

None of this works blind: you need metrics on retry counts, breaker state changes, and DLQ arrivals to know which layer is doing the work. We cover that side in our guide to AI agent monitoring and observability, and the broader recovery playbook in AI agent error recovery strategies for production systems.

Frequently asked questions

How many times should an AI agent retry a failed API call?

Four to five attempts with exponential backoff and jitter is a sound default for transient errors. The Anthropic SDK retries twice by default; if you add your own layer, reduce one or the other so the layers multiply to a sane total rather than an explosive one.

What is the difference between a 429 and a 529 error from the Claude API?

A 429 means your organization exceeded its own rate limit; the response includes a retry-after header you should honor. A 529 means the API is overloaded for everyone, independent of your usage; retry with backoff, and if it persists, fail over to the same model on another provider. A third 429 variant, the monthly spend cap, has no retry-after and never clears by retrying.

Should I retry when the LLM returns malformed JSON?

Yes, but not with the identical prompt. Parse the output, and on failure send the response and the validation error back to the model so it can correct itself. Two or three correction attempts is enough; beyond that, the prompt or schema is the problem.

Is exponential backoff enough on its own?

No. Without jitter, synchronized clients retry in lockstep and re-overload the recovering service. Without a maximum, a permanently failing call retries forever. AWS's Well-Architected Framework lists retries without backoff, jitter, and a cap as a primary anti-pattern.

How do I stop a failing tool from breaking every agent run?

Wrap the dependency in a circuit breaker: after a threshold of consecutive failures, calls fail fast without touching the dependency, and a half-open trial call tests recovery after a cooldown. Define what the agent does while the circuit is open — cached data, a queued task, or a clear message to the user.

Where to go from here

Error handling is what separates a demo agent from one you can trust with real work, and it is the part of an agent build that takes the most production experience to get right. If you want an always-on agent with retries, checkpointing, and monitoring already built in, take a look at our AI Employees service, built on the open-source Hermes agent running Claude.