Claims AI Architecture: Where Fine-Tuning Ends and RAG Begins

Most claims AI projects fail at the architecture decision, not the model decision. Teams pick fine-tuning when the problem is stale knowledge, or build an elaborate retrieval system when the real need is a disciplined classifier, and they discover the mismatch after the budget is spent. The correct answer for almost every insurer is a split: fine-tune the stable, repeatable parts of the claims pipeline and put retrieval in front of everything that changes, everything that needs a citation, and everything a regulator might ask you to explain.

This post maps each stage of a claims pipeline to the technique that actually fits it, with real cost and effort numbers from published experiments, the regulator expectations that should shape your design, and a checklist you can run against your own workflow before you commit.

Why claims is the hardest test of this decision

Claims work sits at the worst intersection for AI: the knowledge changes constantly, the evidence is fragmented, and every decision has to be defensible.

Policy wording updates with each product refresh. Endorsements and exclusions accumulate. Claim files arrive as a mess of PDFs, photos, medical reports, handwritten notes, and email threads. And when a claim is denied, an adjuster, a complaints handler, or a regulator needs to trace the decision back to a specific clause in a specific document.

A Nomura Research Institute experiment shows how badly naive assumptions hold up here. NRI trained models on 1,020 past insurance certification exam questions and compared retrieval-augmented generation against fine-tuning. RAG actually lowered GPT-3.5 scores on the academic test and barely moved GPT-4. Fine-tuning substantially improved academic scores but made practical skills-test performance worse overall, and the fine-tuned model started recommending insurance products that do not exist. Neither technique was a reliable win on its own, even in a controlled setting with clean data.

The lesson is not that both techniques fail. It is that they fail in different places, which is exactly why the pipeline mapping matters.

The pipeline map: five stages, three techniques

A claims pipeline has distinct stages with different failure modes. Treat each one separately.

Stage 1: Document intake and classification — fine-tune

Incoming claims bring a stream of document types: first notice of loss forms, police reports, medical bills, repair estimates, photos, correspondence. Routing these correctly is a stable, high-volume classification task with clear right and wrong answers.

This is the strongest case for fine-tuning. A small model fine-tuned on a few thousand labeled examples will classify documents more cheaply and more consistently than a large general model reading every page. STX Next, describing a production insurance RAG build, uses a small fine-tuned model as a router precisely so the expensive large model is reserved for the reasoning work that needs it.

You do not need a frontier model here. Modern parameter-efficient methods changed the economics completely: the LoRA paper showed you can adapt a large model by training small injected matrices instead of all weights, cutting trainable parameters by a factor of 10,000 and GPU memory by 3x compared with full fine-tuning, with no added inference latency. QLoRA pushed this further, fine-tuning a 65-billion-parameter model on a single 48GB GPU. An 8-billion-parameter open-weight classifier for document routing is a same-day training job on one GPU.

Stage 2: Extraction and normalization — fine-tune, with retrieval backup

Pulling structured fields out of claims documents (dates of loss, policy numbers, injury codes, amounts claimed) is again repetitive and rule-bound. Fine-tuned models or adapters produce consistent schemas and formats, which is what downstream systems need. Insurance AI reference material from Charlee lists exactly these as the defensible fine-tuning targets: classifying correspondence, extracting exposure signals, building structured claim chronologies.

The catch is documents you have never seen. New hospital billing formats, a new repair estimate template, a form from a jurisdiction you just entered. This is where a retrieval layer earns its place as backup: retrieval with a capable multimodal model can handle unseen document formats because it reads the document fresh each time rather than relying on patterns baked into weights. An industry architecture guide from Datos Insights notes that multimodal models can extract information from submissions, loss runs, and inspection reports even when that document format has not been previously encountered.

The practical pattern: fine-tune for the 90 percent of volume that follows known formats, route low-confidence extractions to the retrieval-backed model, and feed corrected examples back into the next training round.

Stage 3: Coverage reasoning — RAG, never fine-tuning

This is the stage where the decision carries the most risk, and where the answer is clearest.

Coverage questions depend on the current policy wording, the specific endorsements on that policy, the jurisdiction, and the facts of this claim. All of that is knowledge, not behavior. Fine-tuning bakes knowledge into model weights where it goes stale, cannot be cited, and cannot be updated without retraining. Retrieval pulls the actual policy document at query time and can show the adjuster the exact clause.

Production-grade claims retrieval needs more than basic vector search. The STX Next engineering account describes what the real version requires:

• Granular citation. An answer that points to a 200-page policy manual is useless. The system must cite the page, section, and clause.
• Conflict detection. Policy documents, historical claims, and internal guidance sometimes contradict each other. The system should flag the contradiction instead of guessing.
• Query expansion. Adjusters type shorthand, typos, and fragments. Techniques like multi-query expansion and HyDE turn a sloppy question into several useful searches.
• Metadata discipline. Every chunk needs lineage back to its source document, or auditability collapses.

If you are weighing this decision across a whole insurance or banking AI portfolio rather than claim by claim, our RAG or fine-tuning decision guide for banks and insurers covers the general framework. This post is the claims-specific version.

Stage 4: Customer communication — fine-tune for voice, retrieve for facts

Denial letters, approval letters, and status updates must use approved wording, mandatory disclosures, and a consistent tone. That is behavioral consistency, and fine-tuning on your approved correspondence library delivers it. Binariks lists claims letters and regulatory disclosures among the strongest insurance fine-tuning cases for exactly this reason: the wording cannot vary.

But the facts inside the letter (the coverage reason, the clause being relied on, the amount) must come from retrieval and the claim system of record, never from the model's memory. NRI's experiment shows why: their fine-tuned model invented a plausible-sounding insurance product when asked for recommendations. A fine-tuned drafting model that hallucinates a coverage reason in a denial letter is a regulatory incident, not a rounding error.

The safe pattern is a template-driven draft: retrieval supplies the cited facts, the fine-tuned model supplies structure and voice, and a human approves anything adverse.

Stage 5: Fraud signals and triage — fine-tune, with rules on top

Fraud pattern detection and severity triage are classification problems trained on historical outcomes, so fine-tuning fits. But two constraints apply.

First, historical labels encode historical bias. If past fraud flags over-represented certain postcodes or demographics, fine-tuning will faithfully reproduce that pattern. Charlee calls this the biggest dataset risk and it is the one regulators look for first.

Second, hard business rules should stay as deterministic rules. A claim over a certain amount goes to a senior adjuster; a claimant with three claims in 12 months gets reviewed. Do not teach a model rules that a rules engine enforces perfectly and audibly. We covered the triage accuracy, human review, and cost trade-offs in detail in LLM claims triage for insurers.

What the work actually costs

Vendor decks rarely attach numbers. Here are the numbers that exist in public sources, plus what they imply for a mid-size insurer handling around 25,000 claims a year.

Data preparation dominates fine-tuning cost. NRI spent 70 person-hours reformatting just 1,020 exam questions into training-ready Q&A format, and noted that unstructured material like operations manuals would be even more labor-intensive. For a claims document classifier needing 3,000 to 5,000 labeled examples, budget 150 to 350 person-hours of cleaning and labeling before any GPU starts. That is the line item projects forget.

Training compute is the cheap part now. The QLoRA result (a 65B model fine-tuned on a single 48GB GPU, with their best model needing 24 hours on one GPU) means an 8B classifier adapter trains in hours on hardware that fits under a desk. The API-era comparison from NRI makes the same point from the other side: fine-tuning GPT-3.5 on their dataset cost $3.30, $6.60, and $13 at 1, 5, and 10 epochs respectively, while accuracy barely moved between 5 and 10 epochs. More training is not more accuracy. Past a point it is just more invoice.

Retrieval cost is front-loaded into engineering, not training. A production claims RAG system needs document cleaning pipelines, chunking and metadata design, evaluation sets, and conflict detection before it answers its first question. There is no $13 shortcut here, but there is also no retraining bill when a policy updates: you replace a document in the index and the system is current.

Ongoing cost profiles invert. The fine-tuned classifier costs almost nothing to run and needs retraining only when document types or performance drift. The RAG layer has steady infrastructure cost (vector storage, embedding, the generation model) but zero retraining cost for knowledge updates. A hybrid pipeline for a 25,000-claim operation typically lands as one modest GPU for the classifiers plus a retrieval stack sized for the policy corpus, with the largest recurring cost being evaluation and monitoring time, not compute. For the training-side practices that keep those costs down and accuracy up, see our LLM fine-tuning best practices guide.

The regulator is part of your architecture

In the United States, the NAIC's model bulletin on insurer AI use, adopted in December 2023, has now been adopted in some form by roughly 29 jurisdictions, with California, Colorado, New York, and Texas running their own regimes. The bulletin itself expects a written AI systems program with senior-management accountability, lifecycle controls from design to retirement, vendor due diligence with audit rights, and documentation a regulator can demand during any market conduct examination.

Two architecture consequences follow directly.

Retrieval gives you evidence; fine-tuning gives you behavior. NRI's framing is the cleanest: with RAG, the way the model reaches an answer from retrieved data is transparent, while fine-tuned inference is a black box. When a regulator asks why a claim was denied, the RAG layer can produce the clause that was retrieved and cited. The fine-tuned classifier can produce its confidence score and training lineage. What neither can do is explain itself after the fact unless you logged everything at decision time.

Logging is a design requirement, not a feature. Charlee's recommended minimum log per decision is a good checklist: model and dataset version, input, instructions, retrieved context, output, confidence flags, reviewer action, and final disposition. Build that into the pipeline from day one. Bolting it on after a regulator asks is the expensive version. And if your models take instructions from document content, treat prompt injection as a layered defense problem, because a claims pipeline that reads claimant-supplied documents is an attack surface.

A decision checklist for your own pipeline

Run each candidate task through these questions, in order:

1. Does the task depend on current documents, policy wording, or case-specific facts? If yes, retrieval. Do not fine-tune knowledge into weights.
2. Is it a stable, high-volume, repeated behavior with clear right answers? If yes, fine-tuning is justified, provided you have or can create labeled examples.
3. Could a rules engine do it deterministically? If yes, use rules. They are cheaper, faster, and audit themselves.
4. Does the output need a citation a third party could verify? If yes, retrieval with granular sourcing is mandatory.
5. Would an error here trigger a regulatory or legal obligation? If yes, add human review regardless of measured accuracy, and log the full decision chain.
6. Do you have the labeled data rights and quality to fine-tune? Historical bias in, historical bias out, and now it is your model doing it.

Common mistakes

Fine-tuning to fix a knowledge problem. If the model gives outdated answers, retraining is the slowest and most expensive patch. Update the retrieval index instead.

Building shallow RAG and blaming retrieval. A single embedding search over unchunked PDFs will produce vague answers, and teams conclude RAG does not work. The failure is the missing metadata, citation granularity, and conflict detection, not the concept.

Training on whatever data exists rather than what the task needs. NRI's skills-test scores dropped partly because their training data (academic exam questions) did not match the practical task. Label relevance beats label volume.

Skipping the held-out test set and regression tests. Fine-tuned models can become more fluent while becoming less accurate, and catastrophic forgetting can quietly degrade capabilities the base model had. Test against cases the model never saw, before and after every retraining.

Letting the drafting model state facts. Correspondence models should receive facts from retrieval and systems of record. A fluent model with write access to factual claims will eventually invent one.

Frequently asked questions

Can I fine-tune Claude or GPT-4 on my claims data? OpenAI offers fine-tuning on some models; Anthropic does not offer customer fine-tuning for Claude. For claims work where data cannot leave your environment, the practical route is fine-tuning open-weight models (Llama, Qwen, Mistral, Gemma) with parameter-efficient methods like LoRA, which keeps everything inside your infrastructure.

How much labeled data do I need to fine-tune a claims classifier? Published work suggests a few thousand high-quality, task-matched examples beat tens of thousands of loose ones. Plan around 3,000 to 5,000 labeled examples per document class set, and budget most of the effort for cleaning and labeling rather than training.

Is RAG accurate enough for coverage decisions? Not alone, and it should not be the decision-maker. Retrieval with granular citation is accurate enough to put the right clause in front of an adjuster with a verifiable source. The adjuster makes the decision; the system's job is evidence assembly with a clean audit trail.

How often does a fine-tuned claims model need retraining? On evidence, not on a calendar. Retrain when measured performance drifts, when document types or workflows change materially, or when evaluation on fresh data shows degradation. Scheduled retraining without a performance trigger wastes money and adds regression risk.

What does a regulator actually ask for? Under the NAIC model bulletin, expect requests for your written AI program, model inventories, validation documentation, third-party due diligence, and the decision records for specific consumer outcomes. If you cannot produce the retrieved evidence and model version behind a specific denial, that is the gap they will find.

Where to go from here

The architecture that survives contact with a claims department is almost always a hybrid: fine-tuned adapters for classification, extraction, and voice; retrieval with granular citation for anything touching policy knowledge; rules for hard controls; logging everywhere. Getting that split right is mostly a mapping exercise against your own pipeline, and it is far cheaper to do on a whiteboard than in production. This is the work our enterprise AI practice for banks and insurers does: fine-tuning open-weight models, building retrieval systems, and writing the governance documentation regulators expect. If your pipeline mixes claims intake with other document workflows, our custom builds team can scope the integration points.