The Real Cost of Fine-Tuning an LLM for a Bank or Insurer

Fine-tuning a 70-billion-parameter open-weight model costs about $20 in GPU time. The full project costs 50 to 200 times that once you count data preparation, evaluation, validation and serving. If you are budgeting an LLM fine-tune for a bank or insurer, the GPU line is the one number your vendors will quote and the one number that matters least. This post breaks down every cost line with current, checkable prices, walks through a worked budget for a mid-size insurer, and tells you when fine-tuning is simply the wrong spend.

Two scope notes before the numbers. First, this is about fine-tuning open-weight models such as Llama, Qwen, Mistral and Gemma inside your own cloud perimeter. Anthropic does not offer customer fine-tuning of Claude models, so for regulated institutions the open-weight path is the one that keeps training data inside your VPC. Second, if you have not yet decided whether you need fine-tuning at all, read our RAG or fine-tuning decision guide for banks and insurers first. Retrieval solves knowledge problems; fine-tuning solves behavior problems. This post assumes you have a genuine behavior problem: consistent classification, strict output formats, extraction into fixed schemas, or house style that prompting cannot hold.

Why the compute got so cheap

Three ideas did it, and they are worth understanding because they determine which GPU tier you pay for.

LoRA (Low-Rank Adaptation, from Microsoft Research) freezes the base model and trains small rank-decomposition matrices injected into each layer. Against GPT-3 175B fully fine-tuned with Adam, the LoRA paper reports 10,000 times fewer trainable parameters and 3 times less GPU memory, with quality on par with or better than full fine-tuning and no added inference latency. You train megabytes of adapter weights, not billions of parameters.

QLoRA goes further: it quantizes the frozen base model to 4 bits and backpropagates through it into the LoRA adapters. The QLoRA paper demonstrated a 65B model fine-tuned on a single 48GB GPU while preserving full 16-bit task performance, and its Guanaco models reached 99.3% of ChatGPT's level on the Vicuna benchmark after 24 hours on one GPU. The same paper found that a small, high-quality dataset beats a large mediocre one, which has direct budget implications: spend on data quality, not dataset size.

GPU rental markets decoupled training from hyperscaler minimums. A single H100 SXM rents on-demand for $3.99 per GPU-hour at Lambda's published rates, with A100 80GB at $2.79. Meanwhile AWS bills its 8-GPU p4d.24xlarge as a whole node, and one vendor analysis found a job needing a single A100 costing $229 on that node versus about $10 on a single-GPU spot rental, because you pay for eight GPUs whether or not the job uses them.

What the compute actually costs

The memory footprint decides the hardware, and the hardware decides the price. Approximate VRAM requirements, from io.net's budget guide:

• 7B model: about 50GB full fine-tune, 20GB LoRA, 8GB QLoRA
• 13B model: about 90GB full, 35GB LoRA, 14GB QLoRA
• 70B model: about 400GB full, 140GB LoRA, 40GB QLoRA

That 40GB figure for 70B QLoRA is the important one. It fits on a single A100 40GB or a single H100, which is why a 70B QLoRA run on 25,000 examples costs roughly $20 in rental compute, about 10 hours on an H100 PCIe at $2.01 per hour in Spheron's worked numbers (prices dated 7 July 2026; check current rates before budgeting). The same analysis puts a full-precision 70B LoRA run on two H100 SXMs at $81 to $142, and a full 70B fine-tune on 100,000 examples across 11 H100s at about $1,785.

The hosted API alternative prices per token instead of per GPU-hour. Together AI's fine-tuning tiers, as quoted in the same analysis, run from $0.48 per million training tokens for LoRA on sub-16B models up to $2.90 per million at the 70-100B tier, with full fine-tuning roughly 2.5 times the LoRA rate. The identical 70B QLoRA job that costs $20 rented comes to $148 to $154 through a per-token API. More importantly for a bank, hosted fine-tuning means your training data leaves your perimeter. For most regulated institutions that ends the comparison before the price does.

So the compute budget for a serious project looks like this:

• Single 70B QLoRA training run: $20 to $40
• Full iteration program, 15 to 25 runs: $400 to $1,000
• Full fine-tune instead of adapters, if you truly need it: $1,500 to $2,500 per run
• Annual compute for a team fine-tuning quarterly with iteration: low four figures

Read that last line again. If a proposal for fine-tuning compute comes in at six figures, the gap is almost certainly in the GPU pricing layer, not the job.

The costs the GPU vendors leave out

Compute is line one of six. The other five are where projects actually spend, and where budgets blow up.

Training data preparation. This is the largest cost in almost every fine-tune. You need thousands of examples that show the behavior you want: reviewed claims classifications, redlined draft letters in your house style, extraction outputs in your exact schema. Each example needs a subject-matter expert to write or review it. For 3,000 to 5,000 examples at 10 to 15 minutes of expert review each, that is 500 to 1,250 hours of your most expensive people's time. At blended internal rates this dwarfs every other line. It is also the line you cannot cut, because the QLoRA result holds: small and high-quality beats large and sloppy. Our complete guide to LLM fine-tuning best practices covers example formats, review workflows and dataset versioning in detail.

Evaluation. Before and after every run you need a fixed set of test cases drawn from real work, scored against thresholds you chose in advance. Building 300 to 500 golden cases with reviewed answers is expert time again, and if you use an LLM-as-a-judge you must calibrate it against 150 to 250 human-labeled outputs and document the agreement rate, because the judge is itself a model your validator will ask about.

Independent validation. Under the Federal Reserve and OCC's SR 11-7 model risk guidance, a system that processes inputs into estimates used for business decisions is a model, and models require effective challenge: critical review by parties independent of development. Your build team cannot validate its own fine-tune. Budget for a separate validation function or external validators, and for the documentation they need: training data lineage, evaluation results, limitations, and monitoring plans. This is people cost, measured in weeks, not GPU cost, measured in dollars.

Serving. A trained model that is not deployed is a paperweight. A quantized 70B model serves from one H100; at Lambda's $3.99 per hour on-demand that is roughly $2,900 per month for one always-on GPU, before redundancy. Reserved capacity or smaller models cut this substantially, and a well-tuned 8B to 13B model serving on a single A100 40GB at $1.99 per hour comes to about $1,450 per month. Serving, not training, is the recurring compute cost.

Governance and maintenance. Model cards, inventory registration, drift monitoring, and re-running your evaluation suite on every adapter, prompt or base-model change. When the behavior spec changes, you retrain; when only facts change, you should be re-indexing a retrieval layer instead, which is the architecture point our decision guide makes in detail.

A worked budget: claims triage at a mid-size insurer

Take an insurer that wants a model to classify inbound claims into a fixed taxonomy, extract eight fields into a strict JSON schema, and draft acknowledgement letters in house style. Facts stay in a retrieval layer; the fine-tune owns behavior only. Here is an honest first-year budget:

• Training data: 4,000 reviewed examples across the three tasks, about 800 hours of claims-expert and operations time. Dominant cost, internal.
• Compute for iteration: 20 QLoRA runs on a rented H100 at $20 to $40 each, $400 to $800 total.
• Evaluation harness: 400 golden cases plus a 200-output judge calibration study, roughly 150 to 250 hours split between operations staff and an ML engineer.
• Independent validation: four to eight weeks of validator time depending on scope and how clean your documentation is.
• Serving: one H100 for a quantized 70B at about $2,900 per month on-demand, or about $1,450 per month for a 13B on an A100, so $17,000 to $35,000 per year.
• Documentation: model card, validation report, monitoring plan, SR 11-7 mapping, 60 to 120 hours.

The pattern matters more than the totals. GPU rental for training rounds to zero. Serving is the largest check written to anyone outside. Everything else is your own people's time, and most of it is data and evidence, not machine learning. For the surrounding workflow decisions, including where human review sits in claims triage, see our post on LLM claims triage for insurers: accuracy, human review and cost.

When fine-tuning is the wrong choice

Spend nothing on fine-tuning if any of these describe your situation:

• The model lacks knowledge, not behavior. If answers are wrong because the model has not seen your current policy wordings, that is a retrieval problem. Fine-tuned facts are hard to update, impossible to cite, and stale the day a policy changes.
• You have not exhausted prompting. Few-shot examples and structured prompts solve more format problems than teams expect, at none of the fixed cost. Fine-tune when prompting has measurably plateaued on your evaluation set, not when it feels insufficient.
• You cannot produce the examples. No reviewed training data means no fine-tune worth shipping. Synthetic data helps at the margins but does not replace expert-reviewed examples for regulated behavior.
• Volume is low. Fine-tuning pays back through consistency at scale. Fifty claims a day does not justify the fixed costs; five thousand a day does.
• The use is customer-affecting and you have no validation capacity. In EU AI Act Annex III territory such as creditworthiness assessment or life and health insurance pricing, an unvalidated fine-tune is a liability, not an asset.

Common mistakes that inflate the bill

• Full fine-tuning by default. Full fine-tunes cost 20 to 80 times more per run than QLoRA and, per both papers, rarely beat adapters for task adaptation. Start with QLoRA; escalate only on evidence.
• Renting hyperscaler nodes for single-GPU jobs. An 8-GPU node billed as a unit turns a $10 job into a $229 one. Match the instance to the VRAM math, not the other way round.
• Big datasets of unreviewed examples. Volume without review trains inconsistency in. Halve the dataset, double the review.
• Skipping the baseline. Always score a prompt-only baseline on your golden set before any training run. A depressing share of proposed fine-tunes die right there, which is the cheapest good outcome available.
• Forgetting the license. Meta's Llama community license is royalty-free for use and modification, but redistribution carries attribution duties, and models trained on Llama outputs must carry Llama in the name. Have counsel read it before distribution, not after.

Frequently asked questions

How much does it cost to fine-tune a 70B model once? About $20 to $40 in rented GPU time for a QLoRA run on roughly 25,000 examples, $81 to $142 for full-precision LoRA, and around $1,800 for a full fine-tune. The training run is never the expensive part; data preparation and validation are.

Is API fine-tuning cheaper than renting GPUs? Per run, hosted per-token pricing is typically 7 to 9 times the rental cost for the same job, and it scales linearly with every retrain. For banks the deciding factor is usually not price but that hosted fine-tuning moves training data outside your perimeter.

How many training examples do we need? For behavior tasks like classification, extraction and house style, 1,000 to 5,000 expert-reviewed examples is the working range. The QLoRA research showed small high-quality datasets beating larger noisy ones, so review effort matters more than count.

Do we need to retrain when our policies change? No, and if you are retraining to keep facts current, the architecture is wrong. Facts belong in a retrieval index you update daily. Retrain only when the behavior spec changes or evaluation scores drift, typically quarterly at most.

Will a fine-tuned open-weight model satisfy our regulator? The model itself satisfies no one; the evidence around it does. Under SR 11-7 style model risk expectations you need documented development, independent validation, a measured error or hallucination rate, and monitoring. Those are budget lines in this post precisely because examiners ask for them.

Should we fine-tune a small model or prompt a big one? For narrow high-volume tasks, a fine-tuned 8B to 13B model usually wins on both cost and consistency, and serves from one modest GPU. Prompting a frontier model wins on open-ended reasoning and low volumes. Many production systems use both: a tuned small model for routing and extraction, a large model with retrieval for the hard questions.

Where to go from here

The short version: training compute is nearly free, serving is the recurring check, and data plus validation is where the budget and the risk actually live. Plan those three honestly and a fine-tune is a routine project; plan only the GPU line and it is an expensive surprise. Our enterprise AI service for banks and insurers covers private fine-tuning of open-weight models inside your VPC, the evaluation and validation evidence around it, and the governance documentation your model risk team needs, and our Claude for financial services practice handles the workloads where a managed frontier model is the better answer. If you want a budget built on your actual volumes and data, that is a conversation we have every week.