AI for AML Alert Triage in Banks: Workflow, Costs, Limits
How banks can use LLMs to clear the 85 to 95 percent of AML alerts that are false positives: the pipeline architecture, per-alert cost math, regulator expectations, and which decisions must stay human.

Banks drown in AML alerts. Between 85 and 95 percent of transaction monitoring and screening alerts are false positives, and each one still costs an analyst 20 to 30 minutes to clear. Large language models can absorb most of that first-line review, but only if you build the pipeline so that every automated decision is evidenced, auditable, and reversible. This post lays out the workflow, the real costs, and the hard line between what you can automate and what must stay human.
The alert problem in numbers
Start with the scale of the problem, because it explains why triage is the right target.
Industry benchmarks put the false positive rate for rules-based transaction monitoring and sanctions screening between 85 and 95 percent. Vendors and consultancies working in this space consistently report figures at or above 90 percent. That means for every 100 alerts your system generates, an analyst reads 90 or more that turn out to be nothing.
The downstream volume is enormous. FinCEN received 4.7 million Suspicious Activity Reports in fiscal year 2024, an average of 12,870 SARs per day. Every one of those SARs started as an alert a human worked.
And the cost of getting the plumbing wrong is public. In September 2024 the FCA fined Starling Bank £28,959,426 for financial crime control failures. The final notice is worth reading in full, because it shows what a broken screening and triage stack actually looks like:
- A misconfiguration meant that from July 2017, customers were screened only against individuals on the consolidated sanctions list with UK citizenship or UK residency: 39 of the 3,088 designated persons at the time.
- When the bank re-screened its back book of 3.5 million customers over two weeks in February 2023, it generated around 48,000 alerts for the financial crime team to review.
- A retrospective review of 3.99 million payments generated 795,712 alerts.
- The FCA found no formal testing or calibration methodology for the screening system, and no management information on alert volumes.
Two lessons matter here. First, alert volume is not an abstraction; it is an operational liability that can explode by orders of magnitude when you fix a defect upstream. Second, regulators now read your alert handling as evidence of whether your whole control framework works.
What an LLM triage pipeline looks like
The naive version of this idea is: paste the alert into a chatbot, ask if it looks suspicious, and auto-close the ones it clears. Do not build that. It produces unevidenced, unrepeatable decisions, and it will not survive your first internal audit, let alone a regulator.
The version that works looks like a research pipeline with strict contracts, not a chatbot. A useful reference architecture comes from a recent paper on explainable AML triage with large language models, which tested exactly this setup against a rules baseline. The design has four parts:
1. Evidence bundles, not raw alerts. Before the model sees anything, a retrieval layer assembles a bundle: the triggering rule and why it fired, customer KYC context, transaction history and counterparty subgraphs, and the relevant passages from your own AML policy. The model never goes hunting on its own; it reasons over a fixed, logged bundle.
2. A structured output contract. The model must return a decision (clear, escalate, needs more evidence), a rationale, and citations pointing at specific items in the evidence bundle. Free-text answers are rejected by the schema.
3. A verifier pass. A second, cheap step checks the output: do the citations actually exist in the bundle, are the numbers in the rationale consistent with the data, does the decision follow the policy. Failures route to a repair loop, then to a human.
4. Counterfactual checks. For escalations, the system perturbs the key evidence (for example, removes the single transaction the rationale leans on) and confirms the decision flips or weakens. This catches rationales that sound right but do not depend on the evidence.
The measured difference between this and simpler approaches is significant. In that study, the rules baseline caught most genuinely suspicious activity but at very low precision, which is the false positive burden in statistical form. The full pipeline reached an Escalate F1 of 0.62, with citation validity of 0.98, meaning the model cited real evidence 98 percent of the time. It is not perfect, and the authors are honest about that. It is dramatically better than asking a model to eyeball alerts.
What to automate and what must stay human
This is the section to print out and pin above the project plan.
Automate these:
- Evidence assembly. Pulling KYC records, transaction histories, counterparty data, prior alerts, and adverse media into one bundle. This is slow, mechanical, and error-prone when humans do it under time pressure.
- First-pass disposition of clear false positives. Alerts where the evidence bundle fully explains the activity (a known payroll pattern, a counterparty that matched a screening list on a common name with a different date of birth) can be closed automatically, with the rationale and evidence logged.
- Draft rationales and narratives. The model writes the first draft of the closure rationale or the escalation summary. Humans edit and sign.
- Adverse media triage. Screening thousands of articles for actual relevance is one of the strongest LLM use cases in the stack. One worked example from a payments company describes 5,000 to 10,000 articles a day landing in manual review queues; relevance filtering is exactly what these models do well.
- Quality assurance sampling. Run the verifier over a sample of human-cleared alerts too. Models are tireless second readers.
Keep these human:
- The SAR decision. Filing a suspicious activity report is a legal judgment with personal accountability attached. Under US BSA rules the filing clock runs 30 calendar days from initial detection, 60 if no suspect is identified. The model can assemble the facts and draft the narrative; a named, accountable human decides and signs.
- Escalation thresholds and policy. Humans set what counts as suspicious. The model applies the policy; it does not write it.
- Any customer-facing or reporting action. Account freezes, exits, and regulator notifications need human sign-off, full stop.
- Model and rule changes. Tuning the transaction monitoring rules, changing the LLM prompts, or swapping models is a governed change process, not something the pipeline does to itself.
A useful framing for your risk committee: the LLM is a very fast, very consistent L1 analyst that writes everything down. You would never let an L1 analyst file SARs or freeze accounts unsupervised. Same rule.
Worked example: a mid-size bank
Take a mid-size bank generating 1,000 alerts a day: 800 from transaction monitoring, 200 from sanctions and PEP screening. Assumptions are stated so you can swap in your own numbers.
The manual baseline. At 25 minutes per alert, the middle of the 20 to 30 minute range reported for L1 review, 1,000 alerts is 25,000 analyst-minutes a day. That is 417 analyst-hours, or roughly 60 analysts working seven productive hours a day, doing nothing but clearing alerts. Most banks this size cannot staff that, so the real baseline is a permanent backlog, rushed closures, and aged alerts.
The LLM pipeline cost. Per alert, assume a 30,000-token evidence bundle in and a 2,000-token structured decision out. At Anthropic API prices for Haiku 5.5 ($0.10 per million input tokens, $0.50 per million output):
- First pass on all 1,000 alerts with a fast model: about $0.004 per alert, $4 a day.
- Full pipeline (stronger model, verifier, one repair loop, roughly 3x the tokens) on the 15 percent of alerts that are genuinely ambiguous, at Sonnet 5.5 prices ($2 in, $10 out): about $0.24 per alert, $36 a day.
- Total: roughly $40 a day, around $1,200 a month in API spend. Overnight batch processing, which cuts token costs 50 percent, fits naturally since most alert queues are not real-time.
Against 60 FTEs of fully loaded analyst cost, the API bill is noise. The real cost of this project is not tokens; it is the integration work, the evaluation harness, and the governance. Budget for those, not for the model.
The outcome to target. If the pipeline auto-clears 80 percent of alerts with documented rationales, the human workload drops from 417 hours a day to about 83, and those hours go to the alerts that actually deserve them. Vendor benchmarks in this space report review-time reductions in the 60 to 80 percent range; treat 80 percent auto-clear as the optimistic end and pilot before you believe your own number.
What regulators expect
Regulators are not blocking this. They have been encouraging better tooling for years, with conditions.
The FATF's 2021 report on new technologies for AML/CFT concluded that new technology can make AML controls faster, cheaper, and more effective, and then listed the obstacles it sees in practice: difficulty explaining and interpreting model outputs, dependence on data quality, the cost of replacing legacy systems, skills gaps, and regulatory hesitation itself, since institutions fear that piloting new systems will expose weaknesses in the old ones.
Read the FATF obstacle list as your governance requirements:
- Explainability is not optional. Every automated disposition needs a rationale a supervisor can follow, tied to evidence. This is why the citation contract in the architecture matters more than the choice of model.
- Data quality is your problem, not the model's. Garbage KYC data produces garbage bundles. Expect the project to surface data defects; that is a benefit, but plan remediation time.
- You are accountable for the model. An LLM used in a control function is a model under your model risk framework. That means documentation, validation, monitoring, and change control. We have written a working guide to LLM model risk documentation for banks and a companion piece on building an evaluation suite that covers the mechanics.
The Starling notice adds one more expectation: management information. The FCA criticized the absence of MI on alert volumes. If you automate triage, your dashboards on volumes, auto-clear rates, verifier failure rates, and override rates are part of the control, not a nice extra. Our guide to LLM monitoring and audit logging for banks and insurers covers what that logging layer needs to capture.
Where this fails
Honest limits, because there are several.
- Data quality is the ceiling. If your KYC records are stale or your transaction data has missing counterparty fields, the evidence bundles are weak and the model's rationales will be weak. No prompt fixes bad data.
- False negatives are the dangerous failure mode. A false positive costs 25 minutes. A false negative, an auto-cleared alert that was real suspicious activity, is a regulatory event. This is why you tune for recall on the auto-clear boundary, run QA sampling on closed alerts, and keep the auto-clear rule conservative. Vendors talk about false positive reduction because it is the easy win; your risk function should obsess over the other side.
- Distribution shift is constant. Typologies change, sanctions lists update weekly, and your own product mix drifts. A pipeline validated in January can degrade by June without anyone touching it. Continuous evaluation, not a one-off validation, is the requirement.
- Token costs grow with diligence. The cost math above assumes disciplined routing: cheap model first, expensive pipeline only for the ambiguous slice. Send every alert through the full pipeline with multiple repair loops and the bill multiplies fast. One vendor working in this space flags token cost as a genuine operational consideration at scale, which matches the math.
- It will not fix a broken rule set. If your monitoring rules are misconfigured, the LLM will diligently triage garbage. Fix detection first; the Starling case is the cautionary tale.
Common mistakes
Teams piloting LLM triage keep making the same errors:
- Starting with SAR writing. It feels like the highest-value target and it is the highest-risk one. Start with evidence assembly and false-positive clearing, earn trust, then move up the stack.
- No output contract. Accepting free-text rationales means you cannot verify citations, cannot measure quality, and cannot audit. Schema first.
- Skipping the verifier. The single-pass chatbot design fails silently. The verifier pass is cheap and it is what makes the system auditable.
- Measuring only precision. If you optimize for cleared-alert accuracy you can end up clearing real suspicious activity. Track recall on known-bad cases, run red-team exercises against the pipeline, and sample auto-closed alerts for human QA. Our guide on how to red-team an LLM before a bank or insurer deploys it covers the adversarial side.
- Piloting without baseline MI. If you do not know your current alert volumes, closure times, and false positive rate, you cannot prove the system helped. Instrument first.
How to implement it
A pragmatic sequence, eight steps:
- Instrument the baseline. Four weeks of data: alert volumes by type, time-to-close, false positive rate from QA samples, backlog age. You need this for the business case and for later validation.
- Fix the worst detection defects. If any rule or list configuration is known-broken, fix it before you automate triage of its output.
- Build the evidence assembler. The retrieval layer that bundles KYC, transactions, history, and policy excerpts for each alert. This is most of the engineering work and it is useful for human analysts even before any model touches it.
- Define the output schema. Decision, rationale, citations, confidence, needs-more-evidence flag. Get your compliance officers to sign off on the fields, since they will live with them.
- Shadow mode. Run the pipeline alongside humans for at least a full monthly cycle. The model dispositions everything; humans still decide. Measure agreement, citation validity, and where they diverge.
- Validate like a model. Backtesting, sensitivity analysis, documented limitations, approval through your model risk process.
- Go live narrowly. Auto-clear only one alert class, only below a conservative risk threshold, with daily QA sampling. Expand by class and threshold as the evidence accumulates.
- Monitor forever. Weekly dashboards on volumes, auto-clear rate, verifier failures, overrides, and drift. Quarterly re-validation. This is an enterprise AI governance function, not a project that ends.
Frequently asked questions
Can an LLM close AML alerts without any human review? For a narrow, conservative slice of obvious false positives, yes, with full evidence logging and QA sampling. For anything that could be real suspicious activity, no: the escalation and SAR decision stay with an accountable human. Regulators care about who is accountable, and "the model" is not an acceptable answer.
What false positive rate should we expect from rules-based monitoring? Published benchmarks cluster between 85 and 95 percent. If your measured rate is far below that, check whether your QA sampling is actually catching misclassified alerts before celebrating.
Does using AI in triage create model risk obligations? Yes. A model used inside a control function falls under your model risk management framework: documentation, independent validation, ongoing monitoring, and change control. Supervisors will ask for all four.
How much does LLM-based triage cost per alert? With a cheap-model-first design, roughly half a cent per alert for the first pass and about 25 cents for the small share that needs the full pipeline, at current API prices. The real project cost is integration, evaluation, and governance, not tokens.
Can AI write SAR narratives? It can draft them from the evidence bundle, and this genuinely saves time. A human investigator must verify every fact, own the judgment, and sign the filing. Treat the draft as a starting document, never a finished one.
What do regulators say about AI in AML? FATF has publicly encouraged new technology for AML since 2021, while listing explainability, data quality, and governance as the conditions. Supervisors like the FCA judge you on outcomes and controls: alert handling, MI, testing methodology. AI that improves those with evidence is defensible; AI that obscures them is not.
Where to go from here
The pattern that works is consistent: assemble evidence mechanically, let the model reason over a fixed bundle with citations, verify everything, and keep humans on every decision with legal weight. Start by instrumenting your current alert queue, then run a shadow pilot on one alert class. If you want a second pair of eyes on the architecture, the evaluation harness, or the governance documentation, our Claude for financial services team works on exactly these systems with banks, fund administrators, and insurers.
About AI Agents Plus Editorial
AI automation expert and thought leader in business transformation through artificial intelligence.



