AI ROI is the question every carrier's board is now asking, and most carriers cannot answer it. Sixty-three percent of insurance companies have operationalized AI. They have models in production across underwriting, claims, customer service, and fraud detection. They committed the budget, hired the data scientists, and built the infrastructure. Most of them still cannot demonstrate a positive return.
Only 22% of carriers have AI in full production at scale. The rest operate in limited deployment, pilot programs, or single-use-case implementations where ROI timelines stretch to 2028 and beyond.
Weak metrics make ROI invisible, which is a separate problem from weak ROI. The AI might be delivering substantial value that the carrier simply cannot see through the activity numbers it reports. Meanwhile, the spend side of the ledger is often just as murky: seat licenses nobody opens, frontier models doing work a cheaper model would handle, and invoices that arrive a month after the money is gone.
This guide covers both halves of the problem: how to measure AI ROI so the value shows up, and how to manage AI cost so the denominator stays honest.
Why AI ROI is so hard for carriers to prove
The headlines do not help. 95% of AI projects fail to reach production, according to an MIT report covered by Fortune. At the same time, G2's 2025 data shows 57% of organizations already have AI agents in production. And McKinsey's survey shows 80% of organizations report no meaningful bottom-line impact from AI.
All three can be true at once. The 95% figure examines custom generative AI programs: internal builds, limited scope, high complexity. The 57% are running purpose-built, vendor-supported agents. The 80% are largely measuring what is easy to count (adoption, usage, tickets handled) rather than what moved the P&L. Adoption is outpacing instrumentation.
Carriers know how to measure ROI on a traditional technology purchase. Buy a new policy administration system, measure processing speed before and after, calculate headcount impact, track error rate reduction. The investment is discrete, the benefits are localized, and the timeline is predictable. AI breaks every one of those assumptions.
Benefits spread across departments. A claims triage AI reduces cycle time in claims, but it also reduces customer churn (a marketing metric), lowers litigation exposure (a legal metric), and improves reserve accuracy (an actuarial metric). The claims team reports a 15% cycle time improvement. Retention impact sits in a different dashboard owned by a different team. Litigation reduction appears in legal's annual report three years later. The total return is real, and no one owns the measurement.
Costs are front-loaded. Data preparation, model development, infrastructure buildout, integration engineering, and team training consume the majority of total project spend before the system produces its first output. A traditional ROI calculation comparing annual cost to annual benefit shows negative returns for the first 12 to 18 months by design, even when the project is on track.
Attribution resists isolation. An underwriting AI improves risk selection accuracy by 8%. Premium adequacy improves and loss ratios decline. In the same period, the carrier also tightened underwriting guidelines, hired two experienced underwriters, and exited three unprofitable geographic markets. Separating the AI's contribution from those concurrent changes requires holdout testing that most carriers are not set up to run.
None of this means the AI is failing. The carrier just cannot tell whether it is succeeding, and that problem has its own fix. It also gets more urgent over time, because boards and CFOs have finite patience for investments that cannot show returns, whether those returns are real but unmeasured or simply absent.
Activity metrics vs. value metrics
The numbers insurance AI teams report to leadership mostly measure activity. That is the core of the measurement problem.
Deflection rate. A customer service AI deflects 40% of inbound contacts, and the team reports success. Deflection does not measure resolution. If 30% of deflected contacts call back within 48 hours because the AI did not solve their problem, the true resolution rate is closer to 28%. The carrier cut contact center volume by 40% on paper while creating repeat contacts and a worse policyholder experience.
The same math applies at higher volumes. If an AI deflects 60% of volume but only 75% of those deflections are genuine resolutions, the effective automation rate is 45%. Savings projections should use the lower number. A 60% deflection rate can mean 60% of customers got help, or it can mean 60% gave up, received wrong information, or walked away. The number looks identical in both cases. Industry analysts draw the line between containment (the bot fully resolved the conversation) and deflection (a potential agent escalation was avoided). Only the first is value. See AI customer service metrics for the measures that hold up.
Hallucinations make deflection worse, because a confidently wrong answer still closes the ticket. Stanford University researchers found that general-purpose LLMs hallucinated in 58–82% of legal queries, and even domain-specific tools produced hallucinations in 17–34% of cases. Every one of those can register as a successful deflection.
Model accuracy. A fraud detection model achieves 94% accuracy. If its false positives generate 200 manual reviews per week consuming 50 hours of investigator time, the cost of those reviews may offset the savings from detected fraud. The metric that matters is net fraud loss reduction after investigation costs.
Automation rate. This tells you how many tasks AI handles without a human. It does not tell you whether those tasks were handled correctly, whether they are the tasks that generate the most value, or whether automation pushed exception handling onto another team.
A carrier reporting 40% deflection, 94% accuracy, and 60% automation can look successful on every dashboard and still deliver negative ROI. It could also be delivering excellent ROI. Those metrics cannot tell the two apart.
Accuracy is a probability, and scale multiplies it
Accuracy figures deserve a second look for another reason. AI systems are probabilistic: run the same prompt twice and you may get different answers. A 93% accurate agent will complete roughly 93 of 100 tasks within acceptable parameters. Seven will not, and you do not know in advance which seven.
At carrier scale, small percentages turn into large numbers. A carrier processing 10 million claims annually at 93% accuracy is looking at 700,000 failed claims. At 99%, that is still 100,000 errors. At 99.9%, which is very hard to reach, it is 10,000. Each of those errors carries rework, leakage, or complaint cost that belongs in the ROI calculation. An honest model plans for the error percentage, routes it to humans, and prices it in.
How to measure AI ROI: a six-step framework
Measuring AI ROI comes down to connecting system performance to a specific business outcome, with the full cost in the denominator. This is the framework we use with carriers.
Step 1: Find out which tasks actually produce value
Start with an inventory. Which AI tools are in use, by whom, for what work, and how often? Most carriers discover usage they did not sanction (see how to detect shadow AI) alongside paid tools that sit idle. Usage data by user, team, and task is the raw material for every later step, because it shows where AI is doing real work and where it is a line item.
Map each AI decision point to a business metric you already act on: loss ratio, cycle time, expense ratio, retention, complaint volume. If an AI use case does not connect to any metric someone owns, that is a finding in itself.
Step 2: Baseline the process before the AI
You cannot show improvement without a before. Capture cycle time, cost per transaction, error or reopen rate, and quality scores for the human process, ideally over the same season and mix of business the AI will handle. Where possible, keep a holdout group running the old process so you can compare like with like.
Step 3: Pair every speed or volume metric with a quality metric
Speed without quality is cost-shifting. The three pairings that hold up best for carriers:
- Cycle time with maintained accuracy. A claims AI that cuts average cycle time from 14 days to 5 while holding settlement accuracy demonstrates clear value. One that cuts cycle time to 5 days but increases reopened claims by 20% has moved cost from one line to another. The cycle time number is identical in both cases.
- Unit economics. Divide the full cost of the AI system (infrastructure, licensing, maintenance, supervision) across actual transaction volume. If a customer service AI costs $500,000 annually and handles 200,000 interactions, the per-interaction cost is $2.50. If a human interaction cost $8, the net savings are $5.50 per interaction, multiplied by volume. A CFO can evaluate that.
- Revenue attribution. Does AI-assisted underwriting produce a better book? Compare loss ratios, premium adequacy, and retention for AI-underwritten policies against a comparison group over a meaningful period (a minimum of 12 months for most lines). It takes patience, and it produces the most defensible ROI evidence available.
Step 4: Count the full cost
Most AI ROI models count two of the four cost layers.
| Cost layer | What it includes | How it usually goes |
|---|---|---|
| Platform | Licensing, per-conversation fees, API and token costs, infrastructure | Budgeted and negotiated; typically 20–30% of total cost of ownership |
| Integration and setup | Development, knowledge base preparation, workflow configuration, testing, training | Anticipated but underestimated by 40–60%; six-month timelines that run to twelve are common |
| Governance and supervision | Pre-deployment evaluation, monitoring, compliance documentation, human review workflows, knowledge base upkeep | Frequently left out entirely |
| Incidents | Remediation, legal exposure, brand damage, churn, engineering time pulled to firefight | Ignored until it happens |
Skipping the third layer does not save money. It converts a predictable infrastructure cost into an unpredictable incident cost. Customers who receive wrong information leave at 3–5x the normal rate, and a single significant hallucination incident in a regulated line can cost more than a full year of governance infrastructure. The organizations that budget for layer three tend to avoid layer four.
Step 5: Track refusals, escalations, and error rates by root cause
If you track one operational metric beyond the business outcome, make it refusal and escalation patterns with root cause tagging. The question is why the AI handed off: ambiguous input, missing data, a policy edge case, low confidence on a valid request, or a guardrail doing its job.
"We handled 10,000 requests" is an activity number. "We refused 3,000 requests for reasons we could have fixed" is a roadmap. Each cluster of refusals is a training opportunity, a process gap, or a sign the AI is being asked to do something it should not. Set an explicit acceptable error rate per use case, and weigh the cost of a false positive against a false negative in that line of business.
Step 6: Report value on a schedule the board can follow
Roll the results into a quarterly view: verified outcomes by use case, total cost by layer, spend by department, error and escalation trends, and incidents caught before they reached a policyholder. Include risk avoidance, even though it is counterfactual. Every prevented incident has a calculable cost, and drift caught in week one costs a configuration change where drift caught in month six costs a crisis response.
Expect the year-one number to come in below vendor projections and the year-two number to come in above them. Quality compounds when supervision feeds failures back into evaluation: an AI agent that improves verified resolution by 5% per quarter delivers very different value in month eighteen than in month one.
AI cost management: keeping spend in line with value
The ROI conversation usually fixates on the numerator. For most carriers in 2026, the denominator is the easier win. AI spend arrives through seat licenses, API tokens, vendor features bundled into existing contracts, and tools employees expense on their own. Most carriers discover the total after the fact, through an invoice or a quarterly report. Good AI cost management means seeing spend as it happens and tying it to the work it bought.
Meter AI spend per user and department in real time
Token consumption and spend should be visible by user, team, department, task, and model while the month is still in progress. Real-time metering answers the questions finance will ask: who is using what, for which work, at what cost, and whether that cost tracks to the value measured in the framework above. It also exposes waste that monthly invoices hide, like a single workflow running a frontier model in a loop.
Set AI budgets and caps per user and team
An AI budget only works if it is enforced before the overspend happens. Set budgets per employee, team, and department, with alerts before thresholds are reached. When a cap is hit, usage should pause and route to an approved human who can review the context and decide whether to extend it. Every alert, override, and resolution should be logged, so spend decisions show up in governance reporting with the same audit trail as everything else.
Right-size the model to the task
Frontier models are priced for frontier work. Summarizing a first notice of loss, drafting a routine policyholder letter, or extracting fields from a dec page rarely needs the most expensive model available. Route each task to the smallest model that meets its quality bar, verified against your own evaluation set, and make switching models a configuration change rather than a procurement project. This is also where avoiding single-vendor lock-in pays off: if one model vendor holds your workflows and context, it also holds your pricing.
Pay for usage, not idle seats
Per-seat licensing charges for every provisioned user, whether or not they open the tool that month. Usage-based access through a governed gateway charges for the work actually done, and the metering data from the first step shows exactly which seats are earning their keep. For many carriers, reallocating spend from idle seats to metered access for the people who use AI daily is the fastest ROI improvement available. A private AI gateway also brings shadow AI usage inside the tent, where it can be metered and budgeted like everything else.
Watch for vendor changes that move your numbers
Your cost and quality baselines can shift without you touching anything. In one evaluation of customer service agents across multiple vendors, one vendor's performance dropped from 93% to 60% between testing cycles. Same prompts, same criteria. The vendor had released a model update, and its customers got no advance notice. A model update can change accuracy, token consumption, or both, so model drift monitoring belongs in your cost management practice as much as your quality practice.
Hidden costs that quietly erode AI ROI
Some AI costs never show up as a line item. They show up as rework.
The clearest data comes from software teams. CodeRabbit's analysis of 470 real-world GitHub pull requests found AI-generated code produces 1.7x more issues than human-written code. GitClear's study of 211 million changed lines found code cloning grew 4x, duplicated blocks increased 8x, and refactoring dropped from 25% of code changes to less than 10%. The Stack Overflow 2025 Developer Survey showed developer trust in AI accuracy fell from 40% to 29%, with two-thirds saying they spend more time fixing "almost right" AI output than they save.
Perceived productivity and measured productivity can diverge sharply. The METR study from July 2025 found experienced developers believed AI made them 20% faster, while a randomized trial showed they were actually 19% slower. The Google DORA 2024 report found increased AI adoption correlated with a 7.2% decrease in delivery stability. When 41% of all code is AI-generated and 48% of that code contains security vulnerabilities, the review burden is a real cost.
The same pattern applies to claims notes, underwriting memos, and policyholder correspondence. If an adjuster spends twenty minutes correcting a draft that took the AI ten seconds, the time saved is negative, and only measurement will show it. Reliability problems compound the issue. OpenAI's reasoning systems have shown hallucination rates reaching 33% for the o3 model and 48% for o4-mini, and research indicates incorrect decisions based on hallucinated AI outputs affected 38% of business executives in 2024. Regulatory readiness is a cost driver too: 55% of organizations report being unprepared for compliance requirements, which have become the top barrier to deployment.
Teams that define quality baselines, put automated checks on output, and monitor for drift avoid most of these regressions. The practical takeaway for an ROI model: count rework and review time, and measure output quality as closely as you measure usage.
Building a board-ready AI ROI case
The supervision layer that monitors AI in production also generates the operational data needed to prove ROI. Continuous monitoring produces performance metrics over time, which makes before-and-after comparisons, drift detection, and outcome tracking possible.
A carrier running a claims triage AI without supervision knows how many claims the model processed. A carrier running the same model with supervision knows how many it processed correctly, how accuracy trended over six months, which claim categories produce the highest error rates, and how the model compares to the human baseline. Add real-time spend data by department and the carrier can show cost per verified outcome, by use case, by quarter.
Pre-launch evaluation earns its cost quickly. In one embedded-insurance customer service deployment, evaluation caught fabricated dollar amounts, information merged across unrelated insurance products, and invented contact details before launch. Each would have counted as a successful deflection in the vendor dashboard. After correction, the deployment reached 60–70% automation with zero customer-facing hallucinations, validated against a test suite of several hundred real customer scenarios. That validation is what let the team cite the number to stakeholders and regulators with confidence.
A board-ready AI ROI package for a carrier typically includes:
- Verified outcomes per use case, each paired with a quality metric
- Total cost across all four layers, with spend by department and model
- Budget adherence, cap events, and how they were resolved
- Error, escalation, and refusal trends, with root causes
- Incidents caught before reaching policyholders, with estimated avoided cost
- The governance record behind it, mapped to the NAIC model bulletin
For the 63% of carriers who have operationalized AI, the technology mostly works. The models run and the systems process real decisions. What most carriers lack is outcome metrics, the operational data to populate them, and visibility into what they are spending. That gap closes by measuring outcomes, metering spend as it happens, and matching each task to the model it actually needs.
If you want to see what real-time metering, per-user and per-department budgets, and model right-sizing look like across every LLM in your environment, see how Swept handles AI cost optimization, or talk to us about the AI ROI questions your board is asking.