# How do I evaluate an AI tool before we buy it?

_Test the tool on your own use cases and data, and check its outputs for accuracy, hallucinations and bias. Confirm where your data goes and what the contract allows, make sure you can supervise the tool after go-live, and model the real cost. Then score every vendor on the same scorecard before the go/no-go call._

Test the tool on your own use cases and data, and check its outputs for accuracy, hallucinations and bias. Confirm where your data goes and what the contract allows, make sure you can supervise the tool after go-live, and model the real cost. Then score every vendor on the same scorecard before the go/no-go call.

If you'd rather not run it yourself, Swept is the trusted third party insurance carriers use to evaluate, supervise and certify AI. We run this whole evaluation at scale on your own data, then supervise the tool continuously after go-live. [Talk to us about an AI evaluation](https://www.swept.ai/contact).

**Fig. 1 · The evaluation flow**

- **01 · Test:** Test on your own data, in an isolated environment.
- **02 · Score:** Score each vendor on the same seven areas.
- **03 · Decide:** Make the go/no-go call from the weighted score.
- **04 · Supervise:** Keep checking it continuously after go-live.

A vendor demo shows what a tool does on the vendor's examples. Your evaluation has to show what it does on yours, in a way you can explain to a COO, a board and a regulator.

This guide works as an AI vendor due diligence checklist if you're doing it by hand, with a scorecard for up to three vendors. Swept runs the same evaluation at scale on your own data.

## What should an insurance carrier evaluate before buying an AI tool?

Evaluate seven areas: fit to the use case, output quality, bias and fairness, data handling and security, supervision and auditability, contract terms, and cost. Weight them by risk, so output quality and data handling count the most, since a failure in either one is hard to undo.

> **With Swept:** Swept evaluates every vendor on your shortlist the same way, on your own data, so the go/no-go call compares like with like.
>
> [Compare your shortlist with Swept](https://www.swept.ai/contact)

## How do I test an AI tool on our own data without exposing policyholder data?

Test in a private or isolated environment, on real cases from your own work, with personal data redacted wherever the task doesn't need it. Before any data moves, get these from each vendor:

- A private tenant, a dedicated test environment or a deployment inside your own cloud.
- A written commitment that test data won't be used for training and will be deleted after the test.
- A list of every system and subprocessor your test data will touch.

> **With Swept:** Swept runs the evaluation inside your environment, on your own cases, so policyholder data stays with you and your experts only review results.
>
> [See how Swept tests on your data](https://www.swept.ai/contact)

## How do I check an AI tool for hallucinations and bias?

Compare the tool's outputs on your own cases with answers your experts would stand behind, and check whether results differ across groups of policyholders or claimants. A hallucination is a confident statement the source documents don't support, so every claim in an output has to trace back to the file.

| The results should show | Why it matters |
|---|---|
| Accuracy on your own cases | A tool can shine on a demo and struggle with your forms and files. |
| No unsupported statements | One invented fact in a coverage letter can become a complaint or an exam finding. |
| Consistent answers | If the same file gets different answers, no single output can be trusted. |
| No unexplained gaps across groups | Differences by region, claim type or policy type need an explanation before go-live. |

> **With Swept:** This is the core of what Swept does: testing models and agents, including vendor AI, for accuracy, hallucinations and bias on your data, with results you can take to a board or an examiner.
>
> [Talk to us about an AI evaluation](https://www.swept.ai/contact)

## How many test cases do we need for a result we can trust?

Enough to reach the confidence level you choose: about 271 test cases per use case for 90% confidence, 385 for 95% and 664 for 99%, each within plus or minus 5 percentage points. A manual test with a few dozen cases is a sensible base level, but it can't tell you whether a 72% pass rate is really 72%.

> **With Swept:** Swept's [scoring methodology](/scoring-methodology) grades every test as a pass or fail against a known right answer, uses three AI judges from different model families, and reports confidence intervals. You pick the confidence level, 90, 95, 99 or higher, and we size the test to it.
>
> [Read Swept's scoring methodology](https://www.swept.ai/scoring-methodology)

## What should the contract say about our data and the model?

The contract should cover training on your data, retention, subprocessors, notice before model changes, audit rights and how you get your data back when you leave. Ask before the pilot, while you still have leverage.

| Clause | What to ask for |
|---|---|
| Training on your data | No training or fine-tuning on your inputs or outputs without written consent. |
| Retention | A set retention period and written confirmation of deletion. |
| Model version changes | Advance notice, and the right to test the new version first. |
| Audit rights | Audit rights or independent audit reports, and cooperation with regulators. |
| Exit | Your data and logs returned in a usable format, then deleted. |

The [NAIC Model Bulletin](https://content.naic.org/sites/default/files/inline-files/2023-12-4%20Model%20Bulletin_Adopted_0.pdf) expects this kind of due diligence on third-party AI systems, including audit rights where appropriate.

> **With Swept:** Swept's results give legal something concrete to negotiate from, and Swept certification gives your board and examiners a record that the tool was independently checked.
>
> [Ask about Swept certification](https://www.swept.ai/contact)

## How do I know it will stay accurate after we buy it?

Supervise it continuously against the standard you used to buy it, and test every new model version before it reaches production. Before you sign, make sure you'll have:

- Continuous supervision of quality, measured against your evaluation results.
- Notice before any model change, and a test against your standard before it goes live.
- Logs and an audit trail you could hand an examiner in a market conduct exam.

For a claims intake agent, see [how carriers monitor a claims intake AI agent](/monitor-claims-intake-ai-agent).

> **With Swept:** Swept supervises AI continuously on your own data and in your environment, pins model versions and tests any change against your baseline before it reaches production.
>
> [See how AI supervision works](https://www.swept.ai/ai-supervision)

## How do I estimate the real cost?

Estimate cost per use case from expected volume, compare usage pricing with seat pricing, and add the internal time to run and supervise the tool. Ask every vendor to price the same usage scenario in writing so the numbers line up.

> **With Swept:** After go-live, Swept tracks usage and budgets by department and use case, so you can compare real spend with the estimate you approved.
>
> [See budgets and usage tracking](https://www.swept.ai/contact)

## Who should be in the evaluation?

Include a business owner who owns the outcome, IT and security, compliance, legal or third-party risk, and a claims or underwriting expert who knows what a right answer looks like. One person can cover more than one role, but every role needs a name next to it.

> **With Swept:** For a small IT team, Swept joins as the independent tester and stays on as the supervisor, so nobody has to become an AI testing specialist overnight.
>
> [Bring Swept onto your evaluation team](https://www.swept.ai/contact)

## What goes on an AI evaluation scorecard?

An AI evaluation scorecard lists the seven areas with a weight for each, a score from 1 to 5, the evidence behind each score and a go/no-go rule. Here is ours, with what a 1, a 3 and a 5 look like in each area.

| Area | Weight | 1 looks like | 3 looks like | 5 looks like |
|---|---|---|---|---|
| Fit to the use case | 15 | Doesn't cover the core task, or needs major changes to how the team works. | Covers the core task, with workarounds the team can live with. | Covers the core task inside the existing workflow, with no workarounds. |
| Output quality | 20 | Frequent wrong or unsupported answers on your cases. | Mostly right, with errors on hard cases that reviewers catch. | Accurate on your cases, hard ones included, with no unsupported statements. |
| Bias and fairness | 10 | Unexplained gaps in outcomes across groups, or no way to test for them. | Small gaps the vendor can explain, with a plan to monitor them. | No material gaps, a documented testing method and ongoing checks. |
| Data handling and security | 20 | Data leaves your control, or may be used to train the vendor's models. | Private processing, with open questions on retention or subprocessors. | Runs privately or in your environment, no training on your data, clear retention and subprocessors, current security audits. |
| Supervision and auditability | 15 | No logs or version history you can get to. | Logs are available, but monitoring and change control are manual. | Full logs, continuous supervision, testing before model changes and an audit trail you can export. |
| Contract and vendor terms | 10 | Terms allow training on your data, or give no notice before model changes. | Most terms work, with open items legal is still negotiating. | No training on your data, set retention, notice before model changes, audit rights and a clean exit. |
| Cost and usage controls | 10 | You can't estimate the cost or cap it. | Cost is predictable, but the controls are coarse. | Clear cost per use case, with budgets, limits and usage reporting by team. |

_Fig. 2 The full scorecard adds Score, Weighted score, Evidence and Notes columns for up to three vendors._

Score each area from 1 to 5 and multiply by its weight. Add the seven weighted scores and divide by 100 to get a weighted average out of 5.

- **4.0 to 5.0 (Go):** A weighted average of 4.0 or more is a go.
- **3.0 to 3.9 (Conditional go):** 3.0 to 3.9 is a conditional go, with a written fix plan and an owner.
- **Below 3.0 (No-go):** Below 3.0 is a no-go.
- **Any 1 in a critical area (No-go):** A score of 1 in Output quality or Data handling is a no-go, whatever the total.

**Get the AI evaluation scorecard.** A printable PDF with the seven weighted areas and the go/no-go rule, with room for three vendors side by side.

- Seven areas, weighted to 100
- What a 1, a 3 and a 5 look like in each area
- Evidence and notes columns for your file
- The go/no-go rule for the final call

[Get the scorecard](https://www.swept.ai/resources/download/ai-evaluation-scorecard)

## Can Swept run the AI evaluation for us?

Yes. Swept is the trusted third party carriers use to evaluate, supervise and certify AI. Your team keeps the decision, and we do the testing, the comparison and the supervision after go-live.

| | By hand | With Swept |
|---|---|---|
| Test cases | A few dozen, built by your experts. | Sized to the confidence level you choose. |
| Hallucinations and bias | Spot checks, within the time you have. | Tested at scale, including vendor AI. |
| Comparing vendors | Different demos and different numbers. | Every vendor tested the same way. |
| After go-live | Re-checks when someone has time. | Continuous supervision, with model changes tested first. |
| Evidence for examiners | Assembled by hand. | Logs, an audit trail and certification. |

Start with one use case and a shortlist of two or three vendors. Tell us what you're evaluating and we'll show you what Swept's evaluation looks like on your data.

[Talk to us about an AI evaluation](https://www.swept.ai/contact) or [get the scorecard](https://www.swept.ai/resources/download/ai-evaluation-scorecard).

## What else do carriers ask about evaluating AI tools?

### How long should an AI vendor evaluation take?

Plan for four to eight weeks for one use case and two or three vendors. Contract terms often take the longest, so start them in the first week.

### Can we evaluate the AI built into our core system vendor's software?

Yes, because it touches the same policyholder data as any standalone tool, so ask for the same evidence: test access on your data, where the data goes and how model changes are handled. See our vendor AI governance and evaluation guides.

### What does the NAIC model bulletin expect for third-party AI systems?

The [NAIC Model Bulletin](https://content.naic.org/sites/default/files/inline-files/2023-12-4%20Model%20Bulletin_Adopted_0.pdf), adopted in December 2023, expects due diligence on AI vendors and their data and models, audit rights or audit reports where appropriate, and vendor cooperation with regulators. It also expects testing and validation of AI systems whether you built them or bought them.

### Should we run a pilot before we buy?

Yes, run a short pilot with real users once the tool passes testing on your data. Agree on exit criteria up front, and make sure the contract lets you walk away if it fails.

### What score should a tool hit to get a yes?

A weighted average of 4.0 or more out of 5 is a go, 3.0 to 3.9 is a conditional go with a written fix plan and an owner, and below 3.0 is a no-go. A score of 1 in Output quality or Data handling is a no-go whatever the total.