The NAIC AI Bulletin: What It Requires, How the AI Evaluation Tool Tests It, and What Carriers Need on File

The NAIC AI bulletin started in December 2023 as a set of principles: fairness, transparency, accountability, risk management. For an industry where most AI was still in pilot, that level of abstraction made sense. Nearly three years later, the NAIC model bulletin is the baseline examiners in about half the country work from, and the NAIC AI Systems Evaluation Tool has given them a standard set of questions to test it with. Regulators have moved from asking whether a carrier has an AI policy to asking for the evidence that policy was supposed to generate.

This guide covers what the bulletin requires, where it has been adopted, what the 12-state evaluation tool pilot asked carriers, why the industry pushed back, how the federal executive order on AI collides with state authority, and what a carrier should have on file before an examiner asks.

What the NAIC AI bulletin requires

The NAIC's Model Bulletin, formally titled "Use of Artificial Intelligence Systems by Insurers," was adopted on December 4, 2023. It creates no new law. It tells carriers that the statutes already on the books (unfair trade practices, unfair discrimination, rate adequacy, corporate governance, market conduct) apply to every decision an AI system touches, regardless of what technology produced it.

The bulletin runs in four sections:

1. Introduction, background, and legislative authority. Grounds the bulletin in laws the department already enforces.
2. Definitions. Terms that carry legal weight, including AI System, Adverse Consumer Outcome, Predictive Model, and Model Drift.
3. Regulatory guidance and expectations. Requires every carrier to maintain a written AIS Program covering governance, risk management, and third-party oversight.
4. Regulatory oversight and examination considerations. Lists the documentation a department can request in an investigation or market conduct exam. This is the section that turns the program from policy into evidence.

The AIS Program

The core obligation is a written AI Systems (AIS) Program governing how the carrier develops, buys, and uses AI. In practice it has four working pillars:

Proportionality

The bulletin does not demand the same rigor for every model. The strength of the safeguards should scale with five factors: the nature of the decision the AI supports, the degree of potential harm to the consumer, how much a human is involved in the final decision, how transparent and explainable the system is, and how much the carrier relies on third-party data and models. A claims-denial model and an internal document-search tool do not carry the same burden. The proportionality argument only works, though, if the carrier has written down which systems sit where and why.

Section 4: what an examiner can ask for

Section 4 is the part to treat as a standing requirement. It lets a department request the written program, evidence of how it was tailored to the carrier's use of AI, model documentation, data and technique records, and validation, testing, and audit results, including drift evaluation. A carrier that keeps those artifacts current can answer an examination letter with an export. A carrier that doesn't will be reconstructing them under deadline.

Where states have adopted the NAIC model bulletin

As of the NAIC Big Data and Artificial Intelligence (H) Working Group's implementation map on April 1, 2026, 25 jurisdictions (24 states plus the District of Columbia) had adopted the model bulletin. Four more run insurance-specific AI guidance of their own instead: California (Bulletin 2022-5), Colorado (SB 21-169 and Regulation 10-1-1), New York (Circular Letter No. 7, 2024), and Texas, which issued Commissioner's Bulletin B-0003-26 on June 12, 2026.

Adoption keeps expanding, and adopting states do not always copy the model word for word. Our state-by-state insurance AI governance tracker lists each state's bulletin citation and effective date, with dedicated pages for the states that diverge most, such as Colorado's ECDIS regime. For the broader legislative picture beyond insurance departments, see our guide to state AI regulations.

From principles to examination procedures

The original bulletin did not prescribe specific documentation. Then AI moved into production across underwriting, claims, pricing, fraud detection, and customer engagement faster than governance kept up. When regulators asked how carriers' AI policies translated into operating safeguards, the answer was often a hastily assembled binder.

The NAIC's response was institutional: a working group coordinating approaches across states, an evaluation tool giving examiners a standardized assessment framework, and examination procedures that ask for evidence rather than statements.

The cybersecurity lens

Regulators are evaluating AI governance through the lens they use for cybersecurity and operational risk: data protection, access management, change management, incident response, vendor oversight. A carrier with a mature cybersecurity program already has most of the organizational muscle; extending it to AI systems is incremental work. Carriers without that foundation have to build both at once, and a carrier that cannot demonstrate cybersecurity maturity is unlikely to convince an examiner its AI governance is credible.

Five areas examiners probe

• Data security and integrity. Protection of the data that trains and informs AI decisions across its lifecycle, with integrity checks on training data, inputs, model parameters, and outputs. Regulators increasingly treat corrupted or unrepresentative training data as a governance failure.
• Access and change management. Picture a data scientist pushing a model update to production on a Friday afternoon. Who authorized it, what changed, how was it tested, and what happens if it misbehaves Monday? Role-based access, segregation of duties, approval workflows, version history, and rollback procedures are the baseline.
• Traceability. For a specific decision on a specific policyholder's claim or premium, the carrier must reconstruct which inputs, which model version, and which human decisions produced the outcome. A general explanation of how the model works will not satisfy an examiner.
• Third-party risk. The obligation stays with the carrier no matter who built or runs the system. Vendor contracts need to address AI governance obligations, incident notification, documentation access, and audit rights.
• AI-specific incident response. A model producing discriminatory outcomes isn't "down" in any traditional sense. It keeps operating and accumulating exposure with every decision. Incident definitions and playbooks need to cover model degradation, biased outcomes, unintended use, and corrupted data.

The NAIC AI evaluation tool and the 12-state pilot

The Big Data and Artificial Intelligence (H) Working Group built the AI Systems Evaluation Tool to give examiners a repeatable way to check what the bulletin only described: the same questions, asked the same way, across every participating state.

The pilot launched on March 2, 2026, in 12 states: California, Colorado, Connecticut, Florida, Iowa, Louisiana, Maryland, Pennsylvania, Rhode Island, Vermont, Virginia, and Wisconsin. It ran through September 2026, with states sending inquiries to carriers and holding monthly coordination calls. Reviews folded into the market conduct and financial examinations carriers already undergo. The working group said it would update the tool on pilot feedback, re-expose it for public comment in October, and bring it to the Fall 2026 National Meeting in November for adoption. After adoption, any state insurance department can deploy it.

What the four exhibits ask

The tool is organized into four exhibits, each more detailed than the last.

Exhibit A: AI Usage Inventory. Pilot states focused here first. Carriers quantify how extensively they use AI: how many systems, in which functions, affecting which decisions. A carrier with a current model registry can answer in days. Without one it gets complicated fast, because "AI systems" includes vendor-embedded models, automated decision components inside larger platforms, and machine learning features nobody internally files under AI.

Exhibit B: Governance Risk Assessment. Oversight structures, risk management policies, accountability chains, and documentation practices. A policy document and a quarterly committee meeting fall short of governance infrastructure, and Exhibit B makes that gap obvious.

Exhibit C: High-Risk AI System Details. For systems regulators classify as high-risk (claims decisions, underwriting, pricing, fraud detection), carriers document model design, training data, validation procedures, performance metrics, and bias testing results.

Exhibit D: AI Data Details. Data sources, quality safeguards, representativeness, and potential for proxy discrimination. The NAIC is particularly focused on data used in rate setting, screening for proxies for race and ethnicity, social media data, and aerial imagery that may correlate with protected characteristics.

The working group circulated a Pilot Project Summary identifying the categories the evaluation focuses on: governance program design, model inventory, third-party oversight, fairness testing, and ongoing monitoring. Foley's readout of pilot mechanics describes a structured request asking carriers to demonstrate those practices with documentation.

How pilot states used it

Each state chose independently which carriers to examine and how to fit the tool into its workflow. Some embedded it in market conduct reviews; others used financial examinations. Selected companies spanned property and casualty, life, and health, at a range of sizes, so regional and mid-sized carriers were in the population.

Regulators said they would prioritize the systems most likely to harm consumers. Back-office automation draws less scrutiny than a model that denies claims, sets premiums, or flags policyholders for fraud investigation. The NAIC's stated position is that existing insurance law applies to AI-driven decisions the same way it applies to a human adjuster's. Where the AI touches the policyholder, the regulatory interest follows.

At the Spring 2026 meeting, regulators also floated a four-tier risk scale (unacceptable, high, medium, low) to direct scrutiny toward the systems that can do the most harm. The tier decides how much documentation, testing, and oversight each system carries. A carrier that hasn't sorted its models into tiers cannot show an examiner it is spending governance effort where the risk sits.

"We bought it from a vendor" is not an answer

Carriers take full responsibility for AI purchased from third parties. A carrier using a vendor's claims triage model has to answer the Exhibit C and D questions about that model: its design, training data, performance, and bias testing history. Many vendor contracts pre-date these expectations and never secured model documentation access, bias testing transparency, or performance data sharing. Amending them takes months, and a regulatory response clock does not pause for procurement.

What comes after the tool

The evaluation tool is a waypoint. The NAIC also appointed a Market Conduct Regulation Modernization Working Group at its Spring 2026 meeting. Illinois DOI Director Ann Gillespie framed its scope: "Artificial intelligence, new distribution models, national scale vendors, and other technological advances are significantly changing both consumers' expectations and insurers' business models and practices." Its recommendations were expected by year-end 2026. Between a standardized instrument and a modernized examination methodology, regulators are on track to have both by early 2027.

The same working group has signaled interest in a formal AI model law, and a separate third-party data and models framework has been advancing since spring 2026, possibly carrying licensing requirements for vendors that sell models into the industry. The direction is toward requirements with statutory weight.

"Voluntary for regulators, compulsory for companies": the industry pushback

In December 2025, a coalition of trade associations representing life, health, property and casualty, mutual, and reinsurance carriers sent the NAIC a joint letter objecting to the pilot. The phrase that made it into trade press called the pilot "one-sided, voluntary for regulators while compulsory for companies." States could opt in or out. Carriers in opting-in states could not.

The letter raised five concerns in all. Beyond the asymmetry, the pilot's duration was undefined; the tool could be used in financial exams, market conduct exams, or both; "negative" findings from pilot data might lead to penalties; and the pilot might launch before the final tool went through public comment. InsuranceNewsNet documented the industry's argument that the tool risked formalizing examination practice before carriers had built the underlying infrastructure.

The trade groups got a revised tool. They did not get a delay. Iowa Commissioner Doug Ommen, filling in for the working group's leadership at a public session, discussed the pilot's structure without addressing the voluntary/compulsory complaint head-on. The revision made minor procedural adjustments and conceded nothing on authority.

From the regulator's seat, the asymmetry is simply how examinations work: departments do not ask permission to open a market conduct exam. What is new is standardization. A single state's exam report on a single carrier rarely changes how other states examine. A standardized tool applied across 12 states produces a corpus of comparable findings that examiners everywhere will read to calibrate their own questions.

Two consequences follow for carriers:

• Pilot responses are a durable record. The trade groups asked for assurances that negative findings would not be used punitively. The revised tool contains none. A weak Exhibit B narrative filed during the pilot sits with that state regulator as a baseline for the next examination cycle.
• Findings travel as patterns. The working group's report would not name carriers, but if a large share of respondents could not produce a current inventory or complete high-risk model documentation, that pattern shapes the final tool and every department's 2027 examination planning.

For carriers outside the pilot states, the practical hedge is a self-assessment against the public version of the tool. Walking the four exhibits internally produces a current inventory (Exhibit A), shows where stated governance and actual operations diverge (Exhibit B), and exposes which high-risk systems lack the audit trail and bias testing record Exhibits C and D require. That last artifact takes longest, because it depends on documentation of design decisions and monitoring history that cannot be created under an examination deadline.

Executive Order 14365 and the federal-state fight over AI

Executive Order 14365, signed December 11, 2025, opened a federal-state preemption fight over AI regulation that the NAIC publicly opposed within days. The order contains no insurance carve-out. The NAIC's December 16, 2025 statement warned that it "could disrupt well-established processes that ensure fairness and transparency in insurance markets and safeguard consumers from unfair or discriminatory practices," and "could implicate routine analytical tools insurers use every day." The NAIC asked the administration to "reconsider this Executive Order and, at a minimum, affirm state regulation of AI in the business of insurance."

The phrase "routine analytical tools" matters. The NAIC is signaling that the preemption claim reaches beyond the recent AI bulletins into scoring models, underwriting algorithms, and rating tools states have regulated for decades.

Why McCarran-Ferguson doesn't settle it

The 1945 McCarran-Ferguson Act provides that no federal statute shall be construed to "invalidate, impair, or supersede" state insurance regulation unless it specifically relates to the business of insurance. Carrier counsel will reach for it first, and the NAIC will make the same argument. Two complications:

1. The Act speaks of "Acts of Congress," and an executive order is not one. The argument that an order issued without a specific congressional grant over insurance cannot displace state regulation is strong, but as of April 2026 it had not been litigated under EO 14365.
2. The order's authority claim presumably rests on a broader theory, such as the Commerce Clause or national security, each with its own preemption analysis. The counter-argument the NAIC has begun assembling is that states have shown effective AI oversight through bulletins adopted in about two dozen states.

The posture is unsettled and likely to stay that way for some time. Assuming the order preempts state authority bets against decades of insurance federalism; ignoring it bets against the text of a presidential order. The defensible position is to comply with both and let the lawyers argue about which binds.

What carriers should do while it plays out

The state calendar does not wait for federal court. As of our last review, bulletins adopted before the order had not been withdrawn, market conduct exams keep incorporating AI inquiries, and rate filings with algorithmic models still require the disclosures state actuarial offices ask for. The pilot proceeded on schedule, and the November adoption decision belongs to state regulators.

A small federal-track work stream covers the rest:

• Cooperate with state inquiries while preserving defenses. A carrier that refuses to respond to a state request on preemption grounds creates an examination problem long before any ruling. Respond, with a clearly noted reservation of rights.
• Document dual-compliance costs contemporaneously. Record where state and federal expectations actually conflict and what dual compliance costs. That record matters if the question is litigated or Congress draws lines. Regional and mid-sized carriers writing in the most active bulletin states face the sharpest version of this problem and benefit most from the record.
• Add an EO 14365 contingency clause to vendor contracts. Allocate compliance cost and disclosure responsibility between carrier and vendor whether preemption materializes or fails. Do not assume vendors are tracking the litigation closely.
• Coordinate with trade groups on amicus filings as challenges reach appellate courts.

What carriers need on file for the NAIC AI bulletin

The evaluation tool, the bulletin's Section 4, and state market conduct practice all point to the same file. These artifacts are required whether or not a carrier was selected for the pilot.

The core evidence checklist

• A model inventory that includes vendor models. One list of every AI system in use. Licensed models count.
• A risk tier for each system. Map every model to the four-tier scale so underwriting, pricing, and claims models carry the documentation their tier demands.
• Testing records, bias and fairness included. Evidence each model was evaluated for accuracy and disparate impact before deployment and on a schedule afterward, against thresholds set in advance.
• Change approvals and version history. Who approved each material change and what testing preceded the release.
• A trail from data to decision. For a contested outcome, which inputs produced which recommendation and what a human did with it.
• Proof of oversight on high-risk paths. Evidence that human-in-the-loop and escalation steps actually fire, beyond a policy saying they should.
• Vendor documentation and contract rights that let the carrier answer Exhibit C and D questions about third-party models.
• Incident records with detection method, timeline, and remediation. An empty incident log usually means nothing was watching.

Five deliverables, in dependency order

Our CIO playbook for the pilot window broke that file into five deliverables. The sequencing still holds, because each depends on the one before.

1. Model inventory. Every production AI system with its business purpose, in-scope states, responsible owner, deployment date, vendor, supervisory cadence, and documentation location. A single accountable executive (often the CRO or CDO) owns it, with model owners per line of business making the scope and cadence calls. Expect six to eight weeks for a mid-sized carrier evaluating one to two thousand candidate systems, most of it discovery: routing models in customer service, agent-side recommendation tools, fraud scoring inherited through an acquisition. Plan for discovery to surface 20 to 40% more in-scope systems than first estimated.

2. Audit trail specification. The evidentiary spine. A written audit trail specification defines what is captured per decision (input features, model version, output, confidence indicators, timestamp, downstream action), retention (typically seven years to match record-retention rules), storage, and query method. The CIO and CCO co-own it. The specification is two to four weeks of policy work; implementation across the inventory is a multi-quarter program, so track each system's status on a dashboard rather than trying to build everything at once.

3. Vendor contract amendments. A documented program covering every AI vendor in the inventory, the renewal calendar, the clauses already in place, the clauses targeted at next renewal, and an owner for each. General Counsel leads, procurement owns the calendar, and the CIO reconciles it against the inventory. Retrofits run on renewal cadence into 2027; trying to renegotiate every contract at once exhausts the legal budget and completes none. Even the vendor registry contemplated under the third-party framework would not shift diligence off the carrier.

4. Bias testing results. A methodology built for market conduct exams runs a four-test minimum (statistical parity, equal opportunity, calibration, counterfactual flip) on every in-scope model across each operating state's protected classes, with proxy validation and a remediation log. The CRO or a dedicated fairness officer owns it, and compliance signs off before results enter the regulatory file. Budget ten to fourteen weeks for a first complete pass and expect findings on 15 to 30% of consumer-facing models.

5. Drift dashboards. Operational drift detection on every consumer-facing model, with thresholds for material drift (typically a 5% accuracy change, a four-fifths rule violation on a monitored protected class, or distribution shift in a key input), an alert path to the model owner, and a link to the remediation log. Six to ten weeks if the audit trail exists, since dashboards consume its data. Without it, carriers fall back on vendor-supplied instrumentation that examiners discount.

Who owns it, and what it costs

Five deliverables with five ownership maps will stall in coordination overhead unless one executive is accountable, typically reporting to the CEO or COO, with dotted-line authority across the CIO, CRO, CCO, GC, and CDO functions and the budget to force scheduling decisions. A steering committee without that person meets every two weeks while deadlines approach.

For budgeting, our benchmark for a mid-sized carrier with two to four thousand in-scope systems was $4 to $8 million in incremental spend across people, infrastructure, and outside counsel. Infrastructure (audit trail, drift monitoring, evaluation tooling) is the largest and most reusable line. Building all five deliverables on one system costs less to run than five point solutions and yields a sturdier artifact set.

Mutuals and smaller carriers

Standardization removes any hope of staying below the line. The questions a national carrier gets are the questions a mutual with a two-person compliance function gets. The difference is which one already has the answers written down. A mutual licensed in any pilot state was inside the test population; one that writes only elsewhere has roughly one National Meeting cycle before the adopted tool reaches its regulator.

Mutuals do have structural advantages here, chiefly short decision chains and a board close to the work. Those advantages only count for questions the mutual can actually answer. A mutual that runs member data through AI inside a private, isolated environment also answers the next set of examiner questions, about where policyholder information travels when a model processes it.

Evidence produced as a byproduct of operations

Examiners can usually tell a program that was lived from one that was assembled. Contemporaneous logs, dated approvals, and test results that line up with deployment history show oversight that operated. A binder built after the exam letter is a single snapshot with no proof that governance ran between examinations.

That is the case for running governance continuously and mapping it to the NAIC bulletin and each state's requirements as you go. A governance layer that logs every AI interaction and configuration change, holds a live inventory of AI applications in production, and rolls it into board-ready reporting turns an examiner's request for the validation history of a specific underwriting model into a query. The file an examiner asks for becomes an export.

If you want a quick read on where your program stands against the bulletin's pillars, take the NAIC readiness scorecard. It takes a few minutes and points you to the fastest gaps to close.