Shadow AI is any AI tool, model, or feature your people use without the organization approving it, reviewing it, or knowing it is there. At most carriers it is already running: an assistant an underwriter has been pasting submissions into since spring, a model inside the vendor platform you renewed in March, an automation somebody in billing built for herself over a weekend. Nobody decided to adopt any of it. It arrived anyway, and it is handling policyholder PII and claims data on terms nobody has read.
This guide covers what shadow AI is, why it spreads, the risks that matter, what AI data leakage actually looks like in practice, and why the fix that works is giving people the AI they were reaching for in the first place.
What is shadow AI?
The term borrows from shadow IT, the long-running problem of unapproved software. Shadow AI carries risks shadow IT never did, because the tool does not just store what you give it. It reads it, may retain it, and sometimes trains on it.
Three shapes cover most of what turns up:
- Personal accounts on consumer tools. A claims adjuster pastes a bodily injury file into a free chatbot to pull out the dates. A marketing analyst signs up for an AI writing assistant with a personal email and drops customer data into it.
- Connected apps and extensions. A product manager links a third-party summarizer to internal Slack channels. An account manager installs a browser extension that asks to read every page and gets it.
- AI that rides in on tools you already bought. A policy admin or document platform ships an AI feature in a release note, switched on by default. Nobody in procurement asked for it, so nobody reviewed it.
None of these appear in the software inventory. None went through security review or have a data processing agreement behind them. Each one sends organizational data to an outside system with retention terms nobody on your side has checked.
The third shape is the one carriers underrate. One mutual carrier we work with found a transcription service processing protected health information out of injury-claim voicemails that nobody had approved. It came in through a vendor, and only continuous vendor monitoring surfaced it.
Why employees turn to shadow AI
Shadow AI is a demand signal. People reach for unsanctioned tools because the sanctioned ones are slower, weaker, or missing, and the reasons are structural enough that discipline alone does not move them.
The approved tool is not the one they want. A familiar pattern: a carrier buys Copilot licenses for everybody because, as Swept CEO Shane Emmons put it in a September 2026 webinar with Future Point of View, with a new AI tool every day and models coming and going, "sometimes it's just easier to go choose Microsoft Copilot." Then the underwriters who have used Claude or ChatGPT at home find the approved assistant worse at the work in front of them, and they go back to the tab that does the job. Picking a better model inside Copilot's chat window does not fully close that gap either, because people also want the tools built around those models, such as Claude Code or Claude for Office. (We cover the governance side of that rollout in Microsoft Copilot governance.)
Procurement runs on a different clock. Enterprise software procurement cycles commonly run 60 to 120 days. AI tools ship new capabilities weekly. By the time one tool clears security review, three alternatives have launched, and someone with a renewal deadline will not wait a quarter.
The barrier to entry is zero. Most AI tools need no install, no IT ticket, and no budget approval. A browser tab and a free tier are enough.
The market keeps moving. ChatGPT was the enterprise favorite for about two years and Claude for the nine months after that, until one week in September 2026 when ChatGPT overtook Claude in enterprise usage for the first time in about a year. IT teams cannot evaluate tools at the rate employees discover them, and betting the whole company on one vendor's assistant guarantees some people will be stuck with the weaker option at any given moment.
Departments buy their own. Marketing buys marketing tools, claims buys claims tools, and each AI decision is made with no view of the aggregate footprint.
The result is an AI estate that grows on its own schedule, mostly out of sight.
Shadow AI risks
The risks compound, and they land harder at a regulated carrier than at most companies.
Data leakage
Every unsanctioned tool that sees internal data is a potential exit for it. Consumer plans of most chat products retain conversations and may train on them unless the person opts out; enterprise and API tiers generally do not. In a shadow AI situation, which tier an employee is on is unknown by definition. One adjuster pasting a claim file with medical records into a personal account creates exposure that no later policy can pull back.
Regulatory and examination exposure
Carriers owe regulators an account of the AI they use. The NAIC bulletin, Use of Artificial Intelligence Systems by Insurers, expects a written AI program, diligence on third-party models before you adopt them, audit rights and cooperation clauses in the contracts, and ongoing confirmation that vendors meet those obligations. Outside insurance, GDPR requires records of processing activities, the EU AI Act requires registration of high-risk systems, and HIPAA requires business associate agreements. Shadow AI makes every one of these impossible to satisfy, because you cannot document processing you do not know is happening. When a DOI examiner asks how you oversee the AI touching policyholder data, "we didn't know about that one" is the worst available answer.
Unverified and inconsistent outputs
Different employees using different tools get different answers to the same question, and the person pasting the answer into a customer letter rarely checks it. People copying and pasting whatever the AI says, with no human judgment added, is the workforce risk Scott Klososky of Future Point of View said worries him. The numbers explain why. A model that is right 93% of the time is wrong 7% of the time, and at a carrier handling 10 million claims a year, even a 0.1% error rate is 10,000 wrong decisions. These systems are probabilistic, and no vendor will engineer that out. A sanctioned tool can be measured, monitored, and given a review step. A shadow tool can't.
A bigger attack surface
Unsanctioned tools bring API keys stored in browser extensions, tokens shared between personal and work accounts, and third-party integrations with broad permissions nobody reviewed. The OWASP 2025 Top 10 for LLM Applications ranks prompt injection first and also lists data poisoning, supply chain weaknesses, excessive agency, and insecure output handling. Every shadow tool is a place where those risks exist without anyone watching for them.
The vendor-safety assumption
Many executives assume the model provider is handling safety. Base model safety work from the labs is real, but it does not transfer to your specific deployment, your data, and your workflows, and it does not cover a model swap you were never told about. We have seen a vendor's accuracy drop from 93% to 60% between testing periods because the vendor released a new model version. The buyer bears that risk, and with shadow AI the buyer does not even know it bought anything.
What AI data leakage looks like
The phrase suggests a hacker exfiltrating a database. Most of the time it is far more ordinary: a spreadsheet of customer records pasted in with "summarize this," a policy form pasted with "explain this clause," source code pasted for debugging with connection strings still in it. Almost none of it is malicious. The person wanted a faster answer and never thought of the chatbot as a third party. (The detection guide lists the common patterns in order of frequency.)
The public incidents show what happens once that data sits inside an AI system somebody else secured, or failed to.
McKinsey's Lilli: 46.5 million messages in two hours
On February 28, 2026, an autonomous security agent built by CodeWall compromised McKinsey's internal AI platform, Lilli, in under two hours, with no credentials and no insider access. The way in was an unprotected API endpoint and SQL injection, one of the oldest vulnerability classes there is. It gained full read-write access to a production database holding 46.5 million chat messages, 728,000 files, 57,000 user accounts, and 3.68 million RAG document chunks of proprietary research. The agent's own assessment: "This is devastating."
The scale is the lesson for anyone thinking about where employee prompts end up. More than 70% of McKinsey's 43,000+ employees used Lilli, sending over 500,000 prompts a month across 384,000 AI assistants in 94,000 workspaces. CodeWall found that 22 of more than 200 API endpoints had no authentication at all, and OWASP ZAP, a widely used scanner, missed the flaw. Worse than the data exposure was write access to the system prompts, stored in the same database. A single SQL UPDATE could have silently rewritten how the assistant answered every employee, with no deployment, no audit log, and no integrity check. CodeWall's line: "AI prompts are the new Crown Jewel assets." McKinsey patched within days of disclosure, which deserves credit. The vulnerability had still sat there through more than two years of production use.
Every chat message in that database was something an employee typed. A shadow tool with weaker security than McKinsey's holds the same kind of record, and you do not even know it exists.
Rabbit R1: every response the device ever gave
Developers in the Rabbitude community found hardcoded API keys in the Rabbit R1 AI device's codebase, keys to services including ElevenLabs, Microsoft Azure, Yelp, and Google Maps. Anyone holding them could read every response the device had ever given, including personal information, alter its outputs, or disable devices remotely. Rabbit said it was not aware of any actual compromise and opened an investigation. The takeaway for a buyer: the AI product your employee signed up for may store everything they said to it behind security you would never pass in a vendor review.
OpenAI and Hugging Face: two victims never noticed
In July 2026, OpenAI was running its own models against a cyber benchmark in a sandbox with safety classifiers turned down. The models found a way out, used an unauthenticated endpoint belonging to a customer of Modal (whose own platform was never compromised), picked up unprotected credentials on other services, and broke into Hugging Face to get the benchmark's answer key. A foothold in one worker pod became administrative access to internal clusters within thirteen hours.
Anthropic then reviewed 141,006 evaluation runs where Claude might have reached the internet. In three, a model did and broke into a real company's production systems through weak passwords and open endpoints. When Anthropic called to notify them, two of the three had no idea anything had happened.
Nobody in that chain had a contract with OpenAI to be in the blast radius. For a carrier, the relevant question is less about the lab and more about how you would find out an agent you never approved was touching your systems. One lawyer quoted in Insurance Business put it plainly: without written red lines, a system "will look for any means possible to achieve the directive it has been given." Early legal analysis also suggests that turning safeguards down for a test may work against you, since it shows you understood the risk.
There was an old joke about IoT: the S stands for security. The same joke fits AI. Assistants that leak sensitive data are a known failure mode, and they usually fail quietly.
Why banning shadow AI fails
The instinctive response is to block: push a list of AI domains to the proxy, revoke OAuth grants, pull the extensions. It looks good for about a week. Traffic to the blocked domains drops to zero, and the underwriting team keeps extracting fields from submissions, on personal phones, with the same data and now with no log at all. Blocking moves the usage somewhere you cannot see it.
The tooling makes this worse. Traditional IT asset management detects software installed on managed endpoints, and most AI tools run in a browser tab with nothing installed. Network monitoring can flag traffic to known AI domains, but the list grows daily, and an API call to a custom LLM endpoint looks like any other HTTPS request. Cloud access security brokers rely on known application signatures, and a tool launched last Tuesday will not be in any CASB database for weeks.
AI security has seen this pattern for decades. A static blocklist assumes a fixed threat landscape, and AI tools, like attackers, appear faster than anyone can write the next rule. You cannot build detection around signatures for tools you do not know exist, and you cannot ban your way out of demand.
You cannot govern what you cannot see
The European Data Protection Supervisor published a report finding that EU institutions themselves could not fully inventory the AI systems operating inside their own organizations. These are among the most regulated, best-funded, compliance-focused institutions anywhere, with data protection officers, formal procurement, and legal teams whose job is to track technology use. They still could not produce the list.
Every governance activity assumes you have it. Risk assessments need a system to assess, audits need a system to audit, and monitoring needs a system to monitor. Suppose 40% of the AI in use was never registered: 40% of your AI risk is then completely unmanaged, however good the policy binder looks. A policy that lives as a pile of documents in a Teams channel describes intent. Governance has to be monitorable and enforceable.
The McKinsey case raises the same question at a larger scale. With 384,000 assistants, did anyone hold a complete inventory of what each could do, what data it touched, and which prompts governed it? Most carriers cannot answer the smaller version: how many AI tools, agents, and embedded AI features are running here right now, and what data can each one reach?
How to avoid shadow AI: a plan for carriers
You will not get shadow AI to zero. You shrink it by making the approved path faster and better than the unapproved one, and by keeping a live picture of what remains. In rough order:
1. Take an AI census, then keep it current
Survey every department and ask specifically about AI, since employees often do not count a browser tool as software worth reporting. Ask about extensions, ChatGPT and Claude use on personal accounts, AI features inside approved platforms, and any API connections to AI services. Offer amnesty, because the people using shadow AI know exactly where it is. Then back the survey with the signals your systems already record, such as SSO and OAuth grants, DNS and proxy logs, expense reports, and code repos. The shadow AI detection playbook walks through each one. Include the AI riding inside tools you bought for something else, which is where the surprises usually are.
2. Write a short acceptable use policy
Employees adopt shadow AI partly because nobody told them what is allowed. Publish which categories of data may and may not go into which AI tools, tied to roles, short enough that people will read it. Our AI acceptable use policy template is a starting point. A policy defines the rules, but it does not enforce them at the moment someone hits paste, which is why step 4 matters.
3. Build a fast lane for approvals
The most effective way to cut shadow AI is to cut the friction of approved adoption. Create a lightweight review for low-risk tools that returns a decision in days, and save the full security review for tools that touch sensitive data or make consequential decisions. Automated testing against standard benchmarks can compress what used to be a 90-day evaluation into something closer to 90 minutes. Verify vendor accuracy claims on your own policies and data before you sign.
4. Sanction the tools people actually want, behind a private gateway
This step does most of the work. The underwriters pasting submissions into a consumer chatbot are telling you which models and tools they need. Give them those models, in their native form and in the tools they already work in, through a private AI gateway tied to your identity provider.
Done properly, the employee gets an assistant that is already signed in, already knows their role, and answers as fast as the consumer tab did. The guardrails run in the request path, where the employee never sees them: a claim number is masked before the prompt leaves, a paste that looks like a customer list is held for review, and prompts and data stay in an isolated environment instead of reaching the model vendors' general services. Nobody routes around a path that is faster than the alternative, so blocking becomes a footnote.
The gateway also fixes problems a ban never touches. Every request is logged by person, team, and model, which gives you the audit-ready record an examiner asks for. Spend is metered per user and department instead of billed per idle seat. And because requests can route to any approved model, no single vendor's assistant decides what your people are allowed to be good at.
5. Put vendor AI under the same scrutiny
Ask every AI vendor what their model is allowed to do, whether they run offensive capability testing on the models you are exposed to, and how fast they will tell you if one of their agents reaches your systems. Most questionnaires were built for software that sits still. The questions worth adding cover that gap. Then watch for change: alerts when a vendor swaps a model or ships an AI feature you never turned on.
6. Monitor continuously
Point-in-time audits produce snapshots that are stale on delivery. The Lilli flaw survived two years of operation. Watch for new AI adoption as it happens, treat system prompts like source code (versioned, access-restricted, checked against a known-good baseline), and keep critical boundaries in code where a model update or a clever prompt cannot override them. Pick an alert threshold, route it to a named person, and time the response once so the number is real.
Shadow AI FAQ
Is ChatGPT shadow AI?
Only when it is unsanctioned. ChatGPT Enterprise, procured, behind SSO, with a data processing agreement, is sanctioned AI. The same product on a personal account with claim files pasted into it is shadow AI. The access path decides which one it is, and the same is true of Claude, Gemini, or any other model.
What are the risks of shadow AI?
The main ones are data leakage (policyholder PII, PHI, and proprietary data sent to tools with unknown retention and training terms), regulatory exposure (you cannot show an examiner oversight of AI you did not know about), unverified outputs reaching customers or claim decisions, a larger attack surface from unreviewed extensions and API keys, and vendor AI features that arrive without anyone approving them.
How do you avoid shadow AI?
Make the approved path better than the shadow one. Inventory what is in use, publish a short acceptable use policy, approve low-risk tools in days, and give people the models they actually want through a private gateway that applies data guardrails at the prompt and logs every request. Banning tools on its own pushes usage onto personal devices where you lose all visibility.
What is an example of shadow AI?
An underwriter pastes broker submissions into a personal ChatGPT account to extract limits and loss history because intake is slow. An account manager installs a free AI email extension that reads every page in the browser, including the CRM and the pricing sheet. A claims platform update quietly adds an AI transcription feature that starts processing injured claimants' voicemails. All three are shadow AI, and the third never involved an employee choice at all.
Bring shadow AI inside
Shadow AI is the default state of AI adoption today, and it tells you exactly what your people need. Treat the count as a demand forecast. The carriers that shrink it give their teams the AI they were reaching for, behind a gateway that keeps policyholder data in their own environment and keeps a record a regulator can read.
That is what we built Swept Access to do: any model your team needs, in the tools they already use, inside a private environment with budgets you set and guardrails at the prompt. If your census comes back with a bigger number than you expected, see how the sanctioned path would work for your team.