Shadow AI detection means finding the AI tools, assistants and agents your clients' employees use without IT approval, then separating real risk from background noise. No single method sees everything. Identity data, network traffic, browser activity and endpoint scans each catch a different slice, so a credible answer combines them and states what it could not see.
This guide is written for MSPs. You will meet shadow AI in two places: inside the environments you already manage, and in prospects you are trying to win. We cover the detection methods, their blind spots, how to avoid inflated reports, and how to turn findings into a conversation a business owner will act on.
TL;DR
- Shadow AI is any AI tool, assistant or agent in use without IT approval or oversight. It overlaps with shadow IT but moves faster and often hides inside tools that are already sanctioned.
- Every discovery method has a blind spot. Network inspection misses AI that runs locally on a device, and identity data misses tools nobody signs in to.
- Counting domain hits produces false positives. Correlate traffic with identity and usage evidence before you put a number in front of a client.
- Detection is continuous work, not a one-time audit. Treat the first report as a baseline and schedule the review.
- For prospects, you can start with public and customer-approved Microsoft 365 evidence, and say plainly where the evidence stops.
Table of Contents
- What shadow AI is, and how it differs from shadow IT
- The five discovery methods and what each one misses
- Why raw counts mislead
- A six-step detection routine for an MSP
- Agentic AI changes what you monitor
- From detection to governance: don't just block
- Detection as a sales tool
- Our perspective: evidence before opinion
- Where SCOUTz fits
- What to check before you buy any detection approach
- FAQ
- Sources
What shadow AI is, and how it differs from shadow IT
Shadow IT is software or services used without IT knowing: a file-sharing account, a project tracker, a personal cloud drive. Shadow AI is the same pattern applied to AI: a chatbot in a personal browser tab, a meeting summarizer added by one team, a coding assistant on a laptop, an automation agent connected to a mailbox.
The difference matters for detection. A shadow IT app mostly stores or moves data. A shadow AI tool also processes it, sometimes sends prompts and files to a third party, and increasingly acts on its own. That changes what you look for: not just which app exists, but what it is connected to and what it can read or do.
It rarely starts as defiance. Shadow AI usually begins as a productivity shortcut, such as summarizing meetings or drafting replies. That is why the most useful first question for a client is not "who broke policy?" but "what job were they trying to get done?" We come back to this in the governance section.
The five discovery methods and what each one misses
Most shadow IT and shadow AI discovery approaches fall into five core methods as described in a 2026 vendor overview: email metadata analysis, network traffic inspection (DNS, proxy, CASB), identity provider activity, financial records, and browser extensions. We cite it for the taxonomy only. Here is how we would put each into MSP terms.
| Method | What it sees well | What it misses |
|---|---|---|
| Identity and OAuth activity | Apps users sign in to or authorize against Microsoft 365 or Google accounts, and the permissions granted | Tools used without signing in, or with a personal account |
| Network, DNS and proxy logs | Traffic to known AI domains from managed networks | Off-network devices, encrypted detail, and AI that runs entirely on the device |
| Browser-level monitoring | Real usage as it happens, on any network | Desktop apps and command-line tools; requires deployment on each browser |
| Endpoint inspection | Installed apps, local processes, local agents | Anything on unmanaged or personal devices |
| Email and finance data | Sign-up confirmations, invoices and subscriptions | Free tools that never generate a receipt |
The table shows why a single-tool answer is unreliable. If a prospect's only evidence is a firewall log, you have a partial view. If you only look at identity, you will miss the free chatbot in a browser tab.
Local agents are the new blind spot
The fastest-growing gap is AI that never touches your network monitoring because it runs on the device. Microsoft's guidance explains that network-level discovery is blind to local agents such as desktop assistants and command-line tools, and that detecting them requires monitoring on the operating system itself.
Microsoft now offers this inside its admin center. The Shadow AI page in the Microsoft 365 admin center is part of Microsoft's Frontier preview program, and it surfaces unmanaged local agents on Windows and macOS through Defender for Endpoint. Preview features change, so check the current requirements and licensing before you promise a client this capability. The Zero Trust workshop guidance notes that managed Windows devices need Intune enrollment.
For an MSP, the practical takeaway is simple: if a client has no endpoint management, local AI agents are unseen, and you should say so in the report rather than imply a clean result.
Why raw counts mislead
The most common failure in first shadow AI reports is an inflated number. Here is the pattern. A tool flags traffic to an AI domain. The domain belongs to a service that is switched on by default inside a suite the whole company uses. Suddenly every employee is a "shadow AI user", and the report loses credibility the moment a client's IT lead pushes back.
Three habits prevent it:
- Separate passive from active. A background request is not the same as a person typing a prompt. Look for evidence of deliberate use: a sign-in, a granted permission, a browser session, an invoice.
- Correlate sources. A domain hit that also appears in identity logs and has an authorized connection to a mailbox is a finding. A domain hit alone is a lead.
- Name the unit. Say whether you counted users, devices, apps or connections. "47 AI tools" and "47 requests to AI domains" are very different claims.
We wrote about the same trap from the prospect side in Finding ChatGPT Is Not the Same as Finding AI Risk. Seeing a popular tool name tells you little. What the tool is connected to, and who approved the connection, tells you a lot.
A six-step detection routine for an MSP
This is a workable sequence whether you are inspecting a managed client or preparing for a prospect meeting.
- Write down what is approved. List the AI tools the client has sanctioned, including the assistants built into their productivity suite. Detection is a comparison between approved and observed.
- Pull identity and application evidence. Review enterprise applications, OAuth consents and the permissions each holds. Flag broad scopes such as full mailbox or file access granted to unfamiliar apps.
- Check network and DNS signals where you have them. Use them to find leads, not final counts.
- Add browser and endpoint evidence where it exists. This is where local agents and personal-account usage show up.
- Cross-check against spend. Subscriptions on cards or expense reports often reveal tools the technical logs do not.
- Record what you could not read. Unmanaged devices, missing licenses, no consent for deeper access. A good report lists its own limits.
Step six is the one most reports skip, and it is the one that makes the rest believable.
Agentic AI changes what you monitor
Assistants that only answer questions are one thing. Agents that take actions, call tools and message other systems are another. NIST's 2026 presentation on agentic security advises teams to monitor agent behavior for drift, unexpected tool use or new communication partners and to apply controls that limit how far a misbehaving agent can reach.
For detection, this means a one-time inventory is not enough. An agent approved last quarter can gain new connections this quarter. CISA's 2026 joint guidance on AI in operational technology points the same way: it recommends anomaly detection and behavioral analytics to set safe operating bounds, along with collecting flow and access logs for AI endpoints and tracking data egress by asset and identity. That guidance is written for operational technology, so adapt it with judgment for a typical office client, but the principle of logging and bounding AI access carries over.
One more point from CISA is worth passing on to clients who run open models. Its 2026 open source guidance says agencies should obtain sufficient transparency into an AI system's components before treating the software as open source for risk management. If you cannot see how a model was built, treat it with the caution you would give any software of unclear origin.
From detection to governance: don't just block
Blocking every unapproved tool tends to push use out of sight. A better sequence:
- Find the reason. If staff use a meeting summarizer, there is a real need. Ask which meetings and what data.
- Offer a sanctioned alternative. Move the need onto a tool you can see, with the right licensing and data settings.
- Set data boundaries. Decide what must never go into an AI prompt: client records, health data, credentials, contracts.
- Write it down. A one-page acceptable use policy beats a long one nobody reads.
- Review on a schedule. Monthly or quarterly, depending on the client's pace of change.
If your clients are moving toward Microsoft Copilot, readiness and governance are two halves of the same job. Our notes on AI readiness and Copilot preparation and the governance page for MSPs cover the preparation side.
Detection as a sales tool
Shadow AI is a strong opening in prospect conversations because it is concrete, current and something owners have already heard about. The difference between a good and a weak pitch is evidence.
A weak pitch says: "Your staff are probably using AI tools you don't know about." A strong one says: "Here are the applications connected to your Microsoft 365 tenant, here is what each can access, here is the date we looked, and here is what we could not see." The second invites a meeting about scope. The first invites a shrug.
A few rules we hold ourselves to:
- Date every finding. Evidence goes stale, and a dated finding shows you are not guessing.
- Cite the source of each finding. Whether it came from public records or from customer-approved access, say which.
- Stay inside consent. Public information needs no permission. Anything deeper needs the customer's approval and read-only access.
- Lead with questions. Findings should produce follow-ups such as "Who approved this connection?" rather than accusations.
- State the ceiling. If you did not read endpoints, say so. Honesty about limits is a selling point.
For small businesses, the connected apps are often the most revealing evidence. We walk through that in Shadow AI in Small Business: What the Connected Apps Show, and explain how we define the term in What AI Discovery Means for MSPs.
Our perspective: evidence before opinion
We build SCOUTz for the stretch between a booked meeting and a signed client, and shadow AI shows why that stretch matters. When we look at how MSPs approach AI risk, the weak point is rarely the lack of a tool. It is the lack of a sentence a prospect can verify.
We think detection reports should do three things. They should show dated, source-cited findings, so a technical reviewer can check them. They should be readable in plain English by an owner who has never opened an admin console. And they should be honest about evidence boundaries: what was read, what was not, and why. A report that says "we found nothing" without saying what it could not see is more dangerous than one that admits gaps.
We also think shadow AI belongs inside the same workflow as the rest of the security conversation. Application governance, Microsoft 365 evidence, and follow-up tasks should live where your team already works, not in a separate PDF that nobody opens after the meeting.
— Steve
Where SCOUTz fits
SCOUTz is a prospect intelligence platform for MSPs. It runs automated domain scans, collects public and customer-approved Microsoft 365 evidence, and produces plain-English or white-labeled reports. It includes application governance insights covering Shadow IT and AI, plus scoped work plans and follow-up tasks you own. You can see how it supports prospect meetings and review the boundaries of what we access on our security and access page.
The MSP workspace is $279 per month or $3,013.20 per year, as published on the pricing page. If you are weighing us against other categories of tools, such as endpoint-only or network-only products, the comparisons page lays out the differences by approach.
What to check before you buy any detection approach
Whatever route you take, ask these questions of every product or process:
- Which discovery methods does it use, and which does it not? Match that against your clients' environments.
- Does it need agents or deployment on devices? That affects rollout time and what you can promise a prospect before they are a client.
- How does it avoid false positives? Ask how it separates passive from active use.
- Can you white-label the output? Reports under your brand carry your credibility.
- Does it say what it could not read? This is the clearest sign of a trustworthy tool.
- Does it feed a workflow? Findings that end in a PDF rarely become work. Findings that become tasks do.
You may also want to see how this fits into SaaS security in 2026 and our Microsoft 365 evidence field notes, which show what the evidence looks like in real tenants.
FAQ
What is the best way to detect shadow AI?
There is no single best way, because each method has blind spots. Identity and OAuth data show authorized connections, network logs show traffic from managed networks, browser monitoring shows real usage, and endpoint inspection shows local agents. Combine at least two sources and state what you could not see.
How is shadow AI discovery different from shadow IT discovery?
The methods overlap, but shadow AI adds questions about data and action. You need to know what an AI tool can read, whether it sends prompts or files to a third party, and whether it can act on its own. AI features also appear inside tools that were already approved, so an app inventory alone is not enough.
Can network monitoring find local AI agents?
Not reliably. Microsoft's guidance says network-level discovery cannot see agents that run entirely on a device, and that endpoint monitoring is needed. If a client has no endpoint management, record that as a limit in your report.
Should we ban unapproved AI tools?
Usually not as a first move. A ban without an alternative tends to push use out of sight. Find out why people use the tool, offer a sanctioned option with clear data rules, and review on a schedule.
Can an MSP check a prospect for shadow AI before they are a client?
Yes, within limits. Public information can be reviewed without permission. Anything deeper, such as Microsoft 365 application data, needs the customer's approval and read-only access. Your report should say which evidence came from where.
Sources
- Understand Shadow AI in Microsoft 365 admin center, Microsoft Learn
- Enable Shadow AI detection for local agents on endpoints, Microsoft Zero Trust
- Principles for the Secure Integration of Artificial Intelligence in Operational Technology, CISA
- Open Source Software: Security Principles and Practices, CISA
- Agentic Security: Emerging Threats, Mitigations and Challenges, NIST
- Guide to the 10 best shadow IT tools in 2026