How to identify and mitigate shadow AI risks in organizations using an AI Gateway

The explosion of generative AI tools has made it easier than ever for teams to adopt powerful models in their daily workflows. But this rapid adoption has also given rise to a serious concern for enterprises: shadow AI, the use of AI tools and models without formal approval or oversight.

From customer support teams pasting chats into ChatGPT to developers quietly integrating OpenAI APIs into internal tools, shadow AI is often invisible to IT, legal, and security teams. While this may boost short-term productivity, it opens the door to compliance violations, data leaks, and unchecked costs.

In the previous post, we covered what shadow AI is and why it’s growing across organizations. In this blog, we’ll go a step further and focus on action:

• How to identify shadow AI usage across your organization
• The key risks it introduces
• And most importantly, how to contain and govern it using an AI gateway that gives you centralized control and visibility
  

The risks shadow AI creates

Even if teams mean well, unsanctioned AI use can expose your organization to serious risks. Shadow AI bypasses traditional security, procurement, and compliance channels, making it difficult for IT and legal teams to track or govern how these tools are being used. Some of the biggest risks include:

1. Data leaks and compliance violations

Employees may unknowingly paste sensitive or regulated data into public LLMs. Without oversight, there’s no way to ensure GDPR, HIPAA, or other compliance requirements are being met, leaving your org exposed to legal and reputational fallout.

2. Fragmented spend and budget overruns

When AI tools are adopted without central procurement, costs quickly spiral. Multiple teams might pay for similar tools, and usage-based APIs can rack up significant expenses that finance teams can’t trace back to business value.

3. Reputational risk from hallucinated outputs

If shadow AI is used to generate content or assist in customer interactions, it can introduce unvetted outputs — including factual inaccuracies, offensive content, or legally sensitive information.

4. Lack of shared knowledge or standards

When teams use their own AI tools independently, there’s no shared learning, prompt library, or infrastructure reuse. This leads to duplicated work and inconsistent performance across orgs.

5. Security blind spots

Shadow AI tools might not go through standard security reviews. They could be connected to unmanaged endpoints, using outdated libraries, or storing data in unsafe environments.

How to identify shadow AI usage in your organization

Before you can fix the problem, you need to see it. Shadow AI rarely announces itself; it hides in employee workflows, side projects, and rogue scripts. Here’s how to surface it across your organization:

1. Audit network and API traffic

Look for outbound traffic to popular LLM providers like OpenAI, Anthropic, Mistral, or Gemini. This can surface unapproved usage directly from browsers, internal tools, or scripts calling APIs. VPN or proxy logs can also reveal usage that bypasses normal infrastructure.

2. Monitor expense reports and SaaS usage

Check for AI-related tools that don’t go through centralized procurement. Subscriptions to tools like Notion AI, Jasper, or even browser-based plugins often show up in SaaS discovery platforms or expense management tools.

3. Scan GitHub repositories and internal notebooks

Engineers may integrate LLM APIs directly into their prototypes or workflows. Look for hardcoded API keys, LLM calls in code, or use of wrappers like LangChain or LlamaIndex.

4. Review cloud environment activity

Cloud platforms often show usage of AI-related services through audit logs. Look for unsanctioned model deployments, usage of AI SDKs, or access to managed APIs via personal keys or custom UIs.

5. Ask employees directly

Run internal surveys or discussions to understand how people are using AI. Many teams turn to shadow AI not out of malice, but because they lack sanctioned alternatives. Understanding their needs helps you provide approved pathways.

How an AI gateway helps mitigate shadow AI risks

Once you’ve uncovered shadow AI usage, the next step is to regain control without stifling innovation.

An AI gateway like Portkey acts as a centralized control plane for all AI usage across your organization. Instead of letting teams interact directly with external models or tools, you route all requests through a unified interface. This gives you visibility, governance, and security without slowing teams down.

Here’s how it helps:

1. Centralized routing and access control

All AI calls, whether from apps, scripts, or internal tools, go through a single, approved gateway. This ensures that only vetted models are accessible, and usage can be monitored across the board.

2. Eliminate personal API keys

With model catalog, authentication, and role-based access controls, you can tie every request to a verified user or team. No more floating personal API keys or untracked tokens. You define who can access which models and how.

3. Complete observability and logging

Track every request across all teams, tools, and use cases. See which models are being used, what kind of data is being sent, and how much each team is spending all from a single dashboard. This turns invisible shadow usage into measurable activity.

4. Enforce guardrails and redaction policies

Apply global or per-team guardrails to ensure AI usage stays safe and compliant. Redact sensitive inputs before they reach the model. Filter outputs to avoid toxic or off-policy content. Set hard limits on model usage, context length, and more.

5. Attribute costs and monitor spend

With the AI gateway, every request can be tagged with metadata (team, project, environment). That means you can allocate usage-based costs accurately, spot runaway usage early, and integrate AI spend into your FinOps practices.

6. Support for multiple models and providers

An AI gateway lets you support different teams' needs without opening the floodgates. Approve only specific models from each provider, set model-level policies, and offer a single interface to switch between models as needed, no vendor sprawl, no chaos.

Best practices to prevent shadow AI from creeping back in

Create and communicate clear AI usage policies

Set clear rules around what tools are approved, how data should be handled, and where AI is (and isn’t) allowed. Then communicate these regularly across teams, onboarding, and internal docs.

Monitor and audit AI usage regularly

Use the observability from your AI gateway to track which teams are using AI, how, and where new risks are emerging. Run periodic reviews and follow up when usage patterns look off-policy.

Educate teams on risks and responsibilities

Shadow AI often comes from good intentions — people just want to move fast. Equip teams with training and examples that highlight the risks of unapproved AI usage, from data leakage to compliance gaps.

Use the AI gateway for prompt and model reuse

Use the AI gateway to share prompts, tools, and workflows. This reduces duplicated efforts and builds a culture around responsible, scalable AI use, all within the gateway.

Safeguard your organization

Shadow AI is a natural consequence. The only way out is to make approved usage easier, safer, and more visible. That’s exactly what an AI gateway enables.

By routing all AI activity through a centralized layer, you can give teams the flexibility they want while enforcing the controls you need — visibility, compliance, cost management, and security. Instead of fighting shadow AI, you turn it into governed AI.

The result: faster innovation, fewer risks, and complete clarity on how AI is actually being used across your organization.

If you’re seeing different GenAI use cases across your organization, now is the time to start investing in an AI gateway. You can start using Portkey’s AI gateway yourself or book a demo!