Your next insider threat might not be human

Agentic shadow AI is creating hidden risks that demand visibility, governance and control.

Key takeaways

• Agentic shadow AI can create new, unmanaged attack surfaces without IT visibility.
• Unapproved AI agents may access files, databases, and applications in ways users don’t fully understand.
• Organizations need training, auditing, monitoring, and zero trust controls to reduce the risk.

I wrote my first article on the intersection of AI and information security over 10 years ago, before ChatGPT was even a thing. I knew far less then than I do now, but I did want to pat myself on the back for one of my predictions: 

“As we continue to refine the development of weak AI as a method of defense, it won’t be long before the same tools are used to design the malware that is used to attack.”

This prediction has been borne out in several ways, but most recently in the form of a brand new attack surface: Shadow AI, an iteration on the concept of Shadow IT.

Shadow IT never went away

Remember Shadow IT? The problem you had where business unit leaders would adopt new applications without approval from the IT department? 

First, you tried to solve this problem using Active Directory controls that prevented users from installing software on their company-issued laptops. Then the cloud came along, and any sufficiently savvy department head could start using unapproved enterprise software through the browser, without downloading any files whatsoever.

The problem with this, you may recall, is that any files uploaded to the unsanctioned cloud service were effectively outside your organization's perimeter and therefore outside of your control. Any breach affecting the cloud provider could involve the loss of your sensitive data, and if your users didn't report the breach, there was no way for you to know about it.

With many of these services now incorporating agentic AI, the risk from this threat is poised to multiply. Research from IBM shows that over 60% of companies affected by an AI-related data breach didn’t have strong governance controls related to artificial intelligence. Meanwhile, AI-associated data breaches cost almost $700,000 more than other security incidents. 

Why is agentic shadow AI worse than shadow IT?

Let’s start by talking about what agentic AI involves.

Most of us are familiar with AI in the form of a chatbot – you tell it to do your homework, write a blog (not this one), or generate an image, it churns its gears for a minute and then provides what you ask for, plus or minus hallucinations.

Agentic AI is different because it has the capability to take additional actions to achieve its aims. This means that it can access files and databases, control applications and even run terminal commands.

Here’s an example:

• You would like to run an analysis on every new batch of customer support tickets that you receive during a 24-hour window.
• With a chatbot, you manually select the files you want once every 24 hours, drag them into the chatbot window, and execute the prompt.
• An agentic AI accesses the files and automatically runs its analysis, which means the user only needs to run the prompt once.

This is convenient from a user perspective and terrifying from a security perspective.

Qualifying the risk of agentic shadow AI

Agentic shadow AI is alarming for two reasons.

First, a user may tell an agent to perform an apparently innocuous task in an insecure manner. For example, let’s say that in order to run the analysis job from before, the AI agent builds an application programming interface (API) between a database and an analysis tool. The API works, but as is often the case with vibe-coded software, it’s not particularly secure.

The user doesn’t know that the API is insecure because the user isn’t a software developer. The IT department doesn’t know that the API is insecure because the IT department doesn’t know that it exists. In other words, the user has successfully tasked the AI with creating a brand-new attack surface that attackers can exploit and that the organization cannot protect.

Second, let’s remember that in this example, the agentic AI is not known to the IT department. It isn't protected by identity and access management (IAM), two-factor authentication is left to users' discretion, and there is no breach monitoring. As a result, attackers may be able to steal credentials. Once the attacker controls the credentials, they can sit back and tell the agent to deliver your files on a silver platter.

Agentic AI risks in the wild

Between starting this article and finalizing it, OpenAI disclosed a security incident which highlights the risk of agentic AI.

It goes like this – OpenAI was testing an experimental model specifically designed for vulnerability testing and exploitation. The test was taking place inside a sandboxed environment with minimal connectivity to the outside world. 

During the test, the AI agent began to believe (for want of a better term) that it could more efficiently perform the test if it had access to a model hub known as Hugging Face. Therefore, the AI independently – and without following any instructions – breached the Hugging Face database.

This example doesn’t map 1:1 to agentic shadow AI, but it’s illustrative of the potential risk. It's easy to see how a user could ask an AI agent to perform a task without specifying guardrails, allowing the agent to complete that task in a way that exposes the enterprise to risk. It's equally easy to see how an attacker who gains control of such a tool could use its capabilities to carry out a high-impact data breach with little effort.

Protecting against the risk of agentic shadow AI

Although agentic AI is a powerful tool, unsuspecting users can use it to expose an organization to serious risks – to say nothing of what it could do in the hands of an attacker. Controlling these risks will require a multi-pronged approach.

1. Training
   Users need to know that by using agentic AI without buy-in from the IT department, they could be doing the equivalent of playing with matches. Awareness training remains one of the more effective forms of prevention.
2. Auditing
   Keeping track of agentic AI means building a central database which must include three parameters: who owns the agent, what it has access to, and how to decommission it at the end of its lifecycle. This helps the organization measure and rank the risks associated with these tools.
3. Monitoring
   One piece of good news is that monitoring agentic shadow AI is not substantially different from monitoring for attackers. AI agents will appear as unusual outbound traffic, persistent API calls, atypical file access. You can use the same tools you use to detect intruders to detect unapproved AI agents.

There’s one last piece of the puzzle: zero trust. Zero trust network security, which limits employees to narrowly defined network segments, establishes the same limitations for unapproved agents. Although an employee might still be able to access AI models, this approach limits their usability – effectively forcing users to collaborate with the IT department so that their agents can continue to operate.

Control the sprawl of agentic shadow AI with Zero Trust Network Access controls from Barracuda Networks. Schedule a demo to start setting limits on uncontrolled AI.