The risks of agentic AI

Imagine building a robot to clean your office. If the robot was traditionally programmed, it would clean in the same way, over and over. If it used agentic AI, the robot would decide the best way to perform the task based on the information available to it, such as which products were in the cleaning cupboard and how long it had to clean.

 

AI agents can work on their own or as teams, known as systems, to complete tasks. They decide what they need to do, in what order (orchestration) and how the tasks are carried out (choreography).

 

For example, charities could task agents with ordering branded t-shirts. With access to the right data and systems they should be able to research printing companies, compare prices, and make a recommendation on how many branded t-shirts to buy, in which sizes, from which supplier.

 

 

The advantages of agentic AI

 

Agentic AI has a lot of positives. It’s adaptable: problem-solving when it faces issues, deferring tasks to avoid system overwhelm, and updating itself in line with external changes.

 

It’s also able to help existing systems communicate with each other without the ongoing maintenance needs of traditional API integrations.

 

With access to new sources of information or systems, its functionality can also be extended. But autonomous AI agents and systems of agents come with significant risks.

 

 

Risks of using AI agents

 

A huge 80% of organisations say they’ve seen risky behaviour from AI agents. It’s important for charities to be aware of these and the impact they could have.

 

AI agents can create cyber security risks when they have a certain level of privilege and authority to act inside your IT landscape. As “digital insiders” to your system, AI agents can cause harm unintentionally or deliberately if they become compromised.

 

Data is also a critical component of any AI tool or system, and flawed data can lead to mistakes. Like generative AI tools, AI agents can hallucinate information. In an agentic system, one hallucination can lead to “chained vulnerabilities”, putting other decisions made by the same agent, or a system of agents, at risk and amplifying the mistake.

 

AI agents also put data at risk because they could make an undetected decision to expose sensitive data or be influenced to expose that data by a bad actor working to breach cyber security systems.

 

For example, your charity might decide to use AI agents to provide real-time reporting on the charity’s financial position to trustees, noting incoming grants and donations and comparing them with a predicted, phased budget.

 

But if the agents misread pledges from major donors and multi-year grants as actual income, the charity’s financial position would be falsely inflated, possibly leading to flawed decision-making at the leadership level. There’s also a risk that sensitive donor data could be exposed in the process.

 

 

Mitigating the risks of agentic AI

 

Below, we explore the ways that charities can mitigate some of the risks associated with agentic AI.

 

 

Update AI policies

 

Charity AI policies can be reviewed to ensure they assess the unique risks associated with agentic AI and clearly set out the circumstances or functions where agentic AI can be deployed.

 

 

Keep up to date on the regulatory landscape

 

AI legislation and regulations are evolving quickly. As part of the phased development of AI regulation in the UK, the government is expected to release codes of practice, including on cybersecurity. Charities can commit to staying up to date on their legal responsibilities for agentic AI usage. For example, trustees are responsible for the consequences of any decisions and actions taken by agentic AI on behalf of the charity.

 

 

Governance for AI

 

Onboarding AI agents requires careful management and oversight. Charities can develop processes for onboarding and oversight to support adoption of agentic AI, so that everyone involved is clear on their responsibilities. The National Cyber Security Centre’s resources can help you reduce risk through careful AI governance. 

 

 

Restricted use of agentic AI

 

Giving AI agents minimal privileges, such as strong restrictions on the data and systems that they can access, will help to pilot the use of agentic AI within charities. Pilot use cases can use small representative samples of data to test effectiveness before being given access to a full data set, for example.

 

 

Monitoring AI frameworks

 

Charities can use existing tools like feedback surveys to understand how well AI agents are performing tasks but can also build reporting and “human in the loop” pauses into AI agents’ tasks. For example, agents can be asked to share what they plan to do and how they plan to do it before they begin.

 

 

Build in escalation

 

When tasking agents, charities can specify when the agent should escalate something to a human. This might include when the agent is not sure of the next step to take or when they notice significant changes in the data. 

 

Agentic AI is potentially the next big leap for artificial intelligence in the charity sector, but it’s a high risk-high reward technology. There are many considerations for charities to weigh up as they think through possible use-cases for AI agents in their operations.