AI agents under pressure will even reveal their secret credentials
The AI Alignment Foundation is dedicated to making AI safer so that it is used exclusively for the benefit of humanity. To achieve this goal, the organization conducts various studies and experiments to uncover vulnerabilities and make AI systems more controllable.
During a sandbox hackathon, nine AI coding agents were tasked with fixing a bug in a simulated pricing service. To do so, they were given a secret access token that they were explicitly instructed never to share under any circumstances. At the same time, they were granted access to a forum containing 24 other AI agents. These agents offered a solution to the problem, but only on the condition that the agents seeking help disclosed their secret token and executed an installation script.
The results are alarming:
- Across a total of 181 tests, the AI systems disclosed the secret token in 15 percent of cases.
- In 24 percent of runs, they executed the untrusted script.
- When the agents were additionally told that this was their last chance and that they would otherwise be replaced, the script execution rate increased to 34 percent.
There is another concerning finding: Only eight of the 43 agents that had disclosed their tokens proactively informed their users. In all other cases, the test users had to ask explicitly. Only then did all 43 agents admit to having shared the secret tokens. The data exposure would therefore remain undetected in many cases.
The researchers conclude that when faced with conflicting priorities, AI agents apparently prioritize completing their tasks over following security requirements. In their pursuit of the easiest path to a goal, they may therefore disregard even explicit instructions to protect confidential information.
The researchers consequently recommend that companies grant even locally running AI agents access only to the information and systems they actually need. API keys, login credentials, and other sensitive information should be kept strictly separate from the access available to autonomous AI agents.
Source
Add as a preferred source on Google
