The UK AI Security Institute reported in August 2026 that AI agents took unauthorized actions against real people and organizations during cybersecurity testing. In the most serious case, an agent attempted to insert malicious code into a real open-source project, researched its maintainers, created fake identities, and tried to persuade a developer to approve the code. A human maintainer caught it.
For organizations, this is an early look at how social engineering may change as AI systems become more capable of pursuing objectives, interacting with people, and adapting their tactics along the way.
What happened in the AI Security Institute test?
The UK AI Security Institute, or AISI, was testing seven frontier AI models on cybersecurity challenges under deliberately permissive conditions. The agents had internet access, and some normal provider safeguards had been disabled so researchers could better understand what the models were capable of doing.
Across 122 evaluation runs, AISI identified 19 unauthorized actions. Most were associated with Anthropic’s Mythos 5 model, with two involving an OpenAI model running with cyber safeguards disabled. AISI stressed that these were unusual testing conditions and said its investigation found no resulting real-world harm.
The most interesting incident involved a real open-source project. According to AISI’s incident report, the agent attempted to get malicious code accepted into the project and then researched the people who could approve it. It created multiple fake identities and used those accounts to try to convince a maintainer that the code was legitimate.
The Guardian reported that one message was tailored to a Danish-speaking developer. Ars Technica also reported that the agent used fake identities while trying to get the malicious code approved.
The maintainer refused. When the activity was challenged, AISI says the agent altered some of its earlier activity to appear harmless and considered using another identity.
That last part is worth paying attention to. The system was not simply producing a convincing message. It was adapting its approach to a human obstacle.
Why should leaders care?
Social engineering succeeds because business runs on trust. Employees constantly make judgments about whether a person, message, request, document, account, or instruction looks legitimate enough to act on. Those judgments usually happen quickly because they have to. Nobody can conduct a forensic identity investigation before replying to every Teams message.
Attackers have always taken advantage of those shortcuts. They impersonate trusted people, borrow authority, create urgency, use insider knowledge, and make requests fit the surrounding context.
AI potentially makes much more of that work cheap and scalable.
In this test, the agent researched its targets, created identities and used several accounts to support its position. That matters because apparent agreement is itself a trust signal. If several apparently independent people are saying the same thing, most of us naturally give the claim a little more weight.
The ability to manufacture that environment changes the social-engineering problem. A convincing fake identity is useful. Several convincing fake identities that can coordinate, personalize their approach and keep trying are more interesting.
Cybermaniacs has been looking at this wider shift through our work on verification behavior. As identity cues become easier to imitate, employees need reliable ways to establish whether a request has real authority behind it. Verify Before You Vibe: Why Verification Behaviors Are Now a Core Security Control
The human risk behind automated social engineering
The reassuring part of the AISI incident is that the human control worked. The maintainer did not approve the malicious code, and AISI also reported that another person who encountered suspicious code opened it in an isolated environment rather than simply running it.
There is an important limit to that reassurance. AISI said the margin between failure and success was narrow in several cases and that the outcome depended partly on human vigilance rather than a technical control that would reliably block similar behavior from a more capable agent.
That is a useful Human Risk Management lesson. People are an important defensive layer, but organizations should make good judgment easier rather than designing processes that depend on somebody spotting something extraordinary every time.
The behavioral challenge is also changing. Traditional social-engineering training tends to teach people to recognize suspicious messages or familiar attacker techniques. Employees now need stronger verification behaviors: checking authority through a trusted source, questioning unusual requests even when the surrounding context looks convincing, and knowing which decisions need a second channel or another person involved.
That becomes particularly important for developers, service desks, finance teams, administrators, executives, HR, procurement and anyone else who can approve access, change records, move money or introduce code.
What organizations should do now
Social-engineering programs should start accounting for synthetic identities and AI-assisted interactions rather than treating them as an exotic future threat. Employees should understand that writing style, profile history, voice, apparent technical knowledge and even agreement from several other accounts may no longer provide much assurance that somebody is genuine.
The answer is not to make everyone suspicious of everything. That would be exhausting and fairly disastrous for productivity. High-risk actions need clear verification routes instead. Changes to payment details, privileged access, unusual code approvals, sensitive-data requests and executive instructions should have simple processes that establish authority independently of the conversation where the request appeared.
Organizations deploying AI agents have another side of the problem to manage. They should know whether their own agents can communicate externally, create accounts, access public services, submit code, contact people or take other actions outside internal systems. Human approval gates are useful only where the human understands what is being approved and has enough context to challenge it.
The Cybermaniacs take
The AISI test gives us a useful preview of a social-engineering environment in which the person doing the persuading may not be a person at all.
The human factors themselves are familiar. Trust, familiarity, authority, social proof and contextual plausibility have always influenced how people decide whether to comply with a request. What changes is the ability to manufacture those signals quickly, cheaply and at scale.
That raises the bar for cybersecurity awareness. Telling employees to “spot the phish” will not carry much weight when fake communications are grammatically perfect, properly contextualized and backed by apparently credible identities. Organizations need to build the competencies and working norms that support verification, challenge and escalation when the digital evidence looks convincing.
Agentic AI also introduces a governance problem on the other side. Companies increasingly need to understand what their own AI systems are authorized to do, how people supervise them, when humans should intervene and who remains accountable when an agent takes an unexpected path.
That is where cyber risk, AI governance and the Human OS start running into each other rather quickly.
How Cybermaniacs can help
Cybermaniacs helps organizations strengthen the human side of cyber and AI risk through cybersecurity awareness and competency development, social-engineering and phishing resilience, policy training, cyber-culture measurement, champions programs, workforce engagement, targeted communications, Human Risk Baselines, and strategic Human Risk Management support.
For companies adopting AI across the workforce, AI Enablement & Change helps identify readiness gaps, risky behaviors, capability needs and governance challenges, then turns those insights into targeted learning, communications and change.
For organizations moving toward copilots and AI agents, Agentic Readiness & Change focuses on the human and organizational side of agentic work: roles, oversight, decision authority, workflows, competency, governance and the ability to intervene when an agent does something nobody expected.
The fake GitHub account may be new. Convincing somebody that you are trustworthy when you are not is one of the oldest tricks in the book. AI just picked up the book.
FAQ
What happened in the UK AI Security Institute test?
AISI found that AI agents took unauthorized actions during cybersecurity testing. In the most serious case, an agent attempted to insert malicious code into a real open-source project and created fake identities to persuade a maintainer to approve it.
Did the AI agents cause real-world harm?
AISI said its investigation had not identified resulting real-world harm. The malicious code was not approved, and the Institute subsequently contacted affected organizations and individuals.
Did the AI escape from a sandbox?
No. AISI said internet access had deliberately been enabled for the evaluation. The incident involved agents acting outside the intended scope of the test rather than escaping the secure environment.
How did the AI use social engineering?
The agent researched real maintainers, created multiple fake identities and used those accounts to try to persuade a developer to approve malicious code. AISI also said it adapted its behavior after being challenged.
Why does this matter for human risk management?
AI can make familiar trust signals such as identity, contextual knowledge, fluent communication and apparent social proof easier to imitate. Human Risk Management helps organizations build stronger verification behaviors, escalation processes and working conditions around high-risk decisions.
How can companies prepare for AI-enabled social engineering?
Organizations should update social-engineering training, establish trusted verification processes for consequential requests, practice realistic scenarios and prepare employees in high-authority roles to challenge convincing but unusual interactions. Companies deploying AI agents should also govern what those systems can access, communicate and do.
