Looking ahead to 2027? This article captured our view in September 2026 as enterprise agent governance began moving rapidly from experimentation toward production. We have since expanded the core argument into our evergreen Guide, AI Agent Governance: The Missing Human Risk Layer, and our 2027 AI Agent Governance Predictions look at what happens next as supervision, accountability, delegation, and workforce risk become harder to ignore.
AI agent governance is rapidly becoming a serious enterprise discipline. It has to. Agents are no longer confined to answering questions in a chat window. They can navigate systems, use tools, access information, make decisions, execute multi-step tasks, and increasingly act on behalf of people inside real business workflows.
The security and governance ecosystem is responding quickly. Identity, permissions, data access, runtime behavior, observability, tool use, prompt injection, containment, and agent-to-agent interaction are all becoming legitimate control domains. In February 2026, NIST launched its AI Agent Standards Initiative to support interoperable and secure adoption, including work around agent authentication, identity infrastructure, and secure human-agent interactions.
OWASP is developing its own increasingly substantial body of agentic security work, while Microsoft and other enterprise providers are building controls around agent identity, permissions, runtime behavior, observability, and human oversight.
All of this is necessary. It is also only part of the system we are trying to govern.
Every time an organization asks a person to supervise an agent, approve its work, interpret its recommendations, decide when to challenge it, escalate a problem, take responsibility for its output, or simply work alongside it, another control surface appears: the human-agent relationship itself.
Agentic systems make this particularly visible because the risk increasingly sits in the human-agent system, not neatly on either side of the relationship.
That is where AI agent governance should overlap with Human Risk Management.
Organizations need a layered control model rather than a single AI security product.
AI governance establishes policy, accountability, acceptable use, and risk ownership. Identity and access management determines what agents and people can access. Data-security controls protect sensitive information. Agent-security and runtime controls constrain what an agent can do and create visibility into its actions, while monitoring and observability help organizations understand and reconstruct what happened.
Human Risk Management addresses another set of questions: whether the people working with agents have the capability, context, authority, and judgment required to use them appropriately; how much they rely on agent outputs; whether they verify, challenge, or override them when necessary; whether escalation actually works; how behavior changes as familiarity grows; and whether the organizational environment quietly undermines controls that appear perfectly sensible on paper.
We have written before about why Human Risk Management increasingly acts as a control plane for AI at work. Agentic systems make that argument more consequential. Technical controls can govern what an agent can do. Human Risk Management helps organizations understand and influence what happens when people trust, direct, supervise, challenge, override, or act on what that agent does.
Enterprise agent governance needs both.
For the durable version of this argument, see our canonical Guide to AI Agent Governance: The Missing Human Risk Layer.
Traditional software usually waits. It may automate a deterministic process, enforce a rule, or surface information, but the relationship between operator and application is relatively easy to understand.
Agentic systems blur that boundary. They can interpret a goal, plan a sequence of actions, choose between tools, retrieve information, make intermediate decisions, and continue moving a task forward with progressively less human involvement. NIST’s 2026 work on agent standards and security reflects the importance of secure interactions between agents, humans, and digital systems as that ecosystem matures.
Agent governance therefore begins with sensible questions about identity, access, tools, data, approvals, logging, stop mechanisms, and the possibility that objectives, memory, context, or instructions could be compromised.
Those questions are becoming more mature quickly.
The approval gate creates another set.
Who is doing the approving? Do they understand what they are approving? Do they have enough information to recognize a problem? Have they learned where the agent is reliable and where it is not? Are they expected to inspect one consequential agent action per day or three hundred routine ones? Does rejecting an agent create ten minutes of extra work? Is the employee already under pressure to move faster? Does the culture reward questioning automated recommendations, or does “the system said so” quietly end the conversation?
The agent may be well governed while the work system around it is not.
That distinction becomes more important as autonomy increases.
“Human in the loop” has become reassuring language. It suggests that even if an AI system becomes uncertain, consequential, or risky, a person remains present to make the final call.
Sometimes that is an excellent control. Sometimes it is a box on a diagram.
The presence of a human does not tell us whether meaningful oversight is occurring. The NIST AI Risk Management Framework and its supporting guidance are more precise about human roles, proficiency, organizational accountability, and the difficulties of human-AI interaction than much of the shorthand surrounding HITL governance.
A human approval point becomes meaningful when the person has enough capability, authority, context, attention, and opportunity to intervene effectively.
Imagine a finance employee reviewing an agent-generated recommendation. The workflow technically requires human approval, but the agent is normally right, the employee is processing dozens of recommendations, the reasoning is difficult to inspect, rejecting the recommendation creates more work, and the team is already behind target.
The organization certainly has a human in the loop.
Whether it has meaningful human oversight is a harder question.
That is why Cybermaniacs argues that agentic governance has to look beyond the existence of approval gates and examine the actual conditions surrounding human judgment. Our work on Human Control Effectiveness addresses that question directly.
For years, cybersecurity often divided controls into neat categories. Technology had vulnerabilities. People made mistakes. Security built protections around both.
Agentic work makes that separation considerably less useful.
An outcome may now emerge from a chain in which a person frames a task, an agent interprets it, several tools are invoked, an output influences the person's judgment, another system carries the decision forward, and the human eventually approves something they did not personally construct.
The better unit of analysis is increasingly the human-agent system: the combination of people, agents, workflows, permissions, interfaces, incentives, organizational conditions, and control mechanisms producing the outcome.
This does not eliminate individual accountability or technical security. It gives both of them context.
AI does not simply insert a new technology into an existing job. It changes how tasks move, where judgment happens, what people can see, when intervention occurs, and sometimes who or what is meaningfully responsible for the work.
Agentic systems accelerate that shift.
A mature Human Risk Management program therefore needs to understand more than whether somebody has read the AI policy or completed AI-awareness training. It needs visibility into how human behavior and capability change as AI begins mediating more consequential work.
Several dimensions deserve particular attention.
How much weight does an employee place on an agent's recommendation or output?
Too little reliance can defeat the purpose of the technology and send employees back toward manual or shadow processes. Too much reliance can turn assistance into deference, particularly when repeated exposure to competent outputs creates a growing assumption that the next output will also be right.
The objective is appropriately calibrated reliance.
That means understanding whether the person's behavior changes sensibly with the task, consequence, expertise available, system performance, and level of uncertainty involved.
Our Guide to AI Reliance in Human-Agent Work examines this relationship in more depth.
“Verify AI output” is easy policy language.
The interesting question is what verification actually means in the work.
Verification becomes harder when the task is complex, the employee lacks deep domain expertise, the evidence is difficult to inspect, or the efficiency benefit disappears if every output has to be manually recreated before somebody can trust it.
Organizations therefore need proportionate verification around actual workflows. A low-consequence drafting task and an agent recommendation that changes a customer's financial position should not require the same level of human scrutiny.
Without that distinction, “verify the output” risks becoming a policy instruction everybody understands in theory and performs very differently in practice.
An agent does not need formal decision authority to exert considerable influence.
It can determine which information appears first, frame the available choices, summarize evidence, suggest language, recommend a course of action, or establish an initial position around which subsequent human thinking gravitates.
Organizations therefore need to consider not only whether a human made the final decision, but how the AI shaped the judgment that produced it.
Our Guide to Human Decision Ownership explores the consequential decisions that should remain explicitly assigned to people even as AI performs more of the surrounding work.
This is part of why the human-agent relationship deserves to be treated as a real control surface rather than a footnote to model governance.
A well-designed agentic workflow needs more than an emergency stop.
Employees have to recognize when intervention is warranted, know what action to take, understand the consequences, and have the authority to act. Workflow design matters here, as do incentives, operating pressure, and culture.
An override mechanism that is technically available but socially discouraged, operationally painful, or routinely ignored is weaker than the architecture suggests.
Our work on Intervention Points, Intervention Authority, and Human Override separates these questions because the ability to intervene depends on much more than having a stop button somewhere in the interface.
The same is true of escalation.
Novel systems produce novel uncertainty.
Employees need somewhere to take the case that does not fit: an output that feels wrong but is difficult to prove wrong, behavior that may not violate policy but still seems concerning, an unexpected interaction between tools, or a workflow nobody quite seems to own.
Escalation is therefore part of the learning system surrounding agent adoption as well as a response mechanism for immediate exceptions.
Our Guides to Escalation in Agentic AI and AI Escalation Paths explore the handoff and organizational route in more detail.
The answer cannot simply be “Security” for technical risk and “HR” for people risk because some of the most interesting agentic failures occur precisely where those categories stop being separable.
Human behavior around AI will not remain static.
People learn the quirks of systems. They develop shortcuts and workarounds. Confidence changes. Teams establish informal norms. Tasks migrate between humans and agents, while skills strengthen, weaken, or simply become different. A workflow that behaves one way during a carefully managed pilot can look very different once several thousand employees have been living with it for nine months.
This is one reason our work on Delegation Drift has become increasingly relevant. The practical scope of delegated work, judgment, and authority can move away from what the organization originally intended without anyone consciously redesigning the governance model.
Agent governance cannot therefore remain a launch checklist.
It has to observe how the human-agent relationship evolves.
It is useful to think of agent governance as complementary control layers rather than a contest between competing technologies.
| Control layer | Core question | Typical capabilities |
|---|---|---|
| AI governance | Should this use exist and under what rules? | Policy, inventory, risk classification, ownership, lifecycle governance |
| Identity & privilege | Who or what can access which resources? | IAM, non-human identity, least privilege, authentication, authorization |
| Data protection | What information can enter or leave the system? | DLP, classification, access controls, data governance |
| Agent security & runtime control | What can the agent do while operating? | Tool controls, runtime policy, containment, monitoring, prompt-injection defenses |
| Observability & evidence | What happened, and can we reconstruct it? | Logging, telemetry, audit, SIEM integration, traceability |
| Human Risk Management | Can people work with and supervise the system safely? | Readiness, capability, reliance, verification, behavior, culture, escalation, intervention |
| Change & workforce enablement | Can the new way of working become safe, useful, and sustainable? | Role redesign, communication, learning, adoption support, workflow change |
These layers are complementary.
An identity product should not be expected to solve a workforce-readiness problem. Employee training should not compensate for wildly excessive agent permissions. A DLP control cannot tell you whether an approver understands the decision they are validating, and Human Risk Management cannot patch a technical containment failure.
Controls work best when they solve the problems they were designed to solve and share enough evidence to illuminate what is happening across the wider system.
The technical control plane is becoming more sophisticated. Organizations now need comparable seriousness around the people operating inside it.
There is a tempting answer to every new human-risk problem: let's train everybody.
Yes, training matters, but it is not an operating model.
If an employee lacks the authority to challenge an agent, training them to challenge it does not solve the problem. If a workflow requires impossible levels of manual verification, another awareness module will not make the workflow sustainable. If incentives reward speed while policy demands deliberation, the organization has created a control conflict.
Managing human-agent risk therefore requires changes across role and decision design, workforce capability, policies and guardrails, workflow controls, communications, measurement, and feedback from real use.
Training supports those elements. It cannot substitute for them.
This is where the existing discipline of Human Risk Management becomes particularly useful because it asks a more productive question than “Did the employee know the rule?”
It asks what is driving the behavior and which intervention is actually likely to change it.
Organizations are becoming very good at measuring AI systems.
They can track adoption, licenses, model performance, errors, latency, usage volumes, and increasingly the actions taken by agents.
Workforce measurement often remains much thinner.
An organization may know that most of a business function uses approved AI every week without knowing whether those employees can identify when an output requires verification. It may know that an approval was recorded without knowing whether the approver had enough understanding to challenge the recommendation, or see adoption rising without knowing whether reliance is changing sensibly as familiarity grows.
This is why Human-AI Risk Intelligence is becoming important.
The objective is not employee surveillance. It is enough visibility to recognize when the human-agent system begins drifting away from the assumptions on which the governance model depends.
Probably nobody alone.
Security has an obvious role in threat, identity, data, technical controls, monitoring, and incident response. AI governance leaders bring policy, lifecycle oversight, and risk classification. IT and technology teams own substantial parts of deployment. Legal, privacy, compliance, HR, and business leaders each own different parts of the environment in which AI-enabled work occurs.
The difficult risks appear between them.
Someone still needs accountability, but the operating model also needs shared evidence and coordinated intervention.
Our Guide to Human-Agent Accountability looks more closely at how responsibility, authority, and control need to remain aligned as agents perform more of the work.
Cybermaniacs does not replace AI governance, IAM, DLP, runtime agent security, or technical observability.
We work on the part of the system those controls cannot solve by themselves: the people, behaviors, capabilities, authority, culture, and changing work surrounding AI.
Our AI Enablement & Change work helps organizations understand workforce readiness, adoption barriers, capability, confidence, behavioral risk, and where targeted intervention can help AI adoption become safe and productive rather than simply widespread.
Our Agentic Readiness & Change work goes further into the operating model created by agents: helping organizations clarify human and agent roles, identify where workflows and controls need to change, prepare people to supervise increasingly capable systems, and build targeted enablement as agentic AI scales.
That work sits within a broader Human Risk Management approach because agentic AI is amplifying questions organizations already need to answer about judgment, trust, competency, behavior, accountability, intervention, and resilience while changing the environment in which those questions have to be answered.
The direction of travel has become clearer.
Technical agent governance is maturing quickly. Enterprise guidance now increasingly incorporates agent identity, lifecycle ownership, risk-based autonomy, observability, human review, escalation, and continuous monitoring.
That progress strengthens the argument we made here rather than replacing it.
As those controls mature, the remaining questions become more visible: whether people are capable of supervising what they have delegated, whether repeated automation is changing skill and reliance, whether accountability remains aligned with real authority, and whether organizations can see how the human-agent relationship changes after deployment.
Our evergreen AI Agent Governance Guide now develops that model in full.
And in our 2027 AI Agent Governance Predictions, we look at what we think comes next: measurable human oversight, supervision-capacity problems, Delegation Drift, Agentic Skill Atrophy, Human-Agent Accountability, and the growing need for Human-AI Risk Intelligence.
Agent security is going to improve quickly. Identities will become better defined, permissions more granular, runtime controls more capable, and observability stronger.
That progress should make agents safer.
It will not make the human side automatic.
The more capable agents become, the more consequential the handoffs between human judgment and machine action become too. Organizations will need to know when people should rely on the system, when they should verify, when they should intervene, whether they are equipped to do so, and what happens when the relationship between person and agent changes over time.
That is why the next phase of AI agent governance needs a broader unit of analysis.
The enduring question is whether the human-agent system is working safely, effectively, and as intended.
That is the human risk layer.
AI agent governance is the set of policies, accountabilities, technical controls, lifecycle processes, and oversight mechanisms used to determine how AI agents may be developed, deployed, accessed, monitored, and used within an organization.
Human risk arises from the way people understand, direct, trust, verify, supervise, challenge, override, and act on AI agents. It can also arise from workflow design, unclear accountability, insufficient capability, excessive or insufficient reliance, weak escalation, conflicting incentives, and organizational culture.
No. Human-in-the-loop describes the presence of a human decision or approval point. Effective oversight depends on whether the person has sufficient capability, context, authority, attention, and opportunity to recognize problems and intervene.
Most enterprises will need a combination of AI governance, identity and access management, data protection, agent-security and runtime controls, observability, Human Risk Management, and workforce change or enablement. These capabilities solve different parts of the same human-agent system.
Human Risk Management provides a workforce-risk layer within the wider AI governance system. It helps organizations understand and influence the capabilities, behaviors, authority relationships, culture, and organizational conditions shaping how people use and supervise AI.
Relevant evidence can include workforce readiness, role-specific capability, reliance and verification behavior, confidence, escalation, intervention and override behavior, supervision capacity, workflow conditions, cultural norms, Delegation Drift, and appropriate enterprise risk signals.
Measurement should be proportionate, transparent, and focused on making the human-agent system safer and more effective rather than turning AI governance into employee surveillance.