Human Error Is No Longer Only at the Keyboard

For years, the cybersecurity industry has placed the human near the end of the risk chain.

The employee receives the phishing email, clicks the link, reuses the password, mishandles the document, approves the fraudulent request, or uploads sensitive information to the wrong system. Awareness programs are then designed around that moment of action: recognize the threat, remember the rule, and make a safer choice.

Those moments still matter. Agentic AI creates an additional set of human-risk decisions much further upstream.

Someone defines the agent’s objective. Someone selects the data it can use. Someone grants access to tools and systems. Someone decides how much autonomy is appropriate. Someone removes a review step, accepts an exception, chooses a vendor, approves a deployment, or assumes another team has considered the consequences.

A single human decision can then be repeated through thousands of automated actions.

That changes both the scale and the location of human risk.

The practical answer is that Human Risk Management must expand beyond end-user awareness and move upstream into AI design, cognitive security, intelligence protection, decision governance, human factors, and organizational culture. The same psychological, behavioral, and anthropological tools used to understand workforce risk should become checks and balances around the people building, authorizing, operating, and relying on AI agents.

The human is still part of the attack surface. The more important question is increasingly which human, at what point in the lifecycle, with what authority, under which organizational pressures.

What does it mean for human risk to move upstream?

A downstream error happens near the visible outcome.

An employee clicks a malicious link. A user shares a credential. A manager approves a fraudulent payment. The action and consequence may be separated by minutes.

An upstream decision shapes the conditions under which many later actions occur.

A product manager defines the use case. An engineer gives the agent broad permissions. A business owner approves automated customer communication. An identity administrator assigns credentials. A governance committee accepts a residual risk. A transformation lead selects a deployment model that makes experimentation easy but visibility difficult.

The visible incident may occur months later.

By that point, the original decision may have been normalized, inherited by another team, or buried inside a workflow nobody wants to disturb.

NIST’s AI Risk Management Framework already treats AI risk as a lifecycle and socio-technical issue. It applies to organizations designing, developing, deploying, or using AI and calls for governance, mapping, measurement, and management throughout the system lifecycle. It also distinguishes a wide range of AI actors, including product managers, data scientists, domain experts, human-factors specialists, governance teams, legal and privacy professionals, deployers, operators, and affected communities.

This is a much broader population than the traditional cyber-awareness audience.

What does this mean for your humans?

Organizations need to identify people according to the decisions they influence, rather than only their department or job title.

A person may become a high-impact human-risk actor because they can:

• define an agent’s objective;
• choose or alter its data;
• connect a tool or integration;
• grant access;
• remove a human approval step;
• approve an exception;
• interpret or act on its output;
• expand its use to new teams or populations;
• decide whether unusual behavior requires intervention.

Some of these people will be developers and administrators. Others will be business users working through no-code tools, product owners, procurement specialists, data stewards, operational leaders, or executives sponsoring transformation.

They may never think of themselves as part of cybersecurity.

The system will not care what they think their remit is.

Why is one upstream decision so consequential?

A traditional human mistake often produces one action.

A person sends one file, approves one request, or enters credentials into one fraudulent page. The resulting harm can still be severe, but the initiating behavior is usually discrete.

A decision embedded into an agent can be replicated.

If a person gives an agent excessive access, that access may be used repeatedly. If a reviewer approves a poorly bounded objective, the agent may pursue it across many tasks. If a team trains users to accept automated recommendations mechanically, the behavior can influence thousands of decisions.

NIST’s work on agent identity and authorization reflects this increased scale. It describes agents as systems capable of autonomously performing tasks and notes that organizations must understand the risks associated with granting them access to diverse datasets, tools, and applications. NIST is specifically examining identification, authorization, auditing, and non-repudiation because agents can move from producing content to taking actions such as deploying code.

A human does not have to make the unsafe choice repeatedly.

They may only need to authorize the system once.

What does this mean for your humans?

The risk attached to a decision should reflect the scale of authority being delegated.

Organizations routinely apply stronger controls to privileged technical access because a privileged account can affect many systems. The same principle should extend to people who can create, configure, approve, or expand agent authority.

A person who can connect an agent to production data may need stronger capability assurance than a person using it to summarize public information. A business owner who can approve autonomous external communication should understand more than the organization’s general AI policy. A product manager defining the agent’s success criteria may need to understand incentive design, downstream effects, and escalation thresholds.

This does not mean surrounding every AI decision with committees and laminated forms.

It means recognizing that some decisions create scalable exposure and preparing the humans making them accordingly.

Why is the traditional awareness remit too narrow?

Traditional awareness programs developed around common workforce behaviors.

They teach employees to identify social engineering, protect credentials, handle information correctly, use approved systems, and report suspicious activity. Those capabilities remain essential.

Agentic systems introduce risks that cannot be addressed adequately through broad all-employee content alone.

The person creating the exposure may need to understand:

• delegated authority;
• system identity and access;
• objective design;
• data provenance;
• model and tool limitations;
• automation bias;
• decision rights;
• human approval thresholds;
• monitoring and intervention;
• intelligence leakage;
• change control;
• emergent behavior;
• downstream and third-party impacts.

NIST’s AI RMF makes this expansion explicit. Its governance function calls for trained personnel, differentiated human-AI roles, executive accountability, diverse multidisciplinary input, and an organizational culture committed to identifying and communicating AI risk.

Human Risk Management therefore needs a broader operating question.

Instead of asking only, “How do we stop employees from making unsafe choices with technology?” it must also ask, “How do we make sure the people shaping technology are equipped to create safe choices for everyone else?”

What is cognitive security?

Cognitive security concerns the protection of human perception, attention, judgment, memory, decision-making, and belief formation from manipulation, distortion, overload, or inappropriate delegation.

The term is used in several contexts and does not yet have one universally adopted enterprise definition. For Human Risk Management, it offers a useful way to describe the security of the human decision system surrounding AI.

Agents can influence cognition in several ways.

They can determine which information a person sees, summarize complex material, recommend an action, prioritize alerts, draft persuasive communications, or decide when information is presented. Their output may become the starting point for the employee’s judgment.

That creates risk even when the information is not overtly malicious.

A confident summary can anchor the reader. A recommendation can narrow the set of options considered. High volumes of generated material can overwhelm review capacity. Repeatedly accurate outputs can cause users to stop checking. Personalized agents may learn how to frame information in ways that are especially persuasive to the individual user.

The Human Risk Management remit should therefore include the conditions under which employee cognition is being supported, shaped, overloaded, or bypassed.

What does this mean for your humans?

Employees need to understand their relationship with AI-supported judgment.

They should be able to recognize when:

• an AI summary has replaced examination of the source;
• a recommendation is anchoring the decision;
• the system has narrowed the available options;
• confidence in the output exceeds the supporting evidence;
• workload is making meaningful review unrealistic;
• personalization may be increasing persuasive influence;
• repeated success is creating complacency;
• decision authority is moving toward the agent without a formal change.

Different roles will need different cognitive defenses.

Executives may need to challenge compressed summaries that remove uncertainty. Analysts may need methods for independent verification. Reviewers may need interfaces that reveal evidence and ambiguity. Managers may need to protect time for actual judgment rather than treating human review as a click added to an automated process.

Cognitive security is partly an education problem. It is equally a system-design and workload problem.

Why does intelligence protection belong in Human Risk Management?

AI systems are extraordinarily useful for assembling, transforming, and interpreting information.

That creates obvious data-security concerns. Employees may expose confidential information through unapproved tools, poorly controlled integrations, prompts, uploaded documents, or agent memory.

Intelligence protection is broader than preventing a sensitive file from being uploaded.

Organizations also need to protect:

• strategic intent;
• internal assumptions;
• decision criteria;
• operating weaknesses;
• customer and partner context;
• research and development direction;
• negotiation positions;
• security controls;
• incident details;
• proprietary methods;
• the patterns that reveal how the organization thinks.

An employee may not perceive these elements as sensitive data because each individual fragment appears harmless. An agent with access across systems may be able to combine them into something much more revealing.

The same risk works in the other direction. External information gathered by an agent may be manipulated, incomplete, or intentionally designed to influence the organization’s decisions.

What does this mean for your humans?

Employees need a more mature understanding of information value.

Traditional classification models often focus on the sensitivity of individual records. Agentic systems make aggregation and inference more important.

The relevant questions include:

• What could this agent infer by combining the data it can access?
• What does the pattern of our questions reveal about strategy?
• Which internal assumptions are being exposed through prompts and instructions?
• Can external sources manipulate the agent’s understanding of our environment?
• Who is checking provenance and credibility?
• Does the user understand that apparently public information may still create intelligence risk when combined with internal context?

HRM teams should work with Security, Data, Legal, Privacy, and intelligence functions to translate these concerns into usable workforce behavior.

“Do not upload confidential information” is still good advice.

It is no longer a complete intelligence-protection strategy.

How do human factors influence agentic risk?

Human factors examine how people interact with systems, tasks, environments, and organizations.

This includes attention, workload, mental models, interface design, expertise, fatigue, incentives, communication, authority, and team dynamics.

Agentic AI depends heavily on these factors because humans are expected to authorize, monitor, interpret, and intervene around systems whose actions may be complex and partially unpredictable.

NIST’s AI RMF Roadmap identifies human factors and human-AI teaming as areas requiring further guidance. It specifically calls for research into how human-AI teams should be configured to reduce harms and how humans understand AI output and oversee systems in real-world settings.

A control may fail because an employee was poorly trained. It may also fail because:

• the alert arrived among hundreds of low-value alerts;
• the interface hid the relevant evidence;
• the person lacked authority to intervene;
• the owner was overloaded;
• the decision occurred under deadline pressure;
• the team assumed another function was responsible;
• challenging the agent carried social or professional risk;
• the safe process was harder than the workaround.

These conditions cannot be corrected by reminding employees to “stay vigilant.”

What does this mean for your humans?

HRM should investigate the environment surrounding the behavior.

When a person makes a risky choice, ask:

• What were they trying to achieve?
• What information was available?
• What information was missing or difficult to interpret?
• What did the system make easy?
• What did it make difficult?
• What did the person believe they were authorized to do?
• What consequences did they expect for delaying or escalating?
• What had previous experience taught them?
• Which team norms influenced the decision?
• Was the person cognitively capable of performing the expected control under the actual workload?

This approach does not remove individual accountability.

It produces a more accurate explanation of behavior and a better chance of preventing repetition.

Why does organizational anthropology matter?

Organizations are not governed through policies alone.

They operate through rituals, stories, symbols, informal hierarchies, shared language, local workarounds, and beliefs about how things really get done.

Two companies can publish almost identical AI policies and produce very different behavior.

In one organization, employees may involve Security early because it is seen as a partner in design. In another, Security involvement may signal that the project is about to slow down. One company may treat an AI pilot as a controlled experiment. Another may use “pilot” as a socially acceptable way to begin production without saying so.

Organizational anthropology helps reveal these lived systems.

It asks questions such as:

• Who actually has influence over AI decisions?
• Which teams are trusted to move without review?
• What stories are told about people who raised concerns?
• What language is used to justify exceptions?
• Which workarounds have become culturally normal?
• Where does authority sit informally?
• Which risks are considered sophisticated and which are dismissed?
• What does a “good employee” do when policy and delivery conflict?

These cultural signals often explain why a technically sound governance model behaves differently in practice.

What does this mean for your humans?

HRM should measure the culture employees experience, not only the culture leadership intended.

That means combining surveys with interviews, focus groups, decision records, exception data, workflow evidence, incident reviews, and observed behavior.

A culture assessment might reveal that employees know the AI policy but believe senior projects are exempt. Exception data may show the same teams repeatedly expanding agent access. Interviews may reveal that operators hesitate to pause systems because they expect blame for disruption.

Together, those signals form a useful risk picture.

The anthropological layer shows how formal governance has been translated into daily life.

What does decisions governance add?

Decisions governance addresses how consequential choices are proposed, evaluated, authorized, recorded, challenged, and revisited.

Agentic AI creates many decisions that cross traditional functional boundaries:

• whether an agent should exist;
• what outcome it should pursue;
• which data it can access;
• which actions it may take;
• which people it may affect;
• how much autonomy is appropriate;
• what level of uncertainty is acceptable;
• when human approval is required;
• who can accept residual risk;
• what changes trigger reassessment.

NIST’s AI RMF emphasizes explicit processes for deployment and go/no-go decisions, clear roles, documented outcomes, organizational accountability, and improved information-sharing about decisions and responsibilities.

The challenge is that organizations frequently govern the system without governing the human decision process around it.

A form may record that the agent was approved. It may reveal little about whether the approver understood the authority being delegated, whether dissenting views were heard, or whether delivery pressure shaped the conclusion.

What does this mean for your humans?

The people participating in AI decisions need decision capability.

They should be able to:

• distinguish technical capability from authorized authority;
• identify assumptions and uncertainties;
• recognize who may be affected;
• assess whether oversight is realistic;
• challenge incentives and metrics;
• detect when the original use case has changed;
• understand when they lack the expertise to decide alone;
• document the rationale and evidence;
• revisit the decision when conditions change.

This is where risk psychology, behavioral design, culture, and governance meet.

A good decision process should make risky assumptions visible and challenge possible. It should also prevent authority from becoming so distributed that nobody feels responsible for the final outcome.

Which new human-risk populations matter most?

Agentic AI creates several populations that should receive more focused HRM attention.

AI and software builders

Builders define objectives, instructions, tools, integrations, data, and escalation behavior.

They need secure design knowledge, but they also need to understand human impact, incentives, decision boundaries, and the assumptions embedded in the system.

Business owners and product managers

These roles translate organizational goals into system behavior.

They may determine how success is measured, who is affected, and which trade-offs are acceptable. Their decisions can shape agent behavior more profoundly than a technical configuration.

AI transformation and rollout teams

These teams influence platforms, adoption models, governance pathways, communications, and the speed of deployment.

They need insight into culture, organizational readiness, decision rights, shadow authority, and behavior change.

Identity, access, and system administrators

Administrators may grant the technical capability through which organizational authority is delegated.

They need enough business and risk context to recognize when a routine permission creates an unusual consequence.

Data owners and information stewards

These people determine which information the agent can use and how that information is governed.

They need to consider aggregation, inference, provenance, intelligence value, and downstream reuse.

Procurement and vendor-management teams

These teams shape third-party access, contracts, accountability, and the organization’s understanding of external agent capabilities.

They need role-specific questions and escalation routes.

Human reviewers and supervisors

Reviewers are expected to notice errors, challenge outputs, and intervene.

They require manageable workloads, appropriate evidence, clear authority, and protection from automation bias.

Consumers of AI-generated intelligence

Executives, analysts, managers, and operational teams may rely on AI-generated summaries, scores, or recommendations.

They need cognitive-security capability: understanding how AI can frame, compress, prioritize, and influence decisions.

Senior leaders and risk acceptors

Executives create incentives and accept exposure.

They need enough understanding to recognize when speed, savings, or transformation targets are weakening the control environment.

What should Human Risk Management measure upstream?

Moving upstream means measuring the conditions that create risk before the final user action.

Useful measures may include:

Authority and access

• Who can build, configure, approve, or materially change agents?
• Who can grant access to tools, data, and production systems?
• Where is high-impact authority concentrated?

Decision capability

• Do approvers understand the consequences of the authority being granted?
• Can owners identify assumptions, affected groups, and escalation thresholds?
• Can builders recognize a poorly bounded objective?

Intelligence protection

• Do employees recognize sensitive combinations and strategic inference risk?
• Are approved agents accessing more information than the use case requires?
• Can people verify the credibility and provenance of external information?

Cognitive security

• Are employees over-relying on AI summaries or recommendations?
• Is workload making meaningful review impossible?
• Do reviewers form independent judgments or confirm the agent’s answer?
• Do users understand uncertainty and evidence quality?

Governance behavior

• Are material changes being reassessed?
• Are exceptions approved and closed with evidence?
• Are agent owners active and capable?
• Are decision rights understood in practice?

Cultural conditions

• Do employees feel safe challenging AI-backed decisions?
• Does leadership reward deployment more visibly than safe judgment?
• Are workarounds and shadow authority becoming normal?
• Can people raise uncertainty without proving a complete failure?

Remediation

• Did the intervention change access, behavior, capability, ownership, or oversight?
• Was the risky condition actually reduced?
• Is there evidence that the improvement persisted?

These measures move HRM closer to the decisions that create the exposure.

What do checks and balances look like for humans and agents?

Agentic systems need technical controls around identity, access, tools, data, logging, monitoring, and intervention.

They also need checks and balances in the psychological, behavioral, and organizational layers.

Psychological checks

These help people recognize cognitive bias, automation reliance, misplaced confidence, risk discounting, and the influence of time pressure.

Behavioral checks

These shape the environment through defaults, gates, timely prompts, approval thresholds, evidence requirements, and usable escalation paths.

Cultural checks

These create norms around challenge, ownership, early disclosure, secure experimentation, and learning from weak signals.

Governance checks

These clarify decision rights, separate proposal from approval, require independent challenge, and preserve the rationale behind material decisions.

Intelligence checks

These protect sensitive information, verify provenance, limit inference exposure, and establish when human or external validation is required.

Capability checks

These ensure that people assigned to build, approve, operate, or supervise agents can actually perform the role.

The objective is not to create friction everywhere.

Checks and balances should be proportionate to the authority, impact, scale, speed, and reversibility of the agent.

A low-risk assistant does not require the governance architecture of a system that can deploy code or act on customer records. NIST similarly emphasizes that the level of risk-management activity should reflect organizational risk tolerance and the context of the system.

How should HRM work with AI transformation teams?

Human Risk Management should be involved before the rollout plan is complete.

AI transformation teams shape the environment in which employee behavior will occur. They select platforms, define use cases, determine adoption incentives, design governance processes, and communicate what successful AI adoption looks like.

HRM can contribute by:

• mapping the people and roles surrounding each use case;
• identifying formal and informal decision rights;
• assessing cultural readiness;
• identifying populations with high-impact authority;
• measuring confidence, capability, and risk perception;
• designing role-specific learning and behavioral support;
• identifying likely workarounds and shadow authority;
• testing whether escalation and oversight work in practice;
• defining human-risk signals for ongoing monitoring;
• linking detected conditions to targeted interventions;
• verifying whether remediation changed the risk.

This is not an invitation for the awareness team to take over AI governance.

It is recognition that AI governance without behavioral, cognitive, and cultural expertise will miss a large part of the system it is trying to govern.

Why does this change the role of Human Risk Management?

Human Risk Management has often been positioned as the evolution of security awareness: more measurement, more targeting, better behavior change, and a stronger connection to organizational risk.

Agentic AI accelerates that evolution.

The function now has an opportunity to contribute to:

• AI readiness;
• transformation governance;
• cognitive security;
• intelligence protection;
• decision architecture;
• human-agent interaction;
• oversight capability;
• culture diagnostics;
• behavioral controls;
• continuous assurance.

This requires new data, partnerships, skills, and operating models.

HRM teams will need to work with engineering, product, data, identity, AI governance, transformation, Legal, Privacy, Compliance, and executive leadership. They will need signals beyond learning systems and phishing platforms. They will need to understand workflows, access, exceptions, ownership, agent activity, and decision processes.

NIST’s 2026 analysis of industry responses on AI-agent security found widespread agreement that agents create novel security threats and that established cybersecurity practices will need to be adapted to address them satisfactorily.

Human Risk Management practices need the same adaptation.

Steadying the humans and the agents they care for

Agentic AI has unbounded some familiar forms of risk.

A person can now delegate action, judgment, access, and persistence to software. The software can operate at a scale and speed the person could never achieve alone. Its behavior remains shaped by human goals, permissions, assumptions, incentives, and culture.

This creates a new kind of shared operating environment.

The humans need to understand what they are authorizing. They need protection from cognitive overload, manipulation, overconfidence, and silent authority creep. They need clear decision rights, meaningful training, usable systems, and a culture that supports challenge.

The agents need boundaries, constrained access, monitoring, evidence, escalation conditions, and owners capable of intervening.

Neither side can be made safe through policy statements and good intentions alone.

Human Risk Management can supply part of the missing stabilizing layer by using risk intelligence, behavioral science, psychology, organizational anthropology, and systems thinking to make the human conditions visible and actionable.

The industry has spent years teaching employees how to avoid becoming the final link in a cyber incident.

The next stage is helping them avoid becoming the person who designed the chain, approved it, or gave it permission to run.

Build Human Risk Management into AI readiness

Cybermaniacs’ AI Enablement and Culture Model (AIECM) and AI Readiness and Capability Framework (ARC) help organizations understand the human system surrounding AI adoption.

They examine how decision rights, cognitive security, intelligence protection, leadership behavior, workforce capability, risk perception, psychological safety, governance, and culture affect the way AI is designed, deployed, used, and supervised.

Together, AIECM and ARC help organizations identify:

• where consequential human-risk decisions occur across the AI lifecycle;
• which populations hold formal or shadow authority;
• whether builders, owners, approvers, and operators are ready for their roles;
• where cognitive overload or automation reliance may weaken judgment;
• how organizational culture influences disclosure, challenge, and escalation;
• which intelligence-protection behaviors need reinforcement;
• what human-risk signals should be monitored;
• whether remediation is producing measurable risk reduction.

This helps organizations build checks and balances around the humans and agents operating together, so AI can be adopted with stronger capability, clearer accountability, and fewer assumptions hiding inside the system.

Explore Cybermaniacs’ AIECM and ARC frameworks to move Human Risk Management upstream and build the human foundations for safer, faster, and more effective AI adoption.

[Learn more about AIECM and ARC]

Frequently asked questions

What does it mean for human risk to move upstream?

It means examining the human decisions that shape AI systems before an incident occurs, including objectives, data, permissions, autonomy, approval, oversight, and governance.

What is cognitive security?

Cognitive security protects human attention, perception, judgment, memory, and decision-making from manipulation, distortion, overload, or inappropriate dependence on automated systems.

How does agentic AI change human error?

A single human decision can be embedded into an agent and repeated across many automated actions. Human error can therefore become scalable, persistent, and separated in time from the eventual consequence.

Why does intelligence protection matter for AI agents?

Agents can combine data from multiple sources, infer sensitive information, expose strategic intent, and rely on manipulated external information. Protecting individual files is not enough to manage these risks.

Which employees should be considered high-impact AI risk populations?

Relevant groups include builders, product managers, business owners, transformation teams, administrators, data stewards, procurement teams, reviewers, consumers of AI output, and senior risk acceptors.

How do human factors affect AI security?

Workload, interface design, fatigue, authority, confidence, incentives, role clarity, and psychological safety influence whether people make safe decisions, recognize abnormal behavior, and intervene effectively.

What should Human Risk Management measure upstream?

HRM can measure decision capability, authority, access, cognitive reliance, intelligence-protection behavior, governance adherence, ownership, speaking-up culture, and evidence that identified risks were remediated.

Does expanding HRM mean taking over AI governance?

No. HRM contributes expertise in behavior, cognition, culture, risk perception, and human-system interaction. It should work alongside technical, legal, governance, transformation, and business functions.

Practical takeaway

For every material AI use case, ask:

1. Which human decisions created this system’s authority?
2. Who can expand or change that authority?
3. What cognitive work are humans expected to perform?
4. Can they perform it under real operating conditions?
5. What intelligence can the agent access, combine, or expose?
6. Which assumptions remain inside people’s heads?
7. What cultural pressures may weaken the controls?
8. How will we detect and correct those conditions before they become incidents?

Human risk did not disappear when software began making decisions.

It moved upstream.