People rarely make one durable decision about whether they trust AI.

Reliance develops through smaller choices made during the work. Someone accepts one output and questions another. They verify a recommendation when the consequence is high, but take a routine result at face value when checking it would cost more than the task seems to justify. As a system becomes familiar, they may give it more work, inspect fewer intermediate steps and allow it to act with less direct supervision.

Those behaviors are more useful to understand through reliance than through trust alone.

Reliance has a long history in human-automation research because effective collaboration with automated systems has always depended on knowing when to use their output and when to hold back. AI makes that question more important, and agents expand it further. The issue is no longer simply whether a person accepts an answer. It is how much work, judgment and authority they are willing to hand to the system before human control returns.

For Human Risk Management, that makes reliance a behavioral condition inside the work rather than merely an attitude toward AI.

What is reliance in Human-AI work?

Reliance is the extent to which a person allows an AI system's output, recommendation, process or action to influence or substitute for their own work, judgment or control in a particular context.

The behavioral part matters. An employee may say they distrust AI and still follow most of its recommendations. Another may report high confidence in a system while continuing to verify every consequential result because that is what professional standards or the surrounding workflow require.

Trust can influence reliance, but it does not tell us exactly what someone will do.

That distinction has been central to automation research for decades. Raja Parasuraman and Victor Riley's influential work on automation described both misuse, where people rely too heavily on automation, and disuse, where useful automation is ignored. John Lee and Katrina See later developed the idea of appropriate reliance, arguing that trust becomes especially important where complex systems make it impossible for people to understand every internal operation.

The contemporary AI version of the problem is much the same: when should the person rely on the system, and how far should that reliance go?

Appropriate reliance is more useful than maximum trust

AI adoption programs often talk about building employee trust, but more trust is useful only when the system and situation justify it.

A person who refuses reliable AI assistance can waste time, create unnecessary manual work and reduce the value of the technology. Someone who accepts a weak recommendation because the system has performed well in the past creates a different problem.

The useful objective is calibrated reliance: the amount of trust, verification, delegation and human control should move with what the system is capable of doing, what the task requires and what happens if it gets something wrong.

Microsoft Research often operationalizes appropriate reliance as accepting correct AI outputs and rejecting incorrect ones. That works well in research settings where correctness can be established. Enterprise work is messier because many consequential tasks involve interpretation, uncertain evidence or decisions whose quality may not become obvious immediately.

An analyst can accept a plausible synthesis. A manager may allow an agent to prepare a recommendation. A procurement team might use AI-generated rankings as one input into a decision. None of those cases necessarily gives the organization a simple correct/incorrect label against which reliance can be scored.

That is why acceptance rate by itself tells us very little.

High acceptance may reflect an excellent system. It may also reflect habit, time pressure, declining scrutiny or weak human capability. Low acceptance may represent sensible skepticism, unnecessary resistance or an AI system that simply is not very good.

The behavior is evidence. The meaning depends on the surrounding conditions.

Trust and reliance are related, but the work tells us more than the attitude

A person's trust in AI can be shaped by the product's reputation, prior experience, interface design, explanations, confidence indicators and their general beliefs about the technology.

Reliance becomes visible in what they do next.

Do they accept the answer? Check the source? Ask a second question? Delegate another step? Allow the agent to proceed without review? Retain control because something about the situation has changed?

That distinction matters for Human Risk Management because asking employees whether they “trust AI” can produce useful psychological information without telling us very much about how Human-AI work actually operates.

One team may describe its trust in AI as fairly cautious while using AI-generated analysis routinely and rarely verifying it. Another may be enthusiastic about AI but subject consequential work to rigorous independent review because that is how the profession operates.

At Cybermaniacs, these differences become part of Human-AI Risk Intelligence. Usage, confidence, verification behavior, task consequence, capability and organizational conditions provide different pieces of evidence, and the useful picture comes from understanding how they interact rather than treating any single measure as the answer.

Agentic AI changes what people are relying on

Much of the traditional reliance literature asks whether the human accepts a recommendation from automation. Generative AI expanded that relationship because the system could produce substantial amounts of content, analysis and reasoning.

Agents expand it again because the person can now rely on the system to perform part of the work itself.

That reliance can appear at several levels:

These forms do not have to grow together.

An employee might rely heavily on an agent to gather evidence while retaining close control of the final decision. Another may scrutinize the text an agent produces but give it considerable freedom to use approved tools and determine how the task is completed.

This is where reliance becomes a much more important agentic-governance issue. The question is no longer only whether the person believes the AI's answer. It is how much of the work can disappear from direct human view before judgment, supervision or control needs to return.

Reliance can expand without anyone formally deciding to expand delegation

Successful systems earn freedom.

A new AI tool usually receives considerable scrutiny because nobody yet knows how well it performs. As employees gain experience and the system repeatedly produces good results, checking naturally becomes faster. Some tasks become routine, and the person becomes willing to delegate more because previous experience gives them a rational basis for doing so.

That is not necessarily over-reliance. It is how useful technology becomes useful.

The governance problem appears when the practical scope of delegation changes faster than the formal operating model.

A system that originally drafted content may gradually begin conducting research. An agent that once prepared recommendations may start executing routine actions. Employees may stop inspecting intermediate steps because they have seen enough successful outcomes to believe that doing so adds little value.

At some point, the actual working relationship can be materially different from the one that was originally approved.

Cybermaniacs uses Delegation Drift for that broader problem: the practical expansion of work, judgment or authority delegated to AI without an equally deliberate reconsideration of human responsibility and control.

Reliance is one of the forces that can drive that drift. Repeated competence naturally earns more autonomy; the challenge is making sure the organization notices when earned confidence has quietly changed the operating model.

Familiarity can improve calibration and weaken scrutiny at the same time

People do not meet the same system as strangers every morning. They accumulate experience with it, and that experience should influence how they work.

If a tool performs reliably on a familiar task, checking every output with the same intensity forever would be irrational. Employees should learn where the system is dependable and where additional scrutiny has proved worthwhile.

The difficulty is that familiarity can make exceptions less noticeable.

A system that is right 98 percent of the time may be easier to supervise operationally than one that fails constantly, but it can be harder to challenge psychologically because success has taught the user that close scrutiny usually produces little value.

Generative AI adds another complication because incorrect outputs can look just as polished as correct ones. Microsoft Research has repeatedly explored this problem, including how explanations and confidence cues can increase reliance even where the underlying answer is wrong.

This is one reason Cybermaniacs treats reliance as something that needs to remain responsive to context. Confidence earned in one class of work should not automatically travel with the system into a different task, a higher-consequence action or a situation outside the conditions in which the employee has learned to trust it.

Interface design can change reliance without changing system quality

People respond not only to performance but to signals about performance.

A well-written explanation can make an answer more convincing. A confidence indicator can change how readily a recommendation is accepted. Citations can make an output feel more authoritative even if the employee never opens the source to determine whether it actually supports the claim.

That means reliance is partly a workforce issue and partly a design issue.

An organization can train employees to question AI while simultaneously presenting the system in a way that encourages passive acceptance. It can tell users that confidence scores are imperfect while designing the interface so that a green 97% badge dominates the screen.

The design goal should not be to make AI appear uncertain for theatrical reasons. It should be to help people interpret the signals they receive without treating polish, explanation or confidence as substitutes for evidence.

Our Guide to How Should Employees Verify AI-Generated Work? explores the same problem from the checking side. Reliance asks the prior question: how much independent checking does the person believe this situation deserves in the first place?

Verification cost is part of the reliance decision

“Check the AI” is easy advice until checking requires recreating the entire task.

If an employee saves fifteen minutes using AI and then needs twenty minutes to establish whether the output is trustworthy, the economics of the workflow become fairly obvious. People will either stop using the system, find faster ways to review it or gradually reduce the depth of checking.

This makes verification cost an important part of reliance.

A low-consequence draft may deserve only ordinary review. A strategic recommendation might justify independent evidence and closer reasoning. An agent about to modify a production system creates a different problem again.

The amount of scrutiny should therefore follow the work rather than one organization-wide rule.

It also means verification requirements should be designed with some respect for human attention. If the operating model depends on employees performing a form of checking that is consistently too expensive to sustain, the organization has not really created a control. It has created a hope about future employee behavior.

Our broader work on Human Control Effectiveness asks exactly that kind of question: whether the human control relied upon in the AI-enabled process can actually perform the risk-reducing function assigned to it under realistic working conditions.

Organizational pressure can recalibrate reliance without changing the AI

Reliance can look like an individual choice while being heavily influenced by the organization around the employee.

Suppose leaders announce that AI should increase productivity by 30 percent. Managers begin asking why familiar work still takes as long as it did before AI arrived. Employees remain formally responsible for checking important AI output, but careful verification is now competing with an explicit expectation that the technology should make everything faster.

People will adapt to that environment.

Local norms begin developing around what is worth checking, what can be accepted, how much delay is considered reasonable and whether challenging the AI is interpreted as good judgment or reluctance to embrace the transformation.

The opposite problem is possible too. If employees are punished heavily for mistakes involving AI, teams may avoid using reliable systems even where doing so would improve performance.

Appropriate reliance therefore depends on more than system accuracy and user knowledge. Workload, incentives, professional norms, management expectations, accountability and psychological safety all shape the amount of human scrutiny that survives in the real workflow.

That is why two teams using the same AI product can develop very different relationships with it.

Reliance becomes a supervision problem when agents start doing the work

Once an agent is executing rather than merely advising, reliance affects how the person supervises the whole process.

A supervisor who believes the agent will behave correctly may monitor lightly and focus attention on exceptions. If that confidence is justified, this can be an efficient and sensible use of human attention.

A supervisor with little confidence may inspect every action, recheck the work and require so many approvals that the organization gains very little from autonomy.

Neither pattern can be called good or bad without understanding the agent's performance and the consequence of the work.

This is where reliance connects directly to Cybermaniacs' concept of Supervision Sufficiency. The organization needs enough human supervisory capacity for the autonomy and consequence involved, but “enough” does not mean constant observation.

Reliance changes what the supervisor notices, what they consider meaningful deviation and when they choose to intervene. A person who assumes the agent is almost always right will experience an Intervention Point differently from someone who expects failure around every corner.

The mechanism can be perfectly designed and still depend on a human judgment about whether the situation has become unusual enough to use it.

Appropriate reliance should move with consequence

The same AI system can deserve different levels of reliance during the same workflow.

Research may be low consequence. A recommendation based on that research may deserve more scrutiny. Turning the recommendation into an external action may change the situation again.

This becomes especially important with agents because consequence can increase while the system is working.

An agent might begin by gathering public information, move into internal analysis, prepare a recommendation and eventually request authority to act. The fact that the employee was comfortable relying on it during the first stage tells us very little about how much scrutiny the final stage deserves.

Organizations should therefore avoid defining one universal level of “trust” for a tool or agent.

Reliance should move with the system, the task, the consequence and the authority being exercised. This is one reason our work on Human Decision Ownership and Intervention Authority matters: a change in what the agent is allowed to decide or do should often change the human relationship around it as well.

Human capability sets a ceiling on how well reliance can be calibrated

People can only judge when to rely on AI if they know enough to assess what it is doing.

An experienced professional may recognize that an answer is beautifully presented but rests on an implausible assumption. Someone with much less expertise may see the same response and have little independent basis for challenging it.

That makes reliance inseparable from capability.

The relationship becomes more difficult when AI performs enough of the work that the person gets less practice doing it themselves. As direct participation falls, so can the experience required to verify, challenge or recover the work.

Our Guide to Agentic Skill Atrophy looks at that longer-term risk. The system's success can reduce the practice through which human expertise was previously maintained, leaving the organization more reliant on AI while weakening some of the capability needed to calibrate that reliance.

This can become self-reinforcing. The harder independent evaluation becomes, the more attractive it is to trust the system, and the less frequently the person exercises the underlying skill.

That is why Human-AI Workforce Resilience is not simply an adoption question. The workforce needs enough retained capability to challenge, supervise and recover AI-enabled work even as more routine execution moves away from people.

What should organizations measure about reliance?

Reliance is worth measuring, but it is easy to turn it into a bad score.

A survey question about AI trust can contribute useful evidence. It should not be interpreted as a direct measure of whether reliance is appropriate.

Organizations need a more contextual picture that can include what people delegate, when they accept or reject AI recommendations, how verification changes with consequence, how often they correct or intervene, the authority assigned to the agent, changes in direct task participation and whether workload gives employees enough time to exercise the judgment the policy assumes.

Outcome quality matters too, but it needs interpretation.

A declining verification rate could mean that employees have developed appropriate confidence in a highly reliable workflow. It could also reflect complacency, workload pressure or loss of the expertise required to verify effectively.

This is precisely why Cybermaniacs frames these signals as Human-AI Risk Intelligence, rather than a single reliance metric. The objective is to understand whether human judgment, capability, authority and system performance remain aligned as the work changes.

The proprietary assessment logic sits inside our broader Human Risk Assessment & Assurance methodology. Publicly, the important principle is that a measurable behavior does not become a diagnosis merely because it is easy to count.

A practical way to review reliance in agentic work

Organizations do not need an elaborate new framework to begin asking better questions.

Start by identifying what the person is actually relying on the AI to do. Generating a draft, forming an analysis, recommending a decision, running a process and exercising delegated authority represent very different relationships.

Then examine what happens if the system gets that part of the work wrong. Consequence and reversibility tell you a great deal about how much independent human scrutiny is reasonable.

Verification needs attention as well. If meaningful checking is quick, the workflow can support more routine validation. If checking requires specialist expertise or reconstruction of the whole task, the organization should be realistic about how often employees will actually perform it.

Finally, look at change over time. Has the agent taken on more of the work? Are employees inspecting less? Has the volume increased? Are interventions becoming rarer? Has direct human capability declined while delegated authority has expanded?

None of those changes proves that reliance is inappropriate.

Together, they tell the organization where it may be worth looking more closely.

Reliance belongs inside Agentic Readiness

Organizations preparing employees for AI often start with tool knowledge: what the system can do, which use cases are approved and what information can be entered.

Reliance requires a more behavioral form of readiness.

People need to understand where greater verification is justified, what uncertainty looks like, when agent authority has changed, how to challenge plausible output and when confidence earned through routine success should be reconsidered.

Managers have an equally important role because they create the environment in which these choices happen. Productivity expectations, workload, incentives and the response to employee challenge can push teams toward either excessive caution or excessive delegation.

This is why Cybermaniacs' Agentic Readiness & Change (ARC) work extends beyond teaching employees how to operate agents. As roles move from execution toward delegation and supervision, the relationship people develop with those agents becomes part of whether the operating model works.

Reliance is one of the things that relationship changes.

Reliance is dynamic because the work is dynamic

One of the easier mistakes in AI governance is to treat trust and reliance as conditions established during rollout.

They move as the system changes, employees gain experience, workflows become more automated and managers alter what they expect from AI-enabled work. Verification that initially felt reasonable may later become burdensome. Tasks once performed directly may become almost entirely agentic. Employees can become more capable in one area while losing practice in another.

The formal workflow can remain unchanged while the real Human-AI relationship shifts considerably.

That makes reliance a valuable longitudinal signal for Human Risk Management. The useful question is not simply whether people rely on AI today, but whether they rely on it differently than they did three or six months ago, what changed around that behavior and whether the new relationship still makes sense for the work.

The objective is not perpetual suspicion.

AI should earn reliance where it performs well, and organizations should benefit from the efficiency that comes with allowing good systems to do more.

The risk appears when reliance quietly outruns the evidence, the human capability or the authority model around it.

In agentic work, that can mean much more than accepting a bad answer. It can mean discovering that the organization has handed over substantially more of the work than anyone remembers deciding to delegate.

Frequently Asked Questions

What is reliance in Human-AI work?

Reliance describes the extent to which a person allows an AI system's output, recommendation, process or action to influence or substitute for their own work, judgment or control in a particular context.

What is appropriate reliance on AI?

Appropriate reliance occurs when the amount of trust, verification, delegation and human control fits the system's capability, the task, the consequence and the conditions under which the AI is operating.

What is the difference between trust and reliance?

Trust is an attitude or expectation about an AI system. Reliance becomes visible in behavior: whether someone accepts an output, follows a recommendation, delegates additional work or allows the system to act.

What is AI over-reliance?

Over-reliance occurs when people rely on AI in circumstances where more verification, skepticism or human control would have produced a better or safer outcome. In enterprise work, that cannot always be inferred simply from how often people accept AI output.

What is AI under-reliance?

Under-reliance occurs when people reject or avoid useful AI in situations where the system could reasonably improve performance, quality or efficiency.

Why is reliance different with AI agents?

Agents can perform processes and exercise delegated authority rather than merely produce outputs. Reliance therefore expands from accepting information to allowing the system to determine intermediate steps, make recommendations and act within defined boundaries.

How does reliance relate to Delegation Drift?

As people become more comfortable with an agent, they may gradually allow it to perform more work or operate with less supervision. When that expansion occurs without a deliberate reconsideration of authority, accountability and control, reliance can contribute to Delegation Drift.

How does reliance affect Supervision Sufficiency?

Reliance influences how closely people monitor agents, which deviations attract attention and when supervisors intervene. Supervision Sufficiency asks whether the total human supervisory capacity remains adequate for the autonomy, complexity and consequence involved.

Can high reliance be appropriate?

Yes. A reliable AI system performing a well-understood, low-consequence task may deserve substantial reliance. The goal is not to minimize reliance but to keep it calibrated to the actual conditions of the work.

Why should organizations monitor reliance over time?

The relationship can change even when the approved system and formal workflow remain the same. Growing familiarity, changing workload, additional delegation, increasing autonomy or declining human capability can all alter how employees rely on AI.