You know security awareness training is changing behavior when you can observe a relevant change in what people do, show that the change persists beyond the immediate training event, and connect it to the risk the organization is trying to reduce. Completion rates and quiz scores can tell you whether people participated or learned something. Phishing simulations and other behavioral signals can show whether people acted differently under specific conditions. Neither automatically proves that the underlying behavior changed sustainably or that organizational cyber risk decreased.
That distinction matters because security awareness programs are increasingly expected to demonstrate outcomes, not activity. The useful question is no longer simply whether employees completed training. It is what changed, why it changed, whether that change lasted, and whether it made the organization meaningfully more resilient.
At Cybermaniacs, we also do not treat behavior as a binary variable: secure or insecure, clicked or did not click, compliant or non-compliant. Human behavior is produced within a context. To understand whether a security program is working, we need to understand the behavior itself, the psychological factors influencing it and the cultural conditions that make the behavior easier or harder to sustain.
What does security awareness training actually measure?
Security awareness programs can generate several different kinds of evidence, and they should not be treated as interchangeable.
A completion rate tells you that an assigned activity was completed. A quiz can provide evidence that someone understood or remembered information. A phishing simulation can show how someone responded to a constructed scenario. Reporting behavior, security events, help-desk data, policy exceptions and other operational signals can provide additional evidence about what people do in the course of real work.
Each measure tells us something different.
This is why Cybermaniacs distinguishes between activity, knowledge, observed behavior, behavioral explanation, sustained change and risk outcome. Moving from one level to the next requires more evidence and more careful interpretation.
Our guide to how to measure human cyber risk explores this progression in more depth, including the difference between events, metrics, signals, patterns, risk conditions, interventions and outcomes.
Does completing security awareness training prove behavior change?
No. Completion proves participation.
That does not make completion meaningless. Organizations still need to know whether required learning was delivered, who received it and whether regulatory or policy obligations were met. Completion data is an important operational measure.
The mistake is asking that measure to answer a different question.
An employee can complete a course without retaining the information. They can retain the information without applying it. They can know exactly what the secure action should be and still behave differently when faced with time pressure, conflicting incentives, confusing processes or social expectations.
Recent research illustrates why this distinction deserves attention. A 2026 study of 965 UK participants found that workplace cybersecurity training was associated with greater self-reported familiarity and confidence, while the researchers did not find corresponding improvements in knowledge of the security practices they tested. That does not mean security training is ineffective. It shows why participation, confidence, knowledge and behavior should be measured separately rather than treated as evidence of the same outcome.
Are phishing simulations evidence of behavior change?
Phishing simulations can provide useful behavioral evidence because they measure what someone does in response to a scenario rather than asking what they know or intend to do.
If an employee who previously clicked simulated phishing messages begins reporting them instead, that is a real observed change within the simulation environment. If that improvement persists across different scenarios and over time, the evidence becomes stronger.
But even here, interpretation matters.
A large field experiment involving 670 organizations and roughly 33,000 employees found that phishing drills produced only limited and inconsistent longer-term reductions in clicking behavior across the population studied. The researchers also found differences associated with characteristics such as trust and risk tolerance, illustrating that people do not respond uniformly to the same intervention.
Other research shows that the design of the intervention itself can matter. A 2026 organizational study involving more than 4,000 employees found that immediate feedback following simulated phishing interactions reduced susceptibility and improved subsequent performance compared with conditions without feedback.
The useful conclusion is not that phishing simulations work or do not work. It is that simulation results are evidence produced under specific conditions, and the design of the intervention, characteristics of the population, organizational context and passage of time can all affect the result.
What is the difference between observed behavior and sustainable behavior change?
Observed behavior tells you what happened in a particular situation. Sustainable behavior change means that the relevant behavior becomes more reliable across time and sufficiently varied conditions.
Cybermaniacs takes this a step further by focusing on the sustained adaptation of preventative and protective actions. Our behavioral model is designed to help distinguish the actions people take to prevent risk, protect themselves and the organization, respond to risk conditions, and reduce the likelihood that human error becomes a security event.
A behavioral taxonomy helps create consistency around what is being observed. It gives the organization a shared vocabulary for distinguishing one type of security-relevant action from another rather than collapsing everything into a generic category of “good” or “bad” behavior.
But a taxonomy alone is not enough.
Knowing what a behavior is does not explain why it occurred.
That distinction is easy to miss when dashboards show a percentage moving in the right direction. Suppose phishing reporting rises from one campaign to the next. That may represent genuine improvement. It may also reflect familiarity with the simulation format, an unusually easy scenario, recent communications about phishing, or temporary attention immediately following training.
The next measurement should therefore ask whether the behavior transfers.
Does the employee respond similarly to a different kind of social-engineering attempt? Does reporting remain strong several months later? Do real security incidents show a related improvement? Are different employee populations changing in the same way? Does the behavior hold when employees are busy, distracted or operating under pressure?
This is one reason the Cybermaniacs Human Risk Management Capability Map treats Measure Risk & Change as part of a larger cycle rather than as a reporting function at the end of a training program. Measurement should help the organization learn whether its interventions are producing the intended effect and what needs to change next.
Why isn't a behavioral taxonomy enough?
A behavioral taxonomy can tell you what type of action occurred. It can create structure around preventative actions, protective actions, risky actions, reporting behaviors and other observable events. That is valuable because without a consistent model, organizations can end up comparing fundamentally different behaviors as though they mean the same thing.
But an ontology or taxonomy primarily helps answer what is this?
Human Risk Management also needs to ask why did it happen this way?
Cybermaniacs connects behavioral evidence to a psychological model because the same observable action can be shaped by very different cognitive and psychological factors. Cognitive biases, confidence, risk perception, trust, attention, habit and protective traits can influence how a person interprets a situation and chooses to act.
That changes the intervention decision.
If a behavior is driven by lack of recognition, the answer may be learning and practice. If it is influenced by an authority bias, excessive trust or time pressure, simply repeating the same information may have little effect. If an employee already has strong protective habits, the organization may need to reinforce those strengths rather than treating them as another person who needs remediation.
The behavior tells us where to look. Psychology helps us understand what may be shaping it.
Why does culture matter when measuring security behavior?
Because individual behavior does not happen in isolation.
Culture determines which behaviors are reinforced, discouraged, copied, questioned or quietly tolerated. It influences whether people feel able to challenge an unusual request, report a mistake, slow down a risky process or ask for help when something does not make sense.
Cybermaniacs therefore connects behavioral and psychological evidence to a broader culture model. We look at whether differences in values, beliefs, norms and interpretation help explain why the same policy or security expectation produces different behaviors across teams, roles or locations.
Consider two departments exposed to the same training, policies and technical controls. One develops strong reporting habits while the other repeatedly bypasses the same process. The difference may not be knowledge. It may lie in how managers respond to mistakes, whether speed is valued above caution, how employees interpret the policy or whether challenging authority is considered acceptable.
A behavior measured without that context can easily be misinterpreted.
This is why we do not think of human risk as simply a collection of individual employee scores. The unit of analysis sometimes needs to be the person, sometimes the team, sometimes the workflow and sometimes the surrounding organizational system.
Can better phishing results prove that cyber risk has decreased?
Not by themselves.
A phishing simulation measures behavior related to a particular class of scenarios. Improved performance is useful evidence, but organizational cyber risk depends on more than one employee action and more than one threat pathway.
The same principle applies to other human-risk metrics. Fewer policy violations, increased reporting, lower simulated click rates or stronger quiz performance can all be encouraging. The question is how strongly those measures connect to the risk condition the organization actually cares about.
This is where measurement validity becomes important.
If the business risk is fraudulent payments caused by executive impersonation, for example, generic phishing click rates may be only one piece of evidence. The organization may also need to understand verification behavior, escalation practices, approval processes, exposure among high-risk roles and whether employees can challenge an unusual request when it appears to come from someone senior.
The metric should follow the risk, rather than the risk being defined by whatever metric the platform happens to produce.
That is also why Human Risk Management is broader than security awareness training. Awareness can be one of the interventions used to influence risk, while Human Risk Management asks the larger questions of what risk exists, what contributes to it, which response is appropriate and whether the result actually improved.
Why does understanding the cause of behavior matter?
Because two people can exhibit the same risky behavior for completely different reasons.
Imagine two employees who both fail the same phishing simulation.
The first does not recognize the warning signs. Additional education, practice and immediate feedback may be exactly what they need.
The second recognizes that the message looks suspicious but is working in an environment where urgent executive requests are expected to be handled immediately. They are juggling interruptions, customer demands and an organizational norm that rewards speed. Their knowledge may be perfectly adequate while the conditions surrounding the decision consistently push them toward the risky action.
The behavioral signal is the same. The intervention should probably not be.
This is the difference between behavioral observation and behavioral explanation that we introduced in our guide to platforms that can help change employee security behavior. Human Risk Management becomes much more useful when behavioral evidence helps trigger investigation rather than prematurely ending it.
At Cybermaniacs, that investigation moves across three connected lenses:
Behavior: What action occurred, and does it support or undermine preventative and protective security outcomes?
Psychology: What cognitive biases, protective traits, perceptions or decision factors may help explain why it occurred?
Culture: What norms, values, beliefs, incentives or interpretations may be reinforcing that behavior across the organization?
None of those lenses is sufficient alone.
Together, they provide a much stronger basis for deciding what the organization should actually do next.
What should organizations measure instead of completion rates?
Organizations do not necessarily need to stop measuring completion. They need to stop treating it as the final outcome.
A more useful measurement chain might look like this:
| Measurement level | What it tells you |
|---|---|
| Activity | Did the intervention happen? |
| Knowledge / capability | Did people understand or learn something? |
| Observed behavior | Did people act differently under the measured conditions? |
| Behavioral context | What psychological, cultural or operational factors may influence the behavior? |
| Behavioral explanation | What appears to be driving the behavior, and does the evidence support that interpretation? |
| Sustained adaptation | Do preventative or protective behaviors persist across time and different conditions? |
| Risk outcome | Is there credible evidence that the relevant organizational risk improved? |
Not every awareness initiative needs an elaborate experimental design. The level of measurement should be proportionate to the importance of the decision being made.
A short communication about a routine security reminder may need very little measurement. A large enterprise program claiming to materially reduce a significant human-related cyber risk should be held to a higher evidentiary standard.
This is part of the broader argument in What Should Human Risk Management Actually Measure?: the goal is not simply to collect more data about people. It is to build enough evidence to make better risk decisions.
How should a CISO determine whether security awareness is working?
Start by defining the outcome before choosing the metric.
What specifically are you trying to change? What preventative or protective behavior would indicate improvement? Under what conditions does that behavior need to occur? Which evidence would give you reasonable confidence that the change is genuine? How long should the improvement persist before you consider it meaningful?
Then look at the context surrounding the behavior.
What does the behavioral evidence show? What psychological factors may influence the action? Do culture, norms, values or local working conditions help explain why one population responds differently from another? Is the intervention addressing the actual cause or simply reacting to the observable signal?
Finally, look at the chain between intervention and outcome.
If training completion increased but behavior did not, the intervention may need to change. If simulated behavior improved but real-world outcomes did not, the simulation may not be measuring the right thing. If one population improves and another does not, the difference may reveal something important about role, exposure, psychology, culture or working conditions.
The goal is not to prove that the awareness program worked.
The goal is to learn what worked, for whom, under what conditions, why it worked, for how long, and whether it changed the risk that mattered.
That is a much more demanding standard than completion.
It is also a much more useful one.
Frequently Asked Questions
How do you measure whether security awareness training works?
Use multiple levels of evidence. Completion measures participation, assessments measure knowledge or capability, simulations and operational signals can measure observed behavior, and longitudinal measurement can help determine whether changes persist. Stronger evaluation also considers the psychological, cultural and operational context influencing the behavior and connects those changes back to the specific organizational risk the program was designed to influence.
Do phishing simulations prove behavior change?
They provide evidence of behavior under simulated conditions. Repeated improvement across varied scenarios and over time strengthens the case that behavior is changing, but simulations alone do not prove that the change transfers to every real-world situation or reduces organizational risk.
Are training completion rates useful?
Yes, for measuring participation, delivery and compliance. They should not be treated as direct evidence that employees learned, changed behavior or reduced cyber risk.
Why is context important when measuring security behavior?
The same observable behavior can arise from different causes. Psychological factors, role, culture, workload, incentives, leadership and local working conditions can all influence how people behave. Without context, organizations may select an intervention that addresses the signal without addressing its cause.
Is security behavior binary?
No. Security behavior cannot be reduced reliably to categories such as good or bad, secure or insecure, compliant or non-compliant. Different actions have different meanings depending on the risk, situation and context, and the same behavior can be influenced by different psychological or organizational factors.
What is the best metric for security awareness training?
There is no single best metric. The best measures correspond to the specific risk, behavior and outcome the organization is trying to influence. A combination of activity, capability, behavioral, contextual and outcome measures is usually more informative than relying on one metric alone.
