Phishing Training: What Works and Why?

Phishing Training Effectiveness: Unpicking the apparent distance between academic evidence and vendor Marketing Claims

Summary

Academic research reveals a stark reality about phishing training effectiveness that contradicts vendor marketing claims. A groundbreaking 2025 study from UC San Diego analyzing 12,511 employees found that eight months of multi-modal phishing training produced only a 1.7% improvement in click rates - statistically insignificant with effect sizes below 0.01[1,2]. This finding challenges the entire foundation of the multi-billion dollar security awareness industry, where vendors routinely claim 80% effectiveness rates and 37-fold returns on investment[3]. The truth is that 65% of organizations that suffered ransomware attacks had conducted anti-phishing training[2,3], highlighting the dangerous gap between perceived and actual protection[4].

The academic consensus from over 42 peer-reviewed studies shows that traditional annual training fundamentally fails to create lasting behavioral change[5,6]. Training effects decay rapidly, with knowledge retention lasting between 7 days and 5 months maximum before performance returns to baseline levels[7,8]. Most critically, researchers have identified a fundamental "knowing-doing gap" where users who pass knowledge tests and understand phishing concepts still fall victim to attacks at nearly identical rates to untrained users, because real-world phishing exploits automatic behavioural responses rather than conscious knowledge.

The Frequency Paradox: Why Annual Training Creates Vulnerability

Traditional annual security awareness training represents one of cybersecurity's most persistent yet ineffective practices. A comprehensive analysis of 19,789 personnel over 8 months found no relationship between time since annual training and phishing susceptibility - employees were equally likely to click malicious links one month versus one year after training[1,9]. The research revealed that more than 50% of training sessions ended within 10 seconds of starting, with only 24% of participants completing courses[1]. Most damaging, employees who completed multiple static training sessions became 18.5% more likely to fall for phishing, suggesting that ineffective training creates dangerous overconfidence[1].

The academic literature consistently identifies optimal training intervals far more frequent than industry practice. Meta-analyses recommend quarterly training as the absolute minimum, with monthly intervals showing superior outcomes[7,8]. Research by Schroeder et al. (2020) established that performance degradation becomes significant after 5 months, making annual programs essentially useless for sustained protection[7]. Organizations implementing continuous training see dramatic improvements: 28% increase in real threat reporting compared to just 7% for periodic quarterly training[10], with some achieving 10x improvements in threat detection rates[11].

The evidence for continuous reinforcement is overwhelming. Healthcare sector studies tracking 5,416 employees across 20 campaigns found that while overall click rates declined with repeated exposure, high-risk "repeat offenders" who clicked 5+ phishing emails remained 10-25% susceptible even after mandatory remedial training[9]. Only through continuous, adaptive training delivered every 10 days did organizations achieve meaningful behavioral change, with failure rates dropping from 34% to below 2% over 12 months[11].

Personalization and Interactivity Transform Training Outcomes

The effectiveness gap between personalized, interactive training and generic awareness content is substantial and measurable. Interactive training programs achieve 19% reduction in phishing failures compared to 9.5% for generic approaches, while requiring 10-20% less time investment[12,13]. A large-scale Israeli financial institution study of 5,000 employees found personalized phishing emails received 1.5-2x more clicks initially but enabled targeted remediation that brought all proficiency groups to equivalent post-training levels[14], effectively eliminating skill gaps across the organization[14].

Gamification emerges as a powerful engagement multiplier, producing 60% increases in user participation and 43% improvements in productivity metrics[15,16]. Role-playing games improved threat identification ability by 36.7%, substantially outperforming traditional lecture-based methods[14,17]. The German public sector's systematic 12-month field study of 409 participants revealed that video-based and interactive training maintained effectiveness for over 6 months, while text-based approaches degraded within just 4 weeks[8].

Adaptive training systems powered by artificial intelligence represent the cutting edge of effectiveness. These platforms create individualized learning paths based on user performance, role, and threat exposure, achieving up to 40x higher engagement rates than static programs[18,19,20]. Organizations using adaptive systems report 500% improvements in resilience ratios and 10x higher real threat reporting rates[11,21]. Microsoft's Attack Simulation Training and similar platforms automatically adjust content difficulty and frequency based on individual performance metrics, reducing administrative overhead by 90% while dramatically improving outcomes.

Just-in-Time Feedback Mechanisms Drive Behavioral Change

Immediate feedback at the point of failure represents one of the most effective interventions identified in the research. A Cambridge study of 11,000+ employees found that just-in-time feedback reduced susceptibility by 10 percentage points for high-risk users (from 50% to 40% failure rate) while increasing threat reporting by 3 percentage points[12,22]. Users spent an average of 97 seconds engaging with embedded training materials delivered immediately after clicking simulated phishing links, compared to just 37 seconds for traditional training content[12].

The psychological mechanisms underlying teachable moments are well-documented. Research identifies a critical window of approximately 2 days where users remain receptive to learning from mistakes, though a 24-hour delay is recommended to allow emotional processing while maintaining relevance[22]. Contextual interactive feedback - where users see their actual clicked email with suspicious elements highlighted - produces the strongest retention and behavior modification[23,12]. Organizations implementing these systems see failure rates drop from 47.5% in untrained groups to 24.5% with embedded just-in-time training[23,12].

Longitudinal tracking over 12+ months demonstrates sustained effectiveness when immediate feedback is combined with continuous reinforcement. Hoxhunt's global study of 2.5 million users showed success rates improving from 34% to 74% after 12 simulations, reaching 80% after 14+ simulations. Financial services achieved the highest industry-specific outcomes at 74% success rates, while healthcare and retail sectors reached 62% and 61% respectively. Critically, 50% of users began reporting real threats by 6 months, increasing to 66% by 12 months[21].

Knowledge Retention Versus Behavioral Change Reveals Training's Fatal Flaw

The most damning finding in phishing training research is the disconnect between knowledge acquisition and behavioral change. Despite users demonstrating understanding of phishing concepts and passing knowledge tests, their actual click rates remain virtually unchanged. The UC San Diego fintech study found no statistically significant improvements in either click rates (p=0.450) or report rates (p=0.417) despite extensive multi-modal training[2]. Paradoxically, University of Maryland research revealed that students who identified themselves as understanding phishing definitions showed higher susceptibility rates than peers with no knowledge whatsoever[17,24].

This "knowing-doing gap" stems from fundamental differences in cognitive processing. Phishing attacks exploit System 1 thinking - fast, automatic, intuitive responses - while training targets System 2 - slow, deliberate, analytical processing. Under realistic workplace conditions with time pressure, competing priorities, and cognitive load, users default to automatic responses that bypass conscious security knowledge. Research consistently shows that 70% of users who open phishing emails click on them[24], regardless of prior training, because habitual email processing behaviors override learned security practices.

The implications are profound. Traditional awareness training creates what researchers term a "dangerous illusion of security improvement" where organizations believe they've addressed human vulnerability when behavioral risk remains unchanged. Studies examining habit formation in cybersecurity contexts find that repeated behaviors shift from conscious prefrontal cortex control to automatic basal ganglia processing, making knowledge-based interventions ineffective against ingrained email habits[25,26]. Only interventions that directly modify automatic behavioral responses show promise for reducing real-world susceptibility.

Vendor Claims Collapse Under Academic Scrutiny

Independent academic research systematically contradicts vendor marketing claims about training effectiveness. While vendors tout "80% reduction in susceptibility" and "37-fold ROI," controlled studies paint a starkly different picture[2]. The comprehensive UCSD study's finding of only 1.7% improvement after 8 months of training directly challenges what researchers diplomatically call "optimistic assessments often promoted by cybersecurity training vendors"[2,1]. Multiple academic voices, including Ariana Mirian (UCSD/Censys) and Christian Dameff (UCSD), explicitly call out the lack of empirical evidence supporting vendor success rates[1,2].

Meta-analyses reveal systematic methodological problems in vendor effectiveness studies. Vendors often use phishing templates aligned with their own training materials, creating artificial success rates. They focus on immediate post-training results within short measurement windows that miss training decay, cherry-pick favorable metrics, and lack proper control groups. Most critically, they conflate correlation with causation, attributing improvements from repeated exposure to their training content rather than the exposure itself.

Real-world evidence further undermines vendor claims. The Cloudian ransomware survey found that 65% of organizations penetrated by phishing had conducted employee anti-phishing training - a damning indictment of training effectiveness[27,28,29,30]. Healthcare institution studies showed mandatory training for high-risk employees had no meaningful impact on click rates, with post-training susceptibility remaining between 10-25% for repeat offenders[9]. The academic consensus is clear: current training methods leave approximately 23% of users susceptible regardless of training intensity[6,4].

Conclusion

The evidence decisively demonstrates that traditional phishing training approaches fail to deliver meaningful security improvements. The knowing-doing gap between security awareness and actual behavior, combined with rapid skill degradation and the dominance of automatic cognitive processing in email interactions, renders annual compliance-focused training essentially worthless. Organizations must abandon the dangerous fiction that human-centered training provides adequate protection against phishing threats.

Effective phishing defense requires continuous, personalized, interactive training with immediate feedback delivered at optimal monthly to quarterly intervals. Adaptive AI-powered systems that adjust to individual performance show 40x engagement improvements over static approaches[18,19,20]. Just-in-time feedback at teachable moments produces measurable behavioral change sustained over 12+ months[12,22]. However, even optimal training approaches cannot overcome fundamental human cognitive limitations. The research strongly supports prioritizing technical controls and process improvements that don't rely on perfect human decision-making, while right-sizing training's role as one component in a comprehensive, layered defense strategy informed by empirical evidence rather than vendor marketing claims.

References

[1] SC Media. (2025). "Phishing training is pretty pointless, researchers find." SC Media. Retrieved from https://www.scworld.com/news/phishing-training-is-pretty-pointless-researchers-find

[2] arXiv. (2025). "Anti-Phishing Training Does Not Work: A Large-Scale Empirical Assessment of Multi-Modal Training Grounded in the NIST Phish Scale." arXiv preprint. Retrieved from https://arxiv.org/html/2506.19899v1

[3] CyberPilot. (2024). "Does phishing training work? Yes! Here's proof." CyberPilot Blog. Retrieved from https://www.cyberpilot.io/cyberpilot-blog/does-phishing-training-work-yes-heres-proof

[4] ACM Digital Library. (2024). "Exploring the evidence for email phishing training: A scoping review." Computers and Security, Vol 139, No C. Retrieved from https://dl.acm.org/doi/10.1016/j.cose.2023.103695

[5] CybSafe. (2024). "Exploring the evidence for email phishing training: A scoping review." CybSafe Research Library. Retrieved from https://www.cybsafe.com/research-library/evidence-for-phishing-training/

[6] ScienceDirect. (2024). "Exploring the evidence for email phishing training: A scoping review." Computers & Security. Retrieved from https://www.sciencedirect.com/science/article/pii/S0167404823006053

[7] SpringerOpen. (2020). "Don't click: towards an effective anti-phishing training. A comparative literature review." Human-centric Computing and Information Sciences. Retrieved from https://hcis-journal.springeropen.com/articles/10.1186/s13673-020-00237-7

[8] CPO Magazine. (2024). "Phishing Awareness Training is Far From Permanent; New Study Shows the Effects Last Only a Few Months." Retrieved from https://www.cpomagazine.com/cyber-security/phishing-awareness-training-is-far-from-permanent-new-study-shows-the-effects-last-only-a-few-months/

[9] PubMed Central. (2019). "Evaluation of a mandatory phishing training program for high-risk employees at a US healthcare system." PMC. Retrieved from https://pmc.ncbi.nlm.nih.gov/articles/PMC6515532/

[10] Hoxhunt. (2025). "Phishing Trends Report (Updated for 2025)." Retrieved from https://hoxhunt.com/guide/phishing-trends-report

[11] Hoxhunt. (2024). "4 Essential Phishing Metrics to Reduce Risk." Hoxhunt Blog. Retrieved from https://hoxhunt.com/blog/4-essential-phishing-metrics

[12] Proofpoint. (2024). "Phishing Training Efficacy: Maximize Simulation Learning." Retrieved from https://www.proofpoint.com/us/blog/security-awareness-training/phishing-training-efficacy-maximize-simulation-learning

[13] PubMed Central. (2022). "How Good Are We at Detecting a Phishing Attack? Investigating the Evolving Phishing Attack Email and Why It Continues to Successfully Deceive Society." PMC. Retrieved from https://pmc.ncbi.nlm.nih.gov/articles/PMC8864450/

[14] ScienceDirect. (2023). "Evaluating organizational phishing awareness training on an enterprise scale." Retrieved from https://www.sciencedirect.com/science/article/abs/pii/S0167404823002742

[15] Keepnet Labs. (2024). "The Power of Gamification in Security Awareness Training." Retrieved from https://keepnetlabs.com/blog/the-power-of-gamification-in-security-awareness-training

[16] Safetechinnovations. (2024). "The Importance of Phishing Training & Awareness - Does Content Matter?" Retrieved from https://www.safetechinnovations.com/the-importance-of-phishing-training-awareness-does-content-matter

[17] Frontiers. (2021). "Phishing Attacks: A Recent Comprehensive Study and a New Anatomy." Frontiers in Computer Science. Retrieved from https://www.frontiersin.org/articles/10.3389/fcomp.2021.563060/full

[18] Hoxhunt. (2024). "No.1 Phishing Training for Employees." Retrieved from https://hoxhunt.com/product/phishing-training

[19] Hoxhunt. (2024). "4 Essential Phishing Metrics to Reduce Risk." Retrieved from https://hoxhunt.com/blog/4-essential-phishing-metrics

[20] Hoxhunt. (2025). "Phishing Trends Report (Updated for 2025)." Retrieved from https://hoxhunt.com/guide/phishing-trends-report

[21] Hoxhunt. (2025). "Phishing Trends Report: Global Analysis of 2.5 Million Users." Retrieved from https://hoxhunt.com/guide/phishing-trends-report

[22] Cambridge University Press. (2023). "Phishing feedback: just-in-time intervention improves online security." Behavioural Public Policy. Retrieved from https://www.cambridge.org/core/journals/behavioural-public-policy/article/phishing-feedback-justintime-intervention-improves-online-security/4F5DF23A7AB0DC81561A1778E06802E2

[23] ScienceDirect. (2023). "Cognitive elements of learning and discriminability in anti-phishing training." Retrieved from https://www.sciencedirect.com/science/article/abs/pii/S0167404823000159

[24] ResearchGate. (2019). "Phishing in an academic community: A study of user susceptibility and behavior." Retrieved from https://www.researchgate.net/publication/335162516

[25] Liebert Pub. (2021). "Exploring Workers' Subjective Experiences of Habit Formation in Cybersecurity: A Qualitative Survey." Cyberpsychology, Behavior, and Social Networking. Retrieved from https://www.liebertpub.com/doi/10.1089/cyber.2020.0631

[26] PubMed. (2021). "Exploring Workers' Subjective Experiences of Habit Formation in Cybersecurity: A Qualitative Survey." Retrieved from https://pubmed.ncbi.nlm.nih.gov/34403600/

[27] GlobeNewswire. (2021). "Cloudian Ransomware Survey Finds 65% of Victims Penetrated by Phishing Had Conducted Anti-Phishing Training." Retrieved from https://www.globenewswire.com/news-release/2021/07/15/2263492/

[28] Intelligent CIO APAC. (2021). "Survey finds 65% of victims penetrated by phishing had conducted anti-phishing training." Retrieved from https://www.intelligentcio.com/apac/2021/07/19/

[29] ACM Digital Library. (2024). "Exploring the evidence for email phishing training: A scoping review." Computers and Security, Vol 139. Retrieved from https://dl.acm.org/doi/10.1016/j.cose.2023.103695

[30] CyberPilot. (2024). "Does phishing training work? Yes! Here's proof." Retrieved from https://www.cyberpilot.io/cyberpilot-blog/does-phishing-training-work-yes-heres-proof