Overview
Evaluated the effectiveness of a statewide cybersecurity training program to determine whether it improved employee behavior and reduced organizational risk. Using Kirkpatrick’s Four Levels of Evaluation, the project assessed learning outcomes, behavioral change, and real-world impact beyond completion metrics.
Role: Instructional Designer / Evaluator
Methods: Surveys, interviews, LMS analytics, phishing simulations, incident reports
Problem / Need
The training achieved a 97 percent completion rate but showed limited impact on employee behavior. Phishing failures remained high, security incidents persisted, and employees viewed the training as a routine compliance task rather than a meaningful learning experience. The core issue was a disconnect between training completion and real-world application.
Analysis
I evaluated the program across reaction, learning, behavior, and results using multiple data sources.
Key findings:
- Knowledge gains were modest (62% → 78%)
- 39% of employees failed phishing simulations after training
- Security incidents increased by 14%
- Policy compliance remained low at 60%
- Engagement was minimal, with no use of interactive learning tools
These results indicated that employees were completing the training but not applying it effectively in practice.
Design Strategy
The evaluation focused on measuring performance, not just satisfaction.
Key decisions:
- Use Kirkpatrick’s model to assess real-world impact
- Compare learning gains against behavioral outcomes
- Identify where breakdowns occurred between knowledge and application
- Prioritize redesign recommendations over incremental content updates
This approach revealed that the primary failure point was behavioral transfer, not access to training.
Solution
Delivered a comprehensive evaluation report with targeted redesign recommendations.
Key recommendations:
- Introduce scenario-based and interactive training
- Develop role-specific learning pathways
- Implement ongoing reinforcement through simulations and microlearning
- Provide targeted follow-up for high-risk employees
- Increase leadership involvement to reinforce security culture
The solution reframed the program from a compliance requirement to a behavior-driven learning system.
Evaluation & Reflection
This project demonstrates my ability to evaluate training effectiveness using real performance data rather than completion metrics alone.
What worked:
- Clear alignment between evaluation data and recommendations
- Strong use of behavioral and organizational metrics
- Identification of the gap between knowledge and application
What I would improve:
- Add a visual summary of key metrics for faster scanning
- Further reduce secondary data points to emphasize the most critical insights
This project highlights my ability to diagnose why training fails and design improvements that target real-world performance.