Measuring & Documenting Training Outcomes
Making Project Management More Effective
Measuring & Documenting Training Outcomes LERN ANNUAL CONFERENCE – Orlando, FL – November 20‐23, 2014
Presented by
Richard T. Walsh, Ph.D. – LERN Consultant
Successful TRAINING is a mutual conversation among two or more individuals that follows a predictable process and leads to superior performance, commitment to sustained improvement, and positive relationships!
Richard T. Walsh, Ph.D. ‐ LERN Consultant
Making Project Management More Effective
Measurement Issues Training outcomes are difficult to measure because they are: Intangible Difficult to quantify Invisible Not tracked through accounting Not easily controlled Subject to multiple influences Cumulative Behavior‐based Not time‐bound
The Training Proposition If you develop employee skills
You influence quality
You reduce cycle time
You increase performance
Exceed customer expectations
Create customer loyalty
Produces return on investment
Richard T. Walsh, Ph.D. ‐ LERN Consultant
Making Project Management More Effective
Training Design Sets the Stage DIMENSIONS
CONTENT‐BASED
FOCUS • Covers information SHORT‐TERM • Training took place OUTCOMES • People were exposed to concepts LONG‐TERM • Training took place OUTCOMES • People were exposed to concepts
OUTCOME • How many people attended and MEASURE for how long NATURE • Event
CPMPETENCY‐BASED • Develops skills and abilities to achieve business goals • Increased capability of participants • People changed attitudes and increased skills • Impact on capability and productivity • Changes in performance – individual and organization • Performance Improvement • Impact on operations/ organization • Strategy
Develop Effective Training KPIs
Richard T. Walsh, Ph.D. ‐ LERN Consultant
Making Project Management More Effective
10 Step Process 1. 2. 3. 4. 5. 6. 7. 8. 9. 10.
Identify the need or problem Define the desired outcomes for participants and for the organization Identify the changes in performance that need to be achieved Identify strategy to measure the current performance and results ‐‐ Measure Determine that training is the appropriate strategy to address the situation Design the training to develop and reinforce competencies and improve performance Develop a plan to measure the impact of training on performance and results Deliver the training Measure immediate effects of the training Measure long‐term effects of the training
Kirkpatrick’s Four‐Level Training Evaluation Model 1. Reaction Determine how your trainees reacted to the training
2. Learning Measure what your trainees have learned
3. Behavior Evaluate how far your trainees have changed their behavior, based on the training they received
4. Results Analyze the final results of your training including outcomes that you have determined to be good for business, good for the employees, or good for the bottom line
Richard T. Walsh, Ph.D. ‐ LERN Consultant
Making Project Management More Effective
Using Kirkpatrick’s Model • Level 1 (Reaction) • completed participant feedback questionnaire • informal comments from participants • focus group sessions with participants
• Level 2 (Learning) • pre‐ and post‐test scores • on‐the‐job assessments • supervisor reports
• Level 3 (Behavior) • completed self‐assessment questionnaire • on‐the‐job observation • reports from customers, peers and participant's manager
• Level 4 (Results) • financial reports • quality inspections • interview with sales manager
Potential Sources for Outcomes Data • Hardcopy and online quantitative reports • Production and job records • Interviews with participants, managers, peers, customers, suppliers and regulators • Checklists • Tests • Direct observation • Questionnaires, self‐rating and multi‐rating • Focus Group sessions
Richard T. Walsh, Ph.D. ‐ LERN Consultant
Making Project Management More Effective
Potential Outcomes • Better productivity • Lower insurance premiums • Improved morale • Enhanced reputation • Reduced stress • No legal costs • No compensation cases • Lower staff turnover • Employer of choice
Richard T. Walsh, Ph.D. ‐ LERN Consultant
Measuring & Documenting Training Outcomes LERN ANNUAL CONFERENCE – Orlando, FL – November 20‐23, 2014
Presented by
Richard T. Walsh, Ph.D. – LERN Consultant
Successful TRAINING is a mutual conversation among two or more individuals that follows a predictable process and leads to superior performance, commitment to sustained improvement, and positive relationships!
Richard T. Walsh, Ph.D. ‐ LERN Consultant
Making Project Management More Effective
Measurement Issues Training outcomes are difficult to measure because they are: Intangible Difficult to quantify Invisible Not tracked through accounting Not easily controlled Subject to multiple influences Cumulative Behavior‐based Not time‐bound
The Training Proposition If you develop employee skills
You influence quality
You reduce cycle time
You increase performance
Exceed customer expectations
Create customer loyalty
Produces return on investment
Richard T. Walsh, Ph.D. ‐ LERN Consultant
Making Project Management More Effective
Training Design Sets the Stage DIMENSIONS
CONTENT‐BASED
FOCUS • Covers information SHORT‐TERM • Training took place OUTCOMES • People were exposed to concepts LONG‐TERM • Training took place OUTCOMES • People were exposed to concepts
OUTCOME • How many people attended and MEASURE for how long NATURE • Event
CPMPETENCY‐BASED • Develops skills and abilities to achieve business goals • Increased capability of participants • People changed attitudes and increased skills • Impact on capability and productivity • Changes in performance – individual and organization • Performance Improvement • Impact on operations/ organization • Strategy
Develop Effective Training KPIs
Richard T. Walsh, Ph.D. ‐ LERN Consultant
Making Project Management More Effective
10 Step Process 1. 2. 3. 4. 5. 6. 7. 8. 9. 10.
Identify the need or problem Define the desired outcomes for participants and for the organization Identify the changes in performance that need to be achieved Identify strategy to measure the current performance and results ‐‐ Measure Determine that training is the appropriate strategy to address the situation Design the training to develop and reinforce competencies and improve performance Develop a plan to measure the impact of training on performance and results Deliver the training Measure immediate effects of the training Measure long‐term effects of the training
Kirkpatrick’s Four‐Level Training Evaluation Model 1. Reaction Determine how your trainees reacted to the training
2. Learning Measure what your trainees have learned
3. Behavior Evaluate how far your trainees have changed their behavior, based on the training they received
4. Results Analyze the final results of your training including outcomes that you have determined to be good for business, good for the employees, or good for the bottom line
Richard T. Walsh, Ph.D. ‐ LERN Consultant
Making Project Management More Effective
Using Kirkpatrick’s Model • Level 1 (Reaction) • completed participant feedback questionnaire • informal comments from participants • focus group sessions with participants
• Level 2 (Learning) • pre‐ and post‐test scores • on‐the‐job assessments • supervisor reports
• Level 3 (Behavior) • completed self‐assessment questionnaire • on‐the‐job observation • reports from customers, peers and participant's manager
• Level 4 (Results) • financial reports • quality inspections • interview with sales manager
Potential Sources for Outcomes Data • Hardcopy and online quantitative reports • Production and job records • Interviews with participants, managers, peers, customers, suppliers and regulators • Checklists • Tests • Direct observation • Questionnaires, self‐rating and multi‐rating • Focus Group sessions
Richard T. Walsh, Ph.D. ‐ LERN Consultant
Making Project Management More Effective
Potential Outcomes • Better productivity • Lower insurance premiums • Improved morale • Enhanced reputation • Reduced stress • No legal costs • No compensation cases • Lower staff turnover • Employer of choice
Richard T. Walsh, Ph.D. ‐ LERN Consultant
