Transcript Name of presentation
Learning: Re-Imagined
Beyond Measuring Usage - Tools & Techniques for Demonstrating Program Value SkillSoft and Knowledge Advisors
Workshop Agenda
•
Welcome and Introduction
•
Metrics in Times of Economic Uncertainty
Jeffrey Berk - Knowledge Advisors •
Metrics That Matter ™ Case Study –CNA
Tom Simon and Mike Castillo - CNA •
SkillSoft Tools & Techniques
Heather McCain, Laura Rexford, Lori Turley - SkillSoft •
Wrap up
Economic Uncertainty Emphasizes Human Capital Measurement
Jeffrey Berk KnowledgeAdvisors
Agenda
Measurement in Economic Uncertainty Business Case for Measurement Measurement for Performance Management 29-Apr-20 © 2008, Knowledge
Advisors
4
Bumpy Economic Times
S&P 500 Down Double Digits $700 Billion Bailout DJIA Record Daily Losses Unemployment on the Rise Well Known Brands in Trouble
© 2008, Knowledge
Advisors
29-Apr-20 5
What are Business Leaders Saying?
• Firms that don’t train are 2.5 times more likely to fail than those that do train.
•“Investing now in building new skills will put us in the strongest position as the economy recovers.” •“The skills of our people are our best guarantee of future prosperity - - and the best investment a business can make in challenging times.” Source: UK Commission for Employment and Skills, November 10, 2008 © 2008, Knowledge
Advisors
29-Apr-20 6
The Economy Changed in 16 Years
1992 0.6:1 Intangible/Tangible = HCI (Human Capital Index) 2008 5:1 66% Productivity Gain in 16 Years Market Value
Intangible Assets Intangible Assets
Market Value
Tangible Assets
Gary Becker Wins Nobel Prize in Human Capital Organizations Invest More in L&D, Leadership, HRIS, LMS, Analytics, Performance Management, Engagement & Recruiting
Tangible Assets
S&P 500 Grows at Record Levels
29-Apr-20 © 2008, Knowledge
Advisors
7
Measuring Talent Maximizes Economic Value
Shareholder Returns of companies investing in L&D measurement have 15.18% higher returns than the S&P 500 Source: Human Capital Measurement and Its Impact on Stock Performance, KnowledgeAdvisors, 2008 29-Apr-20 © 2008, Knowledge
Advisors
8
Identify Gaps in Human Capital Processes
Performance Appraisal Succession Planning Leadership Development On-boarding L&D Coaching Knowledge Mgmt Forecast Workforce Requirements Recruit, Select, and Hire International Assignment Mobile Workforce Employee Turnover Competencies Assessment © 2008, Knowledge
Advisors
Compensation and Benefits Employee Satisfaction Employee Engagement Work/Life Balance Workforce Diversity 9 29-Apr-20
Case: Talent Gap
Background: Printing and publishing company Business Issue: Gaps identified in skilled resources for changing business strategy Measurement Solution: Provided general skills competency set, collected and reported data w/analysis Results: Data used for L&D strategy and for recruiting job descriptions.
© 2008, Knowledge
Advisors
4/29/2020 10
Case: Leadership Evaluation
Background: Major health benefits provider Business Issue: Turnover in first 90 days of employment was 25 to 30% Measurement Solution: Built a leadership program for the Blue Chip hires and used smarter evaluations to measure it and improve it
“Metrics that Matter has provided objective data that supports “gut feelings” about where improvements can be made and where things are going well.” -Director of Leadership & Associate Development
Results: 93% of Blue Chip new hires were retained and no longer flight risks. 73% reduction in admin burden in the measurement process itself
© 2008, Knowledge
Advisors
4/29/2020 11
Case: Template Links L&D to Sales
Background: Consumer Packaged Goods firm Business Issue: Needed reasonable evidence of sales impact from training Measurement Solution: Used actual results and reasonable assumptions via impact template Results: Proved that $74,256/person of sales dollars were tied to a $5,000/person training investment.
Example of template used to showcase ROI from Sales training Source: KnowledgeAdvisors © 2008, Knowledge
Advisors
4/29/2020 12
Case: ERP Technology Company
Background:
ERP company performing customer education
Business Issue:
Show value of training to
Business Results: Productivity Analyze & monetize this result
Total % improvement in productivity including training Training's contribution to improved productivity Percent improvement due to training Adjustment factor for confidence in estimations Adjusted percent improvement due to training customers in terms of increased productivity and quality
Business Results: Quality Analyze & monetize this result
Total % improvement in quality, including training
Measurement Solution:
Collected 12,770 Training's contribution to improved quality Percent improvement due to training Adjustment factor for confidence in estimations Adjusted percent improvement due to training end of program evaluations within 6 months asking participants to forecast impact
Analytics Results:
18.2% estimated
Business Results: Customer Satisfaction Analyze & monetize this result
Total % improvement in customer satisfaction including training Training's contribution to improved customer satisfaction Percent improvement due to training Adjustment factor for confidence in estimations Adjusted percent improvement due to training increase in productivity from the training (adjusted down for self-reported bias)
Post Event
51.0% 57.0% 29.1% 65.0% 18.9%
Post Event
54.0% 57.0% 30.8% 65.0% 20.0%
Post Event
52.0% 57.0% 29.6% 65.0% 19.3% © 2008, Knowledge
Advisors
29-Apr-20 13
Agenda
Measurement in Economic Uncertainty Business Case for Measurement Measurement for Performance Management 29-Apr-20 © 2008, Knowledge
Advisors
14
Fact
1.
Most organizations invest 1% to 4% of revenue in talent development
source ASTD stated L&D is 2 to 3% of revenue
2.
Most organizations waste 50% or more of their talent development expenses
source Metrics that Matter 600,000 follow up data points is 55%, IDC sites 50% waste, Brinkerhoff cites 80% or more waste
3.
Most organizations waste .5% to 2% of revenue on ineffective talent development
calculation from item 1 x 2 above
4.
Effective investments in predictive analytics increases performance by 2.4%
source IDC
Talent Measurement Results: Increases Performance * Reduces Waste * Slashes Admin Burden © 2008, Knowledge
Advisors
29-Apr-20 15
Fact
According to IDC, programs that use predictive analytics solutions, such as Metrics that Matter, generate returns than those that don’t.
60%
better Example: If one invests $1,000,000 in an L&D program that generates a $1,000,000 profit, without using a predictive measurement solution, effective use of one, could have generated an additional $600,000 in profit.
29-Apr-20 © 2008, Knowledge
Advisors
16
Measurement Impact on Increased Performance
1.
Major services firm realized $200,000 increased annual profit by using Metrics that Matter predictive analytics capabilities. 2.
Through measurement a leading federal agency significantly increased output and business impact without increasing budget over a 3 year period.
3.
Major telecommunications firm using Metrics that Matter was able to increase sales by $1.25 million dollars through better supply chain management.
4.
15,000 hours in improved productivity by a floral company in identifying appropriate training investments. © 2008, Knowledge
Advisors
29-Apr-20 17
Measurement Impact on Reduced Waste
1.
25 retail stores avoided $100,000 of wasted training. Retail store training measured after first of six weeks saw 1 set of ineffective trainers and discontinued using them.
2.
Instructor compared herself to other instructors and changed approach. 6 months of additional sessions performed better to avoid at least $100,000 in scrap learning.
3.
$500,000 in funding gained by measuring scrap learning. Services firm measured training results and found high quality but low impact. The program was retained and additional funding rewarded to consult with managers on increasing impact. 4.
88% of learners not applying training is NOT due to bad training. Waste is due to unsupportive employee environment. A billion dollar company can save at least $1,000,000 if it can reduce waste by just 20% by performance managing the employee environment.
© 2008, Knowledge
Advisors
29-Apr-20 18
Measurement Impact on Reduced Admin
1.
75% reduction in administration costs from automating measurement process vs. manual measurement.
2.
15 seconds to retrieve actionable data for decision-making vs. 15 hours prior to automating the measurement process. This automation of 1 report alone saved over $36,000 per year.
3.
$400,000 savings by major accounting firm that used smarter measurement in a practical way to yield similar results compared to complex and burdensome alternative measurement techniques © 2008, Knowledge
Advisors
29-Apr-20 19
Calculate Performance Gain
$1,000,000,000 Revenue 25% Labor Percentage $250,000,000 Labor Cost 1%
$2,500,000
Increased Productivity from Measurement %
Increased Performance from Measurement
© 2008, Knowledge
Advisors
29-Apr-20 20
Calculate Waste Reduction
Description
$1,000,000,000 Revenue 1% L&D as % of Revenue $10,000,000 L&D Investment 50% Waste or Unrealized Value % $5,000,000 Waste or Unrealized Value 5% Waste Reduction from Measurement %
$250,000 Waste Reduction from Measurement
© 2008, Knowledge
Advisors
29-Apr-20 21
Calculate Admin Savings
Description
$55,000 Salary of 1 FTE admin for metrics $50,000 Cost avoided from unnecessary analysis $50,000 Cost to replication fraction of MTM benchmarks 75% Admin savings from MTM measurement and benchmarks
$116,250 Administration Savings from Measurement
© 2008, Knowledge
Advisors
29-Apr-20 22
Total Value
Value Summary
$2,500,000 Performance Gain $250,000 Waste or Scrap Avoided $116,250 Administration Savings
$2,866,250 Total Quantified Value
© 2008, Knowledge
Advisors
29-Apr-20 23
Agenda
Measurement in Economic Uncertainty Business Case for Measurement Measurement for Performance Management 29-Apr-20 © 2008, Knowledge
Advisors
24
Talent and L&D Measurement Tips
Identify and Audit Gaps in Human Capital Processes Build a Robust Evaluation Process Track and Link to Actual Business Results Reduce Errors and Scrap Leverage Technology Use Metrics for Performance Management
© 2008, Knowledge
Advisors
29-Apr-20 25
Benchmarks for Performance Management
Benchmarks: Internal or External Provide a point of reference Motivate by example Available thru Metrics that Matter 29-Apr-20 © 2008, Knowledge
Advisors
26
Goals for Performance Management
Goals: Actual to Goal Set in Metrics that Matter Move in right direction Challenging yet attainable 29-Apr-20 © 2008, Knowledge
Advisors
27
Trends for Performance Management
Trends: Time based analysis Available in Metrics that Matter Shows improvement over time 29-Apr-20 © 2008, Knowledge
Advisors
28
Dashboards for Performance Management
Dashboards: •Concise Data •Summary Data •Visual View •Available in Metrics that Matter for course managers, instructors, impact and operations 29-Apr-20 © 2008, Knowledge
Advisors
29
Track & Link to Actual Business Results
A template for the learning team to conduct more comprehensive business impact and ROI analysis. Templates bring stewardship and collaborative guidance to the process of linking training to results.
© 2008, Knowledge
Advisors
29-Apr-20 30
Build a Robust Evaluation Process
© 2008, Knowledge
Advisors
Agenda
Measurement in Economic Uncertainty Business Case for Measurement Measurement for Performance Management 29-Apr-20 © 2008, Knowledge
Advisors
32
Knowledge
Advisors
Experts in metrics that matter Knowledge
Advisors analytics
is a
human capital
firm. We are the world’s
largest provider
of learning and talent measurement solutions. 29-Apr-20 © 2008, Knowledge
Advisors
33
KnowledgeAdvisors Clients
4/29/2020 © 2008, Knowledge
Advisors
34
www.knowledgeadvisors.com
For further information on the information contact: Jeffrey Berk Chief Operating Officer +1 312 676 4411 (phone) +1 312 676 4401 (fax) [email protected]
29-Apr-20 © 2008, Knowledge
Advisors
Page 35
Measurement at CNA Measurement at CNA Tom Simon & Michael Castillo
Area of Presenter
Measurement CNA At A Glance
• 1 million business and professionals insured • 7th largest commercial lines insurer • U.S. and international operations • 9,000 employees • $7.8 billion in revenues • Assets of $52 billion • “A” rated for financial strength by A.M. Best
Measurement
Measurement
Measurement
Measurement
Measurement
Why ROI?
Benefits of ROI:
• Business cases benefit strongly from ROI • Measuring financial impact helps illustrate importance of learning • Value-based measurements support Learning Growth Model progression
Challenges of ROI:
• Limited time and resources • Hard to evaluate ROI for intangibles (learning) • Securing internal data difficult
SkillSoft Measurement Tools
EZ ROI Calculator Business Impact Analysis Value Measurement Toolkit
time, resources, data involved estimate vs. rigorous measure
• • •
EZ ROI Calculator - “soft measure” Business Impact Analysis – Validated by internal survey Value Measurement Toolkit (VMT) – ROI, Phillips Level 5
EZ ROI Calculator
EZ ROI Calculator Business Impact Analysis
time, resources, data involved estimate vs. rigorous measure
Value Measurement Toolkit
Objectives and Attributes
Objectives:
• • • Deliver more value-based metrics Help strengthen business cases Prospective use (Should we train?) and retrospective use (How did we do?)
Attributes:
• Easy to use • • • • Requires little time and very little data Defensible Immediate analysis Cost displacement and performance improvement measures
EZ ROI CALCULATOR
Components:
• • • • • Introduction and assumptions Cost displacement calculator Cost displacement sample Productivity impact calculator Productivity impact sample
• • •
Application Ideas for EZ ROI Calculator
Business Impact Analysis
EZ ROI Calculator Business Impact Analysis
time, resources, data involved estimate vs. rigorous measure
Value Measurement Toolkit
Objectives and Attributes
Objectives:
• Measure satisfaction, determine motivation and determine job impact by the following: • • • Determine individual skill requirements and needs Evaluate program satisfaction Gauge skill improvement (based on individual responses/group roll-up) to determine productivity gains
Attributes:
• • • • • • Easy to use Company-specific Requires limited time and data Longer timeline Stronger measurement story Potential testimonials
Business Impact Analysis
Components
• • • •
Custom survey Communications to launch the survey Summary of feedback collected Impact/Results Summary
Sample Survey Questions
1.Job Category 2.Salary Range 3.Would you recommend the SkillSoft resources to a colleague?
4.What was the main reason why you accessed SkillSoft training? 5.Primarily, what skill were you seeking to develop?
6.What percent of your job requires you to use this skill? 7.As a result of the training, how much of a skill increase did you experience?
8.If you did not have access to SkillSoft resources, what alternatives would you have considered? 9.In approximate terms, how much do you think the alternate option would have cost? 10.When are you planning to use SkillSoft resources again?
Drop Down of Job Families Drop Down of Salary Ranges Select One (Yes, Somewhat, No) Drop down with list of likely options plus an “other” option that has a blank field Open short text field Drop down with a range of percentages (0%, 1-10%, 11-20%, 21-30%, 31-40%, 41-50%, 51-60%, 61-70%, 71-80%, 81-90%, 91-100%) Drop down with a range of percentages (0%, 5-10%, 11-15%, 15-20%, 21-25%, 26-30%, More than 30%) Drop down of common training types including “no alternatives” and “other” options Open numerical field, includes the options “no cost” and “not sure” Select One (Not sure, within 2 weeks, within 30 days, within 90 days, more than 90 days)
Output of Survey
•
Survey Results document including:
• • • • Background Survey Responses (graphical representation) Comment Highlights ROI Impact
Application Ideas for Business Impact Analysis
• • Utilize survey results for ongoing marketing and increased usage • Create targeted programs based on skill needs identified
Value Measurement Toolkit
EZ ROI Calculator Business Impact Analysis
time, resources, data involved estimate vs. rigorous measure
Value Measurement Toolkit
Objectives and Attributes
Objectives:
• • • Measure the value derived from the implementation and ongoing support of learning initiatives.
A method offered to aid clients to establish a process for measurement.
Deliver value in terms your stakeholders will believe in – cot savings, productivity impact, increased revenue, etc.
Attributes:
• Longer term study based on a defined process • • • Requires internal, validated data Defensible and credible due to stakeholder input Return on investment
The Value Measurement Process
Notice that the 5 levels are not each a step in the process. The process can be used to any level.
Value Measurement Tools
• • • • •
White Paper Readiness Worksheet Planner Calculator Case Studies
Application Ideas for Value Measurement
• • •
Hands-On Exercise
•
Divide into teams
•
Review your scenario
•
Agree on appropriate tool
•