View
3
Download
0
Category
Preview:
Citation preview
BMW Manufacturing Corp.Return on Investment Impact Study
Gallery Tourv Review the posters with questions that will be
addressed at each evaluation levelv Complete a Post-It note with additional questions
you want addressed today
Founders of GAINS ProfileDr. Ivan Misner & Robert Davis
Purpose: To strengthen both new and established relationships
Five Elements of the GAINS Profile
1. Goals2. Accomplishments (Assets)3. Interests (Passions)4. Networks (IRS)5. Skills
Results
Let’s Network using the GAINS ProfilePresenter: Ylonda Glover, MS
Tri-Success Management Institute
GAINS ProfileFocus on I for Interests
§People are the Drivers
§Understanding People - Improves ROI
§What are your personal and professional interests?
§What are you most passionate about or what do you like doing?
Toni HodgesTH & Company
-Independent consultant, providing consulting services and workshops to private and public organizations
-Formerly managed Measurement and Evaluation for Verizon Communications and Bell Atlantic
-Bell Atlantic selected APQC Best Practice for “Measuring the Impact of Learning”
-Selected “Training’s New Guard”
-ROI Network Practitioner of the Year
-Author “Linking Learning and Performance”
Michelle M. WentzAssistant ManagerBMW Manufacturing
•Assistant Manager of Training and Associate Development at BMW Manufacturing
Ø 5000-associate company manufacturing the Z4 Roadster and X5 Sport Activity VehicleØMain responsibilities: Learning Technologies, e-Learning, Computer Applications Training, Training Measurement
•12 years of broad-based training and development experience working in the manufacturing and financial services industries
•Master of Science degree in Adult Education from the University of Wisconsin
•Bachelor of Business Administration degree in Management, also from the University of Wisconsin
Overview1. ROI Methodology
2. Program Background
3. BMW Process
4. Results
5. Conclusions
6. Recommendations
7. Lessons Learned
Collect Data After Solution
Implementation
Collect Data During Solution Implementation
Develop Evaluation Plans and
Baseline Data
Develop Objectives of
Solution
DATA COLLECTIONEVALUATION PLANNING
1. Satisfaction /Reaction
3. Application/Implementation
2. Learning 4. Business Impact
ROI Process™Methodology
Provides 6 types of data
Isolate the Effects of the
Solution
Convert Data to Monetary
Values
Calculate the Return on Investment
Tabulate Costs
Identify Intangible Measures
GenerateImpactStudy
REPORTINGDATA ANALYSIS
5. ROI
6. Intangible Benefits
Provides 6 types of data
ROI Process™Methodology
Benefits / Project BenefitsCosts =Ratio Project Costs
Net Project Benefits
ROI =Project Costs
ROI Process™Calculating the BCR and ROI
Leadership Mastery ProgramBackground
-To continue leaders’ professional development after the BMW Life Program
-Emphasis on business competencies, communication, DISC style adapting and conflict resolution skills
-3 classroom days followed by 8 hours of individualized coaching
-Target Associates: Managers, Supervisors, and Individual Contributors with Project Responsibilities
1. Two focus groups convened to determine potential business objectives (21 LMP participants)
2. Reaction data collected from BMW evaluation form (158 LMP participants)
3. Job Application data collected with 360 Degree Assessment (pre and post data collected from 16 out of 34 associates)
4. Business Impact data collected from Leadership Mastery Program Review survey (128 LMP participants responded)
5. Tangible Benefits collected from Leadership Mastery ROI Survey and actual program costs (24 out of 36 LMP participants responded)
6. All data compiled and analyzed by Return On Training (ROT) Analysis Team (3)
7. Logic Map Task Force convened for input into Conclusions/ Recommendations (2 coaches, 4 associates)
Methodology for BMWProcess Steps
-Reaction data high – ratings between 3.8 to 5.0
-Participants saw improvement in 13 of 18 competencies, 4 significantly so:
1. Communicates with Impact2. Fresh Thinking3. Positive Disposition 4. Decision Making
-Training and Coaching perceived to have a positive impact on meeting business and performance objectives
-Those that received more coaching perceived higher impact to business/performance objectives (Impact increased more than threefold for those who had more than 5 coaching sessions)
Results Reaction and Job Application
1. Select the business benefits the Leadership Mastery program most positively impacted.
2. What is the monetary value of each unit of improvement?
3. What was the change in this unit over a 30-day period?
4. To what extent do you credit the Leadership Mastery program for this change?
5. How confident are you of this estimate?
Question
1. Fewer hours spent in meetings
2. $30 per hour
3. 4 hours per month
4. 70% percent of change accredited to Leadership Mastery program
5. 80% confident
Example
ResultsCalculating Tangible Benefits
ResultsTangible Business Benefits
-14 out of 24 respondents were able to estimate tangible benefits(used in calculation)
-“Reduced number of issues reported” received highest estimated tangible benefit of $50,284
-“Fewer hours spent in meetings” was the benefit realized most often (2nd highest tangible benefit)
Participant example from Assembly Quality Department
Ø Reduced number of issues reported (Quality Improved)
Ø 10 issues reduced per month at 3 hours each @$30/hr. Ø Isolation @ 40%Ø Confidence level @ 60%Ø Annual monetary benefit = $259
ResultsBCR and ROI CalculationBenefits(Perceived Annual Monetary Value from 24 Participants -Adjusted for Isolation and Confidence)
Costs(Actual Costs for 24 Participants)
Reduced number of issues reported $50,284Reduced overtime $46,444Fewer hours spent in meetings $27,446 Reduced time to full productivity $14,275Reduced turnover $13,970Fewer associate complaints $9,714Improved customer satisfaction $3,200Reduced minutes in rework $300Improved vehicle quality measures $175
Benefit Attributable to Training: $124,008
Design and Development:($190.80 X 24 associates ) $4,579
Leadership Mastery Delivery Costs: Facilities + Instructor + Materials($1924 X 24 associates) $46,176
Associate ParticipationCosts: $26,910
$57.48/hr (FL IV) X 24 hrs X 4 associates +$53.01/hr (FL V) X 24 hrs X 9 associates +$34.02/hr (FL VI) X 24 hrs X 11 associates )(fully loaded hourly rates)
Report:$150/report X 24 associates $3,600
Coaching:$250/hour X 75 coaching hours $18,750$45.02/hour X 75 coaching hours(Participant time in coaching sessions ) $3,377
Total Training Costs: $103,392 ROI Calculation:
$124,008 (benefit attributable to training) - $103,392 (training costs ) = .199 x 100 =19.9%$103,392 (training costs)
Benefit/Cost Ratio Calculation:
$124,008 (benefit attributed to training) = 1.20 = 1.20:1$103,392 (training costs )
-Improved teamwork – 83%
-Reduced conflict – 74%
-Improved clarity of roles and responsibilities – 61%
–Improved associate job satisfaction – 39%
–Improved associate development plans – 35%
–Increased cross functionality – 30%
–Others noted: improved customer satisfaction, organizing skills, and communication
ResultsIntangible Benefits
1. Participants believed they would be able to apply the skills they learned on the job
2. Leadership competencies improved as a result of the program, according to participants: Decision-making, Communicates with Impact, Positive Disposition, and Fresh Thinking.
3. Training and coaching had a positive impact on participants’ performance
4. Training and coaching may be improved with more realistic and relevant work-related focus
5. A better understanding by participants’ management of LMP skills acquired and value expected would be beneficial
6. The LMP is leading BMW employees toward leadership with a financial return
Conclusions
RecommendationsNext Steps
1. Continue Leadership Mastery program
2. Share ROI project results with key audiences
3. Prioritize and implement LMP recommendations
4. Implement the following measurement best practices:
- Change our training design process to include the UPFRONT development of business objectives for future courses. Seek input from Senior Management so that training programs can be strategically linked to BMW’s business/ organizational objectives.
- Focus measurement efforts on learning and on-the-job application / performance impact
- Measure ROI and Business Impact for a few select training programs
Lessons LearnedPutting the Pieces Together…
•Training Design process must include link to business need, initiative, or improvement area
ØLearning objectives are not enough
ØObjective mapping process is key
ØProvides strategic link to business
ØEnsures concrete measures
•Prepare participants for accountability
ØEspecially when collecting data using participant estimates
ØTrain instructors, designers, and learners in documenting, and tracking tangible benefits
ØAction plans, business objectives, course design play a key role
Lessons LearnedPutting the Pieces Together…
•Partner with Senior Management and Financial departments
ØCommunicate ROI process (for training)
ØGet financial requirements
ØLook for ways to customize to organization in order to establish an ‘approved’ methodology
Lessons LearnedPutting the Pieces Together…
•Keep participation high
ØUtilize ‘field’ perspective whenever possible through focus groups, and task forces
ØInvolve course participants, trainers, designers, managers, etc.
ØProvides credibility, buy-in and more reliable data
Lessons LearnedPutting the Pieces Together…
•Be careful using 360º feedback for performance data (level 3)
ØIn this ROI study, used only as supporting data
Ø360º Surveying is labor intensive
ØDifficult to draw conclusions, since over time, feedback givers will change
Lessons LearnedPutting the Pieces Together…
•Anticipate personal bias
ØExpect parties with vested interest to try to influence study results
ØEvaluator needs to remain neutral and ensure objectivity
ØExplain ROI process (what’s okay and not okay to do)
ØPush back if you need to
Lessons LearnedPutting the Pieces Together…
Executive Response
Ø CFO reaction to “ROI”
Ø VP reaction to “intangibles”
Ø CEO reaction to the importance of training
Ø Overall reaction to the training organization’s link to the business
Lessons LearnedExecutive Response
Executive Response
Ø ISO Matrix Certification
Ø Auditor asked about training measurements, we offered the ROI impact study, and auditor asked for a copy of the study
Ø Received an “Honorable Mention” at Auditor’s Closing meeting as a best practice taking place within the organization!
Lessons LearnedHonorable Mention
Correlation Pyramid
-
Reaction, Learning, Performance, ROE and ROI Data
Reaction, Learning, Performance and ROE Data
Reaction, Learning and Performance Data
Reaction and Learning Data
Reaction Data
Why Are Levels of Evaluation Important?
Thank you for your attention. Let’s take a break!
Michelle M. WentzBMW ManufacturingAssistant Manager, Training and Development
Michelle.Wentz@bmwmc.com
Toni HodgesTH & Company
410-956-0475
Tonihodges@mindspring.comwww.tonihodges.com
Rotating Table Discussions
Process
Everyone chooses a table
3 Rotations: Ø15 minute table discussion
Ø 5 minute for rotation to next table
ØBe prepared to share insights from each table discussion
Debrief
ØInsights from table discussions
ØAdditional Questions
ØSpecific challenges going forward
ØForm Networks for support and idea sharing
Michelle M. WentzBMW ManufacturingAssistant Manager, Training and Development
Michelle.Wentz@bmwmc.com
Toni HodgesTH & Company
410-956-0475
Tonihodges@mindspring.comwww.tonihodges.com
Recommended