Manitoba Public Insurance

Quality of Repair - Phase II

Agenda

• Overview of KPIs

• Quality of Repair (QoR)

components

– Repair Capability

– Repair Records on File

– Repair Accuracy

• Interim and final score card

• Support

• Timeline

2

Learning Outcomes

• By the end of this training, you should be able to:

– Summarize the overall Phase II Quality of Repair

(QoR) Key Performance Indicators (KPI)

– Review the Repair Capability component of QoR

– Explain new components of QoR

– Interpret the interim and final score card

– Identify support options

– Summarize the timeline and implementation of Phase

II Quality of Repair (QoR)

3

Overview of KPIs

4

Quality of Repair: Components

5

Component What is it? Purpose Weight Implementation

Schedule

Repair

Capability

Shop Profile-

Based Indicator

– Sets Global

Minimum

Standard

To confirm that a shop’s

technicians are trained for physical

damage repair and track its

capability in complex material

repair as well as access to

recommended tools & equipment

5 Points Implemented

Repair

Records on

File

Sample-Based,

Claim-Specific

Indicator

To ensure that a repair shop has

proper operational procedures

supported by documentation

attached to its claims

5 Points Phase II

Repair

Accuracy

Physical

Inspection-

Based Indicator

To verify that a shop is performing

collision repair as per its stated

capability – covers all accredited

repair shops

15 Points Phase II

Total 25 Points

REPAIR CAPABILITY

6

Minimum Required Skills &

Training

I-Car Gold Class

• Platinum level technicians

in the 4 key-roles

• Continuous learning to

maintain Platinum

designation C

MPI Mandated Training

• MPI Estimating

Standards, Glass and

other future MPI

mandated training

7

I-CAR

NST

EST

RT

SST

* NOTE: Repair Capability

QoR is already implemented

• Steel Welding Certification – Current

– Recorded in the repair shop profile but not scored:

• Tools and equipment a shop has access to (from

Accreditation application)

• Journeyperson / Red Seal status of technicians

• Capability to repair vehicles with complex materials

Minimum Required Skills &

Training

8

* NOTE: Repair Capability

QoR is already implemented

REPAIR RECORDS ON FILE

9

Business Purpose

• Sample-based, claim-specific indicator assesses

whether a shop is following proper operational

procedures

• Measures the artifacts being captured in the

repair process

• Focus on Structural Repairs

• Contributes 5 points to QoR KPI

10

Repair Status & Artifacts

11

As included in Shop Measures Information Guide

Note: The above artifacts, aside from photos of undressed welds, are already required as part of

a proper repair process.

1. Mitchell Status Reporting

through all Repair Statuses:

2. Relevant artifacts to be

uploaded to each claim:

Arrived At Shop

Disassembly Repair in Progress Paint Ready for Delivery

Delivered

1. OEM Repair Data

2. Diagnostic Scan –

SRS, ABS, EMS

3. Photos of Undressed Welds

4. Frame Inspection Sheet

5. 3D Measurement Data

6. Wheel Alignment Data

7. FRA Sheet

Artifacts

12

Document Why is this required?

1 OEM Repair Data • Ensures technicians follow OEM procedures

• Allows shops to enhance customer confidence in quality of repair

2 Diagnostic Scans -SRS, ABS, EMS

• Scanning prior to repairs can prevent hidden repair costs

3 Frame Inspection Sheet

• Ensures frame pulling is being billed in alignment with MPI

Estimating Standards

4 3D Measurement Data

• Supports the condition of the vehicle’s dimensions prior to repairs

and the final repaired dimensions

5 Photos of Undressed Welds

• Supports selected welding method

• Ensures technicians follow the correct sectioning procedures

6 Wheel Alignment Data

• Supports the condition of the vehicle’s dimensions prior to repairs

and the final repaired dimensions

7 Final Repair Account Signature sheet

• Supports customer authorization to repair and satisfaction with

completed repairs

Sample & Scoring Approach

• Recommended annual sample size based on

ISO Sampling Procedure (ISO2859-1) provides

a 95% confidence level and 2% margin of error

– Sample is based on 2016 Ready For Payment claims

– Type of claims audited:

• Claims above $3,000 and 1 hour of Frame Repair

• Claims above $2,000 in Total Repair

– Score is calculated each month using last month’s

data

– Scoring is based on existence of documents; claims

may be referred for follow up with the repair shop

13

Scoring

• How it is scored

– Each claim is scored out of 5 points

– Simple average is then taken of all scores for the

month

– To mitigate fluctuations, a rolling average of scores of

the previous 3 months is used for reporting

14

Scoring

• Contribution to composite score

– Weighted ranking of all measures

– Calculated as a rolling average of RRoF scores for

the previous 3 months

– Contributes 5 points towards a shop’s composite

score

15

REPAIR ACCURACY

16

Business Purpose

• Indicator assesses whether a shop is performing

collision repair as per its stated capability and is

following standard repair procedures

• Seeks to cover all accredited autobody and

autobody & frame repair shops through in-

progress inspections via shop visits as well as

vehicle inspections arising out of customer

referrals

• Contributes 15 points to QoR KPI

17

What to Audit -

Actual Behaviour vs. Stated Capability

• Entry Criteria:

– Vehicles are being accepted and repaired according

to the shop profile/capability

• If not, score for the entire visit is zero

• Accredited Repair monitors a daily report of

Mitchell in-progress claims for compliance

18

What to Audit –

Actual Behaviour vs. Stated Capability

• Scoring Criteria “for each repair observed”:

– Trained technicians (i.e., as per Shop Capability) are

working on vehicle repair

– Right tools are calibrated and being used as per

OEM recommendation

– Technician working with a tool is trained in that tool

– OEM/MPI Repair Procedures are available to the

technician working on repair

– OEM/MPI Repair Procedures are being followed by

technicians

19

Sample & Scoring Approach

• All accredited repair shops will be scored on

Repair Accuracy at least once a year

– All MPI claims in progress may be scored

– Focus will be on Structural Repairs

– Shop visits may be announced or unannounced

– Repair processes may be scored virtually

– If no repair is observed in progress at the time of visit,

shop’s repair processes will be reviewed

20

Sample & Scoring Approach

• All Customer referrals will be scored as part of

Repair Accuracy using the same methodology

• Repair of damage unrelated to the claim will be

noted and reported

• Non-MPI claims may be observed and reported

to Vehicle Safety for deficiencies

21

Scoring

• How it is scored

– All shop visits, customer referrals and claims

reviewed are scored out of 15

– Simple average is then taken of all the scores during

the last three months and counts towards the

composite score

– In the event no evaluation has taken place in last

three months, the last available score will carry

forward

22

Scoring

• Contribution to composite score

– Calculated as a rolling average of the previous three

monthly scores

– Contributes 15 points towards the composite score

– Score is carried forward from the previous period if a

shop hasn’t been scored in last three months through

shop visits, customer referrals or review of claims with

the shop

23

Inspection Frequency

• Based on the following criteria, a shop may be

visited once a week or once a year:

– Number of qualified claims

– Customer referrals

– Reports on non-qualified repairs

– All KPI scores, including Customer Satisfaction,

Financial/Administrative and QoR KPIs

– Corrective actions (per QA Corrective Action Plan)

– Performance trends

24

SCORE CARD

25

QoR Phase II - Interim Score Card(January – March 2018)

Monthly Shop Measures - December 2017 FOR INFORMATION ONLY

REPORT DATE: January 15 2018SHOP NAME: Repair Shop X

REG ACCT NO: A****

KEY MEASURES:

Optimal Value Direction

Month 3 Month AverageIndustry

3 Month Average3 Month Average Contribution to

Composite(December 2017) (Oct 2017 To Dec

2017)(Oct 2017 To Dec

2017)

Ask-Approve Variance Closer to 0 is better 0.2% 0.0% 1.6% 25.0

Average Supplement per Estimate Closer to 0 is better 0.98 1.15 1.23 7.9

Alternate Parts Usage (APU) Higher is better 55.4% 47.5% 49.7% 9.0

Labour % of Gross Total Higher is better 44.6% 41.8% 44.2% 3.4

Net Promoter Score (NPS) Higher is better 65.0% 75.5% 69.6% 7.6

QUALITY OF REPAIR:

3 Month AverageIndustry

3 Month Average Standardized QoR KPI Score(Oct 2017 To Dec

2017)(Oct 2017 To Dec

2017)

Shop Capability Score Higher is better 58.8% 71.6% 2.9

Repair Records On File Higher is better 80.0% 62.2% 4.0

Repair Accuracy Higher is better 66.7% 68.0% 10.0

Confidential: This report is intended only for the party or parties to whom it is addressed. While every attempt has been made to ensure that the information provided is accurate, Manitoba Public Insurance assumes no responsibility for any damages, losses, or negligence arising out of your reliance upon or use of its content. It may contain information that is privileged, confidential, and/or exempt from disclosure under applicable law. Any dissemination, distribution, copying, disclosure, or action taken in reliance upon this communication by parties other than those to whom it is addressed is strictly prohibited. If you have received this message in error, or if you are not an intended addressee, please immediately notify the sender and irrecoverably delete this message at once.

Total Out of 100 69.7

Composite Score 69.7%

QoR Phase II - Final Score Card(April 2018 Onwards)

Monthly Shop Measures – March 2018

REPORT DATE: April 15 2018SHOP NAME: Repair Shop X

REG ACCT NO: A****

KEY MEASURES:

Optimal Value Direction

Month 3 Month AverageIndustry

3 Month Average3 Month Average Contribution to

Composite(March 2018) (Jan 2018 To March

2018)

(Jan 2018 To March

2018)

Ask-Approve Variance Closer to 0 is better 0.2% 0.0% 1.6% 25.0

Average Supplement per Estimate Closer to 0 is better 0.98 1.15 1.23 7.9

Alternate Parts Usage (APU) Higher is better 55.4% 47.5% 49.7% 9.0

Labour % of Gross Total Higher is better 44.6% 41.8% 44.2% 3.4

Net Promoter Score (NPS) Higher is better 65.0% 75.5% 69.6% 7.6

QUALITY OF REPAIR:

3 Month AverageIndustry

3 Month Average Standardized QoR KPI Score(Jan 2018 To March

2018)

(Jan 2018 To March

2018)

Shop Capability Score Higher is better 58.8% 71.6% 2.9

Repair Records On File Higher is better 80.0% 62.2% 4.0

Repair Accuracy Higher is better 66.7% 68.0% 10.0

Confidential: This report is intended only for the party or parties to whom it is addressed. While every attempt has been made to ensure that the information provided is accurate, Manitoba Public Insurance assumes no responsibility for any damages, losses, or negligence arising out of your reliance upon or use of its content. It may contain information that is privileged, confidential, and/or exempt from disclosure under applicable law. Any dissemination, distribution, copying, disclosure, or action taken in reliance upon this communication by parties other than those to whom it is addressed is strictly prohibited. If you have received this message in error, or if you are not an intended addressee, please immediately notify the sender and irrecoverably delete this message at once.

Total Out of 100 69.7

Composite Score 69.7%

Support for QoR Phase II –

Partner’s Site

• Material available:

– Job Aids and Q & A’s

– Videos

– Info Session Webinars

– Overview Training

PowerPoint deck

– Shop Measures Information Guide

• For further information:

– Website: www.mpipartners.ca

– Email: partners@mpi.mb.ca

28

Support for QoR Phase II –

Transition

• Training:

– In addition to the material posted on the website, all

shops will receive an invitation to attend a web-based

seminar on what to expect from QoR Phase II.

• On-Site Support:

– Accredited Repair Inspectors will visit repair shops

starting October, 2017. They will perform inspections,

answer questions and provide support to ensure the

shops have the right process in place.

29

Timeline

Sep. – Dec. ’17

• Training package available on MPI Partner’s website (Sep.)

• Info session webinars (Sep.)

• Accredited Repair inspections begin (Oct. – Dec.)

• Repair Records on File and Repair Accuracy evaluation & feedback (Oct. – Dec.)

Jan. – Mar. ’18

Soft launch

• All interim monthly QoR scores reported with no impact to January – March composite

• Claims and visits made in January – March count towards April composite

Apr. ’18

All scores live

• Composite to include QoR scores based on evaluations during January – March

30

Course Summary

• Having completed this training, you should now

be able to:

– Summarize the overall Phase II Quality of Repair

(QoR) Key Performance Indicators (KPI)

– Explain new components of QoR

– Interpret the interim and final score card

– Identify support options

– Summarize the timeline and implementation of Phase

II Quality of Repair (QoR)

31