Upload
haroon-abbu
View
8.928
Download
6
Tags:
Embed Size (px)
Citation preview
Copyright T. M. Kubiak 1996-2010
Performance Improvement Solutions
Conquering the Challenge!™ 1
Presented to
ASQ St. Petersburg-Tampa-Section 1508
08 February 2010
by
T. M. Kubiak, President
Performance Improvement Solutions
Developing Meaningful Metrics:
Driving Action for Results!
Copyright T. M. Kubiak 1996-2010
Performance Improvement Solutions
Conquering the Challenge!™ 2
If you don’t have confidence in
the diagnosis, you won’t have
confidence in the prescription.
— Steven Covey
Copyright T. M. Kubiak 1996-2010
Performance Improvement Solutions
Conquering the Challenge!™ 3
Copyright T. M. Kubiak 1996-2010
Performance Improvement Solutions
Conquering the Challenge!™ 4
Can any of you help me with
metrics? My boss wants a
whole new set by the end of
the week and I don’t even
know where to start! HELP!
Copyright T. M. Kubiak 1996-2010
Performance Improvement Solutions
Conquering the Challenge!™ 5
The Driving Force
Definitions
Why Measure?
Barriers to Measurement
Pitfalls to Avoid
What Measures Measure
The “E’s” of Measurement
Different Views of Measures
Overview
Characteristics of Measures
Development Considerations
Principles of Measurement
Where Should You Measure?
Goals, Targets, & Benchmarks
Evolution of a Measure
Examples of Measures
Some Final Words...
Copyright T. M. Kubiak 1996-2010
Performance Improvement Solutions
Conquering the Challenge!™ 6
Measure (Metric)
Refers to numerical information that quantify
input, output, and performance of processes,
products, and services
Indicator
Measurement that relates to performance, but
is not a direct or exclusive measure of such
performance
Measure that is a predicator of some more
significant performance
A Few Definitions, Of Course!
Measure, Metrics, and Indicators! Oh my!
Copyright T. M. Kubiak 1996-2010
Performance Improvement Solutions
Conquering the Challenge!™ 7
If you cannot measure it, you cannot control it.
If you cannot control it, you cannot manage it.
If you cannot manage it, you cannot improve it.
What gets measured and rewarded is what gets
done. Corollary: If the measurements don’t
change, neither do the results.
Measurements allow us to track where we have
been, where we are, and where we are going
Shows how effectively we use our resources
Why Measure?
Measurement: The Roadmap to Progress
Copyright T. M. Kubiak 1996-2010
Performance Improvement Solutions
Conquering the Challenge!™ 8
Focuses attention on factors contributing to achieving the organization’s mission
Assists in setting goals and monitoring trends and progress
Provides input for analyzing root cause and sources of errors
Identifies opportunities for on-going improvement
Provides a means of knowing whether we’re winning or losing
More Reasons for Measuring
Measurement: The Roadmap to Progress
Copyright T. M. Kubiak 1996-2010
Performance Improvement Solutions
Conquering the Challenge!™
Integrated System of Metrics
Driving Customer Satisfaction and Business Success!
Corporate
Objectives
Business
Requirements
Corporate
Strategy
Operational
Processes
Customer
Requirements
Business
Strategy
Support
Processes
Supplier
ProcessesAre these processes capable of
meeting business requirements?
Lean Six Sigma drives
improvement at the individual
process level.
The Driving Force
Copyright T. M. Kubiak 1996-2010
Performance Improvement Solutions
Conquering the Challenge!™ 10
Thinking some jobs can’t be measured
Thinking there isn’t enough time to measure
The process of accomplishing a task is not
understood
Fear that the measurement will become a club
Barriers to Measurement
Copyright T. M. Kubiak 1996-2010
Performance Improvement Solutions
Conquering the Challenge!™ 11
Using the measure as a carrot or a stick
Reason: the goal becomes to manipulate the number
Pitfalls to Avoid!
Copyright T. M. Kubiak 1996-2010
Performance Improvement Solutions
Conquering the Challenge!™ 12
Measuring everything that is possibly helpful
Reason: dilutes the effort
Pitfalls to Avoid!
Copyright T. M. Kubiak 1996-2010
Performance Improvement Solutions
Conquering the Challenge!™ 13
Measuring parameters where the organization excels
Reason: reduces the effort to a publicity campaign
Pitfalls to Avoid!
Copyright T. M. Kubiak 1996-2010
Performance Improvement Solutions
Conquering the Challenge!™ 14
Requiring precision in the data beyond the
requirements of the decision
Reason: exasperates all involved
Pitfalls to Avoid!
Copyright T. M. Kubiak 1996-2010
Performance Improvement Solutions
Conquering the Challenge!™ 15
Quality (acceptable, good, in-spec, etc.)
Quantity (Number of...)
Time (Cycle time, rates)
Cost
What Measures Measure
Copyright T. M. Kubiak 1996-2010
Performance Improvement Solutions
Conquering the Challenge!™ 16
Extent (volume-based or deployment)
How much?
Efficiency (productivity-based, usually
expressed as a rate)
How fast?
Effectiveness (customer-based or
objective-based)
How well?
The “E’s” of Measurement!
If It Were Only That Easy!
Copyright T. M. Kubiak 1996-2010
Performance Improvement Solutions
Conquering the Challenge!™ 17
W IPM W IPM W IPM W EOPM
W IPM W IPM W IPM W EOPM
Organizational level
Operational measures (invisible to the customer)
Predictive measures (in line of sight of the customer)
Process Level
In-process measures (evaluate the method that
creates results)
End-of-process measures (measure the results of a
method)
Different Views of Measures
Copyright T. M. Kubiak 1996-2010
Performance Improvement Solutions
Conquering the Challenge!™ 18
Actionable (So What?)
Easily measurable
Economical to collect
Focused on processes
Objective
Relevant
Specific
Timely
Traceable
Understandable
Characteristics of Measures
If It Ain’t Actionable...
Copyright T. M. Kubiak 1996-2010
Performance Improvement Solutions
Conquering the Challenge!™ 19
How often will the measure be computed and
posted?
What about data stratification/aggregation?
Where will the data come from?
Is the data source reliable?
Can/should the data be normalized?
How will the measure be presented?
Who will see them?
Development Considerations
Critical Questions Require Thoughtful Answers
Copyright T. M. Kubiak 1996-2010
Performance Improvement Solutions
Conquering the Challenge!™ 20
Answers: Who, How, What, Where, When
Defines: formulas and terms
Defines: interpretation (e.g., up is good)
Example:
Operational Definition
Establishes a common language and understanding
What is a “month?”
Copyright T. M. Kubiak 1996-2010
Performance Improvement Solutions
Conquering the Challenge!™
Cash for Clunkers Example Issue: 78 cars bumped from the list
Threshold: 18 mpg
Reason: More precise data (4 decimals) caused the revisions (Note: 0.0001 miles is ½ drop of fuel based on 18 mpg)
Measurement Methods:
Older: Dynamometer, fuel consumption measured, measured to 4 decimals, numbers rounded
Newer: Tailpipe emissions, measured to 4 decimals
Fix: Old data updated to be “effectively equivalent”
Quote: “Repeatability and accuracy is something we spend a lot of time on.” EPA scientist
21
Hey, Joe! Gotta light?
Source: WSJ 09-08-05
Copyright T. M. Kubiak 1996-2010
Performance Improvement Solutions
Conquering the Challenge!™ 22
Cancer Deaths Example
Normalized
Stratified by age
•Gaps in age
stratification
•Aggregated across
gender
•Aggregated across
cancer type
Does this chart support
the conclusion?
Source: WSJ 09-02-09
Copyright T. M. Kubiak 1996-2010
Performance Improvement Solutions
Conquering the Challenge!™
Pareto Chart Example
23
0
2
4
6
8
10
12
14
16
18
20
Def 1 Def 2 Def 3 Def 4 Other
You can’t fix stupid!
0
2
4
6
8
10
12
14
16
18
20
Def 1
Def 3
Def 5
Def 7
Def 9
Def 11
Copyright T. M. Kubiak 1996-2010
Performance Improvement Solutions
Conquering the Challenge!™
Air Travel Example
What is your understanding of an “on-time arrival?”
How does plus or minus 15 minutes of the scheduled arrival time sound?
Is this a good metric or a bad metric?
How many customers are aware of it?
Related metrics (by carrier):
Percentage of flights arriving on-time
Percentage of canceled flights
Mishandled bag reports per 1,000 passengers
Complaint reports per 1,000,000 passengers
Bumped passengers per 10,000
24
Do key players understand the metrics?
Source: WSJ 1--01-07
Copyright T. M. Kubiak 1996-2010
Performance Improvement Solutions
Conquering the Challenge!™ 25
ExtentEfficiency
Effectiveness
Measurements must be specific
Measure the outputs of highest value to
the customerMeasures can be applied to all performance
dimensions - external as well as internal
Understand the game before you decide
how you’ll keep score
Measure the process as well as the results
There is no single perfect measure
Principles of Measurement
Copyright T. M. Kubiak 1996-2010
Performance Improvement Solutions
Conquering the Challenge!™ 26
Early in the process to promote prevention
At the point where a cause-and-effect
relationship can be established
Functional boundaries
Points of convergence
Points of divergence
At critical processes
Where Should You Measure?
Copyright T. M. Kubiak 1996-2010
Performance Improvement Solutions
Conquering the Challenge!™ 27
Goals, Targets, & Benchmarks
Copyright T. M. Kubiak 1996-2010
Performance Improvement Solutions
Conquering the Challenge!™ 28
If you can’t win the race, make
sure the guy ahead of you
breaks the record.
— Steve Prefontaine (1951-
1975), American runner
Copyright T. M. Kubiak 1996-2010
Performance Improvement Solutions
Conquering the Challenge!™ 29
Extent
Efficiency
Effectiveness
Number of computer reports
delivered
Number of acceptable computer
reports delivered on time
Number of computer reports
delivered on time
Percent of acceptable computer
reports delivered on time
Evolution of a Measure
Copyright T. M. Kubiak 1996-2010
Performance Improvement Solutions
Conquering the Challenge!™ 30
The value of this measure is unclear. It is not
customer-focused nor does it measure the
ability of an organization to deal with incoming
trouble calls since no processing of trouble
calls is evident in the metric. Essentially, this
metric measures nothing more than the
volume of incoming trouble calls. The fact that
it is expressed in the form of a rate is
misleading and may fool one into thinking it is
a measure of efficiency. Type of Measure: Extent
Measure: Trouble calls received/week
Examples of Measures
Copyright T. M. Kubiak 1996-2010
Performance Improvement Solutions
Conquering the Challenge!™ 31
This metric focuses on measuring nothing
more than the volume of jobs scheduled. Other
than that, it provides little insight into the
underlying process of “scheduling jobs.” The
metric is not actionable.
Type of Measure: Extent
Measure: Job Scheduled
Examples of Measures
Copyright T. M. Kubiak 1996-2010
Performance Improvement Solutions
Conquering the Challenge!™ 32
Measures the planning/forecasting process.
Type of Measure: Effectiveness
Measure: Actual versus planned sales
Examples of Measures
Copyright T. M. Kubiak 1996-2010
Performance Improvement Solutions
Conquering the Challenge!™ 33
This metric is highly misleading and often
used for organization to organization
comparisons. It falsely assumes that all
employees generate sales and does not
consider the product being sold or the
associated supporting infrastructure required
to produce and sell the product.
Type of Measure: Efficiency
Measure: Sales/employee
Examples of Measures
Copyright T. M. Kubiak 1996-2010
Performance Improvement Solutions
Conquering the Challenge!™ 34
This metric measures the deployment of
training hours to the workforce.
Type of Measure: Extent
Measure: Number of training hours/employee
Examples of Measures
Copyright T. M. Kubiak 1996-2010
Performance Improvement Solutions
Conquering the Challenge!™ 35
This metric measures the impact of training the
sales workforce so long as a clear cause-and-
effect relationship can be established between
training and revenue.
Type of Measure: Effectiveness
Measure: Revenue/sales training hour
Examples of Measures
Copyright T. M. Kubiak 1996-2010
Performance Improvement Solutions
Conquering the Challenge!™ 36
This metric measure the design process from
the point of view of the drawings. However,
note that this metric does not normalize or
adjust for the complexity of a given drawing.
Type of Measure: Effectiveness
Measure: Engineering change orders/drawing
Examples of Measures
Copyright T. M. Kubiak 1996-2010
Performance Improvement Solutions
Conquering the Challenge!™ 37
Measure, Control, Manage, Improve
Extent, Efficiency, Effectiveness
Performance measurement is
irrelevant without a frame of
reference
Measure twice! But cut once!
Some Final Words...
Copyright T. M. Kubiak 1996-2010
Performance Improvement Solutions
Conquering the Challenge!™ 38
People without information are
unable to change; those with
information are compelled to
change.
Copyright T. M. Kubiak 1996-2010
Performance Improvement Solutions
Conquering the Challenge!™ 39
Copyright T. M. Kubiak 1996-2010
Performance Improvement Solutions
Conquering the Challenge!™ 40