Upload
moris-doyle
View
217
Download
0
Embed Size (px)
Citation preview
Baseline Data
• Measure• Kaizen Facilitation
2
Objectives
• Define data types and purpose• Explain concepts of efficiency and effectiveness• Provide tips on establishing baseline metrics• Understand importance of data collection• Review elements for data planning and usage
3
Types of Data (Definition)
A metric obtained by observing a population, product, process, or service...
• Qualitative • Job is expensive and takes too long
• Quantitative• ATTRIBUTE DATA
• (Categories, Yes/No, Pass/ Fail, Machine 1 vs. Machine 2, etc)• CONTINUOUS DATA
• (Time, Temperature, Weight, Pressure, etc)
4
Purpose of Data Collection
• The “generic” purpose of all data (metrics) is to help make better decisions
• A Metric is an input to a decision system which has the following elements:• Who looks at it?• When / Where / How do they look at it?• What do they compare it to?• What are their choices?• What actions are taken?
Better Data = Better Decisions
5
System Feedback
• How do you know if your system is working right?• All processes exist to fulfill needs• Your system is working ‘right’ if it fulfills those needs
• Who defines these needs?• The customer (recipient of good or service)
• Typically associated with the output of the process• The business (provider of good or service)
• Typically associated with the process itself
6
Establishing Metrics
• For each product or process requirement, there should be one or more objective measures
• These measures (metrics) allow us to verify that requirements have been met through the general relationship:
MeasuredPerformance
_ RequiredPerformance
= (Delta or Variance)
7
Sources of Process Feedback
• Typical organization has two main sources:• Metrics
• Quantitative performance indicators are compared to defined requirements• If these indicators do not meet or exceed the requirements, then a “problem” is said to
exist
• Squeaky Wheel• Anecdotal and experiential feedback for the process indicate the potential for a
“problem”• Squeaky wheels can be a trigger for the creation of a metric
8
Flow versus Performance
• Process mapping helps us understand process flow• Now we must collect data to more objectively evaluate and measure
performance• To be successful, we must meet both Customer and Business
requirements• Business (VOB) - efficiency measures• Customer (VOC) - effectiveness measures
• Measurement systems incorporate both
9
Business Feedback (VOB) - Efficiency• In order to remain profitable, a business must deliver goods and
services such that their cost is lower than their price• Business systems should operate at:• High speed• Low cost• With minimum resources (task sequence, strain, waste)
• Voice Of the Business are “Efficiency” measures
10
What to Measure? (VOB) - Efficiency
• Outputs• Highest level• Look for:• Quality
• Defects, Yield, Rework (Overall and by process step)• Cycle Time
• Lead time (Overall and by process step)• Cost / Bottlenecks (Overall and by process step)
11
VOB Metrics (How Capable is the Process?)• Quality: Does the product or service meet the customer requirements
(specifications)?
• Cycle time: How much time do various steps in the process take? Are there delays in some steps?
• Bottlenecks: What types of bottlenecks are you seeing? How frequently? How long is the delay?
12
Customer Feedback (VOC) - Effectiveness
• Goods and services that your Customer receives is an output of your process• Customers want goods and services:• On-time• With highest quality• At an excellent value (competitive price, et al)
• Voice Of the Customer are “Effectiveness” measures
13
What to Measure? (VOC) - Effectiveness• Unacceptable product or service• Customer complaints• High warranty costs• Decreased market share• Backlog• Redoing completed work (cost)• Late output• Incomplete output (yield)
14
VOC Metrics (How Satisfied is the Customer?)
• On-time: How many of your products or services out of the total that were made or delivered meet customer requirements?
• Quality: Does the product or service meet the customer requirements?
• Cost: How much does it cost to produce the product or service and how does the cost compare to your competitor’s costs?
Plan
Data Collection
USAGE
Elements of Data Collection• Data collection is the process of gathering the
information you need to be able to make a better decision
15
16
Planning: Clarifying Goals
• Decide why you are collecting the data• Determine factors that could cause the measurement of an item to
vary• Find ways to reduce the impact of those factors• Decide how the data will help you• Decide what you will do with the data once you have it
17
Planning: Consistency and Stability • Know your process• Decide what data you need to collect• Fresh (current) data or historical (past) data
• Develop an “operational definition” of metric• Test your data collection forms• Make it easy to collect data• Communicate the what and why to the data collectors and process
participants
18
Usage: Operational Procedures
• Decide what you are trying to evaluate• Decide how you will attach a value to what you are trying to measure• Decide if you need to collect new data• If so, decide how you will collect the data• Decide how you will record the data• Determine the period of time you will study• Estimate how many observations you will need
19
Usage: Oversight Required
• Train everyone who will be collecting data• Make data collection procedures error-proof• Be there in the beginning to oversee data collection• Confirm understanding of operational definitions• Check to make sure data measurements are stable and that the data
looks ‘reasonable’
• Entered as part of workflow• Mined with queries and reportsDatabase
• Sheets that move with product or fileTravelers
• Every item vs. selected ones• Links data to a specific recordLogs
• Easy to use• Aggregates dataCheck sheets
Collection: Sources
20
Collection: How Much ?
Too Little Data• Bad decision making• Extends project timeline• Extra work to collect additional
data
Too Much Data• Wasted time collecting data• Frustrated team members• Wasted time spent managing
data• Important information hidden in
piles of data
21
22
Types (Pros and Cons)
• Attribute Data• Few Tools for Analysis• Need more data
• Continuous Data• Gain more insight to behavior of the process• Can use many tools to analyze• Need less data
Whenever possible its generally better to use Continuous Data
Data
Continuous
Attribute
Binary
Measured
Categorical
Ordinal
Count
Types (Safety example)
23
Time of Incident
Gender
Type of Incident
Severity Rating
Number of LTI’s
(Can be sub-divided)
• Collecting data on a small subset of a population to make inferences about the characteristics of the whole population• Benefits• Lower cost• Saves time
• Drawbacks• Possibility of errors
Sampling uses this…
Sampling
24
…to predict
about all of this
Sampling
25
26
Integrity (Some things to think about)
• Is recorded data what we meant to record?• Does it contain information that was intended?• Does the measure discriminate between items that are
different?• Does it reliably predict future performance?• Does it agree with other measures designed to get at the
same thing?• Is the measure stable over time?• Is the data correct in the system?
Evaluate what’s right and wrong with your process
• The way baseline data is collected can have an adverse effect on the outcome of the project• A data collection plan is an organized, written strategy for gathering
information for your project• Data are reliable every time• Only relevant data are collected• All necessary data (VOB, VOC) are collected• Resources are used effectively
• Data collection form – a very important check sheet for your project…
Tips
27
28
Review
• Don’t assume people know how to collect data• Take a careful, methodical approach
• Develop a systematic method to collect, review and analyze the data • Chart data and Analyze for trends• Ask: Do the results pass the common sense test?
• Monitor process using your metrics• See how trends project into the future• Share the results with appropriate personnel