1
Identify and develop the current state “as-is” value stream process map
Confirm KPOV’s and Identify Key Process Input Variables (KPIV’s)
Define performance standards and the defect
Identify sources of existing data
Establish priority and plan data collection of non-existent data
Conduct measurement system analysis (MSA)
Collect baseline data
22
Review and amend cost of poor quality (COPQ) estimates
Evaluate process capability and performance
Review problem statement and project objective
Develop project measure phase tollgate presentation
3
Process Value Stream Mapping
Check sheets, Data Collection Worksheets, Process Observation
Measurement Systems Analysis
Gage R & R
Baseline performance analysis
QI Macros Statistical Software◦ Histogram with cp/cpk, DPMO
◦ Pareto Chart
◦ Capability Analysis
4
Phase Objectives Key Activities Possible Tools and Techniques Key Deliverables Document the problem statement and establish the charter. Demonstrate alignment with the Business metrics and strategies. Determine Customer requirements and performance standards.
▪ Select Team with Champion
▪ Develop problem statement
▪ Develop Charter ▪ Create SIPOC ▪ Address gap between VOC
and process ▪ Estimate financial benefits
▪ Problem Statements ▪ Project Charter ▪ SIPOC map ▪ COPQ or CODND ▪ Communication Plan
Develop a reliable and valid measurement system of the business process to effectively evaluate the success of meeting customer requirements.
▪ Obtain Customer requirements
▪ Create overall project plan ▪ Develop measurement
plan & compile project metrics
▪ Determine defect tracking requirements
▪ Assess baseline performance-estimate process capability
▪ Measurement Systems Analysis
▪ Process Description ▪ Project Plan & Timeline ▪ Metrics and collection plan ▪ Baseline Performance results ▪ Process capability analysis ▪ Lean Tools Assessment ▪ Measurement Systems
Analysis ▪ Process model – ‘as is’
Utilization of data techniques to gain insight into process. Divide data into groups based on key characteristics and assess the root causes of errors and poor performance. Determine where to focus efforts for improvement.
▪ Statistical tests / tools ▪ FMEA ▪ Pareto chart ▪ Correlation/Regression ▪ Fishbone Diagram ▪ Box plot ▪ Hypothesis Testing
▪ Describe findings – identify potential root causes RCA
▪ Validate findings
▪ Data relationships ▪ Validated Key Input Variables
(KPIVs) & Key Output Variables (KPOVs)
▪ Prioritize sources of variation ▪ root causes
▪ Identify & communicate potential improvements
▪ Summarize benefits & annualized financial benefits
Identify key change opportunities and proactively test for optimization. Develop implementation and communication plan including a change management approach to assist the organization in adaptation of the improvements.
▪ Design of Experiments – describe purpose & build test/ analysis strategy
▪ Evaluate and Confirm results
▪ Analyze KPIVs ▪ Create action plan for
implementation including change management and communication needs
▪ Buy-in assessment
▪ Quantified relationship between key
input and key output variables ▪ Defined process improvements
including impacts and benefits ▪ Implementation Plan ▪ Process model – ‘Should be’ ▪ Impacted Employees are Trained
Definition of optimal process settings and conditions with specified metrics. Implementation of improvements with a control plan to assess & maintain gains.
▪ Implement improvements ▪ Evaluate results ▪ Integrate & manage
improvements in work processes
▪ Complete closure activities
▪ Document process change ▪ Control plan ▪ Determine new process capability ▪ Leverage opportunities for replication ▪ Communicate results ▪ Financial audit ▪ Hand-off to process owner
1.0 Define
Opportunity
2.0 Measure
performance
3.0 Analyze
Opportunity
4.0 Improve
Performance
5.0 Control
Performance
5
Six Sigma Process Improvement Road Map
1
Migration e-Pro Process ImprovementProject Charter
Project Description Error corrections and clarification of benefits are generatingrework throughout the migration and case installationprocesses, accounting for 20% of the total number of e-Prochange transactions. It is estimated that the volume of errorand rework will grow proportionally as the number ofaccounts migrating by 1/1/2004 increases, driving aproportionate increase in cost and potentially dissatisfyingcustomers.
Start Date April 1, 2003
Completion DateScheduled to be completed by September 5, 2003
Baseline Metrics For 1/1/03 migrated accounts:National Accounts- Average number of change transactions: 14.3, of which
2.9 are due to error and rework- Average hours of rework: 309 hoursRegional Accounts- Average number of change transactions: 8.0, of which
1.6 are due to error and rework- Average hours of rework: 137 hours
Primary Metrics 1. Total e-Pro change transactions2. Percentage of change transactions due to error and
benefits clarification3. Average rework hours per error and benefits clarification
Secondary Metrics none
Goal Reduce error and rework in the migration process by 50%starting with 1/1/04 migrating accounts
Customer Customer migration survey results
Financial CODND (Cost of Doing Nothing Differently)4
th Qtr 2003: $500K
Year 2004: $2.5M
Ben
efits
Internal Productivity Estimated cycle time reduction of 18,868 hours (assuming195 accounts migrating 1/1/04).
Define April 1 – April 21, 2003
Plan Projects & Metrics April 14 – April 18, 2003
Baseline Project April 21 – May 2
Consider Lean Tools May12 – May 16, 2003
MSA May 19 – June 2, 2003
Wisdom of the Org. June 2 – June 6, 2003
Passive Analysis June 9 – June 20, 2003
Proactive Testing June 23 – August 4, 2003
Ph
as
e M
ilesto
nes
Control August 4 – September 5, 2003
SUPPLIER INPUT PROCESS OUTPUT CUSTOMER
Sales
Client / Policy HolderHR BenefitsCoordinator
Client Consultant
Third Party BenefitsVendor
Member
GO Decision
Policy Renewal Date
Summary of Benefits
AdministrativeRequirements
AccountOrganizational
Structure
Detail Benefits
Account DataLoaded in System
Member andDependentEligibilityInformationLoaded in System
Member ID Card
Client / PolicyHolder
Third Party BenefitsVendor
Member andDependent
Providers
Claim
Call
1. Conduct migration analysis
2. Complete account profile
3. Load account structure in system
4. Set up and validate account benefits in system
5. Produce account eligibility record
6. Load account data in product claim engines
0Subgroup 10 20 30 40
0
10
20
30
Ind
ivid
ua
l V
alu
e
Mean=10.98
UCL=26.81
LCL=-4.854
0
10
20
Mo
vin
g R
an
ge
1
R=5.952
UCL=19.45
LCL=0
Total e-Pro Change Transactions by Account from Sep 2002 thru Mar 2003Conduct
Analysis
Create Implementation
Guide
Expert Team
Meeting
Draft
EPRO
Draft
e-
PRO
Release e-PRO
Record
e-
PRO
Impl.
Guid
e
Update
EPRO
OK For
Release
to Vendor
Track
Systems
Loads
IMPLEMENTATION
ERW
From
Eligibilit
y
GO
Decisio
n
SALE
S
Set Up Client ID in
End State
Structure
Request
Codes
STRUCTURE
Complete
Structure
Inspection/Verify
with e-PRO
Get
Underwriting
Approval
Yes
No OK? Yes
No
Go back to
Rates
Structur
e in
CDB
No
Yes
Release
ATC
To Vendor
OK?
Review
Draft e-
PRORequest/Receive
Codes
Enrollmen
t File
CLIENT /CUSTOMERClient
Input
To Sales
Run Legislative
Tool
Check vs. e-
PRO
e-PRO
Redo?
Yes
No
YesOK?
Review
Draft
e-PROCreate
Codes
Legislative Tool
Review
BPC &
Class
Codes
To
Structure
BENEFITS
TS
ID Claim
Scenarios
Load Data into
Downstream
Systems
OK?Yes
No
Data
Engines
Loaded
(e.g ATC,
DocGen,
etc)
Test Scenarios
Check vs. e-
PRO
OK?No
Yes
Fix Claim
Errors
No
No
VOB
Yes
EPRO
Rework
CDB
From
Vendor
Member
cancelle
d
in
Legacy
Reformat
Client
Eligibility DataReview
Draft
EPRO
ELIGIBILITY
Receive Enrollment
Data
Match &
Merge
Load data in
CEO
Are
errors
resolve
d?
Fix Errors
YesNo Cancel
Member in
Legacy
Create
ERW
ERW
Eligibility
In CED
VENDO
R(ID
CARDS)
ID
Card
s
From
Benefit
s
Get
Underwriting
Approval
ERW To
Implementatio
n
ERW To
Implementatio
n
Create Client IDClient
ID
To
Structure,
Benefits,
and
Eligibility
Migration
Structure
Mapping Job
Aid
GO
Decision
To
Structure,
Benefits,
and
Eligibility
e-PRO
Rework
EPRO
Rework?
SMT linking
legacy
structure to
end state
To
Eligibility
CAIP
Processes shaded in green are specific to
migration
Processes out of scope, but critical to
Employer Services
Employer Services functional areas
OUT OF SCOPE
PROCESS STEPS
Production
Migration
Support
Cancel
Legacy
Structure
Elig.
Rework
e-Pro
Rework
Rework Loops highlighted in Red
10
100
50
0
Contracting
T4
-T1
PMHS
Overpayment
Process Data / Materials
PeopleTechnology
Ÿ Auth / Referral Info missing/incomplete/incorrect
Ÿ OI Info missing/incomplete/incorrect
Ÿ Member Eligibility Info missing/incomplete/
incorrect
Ÿ Benefit Info missing/incomplete/incorrect
Ÿ Provider Fee Schedule Info missing/incomplete/
incorrect
Ÿ Provider/Vendor TIN/SSN Info missing/
incomplete/incorrect
Ÿ Additional Information Necessary to Process
Claim
Ÿ Transaction/Codeset data excluded at gateway
Ÿ Standard Operating Procedures (SOPs)
Ÿ Claim Audit Process >$5K
Ÿ Second/Third Party Internal Review
(Medical Management, Claim Benefit
Build)
Ÿ iTrack - drives usage of paper reports to
sort older claims
Ÿ Skill Level of Processor
Ÿ Accessiblity of Site Coach/Training Staff
Ÿ Aggressive Productivity goals conflict with low
quality requirements
Ÿ Rushed Training Schedule
Ÿ Lack of up-training / reinforcement training
Ÿ Best Practice / Skill Training not conducted
Ÿ OJT training on SOP usage
Ÿ System Error During Processing
Ÿ Data Fallout
Ÿ Aurhorization Mis-Match
Ÿ System Restrictions - LPI Manual Calc
Ÿ Data Set Up Issues (eligibility, provider, benefits)
Ÿ Timeliness of Batch Processing
Ÿ Bank Acct Set-Up Delays
Ÿ Customer Touchpoints Delays
Ÿ Inappropriate assignment or missing hold codes
Ÿ Provider Mis-Match
Ÿ Transaction Limitations on data collected at
gateway
Manual Adjudication &
PMHS Provider Selection
- Manual
End
Manually check
provider/ vendor
on claim system
Check provider
data and claim
data against iView
image
Mismatch?Manually try to find
correct data
Found data
Correct data &
verify COB
Service request to
appropriate area
YES
NO
YES
iTrack
Verify in claim
screen and follow
COB Checklist
Attempt to
adjudicate claim
NO
Claim processed
Hold codes that
require further
research
NO
Re-open the claim
YES
Process will
depend on Hold
Code & SOP/ Job
Aid
CIRF
Attempt to resolve
all Hold Codes at a
line level
Resolve service
requests
Adjudicate claim
(manual or
systematic)
End
ID Task Name
1 DEFINE PHASE
12 MEASURE PHASE
13 Plan Project and Metrics
22 Baseline the Project
23 Select KPOV metric to track process output
24 Estimate process capability/performance at the 30,000-foot-level
25 Categorical failures
26 Create pareto chart
27 Rescope project to a large Pareto category
28 Repeat Baseline the Project steps 23 through 27
29 Non categorical failures
30 Revise estimate for COPQ/CODND
31 Project status update w ith executive sponsor
32 Consider Lean Tools
39 Conduct Measurement Systems Analysis (MSA)
40 Ensure data integrity
41 Perform Gauge R&R
42 Improve gauge
43 Project status update w ith executive sponsor
44 Wisdom of the Organization
55 ANALYZE PHASE
56 Use visualization of data techniques to gain insight to processes
57 Conduct inferential statistical tests and confidence interval calculations on individual KPOVs
58 Conduct appropriate sample size calculations
59 Conduct hypothesis tests
60 Describe statistical f indings to others using visualization of data techniques
61 Implement agreed-to process improvement findings
62 Project status update w ith executive sponsor
63 IMPROVE PHASE
65 d13 d
18 d
18 d
18 d
20 d
20 d
20 d
23 d
23 d
23 d
27 d
76 d33 d
69 d38 d
38 d
40 d
43 d
69 d48 d
61 d48 d
53 d
60 d53 d
60 d53 d
60 d53 d
60 d53 d
53 d
58 d
62 d
63 d
02 09 16 23 30 06 13 20 27 04 11 18 25 01 08 15 22 29 06 13 20 27 03 10 17 24 31 07 14 21 28 05
March April May June July August September Octob
PMHS
# of Audits
7,321
04/05/2003
# $'s
Under 300 4% Under 1,030,680$
Over 676 9% Over 2,303,562$
No $ Error 914 12% No $ Error -$
No Error 5,431 74% No Error -$
7,321 3,334,243$
A)SEVERITY B)OCCURRENCE
Probability
C)DETECTION
Probability
RISK
PRIORITY
NUMBER ACTION TO IMPROVE
Rate 1-10 Rate 1-10 Rate 1-10 RPN
10=Most
Severe
10=Highest
Probability
10=Lowest
Probability AxBxC A B C
Provider Mis-Match 10 8 9 720
Provider Data Incorrect/Incomplete 9 8 9 648
Data Fallout 9 6 10 540
Data Set Up Issues 9 6 10 540
Provider Fee Schedule Unclear 9 6 9 486
OI Information Needed 9 6 6 324
System Restrictions 6 6 9 324
Hold Codes 9 3 10 270
FAILURE MODE
Process Name: PMHS Claim Processing
Date: 6/30/2003 Revision Level: 3
REVISED VALUES
0Subgroup 10 20 30 40
0
10
20
30
In
div
idu
al
Valu
e
Mean=10.98
UCL=26.81
LCL=-4.854
0
10
20
Mo
vin
g R
an
ge
1
R=5.952
UCL=19.45
LCL=0
Total e-Pro Change Transactions by Account from Sep 2002 thru Mar 2003
Gage R&R http://www.aiag.org/ Part Number http://www.qimacros.com/free-lean-six-sigma-tips/aiag-msa-gage-r&r.html
Average & Range Method 1 2 3 4 5 6 7 8 9 10 Sum
Appraiser 1 Trial 1 0.65 1 3.250
Enter your data here-> Trial2 0.6 1
Trial3
Trial4
Trial 5
Total 1.25 2
Average-Appraiser 10.625 1 #N/A #N/A #N/A #N/A #N/A #N/A #N/A #N/A
Range1 0.05 0 #N/A #N/A #N/A #N/A #N/A #N/A #N/A #N/A
Appraiser 2 Trial 1 0.55 1.05 3.100
Enter your data here-> Trial2 0.55 0.95
Trial3
Trial4
Trial 5
Total 1.1 2
Average-Appraiser 20.55 1 #N/A #N/A #N/A #N/A #N/A #N/A #N/A #N/A
Range2 0 0.1 #N/A #N/A #N/A #N/A #N/A #N/A #N/A #N/A
Appraiser Trial 1
Enter your data here-> Trial2
Trial3
Trial4
Trial 5
Total
Average-Appraiser 3#N/A #N/A #N/A #N/A #N/A #N/A #N/A #N/A #N/A #N/A
Range3 #N/A #N/A #N/A #N/A #N/A #N/A #N/A #N/A #N/A #N/A
EV (Equipment Variation)0.0332 Equipment Variation (EV)
%EV 11.3% 39.9% # Parts #Trials #Ops % of Total Variation (TV)
AV: (Appraiser Variation)0.02066 2 2 2 Appraiser Variation(AV)
%AV 7.0% 24.8% % of Total Variation (TV)
R&R (Gage Capability) 0.0391 Repeatability and Reproducibility (R&R)
%R&R 13.3% 47.0% NDC 11 % of Total Variation (TV)
PV (Part Variation) 0.2917 Part Variation (PV)
%PV 99.1% 350% % of Total Variation (TV)
Skewness0.41Stdev1.670.20Max6.20
QI Macros is loaded onto Excel as an “add on”
QI Macros has an extensive menu system with Statistical Tools, Template and Charts
6
Clicking on any one of the menus will reveal the options
Some of the tools require you to preload data into Excel while others will ask you to enter data by hand
The nomenclature for accessing the QI Macros tools to be used for each exercise
will be MENU > TOOL7
To open the tool to conduct a Variable Gage R&R study we would go to QI Macros > DOE Gage R&R FMEA > Gage R&R
8
Statistical Tools
Lean Six Sigma Templates
Data Transformation
Capability Charts
Control Charts
Improvement Charts
Other Charts
9
10
Current State Process Map
Customer
Business ProcessX
X’s (Inputs)
• There is a process output (Y) that is influenced by multiple process inputs (Xs).
• The inputs (Xs) in the process are manipulated until the variation in output (Y) is minimized and/ or the benefit of the output (Y) is optimized.
Y (output)
Here’s where we are right now – focusing in this area
11
X
X
X
X
.
.
Also known as:◦ The “as-Is” process map
◦ A flow diagram or flow chart
◦ Swim-lane cross functional process map
Purpose is to graphically document the current flow of work through the process to be improved
Useful to people familiar with the process, as well as those that have a need to understand the process
12
Each process step is listed, reviewed and analyzed
Should be done by the team members who are actively participating in the process on a day-in and day-out basis
One of the most critical tools the team uses during the measure phase
Before a process can be improved it must be documented and then measured
13
Some standardized symbols for flow charting or process mapping:
Major Process Step
Wait
Process Start
Decision Point
Re-Work Loop
Major Process Step
Alternate Process
Alternate Process
Re-Work Loop
Major Process Step
Alternate Process
Process Stop
14
15
As a class and facilitated by the instructor, we will spend 20 minutes developing a process map for the first segment of the class scenario◦ Use sticky notes on a white board or flip chart
Process Steps – use notes oriented at 90 degrees
Process Decisions – use note oriented at 45 degrees
◦ The process map should follow the process from the beginning to the end of the project scope
◦ Be sure to scrutinize the map for likely decisions
16
Measure 3 – Process Map Worksheet
• KPOV’s are the key process performance indicators for the process Lead time
Quality – defect rates
Customer Satisfaction scores
Inventory Costs
Inventory Levels/Turns
Right First Time – First Pass Yield
Total Good Output – Rolled Throughput Yield
Delivery to Schedule
Cycle Time
What other metrics have you seen?
17
• Lead Time: The length of time between the initiation and
delivery of a product or service. Example: in manufacturing it would be the time from customer order to delivery.
• Lead time profiles can help to:
document “queue time” versus the “cycle time” content of individual parts of process steps
identify constraining resources (bottlenecks)
Identify non-value added activity (trouble spots) during the process
1818
Some Potential Lean Six Sigma Metrics
• Average Lead time = Work In ProcessCustomer Requirements
• Example: 467 parts Work In Process
102 parts scheduled weekly
= 4.6 weeks
= 22.5 days lead time
19
• Is a function of Lead time
• The longer the lead time the greater the level of inventory
• If the lead time is significantly longer than the process time (VA/NVA) then a proportional part of the inventory can be classed as waste.
• In the Lean world levels of inventory are planned, tested and constantly monitored
• By eliminating the waste, which causes inventory to increase, we can reduce lead times and inventories
20
• Takt is German word for Metronome• In production terms the takt time sets the beat/rate
at which we need to produce parts to meet customer demand
Takt time is: time available
customer requirements
• Takt time has nothing to do with how long a task takes but instead how long you have to do the task
2121
• Time available in printing area: Monday 7AM to Friday 6PM = 108 hours
Time is reduced by
Meal breaks
Shift changeovers
Personnel needs
Meetings
Meeting (1 hour)
Total hours reduced by 25% = 81 hours Convert to minutes 81 x 60 = 4,860 minutes
Divide by Customer Requirements of 91 per week
=1 every 53.4 minutes
22
• A metric that is useful in looking at what goes on inside the process
• Looking at the internal sub processes can help the team target improvement efforts
23
• Overall process has 3 sub-processes• Each sub-process step is calculated as it’s own yield
1,500 Input Units
Sub-process # 1
Rework of 30 Units or .98 Yield
Sub-process # 2
Rework of 15 Units or .99 Yield
Sub-process # 3
Rework of 44 Units or .97 Yield
1,411 Output Units
Rework 30 units Rework 44 unitsRework 15 units
1,500 –(89 Units Reworked) = .94 First-Pass Yield1,500 Units Input
When Identifying improvement opportunities: Sub-process # 3 has the lowest yield and possibly the most opportunity for improvement
24
• First pass yield does not account for internal rework in the process – another metric is needed
• RTY - A metric that measures the probability that a process, or service will make it through the process, the first time, without a defect occurring.
• RTY shows the actual yield of a process that has a hidden factory (adds no-value, rework)
• Two methods for calculating:
Method 1 – uses Defects per unit, when we know the total units produced and the number of defects
Method 2 – uses the yield at each opportunity for a defect
25
• Cycle time: The time it takes to successfully complete the
tasks required for a work process.
• Can also be broken down into 2 components of process cycle time:
Work time
Wait time
26
Example: • Process Start: Time a patient enters an emergency
department door – either by foot or ambulance
• Process Stop: Time the patient is officially discharged and leaves the Emergency Department
27
• Defects a part, product, or service that does not conform
to specifications or the customer’s expectations.
• Defects are caused by errors
• Defectives – a unit with one or more defects
• Customer satisfaction is typically measured through a pro-active customer satisfaction survey
28
• Defects per Million Opportunities (DPMO) help to determine the sigma capability at the end of a process, based on the results or output
• Focuses on the capability of the process to meet external customer requirements
DPMO = Defects X 1,000,000
(Units x Opportunity)
29
][
Imagine looking for defects on deck planking. These would constitute 4 opportunities on a single board.◦ Width measurement◦ Length measurement◦ Thickness measurement◦ Presence of knot-holes
Say there were 15 total defects on 55 boards; the DPMO would be:
68,182 = 15 X 1,000,00055 x 4
You can use the file – Distribution DPMO Example in the General Files to calculate DPMO
30
31
Data drives decisions and actions!
Data are measurements or observations we record and use to describe, understand, optimize, or control something such as a process.
32
Attribute
or Discrete
Data
Hours with
Temperature
over 85 Degrees
# of Calls on
Hold Past 30
Seconds
Tank Empty or
Not Empty
Units Exceeding
Target Costs
# of defective items# of errors
33
Advantages:◦ Easy to obtain and calculations are simple
◦ Everyone understands the data
◦ Often readily available
◦ Used to set base line performance
◦ Used by management
Disadvantages:◦ No clue about why a defect happened or how
the process changed
◦ Not accurate at low defect rates (need large sample size)
◦ Cannot predict trends or future events
34
Variable or
Continuous
Data
Hold Time
per Incoming
Call
Average
Temperature
per Hour
Minutes to
Board a Plane
Quantity of
Gas in Tank
Dimensions, Weight, Strength, Time, Cost
35
Advantages:◦ Provides detailed information about the process
using relatively small sample size
◦ Can be used at low defect rates
◦ Can predict trends and future conditions
◦ Key for improvement – DOE, Regression
Disadvantages:◦ Often more difficult to get the data
◦ Analysis is more complex
3636
Attribute/Discrete Variable/Continuous
Pass/Fail Actual Value
Poor Data Definition Excellent Data Definition
Larger Sample Required Smaller Sample Required
Usually Binomial or Poisson Distribution
Often a Normal Distribution
Count Data Continuous Data
37
If you can have ½ of something it is continuous; otherwise it is attribute◦ ½ a minute?
◦ ½ a customer?
◦ ½ of a widget?
◦ ½ second?
Note: Dividing attribute data by other attribute data will give a percentage but the result is still NOT continuous data.
• Number of people in California who drink wine out of the total population
• The resulting percentage is still attribute data
38
1. Number of foam pads with tears Variable Data ? Attribute Data?
2. The number of tears in each foam pad
Variable Data ? Attribute Data?
3. Delivery time in days for each shipment
Variable Data ? Attribute Data?
4. The number of seats that fail inspection
Variable Data ? Attribute Data?
5. The number of times a shipment is late
Variable Data ? Attribute Data?
6. Weld button diameter in inches Variable Data ? Attribute Data?
7. The length of time spent in the Emergency Department
Variable Data? Attribute Data?
8. The number of bad welds Variable Data ? Attribute Data?
39
Why Sample? ◦ The entire population of a dataset can rarely be
measured
◦ A sample is used to infer the characteristics of the population
Total Population
Sample
Generate Statistics
Inferences about the characteristics of the
population
40
A good sampling plan:◦ Is representative of the population
◦ Includes the variability of the population
◦ Is of a large enough size to be able to estimate the parameters to desired accuracy
Sampling Plan Considerations:◦ Goal of the sampling plan
◦ Levels of sampling
◦ Type of variation present
◦ Level of Risk for customer and manufacturer or organization
41
Questions to consider before sampling:◦ What is the goal of the data collection?
◦ What is the population to be sampled?
◦ How many samples should be measured from each group, shift or run?
◦ How will the random samples be chosen?
◦ How many measurements should be taken?
◦ What is the timeframe?
42
To avoid Bias◦ Utilize systematic sampling
good for business processes
take data samples at certain time intervals
take data sample from consistent transaction
◦ Don’t forget process variation when sampling Different work periods
Different experience levels with the same role
Previous experience with other organizations vs.. seasoned organization associates
Newly graduated vs.. career associates
43
Random Sampling:◦ Sampling without randomness ruins the
effectiveness of any plan
◦ Random sampling gives every part or process an equal chance of being selected for the sample
◦ The sample should be representative of the lot
◦ The sample can be determined by selecting a sample based on a random number table
44
Sequential Sampling:◦ Used for costly destructive sampling
◦ A sample size of one
Stratified Sampling:◦ Used when the group may not be homogenous
(machined with different machines or on different lines)
◦ Select random samples from each group
◦ Observations are based on relevant group not the overall system
45
When sampling a process:◦ Collect smaller samples more frequently
◦ Can take action and track changes through a control chart
Sampling approach:◦ Random sampling
◦ Stratified random sampling
◦ Systematic sampling (every Kth sample)
46
When sampling a large population:◦ Utilize statistics in our sampling
◦ Estimated with a certain degree of confidence typically 95%
47
48
1 An improvement team is interested in studying "first Call Resolution".
Approximately 15% of the callers are sampled to respond to a customer
survey. Only 10% of those sampled were willing to respond.
2 An improvement team is interested in improving billing accuracy. They have
decided to collect a sample of bills processed from 4 to 5 PM every day for
the next 4 days
3 An improvement team is studying the total cycle time for large deals. To
develop an understanding of the current cycle time, the team sampled deals
from the Atlanta Regional Office, who had data readily available.
4 An improvement team is interested in improving billing accuracy. They
decided to sample and pull every 20th bill processed over the next 30 days
5 An improvement team is interested in studying the likelihood that customers
will purchase new service. Approximately 10% of all 7,000 customers are
randomly selected to survey; approximately 80% of those sampled
responded to the survey
1. Define clear, concise data collection goals
2. Decide on what you are trying to measure and the procedures you will use to collect the data
3. Attempt to get an accurate picture of what’s is happening in the process over time
4. Begin collecting data, making sure data is consistent and stable in the way it is collected
5. Continue improving measurement stability & consistency
49
50
Step Action Airline Queue Example
1
Define clear & concise data collection
goals
• desired outcomes
• what should be measured
• customer comments
• why are you collecting data?
• how will the data help you
• what will you do with data
US Air decided to find out how to reduce the time that people had to
wait in lines at the airport:
• average wait time
• when are times particularly bad
• factors contributing to long waits
• special causes associated with long wait
2
Decide on what you are trying to
measure & the procedures you will
use to collect the data
• operational definition
• types of data
• inputs (what do you look at)
• process (efficiency factors)
• output (key quality aspects)
• sampling
• critical measurement points
• timelines
Developed operational definition:
• time people wait - measured in minutes & seconds
• cycle - when a person steps in line to the service representative
• data collection form - stratify time of day; day of week; no. of
representatives in service, no. of people in line.
• use digital stop-watch
• start watch when person enters line, then stop the watch as soon as
the service representative begins to speak
• sample one person every 10 minutes, each day, over a three week
period
• sample at a different time very day
51
Step Action Airline Queue Example
3
Attempt to get an accurate
picture of what is happening
over time (consistency &
stability over time)
• measurement plan
• who will collect
• collection form
• agreement on terms
• collecting & recording data
Measure the same thing in the same way throughout the data collection
process:
• looked for data collection stability by determining what factors might
cause measurements to vary
• observers must start the timer the same way every time
• observers must count the number of people in line the same way - do
not count the person at the service counter
• observers must count the number of service representatives the same
way - just those who are assisting customers and not supervisors or
managers
• everyone must agree on what to put down on the collection form by
giving definition to terms
4
Begin collecting data (make
sure data is consistent and
stable in the way it is collected
• check for understanding
• check for agreement
• assign responsibility
Train & practice
• trained all observers on how to use the stop-watch & fill out forms by
having them practice at the airport
• reviewed observer gathering data to make sure the data was collected
accurately
• all observers met to discuss procedures and recommend modifications
for a smooth approach
52
Step Action Airline Queue Example
5
Continue improving
measurement stability &
consistency (document any
special issues associated with
study)
• descriptive statistics
• special causes
• common causes
Complete data collection effort:
• calculated the average wait time
• correlated wait time with time of day
• looked at times of day over the course of the week
• data showed that the longest wait times occurred on Sundays between
1pm and 6pm, and during the week on Mondays & Fridays from 7:30am to
10am
• longer wait lines occurred when airlines & travel agencies had special
promotions
QI Macros > Calculators > Sample Size Calculator > Sample Size Calculator Tab
53
Set the a and b levels• How confident you
are that the true mean is between
the Interval
Set the interval (1/2 of the interval you are willing to accept e.g.
your specification range)
• If your spec range is 0.9 to 1.1 then
your interval would be 0.1)
54
Enter the population if known
If you are using attribute data, enter the current defect
rate (approximate if needed)
Enter the standard deviation for the
process
55
Currently estimating that you are late 35% of the time. You are willing to be +/- 3% (0.03). Using the typical a and b levels (0.05 and 0.10 respectively), how many samples would you need to accurately assess the actual rate?
How about if you estimated that you were late 10% of the time?
Try on your own
56
Estimating a 35% rate would require 971 samples to verify…
Estimating a 10% rate would require 384 samples to verify…
57
58
We are at the mercy of chance!
We can’t control it
then
We don’t know enough about it
then
If we can’t accurately measure something
then
59
USABLE INFORMATION
that statistics converts to…
DATA
We measure to get
60
A lot of data is available but not necessarily used
Measure and gather data effectively which provides the improvement team with specific valuable information
Remember to gather data which reflects what your process or product looks like to your customers
6161
Think through what you are trying to learn about the process inputs (X’s or causes) and decide what specifically needs to be measured
There are many inputs (X’s) that can be measured in a process - remember to measure against customer requirements –this will help to keep the team focused
62
Measuring allows the improvement team to see how input variables (X’s or causes) upstream in the process can affect the outputs (Y) of the process under investigation.◦ as an example: the time of day in which a
process happens could impact the overall cycle time a customer experiences when completing the process under investigation
63
64
Example & Question:◦ The improvement team needs to gather data on
a process input (x) of the time a patient arrives in the hospital emergency department
◦ This time has been defined by the team as time the patient walks through the doors of the ED or arrives by ambulance
◦ The existing data available is when the patient checks in at the emergency department registration or triage desk
◦ Will this data work?
65
Example & Question:◦ The improvement team has decided to gather
data on a process input (X) of the time a customer spends on the phone with customer service resolving an issue
◦ This time has been defined as the time the customer reaches the first person they speak with to the time the issue is resolved
◦ The existing data available is time stamped with each customer service rep the customer speaks with but does not account for the time spent in-between on hold or transferring
◦ Will this data work?
66
Why won’t the previous 2 examples work?
What do you do?◦ COLLECT REAL TIME DATA THAT REPRESENTS
THE SPECIFIC PARAMETERS
However, use the historical data to provide additional segmentation and trend information
6767
Uncovering what specific data the improvement team needs to gather
Review existing data to ensure the data exactly matches what is needed
Also make sure the existing data is representative of the sample and is of the stratification the improvement team has decided is within the scope of the project
6868
Historical Data◦ Often contains data inaccuracies
◦ Records are often incomplete often specific information on some variables might be
missing
attention was placed on variables that were not vital
some important variables might be missing
◦ Process variables might be strongly correlated but not understood
always validate historical data with real time observations
69
70
Measurement is critical to understanding the process
Determining WHAT to measure is often the most difficult part
Process measurements collected during the Measure Phase are to understand and baseline the current process
71
1. Observe the process
2. Select what to measure
3. Measure for a reason
4. Have a process for measurement
5. Plan and measure performance
72
1. Observe the Process◦ Observe first, then measure
◦ Watch the process step by step
◦ Validate the current state value stream process map
◦ Begin to identify failure modes
73
2. Select what to Measure◦ Understand the measurement type
Attribute inspection a special attribute is inspected resulting in a pass or fail
Defect inspection inspection for defect levels
classified by critical, major or minor defects
Variable measurement a specific parameter is measured at single or multiple
measurement sites
74
3. Measure for a Reason Ask yourself: Why is this particular part of the process
being measured? How does this measurement relate to the customer CTQ’s
There are two reasons to collect data:
◦ Measure efficiency and / or effectiveness
Efficiency measures how well your internal process are running (defect rates, process capability, etc.)
Effectiveness measures how your product of process looks to your customer (CTQ outputs)
◦ Discover how variables (X’s) upstream in the process affect the outputs (Y’s) delivered to the customer
The number of defects affect the cycle time due to rework processes
75
4. Have a process for measurement◦ Review your CTQ tree and SIPOC for KPOV’s and
KPIV’s
◦ Select the best measurements for each CTQ
◦ Develop Operational Definitions
What the measurement is
What the measurement isn’t
What is outside of the measurement scope
The basic definition of the measurement
Detail on how to take the measurement
76
Operational Definition
Elements Examples
What are you
trying to
measure?
Basic definition
or the measure
What the
measurement
isn't
How to take the
measument (in
detail)
➢ Satisfaction of customers in the Northeast
region with telephone support services
➢ Number of surface defects on the rear panel
➢ On-time delivery for Product X
➢ Are "customer comments" included under
"complaints"?
➢ Does "surface defects" include smears or only
scratches and dents?
➢ Satisfaction = X% of customers giving us a
score or 80 or above
➢ Surface defect = any dent or scratch visible from
a distance of 3 feet under normal light
➢ "Start the stopwatch when the customer steps
into the line, and stop it whenthe customer
leaves the front desk."
➢ "Use the standard calipers placed at the X-
junction to measure width in centimeters."
Definition – a clear, understandable description of what is to be observed and measured, so that different people taking or interpreting the data will be consistent
Factor in…Different terminologyAbility to focusSituations which might occurEvent which can be interpreted in several different ways
77
Measure 9 - Operational Definition Instructions
5. Plan and Measure◦ Creating the data collection plan
◦ Stratification factors
◦ Data collection plans
Hints for data collection
78
Determine:◦ What will be measured?◦ Who will collect the data?◦ What are the units of measurement?◦ Stratification factors (i.e. KPIVs)?◦ Over what timeframe will the data be collected
(sampling frequency)?◦ What number of samples will be collected at
each point in time (sample size)? ◦ Does the data already exist, or will it need to
be collected?
79
Measure 10 - Define Measure Data Collection Process
Key Process Input Variable (KPIV) information collected as part of the VOP data collection.
For example:
◦ KPOV → time for inspection
◦ Stratification factors (KPIVs that affect this KPOV):
Type of part
Severity of defect
Experience level of inspector
Time of Data
Others??
80
Data Stratification
FactorsExamples (Slice the
data by...)
Who
When
What
Where
➢Department
➢ Individual
➢Customer type
➢Region
➢ Plant
➢ Type of complaint
➢Defect category
➢Reason for incoming call
➢Month
➢Quarter
➢Day of week
➢ Time of day (hour)
➢Region
➢City
➢ Specific location on product (e.g. top
right corner)
You want to be able to “slice and dice” the data from the CTQ tree so that you can identify where there might be differences (good and bad)
You must gather the stratification information at the same time as the data – otherwise, you won’t be able to link the two
81
Measure 11 - Data Stratification Worksheet
▪ Example Data Collection Plan:
Data Collection Plan
Metric Name KPOV/KPIV
Who will
collect
data ?
Units of
measure
Stratification
factors
Sample
Freq
Sample
Size
Source
/locatio
n
Collection
method
Out of
bounds
How will
data be
used?
How will
data be
displayed
CMM inspection
time KPOV Jen Minutes Part number xx 50/month
CMM
reports/I
nfinity First piece
CAT
manifolds
only
correlation
analysis,
capability
analysis
Control
chart,
scatter plot
Read CMM report KPIV Bob Minutes Inspector
Deviation look up KPIV Dan Minutes Inspector
Notification time KPIV Bill Minutes
Supervisor,
opertor time
82
Data should be collected in it’s “rawest” form – no summary data.
Keep data sheets for future reference
Put data into Excel to facilitate Statistical Analysis
Sometimes data will be collected that is not immediately used…that’s O.K., keep it for future reference.
83
▪ Example Checksheet (with “Whys”):
84
Complete? Accurate? Timely?
▪ Example Process Observation Worksheet:
85
Measure 1 – Process Observation Worksheet
86
Measure 12 - Checksheet
H
W
S
HHH
Types of Damage
H = Hole
S = Smear
W = Wrinkle
This shipping box has several types of damage which can occur during packing and shipping. The relative location of each type of damage is shown on one diagram which represents many boxes. Note the large number of holes in the one location – this would lead the team to believe that some systematic cause is poking holes in the boxes and needs to be investigated.
87
Measure 13 - Concentration Diagram
Transactional Manufacturing Done?Planned
Date
Actual
Date
Planned
Duration
Actual
DurationNotes
Start Date Start Date X 14-Jun 15-Jun n/a n/a
Information Collected
from CustomerOrder Received X 15-Jun 19-Jun 1 days 4 days
Data from
customer
Initial Draft PreparedOrder/Press Set
UpX 17-Jun 18-Jun 2 days 1 day
Review with
CustomerOrder Run X 21-Jun 23-Jun 4 days 5 days
Final Version
PreparedOrder Packaged 22-Jun 1 days
Final Report
SubmittedOrder Shipped 23-Jun 1 days
Completion DataStep
Shows where you are in the process
Tracks data on time taken to complete
steps – lets you track differences
Notes are data which
help explain differences
88
Measure 14 - Confirmation Checksheet
Working as a team, spend 5 minutes…
Based on your understanding, what data do you need to collect on the class process; Use the Measure 6 – Data Collection Sheet to determine the data to be collected
Homework - Determine which check sheet(s) you might need to collect the data (Measure 7 – Data Collection Checksheets or other)
89
Measure 6 – Data Collection Sheet
Measure 7 – Data Collection Checksheets
90
Making sure your measurements are accurate◦ Variable and Attribute data – Gage R&R
Repeatability – can a single operator recreate his/her measurement on each part accurately?
Reproducibility – can two or more operators recreate the measurement on each part accurately?
Analyzing your data graphically and statistically◦ Plots◦ Capability◦ Etc.
91
GRR is a test of your measurement system◦ System includes:
Defect Definition
Measurers
Process
Documentation Tool
GRR becomes even more critical with transactional processes as you have many different people collecting the same data in different areas
92
GAGE R&R Instructions
1. Select 10 DIFFERENT parts and at least two "appraisers" and one gage
2. RANDOMLY, have appraisers measure each part at least twice
3. Then enter the results of each measurement in Cells C3:L7, C11:L15, C19:L23
4. The template will automatically calculate all of the values based on your data
5. Evaluate your measurement system based on %R&R and make adjustments as required.
Gage Repeatability and Reproducibility
Measurement is one common cause of variation. Gage R&R helps improve measurement systems
If repeatability (EV - Can the same person using the same gauge measure the same thing consistently)
is larger than reproducibility, reasons include:
1. Gage instrument needs maintenance
2. Gage needs to be redesigned
3. Clamping or location needs to be improved
4. Excessive within-part variation
If reproducibility (AV-can two appraisers measure the same thing and get the same result)
is larger than repeatability, reasons include:
1. Operator needs to be better trained in how to use and read gage
2. Calibrations on gage are not clear
3. Fixture required to help operator use gage more consistently
Gage System Acceptability
% R&R<10% Gage System Okay (Most variation caused by parts, not people or equipment)
% R&R<30% May be acceptable based on importance of application and cost of gage or repair
% R&R>30% Gage system needs improvement
(People and equipment cause over 1/3 of variation)
93
Measure 15 - Gage R&R Instructions
Scenario - One of the reasons that your boss needs you there on time is that you are the only one who seems to get good data with a variable gage on the shop floor. Over the past few weeks he has had you and another employee run 10 samples two times each to see how well that gage works.
Open the file Gage R&R and follow along with your instructor
94
Gage R&R
Gage R&R http://www.aiag.org/ Part Number http://www.qimacros.com/free-lean-six-sigma-tips/aiag-msa-gage-r&r.html
Average & Range Method 1 2 3 4 5 6 7 8 9 10 Sum
Appraiser 1 Trial 1 0.65 1 3.250
Enter your data here-> Trial2 0.6 1
Trial3
Trial4
Trial 5
Total 1.25 2
Average-Appraiser 10.625 1 #N/A #N/A #N/A #N/A #N/A #N/A #N/A #N/A
Range1 0.05 0 #N/A #N/A #N/A #N/A #N/A #N/A #N/A #N/A
Appraiser 2 Trial 1 0.55 1.05 3.100
Enter your data here-> Trial2 0.55 0.95
Trial3
Trial4
Trial 5
Total 1.1 2
Average-Appraiser 20.55 1 #N/A #N/A #N/A #N/A #N/A #N/A #N/A #N/A
Range2 0 0.1 #N/A #N/A #N/A #N/A #N/A #N/A #N/A #N/A
Appraiser Trial 1
Enter your data here-> Trial2
Trial3
Trial4
Trial 5
Total
Average-Appraiser 3#N/A #N/A #N/A #N/A #N/A #N/A #N/A #N/A #N/A #N/A
Range3 #N/A #N/A #N/A #N/A #N/A #N/A #N/A #N/A #N/A #N/A
EV (Equipment Variation)0.0332 Equipment Variation (EV)
%EV 11.3% 39.9% # Parts #Trials #Ops % of Total Variation (TV)
AV: (Appraiser Variation)0.02066 2 2 2 Appraiser Variation(AV)
%AV 7.0% 24.8% % of Total Variation (TV)
R&R (Gage Capability) 0.0391 Repeatability and Reproducibility (R&R)
%R&R 13.3% 47.0% NDC 11 % of Total Variation (TV)
PV (Part Variation) 0.2917 Part Variation (PV)
%PV 99.1% 350% % of Total Variation (TV)
95
After opening the file, go to DOE Gage R&R FMEA > Gage
R&R
Enter the data from the file into the proper cells of the Variable Gage R&R template (you can cut and paste it)◦ Be sure to paste the values!
The part numbers are in row 2
96
The trials for each appraiser on in column B
%EV – Repeatability Red (Not good)%AV – Reproducibility Yellow (Borderline)%R&R Yellow (Borderline)NDC – Number of Distinct Categories Yellow (Borderline)
97
Range Average 0.0340 Constants
XDiff 0.0240 10 Trials9Trials 8 Trials 7 Trials 6 Trials 5 Trials 4 Trials 3 Trials 2 Trials # Trials 2
UCL 0.1112 1.777 1.816 1.864 1.924 2.004 2.11 2.28 2.58 3.27 D4 3.27
LCL 0.0000 0.223 0.184 0.136 0.076 0 0 0 0 0 D3 0
Repeatability(EV) 0.0301 0.308 0.337 0.373 0.419 0.483 0.577 0.729 1.023 1.88 A2 1.88
Reproducibility(AV) 0.0156 0.3249 0.3367 0.3512 0.3698 0.3946 0.4299 0.4857 0.5908 0.8862 K1 0.886226
Gage Capability(R&R) 0.0339 0.7071 0.5231 K2 0.7071
Spec Tolerance 0.7 2 Ops 3 Operators
3.0775 2.97 2.8472 2.7044 2.5344 2.3259 2.0588 1.6926 1.1284 d2
% Using % Using Gage system may be acceptable based on importance of application and cost
AIAG - Automotive Industry Action Group FormulasTV Tolerance Gage may need maintenance, redesign, or better clamping
EV (Equipment Variation)0.0301 Equipment Variation (EV)
%EV 19.7% 25.8% # Parts #Trials #Ops % of Total Variation (TV)
AV: (Appraiser Variation)0.01558 10 2 2 Appraiser Variation(AV)
%AV 10.2% 13.4% % of Total Variation (TV)
R&R (Gage Capability) 0.0339 Repeatability and Reproducibility (R&R)
%R&R 22.1% 29.1% NDC 6 % of Total Variation (TV)
PV (Part Variation) 0.1494 Part Variation (PV)
%PV 97.5% 128% % of Total Variation (TV)
TV (Total Variation) 0.1532 Total Variation (TV)
19.7%10.2%
0.0%
20.0%
40.0%
60.0%
80.0%
100.0%
120.0%
%EV %AV
Components of Variation
98
• A good gage will have an %R&R < 10%; a bad gage should be > 30%
• The Number of Distinct Categories should be >6• You want most of the variation contribution to be made up by
the parts
Range Average 0.0340 Constants
XDiff 0.0240 10 Trials9Trials 8 Trials 7 Trials 6 Trials 5 Trials 4 Trials 3 Trials 2 Trials # Trials 2
UCL 0.1112 1.777 1.816 1.864 1.924 2.004 2.11 2.28 2.58 3.27 D4 3.27
LCL 0.0000 0.223 0.184 0.136 0.076 0 0 0 0 0 D3 0
Repeatability(EV) 0.0301 0.308 0.337 0.373 0.419 0.483 0.577 0.729 1.023 1.88 A2 1.88
Reproducibility(AV) 0.0156 0.3249 0.3367 0.3512 0.3698 0.3946 0.4299 0.4857 0.5908 0.8862 K1 0.886226
Gage Capability(R&R) 0.0339 0.7071 0.5231 K2 0.7071
Spec Tolerance 0.7 2 Ops 3 Operators
3.0775 2.97 2.8472 2.7044 2.5344 2.3259 2.0588 1.6926 1.1284 d2
% Using % Using Gage system may be acceptable based on importance of application and cost
AIAG - Automotive Industry Action Group FormulasTV Tolerance Gage may need maintenance, redesign, or better clamping
EV (Equipment Variation)0.0301 Equipment Variation (EV)
%EV 19.7% 25.8% # Parts #Trials #Ops % of Total Variation (TV)
AV: (Appraiser Variation)0.01558 10 2 2 Appraiser Variation(AV)
%AV 10.2% 13.4% % of Total Variation (TV)
R&R (Gage Capability) 0.0339 Repeatability and Reproducibility (R&R)
%R&R 22.1% 29.1% NDC 6 % of Total Variation (TV)
PV (Part Variation) 0.1494 Part Variation (PV)
%PV 97.5% 128% % of Total Variation (TV)
TV (Total Variation) 0.1532 Total Variation (TV)
19.7%10.2%
0.0%
20.0%
40.0%
60.0%
80.0%
100.0%
120.0%
%EV %AV
Components of Variation
99
Change the value in cell I4 to match I3 and see how it changes the analysis…
Small changes make a difference….
Reproducibility(AV) 0.0108 0.3249 0.3367 0.3512 0.3698 0.3946 0.4299 0.4857 0.5908 0.8862 K1 0.886226
Gage Capability(R&R) 0.0262 0.7071 0.5231 K2 0.7071
Spec Tolerance 0.56 2 Ops 3 Operators
3.0775 2.97 2.8472 2.7044 2.5344 2.3259 2.0588 1.6926 1.1284 d2
% Using % Using Gage system may be acceptable based on importance of application and cost
AIAG - Automotive Industry Action Group FormulasTV Tolerance Gage may need maintenance, redesign, or better clamping
EV (Equipment Variation)0.0239 Equipment Variation (EV)
%EV 17.0% 25.6% # Parts #Trials #Ops % of Total Variation (TV)
AV: (Appraiser Variation)0.01076 10 2 2 Appraiser Variation(AV)
%AV 7.6% 11.5% % of Total Variation (TV)
R&R (Gage Capability) 0.0262 Repeatability and Reproducibility (R&R)
%R&R 18.6% 28.1% NDC 7 % of Total Variation (TV)
PV (Part Variation) 0.1384 Part Variation (PV)
%PV 98.3% 148% % of Total Variation (TV)
TV (Total Variation) 0.1409 Total Variation (TV)
17.0%7.6%
0.0%
20.0%
40.0%
60.0%
80.0%
100.0%
120.0%
%EV %AV
Components of Variation
The tolerance is the span of your customers specification range.
The tolerance dictates the resolution of your gage.◦ If you gage does not allow you to see the difference
between two different values, you will have difficulty accurately measuring the product or service.
100
101
The first value is the contribution of that element to the total variation in the gage
The second value is how much of the part tolerance is taken up by that element• The wider the tolerance, the more useful
the gage will be in that application
Your biggest concern is getting the % of Total Variation down
You must designate a “master” for the MSA; this should be a person who is regarded as the most reliable inspector (this could even be the customer)
MSA Prep◦ Select at least 30 parts from your process with an approximate mix of 50/50 split on
good and bad items; don’t make them obvious one way or the other (use just 10 for this experiment)
◦ Select current inspectors and/or back ups◦ Review the attribute(s) in question with the inspectors
MSA Inspection◦ Label or locate each item so that they can be identified◦ Have each inspector look at the parts ONCE in random order to determine pass and
fail WITHOUT the other inspectors watching (enter the data in the Attribute R&R matrix as they are inspected)
◦ After each inspector has completed the first run, start on a second run with each inspector
Use the tab Attribute GageR&R on the Gage R&R Excel Sheet to enter your data (from the next page); be sure to enter the “experts” column
102
You are also the head inspector for the product that your company produces. Lately there have been complaints about the quality of the product from customers. You need to have 3 inspectors look at the candy to see if they can agree.
Use the data on the next slide to enter into the Attribute MSA in QI Macro; do this exercise on your own
The file is in the Attribute Gage Worksheet tab in the Gage R&R menu you just opened
103
Appraisal ResultsThe Masters
Appraisal
Part Number
104
Appraiser A Appraiser B Appraiser C
Part # Trial 1 Trial 2 Trial 3 Trial 1 Trial 2 Trial 3 Trial 1 Trial 2 Trial 3 Reference
1 1 1 1 1 1 1 1 1 1 1
2 1 1 1 1 1 1 1 1 1 1
3 1 1 1 1 1 1 1 1 1 1
4 1 1 1 0 1 1 1 1 1 1
5 1 1 1 1 1 0 1 1 1 1
6 1 1 1 1 1 1 1 1 1 1
7 1 1 1 1 1 1 1 1 1 1
8 1 1 0 1 1 0 1 1 0 1
9 1 1 1 1 1 1 1 1 1 1
10 1 1 1 1 1 1 1 1 1 1
Ultimate Score
Score of the gage “system” by itself; helps to determine
if the Appraisers need to confer with the “expert”
105
%
Appraiser
% Score
vs
Attribute
A B C A B C
Total 10 10 10 10 10 10
# Match 9 7 9 9 7 9
Mixed 1 3 1
95% UCI 100% 93% 100% 100% 93% 100%
Score 90% 70% 90% 90% 70% 90%
95% LCI 55% 35% 55% 55% 35% 55%
System %
Effective
Score
System %
Effective
Score vs
Ref
Total 10 10
# Match 8 7
95% UCI 97% 93%
Score 80% 70%
95% LCI 44% 35%
Typically, you want a System % Effective Score vs. Reference to be > 80% although through consistent gage studies, visual aids and training, it can go
higher
Appraiser “B” needs to be retrained on the process,
expectations and the visual aids
Statistical analysis is used to:◦ Describe our products and processes
◦ Draw conclusions and predict results
When applying statistics:◦ Measure the right thing (KPIVs and KPOVs)
◦ Make sure the data is “Good” (your R&R)
◦ Be confident the conclusions are correct (have enough data)
◦ Be efficient
106
Descriptive versus Inferential Statistics◦ Descriptive – describes the feature of a set of
any set of data (average, skew, etc.)
◦ Inferential – analysis of a sample to learn about a population
Data Sets can be characterized by:◦ Shape of distribution
Skew
◦ Central Tendency Mean (average), Median, Mode
◦ Variation Range, Standard Deviation, Variance
107
You can:1. Graph it
2. Calculate it Location – average, median, etc.
Dispersion – standard deviation, range
3. Predict Normality, z score
Non normal data
108
Histograms and Dotplots◦ Shows a snapshot in time
Run Chart◦ Shows a snapshot over time period
Control Chart◦ Is my process in control?
Pareto Chart◦ The vital few
Box and Whisker (Boxplots)
109
A normal (bell curved) distribution is 6 standard
deviations wide! Most of the data is in the middle (+/-
1 standard deviation).
s s s sss
68%
13.5%13.5%2.5%2.5%
Upper Customer Specification Limits
(USL)
Process Performance at a Snapshot in Time
Lower Customer Specification Limits
(LSL)
111
CenterProcess
ReduceSpread
Off-Target Too Much Variation
Centered On-Target
112
The more potential space between the tails of the curve and the specification limits – the higher the sigma level!
Understand the customer requirements◦ Specification limits
Reduce process variation◦ Narrower bell curve
Perform the process consistently time after time
◦ Centered Process between the customer expectations
113
Represent different ways of characterizing the central value of a collection of data
Six Sigma will utilize three measures:◦ Mean
◦ Median
◦ Mode
114
Mean calculation:◦ the average
◦ the sum of the data points divided by the total number of data points
◦ What could be wrong if you only used this measure?
Example: Salaries in the room
How do outliers affect our mean?
115
Median calculation:◦ the middle of the data set
◦ numbers have to be ordered
◦ if the number of values is odd, the median is the center number
example: 4, 6, 6, 7, 8, 9, 10
◦ if the number of values is even, the median is the average of the two center numbers
example: 4, 6, 6, 7 (7.5) 8, 9, 10, 10
116
Mode calculation:◦ the most frequently occurring number within
the data setexample: 4, 6, 6, 7, 8, 9, 10
◦ it is possible for a group of data to have more than one mode
example: 4, 6, 6, 7 , 8, 9, 10, 10
117117
Represents the spread of a distribution
Six Sigma will utilize four measures:◦ Range
◦ Standard Deviation
◦ Variance (standard deviation squared)
◦ Coefficient of Variation (COV)
Standard deviation / mean (or average)
Used to compare the variation in different processes
118
Range calculation:◦ easy way to measure variation
◦ the difference between the largest and the smallest value in a data set
◦ tells us how wide the data set isexample: 4, 6, 6, 7, 8, 9, 10
10 – 4 = 6
range = 6
119
Variance calculation:◦ is a measure of spread, just like range
◦ is equal to standard deviation squared
• The formula for the variance is:
s = (X - u) / N2
120
Where X = the individual data point; u = the overall average; N = the number of samples
2
Standard Deviation:◦ The square root of the variance
◦ The square root of the average deviations of all data values from the mean divided by the number of samples
Why Use Standard Deviation?◦ Unlike the range, the standard deviation takes into
account all the data values in the sample
◦ Unlike the variance, the standard deviation has the same units of measurement as the original data
121121
122
Population Mean
u = Σ ( Xi ) / N i = 1 to N
Population Standard Deviation
O= Σ ( Xi - u ) / N i = 1 to N
-
Sample Mean
X = Σ ( Xi ) / n i = 1 to N
Sample Standard Deviation
O = Σ( Xi - X ) / N - 1 i = 1 to N
-
_
_
Standard Deviation: the square root of the variance
N is used for a population, and N-1 for a sample (to remove potential bias in relatively small samples - less than 30)
Standard Deviation
2 2
A bar graph in which data are grouped into classes.
The height of each bar shows how many data values fall into each class
Shows how well a process is operating
Look at:◦ Is it centered correctly
◦ Wide or narrow dispersion
◦ Symmetrical, lopsided, cliff-like, twin-peaks, flat
◦ Irregularities
123
Enter the data on the right into Excel and run:◦ Be sure to highlight the data
Histograms and Capability > Histograms with Cpand Cpk◦ Subgroup size = 1
◦ Specification Limits
10 for the upper; 0 for the lower (we will cover this later)
◦ Choose 7 bars◦ Note: Rule of thumb depending on the # of data points
if under 50 Need 5 to 7 classes (bars)
50 to 100 Need 6 to 10 classes (bars)
100 to 250 Need 7 to 12 classes (bars)
over 250 Need 10 to 20 classes (bars)
◦ Click OK for the default titles
3.3
1.9
6.2
4.6
3.9
3.5
2.7
1.6
3.2
0.8
0.2
5
1.9
2.2
1.1124
Note: The number of bars is not exact; QI Macros may adjust the number
125
Any Observations?Note: Changing the bars will change
the shape!
126
UCLLCL
Type: symmetric bell-shaped
Behavior: if chart shows no special causes, data may come from a stable process
Action: determine when common causes will occur & control
127
UCL
LCL
Type: skewed
Behavior: large pile up of observations around a maximum reading
Can be skew to one side or the other depending on the
desired target of the process
128
UCLLCL
Type: bi-modal: two humps with special causes
Behavior: data may come from a mixture of two processesAction: investigate before further analysis
129
UCL
LCL
Type: concentration outside of UCL - special cause ?
Behavior: could be different operators / one re-measuring dataAction: standardize process for recording & repeating readings
Cp= Process Capability. A simple and straightforward indicator of process capabilityCpk= Process Capability Index. Adjustment of Cp
for the effect of non-centered distribution◦ Cpk is a measure of both the spread and centeredness of
a process
◦ Cpk looks at the 6 sigma limit of the tail of the curve farthest from the specification midpoint (e.g. tail closest to a specification limit)
◦ The larger the Cpk the better for continuous improvement (look for values of > 1.33)
130
You pull into a your garage and get out of the car and view your parking within the parameters of the space
Cp will look at where you parked in relation to the outside walls and how much room you have before hitting the wall
Cpk will look at how consistently you park in the same spot over and over again
131
Capability – You decide to run capability studies on how long it takes you to get ready to do several stages to see how capable you are at meeting your targets:◦ Stage 1 - Alarm to going downstairs for breakfast (20-
25 minutes)◦ Stage 2 - Fixing breakfast to out-the-door (15-20
minutes)◦ Stage 3 - Out-the-door to arrival at work (30-40
minutes)
Open the file Stage Times and follow along with the instructor as we run a capability study on the First Stage (see the next slide for the steps)
132
Stage Times
1. Highlight the data for the First Stage only
2. Select Histograms & Capability > Histogram with Cp Cpk◦ Choose the subgroup size (chose 1 for this
exercise)
3. Enter the upper specification limit (25)
4. Enter the lower specification limit (20)
5. Select OK on the number of bars and the titles
133
What do you see?
134
0
2
4
6
8
10
12
14
16
18
20
19.66 20 20.34 20.68 21.02 21.36 21.7 22.04 22.38 22.72 23.06 23.4 23.74 24.08 24.42 24.76 25.1 25.44
Nu
mb
er
Values
HistogramLSL 20.000 USL 25.000
Mean 23.044Median 23.047Mode 23.044n 50 Cp 1.836
Cpk 1.437CpU 1.437CpL 2.236Cpm 1.165Cr 0.545ZTarget/DZ1.170Pp 1.793Ppk 1.403PpU 1.403PpL 2.183Skewness 0.172Stdev 0.465Min 22.020Max 24.409Range 2.389Z Bench 4.310% Defects 0.0%PPM 0.000Exp PPM ST8.145Exp PPM LT12.775Sigma 5.810
0 1 2 3 4 5 6 7 8 9 10 11
You might gather data over time by taking multiple samples at regular intervals. This would result in “rational subgroups”.
Subgroups are important to the analysis of capabilities.
Long term
Short Term
135
GB+
12/10/2013
Pg 136
LSL USL
Remember – a single bell curve spans 6 standard deviations. At a Cp of 1.0,
exactly one bell curve would fit within the Upper and Lower Specification Limits
(this would be a 3 sigma process). A Cp of 2.0 would signify that two bell curves
would fit within the specification limits (this would be a 6 sigma process)
A 1.0 Cp / Pp process has no wiggle room for error or
you will be out of specification
12/10/2013
P
g
1
3
7
Which ever value (Cpl or Cpu) is smaller will determine the Cpk for the
process. The lower the Cpk, the more likely the process will be out of spec.
Cpk = minimum value of Cpl or Cpu• Cpl = (average – LSL) / (3 x short term standard
deviation)• Cpu = (USL – average) / (3 x short term standard
deviation)
The calculations for Ppk is identical except you use the long term standard deviation
The “C” values are for short term data; the “P” values are for long term data◦ Process Capability (Cp) is based on short-term
sigma (standard deviation within the subgroup)
◦ Process Performance (Pp) is based on long-term sigma (standard deviation between the subgroups)
The larger the subgroups, the smaller the Cpvalue since the standard deviation rises with the addition of more sample◦ As the subgroup size approaches the total sample
population, Cp approaches Pp
138
Cp/Cpk can be used to predict the “entitlement” of the process.◦ In other words – how good can the process operate
in the ideal world with no outside influence
Pp/Ppk is a more accurate reflection of the long term of the process over time◦ Long term is defined as a situation where 80+% of
the variables have fluctuated across their normal range
139
Homework – Run the 2nd and 3rd stage capabilities and save the resulting analysis; what are your observations? Run different subgroup sizes to see how Cpk changes
140
141
Formulated by Joseph M. Juran
Juran used the 80-20 Principle to focus on the most frequent and serious causes of poor quality
A graphical method of identifying causes of issues
Graphically represents the largest areas of defects
142
Demonstrates the 80-20 Principle, a scientific rule proven in business and economics◦ for many events, 80 percent of the effects come
from 20 percent of the causes
Utilizes discrete data (categories, groupings) ◦ example: complaint categories
143
Open the file Pareto
Highlight columns A and B
Run Pareto Chart in QI Macros
Click OK on the pop up menus
Afterward, run a Pareto on column E (reasons for shipments not being sent)
144
Pareto
34
28
1612
10
34.0%
62.0%
78.0%
90.0%
0.0%
10.0%
20.0%
30.0%
40.0%
50.0%
60.0%
70.0%
80.0%
90.0%
100.0%
0
10
20
30
40
50
60
70
80
90
100
Problem Not
Resolved
Hold Time Problem Not
Resolved to
Satisfaction
Passed to Operator
>2 Times
Unfriendly
Customer Rep
Pare
to C
hart
Categories
Pareto Chart
145
Biggest Hitter
146
Can be used in the Charter to show the state of the current process (KPOV’s)
Also called a time series plot
A trend line can be added to a run chart
Graphical method showing current process performance over time
Utilizes continuous data and requires time/date labels
A run chart shows:◦ Variation over a period of time versus a Histogram which
is a snapshot in time
◦ The data points in the order in which they occur
◦ Shows the variation in the process result over time
147
Pro
cess
Mea
sure
men
t
Time Period
Median or
average
148
QI Macros > Run Charts > Average Use the file Run Chart
Highlight columns A & B
QI Macros > Run Charts > Average
Click “OK” for all titles
Note – this is data for the drive time in the class scenerio
149
Run Chart
150
33.41
33.91
34.41
34.91
35.41
35.91
36.41
36.91
37.41
37.91
38.41
Tim
e t
o W
ork
Date
Time to Work
Time to Work
Average
To add data labels, click directly on the lines formed by the data
Right click and select “Add Data Labels” and data
values will appear
To add a trendline, click directly on the lines formed by the data
Right click and select “Add Trendline…”
Select the type of trendline
151
152
34.4
37.7
35.6
34.3
34.9 34.9
33.8
34.6
37.5
34.5
34.2
36.5
35.6
34.9
34.3
34
34.3
34.7
35.435.6
33.41
33.91
34.41
34.91
35.41
35.91
36.41
36.91
37.41
37.91
38.41
Tim
e t
o W
ork
Date
Time to Work
Time to Work
Average
Linear (Time to Work)
153
Can be used in the Charter to show the state of the current process (KPOV’s)
Another graphical method of showing current process performance
◦ Control Charts can be used in the Define/Measure phases as well as the Control phase
Shows how process performance varies over time Utilizes continuous data
Control Charts:◦ Can determine if a process is in control
◦ Can identify specific causes of noise variation
154
155
0Subgroup 10 20 30 40
0
10
20
30
In
div
idu
al
Valu
e
Mean=10.98
UCL=26.81
LCL=-4.854
0
10
20
Mo
vin
g R
an
ge
1
R=5.952
UCL=19.45
LCL=0
Total e-Pro Change Transactions by Account from Sep 2002 thru Mar 2003
Generally, anything that falls outside the red lines is “Out of Control”. Trends and other conditions can occur and will be marked in red.
QI Macros > Control Charts (SPC) > XmRIndividuals Use the file Run Chart
Highlight columns A & B
Click “OK” for all titles
Note – this is data for the drive time in the class scenario
156
Run Chart
UCL 38.14
CL 35.09
LCL 32.03
31.0
32.0
33.0
34.0
35.0
36.0
37.0
38.0
39.0
X V
alu
es
1/1/2014 - 1/28/2014
X Chart
The light blue points are the actual data points…
157
UCL 3.75
CL 1.15
0.0
1.0
2.0
3.0
4.0
Ran
ge
1/1/2014 - 1/28/2014
R Chart
This is a range chart – it is looking at the difference between a point and the previous point
The red lines and points denote Out-of-Control conditions
158
A point outside of the Control Limits◦ The control limits are +/- 3 standard deviations
from the mean
6 points in a row increasing or decreasing
2 out of 3 points more than 2 standard deviations from the mean
8 points in a row on one side of the center line
159
160
Champion Measure Phase Checklist
161
Measure 8 – Tollgate and Approval
Measure Phase CommentsWhat are your deliverables for this phase? Summarize the findings.
Has your Problem Statement or Objective Statement changed? If yes, why?
Have you completed a process map? Who was involved in its development? What
did you discover?
Have you determined the current defect level and/or KPOV's (ppm, DPMO, etc. on
metric chart)?
Have you developed your Data Collection Plan?
Have you studied the measurement systems used to collect the data (MSA)? Are
better gauges required and what would be the cost?
Have you implemented your Data Collection Plan?
Have you determined the capability of the process?
Have you completed a Pareto analysis?
Do you have any Run Charts or Control Charts on you KPIVs or KPOVs
Have you determined your next steps?
Have you updated your financial valuation?
What are your conclusions from this phase?
Are you on track to meet the scheduled completion date?
Are you satisfied with the level of cooperation and support you are getting?
Have you obtained the signatures from leadership to move on to the next phase?
Project Team Measure Phase Checklist
162
Measure Tollgate Approval
Champion Approval Signature/Date:
Tollgate review approved unconditionally:
Tollgate review approved with the following contingencies:
Tollgate review dis-approved, list issues for resolution:
163