1.5 LSS Quality Files Bus. Risk Management (1)

8/13/2019 1.5 LSS Quality Files Bus. Risk Management (1)

1/81

ASU Lean Six Sigma Green Belt DMAIC

Measure Phase

Introduction to Measure


2/81

Is the team operating within the context of the project charter?project scopegoal statement

project plan

Introduction to Measure

Staying on track ------------------------------------ Maintaining Perspective

1.0Define

Opportunities

2.0Measure

Performance

3.0Analyze

Opportunity

4.0Improve

Performance

5.0Control

PerformanceDefine Oppor tunit ies

Measure Perform ance

Analyze Oppor tuni ty

Improve

Performance

Contro l

Performance

Has fact- based decision making been consistently applied? Are the teams assumptionscontinuously validated?Is business risk being actively managed?Is leadership informed & on board with the teams findings & conclusions? Can the likelihood of success be improved by revisiting previous conclusions or analysis?



3/81

Changing Our Perspective

Select the Y that the customer uses to judge your performance

Start with the customer

Measure the same as the customer does

Understand the variation in the output (what your processproduces that is of value to the customer)

Use data to find the process keys that drive the variation

Outside-In Focus Drives DMAIC Success


4/81

What Is A Measure?

Measurement can be applied to any type of process or productto assess performance

For a process: Time required to complete process stepsTimely execution of processNumber of errors in different process steps

Percent yield of a processFor a product: Errors in a productProduct arrives/shipped on timeNumber of products shipped per month

A Measure Is A Description Of One CharacteristicOf An Object Or Activity


5/81

Uses Of Data And Measurement

Relation To The DMAIC Methodology

Collect facts about a problem or opportunity and voiceof the customer information

Establish a baseline performance for Project Y to

understand how well we meet customer expectationsIdentify the root cause of a problem and find the key tosolving the problem

Evaluate competing solutions based on their impact onperformance; degree and direction of change; to compareprocess performance before and after the solution isimplemented

Quantify the change in process performance to ensureimprovement gains are sustained

Define

Measure

Control

Improve

Analyze


6/81

Process Context For Measurement

CSuppliers Inputs Process Outputs Customers

CTQ CTQ

Input and process measures quantify someaspect of an input to the process or theperformance at one or more process steps. Theyare often referred to as independent variables .

Output measures typically quantify product orservice characteristics or process outcomes.They are often referred to as dependentvariables .

SSupplier CTQs

IInput PProcess OOutput CCustomer S t o p S t a r t

Output Measures Should Reflect What The Customer Feels


7/81

The Relationship Between Process Ys And Xs

Subprocess Ys May Be High-Level Xs

SubprocessL2

CoreProcess

L1

Credit Documentation FundingY YY

x = cycle time x = cycle time x = cycle time

Cash Delivery Process

Total Cycle Time = Y

NotificationX

Decision

X

Application

X

Y = Cycle Time


8/81

Effectiveness:The degree to which customer CTQs are met and exceeded

Some examples:Percent defectiveResponse time

Efficiency:

The amount of resources allocated in meeting and

exceeding customer CTQsSome examples:

Cost per transactionTurnaround time

Quality Measurement

Time per activity Amount of rework

Billing accuracy

Two Aspects Of Measuring Performance


9/81

Why Is Type Of Data Important?

Choice of data display and analysis tools Amount of data required: continuous data oftenrequires a smaller sample size than discrete data

Information about current and historical processperformance

Use Continuous Data Whenever Possible


10/81

Types Of Data

Discrete Data

Binary (Yes/No, Defect/No Defect)

Ordered categories (1-5)

Counts

Continuous Data

Can be broken down into increments

Infinite number of possible values

Examples

Number of incomplete applicationsPercent of responding with a 5 onsurveyNumber of Green Belts trained

Examples

Cycle time (measured in days, hours, minutes,etc.)Weight (measured in tons, pounds, etc.)

Data Type Is An Important Consideration

Discrete

C y cl eT i m

e

C o un

t D

a t a

( M an

y p o s si b i l i t i e

s )

Or d

er e

d C a t e g or i e

s

( M an

y o p t i on

s ,i . e. ,1 -1

0 0 )

Continuous

Bi n

ar y

( Y / N )

Technically Discrete, but can oftenbe analyzed as Continuous

C o un

t D

a t a

( L i mi t e

d p o s si b i l i t i e

s )

Or d

er e

d C a t e g or i e

s

( L i mi t e

d o p t i on

s ,i . e. ,1 -1

5 )


11/81

Importance Of Data Type

Project Y Discrete Y Measure Continuous Y Measure

Time to process % within specifications Actual times for each unit

Delivery time Number late Actual time deviated from target

Customer satisfaction Yes/no questions Rating 1-100

Policies lost due to price Number lost Difference from competition

The More Continuous We Can Make The Data,The More It Will Tell Us About Our Process

Sometimes we have choices. When we do, we shouldchoose continuous data


12/81

Plan For Data Collection

Ensure DataConsistencyAnd Stability

EstablishData CollectionGoals

DevelopOperationalDefinitions AndProcedures

Clarify purpose ofdata collection

Identify what datato collect

Test and validatemeasurementsystems

Write and pilotoperationaldefinitions

Develop andpilot datacollection forms

and proceduresEstablish asampling plan

Collect DataAnd MonitorConsistency

Train datacollectors

Pilot process andmakeadjustments

Collect data

Monitor dataaccuracy andconsistency

1 432

Data Collection Is The First Step To UnderstandingThe Variation The Customer Feels


13/81

What To Collect?

Operational Definitions & Procedures

Data CollectionForm

Date # Start StopInstructions

Start:

Stop:Measure Data Type UnitsCycle Time C Days

Sampling Plan

How Many Who

When Where

Training Plan For Collectors

Needs

Form Use

Audit Plan

Analysis Plan

Measure MSA% Error ImprovementSources Plan

MSA / Gage R & R


14/81

Establish Data Collection Goals

In order to establish your data collection goals you must:

State the purpose of the data collection

Identify what data is required

Asking these questions may help you clarify your goals:

What do I need to know about my process?

What data do I need?

What is the plan for analysis once the data is collected?

What data is already available?


15/81

Segmentation

Collect output data (Y)

To identify patterns and performance trends

Collect segmentation factor data

To be used for later analysis

Segmentation Helps Us Understand Variation In Project Y

Preparing for the Analyze Phase


16/81

Step 1: Establish Data CollectionGoals

Segmenting by external factors will help us identifythe drivers of variation in the process

Possible categories:

Product Customer Market Time Geography


17/81

Common Factors Used ForSegmentation

Factor Example

What type Complaints, defects, problems

When Year, month, week, day

Where Country, region, city, work site

Who Business, department, individual,customer type, market segment

Tip: Begin with factors outside the process box - often these are factorsthat were not considered when the process was first designed

Other Categories


18/81

How To Collect Data ForSegmentation

Identify the factors for segmentation before you startcollecting dataMake sure the segmentation factors can be measuredreliably

Record the segmentation factors for each Y data pointcollectedSegmentation factors are typically easy to collect, socollect more segmentation factors rather than less


19/81

Step 1: Establish Data CollectionGoals (5 Minutes)

Table Team Exercise

For your team project:Brainstorm a list of segmentation factors for yourprojectRemember to also segment on unlikelyparametersReport out to group


20/81

Develop Operational DefinitionAnd Procedures

Clearly specify variables to be collected:Operational definitions for all metricsSpecific descriptions of how to take themeasurement

Specify the details of the data collection process:How to collect the dataHow to record the dataThe period of time for data collection

The sampling plan to be followed



21/81

Defining The Measure

Purpose

An operational definition is a clear, concisedescription of a measurement and the process bywhich it is to be collected.

To remove ambiguity Everyone has a consistent understanding

To provide a clear way to measure the characteristic Identifies what to measure

Identifies how to measure it Makes sure that no matter who doesthe measuring, the results are consistent

Definition

Always Pilot Your Operational Definitions

Operational Definitions


22/81

Features

Operational Definitions For OutputMeasures

What: Must have specific and concrete criteria How: Must have a method to measure criteria Must be useful to both you and the customer

(the wing -to- wing concept)

Example: Loan Application Cycle Time

What: Loan application cycle time is the number of hours fromreceipt of a loan application, to successful notification ofdecision for the loan application

How: The clock starts when the computer attaches the time ofapplication receipt at data entry

The clock stops when the phone caller notes timeof completed application decision notification in desk log


23/81

Operational Definitions Scale OfScrutiny

Measure one scale or level smaller than what your customermeasures

For Example:

If your customer measures cycle time in days, your scale ofscrutiny would be hours

If your customer measures cycle time in hours, your scaleof scrutiny would be in minutes

Scale of scrutiny may expose larger true variation

Choo s ing The Level Of Measurement


24/81

Operational Definition PartnerExercise (25 Minutes)

What How Who Time

AllPartner Preparation

Develop AnOperationalDefinition

Measure AndRecord Data

Close Exercise

Develop an operational definition for oneof the defect types found in an M&M,either, 1) chips and cracks, or 2) unclear and illegible M.

The definition should include: What How Importance to customer

Find a partner for the exercise.

Determine timing for each activity below.

Read the background information.

With your partner apply your operational definition to your package of M&Ms .

Use the form on the following pageto record the total number of M&Ms you inspect and thenumber of defective M&Ms .

Note: If an M&M has one or m orechips/cracks, classify the M&Ms as defective.

Brainstorm the challenges of developingan operational definition for this exercise,and how these challenges may impactyour own project work.

Choose a spokesperson to report out onyour operational definition, the challengesyou experienced, and how these may

impact your project work in the future.

Partners

Partners

Partners

Desired OutcomesPractice applyingoperationaldefinitionsCollect data on the

number of defects ina package ofM&Ms


25/81

Operational Definition PartnerExercise (continued)

BackgroundCustomers of M&Ms candy have various needs related to the consumption of the candy.Because the candy should melt in your mouth, not in your hands, one of the Project YCTQs is for the candy to have no chips or cracks.

Part of the internal process for making the candy is printing the letter M on the candy.While not a high priority for external customers it is important to internal customers formarketing and product branding.

Customer Need

No chips or cracks

Project YCore Process M&Ms Production

Subprocesses

1 2 k

Business Need

Clear and legible Mon the M&Ms

MM

A defective M&M is . . .

1. Any M&M with a chip or crack.2. Any M&M with an unclear or

illegible M.

The two defect types should bemeasured separately.


26/81

Operational Definition PartnerExercise (continued)

Data Collection Check Sheet

Date: Location:

Data CollectorsName

# Of PiecesInspected

# Of Pieces ChippedOr Cracked

# Of Pieces WithUnclear Or Illegible M

Data Summary Sheet

Data CollectorsName

# Of PiecesInspected

% Of PiecesChipped

Or Cracked

% Of Pieces WithUnclear Or

Illegible M

% Of PiecesDefective

Operational Definition:


27/81

Operational Definition TableTeam (5 Minutes)

Write an operational definition for your Project Y Write the definition on a flip chart

Report out

Define Your Project Y


28/81

Understand the purpose and advantages of sampling

Understand the application of different samplingtechniques to ensure accurate process representation

Gain experience in asking appropriate questions toensure a robust sampling plan is implementedeffectively and efficiently

Understand guidelines and formulas used to

determine sample size

Objectives

Sampling


29/81

Basic Definitions And Symbols

Population (N): The entire set of objects or activities for aprocess

: the mean (arithmetic average) calculated for a

population : the standard deviation calculated for a population

Sample (n): a group that is a part or subset of a populationx: the mean (arithmetic average) of a sample

s: the standard deviation of a sample

Sampling


30/81

Sampling

Sampling is the process of:Collecting only a portion of the data that is available orcould be available, and drawing conclusions about thetotal population (statistical inference)

Population Sample

xx

x

xxx

x

x

x

xx

x x

x

x

xx

x

xxx

x

N = 5,000 n = 100

From the sample,we infer that theaverage resolutiontime (x) is 1.2 days

What is theaverage resolution

time?


31/81

When To Sample?

When to sample Collecting all the data is impractical or too costly Data collection can be a destructive process When measuring a high-volume process

When not to sample A subset of data may not accurately depict the process,

leading to a wrong conclusion (every unit is unique e.g., structured deals)

Statistically Sound Conclusions Can Often Be DrawnFrom A Subset Of The Total Available Data


32/81

Goal Of A Useful Sample

Representative Sample: All parts of the target population are represented

(i.e., selected for measurement) equally The customers view is captured

How to guarantee a representative sample: Designing the sampling strategy Understand special characteristics of the population

before sampling

Representat ive Samp les


33/81

Important Sampling Concepts

Bias occurs when systematic differences are introducedinto the sample as a result of the selection process

Not representative of the population

Will lead to incorrect conclusions about thepopulation

Bias


34/81


Convenience sampling: the ones I can reach Systematic sampling: at noon every day

How Is Bias Introduced?

Selection Bias

EnvironmentalBias

Strategic Level Developing The Sampling Plan

Outdated sample: 1996 external survey results


35/81


How Is Bias Introduced?

Measurement Bias

Gage R&R Issues

Non-Response

Bias

Tactical Level Carrying Out Sampling Plan

Initiated by respondents: only a subset of the

population responds to survey (typically the1s and 5s)

Inconsistent operational definitions

Inconsistent collectors or procedures (assess usingMeasurement Systems Analysis)


36/81

Determine Your Sampling Strategy

Population ApproachMake probability statements about the population from the sample

I have 95% confidence that the mean of the population isbetween1.5 and 2.5 seconds

Use sample size formulaProcess Approach

Assess the stability of the population over time Are the shifts, trends, or cycles occurring?Do I take a special or common cause variation approach toprocess improvement?

Use rational subgrouping

Where Are You Standing?

Process Data

Population Data


37/81

Determine Your Sampling Strategy

Think about these two questions:1. What should you do if you are standing at process but wish to

use a population approach?2. What should you do if you are standing at population data but

wish to use a process approach?

Where Are You Standing?

Process Data

Population Data


38/81

Sampling Strategy: Random Sampling

Population Sample Description

Each unit (X) hasan equal probabilityof being selected ina sample

Popula t ion S tudy

N n

X X XX X XX X X X XX X X X XX X X


39/81

Description

Population Study

Sample

L

MMMM

SS

Population

Segments Units

Large

Medium

Small

LLLLL

MMMMMM MMMMMM

S S SSSSSS

S S

Sampling Strategy: Stratified RandomSampling

Randomly samplewithin a stratifiedcategory or groupSample sizes foreach group aregenerally proportionalto the relative size of thegroup


40/81

Sampling Strategy: SystematicSampling

Process Sample Description

Process S tudy

Must select sampling frequency

Sample every n th one (e.g., 4thone)

X XX X X X X X X X X


41/81

Sampling Strategy: RationalSubgrouping

Process Sampling

X X X X X X X X X X X X X X X X X X X X X X X X

Hour 1 Hour 2 Hour 3Subgroup ofsamples

Process Sample

Description

Sample at point A in the process every Xth hour


42/81

Sampling Strategy: Process Study

Key points to considerMonitor process frequently enough to catch it goingfrom good to bad

Better to collect several small samples over differenttimes then one large sample at a single point in timeUnstable process more frequentlyStable process less frequentlyRapid cycle process time more frequentlyLong cycle process time less frequently

Determine Sam pl ing Frequency


43/81

Sampling Situations TableExercise (continued)

You are interested in estimating the proportion of callers (within1%) who experience first call resolution. A customer survey willbe used to gather the data. A random sample will be used to selectpotential survey respondents.1. Are there any potential problems with the approach described?

2. What other approaches might be used in this sampling situation?3. What other information is required?

You are interested in improving billing accuracy and have decided tocollect a subgroup sample of 30 bills processed from 4 to 5 p.m., everyday for the next 4 days.1. What sampling scheme(s) is planned?2. Is there a potential to introduce bias using the plan described? If

yes, how would the bias be introduced?3. What other approaches might be used in this sampling situation?

A

B

S li g Sit ti T bl


44/81

Sampling Situations TableExercise (continued)

A business wants to estimate the total cycle time for deals. There are three types ofdeals (large, medium and small) and four regional offices (Atlanta, New York, Chicagoand Los Angeles). The business randomly sampled deals from the Atlanta regionaloffice who had data readily available.

1. Are there any potential problems with the approach described?

2. What other approaches might be used in this sampling situation?

3. What other information is required?

.

D

C

An improvement team is interested in improving billing accuracy. They

decided to sample and pull every 20th bill processed over the next 30 days

1. What sampling scheme(s) is planned?2. Is there a potential to introduce bias using the plan described? If yes, how

would the bias be introduced?

3. What other approaches might be used in this sampling situation?


45/81

How Do I Determin e Sam ple Size?

Sample Size For Continuous Data

Sample size (n) depends on three things Level of confidence required for the result, How confident

I am that the result represents the true population Level of confidence increases as sample size increases

Precision or accuracy ( ) required in the result, The errorbars or uncertainty in my result

Precision increases as sample size increases Standard deviation of the population (s), How much

variation is in the total data population?

As standard deviation increases, a larger sample size is needed toobtain reliable results

In this equation, 1.96 represents a 95% confidence level

n = 1.96s ( )2


46/81

What Is The Perform ance For Del ivery Time?

Sample Size For Continuous Data

Calculate sample size (n) based on:Precision ( )95% confidence Level (1.96)Standard deviation (s)

Y = DeliveryTime

(Days)

NValues

Conclusion:

I know with 95% confidence that the population mean is X +

Calculate average (X)

Population

Sample

X - X + X

n = 1.96s ( )2


47/81

Finite Population Correction

1. Calculate sample size (n)

2. If n / N > .05

ORIf n > N

3. Calculate n finiten finite = n / 1+ n / N

If You Have A Fini te Pop ulat ion

Where n = sample size;N = population size


48/81

How Do I Determine Sample Size?

Sample Size For Discrete Data

Sample size (n) depends on three things:Level of confidence required for the result, How confident Iam that the result represents the true population

Level of confidence increases as sample size increasesPrecision or accuracy ( ) required in the result, The error bars or uncertainty in my result

Precision increases as sample size increasesEstimated proportion defective of the population (P)

Sample size is maximized at P = 0.5In this equation, 1.96 represents a 95% confidenceinterval

n = P(1-P)1.96 ( )2


49/81

Wh at Is Th e Defect Rate (P) Of A Proces s?

Sample Size For Discrete Data

n =1.96

( )2 P (1-P )

Y = ProportionDefective

nValues

Conclusion:

I know with 95% confidence that the populationproportion defective is P +

Calculate ProportionDefective (P)

Population

Sample

Recalculate n* based on the calculated P. If the new required sample size (n*) is more than thenumber of samples taken, take (n*-n) samples and recalculate P base on the full sample size. If it isnot practical to take more samples, then use the actual n and P to recalculate the actual precision ( )

Calculate sample size (n) basedon:

Precision ( )95% confidence Level (2)Estimated proportion defective (P)


50/81

Beyo nd The Form ulas . . .

Sample Size Considerations

The formulas give an approximate sample size Dont forget these important factors!

Is the population homogeneous?If not, you will need to segment before sampling

What is the opportunity segment for bias?Plan ahead to make sure your data is representativeof the true population

What Is The Impact On The Customer If YourSample Size Is Not Representative Of The Process?


51/81

Variation In Measurement Systems

Address Measurement System Variation Before

Collecting Data To Analyze Process Variation

Actual ProcessVariation

Observed Variation In Data

MeasurementSystem

Variation

Long Term Short Term Variation In TheMeasurement Tool

Repeatability

Reproducibility

Accuracy

Stability

Linearity

Variation In The Act OfMeasuring

Random

Bias

GageR&R

Issues


52/81

Two Cases Of Measurement Error

+ =

+ =

True

Process

Random Variation

Due ToMeasurement

Total Process

Variation Observed

TrueProcess

Random VariationDue To Bias

Total ProcessVariationObserved


53/81

Measurement System Analysis (MSA)

MSA is a set of methods for estimating the currentamount of variation in the measurement process


54/81

Plan For Consistency And Stability

Data is only as good as the process that measures itIdentifies how much variation is present in the measurementprocessUnderstanding measurement variation is necessary foridentifying true process variation and maximizing true YimprovementsWithout MSA, you run the risk of making decisions based onan inaccurate picture of your process

MSA helps direct efforts aimed at decreasing measurementvariationExcessive measurement variation distorts our understandingof what the customer feels

Why MSA Is Impo rtant


55/81

Measurement Systems Analysis

How much variation is caused by the measurement system?Determine which MSA is appropriate based on the type of data collectedDetermine which aspects of measurement are most relevant for the MSA study(accuracy, repeatability, reproducibility, stability, linearity)Measure units repeatedly. How items are measured depends on the aspectbeing quantifiedQuantify the measurement process variation

How much error or uncertainty is allowable for this data?Determine if the measurement process must be improved

What are the sources of measurement error?Determine how the measurement process will be improved

How can the error sources be eliminated or minimizedDetermine how the measurement process will be improvedAudit the measure process to ensure accurate and consistent measurement

Key Ques t ions And Procedures To Answ er Them1.

2.

3.

4.


56/81


57/81

Measurement System Analysis

The type of Measurement System Analysis conducteddepends on the type of data:

When using continuous data, MSA is conductedthrough a Gage R&R (repeatability andreproducibility) studyWhen using discrete data (discrete, count or orderedcategories), MSA is conducted through a DDA(discrete data analysis) study

Step 1: Determine Which MSA Tool Is Appropriate


58/81

Step 2: Determine Which Aspects Of Measurement Error Are Relevant For

Your MSA Study

Measurement System Analysis

Accuracy the differences between observed averagemeasurement and a standard

Repeatability variation when one person repeatedlymeasures the same unit with the same measuring equipment

Reproducibility variation when two or more peoplemeasure the same unit with the same measuring equipment

Stability variation obtained when the same personmeasures the same unit with the same equipment over anextended period of time

Linearity the consistency of the measurement systemacross the entire range of the measurement system

Measurement System Analysis:


59/81

StandardValue

ObservedAverage

Accuracy

Measurement System Analysis:Accuracy

The difference between observedaverage measurement and a master or standard

Continuous Difference between observed and standard inmeasurement units

Discrete Number of instances where the wrong answerwas observed



60/81

Measurement System Analysis:Accuracy (continued)

Validating accuracy involves repeated measurement ofsomething with a known value (master/standard) The difference between the average of repeated

measurementsof the same master/standard and the true value of

the master/standard represents the amount ofinaccuracy or bias in the measurement system Service application: Validating the judgement of the

person making the measurement against an agreed-upon master/standard



61/81

Step 3: Measure Un its Repeatedly

Word Spell Checker 1 Standard DictionaryCommittee C CBattallion C I

Asbestos C CSeperate I I

Flamboyant I C Abbacus I ICatagory I I

Lieutenant C COccassionally I I

Liquefy C C

Discrete Data Example

A marketer wants to understand the accuracy of his measurement process to measure the number of misspelled wordsin the first draft of a marketing brochure. A spell checker is given a brochure and asked to identify the words that arespelled incorrectly. Any difference between the words the spell checker identifies as misspelled, that are in realitycorrectly spelled (as defined by the standard dictionary), represents the accuracy of the measurement process.

C = Spell Checker identified the word as correctly spelledI = Spell Checker identified the word as incorrectly spelled

= Differences between Spell Checker and Standard Dictionary

DataSummary

Measurement System Accuracy = [(4 + 4)/(4 + 1 + 1 + 4) ] x 100 = 80%

Correct Incorrect

Correct 4 1

Incorrect 1 4

StandardDictionary

Spell Checker 1

5

5

10

5 5

Step 4: Quantify The Measurem ent Process Variat ion




62/81

Step 3: Measure Un its Repeatedly

20100

9

8

7

6

5

4

3

2

1

Standard Deal #

C y c l e

T i m e

( i n

d a y s

)

Measure of Deal Cycle Time

X=5.063

3.0SL=8.526

-3.0SL=1.599

(20 "Standard Deals")

Continuous Data ExampleA deal business is measuring the cycle time on deals. A reference set of 20 deals (standard) is collectedand a panel of experts determines the true cycle time for each deal. The average cycle time for the 20deals, as measured by the panel of experts is 5.5 days. The normal measurement process thenmeasures the cycle time for the 20 deals (see chart below).

Step 4: Quantify The Measurement Process Variation

The difference between the average of the 20 deals measured by the panel of experts (the standard) and theaverage of the normal measurement process represents the bias, or inaccuracy, of the measurement process.Thus bias is X standard Xnormal , or 5.5 5.1 = 0.4. This means the measurement process has a bias or inaccuracy of0.4 of a day in measuring cycle time.




63/81

(Minimum Variation)

Repeatability

Measurement System Analysis:Repeatability

The variation when one personrepeatedly measures the same unit withthe samemeasuring equipment

Continuous Calculate variation in terms of measurement units(standard deviation, span, etc.)

Discrete Count number of times the same result is achieved fora given point (% correct)



64/81

Measurement System Analysis:Repeatability (continued)

Validating repeatability involves repeatedmeasurement of the same item by one person with thesame measurement deviceThe difference between the first time an item ismeasured and the second time represents the error ofthe measurement processTherefore repeatability is the ability of the person ormeasurement device to consistently repeatmeasurements for the same items



65/81

Measurement System Analysis:Repeatability (continued)Discrete Data Example

Data Summary

Word First Check Second Check Committee C CBattallion I I


Flamboyant C C Abbacus I ICatagory I I

Lieutenant C COccassionally I C

Liquefy C C

A marketer wants to understand the repeatability of his measurement process tomeasure the number of misspelled words in the first draft of a marketing brochure. Aspell checker is given a brochure and asked to identify the words that are spelledincorrectly. After a short period, the spell checker is given the brochure and asked torepeat the process. The difference between the first and second spell check representsthe spell checkers repeatability.

C = Spell Checker identified the word as correctly spelled

I = Spell Checker identified the word as incorrectly spelled

= Differences between Spell Checks



66/81

Measurement System Analysis:Repeatability (continued)Cont inuo us Data Example

A deal business is measuring the cycle time on deals. A reference set of 20 deals is collected andmeasured by one person. The same person then re-measures the 20 deals. The difference between thefirst set of measurements and the second represents the repeatability of the measurement process.

1st 2nd5.4 5.54.7 4.65.5 5.56.3 6.33.9 3.94.7 4.54.8 4.85.5 5.54.6 4.64.7 4.76.2 6.25.0 4.83.9 3.96.9 6.84.4 4.5

4.0 4.04.8 4.87.1 7.23.7 3.75.2 5.4

Source % ContributionTotal Gage R&R 0.43Repeatability 0.43Part-to-Part 99.57Total Variation 100.00

Using either Minitab or Excel we can calculate the percent of the TotalVariation contributed by the measurement process in terms ofrepeatability (see partial printout above).

In this case we see that the repeatability of the measurement process is99.57%, which means that nearly all of the observed variation is comingfrom the process, not the measurement system.

Measurement SystemVariation

Actual Deal Variation

Total Observed Variation+



67/81

Reproducibility

Data Collector 1

Data Collector 2

Measurement System Analysis:Reproducibility

The variation when two or more peoplemeasure the same unit with the samemeasuring equipment

Continuous Calculate the difference between two people in terms ofmeasurement units

Discrete Calculate the difference in number of times each personachieved a given result (% difference)



68/81

Measurement System Analysis:Reproducibility (continued)

Validating reproducibility involves repeatedmeasurement of the same item by two people usingthe same measurement device

The difference between the two measures representsthe ability of the measurement process to bereproducible

Therefore reproducibility is the ability of themeasurement process to consistently reproducemeasurements for the same items across people



69/81

Measurement System Analysis:Reproducibility (continued)Discrete Data Example

Word Spell Checker 1 Spell Checker 2Committee C CBattallion C I


Flamboyant I C Abbacus I ICatagory I I

Lieutenant C COccassionally I C

Liquefy C C

A marketer wants to understand the reproducibility of his measurement process to measure the number of misspelled words in the firstdraft of a brochure. Two people called spell checkers are given a list of words and asked to identify the words that are sp elled incorrectly.Any difference between the words the two spell checkers identify as misspelled represents the reproducibility of the measurement process.

C = Spell Checker identified the word as correctly spelled

I = Spell Checker identified the word as incorrectly spelled

= Differences between Spell Checkers

Spell Checker 1

Correct Incorrect

Correct 4 2

Incorrect 1 3

Measurement System Reproducibility = [(5 + 4)/(5 + 1 + 0 + 4)] x 100 = 90%

Spell Checker 2

6

4

5 5 10

5 1

0 4



70/81

y yReproducibility (continued)Cont inuo us Data Example

A deal business is measuring the cycle time on deals. A reference set of 5 deals is collected

and measured by two people, and then one of the same people re-measures the 5 deals.The difference between people represents the reproducibility, while the differencebetween the first and second measure by the same person represents repeatability.

Source % Contribution

Total Gage R&R 1.48Repeatability 0.23Reproducibility 1.25Part-To-Part 98.52Total Variation 100.00

Using either Minitab orExcel we can calculate thepercent of the TotalVariation contributed by themeasurement process interms of reproducibility andrepeatability (see partialprintout below).

In this case we see that thereproducibility andrepeatability of themeasurement process is98.52%, which means thatnearly all of the observed

variation is coming from theprocess, not themeasurement system.

Cycle Time Deal # Measurer #5.4 1 15.4 1 15.5 2 15.5 2 13.9 3 13.9 3 14.8 4 14.7 4 1

4.7 5 14.7 5 15.3 1 25.3 1 25.5 2 25.5 2 23.8 3 23.9 3 24.7 4 24.7 4 24.5 5 24.5 5 2


4.7 5 14.7 5 15.3 1 25.3 1 25.5 2 25.5 2 23.8 3 23.9 3 24.7 4 24.7 4 24.5 5 24.5 5 2


4.7 5 14.7 5 15.3 1 25.3 1 25.5 2 25.5 2 23.8 3 23.9 3 24.7 4 24.7 4 24.5 5 24.5 5 2


4.7 5 14.7 5 15.3 1 25.3 1 25.5 2 25.5 2 23.8 3 23.9 3 24.7 4 24.7 4 24.5 5 24.5 5 2

Variation due to measurement system

Actual deal variation



71/81

Stability

Time 1

Time 2

Measurement System Analysis:Stability

The variation obtained whenthe same person measuresthe same unit with the sameequipment over an extendedperiod of time

Continuous Calculate variation change in measurement over time

Discrete Calculate change over time (% correct or consistent)



72/81

y yLinearity

Linearity is the consistency of the measurement systemacross the entire range of the measurement scale

Continuous: The endpoints of a pressure gageare typically not as accurateas the center of the gages range

Discrete: Assessing items for defects is easy in veryobvious cases, but can be very

difficult in borderline or less clear cases. Operators

may have very consistent judgement performance at thebeginning of a shift, but poor consistency before breaksor near the end of a work shift

C d ti A MSA


73/81

Step 5: Determine If The Measurement System Must Be Improved

Conducting An MSA

Examine context of business environment,process, and customerHow critical is the measurement?

What are the risks of making an error?

Review results of MSA studyTypical Gage R&R specs:

% < 30% of total processvariation


74/81

Conducting An MSA

Identify factors that could causemeasurement process variation(measurement error)

Reduce the impact of those factors

Step 6: Determine How The Measurement Process Will Be Improved

C d ti g A MSA


75/81

Conducting An MSA

Determine how often (frequency of audits)Determine what to auditProcedures/documentation up-to-date

Procedures/documentation usedQuantify MSA

Step 7: Audit The Measure Process To Ensure Accurate AndConsistent Measurements

Meas rement S stems


76/81

Measurement Systems

Measurement error is always presentin your total observed variation

Minimize the measurement processvariation

Use MSA to identify the amount ofprocess variation

Measurement error is always a biggerdeal than you think

Understand how measurement errorimpacts your customer

Summary

Make Sure Your MSA Is ExaminingThe Actual Measurement System Itself

Measurement Systems Analysis


77/81

y yBreakout (35 Minutes)

What How Who Time

AllTeamPreparation

Part 1: MSA For Your Project

Develop an MSA for your project Y data

1. Determine which MSA is appropriatefor your data

2. Determine which aspect of measurement are relevant (accuracy,repeatability, etc.)

3. Develop the plan for how you willcollect the data

4. List factors that might cause themeasurement of an item to vary andhow you would reduce the impact of those factors

OR

Choose facilitator, timekeeper, scribe

All

All

1 min

Part 2: If You DoNot Have AProject

1 min

34 min

5 min

15 min

13 min

How would you run a MSA on thefollowing conditions:

1. Collecting dates from insurance files

2. Errors on billing statements

3. The number of customerscontacted/converted by a broker

4. Call-center: categorizing call types


DevelopOperationalDefinitions And

Ensure DataConsistencyAnd Stability

Collect DataAnd MonitorConsistency


78/81

Collect Data And Monitor

Consistency Communicate the what and why to the data collectors

and process participants Train everyone who will be collecting data Pilot the data collection process and adjust as needed Confirm understanding of operational definitions Make data collection procedures error-proof Be there in the beginning to oversee data collection

Procedures

Collect Data And Monitor


79/81

Consistency

Check to make sure the measurementsystem is stable Check to make sure the measurement

proceduresremain consistent (over time, and from datacollector todata collector)

Check to see if the data look reasonable

Summary of Measure


80/81

yPerformance

2.1 Determine what to measureUnderstand the role that data plays in process improvementUnderstand the cause and effect relationships that occur inside theteam's processDetermine the indicators needed to evaluate current processperformance

2.2 Manage measurementUnderstand different types of data and how each type can provide theteam with different insights and knowledge of a processDevelop operational definitions and data collection plans that buildvalidity and consistency in the data which the team gathers

2.3 Understand variationUnderstand the concept of variation and how a process can be evaluatedby assessing its variation over timePlot and calculate the variation of the team's business processGain hands-on experience with the use of the statistical softwarepackage MINITAB



81/81

Summary of Measure Performance (contd)

2.4 Determine Sigma performanceUnderstand the various calculations associated with determiningprocess sigmaCalculate the sigma performance of the team's processUnderstand the difference between First Pass Yield and RolledThroughput Yield

2.5 Managing the measurement systemUnderstand the different uses of our measurement systemsUnderstand the language of measurementUnderstand how to conduct a measurement system analysisUnderstand how to interpret the results of a MSA study

Understand MSA in administrative processes

Documents

1.5 LSS Quality Files Bus. Risk Management (1)