Why Projects Fail - ASQ Milwaukee Section · PDF fileMacro view –Why Projects Fail ... presented a paper titled “What’s Normal & Why Care”. It STARTED with a strongly worded

© 2017 Minitab, Inc.11/24/2017

Why Projects Fail

ASQ MilwaukeeMay 15, 2017

Paul Sheehy, Technical Training SpecialistJeff Harding, Regional Sales ManagerDennis Corbin, Technical Training Specialist

© 2017 Minitab, Inc.21/24/2017

Agenda

Macro view – Why Projects FailMinitab’s ResponseMicro view – Why Data Analysis FailsCorrecting Common MistakesQuestions

© 2017 Minitab, Inc.31/24/2017

Only 30% of improvementinitiatives succeed.

© 2017 Minitab, Inc.41/24/2017

Lack of Management Support

Project Risk Poorly Assessed

Project Scope Poorly Defined

No Data or Bad Data

© 2017 Minitab, Inc.51/24/2017

Variation in Project Execution

Projects are not Aligned with

Corporate Initiatives

Project Benefits are Hard to

Communicate

Stakeholders have Poor Visibility

© 2017 Minitab, Inc.61/24/2017

Minitab’s Response

Minitab Product Evolution

© 2017 Minitab, Inc.71/24/2017

Introducing Companion by Minitab

Cloud based project management solution focused on continuous improvement methodologies

© 2017 Minitab, Inc.81/24/2017

© 2017 Minitab, Inc.91/24/2017

Introducing Companion by Minitab

Desktop AppProject execution

Web AppCloud‐based dashboard

Standardized Roadmaps

Integrated Tools

Data sharing

Program Impact

Project Visibility

Configurable

© 2017 Minitab, Inc.101/24/2017

© 2017 Minitab, Inc.111/24/2017

• Project Charters• Financial Tracking• Fishbone• FMEA• Process maps• Value Stream maps• Monte Carlo Simulation

• C&E Matrix100+ Forms and Tools

© 2017 Minitab, Inc.121/24/2017

© 2017 Minitab, Inc.131/24/2017

© 2013 Minitab, Inc.

Running With ScissorsASQ – MilwaukeeMay 2017

1 © Minitab Inc., 2017



The best laid schemes o’ mice an’ men, gang oft agley

Project Management

A Macro Problem

(Jeff’s discussion)


Quality of Analysis

Or

A Micro Problem



RUNNING WITH SCISSORS



What if we do everything right

• Get executive buy in.• Select candidates to lead improvement projects. • Put then through extensive crash course in Six Sigma or other

statistically rigorous improvement methodology.• Select good Champions.• Select good projects.

►AND THEN…



The analyses are done by…

► Process improvement leaders (belts) who may have• Received relatively little training time on proper analytical

processes.• Scarce access to qualified statistical help.• Used statistical software (analyze data) infrequently. • Received most of their guidance from others equally trained

(blind leading the blind).• Etc.

► In other words… Running with scissors



Let me count the ways• What’s Normal & Why Care?• Subgroups• Measurement and Not• Capability• Stray Thoughts



Some comments

►I am sure that today’s audience is aware of most of these issues.

►The purpose of this talk is to remind us what kind of thinking and actions may be going on in the trenches.

►These examples have for the most part been culled from real life. (Dragnet)


Normality(and equal variance)



Normality – the monster under the bed

► Early on, Six Sigma presented the following:

► Many took the normality assumption too seriously.

Hypothesis test for means – two independent samples



Example

► A major industrial• In 1997 the Six Sigma Statistician for the company wrote and

presented a paper titled “What’s Normal & Why Care”. It STARTED with a strongly worded conclusion that for t and F tests, only “severely non-normal” data would have a significant effect on the test outcome.

• Many GBs and BBs were stopping their projects claiming their data was not normal.



Two-Sample t-test, Simulations Using Non-Normal DataRandom data from a Chi-Square distribution with 5 df20,000 simulated tests run for n = 5, 8, 10, 15, 20, 25, 30, 40, 50 (both samples)

50403020100

1.00

0.95

0.90

0.85

0.80

X-Data

Y-D

ata

0.95

20100-10

99.9

99

90

50

10

1

0.1

Acceptance Rate vs Sample Size for 2-Sample t Test

Test Population Distribution vs Normal Distribution Normal Probability Plot of Test Population DistributionNormal - 95% CI

SampleSize

AcceptRate(standard)

AcceptRate(Welch's)

0.95230.95010.95290.95600.95090.94740.95010.95290.9487

0.94690.94840.95040.95690.95280.94890.95230.95620.9520

5 810152025304050

Accuracy of p-value:

Standard Test assumes equal variances

Welch’s Test does not assume equal variances

Both tests perform very well, even for samples as small as 5

At n = 5, accept rate is just above 95% for Standard, just below 95% for Welch’s (theoretical rate for both tests is 95%)

Welch’s Test

Standard

Bottom Line:The two-sample t-test is EXTREMELY robust to this type of non-normal data.


From Minitab’s Assistant Menu




Transformations & NonparametricsOne way to safely handle severely non-normal data is to use transforms.

• How well is this taught to GBs, etc.?• If a user only needs this once a year…?

A second alternative is the use of nonparametric tests such as the Wilcoxon or Mann-Whitney (for a one or two sample test of medians).

• These tests also have assumptions (in this case “symmetric data” and “equal shape”).

• Have users been taught these assumptions?



Two actual illustrationsSeveral years ago a Black Belt told me that his Master Black Belt said not to use Xbar-R charts because his data was non-normal.

During a VOC data gathering initiative by Minitab, a Black Belt told us that his MBB would say “Get that out of here, your data is not normal. I don’t even want to look at it and we can’t use it.”



What about Equal Variance(2-sample t)►If you assume equal variances (standard test) the results are quite sensitive to the interaction of sample size and unequal variances.

►If you assume unequal variances (Welch’s test), the results are robust even for huge differences in variation and sample size.


Subgroups,Control Charts, andCapability



Would you believe…

Minitab’s Tech Support has seen the following relative to capability analysis:

• Sort the data by the output value to decrease the within subgroup variation and thus improve the Cpk.

• Users tried different subgroup sizes to see what number gave them a good Cpk.

• Presented capability with NO validation of stability.• Use of Capability Analysis-Normal when data is not normal.• Asked how to “back-calculate” spec limits for a particular z.usl

or z.lsl value. Etc.



Subgroups and Control Charts

Tech support had a call• “ I have an I-MR chart with a lot of points out of control.

I know why these particular points are so extreme, but what can I do to make my chart look better.”

Another issue• Improper selection of subgroups leads to overly

generous (and incorrect) UCL and LCL values on an X-Bar chart. Assume a four cavity mold for soap. We could choose as a rational subgroup the four bars made each time the press cycled and we would get…



UCL=109.19

LCL=96.60

Because the die was improperly set up, additional (assignable) variation was introduced into the subgroup



With correct subgroupingCavity 1 is shown out of control (note the different UCL and LCL). n = 4

UCL=105.41

LCL=94.20


Measurement



Basic Failure Modes for both Gage R&R and Attribute Agreement Analysis

• The measurement is poorly defined (e.g. diameter of dowel).

• A “ringer” is thrown in as one of the appraisers.• Test not done in “actual” conditions (time, noise, light,

etc.)• Conversion of continuous data into attribute data.• Use of continuous Gage R&R to analyze attribute data.• When Gage R&R or Attribute Agreement Analysis do

not “fit”, the analyst claims (incorrectly) that MSA is not needed.



A warning re: Gage R&R

Gage R&R is essentially a random effects ANOVA and as such the parts sampled (usually 10) should be selected randomly. HOWEVER, the AIAG standards that use the ratio of 0 to10% for ideal and 10 to 30% for marginal are based on using the process width (defined as 6σ) as the denominator. If we use the standard deviation obtained from our sample without validating it against some historical or benchmark value we could be seriously off.



Let us set up an example

We have a process that is known to the gods on Mt. Olympus to have a standard deviation of 10 and a mean of 300.

• Sample 1: We tell the supervisor to get us 10 random parts and she pulls them over the course of two days that have similar environmental conditions resulting in a sample with a small process σ of 5.56.

• Sample 2: We need a sample and the process is not currently running so we go to the engineer’s wall of shame and randomly grab 10 parts (note these are extreme highs and lows – all parts more than ±3σ).

• Sample 3: We grab a truly random sample of 10 parts.



Some Data

• 6σ of the measurement system actual but unknown is 12.78

• Sample 1: 6σ of the process is 6 x 5.56 = 33.36

• Sample 2: 6σ of the process is 6 x 35.00 = 210.00

• Sample 3: We will look at the distribution of standard deviations of 1,000 samples of n = 10

6σ MS

6σ ProcessTotal Variation



Raw Data

If you look at all 1,000 sample standard deviations:

The max of 17.56 is 496% of the min of 3.54



Sigma Distribution (n = 10)



Results of Gage/TV

Pulling a representative sample is vital


CapabilitySome interesting issues

• Not validating stability (and even the measurement system) prior to calculating capability. (Small sample size issue).

• Cpk vs. Ppk.• Possible lack of understanding of the defect rate given only a

Cpk or Ppk.• We have seen capability analysis continuous used with

dichotomous data (essentially 0/1).• No gut check before running the computer command.



Some Stray Thoughts



(Un)common sense examples

• In an attribute inspection process, a Black Belt claimed 10 opportunities per part (denominator), but discarded the part without further inspection when any single defect was found.

• With large data sets the AD normality test can prove the slightest departure from normality.



Additional interesting items

• Confusion between paired t and 2-sample t.• Use of a t test to prove equivalence (can’t prove the null).• Chi-Sq vs. a series of proportion tests.• Individual error rate vs. family error rate when doing a series of

tests.• Too much data (prove differences that are not of practical

significance).• Lose time sequence of data.• Convert variable data to discrete data.• Unchecked use of historical data.


Minitab Available Resources:

1. Customized webinars – Feel free to have people reach out to me at my contact information below. Whether they are looking for help on basic training for Minitab or a more advanced topic like DOE I can help pull in the appropriate resources to help get them up to speed on Minitab related topics through a webinar or even onsite visit

2. Techinical and Statistical Support Line – 814-231-2682

a. Most people just think to call our support line for installation and technical questions. Through this line we also have an entire team of Statistical experts ready to help you answer questions about how to do certain analyses.

3. Free Public Webinars - http://www.minitab.com/en-us/support/webinars/

a. This is an ever changing schedule taught by one of our statisticians or trainers on various topics. They are open for anyone to attend are also recorded for review later

4. Support Page

a. This has video tutorials, example data sets, our blog, and documentation to help you learn Minitab or get your questions answered.

Free case study: www.minitab.com/crayola

http://www.minitab.com/en-us/support/webinars/

http://www.minitab.com/crayola

Documents

Why Projects Fail - ASQ Milwaukee Section · PDF fileMacro view –Why Projects Fail ... presented a paper titled “What’s Normal & Why Care”. It STARTED with a strongly worded