This article was downloaded by: [Nipissing University]On: 07 October 2014, At: 21:46Publisher: Taylor & FrancisInforma Ltd Registered in England and Wales Registered Number: 1072954Registered office: Mortimer House, 37-41 Mortimer Street, London W1T3JH, UK

Journal of Applied StatisticsPublication details, including instructions forauthors and subscription information:http://www.tandfonline.com/loi/cjas20

SPC as a tool for creatinga successful businessmeasurement frameworkS.Y. Coleman a , G. Arunakumar b , F. Foldvary c &R. Feltham da University of Newcastle upon Tyneb Advantica Technologies Ltdc Transcod Formerly Advantica Technologies LtdPublished online: 02 Aug 2010.

To cite this article: S.Y. Coleman , G. Arunakumar , F. Foldvary & R. Feltham (2001)SPC as a tool for creating a successful business measurement framework, Journal ofApplied Statistics, 28:3-4, 325-334, DOI: 10.1080/02664760120034063

To link to this article: http://dx.doi.org/10.1080/02664760120034063

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of allthe information (the “Content”) contained in the publications on ourplatform. However, Taylor & Francis, our agents, and our licensorsmake no representations or warranties whatsoever as to the accuracy,completeness, or suitability for any purpose of the Content. Any opinionsand views expressed in this publication are the opinions and views ofthe authors, and are not the views of or endorsed by Taylor & Francis.The accuracy of the Content should not be relied upon and should beindependently verified with primary sources of information. Taylor andFrancis shall not be liable for any losses, actions, claims, proceedings,demands, costs, expenses, damages, and other liabilities whatsoeveror howsoever caused arising directly or indirectly in connection with, inrelation to or arising out of the use of the Content.

This article may be used for research, teaching, and private studypurposes. Any substantial or systematic reproduction, redistribution,reselling, loan, sub-licensing, systematic supply, or distribution in any formto anyone is expressly forbidden. Terms & Conditions of access and use canbe found at http://www.tandfonline.com/page/terms-and-conditions

Dow

nloa

ded

by [

Nip

issi

ng U

nive

rsity

] at

21:

46 0

7 O

ctob

er 2

014

Journal of Applied Statistics, Vol. 28, Nos. 3&4, 2001, 325± 334

SPC as a tool for creating a successfulbusiness measurement framework

S. Y. COLEMAN1, G. ARUNAKUMAR2, F. FOLDVARY 3 &

R. FELTHAM4, 1Industrial Statistics Research Unit, University of Newcastle uponTyne, 2Advantica Technologies Ltd, 3Transco, 4Formerly Advantica Technologies Ltd

abstract Many companies are trying to get to the bottom of what their main objectivesare and what their business should be doing. The new Six Sigma approach concentrateson clarifying business strategy and making sure that everything relates to companyobjectives. It is vital to clarify each part of the business in such a way that everyone canunderstand the causes of variation that can lead to improvements in processes andperformance. This paper describes a situation where the full implementation of SPCmethodology has made possible a visual and widely appreciated summary of the perfor-mance of one important aspect of the business. The major part of the work was identifyingthe core objectives and deciding how to encapsulate each of them in one or more suitablemeasurements. The next step was to review the practicalities of obtaining the measurementsand their reliability and representativeness. Finally, the measurements were presented inchart form and the more traditional steps of SPC analysis were commenced. Data fromfast changing business environments are prone to many diþ erent problems, such as theshort previous span of typical data, strange distributions and other uncertainties. Issuessurrounding these and the eventual extraction of a meaningful set of information will bediscussed in the paper. The measurement framework has proved very useful and, from aninitial circulation of a handful of people, it now forms an important part of an informationprocess that provides responsible managers with valuable control information. The measure-ment framework is kept fresh and vital by constant review and modi® cations. Improvedelectronic data collection and dissemination of the report has proved very important.

1 Introduction

Many companies are trying to get to the bottom of what their main objectives areand what their business should be doing. The new Six Sigma approach concentrates

Correspondence: S. Y. Coleman, Industrial Statistics Research Unit, University of Newcastle uponTyne, UK.

The paper reports on a project initiated by R. Feltham, developed by F. Foldvary and implementedby G. Arunakumar with statistical advice from S. Y. Coleman.

ISSN 0266-4763 print; 1360-0532 online/01/030325-10 � 2001 Taylor & Francis Ltd

DOI: 10.1080/02664760120034063

Dow

nloa

ded

by [

Nip

issi

ng U

nive

rsity

] at

21:

46 0

7 O

ctob

er 2

014

326 S. Y. Coleman et al.

on clarifying business strategy and making sure that everything relates to companyobjectives. It is vital to investigate each part of the business in such a way thateveryone can understand the causes of variation that can lead to improvements inprocesses and performance.

Statistical Process Control (SPC) is a scienti® c approach to quality improvementin which data are collected and used as evidence of the performance of a process,organization or set of equipment. Relevant data are presented in charts and thenatural variability of the measurements and any special features can readily beseen. SPC works best in a team environment and the charts provide a focus fordiscussion. Where appropriate, control limits are added to the charts to show theexpected range of future measurements. This has the advantage of showing whenperformance is typical and when there are changes. By investigating the changes,special causes of variation in performance can often be eliminated or exploitedleading to quality improvement.

SPC is used throughout manufacturing industry to monitor production andprocesses. SPC in a wider sense incorporates looking at data for an insight intowhat causes variation and uncertainty. SPC has been successfully applied inmanagement (Wheeler, 1993) even though the data and circumstances diþ erconsiderably from those in production.

This paper describes the challenge of developing a measurement framework forthe safety aspect of a business. The issues involved are typical of the application ofSPC to management information. The safety measurement framework wasdeveloped to provide an objective representation of safety performance for activitiesthat impact on public and system safety.

Using SPC with safety measurements leads to continual improvement because

· attention is directed to areas that need it· eþ ort is not wasted

The aim of the SPC safety measurement framework is to

· summarize the situation in each of the core activities· use data to provide a means for evaluating performance· avoid wasting time investigating minor ¯ uctuations in performance· leave more time for investigating signi® cant problems

The SPC methodology is described in the next section, which is followed by theconstruction of the SPC charts. The data encountered are often non-standard andsparse. These and other features of the data are considered in Section 4. Becauseof the nature of the application, control limits that err on the side of caution arefavoured whenever the data lead to uncertainty, as discussed in Section 5. Theway the data are presented and reviewed is considered in Sections 6 and 7.

2 Methodology

The major part of the work is in identifying the core objectives and deciding howto encapsulate each of them in one or more suitable measures. The approach todetermining the measures deploys process thinking and looking at the business interms of inputs, outputs and processes. The next step is to tackle the practicalitiesof obtaining suitable measurements and ensuring their reliability and representa-tiveness. This can take some time, extensive organization and communication andis facilitated by a personal approach and hard work.

Dow

nloa

ded

by [

Nip

issi

ng U

nive

rsity

] at

21:

46 0

7 O

ctob

er 2

014

Creating a successful business measurement framework 327

AugJunApr00FebDecOctAugJunApr

99FebDecOct

98Sep

200

100

0

Y ear

X=103.4

2.0SL=165

3.0SL=196

-2.0SL=41

-3.0SL=10

Fig. 1. Example of an SPC chart.

Past data are analysed to identify natural variation and variation due to specialcauses.

Special causes tend to

· Have a large eþ ect and be long-lasting· Be identi® able and sometimes possible to rectify

Natural variation tends to

· Be diý cult to identify· Be transient with small, short-lasting eþ ects.

Past data are used to predict a range of expected values. The data are modelled bya standard statistical distribution where possible. The range of expected values isrepresented by control limits and these are presented in a control chart with currentdata, as shown in Fig. 1.

In SPC terminology, the control limits are based on natural variation so thatfuture values that are typical will fall within the limits and special cause variationwill cause points to fall outside the control limits and alert to possible problems.In practice, past data contain a lot of unexplained variation and SPC is a dynamicprocess following Deming’s (1986) PDCA (plan, do, check, act) cycle like otherquality improvement ventures.

Use of a control chart leads to process improvement because

· The control chart provides a focus for discussion· Attention is focused on exceptionally good or poor performance· Performance in line with past data is noted and attempts to improve are made

once alerts to possible problems have been dealt with· It provides a means of quantifying safety performance levels and monitoring

progress to target performance level· Implementation of improvement measures can be readily assessed and

monitored· It illustrates delivery of business requirements to demonstrate system integrity

To help decide how best to present the measures in SPC charts, a ¯ owchart wasdeveloped. In addition, adherence to the ¯ owchart is used as a tool for ensuring

Dow

nloa

ded

by [

Nip

issi

ng U

nive

rsity

] at

21:

46 0

7 O

ctob

er 2

014

328 S. Y. Coleman et al.

the approach is in line with standard statistical practices and to acknowledgerobustness in methodology. A condensed form of the ¯ owchart is given inAppendix 1.

Most of the checks in the ¯ owchart are aimed at learning more about the dataand helping to understand the measurements. Some of the checks, however are toprevent non-standard data giving misleading results.

3 SPC charts

In the safety measurement framework, charts are constructed for each measureshowing the changing values and the historic mean. Warning limits containing 95%(cf. 2 Sigma Limits) of the data are added and action limits contain 99.7% (cf. 3Sigma Limits) of the data. Control limits are omitted from the chart if there arenot enough data for a reasonable prediction to be made.

Once the control limits have been constructed they are left in place unless thereis evidence that a signi® cant change has taken place. It is therefore possible toautomate the data collection and updating of the charts.

Some of the data are available monthly, some quarterly and some yearly. Aquarterly report is produced. Each of about 30 measures has a page with a briefde® nition of the measure, description of the historical data, control chart andcomment on current performance highlighted using traý c light symbology.

4 Special features

Management-type data have the following special features compared with data inmanufacturing applications:

· Rather than aiming for a target, smaller values are usually wanted· Data occur singly rather than in groups· Most of the data are in the form of counts or rates· There is often insuý cient data to investigate seasonal eþ ects· Data is often rather infrequent· There is often a short start-up span of data· The underlying population of the measurements is often changing· At the start of the SPC investigation there is a lot of unexplained variation· The data often have a non-standard distribution

Statistical aspects of these points are discussed in Appendix 2.

5 Conservative approach to control limits

The nature of the safety measurement framework means that it is more importantto detect real changes than to avoid false alarms. For this reason

· Warning limits act as action limits· Extensive checks are made on the data before constructing the control limits

Statistical aspects of these points are discussed in Appendix 3.

Dow

nloa

ded

by [

Nip

issi

ng U

nive

rsity

] at

21:

46 0

7 O

ctob

er 2

014

Creating a successful business measurement framework 329

31302928272625242322212019181716151413121110987654321

4

3

2

1

0

-1

-2

-3

-4

-5

-6

Safety Measures

Sco

res

Safety Measurements for Quarter 2 2000

Fig. 2. Bar chart summary of safety measures.

6 Summary of measures

A summary of all the measures is included in the report to give an overview orsnapshot of the performance in that quarter. Each measure is translated into traý clight symbology, based on the position of the value compared with the controllimits. As most measures are better the smaller they are, purple and blue are usedfor extra small values and red and amber are reserved for extra large values asfollows:

greenÐ performance is consistent with historical data and need not beinvestigated

amberÐ value exceeds the upper warning limitredÐ value exceeds upper action limitpurpleÐ value is below lower warning limitblueÐ value is below lower action limit

To enable all the measures to be summarized in one diagram, the current valueshave been standardized into Q values (Quesenberry, 1991) and plotted in a barchart.

For Normal1 data, the historical mean points are subtracted from the latestpoints and divided by the standard deviation used in the control chart. For Poissondata2, the Q value is the equivalent Normal variable with equal probability (seeAppendix 4). The measures are presented in a logical order rather than in ascendingor descending Q values. An example is given in Fig. 2.

As the report is quarterly and some measures are monthly, guidelines for whichpoint to plot in the summary chart have been devised (see Appendix 4).

7 Review and interpretation

Several measures may relate to one concept as diþ erent ways of looking at thesame thing. If the framework is to be relied upon then it has to be recognized that

Dow

nloa

ded

by [

Nip

issi

ng U

nive

rsity

] at

21:

46 0

7 O

ctob

er 2

014

330 S. Y. Coleman et al.

serious decisions may be made depending on it. The framework is periodicallyreviewed to cover the following aspects:

· the design of the framework: are the measures appropriate and complete?· the robustness of the data: are the measures a reliable re¯ ection of reality?· the quality and rigour of the data analysis· the application of the framework: does it contribute to eþ ective decision-making?

The measurement framework is kept fresh and vital by constant review andmodi® cations.

The readership of the report is monitored carefully. Each report contains a tear-oþ page for comments to be returned so that the readership and reception can beassessed.

It is important to have endorsement by senior management. There have been anumber of brie® ng sessions and these have helped to encourage speedier volun-teering of the data. Improved electronic data collection and dissemination of thereport has proved very important.

Ownership of the charts has remained in the hands of the person who requeststhe data and constructs the control charts. The charts are then presented back tothe people concerned with the particular processes or activities. This keeps themeasurement framework rather distant and is a problem associated with the diversesources of the data involved.

8 Conclusions

The safety measurement framework has been a focal point for information collec-tion and exchange. It has forced consistent data collection and attention to datareliability. The various charts have made the trends visible. The charts have alertedattention to possible problems. SPC is a suitable and valuable medium to publicisethe current situation in each activity, give guidelines for expected performance andlead to continual improvement.

The measurement framework report now forms an important part of an informa-tion process that provides responsible managers with valuable control information.

Notes

1. The standard distribution for variables with a bell-shaped histogram.2. The standard distribution for counts where events occur randomly at a constant rate.

REFERENCES

Bissell, D. (1991) Estimating variation from data with varying element sizes, Journal of Applied

Statistics, 18, pp. 287± 295.Bissell, D. (1994) Statistical Methods for SPC and TQM (Chapman and Hall).Bowman, K. O. & Shenton, L. R. (1975) `Omnibus test contours for departures from normality based

on Ï b1 and b2’ , Biometrica, 62(2), p. 243± 250.Deming, W. E. (1986) Out of the Crisis (Cambridge University Press).Hillier, F. S. (1969) X and R chart control limits based on a small number of subgroups, Journal of

Quality Technology, 1(1), pp. 17 ± 25.Metcalfe, A. V. (2000) Statistics in Management Science (Arnold).Montgomery, D. C. (1991) Statistical Process Control (Wiley)Quesenberry, C. P. (1991) SPC Q Chart for a Poisson parameter short or long runs, Journal of

Information Technology, 23(4), pp. 296 ± 303.Shewhart, W. A. (1931) Economic Control of Quality of Manufactured Product (USA, Reinhold)Wetherill, G. B. & Brown, D. (1991) Statistical Process Control (Chapman and Hall).Wheeler, D. (1993) Understanding VariationÐ The key to managing chaos (SPC Press).

Dow

nloa

ded

by [

Nip

issi

ng U

nive

rsity

] at

21:

46 0

7 O

ctob

er 2

014

Creating a successful business measurement framework 331

Appendix 1

Dow

nloa

ded

by [

Nip

issi

ng U

nive

rsity

] at

21:

46 0

7 O

ctob

er 2

014

332 S. Y. Coleman et al.

Appendix 2. Special features

Unlike manufacturing-based SPC, which aims to focus on processes rather thanthe end product, the measurement framework encompasses the full range ofactivities and includes input measures, process measures and output measures.Most of the measures are such that smaller values (i.e. lower rate or number ofoccurrences) are preferable, rather than a speci® c target. The data are usually one-at-a-time measurements rather than in rational groups, and this makes the chartsless sensitive to change and more sensitive to non-standard distributions.

Most of the data are in the form of counts or rates. When the average rate ofoccurrence is high (over 30) these data behave like variables and tend to have asymmetrical distribution leading to control limits that are equally spaced about themean. When the average rate of occurrence is low, as often happens with safetydata, the natural distribution is asymmetric, leading to upper control limits furtherfrom the mean than lower control limits (see Fig. 3).

The lower the average rate of occurrence the more asymmetric the control limits.It is not sensible, however, to combine data over a larger time period to obtain ahigher rate of occurrence as it is preferable to use the data as soon as possiblerather than waiting for two or three results.

Sometimes there are not enough data to establish whether there are seasonaltrends. In this case a run chart is useful but a control chart cannot be constructed.

The frequency of the data is often low (quarterly or yearly) as complex manage-ment data that amalgamates information from a number of sources tends tobecome available periodically and often well after the event. In the safety measure-ment framework most of the data are seasonal and have traditionally been gatheredon a yearly basis. This often leads to a short start up span of data. Even if dataexist for 20 or so years in the past, the conditions in a fast moving industry willhave changed markedly over that time. Hillier (1969) and Quesenberry (1991)describe how such a short span of start-up data can still be used to set up a controlchart. The upshot is that, for variables that can be assumed to follow the Normaldistribution, the correct control limits are wider than when there are suý cientstart-up data. For count data that can be assumed to follow the Poisson distribution,the eþ ect can be for the limits to be wider or closer. In both cases, control limits

J F M A M J J A S O N D J F M A M J J A S O N D J

1998 1999 2000

0

1

2

3

4

5

6

7

8

9 UCL=9

UWL=7

Mean=2.4

Month/year

Year

asse

t fai

lure

Control chart for monthly asset failur

e

Fig. 3. SPC chart for low count data.

Dow

nloa

ded

by [

Nip

issi

ng U

nive

rsity

] at

21:

46 0

7 O

ctob

er 2

014

Creating a successful business measurement framework 333

need to be calculated each time a new point is added. The limits gradually convergeto those obtained assuming suý cient start-up data as more data are added, andthe diþ erence is small after only 10 or so points.

With changing business environments, the pool from which the counts, rates ormeasurements are collected may be changing markedly. For example, certain typesof storage are being phased out so that a count of failures applies to fewer andfewer sets of equipment. The dispersion test for Poisson data assumes that thesample size is fairly constant, the rule of thumb quoted in Wetherill and Brown(1991) being for all sample sizes to be within 25% of the mean sample size. Whenthis is not the case, a test described in Bissell (1991) is used.

At the start of an SPC exercise, even the larger causes of variation are notknown. The data therefore tend to wander around more than if just naturalvariation is present. If this eþ ect is large then it shows up in a test for additionalvariation (Bissell, 1994) and can alter the way the control limits are calculated.

Management data often do not follow standard distributions and there are notenough data to model in other ways or to look for trends or seasonal patterns.Often the data are approximately Normal and can be treated as such. Shewhart’s(1931) original description of control charts does not specify that data should beNormal, but that control limits at 3 standard deviations either side of the meanare intuitively reasonable. If there are no obvious features then this gives a good® rst approach. There is one exception however, that if the data are platykurtic(that is less peaked than the bell-shaped Normal curve) then 3 standard deviationsmay contain all of the data rather than most of it, so that points can never exceedthe control limits. As this could be dangerous in a safety context, a check is madeon the kurtosis of the data (Bowman and Shenton, 1975) and only if they aresuitable are the data used for the control chart.

Appendix 3. Conservative approach to control limits

SPC control limits usually contain 99.7% of the data. This ensures a low falsealarm rate where a point gives an action signal but there is no underlying changeto the process (see for example, Montgomery, 1991). Such limits mean that it maytake a long time before a real change is detected. For example, if the data areapproximately Normal with a standard deviation of 1.0, then a change of 1.0 inthe overall level of the data may take 26 points before it shows up as an actionsignal. This is 2 years’ worth of monthly data or 6 years’ worth of quarterly data.Using warning limits that contain 95% of the data as action limits in the controlchart gives a higher false alarm rate but speeds up the detection of real changes.Note that larger changes, e.g. 2.0 units, will be detected much more quickly.

The correct placing of the control limits assumes that there is a good model forthe data. If there is uncertainty, it is better to err on the side of safety and take aconservative approach making the control limits closer together.The ¯ ow chart includes checks for these uncertainties to try to avoid unsafeconclusions being made from the charts.

The safety measurement framework presents 31 diþ erent aspects of the safetyperformance of the company. There are opportunities for multivariate analysis ofthese measures to look for patterns and associations between them. The data donot all occur with the same frequency, however, but the analysis is certainly worthtrying and may produce some very interesting relationships.

Dow

nloa

ded

by [

Nip

issi

ng U

nive

rsity

] at

21:

46 0

7 O

ctob

er 2

014

334 S. Y. Coleman et al.

Type of uncertainty Eþ ect on lower limit Eþ ect on upper limit

Standard count data assumed Normal Further from mean but Closer to meanusually adjusted to zero

Over-dispersed count data assumed Poisson Closer to mean Closer to meanRate data falling sample size assumed constant Uncertain Uncertain

when testing for PoisonWandering mean assumed constant Closer to mean Closer to meanEþ ect of short start-up span of data for Closer to mean Closer to mean

Normal ignoredEþ ect of short span of start-up data for Uncertain Uncertain

Poisson ignoredNon-standard data with ¯ at (platykurtic) Further from mean Further from mean

distribution assumed standardNon-standard data with peaked distribution Closer to mean Closer to mean

assumed standard

Appendix 4. Q scores for the summary

For approximately Normal data, the Q score is (value± mean)/standard deviation.Points outside the action limits will have scores greater than 3 or less than 2 3.Scores outside of 5 and 2 5 are unlikely.

For Poisson data, the Q score is obtained by ® nding the cumulative probability ofobtaining a point as extreme as that observed and converting it to the correspondingstandard Normal score. For example, if there is one incident where the mean rateof occurrence is 0.65, then the cumulative probability of 0 or 1 incidents is(0.522 + 0.339) 5 0.861 using the Poisson probability formula (see for example,Metcalfe, 2000). Using statistical tables for standard Normal cumulative probabili-ties, 0.861 corresponds to a Normal score of 0.7, which is therefore the Q score.This de® nition gives rise to some peculiarities, for example if the mean rate ofoccurrence is 2.42, the warning limit is 7 and a point on the warning limit willhave a Q score of 2.7, which is over half way between the warning and action limitson the summary chart. If the mean rate of occurrence is less than 0.05, then evenzero will have a Q score greater than + 2 and give an action signal.

Guidelines for selecting which point should be featured in the summary chartare as follows: plot the last point in the quarter unless one of the other points inthe quarter has gone outside of the control limits. If points are outside of bothaction limits, choose the most recent point outside the upper action limit. If pointsare outside warning and action limits, choose the most recent point outside theaction limit. If points are outside of both warning limits, choose the most recentpoint outside the upper warning limit. If points are outside a warning limit, choosethe most recent point.

The summary chart is eþ ectively illustrating 31 simultaneous tests of statisticalsigni® cance. If there is an action signal whenever a warning limit is breached, thenthe probability of a false alarm is 0.05. However, overall, the probability of a falsealarm occurring anywhere in the framework is much greater than 0.05. It isimportant to be aware of this, but in line with the requirement for erring on theside of caution it does not impair the usefulness of the framework.

Dow

nloa

ded

by [

Nip

issi

ng U

nive

rsity

] at

21:

46 0

7 O

ctob

er 2

014