NEW MODEL OF PERFORMANCE MEASUREMENT SYSTEM

Management and Industrial System Development Lab

Department of Industrial Engineering – ITS

Lack of relevance Lagging metrics Short-termism Inflexible Does not foster improvement Cost distortion

Traditional performance measurement systems produce information that are too late, too aggregate,

and too distorted to be relevant for managers planning and control decisions (Kaplan and Johnson)

TRADITIONAL PERFORMANCE MEASUREMENT SYSTEM:

PERFORMANCE MEASUREMENTS FORWORLD-CLASS MANUFACTURING (BRIAN MASKELL)

▪ Are directly related to the manufacturing strategy

▪ Primarily use non-financial measures

▪ Vary between locations

▪ Change over time as needs change

▪ Are simple and easy to use

▪ Provide fast feedback to operators and managers

▪ Are intended to foster improvement rather than just monitor performance

FINANCIAL AND NON-FINANCIAL MEASURES (BRIAN MASKELL)Financial Measure

▪ Needed for external reporting

▪ Not needed for internal reporting

▪ Irrelevant

▪ Misleading and confusing

Non Financial Measures

▪ Relevant, clear, focused (measures of flexibility, and the ability to achieve quality objectives, schedule position, and reduce inventory, for example can help a company achieve significant improvement goals).

CHARACTERISTICS OF AN EFFECTIVE MEASUREMENT SYSTEM: GENERAL INSTRUMENT CORPORATION

▪ Meaningful and performance traceable to the financial statements (everything linked to the bottom line)

▪ Clear, easy to understand

▪ Visible and shared with all employees

▪ Used to drive the continuous improvement process

▪ Performance objectives must reflect process maps, competitive analysis, and benchmark studies

▪ Heavy use of ratios and time-based metrics including velocity of processes and responsiveness

▪ The basis of incentive compensation.

PERFORMANCE CRITERIA SYSTEM(Shlomo Globerson)

• The value of P.C must be the same for the same performance

• P.C must be derived from the company’s objective

• They make it possible to compare organisations which are in the same business

• Their purpose is clear

• Data collection and calculation method are clearly defined

• Ratio P.C are preferred to absolute numbers

• P.C should be under the control of the evaluated organisational unit

• Objective P.C are preferred to subjective P.C

NEW MODELS OF PERFORMANCE MEASUREMENT SYSTEM

• Performance Measurement Questionnaire

• SMART System

• Balanced Scorecard

• Cambridge Model

• Integrated Performance Measurement System

• PRISM

PERFORMANCE MEASUREMENT QUESTIONNAIRES(Dixon, Nanni, and Vollmann)

PMQ▪ This approach uses a performance measurement

questionnaire (PMQ) for evaluating and developing the measures already in use in an organisation.

▪ This is based on the premise that measures should appraise, reinforce and reward improvements in performance.

▪ Managers need an effective process for designing measures which they can use to meet their unique and evolving needs.

PMQ STRUCTUREPART 1

▪ Asks respondents to score the importance of specific improvement areas for the business and to score how effectively the current performance measures gauge improvement

PART 2

▪ Asks respondents to score specific performance measures on the extent to which they believe that achieving excellence on the measure is important for the long term health of the company and the extent to which they believe that the company places emphasis on that measure

• management level and manufacturing affiliation of the respondents to examine the degree of consensus among managerial levels and functional areas

Questionnaire II :

Questionnaire I :

• the relative degree of importance of improvement area• to the extent the current performance measures support or inhibit the improvement in the area

Questionnaire III : • the degree of importance of achieving excellence in performance factors or measures for the long-run health of the company• the company current emphasises on performance measures

Questionnaire IV : • the most important measures against which respondents’ individual performance should be judged

GENERIC IMPROVEMENT AREAS

•QUALITY

•LABOUR EFFICIENCY

•MACHINE EFFICIENCY

•NEW PRODUCT INTRODUCTION

•VOLUME FLEXIBILITY

•PRODUCT MIX FLEXIBILITY

•PRODUCT TECHNOLOGY

•PROCESS TECHNOLOGY

•MANUFACTURING THROUGHPUT TIMES

•INTEGRATION WITH SUPPLIERS

•INTEGRATION WITH CUSTOMERS

•INFORMATION SYSTEMS

•DIRECT COST REDUCTION

•OVERHEAD COST REDUCTION

•INVENTORY MANAGEMENT

•JOB RESPONSIBILITIES

•PERFORMANCE MEASUREMENT

•CUSTOMER SATISFACTION

•ENVIRONMENTAL CONTROL

•MANUFACTURING STRATEGY

•PROCUREMENT PRACTICES

•OFFSHORE MANUFACTURING

•CIM

•EDUCATION AND TRAINING

EXAMPLE OF QUESTIONNAIRE II

How Much Long-RunImprovement Is Required? IMPROVEMENT AREAS

Do Current PerformanceMeasures Support Improvement

RELIABILITY OF PRODUCTS IN FIELD

COMPETITIVENESS OF COMPANY PRICES

ABILITY TO MEET PROMISED DELIVERY DATES

ABILITY TO SATISFY CUSTOMER REQUESTS

1 2 3 4 5 6 7

1 2 3 4 5 6 7

1 2 3 4 5 6 7 1 2 3 4 5 6 7

NONE GREAT INHIBIT SUPPORT

1 2 3 4 5 6 7

1 2 3 4 5 6 7

1 2 3 4 5 6 7 1 2 3 4 5 6 7

QUESTIONNAIRES III

Relative Importanceto the Company Performance Factors

Emphasis ofMeasurement

Inventory turnover

Conformance to specification

Cost of quality

1 2 3 4 5 6 7

1 2 3 4 5 6 7

1 2 3 4 5 6 7

VeryUnimportant

VeryImportant

NoEmphasis

MajorEmphasis

1 2 3 4 5 6 7

1 2 3 4 5 6 7

1 2 3 4 5 6 7

PERFORMANCE MEASUREMENT QUESTIONNAIRES ANALYSIS

• Alignment analysis– to identify to the extent to which a company’s strategy, actions, and measures

line up with each others– ranks the importance of the improvement areas and the emphasis on

measurement. – These rankings can be used to assess fit with strategy and fit between the

importance of improvement and the emphasis placed on measurement

IMPROVEMENT AREAS

Alignment of Improvement Areas (Actions) with Strategy

Top Quartile Bottom Quartile

New product introductionCustomer satisfactionProduct technologyQualityIntegration with customersManufacturing throughput time

Job responsibilityMachine efficiencyDirect cost reductionEnvironmental control Labour efficiencyOffshore manufacturing

• Congruence analysis – to provide a detailed look at how well the measurement system supports an organisation’s actions

and strategy through gap and false alarm signal – shows the differences between what is considered important to improve and whether the measures

support this improvement. – They also show which performance measures are important to the company and whether the

company emphasises these measures. – The term ‘gap’ is used where the business is failing to measure what matters and ‘false alarms’ where

the business is reacting to measures which do not matter.

Importance= high

Support = low Gap = Importance - Support

Importance = low

Support = high False Alarm = Importance - Support

• Consensus analysis – to contrast the perceptions between hierarchy levels and across functional

organisations about the importance of improvement area and performance measures (ranking the means)

– partitions the data between management levels and functions. – The comparison of results can lead to the identification of communication

problems when the scores do not agree

Importance

High

Low

Division Plant Middle Supervisor

Consensus across management level of Indirect Labour Productivity

• Confusion analysis – to determine the relative extent of consensus in opinions on each improvement

area and performance factor item within a group (ranking the standard deviations)

– assesses the variation in responses within groups

IMPROVEMENT AREAS

Top and Bottom Quartiles of Strategies and Ranks of Standard Deviations

Top Quartile Bottom Quartile

New product introduction (24)Customer satisfaction (19)Product technology (23)Quality (16)Integration with customers (22)Manufacturing throughput time (21)

Job responsibility (9)Machine efficiency (4)Direct cost reduction (5)Environmental control (3)Labour efficiency (1)Offshore manufacturing (2)

PERFORMANCE MEASUREMENT QUESTIONNAIRES(Dixon, Nanni, and Vollmann)

Financial Perspective

Objectives

Measures

Targets

Initiatives

1 2 3

Learning and GrowthPerspective

Objectives

Measures

Targets

Initiatives

1 2 3

Customer Perspective

Objectives

Measures

Targets

Initiatives

1 2 3

Internal Business ProcessPerspective

Objectives

Measures

Targets

Initiatives

1 2 3

Business Strategy

“To succeed financially, how should we appear to our shareholders?.”

“To satisfy our shareholders and customers, what business processes must we excel at?.”

“To achieve our vision,how will we sustain our ability to change and improve”

“To achieve our vision, how should we appear to our customers?.”

BALANCED SCORECARD(Kaplan and Norton)

Part 1Grouping Products

Part2 Agreeing businessobjectives

Part 3Agreeing performancemeasures for thebusiness objectives

Part 4Mapping performancemeasures and activities

Part 7Sign off

Part 5Evaluating options- which of these are key?

Part 6Agreeing performancemeasures for the keyactivities

Part 8Implementationand ongoing management

CAMBRIDGE MODEL(Andy Neely)

INTEGRATED PERFORMANCE MEASUREMENT SYSTEM

(Centre for Strategic Manufacturing)

The Business

Business Units

Business Processes

Activities

IPMS BUSINESS STRUCTURE

The Business

Business Units

Business Processes

Activities

Stakeholders

Objectives

Measures

Business Units

Business Unit Classification, RONA,ROI, Performance Measures Classi-fication

External Monitor

Stakeholders

External Monitor

Objectives

Measures

The Business

EFQM, Financial Models, RONA and ROI trees, etc.

Stakeholders

External Monitor

Objectives

Measures

Stakeholders

External Monitor

Objectives

Measures

Stakeholders

External Monitor

Objectives

Measures

Business Units

Business Units Classification, RONA/ROI Tress,Performance Measures Classifications

Systems Classification,Performance Measures Classification

Systems Classification,Performance Measures Classification

Business Processes

Activities

Financial

Obj

ectiv

esM

easu

res

Targ

etIn

itiati

ves

“To succeed financially, howshould weappear to ourshareholders ?”

Customer

Obj

ectiv

esM

easu

res

Targ

etIn

itiati

ves

“To achieve ourvision, howshould weappear to ourcustomer?”

Internal Business Process

Obj

ectiv

esM

easu

res

Targ

etIn

itiati

ves

“To satisfy ourshareholdersand customer,what businessprocesses mustwe excel at ?”

Learning and GrowthO

bjec

tives

Mea

sure

sTa

rget

Initi

ative

s

“To achieve ourvision, how willwe sustain ourability tochange andimprove ?”

Visionand

Strategy

Company Objectives:- Improve OTD

- Develop Employee PM

Target Setting

Company Target:- OTD = 100 %

-Employee PM Finish 2001

Objectives Deployment

Target Deployment

Department Objectives :

- Train Employee (HRM)- Develop PM (Dev. M)

Department Target :- Train Emp. Ratio = 25%

-Employee PM Finish 2001

ResourcesBargaining

DefineActivity

Activity Plan:- Develop training plan- Set up project team

Resources Requirement:

- No Additional Res. Req.- Hire 2 programmers

- Hire 1 business analyst

Last year performance(Improvement Programmes):

- On Time Delivery (OTD)

Strategic plan:2001Develop Employee PM

Business environmentchanges:

- 2000 Local players- Foreign players

Objectives Setting

Performance Planning PT. X

Performance Accomplishment PT. X

Progress Reports :- Training report

- Software dev. report

Activity Plan Execution : - Develop training plan

- Set up project tim

Activity Plan:- Develop training plan- Set up project team

Resources Deployment :- Allocate 2 programmers and 1business analyst (additional) to

System Development Department

Resources Requirement:

- No Additional Res. Req.- Hire 2 programmer

- Hire 1 business analyst

Performance Measurement PT. X

Balanced ScorecardModel

Identification of Key Performance Indicator (KPI)

Corporate, Business Units

Key Performance Indicator

Corporate, Business Unit

Performance ReportsCorporate, B. Unit, B.Process, Activity

Activity Plan- Routines

- Improvements

CompanyTarget

DepartmentTarget

Performance Measurement

Key Performance Indicator

B. Process, Activity

Identification of Key Performance Indicator (KPI)

Business Process, Activity

Business ProcessMap

Performance Review

Performance Problems

ORDER QUALIFIER AND ORDER WINNER CRITERIA

Order qualifier criteria :• Price

• Modules• Technology• Reliability

Order winner criteria :• Support

• Customization

Identification of Key Performance Indicator (KPI)Corporate, Business Units

STRATEGY OF PT. X

In the period of 1999 to 2004 PT. X provides customizedconsulting services in performance management and its

related areas. We adopt customer intimacy strategy to fully understand our customer requirement and deliver services

beyond their expectation.

Only 10 percent of effectively formulated strategies were successfully implemented

Identification of Key Performance Indicator (KPI)Corporate, Business Units

STRATEGY IMPLEMENTATIONCUSTOMER INTIMACY

FULLY UNDERSTAND CUSTOMERREQUIREMENTS

DELIVER BEYOND EXPECTATION

- MORE THAN WHAT WE PROMISED- FASTER THAN WHAT WE PROMISED

BUILT IN ORGANIZATION ANDMANAGEMENT SYSTEM

IMPLEMENTED IN DAILY OPERATIONSand MEASURES THE PERFORMANCE

Identification of Key Performance Indicator (KPI)Corporate, Business Units

STRATEGIC MAP OF PT.X

PROFIT GROWTH

ASSET GROWTH

CASH-FLOW COVERAGE

FINANCIAL

RETURNCUSTOMER

COMPLAIN RESOLVEDRATIO

REQUEST FULFILLEDRATIO

CUSTOMER

ON TIME DELIVERY NUMBER OF INNOVATION

EMPLOYEE DICIPLINE

EMPLOYEE PRODUCTIVITYGROWTH

INTERNAL BUSINESS PROCESS

TRAINED EMPLOYEERATIO

EMPLOYEETURNOVER

TAKE HOME PAYGROWTH

LEARNING AND GROWTH

FULLY UNDERSTAND CUSTOMERREQUIREMENTS

- MORE THAN WHAT WE PROMISED- FASTER THAN WHAT WE PROMISED

DELIVER BEYOND EXPECTATION

COMPANY STRATEGY

KPI Weight Target Unit Achievement Scoring System Colour

C Company Performance 94.65 C Financial 55% 100 54.20 98.55 S Profit Growth 55% 100% 98% Higher Is Better 53.90 98.00 S Asset Growth 35% 100% 115% Higher Is Better 40.25 115.00 S Cash-flow coverage 10% 75% 33% Higher Is Better 4.40 44.00 C Customer 20% 100 20.00 100.00 S Return Customer 40% 100 % 100 Higher Is Better 40.00 100.00 S Resolved compalin ratio 30% 100 % 100 Higher Is Better 30.00 100.00 S Fulfilled request ratio 30% 100 % 100 Higher Is Better 30.00 100.00 C Internal Business Process 15% 100 15.58 103.87 S On time delivery 25% 11 Month 9 Higher Is Worse 29.55 118.18 S Number of innovation 25% 2 3 Higher Is Better 30.00 120.00 S Employee diciplinary 25% 5018 hour 4999 Higher Is Better 24.91 99.62 S Employee productivity growth 25% 105 % 82 Higher Is Better 19.42 77.69 C Learning and Growth and Employee Delight 10% 100 4.86 48.63 S Trained employee ratio 35% 25% 24% Higher Is Better 32.94 94.12 S Employee turnover 20% 100 % 70.58 Higher Is Better 14.12 70.58 S Take Home Pay Growth 45% 30% 1.05% Higher Is Better 1.58 3.50

STOP

Score

Performance Measurement PT. X

Strategy

B

S

C

Translate theStrategy to OperationalTerms

• Strategy maps• Balanced Scorecards

Mobilize change throughExecutive leadership

• Mobilization• Governance process• Strategic management system

Make strategy a Continual process

• Link budget and strategy• Analytics and IS• Strategic learning

Align the organizationto the strategy

• Corporate role• Business unit synergies• Share services synergies

Make strategyEveryone’s everydayJob

• Strategic awareness• Personal scorecards• Balanced paycheck

EXAMPLE

North America Marketing and Refining :

• product leadership strategy : brand image and product characteristic

• competitor pursued similar strategy

• NAMR lose competitive advantage

NAMR adopted business strategy

• Revenue Growth Strategy

• Productivity Strategy

FINANCIAL PERSPECTIVE

INCREASE ROCE TO 12 %

Revenue Growth Strategy Productivity Strategy• ROCE• Net Margin vs Industrty

New Sources of Non-gasoline

Revenue

Increase Customer Profitability through

Premium Barnds

Become Industry Cost Leader

Maximize use of Existing Assets

• Non-gasoline revenue and margin

• Volume vs industry• Premium Ratio

• Cash Expense (Cents per gallon) vsIndustry

• Cash Flow

CUSTOMER PERSPECTIVES

ROAD WARRIORS16 %

Generally higher-income, middle-aged men who drive 25.000 to 50.000 miles a year….. Buy premium gasoline with credit card…Purchase sandwiches and drinks from convenience store……will Sometimes wash their cars at the carwash

TRUE BLUES16 %

Usually men and women with moderate to high incomes who Are loyal to a brand and sometimes to a particular station…Frequently buy premium gasoline and pay in cash

GENERATION F327 %

Fuel, Food and Fast : Upwordly mobile men and women – half under25 years of age – who are constantly on the go…..Drive a lot and snack heavily from the convenience store.

HOMEBODIES21 %

Usually housewives who shuttle their children around during the day And use whatever gasoline station is based in town or along their route or travel

PRICE SHOPPERS20 %

Generally aren’t loyal to either a brabd or a particular station, and Rarely buy the premium lone …… frequently on the tight budgets.

SERVICE PROFILE

• Immediate access to a gasoline pump • Self-payment mechanism at the pump• Covered area for gasoline pumps• 100 % availability of product, especially premium grades• Clean restrooms• Satisfactory exterior station appearance• Safe, well-lit station• Convenience store, stocked with fresh, high-quality merchandises• Speedy purchase• Ample parking spaces near convenience store• Friendly employees• Availability of minor services

Speedy Purchase

Friendly,Helpful

employee

Recognize loyalty

More Consumer

product

Help developBusiness

skill

Basic• Clean• Sale• Quality• Trusted brand

Differentiators

Delight the customerWin-win Dealer Relationship

• Mystery shopper rating

• Share of segment

CUSTOMER PERSPECTIVES

INTERNAL BUSINESS PROCESS PERSPECTIVES

Two important internal processes :

1. Develop new products and services2. Generate dealer profits from non-gasoline revenues

Create Non-gasolineProduct and

services

• New Product ROI• New product Acceptance Rate

Understand CustomerSegment

• Share of Target segment

Best-In-Class Franchise

Team

• Dealer quality rating

Improve Hardware

performance

• Yield gap• Un-planned downtime

On SpecOn Time

• Perfect Orders

Improve Inventory

management

• Inventory level• Run-out rate

Industry CostLeader

• Activity cost

vsCompetitors

ImproveSHE

• Environmental Incident• Safety Incident

Build the franchiseIncrease customer

value Achieve operational excellence Be a good neighbor

• Functional excellence• Leadership skill• Integrated view

• Personal scorecard• Employee feedback

• Process improvement• Y2K

LEARNING AND GROWTH PERSPECTIVES

• Aligned• Personal growth

A motivated and prepared workforce

Climate for action Competencies Technology

• Strategic skill coverage ratio

• System milestone

NAMR BALANCED SCORECARD

PERSPECTIVE STRATEGIC THEMES STRATEGIC OBJECTIVES STRATEGIC MEASURES

FINANCIAL Financial Growth F1 Return On Capital EmployedF2 Asset utilizationF3 Profitability]F4 Industry cost leaderF5 Profitable growth

• ROCE• Cash flow• Net margin rank• Fuel cost per gallon• Volume growth rate• Premium ratio• Non-gasoline revenue and margin

CUSTOMER Delight the customer

Win-win dealer relations

C1 Continuously delight targeted customer

C2 Build win-win relations with dealer

• Market share in targeted market• Mystery shopper rating

• Dealer gross profit growth• Dealer survey

NAMR BALANCED SCORECARD

PERSPECTIVE STRATEGIC THEMES STRATEGIC OBJECTIVES STRATEGIC MEASURES

INTERNALBUSINESSPROCESS

Build the franchise

Safe and reliable

Competitive supplier

Quality

Good neighbor

I1 Innovative products and services

I2 Best-in-Class Franchise teams

I3 Refinery performance

I4 Inventory management

I5 Industry cost leader

I6 On spec, on time

I7 Improve SHE

• New product ROI• New product acceptance rate

• Dealer quality score

• Yield gap• Unplanned down time

• Inventory level• Runout rate

• Activity cost vs competitor

• Perfect orders

• Number of environmental incidents• Days away from work rate

LEARNINGANDGROWTH

Motivated and preparedworkforce

L1 Climate for actionL2 Core competencies and skillsL3 Access to strategic Inf.

• Employee survey• Personal BSC• Strategic Competency av.• Strategic Inf. availability

Performance Review PT. X

- Overall performance is good- Take home pay growth and Cash flow coverage need improvement

- Set up improvement programmes

Performance Improvement PT. X

Performance Problems :- Take home pay growth

- Cash flow coverage

Identification of Causal Factors :- Take home pay growth is poor because Cash flow

coverage is poor- Cash flow coverage is poor because of uncertainty of

payment

Setting Improvement Programmes :- Improve customer mix ( large company Vs small company )

Improve customer mix :- Target ( 1 large company 2

small company)- Schedule (2002)

- Resources ( No additional resources )

- Responsibility (Partnership Manager)

EXERCISE

1. Define your strategy

2. Develop the strategy map

3. Identify the KPIs(Use Appendix !)

PERFORMANCE PRISM FRAMEWORK (1)

PERFORMANCE PRISM FRAMEWORK (2)

PERFORMANCE PRISM FRAMEWORK (3)