The PERC Methodology

Improving OperationsUsing Data Mining Techniques with a

Common Sense Approach to Doing Business

The PERC Methodology

Performance Evaluation Report Card (PERC) Methodology

One of the ten finalists in the 2010 WIPRO/Knowledge@WhartonInnovation Tournament

The PERC MethodologyTypical Application Scenario

• Business or Government Entity with Several Similar Operating Units– General Motors, Ford -- Automotive Dealerships– Health and Human Services – Medicare, Medicaid, etc.– McDonald’s -- Hamburger Franchises – FDIC – Commercial and Savings Banks – Walmart, Target, Retail Stores – Multiple Outlets– Department of Education -- Schools, Guaranty Agencies– Etc. Etc. Etc. . . . Etc. Etc. Etc.

• Large Volume of Data Resources That:– Seem Unmanageable or Incomprehensible– Unwieldy, Making Easy Access to Important Information Difficult– Typically Require Non-Business IT Types to Access– Result in Lost Opportunities From Information Mismanagement

The PERC MethodologyTypical Application Scenario (continued)

• Opportunities Exist to Improve Operations with Better Information Management and Technology– Improve Productivity

– Move All Units Towards Best Practices

– Gain Competitive Advantage

– Increase Shareholder/Stakeholder Value

– Reduce Costs

– Reduce Risk or Potential Fraudulent Activities

– Increase Awareness of Critical Success Factors Throughout Organization

– Increase Customer/Client Satisfaction

The PERC MethodologyThrough Improved Data Management

Quantitative AnalysisStatistical Analysis

Expert Systems

Data Source 1 Data Source 2 Data Source 3 Data Source 4

FraudDetection

ExpertSystems

ExecutiveInformation

Systems

ManagementSummary Reports

ForecastingModels

The PERC MethodologyDrive Toward Best Practices

Time

Per

form

an

ce R

ati

ng

s

Worst

Best

Identifying performance differences drives business

towards best performances. As the poor performers

improve, so does the average or normal business improve.

The PERC MethodologyThrough Quantitative Business Analysis

Risk ManagementCost Reduction

Growth OpportunityBudget Solution

Policy DevelopmentProfit Increase

Operational Efficiency

NormsVariances

RegressionsCorrelationSampling

AlgorithmsNPV

FinanceOperationsBudgetingMarketingAccounting

Planning

TrendExpert

ModelingGraphics

RatiosDemographics

Quantitative

Business

AnalysisStrategic

ManagementInformation

The PERC MethodologyThrough Improved Data Management

Quantitative AnalysisStatistical Analysis

Expert Systems

Data Source 1 Data Source 2 Data Source 3 Data Source 4

FraudDetection

ExpertSystems

ExecutiveInformation

Systems

ManagementSummary Reports

ForecastingModels

Changes inPolicy & Operation

Improving OperationsUsing the PERC Methodology

Approach

The PERC MethodologyThe Eight-Step Approach

• Step 1: Establish a Strong Business Oriented Focus Group or Executive Dictum

• Step 2: Clearly Define Business Objectives at the Start

– Simplify and Clarify

• Step 3: Identify and Evaluate Available Data Sources for Solving Objectives

– Determine Data Quality -- Comprehensiveness, History, Value & Integrity

– Depending on Findings -- Stop or Move Forward -- If Forward,

• Step 4: Perform Rapid Quantitative Business Analyses

– Identify Relevant Indicators, Establish Benchmarks, Trend History

– KISS -- Keep It Simple Stupid

– Add Complexity Only As Necessary -- Probably Better in Feedback Cycle

• Step 5: Present Findings and Concept for “Smart” System Products

• Step 6: Design & Develop “Smart” System Products for Accomplishing Objectives

• Step 7: Implement, Evaluate, and Obtain Operational Feedback

• Step 8: Refine or Revise and Update Appropriately

The PERC MethodologyThe Eight-Step Approach

• Step 1: Business Oriented Focus Group Phase

• Step 2: Business Objective Phase

• Step 3: Data Evaluation Phase

• Step 4: Rapid Quantitative Business Analysis Phase

• Step 5: “Smart” System Concept Phase

• Step 6: Design & Development Phase

• Step 7: Implementation Phase

• Step 8: Refine & Update Phase

The PERC Philosophyto Improving Operations

KEEP IT SIMPLE STUPID

ADD

COMPLEXITY ONLY AS NEEDED

KISS + COAN

If what is done cannot be understood

then the value, itself, is questionable

The PERC MethodologyStep 1: Establish Business Oriented Focus Group

• Organize a Team of Individuals With:

– A Business Oriented Focus

– Relevant Systems Knowledge

– Operational Knowledge and Experience

• Include Both Line and Staff Level Personnel

– Decision Making Authority

– Enthusiasm for Rapid Development Projects

• Alternatively– CEO Directive

– Executive Dictum

The PERC MethodologyStep 2: Business Objective Phase

• Clarify the Fundamental Business Objectives – Improve Productivity?

– Reduce Costs?

– Increase Sales & Revenue?

– Increase Shareholder/Stakeholder Value?

– Improve Customer Satisfaction?

– Investigate Fraudulent Activities?

• Keep the Objectives Simple -- But Not Too Narrow (e.g.,– Correct Payments or Cost Savings -- Why not Both?

– Default Risk or Improved Operations -- Why not Both?

– Improved and Timely Management Information

• Identify Data Resources for Benchmarking Objectives– Obtain Access for Analytical Purposes

The PERC MethodologyStep 3: Data Evaluation Phase

• Evaluate Available Data Sources for Solving Objectives– Determine Data Quality -- Comprehensiveness, History, Value & Integrity– Depending on Findings -- Stop or Move Forward

• Discuss -- How Do We Go About Doing This?– Simply, Logically and Quickly– Entire Population, Sample Population, or Pilot Group– Frequency Distributions of Coded and Date Fields– Identify Potential Outlyers -- Negatives, Min’s, Max’s, Variances, Missing Data– Obtain Sums on Every Numeric Field -- Begin Establishing Controls or Numbers to

Validate -- Eliminate Outlyers If Appropriate– Slice & Dice the Key Numeric Fields by Codes and by Month/Year, Year, Etc.– Create New Variables to Help in Validation Process -- Algebra is o’k– Putting It All Together -- Does It Make Sense? Are Key Variables Available?

Can Benchmarks Be Established for Performance Measurements?

The PERC MethodologyStep 3: Data Evaluation Phase (Continued)

• Data Quality Evaluation -- What Did You Learn? -- Presentation of Findings

• Graphically Present Findings -- • By Program Meaningful Codes • By Time Intervals (e.g., Month/Year, Year, Week, etc.)• By Logical Peer Groups (size, product type, etc.)• While Seeking New & Creative Perspectives for Evaluation

– Provide Supplemental Detailed Outputs Supporting Findings

• Decision Point -- Stop or Move Forward to Analytical Phase– Provide Recommendation– Decision Should Be Clear

The PERC MethodologyStep 4: Rapid Quantitative Business Analysis Phase

• Perform Rapid Quantitative Business Analyses

– Determine if Conceptual Control Groups Exist

• Establish Best Practice Groups -- Poor Performing Groups

• If Unable, Then Establish Common Sense Business Practices as Initial Measuring Tools (e.g., net income, sales/employee)

– Begin Developing Set of Potential Performance Indicators

– Segregate Data by Peer Groups and/or Other Logical Criteria

– Evaluate Effectiveness of Individual Indicators using Statistical Correlations by Peer Group

• Determine Norms, Variances and Outlyers for Performance Indicators

• Determine Indicators’ Ability to Be Applied Comprehensively

• Consider Amount of Bias in Indicator (May Require Feedback)

The PERC MethodologyStep 4: Rapid Quantitative Business Analysis Phase (Continued)

• Perform Rapid Quantitative Business Analyses (Continued)

– Seek Monitoring Solution with Multiple Indicators

– Mix and Match Indicators with Different Weighting Schemes to Look for Best Monitoring Mechanism

– Consider Different Rating Evaluation Methodologies• A, B, C, D, F

• 1st Quartile, 2nd Quartile, . . .

• Relationship to an Expected Norm for the Population

• Simple Ranking per Peer Group (e.g., 21st out of 212)

• Ten Point / Hundred Point Scale

– Begin Formulating Concept For “Smart” System Design

The PERC MethodologyStep 5: “Smart” System Concept Phase

• Determine How This New Knowledge Should Be Implemented– Alternative Media Options

• Database -- LAN, Web-based Intranet, Software

• Report Card -- Hardcopy

– Periodicity of Rating Evaluation

– Forcefully or Experimentally

– Peer Group or Entire Population

– Relative Measurement or Absolute Measurement

• Develop Alternative “Smart” System Concepts, Providing– Pros and Cons

– Recommendation

The PERC MethodologyStep 6: Design & Development Phase

• Design Output Formats Appropriate For Chosen Media

– Group Information Logically

– Provide User-Friendly Textual and Numeric Displays

• Good, Average, Poor --- High, Medium, Low --- A, B, C

• Color Code Numerics -- Percentiles, Standard Deviations -- Factor Above/Below Norm

– Present Rating Mechanism or Evaluation Clearly

– Provide Information Regarding Peer Group Benchmark Norms

– Show Trend -- Emphasizing Improvements– Seek Self-Improvement Through Fairness

The PERC MethodologyStep 6: Design & Development Phase (Continued)

• Develop Design

– Program Design Layouts

– Program Rating or Monitoring Mechanism

– Test Program Logic

– Document Program Logic and Production Update Process

– Develop User Documentation / Operating Manuals, Explaining• Business Objectives

• Performance Measurements

• Rating Mechanisms or Methodology

• All Relevant Output Variables

• Feedback Mechanism for Improvement Considerations

The PERC MethodologyStep 7: Implementation Phase

• Implement System– Install System Pilot or Full Comprehensive Implementation

– Provide User Training• Explain Objectives

• Key Operating Features

• Stress Importance of User Feedback

– Provide Users with Operating Manuals and Contacts for Feedback

– Over Time -- Monitor Benchmarks for Improving Operations

– Focus Group -- Seek Feedback for Improving System

– Evaluate Usefulness and Value of Making System Enhancements

The PERC MethodologyStep 8: Revise and Update Phase

• Update System

– Update System Based upon Logical Processing Periodicty

– Ensure Update Receives Quality Control

– Identify Major Changes between Previous Period and New Update

– Keep Trend Information

• Revise System– If Feedback Points to Cost-Effective Improvements--Revise

– Can Help Ensure User Acceptance

– Can Reduce Earlier Unforeseen Biases

– Revisions Should Be Somewhat Minor and Quick to Implement

– Remember, Add Complexity Only as Necessary -- KISS

Improving OperationsUsing the PERC Methodology

Analyze Performance

Standards

Analyze Performance

Standards

Analyze Performance

Standards

Drive Towards Best Practices

Focus on Poor Performers

Evaluate Success and Refine Approach

Improving Operations thru PERCDrive Toward Best Practices

Time

Per

form

an

ce R

ati

ng

s

Worst

Best

Placing pressure on poor performers drives them

towards best practices. As the poor performers improve, so does the industry average.

The PERC MethodologyTypical Project – “Fast Track” Implementation

• Focus Group Phase -- (0 weeks) --Done Prior to Project Start

• Business Objective Phase -- (1-2 Days)

• Data Evaluation Phase -- (2-4 weeks) -- Starts Upon Receipt of Data

• Rapid Analysis Phase -- (3-4 weeks) -- Can Start During Previous

• “Smart” Concept Design Phase -- (2-3 weeks)

• Development Phase -- (3-6 weeks) -- Depends on Design Complexity

• Implementation Phase -- (2-4 weeks)

• Refine and Update -- (1-2 weeks) -- Updates Dependent on Periodicity

Typical Project: Fast Track Implementation Requires 3-6 Months