Detection of Fraud

2008 FRAUD FROUMMAY 14-16, 2008 / HYATT REGENCY COCONUT POINT RESORT & SPA / BONITA SPRINGS, FL, USA

Detection ofDetection of FraudFraudPatterns UsingPatterns Using

May 16, 2008

Digital AnalysisDigital Analysis

Slide 2

Session objectives1. Understand why and how2. Understand statistical basis for

quantifying differences3. Identify ten general tools and

techniques4. Understand use of Excel5. How pattern detection fits in

Slide 3

Overview• Fraud patterns detectable with

digital analysis• Basis for digital analysis

approach• Usage examples• Continuous monitoring• Business analytics• Using Excel

Slide 4

The Why and How

• Three brief examples• IIA Guidance Paper• Auditors “Top 10”• Process Overview• Who, What, Why, When &

Where

Objective 1

Slide 5

Example 1Wake County Transportation Fraud

• Supplier Kickback – School Bus parts

• $5 million• Jail sentences• Period of years

Objective 1a

Slide 6

Too little too late

• Understaffed internal audit• Software not used• Data on multiple platforms• Transaction volumes large

Objective 1a

Slide 7

Preventable• Need structured, objective

approach• Let the data “talk to you”• Need efficient and

effective approach

Objective 1a

Slide 8

Cost of six types of AIDS drugs

Total Cost of AIDS Drugs

0

50

100

150

200

NDC1 NDC2 NDC3 NDC4 NDC5 NDC6

Drug Type

Dol

lar

Am

ount NDC1

NDC2

NDC3

NDC4

NDC5

NDC6

Example 2 Objective 1a

Slide 9

Typical Prescription Patterns

AIDS Drugs Prescription Patterns

0.0

10.0

20.0

30.0

40.0

50.0

60.0

Prov 1 Prov 2 Prov 3 Prov 4 Prov 5 Prov 6

Prescriber

Dol

lar

Val

ue

NDC1

NDC2

NDC3

NDC4

NDC5

NDC6


Slide 10

Prescriptions by Dr. X

Dr. X compared with Total Population

050

100150200250

300350

NDC1 NDC2 NDC3 NDC4 NDC5 NDC6

Drug Type

Dol

lar

Am

ount

Population

Dr. X


Slide 11

Off-label use• Serostim

– Treat wasting syndrome, side effect of AIDS, OR

– Used by body builders for recreational purposes

– One physician prescribed $11.5 million worth (12% of the entire state)


Slide 12

Revenue trends

Overall Revenue Trend

0.9

0.95

1

1.05

1.1

1.15

1.2

2001 2002 2003

Calendar Year

Ann

ual B

illin

gs

Overall

Linear (Overall)


Slide 13

Dental Billings

Rapid Increase in Revenues

0

1

2

3

4

5

2001 2002 2003

Calendar Year

Ann

ual B

illin

gs

($m

illio

ns) Billings A

Billings B

Linear (Billings A)


Slide 14

Guidance Paper• A proposed implementation

approach• “Managing the Business Risk of

Fraud: A Practical Guide” http://tinyurl.com/3ldfza

• Five Principles• Fraud Detection• Coordinated Investigation

Approach

Objective 1b

Slide 15

Managing the Business Risk of Fraud: A Practical Guide

• Exposure draft of IIA, AICPA and ACFE

• Exposure draft issued 11/2007

• Section 5 – Fraud Detection

Objective 1b

Slide 16

Section 5 – Fraud Detection

• Detective Controls• Process Controls• Anonymous Reporting• Internal Auditing• Proactive Fraud Detection

Objective 1b

Slide 17

Proactive Fraud Detection

• Data Analysis to identify:– Anomalies– Trends– Risk indicators

Objective 1b

Slide 18

Specific Examples Cited

• Journal entries – suspicious transactions

• Identification of relationships• Benford’s Law• Continuous monitoring

Objective 1b

Slide 19

Data Analysis enhances ability to detect fraud

• Identify hidden relationships• Identify suspicious transactions• Assess effectiveness of internal

controls• Monitor fraud threats• Analyze millions of transactions

Objective 1b

Slide 20

Peeling the Onion

Population as Whole

Possible Error Conditions

Fraud Items

Objective 1c

Slide 21

Fraud Pattern Detection

Market Basket

Stratification

Trend Line

Holiday

Day of Week

Duplicates

Univariate

Gaps

Benford’s Law

Round Numbers

Target Group

Objective 1d

Slide 22

Digital Analysis (5W)

• Who• What• Why• Where• When

Objective 1e

Slide 23

Who Uses Digital Analysis

• Traditionally, IT specialists• With appropriate tools,

audit generalists (CAATs)• Growing trend of business

analytics• Essential component of

continuous monitoring

Objective 1e

Slide 24

What - Digital Analysis• Using software to:

– Classify– Quantify– Compare

• Both numeric and non-numeric data

Objective 1e

Slide 25

How - Assessing fraud risk • Basis is quantification• Software can do the “leg work”• Statistical measures of

difference– Chi square– Kolmogorov-Smirnov– D-statistic

• Specific approaches

Objective 1e

Slide 26

Why - Advantages• Automated process• Handle large data populations• Objective, quantifiable metrics• Can be part of continuous monitoring• Can produce useful business analytics• 100% testing is possible • Quantify risk• Repeatable process

Objective 1e

Slide 27

Why - Disadvantages

• Costly (time and software costs)

• Learning curve• Requires specialized

knowledge

Objective 1e

Slide 28

When to Use Digital Analysis

• Traditional – intermittent (one off)

• Trend is to use it as often as possible

• Continuous monitoring• Scheduled processing

Objective 1e

Slide 29

Where Is It Applicable?

• Any organization with data in digital format, and especially if:– Volumes are large– Data structures are complex– Potential for fraud exists

Objective 1e

Slide 30

Objective 1 Summarized

• Three brief examples• IIA Guidance Paper• “Top 10” Metrics• Process Overview• Who, What, Why, When & Where

Objective 1

Slide 31

Objective 1 - Summarized1. Understand why and how 2. Understand statistical basis for



Next is the basis …

Slide 32

Basis for Pattern Detection

• Analytical review• Isolate the

“significant few” • Detection of errors• Quantified approach

Objective 2

Slide 33

Understanding the Basis

• Quantified Approach• Population vs. Groups• Measuring the Difference• Stat 101 – Counts, Totals,

Chi Square and K-S• The metrics used

Objective 2

Slide 34

Quantified Approach

• Based on measureable differences

• Population vs. Group• “Shotgun” technique

Objective 2a

Slide 35

Detection of Fraud Characteristics

• Something is different than expected

Objective 2a

Slide 36

Fraud patterns• Common theme –

“something is different”• Groups• Group pattern is different

than overall population

Objective 2b

Slide 37

Measurement Basis

•Transaction counts

•Transaction amounts

Objective 2c

Slide 38

A few words about statistics• Detailed knowledge of

statistics not necessary• Software packages do the

“number-crunching”• Statistics used only to

highlight potential errors/frauds

• Not used for quantification

Objective 2d

Slide 39

How is digital analysis done?• Comparison of group with

population as a whole• Can be based on either counts or

amounts• Difference is measured• Groups can then be ranked using

a selected measure• High difference = possible

error/fraud

Objective 2d

Slide 40

Histograms• Attributes tallied and categorized

into “bins”• Counts or sums of amounts

Objective 2d

Slide 41

Two histograms obtained• Population and group

Population

0

100

200

300

400

500

600

700

Jan-07

Feb-07

Mar-07

Apr-07

May-07

Jun-07

Jul-07

Aug-07

Sep-07

Oct-07

Nov-07

Dec-07

Group

01020304050607080

Jan-07

Feb-07

Mar-07

Apr-07

May-07

Jun-07

Jul-07

Aug-07

Sep-07

Oct-07

Nov-07

Dec-07

Objective 2d

Slide 42

Compute Cumulative Amount for each

Count by Month

0

10

20

30

40

50

60

70

80

Month

Cou

ntCum Pct

0.0%

20.0%

40.0%

60.0%

80.0%

100.0%

120.0%

Jan-

07

Mar

-07

May

-07

Jul-0

7

Sep-0

7

Nov-0

7

Objective 2d

Slide 43

Are the histograms different?

• Two statistical measures of difference

• Chi Squared (counts)• K-S (distribution)• Both yield a difference

metric

Objective 2d

Slide 44

Chi Squared

• Classic test on data in a table

• Answers the question – are the rows/columns different

• Some limitations on when it can be applied

Objective 2d

Slide 45

Chi Squared

• Table of Counts• Degrees of Freedom• Chi Squared Value• P-statistic• Computationally intensive

Objective 2d

Slide 46

Kolmogorov-Smirnov

• Two Russian mathematicians

• Comparison of distributions• Metric is the “d-statistic”

Objective 2d

Slide 47

How is K-S test done?• Four step process

1. For each cluster element determine percentage

2. Then calculate cumulative percentage

3. Compare the differences in cumulative percentages

4. Identify the largest difference

Objective 2d

Slide 48

Kolmogorov-SmirnovObjective 2d - KS

Slide 49

Classification by metrics• Stratification• Day of week• Happens on holiday• Round numbers• Variability• Benford’s Law• Trend lines• Relationships (market basket)• Gaps• Duplicates

Objective 2e

Slide 50

Auditor’s “Top 10” Metrics1. Outliers / Variability2. Stratification3. Day of Week4. Round Numbers5. Made Up Numbers6. Market basket7. Trends8. Gaps9. Duplicates10. Dates

Objective e

Slide 51

Understanding the Basis

• Quantified Approach• Population vs. Groups• Measuring the Difference• Stat 101 – Counts, Totals, Chi

Square and K-S• The metrics used

Objective 2

Slide 52



techniques4. Understand examples done using

Excel5. How pattern detection fits in

Next are the metrics …

Slide 53

The “Top 10” Metrics

• Overview• Explain Each Metric• Examples of what it can detect• How to assess results

Objective 3

Slide 54

Trapping anomaliesObjective 3

Slide 55

Fraud Pattern Detection

Market Basket

Stratification

Trend Line

Holiday

Day of Week

Duplicates

Univariate

Gaps

Benford’s Law

Round Numbers

Target Group

Objective 3

Slide 56

Outliers / VariabilityOutliers are amounts which are significantly different from the rest of the population

1 - Outliers

Slide 57

Outliers / Variability

• Charting (visual)• Software to analyze “z-

scores”• Top and Bottom 10, 20 etc.• High and low variability

(coefficient of variation)

1 - Outliers

Slide 58

Drill down to the group level

• Basic statistics– Minimum, maximum

and average– Variability

• Sort by statistic of interest– Variability (coefficient of

variation)– Maximum, etc.

1 - Outliers

Slide 59

Example ResultsProvider N Coeff Var

3478421 3,243 342.23

2356721 4,536 87.23

3546789 3,421 23.25

5463122 2,311 18.54

Two providers (3478421 and 2356721) had significantly more variability in the amounts of their claims than all the rest.

1 - Outliers

Slide 60

Next Metric1. Outliers2. Stratification3. Day of Week4. Round Numbers5. Made Up Numbers6. Market basket7. Trends8. Gaps9. Duplicates10. Dates

Slide 61

Unusual stratification patterns

Do you know how your data

looks?

2 - Stratification

Slide 62

Stratification - How

• Charting (visual)• Chi Squared• Kolmogorov-Smirnov• By groups

2 - Stratification

Slide 63

Purpose / types of errors• Transactions out of the

ordinary• “Up-coding” insurance

claims• “Skewed” groupings• Based on either count or

amount

2 – Stratification

Slide 64

The process?1. Stratify the entire population into

“bins” specified by auditor2. Same stratification on each group

(e.g. vendor)3. Compare the group tested to the

population4. Obtain measure of difference for

each group5. Sort descending on difference

measure


Slide 65

Units of Service Stratified - Example Results

Two providers (2735211 and 4562134) are shown to be much different from the overall population (as measured by Chi Square).

Provider N Chi Sq D-stat

2735211 6,011 7,453 0.8453

4562134 8,913 5,234 0.7453

4321089 3,410 342 0.5231

4237869 2,503 298 0.4632


Slide 66


Slide 67

Day of Week

• Activity on weekdays• Activity on weekends• Peak activity mid to late

week

3 – Day of Week

Slide 68

Purpose / Type of Errors• Identify unusually

high/low activity on one or more days of week

• Dentist who only handled Medicaid on Tuesday

• Office is empty on Friday

3 – Day of Week

Slide 69

How it is done?• Programmatically check entire

population• Obtain counts and sums by day of

week (1-7)• Prepare histogram• For each group do the same

procedure• Compare the two histograms• Sort descending by metric (chi

square/d-stat)

Slide 70

Day of Week - Example Results

Provider 2735211 only provided service for Medicaid on Tuesdays. Provider 4562134 was closed on Thursdays and Fridays.

Provider N Chi Sq D-stat

2735211 5,404 12,435 0.9802

4562134 5,182 7,746 0.8472

4321089 5,162 87 0.321

4237869 7,905 56 0.2189

3 – Day of Week

Slide 71


Slide 72

Round Numbers

It’s about….

Estimates!

4 – Round Numbers

Slide 73

Purpose / Type of Errors• Isolate estimates• Highlight account numbers

in journal entries with round numbers

• Split purchases (“under the radar”)

• Which groups have the most estimates

4 – Round Numbers

Slide 74

Round numbers• Classify population amounts

– $1,375.23 is not round– $5,000 is a round number – type 3 (3

zeros)– $10,200 is a round number type 2 (2

zeros)• Quantify expected vs. actual (d-

statistic)• Generally represents an estimate• Journal entries

4 – Round Numbers

Slide 75

Round Numbers in Journal Entries - Example Results

Two accounts, 2735211 and 4562134 have significantly more round number postings than any other posting account in the journal entries.

Account N Chi Sq D-stat

2735211 4,136 54,637 0.9802

4562134 833 35,324 0.97023

4321089 8,318 768 0.321

4237869 9,549 546 0.2189

4 – Round Numbers

Slide 76


Slide 77

Made up Numbers

Curb stoning

Imaginary numbers Benford’s Law

5 – Made up numbers

Slide 78

What can be detected• Made up numbers

– e.g. falsified inventory counts, tax return schedules

5 – Made Up Numbers

Slide 79

Benford’s Law using Excel

• Basic formula is “=log(1+(1/N))”• Workbook with formulae available

at http://tinyurl.com/4vmcfs

• Obtain leading digits using “Left” function, e.g. left(Cell,1)


Slide 80

Made up numbers

• Benford’s Law• Check Chi Square and d-statistic• First 1,2,3 digits• Last 1,2 digits• Second digit• Sources for more info


Slide 81

How is it done?• Decide type of test – (first 1-3 digits,

last 1-2 digit etc)• For each group, count number of

observations for each digit pattern• Prepare histogram• Based on total count, compute

expected values• For the group, compute Chi Square

and d-stat• Sort descending by metric (chi

square/d-stat)


Slide 82

Invoice Amounts tested with Benford’s law - Example Results

During tests of invoices by store, two stores, 324 and 563 have significantly more differences than any other store as measured by Benford’s Law.

Store Hi Digit Chi Sq D-stat

324 79 5,234 0.9802

563 89 4,735 0.97023

432 23 476 0.321

217 74 312 0.2189


Slide 83


Slide 84

Market Basket

• Medical “Ping ponging”• Pattern associations• Apriori program• References at end of slides• Apriori – Latin a (from) priori (former)• Deduction from the known

6 – Market Basket

Slide 85

Purpose / Type of Errors

• Unexpected patterns and associations

• Based on “market basket” concept

• Unusual combinations of diagnosis code on medical insurance claim

6 – Market basket

Slide 86

Market Basket

• JE Accounts• JE Approvals• Credit card fraud in Japan –

taxi and ATM

6 – Market basket

Slide 87

How is it done?

• First, identify groups, e.g. all medical providers for a patient

• Next, for each provider, assign a unique integer value

• Create a text file containing the values

• Run “apriori” analysis

6 – Market basket

Slide 88

Apriori outputs

• For each unique value, probability of other values

• If you see Dr. Jones, you will also see Dr. Smith (80% probability)

• If you see a JE to account ABC, there will also an entry to account XYZ (30%)

6 – Market basket

Slide 89


Slide 90

Trend BustersDoes the pattern make sense?

ACME Technology

05,000

10,00015,00020,00025,00030,000

Date

Am

ount Sales

Employee Count

7 - Trends

Slide 91

Trend Busters

• Linear regression• Sales are up, but cost of

goods sold is down• “Spikes”

7 – Trends

Slide 92

Purpose / Type of Errors

• Identify trend lines, slopes, etc.

• Correlate trends• Identify anomalies• Key punch errors where

amount is order of magnitude

7 – Trends

Slide 93

Linear Regression•Test relationships (e.g.

invoice amount and sales tax)

•Perform multi-variable analysis

7 – Trends

Slide 94

How is it done?• Estimate linear trends

using “best fit”• Measure variability

(standard errors)• Measure slope• Sort descending by slope,

variability, etc.

7 – Trends

Slide 95

Trend Lines by Account - Example Results

Generally the trend is gently sloping up, but two accounts (43870 and 54630) are different.

Account N Slope Std Err

32451 18 1.230 0.87

43517 17 1.070 4.3

32451 27 1.023 0.85

43517 32 1.010 0.36

43870 23 0.340 2.36

54630 56 -0.560 1.89

7 – Trends

Slide 96


Slide 97

Numeric Sequence Gaps

What’s there is interesting, what’s not there is critical …

8 - Gaps

Slide 98

Purpose / Type of Errors• Missing documents (sales,

cash, etc.)• Inventory losses (missing

receiving reports)• Items that “walked off”

8 – Gaps

Slide 99

How is it done?• Check any sequence of

numbers supposed to be complete, e.g.

• Cash receipts• Sales slips• Purchase orders

8 – Gaps

Slide 100

Gaps Using Excel

• Excel – sort and check• Excel formula• Sequential numbers and

dates

8 – Gaps

Slide 101

Gap Testing - Example Results

Four check numbers are missing.

Start End Missing

10789 10791 1

12523 12526 2

17546 17548 1

8 – Gaps

Slide 102


Slide 103

Duplicates

Why is there more than one?

Same, Same, Same, and

Same, Same, Different

9 - Duplicates

Slide 104

Two types of (related) tests• Same items – same vendor,

same invoice number, same invoice date, same amount

• Different items – same employee name, same city, different social security number

9 – Duplicates

Slide 105

Duplicate Payments

• High payback area

•“Fuzzy” logic• Overriding

software controls

9 - Duplicates

Slide 106

Fuzzy matching with software

• Levenshtein distance• Soundex• “Like” clause in SQL• Regular expression

testing in SQL• Vendor/employee

situations

Russian physicist

9 - Duplicates

Slide 107

How is it done?

• First, sort file in sequence for testing

• Compare items in consecutive rows

• Extract exceptions for follow-up

9 - Duplicates

Slide 108

Possible Duplicates - Example Results

Five invoices may be duplicates.

VendorInvoice

DateInvoice Amount Count

10245 6/15/2007 3,544.78 4

10245 8/31/2007 2,010.37 2

17546 2/12/2007 1,500.00 2

9 - Duplicates

Slide 109


Slide 110

Date Checking

If we’re closed, why is there …

Adjusting journal entry

Receiving report

10 - Dates

Slide 111

Holiday Date Testing• Red Flag indicator

10 – Dates

Slide 112

Date Testing challenges• Difficult to

determine• Floating holidays

– Friday, Saturday, Sunday, Monday

10 – Dates

Slide 113

Typical audit areas• Journal entries• Employee expense

reports• Business telephone

calls• Invoices• Receiving reports• Purchase orders

10 – Dates

Slide 114

Determination of Dates

• Transactions when business is closed

• Federal Office of Budget Management

• An excellent fraud indicator in some cases

10 – Dates

Slide 115

Holiday Date Testing

• Identifying holiday dates:–Error prone–Tedious

• U.S. only

10 – Dates

Slide 116

Federal Holidays

• Established by Law• Ten dates• Specific date (unless

weekend), OR• Floating holiday

10 – Dates

Slide 117

Federal Holiday Schedule• Office of Personnel Management• Example of specific date –

Independence Day, July 4th (unless weekend)

• Example of floating date – Martin Luther King’s birthday (3rd Monday in January)

• Floating – Thanksgiving – 4th Thursday in November

10 – Dates

Slide 118

How it is done?• Programmatically count

holidays for entire population• For each group, count holidays• Compare the two histograms

(group and population)• Sort descending by metric (chi

square/d-stat)

10 – Dates

Slide 119

Holiday Counts - Example Results

Two employees (10245 and 32325) were “off the chart” in terms of expense amounts incurred on a Federal Holiday.

Employee Number N Chi Sq D-stat

10245 37 5,234 0.9802

32325 23 4,735 0.97023

17546 18 476 0.321

24135 34 312 0.2189

10 – Dates

Slide 120

The “Top 10” Metrics• Overview• Explain Each Metric• Examples of what it can

detect• How to assess results

Objective 3

Slide 121

Objective 3 - Summarized

1. Understand why and how 2. Understand statistical basis for quantifying

differences3. Identify ten general tools and techniques4. Understand examples done using Excel5. How pattern detection fits in

Next – using Excel …

Slide 122

Use of Excel• Built-in functions• Add-ins• Macros• Database access

Objective 4

Slide 123

Excel templates• Variety of tests

– Round numbers– Benford’s Law– Outliers– Etc.

Objective 4

Slide 124

Excel – Univariate statistics• Work with Ranges• =sum, =average, =stdevp• =largest(Range,1),

=smallest(Range,1)• =min, =max, =count• Tools | Data Analysis |

Descriptive Statistics

Objective 4

Slide 125

Excel Histograms

• Tools | Data Analysis | Histogram

• Bin Range• Data Range

Objective 4

Slide 126

Excel Gaps testing• Sort by sequential

value• =if(thiscell-lastcell <>

1,thiscell-lastcell,0)• Copy/paste special• Sort

Objective 4

Slide 127

Detecting duplicates with Excel

• Sort by sort values• =if testing• =if(=and(thiscell=lastcell,

etc.))

Objective 4

Slide 128

Performing audit tests with macros

• Repeatable process• Audit standardization• Learning curve• Streamlining of tests• Examples -

http://tinyurl.com/576tp8

Objective 4

Slide 129

Using database audit software• Many “built-in” functions right off

the shelf with SQL• Control totals• Exception identification• “Drill down”• Quantification• June 2008 article in the EDP Audit &

Control Journal (EDPACS) “SQL as an audit tool”

Objective 4

Slide 130

Use of Excel

• Built-in functions• Add-ins• Macros• Database access

Objective 4

Slide 131

Objective 4 - Summarized

1. Understand why and how 2. Understand statistical basis for quantifying

differences3. Identify ten general tools and techniques4. Understand examples done using Excel5. How Pattern Detection fits in

Next – Fit …

Slide 132

How Pattern Detection Fits In• Business Analytics• Fraud Pattern

Detection• Continuous monitoring

Objective 5

Slide 133

Where does Fraud Pattern Detection fit in?

• Business Analytics• Fraud Pattern Detection• Continuous fraud

pattern detection• Continuous Monitoring

Right in the middle

Objective 5

Slide 134

Business Analytics

• Fraud analytics -> business analytics

• Business analytics -> fraud analytics

Objective 5

Slide 135

Role in Continuous Monitoring (CM)

• Fraud analytics can feed (CM)

• Continuous fraud pattern detection

• Use output from CM to tune fraud pattern detection

Objective 5

Slide 136




Next: Links …

Slide 137

Links for more information• Kolmogorov-Smirnov • http://tinyurl.com/y49sec• Benford’s Law

http://tinyurl.com/3qapzu• Chi Square tests

http://tinyurl.com/43nkdh• Continuous monitoring

http://tinyurl.com/3pltdl

Slide 138

Market Basket

• Apriori testing for “ping ponging”

• Temple University http://tinyurl.com/5vax7r

• Apriori program (“open source”) http://tinyurl.com/5qehd5

• Article – “Medical ping ponging” http://tinyurl.com/5pzbh4

Slide 139

Excel macros used in auditing

• Excel as an audit software http://tinyurl.com/6h3ye7

• Selected macros - http://tinyurl.com/576tp8

• Spreadsheets forever - http://tinyurl.com/5ppl7t

Slide 140

Questions?

Slide 141

Contact info

• E-mail: [email protected]• Web: http://ezrstats.com

Documents

Detection of Fraud