Benford's Law: Tables of Logarithms, Tax Cheats, and The

History Applications Benford and Integer Sequences Formalism Benford and Recurrence Relations

Benford’s Law: Tables of Logarithms, TaxCheats, and The Leading Digit

Phenomenon

Michelle Manes ([email protected])

9 September, 2008

1


History

(1881) Simon Newcomb publishes “Note on thefrequency of use of the different digits in naturalnumbers.” The world ignores it.

(1938) Frank Benford (unaware of Newcomb’s work,presumably) publishes “The law of anomalousnumbers.”

2


History

(1881) Simon Newcomb publishes “Note on thefrequency of use of the different digits in naturalnumbers.” The world ignores it.

(1938) Frank Benford (unaware of Newcomb’s work,presumably) publishes “The law of anomalousnumbers.”

3


Statement of Benford’s Law

Newcomb noticed that the early pages of the book oftables of logarithms were much dirtier than the laterpages, so were presumably referenced more often.

He stated the rule this way:

Prob(first significant digit = d) = log10

(1 +

1d

).

4


Statement of Benford’s Law

Newcomb noticed that the early pages of the book oftables of logarithms were much dirtier than the laterpages, so were presumably referenced more often.

He stated the rule this way:


(1 +

1d

).

5


Benford Distribution

DefinitionA sequence of positive numbers {xn} is Benford if


(1 +

1d

).

A sequence of positive numbers {xn} is Benford base b if

Prob(first significant digit = d) = logb

(1 +

1d

).

6


Benford Distribution

DefinitionA sequence of positive numbers {xn} is Benford if


(1 +

1d

).

A sequence of positive numbers {xn} is Benford base b if

Prob(first significant digit = d) = logb

(1 +

1d

).

7


Benford’s Law

Base 10 Predictionsdigit probability it occurs as a leading digit

1 30.1%2 17.6%3 12.5%4 9.7%5 7.9%6 6.7%7 5.8%8 5.1%9 4.6%

8


Benford’s Data

9


More Data

Benford’s Law compared with: numbers from the frontpages of newspapers, U.S. county populations, and theDow Jones Industrial Average.

10


Dow Illustrates Benford’s Law

Suppose the Dow Jones average is about $1,000. If theaverage goes up at a rate of about 20% a year, it wouldtake four years to get from 1 to 2 as a first digit.

If we start with a first digit 5, it only requires a 20%increase to get from $5,000 to $6,000, and that isachieved in one year.

When the Dow reaches $9,000, it takes only an 11%increase and just seven months to reach the $10,000mark. This again has first digit 1, so it will take anotherdoubling (and four more years) to get back to first digit 2.

11






12






13






14


Benford’s Law and Tax Fraud (Nigrini, 1992)

Most people can’t fake data convincingly.

Many states (including California) and the IRS now usefraud-detection software based on Benford’s Law.

15





16





17


True Life Tale

Manager from Arizona State Treasurer wasembezzling funds.

Most amounts were below $100,000 (criticalthreshold for checks that would require morescrutiny).Over 90% of the checks had a first digit 7, 8, or 9.(Trying to get close to the threshold without goingover — artificially changes the data and so breaks fitwith Benford’s law.)

18


True Life Tale

Manager from Arizona State Treasurer wasembezzling funds.Most amounts were below $100,000 (criticalthreshold for checks that would require morescrutiny).

Over 90% of the checks had a first digit 7, 8, or 9.(Trying to get close to the threshold without goingover — artificially changes the data and so breaks fitwith Benford’s law.)

19


True Life Tale

Manager from Arizona State Treasurer wasembezzling funds.Most amounts were below $100,000 (criticalthreshold for checks that would require morescrutiny).Over 90% of the checks had a first digit 7, 8, or 9.(Trying to get close to the threshold without goingover — artificially changes the data and so breaks fitwith Benford’s law.)

20


True Life Tale

21


What types of sequences are Benford?

Real-world data can be a good fit or not, depending onthe type of data. Data that is a good fit is “suitablyrandom” — comes in many different scales, and is a largeand randomly distributed data set, with no artificial orexternal limitations on the range of the numbers.

Some numerical sequences are clearly not Benford:1,2,3,4,5,6,7, . . . (uniform distribution)

1,10,100,1000, . . . (first digit is always 1)

22




Some numerical sequences are clearly not Benford:

1,2,3,4,5,6,7, . . . (uniform distribution)


23






24






25


Powers of Two

1,2,4,8,16,32,64, . . .

tn = 2n

26


Powers of Two

1,2,4,8,16,32,64, . . .

tn = 2n

27


Powers of Two

1,2,4,8,16,32,64, . . .

tn = 2n

28


Powers of Two

29


Fibonacci Numbers

1,1,2,3,5,8,13,21, . . .

Fn+1 = Fn + Fn−1

30


Fibonacci Numbers

1,1,2,3,5,8,13,21, . . .

Fn+1 = Fn + Fn−1

31


Fibonacci Numbers

1,1,2,3,5,8,13,21, . . .

Fn+1 = Fn + Fn−1

32


Fibonacci Numbers

33


Mantissa Function

DefinitionDefine the mantissa function

M : R+ → [1,10)

x 7→ r

where r is the unique number in [1,10) such thatx = r × 10n for some n ∈ Z.

In other words, write the number in scientific notation.

34


Mantissa Function


M : R+ → [1,10)

x 7→ r


In other words, write the number in scientific notation.

35


Mantissa Function


M : R+ → [1,10)

x 7→ r


ExamplesM(9,000,001) = 9.000001.M(0.01247) = 1.247.

36


Logarithms and Benford’s Law

Fundamental EquivalenceData set {xi} is Benford if {yi} is equidistributed mod 1,where yi = log10 xi .

Proof:x = M(x) · 10k for some k ∈ Z.First digit of x is d iff d ≤ M(x) < d + 1.log10 d ≤ y < log10(d + 1), wherey = log10(M(x)) = log10 x mod 1.If the distribution is uniform (mod 1), then theprobability y is in this range is

log10(d+1)−log10(d) = log10

(d + 1

d

)= log10

(1 +

1d

).

37




Proof:x = M(x) · 10k for some k ∈ Z.

First digit of x is d iff d ≤ M(x) < d + 1.log10 d ≤ y < log10(d + 1), wherey = log10(M(x)) = log10 x mod 1.If the distribution is uniform (mod 1), then theprobability y is in this range is


(d + 1

d

)= log10

(1 +

1d

).

38




Proof:x = M(x) · 10k for some k ∈ Z.First digit of x is d iff d ≤ M(x) < d + 1.

log10 d ≤ y < log10(d + 1), wherey = log10(M(x)) = log10 x mod 1.If the distribution is uniform (mod 1), then theprobability y is in this range is


(d + 1

d

)= log10

(1 +

1d

).

39




Proof:x = M(x) · 10k for some k ∈ Z.First digit of x is d iff d ≤ M(x) < d + 1.log10 d ≤ y < log10(d + 1), wherey = log10(M(x)) = log10 x mod 1.

If the distribution is uniform (mod 1), then theprobability y is in this range is


(d + 1

d

)= log10

(1 +

1d

).

40




Proof:x = M(x) · 10k for some k ∈ Z.First digit of x is d iff d ≤ M(x) < d + 1.log10 d ≤ y < log10(d + 1), wherey = log10(M(x)) = log10 x mod 1.If the distribution is uniform (mod 1), then theprobability y is in this range is


(d + 1

d

)= log10

(1 +

1d

).

41




42




0 1

1 102

log 2 ! log 10

43




Kronecker-Weyl TheoremIf β 6∈ Q then nβ mod 1 is equidistributed.(Thus if log10 α 6∈ Q, then αn is Benford.)

44


Powers of 2

TheoremThe sequence {2n} for n ≥ 0 is Benford.

Proof:Consider the sequence of logarithms {n(log10 2)}.By the Kronecker-Weyl Theorem, this is uniform(mod 1) because log10 2 6∈ Q.Since the sequence of logarithms is uniformlydistributed (mod 1), the original sequence isBenford.

45


Powers of 2

TheoremThe sequence {2n} for n ≥ 0 is Benford.

Proof:Consider the sequence of logarithms {n(log10 2)}.By the Kronecker-Weyl Theorem, this is uniform(mod 1) because log10 2 6∈ Q.Since the sequence of logarithms is uniformlydistributed (mod 1), the original sequence isBenford.

46


Fibonacci Numbers

TheoremThe sequence {Fn} of Fibonacci numbers is Benford.

Heuristic Argument:Closed form for Fibonacci numbers:

Fn =1√5

[(1 +√

52

)n

−

(1−√

52

)n].

∣∣∣(1−√

52

)∣∣∣ < 1, so the leading digits are completely

determined by 1√5

(1+√

52

)n.

This sequence is Benford because log10

(1+√

52

)6∈ Q.

47


Fibonacci Numbers



Fn =1√5

[(1 +√

52

)n

−

(1−√

52

)n].

∣∣∣(1−√

52


determined by 1√5

(1+√

52

)n.


(1+√

52

)6∈ Q.

48


Fibonacci Numbers



Fn =1√5

[(1 +√

52

)n

−

(1−√

52

)n].

∣∣∣(1−√

52


determined by 1√5

(1+√

52

)n.


(1+√

52

)6∈ Q.

49


Fibonacci Numbers



Fn =1√5

[(1 +√

52

)n

−

(1−√

52

)n].

∣∣∣(1−√

52


determined by 1√5

(1+√

52

)n.


(1+√

52

)6∈ Q.

50


Linear Recurrence Sequences

Consider the sequence {an} given by some initialconditions a0,a1, . . . ,ak−1 and then a recurrence relation

an+k = c1an+k−1 + c2an+k−2 + · · ·+ ckan,

with c1, c2, . . . , ck fixed real numbers.

Find the eigenvalues of the recurrence relation and orderthem so that |λ1| ≥ |λ2| ≥ · · · ≥ |λk |.

There exist numbers u1,u2, . . . ,uk (which depend on theinitial conditions) so that an = u1λ

n1 + u2λ

n2 + · · ·+ ukλ

nk .

51








n1 + u2λ

n2 + · · ·+ ukλ

nk .

52








n1 + u2λ

n2 + · · ·+ ukλ

nk .

53



TheoremWith a linear recurrence sequence as described, iflog10 |λ1| 6∈ Qand the initial conditions are such thatu1 6= 0, then the sequence {an} is Benford.

Sketch of Proof:

Rewrite the closed form as an = u1λn1

(1 +O

(kuλn

2λn

1

))where u = maxi |ui |+ 1.Some clever algebra using our assumptions to findthat yn = log10(an) = n log10 λ1 + log10 u1 +O(βn) forsome appropriate β.Show in the limit the error term affects a vanishinglysmall portion of the distribution.

54




Sketch of Proof:


(1 +O

(kuλn

2λn

1

))where u = maxi |ui |+ 1.

Some clever algebra using our assumptions to findthat yn = log10(an) = n log10 λ1 + log10 u1 +O(βn) forsome appropriate β.Show in the limit the error term affects a vanishinglysmall portion of the distribution.

55




Sketch of Proof:


(1 +O

(kuλn

2λn

1

))where u = maxi |ui |+ 1.Some clever algebra using our assumptions to findthat yn = log10(an) = n log10 λ1 + log10 u1 +O(βn) forsome appropriate β.

Show in the limit the error term affects a vanishinglysmall portion of the distribution.

56




Sketch of Proof:


(1 +O

(kuλn

2λn

1

))where u = maxi |ui |+ 1.Some clever algebra using our assumptions to findthat yn = log10(an) = n log10 λ1 + log10 u1 +O(βn) forsome appropriate β.Show in the limit the error term affects a vanishinglysmall portion of the distribution.

57


Elliptic Divisibility Sequences

DefinitionAn integral divisibility sequence is a sequence of integers{un} satisfying

un | um whenever n | m.

An elliptic divisibility sequence is an integral divisibilitysequence which satisfies the following recurrence relationfor all m ≥ n ≥ 1:

um+num−nu21

= um+1um−1u2n − un+1un−1u2

m.

58


Boring Elliptic Divisibility Sequences

The sequences of integers, where un = n.

The sequence 0,1,−1,0,1,−1, . . ..

The sequence1,3,8,21,55,144,377,987,2584,6765, . . . (this isevery-other Fibonacci number).

59




The sequence 0,1,−1,0,1,−1, . . ..


60




The sequence 0,1,−1,0,1,−1, . . ..


61


Not-So-Boring Elliptic Divisibility Sequences

The sequences which begins0,1,1,−1,1,2,−1,−3,−5,7,−4,−28,29,59,129,−314,−65,1529,−3689,−8209,−16264,833313,113689,−620297,2382785,7869898,7001471,−126742987,−398035821,168705471, . . .(This is sequence A006769 in the On-LineEncyclopedia of Integer Sequences.)

The sequence which begins1,1,−3,11,38,249,−2357,8767,496036,−3769372,−299154043,−12064147359, . . ..

62


Not-So-Boring Elliptic Divisibility Sequences

The sequences which begins0,1,1,−1,1,2,−1,−3,−5,7,−4,−28,29,59,129,−314,−65,1529,−3689,−8209,−16264,833313,113689,−620297,2382785,7869898,7001471,−126742987,−398035821,168705471, . . .(This is sequence A006769 in the On-LineEncyclopedia of Integer Sequences.)

The sequence which begins1,1,−3,11,38,249,−2357,8767,496036,−3769372,−299154043,−12064147359, . . ..

63


Why We Like Elliptic Divisibility Sequences

Special case of Somos sequences, which areinteresting and an active area of research.

Connection to elliptic curves, also an active area ofresearch.

Elliptic curve

E over Qwith rational point

P on E

↔ EDS: denominators

of the sequenceof points {P,2P,3P, . . .}

Applications to elliptic curve cryptography.

64





Elliptic curve


P on E



Applications to elliptic curve cryptography.

65





Elliptic curve


P on E



Applications to elliptic curve cryptography.66


Elliptic Divisibility Sequences are Benford?

67



68



69


Heuristic Argument

It’s well-known that elliptic divisibility sequencessatisfy a growth condition like un ≈ cn2 where theconstant c depends on the arithmetic height of thepoint P and on the curve E .

Weyl’s theorem tells us that {nkα} is uniformdistributed (mod 1) iff α 6∈ Q.

So we should at least be able to conclude that a givenEDS is Benford base b for almost every b.

But: The argument with the big-O error terms isdelicate, and not enough is known in the case of EDS.

70


Heuristic Argument





71


Heuristic Argument





72


Heuristic Argument





73

Documents

Benford's Law: Tables of Logarithms, Tax Cheats, and The