1 Efficient Computation of Frequent and Top-k Elements in Data Streams

1

Efficient Computation of Frequent and Top-k Elements in Data Streams

2

Motivation

Motivated by Internet advertising commissioners Before rendering an advertisement for user, query clicks

stream for advertisements to display. If the user's profile is not a frequent “clicker”, then s/he

will probably not click any displayed advertisement.– Show Pay-Per-Impression advertisements.

If the user's profile is a frequent “clicker”, then s/he may click a displayed advertisement.– Show Pay-Per-Click advertisements.

– Retrieve top advertisements to choose what to display.

3

Problem Definition

Given alphabet A, stream S of size N, a frequent element, E, is an element whose frequency, F, exceeds a user specified support, φN

Top-k elements are the k elements with highest frequency

Both problems:– Very related, though, no integrated solution has been

proposed– Exact solution is O(min(N,A)) space

approximate variations

4

Practical Frequent Elements

-Deficient Frequent Elements [Manku ‘02]:– All frequent elements output should have

F > (φ - )N, where is the user-defined error.

φ N

(φ - ) N

5

Practical Top-k

FindApproxTop(S, k, ) [Charikar ‘02]:– Retrieve a list of k elements such that every

element, Ei, in the list has Fi > (1 - ) Fk, where Ek

is the kth ranked element.

F4

(1 - ) F4

6

The Space-Saving Algorithm

Space-Saving is counter-based Monitor only m elements Only over-estimation errors Frequency estimation is more accurate

for significant elements Keep track of max. possible errors

7

Space-Saving By ExampleElement

Count

error (max possible)

A B B A C A B B D D

Element A B C

Count 2 2 1

error (max possible) 0 0 0

Element A B C

Count 3 2 1


Element B A C

Count 4 3 1


Element B A D

Count 4 3 2


Element B A D

Count 5 3 3

error (max possible) 0 0 1E

Element B E A

Count 5 4 3


Space-Saving Algorithm– For every element in the stream S

– If a monitored element is observed• Increment its Count

– If a non-monitored element is observed, • Replace the element with minimum hits, min• Increment the minimum Count to min + 1• maximum possible over-estimation is error













C

Element B E C

Count 5 4 4

error (max possible) 0 3 3B

8

Space-Saving Observations

Observations:– The summation of the Counts is N

Element B E C

Count 5 4 4


S = ABBACABBDDBEC N = 13

– Minimum number of hits, min ≤ N/m– In this example, min = 4

Element B E C

Count 5 4 4


– The minimum number of hits, min, is an upper bound on the error of any element

Element B E C

Count 5 4 4


9

Space-Saving Proved Properties

1. If Element E has frequency F > min, then E must be in Stream-Summary. F(B) = F1 = 5, min = 4.


Element B E C

Count 5 4 4


2. The Count at position i in Stream-Summary is no less than Fi, the frequency of the ith ranked element. F(A) = F2 = 3, Count2 = 4.

Element B E C

Count 5 4 4



10

Space-Saving Data Structure

We need a data structure that– Increments counters in constant time– Keeps elements sorted by their counters

We propose the Stream-Summary structure, similar to the data structure in [Demaine ’02]

11

Frequent Elements Queries

Traverse Stream-Summary, and report all elements that satisfy the user support

Any element whose

guaranteed hits = (Count – error) > φN

is guaranteed to be a frequent element

12

Frequent Elements Example

For N = 73, m = 8, φ = 0.15:– Frequent Elements should have support of 11 hits.– Candidate Frequent Elements are B, D, and G.

Element B D G A Q F C E

Count 20 14 12 9 7 5 3 3

error 1 0 4 1 3 0 1 2

Guaranteed Hits = Count - error 19 14 8 8 4 5 2 1

– Guaranteed Frequent Elements are B, and D, since their guaranteed hits > 11.


Count 20 14 12 9 7 5 3 3

error 1 0 4 1 3 0 1 2


13

Top-k Elements Queries

Traverse the Stream-Summary, and report top-k elements.

From Property 2, we assert:– Guaranteed top-k elements:

• Any element whose guaranteed hits = (Count – error) ≥ Countk+1, is guaranteed to be in the top-k.

– Guaranteed top-k’ (where k’≈k):• The top-k’ elements reported are guaranteed to be the

correct top-k’ iff for every element in the top-k’, guaranteed hits = (Count – error) ≥ Countk’+1.

14

Top-k Elements Example

For k = 3, m = 8:– B, D, and G are the top-3 candidates.


Count 20 14 12 9 7 5 3 3

error 1 0 4 1 3 0 1 2


– B, and D are guaranteed to be in the top-3.


Count 20 14 12 9 7 5 3 3

error 1 0 4 1 3 0 1 2


– B , D, G and A are guaranteed to be the top-4. Here k’ = 4.


Count 20 14 12 9 7 5 3 3

error 1 0 4 1 3 0 1 2


– B , and D are guaranteed to be the top-2. Another k’ = 2.


Count 20 14 12 9 7 5 3 3

error 1 0 4 1 3 0 1 2


15

Frequent Elements Precision

Precision for Frequent Elements (>100,000 Hits) on Synthetic Data

0 0

1111111 11111 1

0.833333

0.08890.05260.0707

0.2157

0.1053

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 0.5 1 1.5 2 2.5 3

Zipf Alpha

Pre

cis

ion

Space-Saving GroupTest Frequent

16

Frequent Elements Run Time

Run Time for Frequent Elements (>100,000 Hits) on Synthetic Data

4793745172 43844 43734 43141

27250272182590626125280152650024281

5003149578

6704759375167453103751228111906

0

10000

20000

30000

40000

50000

60000

0 0.5 1 1.5 2 2.5 3

Zipf Alpha

Ru

n T

ime (

ms)


17

Frequent Elements Space Used

Space Used for Frequent Elements (>100,000 Hits) on Synthetic Data

2796

58460

78460

38240

67756

165885636

168260168260 168260 168260 168260 168260168260

13760 13760 1376013760 13760

13760 13760

0

20000

40000

60000

80000

100000

120000

140000

160000

180000

0 0.5 1 1.5 2 2.5 3

Zipf Alpha

Sp

ace U

sed

(B

yte

s)


Max freq. element in stream

Can we promise to find it with less than m buckets?

18

Documents

1 Efficient Computation of Frequent and Top-k Elements in Data Streams