One-Pass Wavelet Decompositions of Data Streams
TKDE May 2002
Anna C. Gilbert,Yannis Kotidis, S. Muthukrishanan, Martin J. Strauss
Presented by James ChanCSCI 599 Multidimensional Databases
Fall 2003
Outline of Talk
• Introduction
• Background
• Proposed Algorithm
• Experiments
• End Notes
Streaming Applications
• Telephone Call Duration
• Call Detail Record (CDR)
• IP Traffic Flow
• Bank ATM Transactions
• Mission Critical Task:– Fraud– Security– Performance Monitoring
Data Stream Model
Data Stream Problem• One Pass – no backtracking• Unbounded Data – Algorithms require small memory usage• Continuous – Need to run real time
Stream ProcessingEngine
(Approximate) Answer
Synopsis in Memory
Data Streams
Data Stream Strategies
• Many stream algorithms produce approximate answers and have:– Deterministic Bounds: answers are within ±– Probabilistic Bounds: answers have high
success probability (1-) within ±
Data Stream Strategies
• Windows: New elements expire after time t
• Samples: Approximate entire domain with a sample
• Histograms: Partitioning element domain values into buckets (Equi-depth, V-Opt)
• Wavelets: Haar, Construction and maintenance (difficult for large domain)
• Sketch Techniques: estimate of L2 norm of a signal
Proposed Stream Model
Background: Cash Register vs. Aggregate
• Cash Register: incoming stream represents domain (increment or decrement range of that domain)
• Aggregate: incoming stream represents range, (update range of that domain)
Note: Examples in this paper assume– each cash register element as +1 unit– no duplicate elements in aggregate models
Background: Cash Register vs. Aggregate
Cash Register
(domain)
Aggregate
(range)
Ordered Easiest
Eg. Time Series
Unordered General Challenging
Eg. Network volume
Contiguous Same as aggregate unordered n/a
Background: Wavelet Basics
• Wavelet transforms capture trends in a signal
• Typical transform involves log n passes
• Each pass creates two sets of n/2 averages and differences.
• Process repeated on averages
• Output: Wavelet Basis vectors – one average and n-1 coefficients
Background: Haar Wavelet Notation
• High pass filter
• Low pass filter
• Input: signal a
• Basis Coefficients
• Coefficients
• Scaling Factor
• Psi Vectors (un-normalized)
}2/1,2/1{
}2/1,2/1{
],...,[ 21 naaa
kjjkj asd ,, ,
}{}{],...,[ ,0,0110 kjn dcwww
jj Ns 2/
kj ,
]0,0,0,0,1,1,1,1[
]0,0,0,0,1,1,0,0[
]0,0,0,0,0,0,1,1[
0,2
2,3
0,3
kj
kj
kj
Background: Haar Wavelet Example
Background: Small B Representation
• Most signals in nature have small B representation
• Only keep largest B wavelet coefficients to estimate energy of signal
• Additional coefficients do not help reduce squared sum error
Energy:
SSE:
2
2R
2
2Ra
Background: Storage
• Highest B wavelet coefficients
• Log N Straddling coefficients, one per level of the wavelet tree
2 2 0 2 3 5 4 4
-1.25
2.75
0.5 0
0 -1 0 -1
+
-+
+
+ + +
+
+
- -
- - - -
Original Signal
Background: Bounding Theorems
Theorem 1• Given O(B+logN) storage (B is number of dimensions)• time to compute new data item is O(B+logN) in ordered
aggregate model
Theorem 2• Any algorithm that calculates the 2nd largest wavelet coefficient of
the signal in unordered CR / unordered agg uses at least N/polylog(N)
• This holds if:– You only care about existence, not the coefficients value– Only calculating up to a factor of 2
Proposed Algorithm: Overview
• Avoid keeping anything domain size N in memory
• Estimate wavelet coefficients using sketches which are size log(N)
• Sketch is maintained in memory and is updated as data entries stream in
What’s a Sketch?
• Distortion Parameter (epsilon)• Failure Probability (delta)• Failure Threshold (eta)• Original Signal a• Random vector of {-1,+1}s r• Seed for r s• Atomic Sketch <a,r> dot product of a and r• Sketch O(log(N/ )/ ^2) atomic sketches
• We use the same j to index the atomic sketch, seed, and random vector, so there are j atomic sketches in a sketch
Updating a Sketch
• Cash Register– Add corresponding to the j atomic sketches
• Aggregate– Add corresponding to the j atomic sketches
Use generator that takes in seed which is log(N) to compute
jir
jiria )(
jis
),( isGr jji
jir
Reed Muller Generator
• Pseudo random generator meeting these requirements:– Variables are 4 wise
independent• Expected value of product of any
4 distinct r is 0
– Requires O(log N) space for seeding
– Performs computation in polylog(N) time
{0} {d} {c} …. {d,c,b,a}
X = median ( )
Estimation of Inner Product
…
O(log(1/))
O(log(1/^2))
= mean ( )
… … …
…
Boosting Accuracy and Confidence
• Improve accuracy to by averaging over more independent copies of for each average
• Improve Confidence by increasing number of averages to take median over
j
iijii rbra
O(log(1/^2)) copies of …
X = median of ( )
= means ( )…
…
O(log(1/)) copies of …
Using the sketches
j
iijii rbraba ~,
• We can approximate <a,> to maintain Bs• Note a point query is <a,ei> where e is a vector
with a 1 at index i and 0s everywhere else
jj rbra ,,Atomic Sketches in memory
Maintaining Top B Coefficients
• At most Log N +1 coefficient updates
• May need to approximate straddling coefficients to aggregate with already existing or near variables
• Compare updates with top B and update top B if necessary
ia
updated
unaffected
Algorithm Space and Time
Their algorithm uses polylog(N) space and per item time to maintain B terms (by approximation)
Experiments• Data: one week of AT&T call detail (unordered cash
register model)
• Modes– Batch: Query only between intervals– Online: Query anytime
• Direct Point: calc sketch of <ei,a> (ei is zero vector except with 1 at i)
• Direct Wavelets: estimate all supporting coefficients and use wavelet reconstruction to calculate point a(i)
• Top B: Reconstruction of point is done with Top B (maintained by sketch)
Top B – Day 0
Top B - 1 Week
(fixed-set) Value updates only. no replacement
Sketch Size on Accuracy
Heavy Hitters
• Points that contribute significantly to the energy of the signal
• Direct point estimates are very accurate for heavy hitters but gross estimates for non heavy hitters
• Adaptive Greedy pursuit: by removing the first heavy hitter from the signal, you improve the accuracy of calculating the next biggest heavy hitter
• However an error is introduced with each subtraction of a heavy hitter
Processing Heavy Hitters
Adaptive Greedy Pursuit
End Notes
• First Provable Guarantees for haar wavelet over data streams
• Can estimate Haar coefficients ci=<a,>• Top B is updated in:
• This paper is superseded by "Fast, Small-space algorithms for approximatehistogram maintenance" STOC 2002– Discusses how to select top B and find heavy hitters
))()log((log 3 BNNO
Recommended
