PhD Defensetildeweb.au.dk/au121/advising/slides/jesper-sindahl-nielsen.pdf · PhD Defense Jesper...

AARHUSUNIVERSITY

PhD Defense

Jesper Sindahl Nielsen

Implicit Data Structures,Sorting, and Text Indexing

Overview

1. Algorithms and Data Structures(a) Abstract description(b) Example

2. Implicit Data Structures(a) Finger Search(b) Priority Queues

3. Integer Sorting4. Text Indexing5. State and University Library

Overview

Algorithms

Algorithm

Algorithms

Algorithminput

Algorithms

Algorithminput output

Algorithms

x + yAddition

Algorithms

x=5y=7 x + y

Addition

Algorithms

12x=5y=7 x + y

Addition

Algorithms

12x=5y=7 x + y

Addition

Sort L

Algorithms

12x=5y=7 x + y

Addition

L=5,1,3,2,4,6 Sort L

Algorithms

12x=5y=7 x + y

Addition

L=1,2,3,4,5,6L=5,1,3,2,4,6 Sort L

We want to design fast algorithms

Data Structures

DataData

Structure

Data Structures

DataData

Structure

Data organized insome clever manner

Data Structures

Data BuildData

Structure

Data Structures

Data BuildData

Structure

Mom,375874Dad,278393Brother,681958Advisor,359617Arnold,123456Sylvester,987654Jean-Claude,456321Wesley,789123

Data Structures

Data BuildData

Structure

create sorted phonebook

Data Structures

Data BuildData

Structure

Advisor,359617Arnold,123456Brother,681958Dad,278393Jean,456321Mom,375874Sylvester,987654Wesley,789123

Data Structures

Data BuildData

Structure

DataStructure

Data Structures

Data BuildData

Structure

DataStructure

Queryinput

Data Structures

Data BuildData

Structure

DataStructure

Queryinput

answer to query on data

Data Structures

Data BuildData

Structure

DataStructure

Queryinput

Arnold

Data Structures

Data BuildData

Structure

DataStructure

Queryinput

find phone number

Arnold

Data Structures

Data BuildData

Structure

DataStructure

Queryinput

find phone number

Arnold 123456

Data Structures

Data BuildData

Structure

DataStructure

Queryinput

find phone number

Mom 375874

Arnold 123456

Data Structures

Data BuildData

Structure

DataStructure

Queryinput

find phone number

Mom 375874

Arnold 123456

efficient when data is organized

Data Structures

Data BuildData

Structure

Advisor,359617Arnold,123456Brother,681958Dad,278393Jean,456321Mom,375874Sylvester,987654Wesley,789123Data

StructureUpdate

inputUpd. DataStructure

Data Structures

Data BuildData

Structure

StructureUpdate

Dad,357951

Data Structures

Data BuildData

Structure

StructureUpdate

Dad,357951

Data Structures

Data BuildData

Structure

StructureUpdate

Dad,357951

Change phone number

Data Structures

Data BuildData

Structure

StructureUpdate

Dad,357951

Change phone number

Data Structures

Data BuildData

Structure

StructureUpdate

Dad,357951

Change phone number

We want space efficient structures,fast queries, and fast updates

Outline

Implicit Data Structures

Primary goal: space efficiency

Data Structure has n comparable elements, laid out in an array of size n.

n− 1

Only the array and n is maintained between operations.

n− 1

Strict implicit model: Nothing but elements and n stored

n− 1

Weak implicit model: Allow a constant number of extra memory cells

n− 1

We work with the strict implicit model

n− 1

Fundamental trick: pair encoding bits.

n− 1

Fundamental trick: pair encoding bits. x yx<y ⇒1

x>y ⇒0

n− 1

x>y ⇒0Requires distinct elements.

n− 1

x>y ⇒0Requires distinct elements.

Procedures can use a constant number of working memory/registers

Dictionary and Fingersearch

Maintain a (dynamic) set of n keys, while supporting:

Insert(k)

Delete(k)

Insert(k)

Delete(k)

Find(k)

Insert(k)

Delete(k)

Find(k)

Predecessor(k)

Insert(k)

Delete(k)

Find(k)

Predecessor(k)

Successor(k)

Insert(k)

Delete(k)

Find(k)

Predecessor(k)

Time: O(log n)

Successor(k)

Insert(k)

Delete(k)

Find(k)

Predecessor(k)

Only in dynamic structures

Time: O(log n)

Successor(k)

Insert(k)

Delete(k)

Find(k)

Predecessor(k)

Time: O(log n)

Successor(k)

Special element: the finger element

Insert(k)

Delete(k)

Find(k)

Predecessor(k)

Time: O(log n)

Successor(k)

a c d g j ` m o q s z0 1 2 3 4 5 6 7 8 9 10

Insert(k)

Delete(k)

Find(k)

Predecessor(k)

Time: O(log n)

Successor(k)

a c d g j ` m o q s z0 1 2 3 4 5 6 7 8 9 10

Insert(k)

Delete(k)

Find(k)

Predecessor(k)

Time: O(log n)

Successor(k)

a c d g j ` m o q s z0 1 2 3 4 5 6 7 8 9 10

finger

Insert(k)

Delete(k)

Find(k)

Predecessor(k)

Time: O(log n)

Successor(k)

a c d g j ` m o q s z0 1 2 3 4 5 6 7 8 9 10

finger

Find(m)

Insert(k)

Delete(k)

Find(k)

Predecessor(k)

Time: O(log n)

Successor(k)

a c d g j ` m o q s z0 1 2 3 4 5 6 7 8 9 10

finger

Find(m)

Distance: t = 3

Insert(k)

Delete(k)

Find(k)

Predecessor(k)

Time: O(log n)

Successor(k)

a c d g j ` m o q s z0 1 2 3 4 5 6 7 8 9 10

finger

Find(m)

Distance: t = 3

Time: O(log t) forFind, Predecessor, and Successor

Implicit Finger Search Idea

· · ·

· · · · · · · · ·

· · ·

· · · · · · · · ·

· · ·

· · · · · · · · ·

· · ·

D0 D1 D2 D`

· · · · · · · · ·

· · ·

D0 D1 D2 D`

· · ·

Elements in sorted order

· · · · · ·

· · ·

D0 D1 D2 D`

· · ·

· · · · · ·

· · ·

D0 D1 D2 D`

· · ·

· · · · · ·

· · ·

D0 D1 D2 D`

· · ·

· · · · · ·

· · ·

D0 D1 D2 D`

· · ·

· · · · · ·

· · ·

D0 D1 D2 D`

· · ·

· · · · · ·

· · ·

D0 D1 D2 D`

· · ·

· · · · · ·

· · ·

D0 D1 D2 D`

· · ·

· · · · · ·

· · ·

D0 D1 D2 D`

· · ·

· · · · · ·

· · ·

D0 D1 D2 D`

· · ·

· · · · · ·

· · ·

Operation Time

Insert

Delete

Find O(log t)

Change-Finger

O(logn)

O(nε)

· · ·

D0 D1 D2 D`

· · ·

· · · · · ·

· · ·

Operation Time

Insert

Delete

Find O(log t)

Change-Finger

O(logn)

O(nε)

optimal trade-off

Priority Queues

Maintain a set of n keys and support:

• FindMin: Return the minimum element• ExtractMin: Remove the minimum element from the set• Insert(k): Add the key k to the set

Priority Queues

Insert ExtractMin Moves StrictIdenticalElements

Amortized bounds are marked with ?

Priority Queues

Williams

log nlog n log n yes yes

Priority Queues

Williams

Carlsson et al.

log n1 log n no yes

Priority Queues

Williams

Carlsson et al.

log n1 log n no yes

log n1 log n no yesEdelkamp et al.

Priority Queues

Williams

Carlsson et al.

Harvey & Zatloukal

log n1 log n no yes

? log n? 1 ? log n yes yes

Priority Queues

Williams

Franceschini & Munro

Carlsson et al.

Harvey & Zatloukal

log n1 log n no yes

? log n? log n ? 1 yes no

Priority Queues

Williams

Carlsson et al.

Harvey & Zatloukal

log n1 log n no yes

? log n? 1 ? 1 yes yesNew

Priority Queues

Williams

Carlsson et al.

Harvey & Zatloukal

log n1 log n no yes

? log n? 1 ? 1 yes yes

log n1 log n yes no

Outline

Integer Sorting

Comparison based sorting: only allowed operation ≤

Integer Sorting

conclude A > B

Integer Sorting

conclude A > B

To sort n gold pieces we needΘ(n log n) scalings/comparisons

Integer Sorting

conclude A > B

287315

Integer Sorting

conclude A > B

287315

Question: Can we sort faster ifwe have access to a digital scale?

Integer Sorting

conclude A > B

287315

Answer: Yes.

Integer Sorting

conclude A > B

287315

Answer: Yes.

Question: How much faster?

Integer Sorting

conclude A > B

287315

Answer: Yes.

Depends on accuracy of the scale

Question: How much faster?

Integer Sorting

Time Space NotesAlgorithm

w is the number of bits per word.

Integer Sorting

Radix sort (1887?) O(n · wlogn

) O(n) optimal for w = O(log n)

Integer Sorting

vEB sort (1975) O(n log w) O(2w) Only O(n) space with Y-fast trie

Integer Sorting

Kirkpatrick &Reisch (1984)

O(n log wlogn

) Only asymptotically better thanvEB when w = O(log n)

Integer Sorting

O(n log wlogn

Signature sort(1995)

O(n) O(n) Only for w = Ω(log2+ε n)

Integer Sorting

O(n log wlogn

Han & Thorup(2002) O(n

√log(w/ log n)) Randomized with mulitplicationO(n)

Integer Sorting

O(n log wlogn

√log(w/ log n)) Randomized with mulitplication

Thorup (2002) O(n log log n) O(n) Randomized, AC0

Integer Sorting

O(n log wlogn

√log(w/ log n)) Randomized with mulitplication

Thorup (2002) O(n log log n) O(n) Randomized, AC0

Han (2004) O(n log log n) O(n) Deterministic, with multiplication

Integer Sorting

Time NotesAlgorithm

) Optimal for w = O(log n)

Signature sort (1995) O(n) Only for w = Ω(log2+ε n)

Han & Thorup (2002) O(n√log(w/ log n)

)Randomized with mulitplication

Time per element

O(√log log n)

log2 n log log n log2+ε n

Overview

Text Indexing

Most of us daily use services like Google, Bing, etc.

Text Indexing

We want documents that contain both terms

Text Indexing

Find all emails sent to madalgo that are not seminars

Text Indexing - Two Patterns

Problem definition:

Given documents D = d1, d2, . . . , dD

Problem definition:

preprocess and store D such that

Problem definition:

when given two text strings P1, P2 (online)

Problem definition:

we can report all di where both P1 and P2 occur

Problem definition:

we can report all di where both P1 and P2 occur(The two-pattern problem)

Problem definition:

we can report all di where both P1 occurs but P2

does not occur

(The two-pattern problem)

Problem definition:

we can report all di where both P1 occurs but P2

does not occur

(The two-pattern problem)

(The forbidden pattern problem)

Space Query Time AuthorsProblem

Two-Pattern (2P) n2−α logc n nα + t Ferragina et al. ’03

Two-Pattern n logn√nt log n log2 n + t Cohen and Porat ’10

Two-Pattern n logn√nt log n log2 n + t

Two-Pattern n√nt log n log n + t

Cohen and Porat ’10

Hon et al. ’10

Forbidden Pattern (FP) n3/2/ logn√n + t Fischer et al. ’12

Hon et al. ’10

Forbidden Pattern Hon et al. ’12n√nt log n log2 n + t

Hon et al. ’10

Both 2P and FP n√nt log n logε n + t New

Hon et al. ’10

Both 2P and FP n√nt log n logε n + t New

Is√n query time really necessary?

Build suffix tree on concatenation of the documents

S = d1$d2$ · · · $dD$

Annotate each leaf with thedocument id the suffx starts in

4 5 4 7 6 4 8 9 9 1 2 3

S = d1$d2$ · · · $dD$