1/24

Advanced Graph Algorithms (I)

• What we do not cover but you are expected to know– Mathematical induction, basic data structure,

sorting, shortest path, minimum spanning tree, dynamic programming, divide-and-conquer

– Homework assignment

• Better if you know NP-complete, NP-hard– Pick it up yourself if not. It becomes a common

sense in computer science.

2/24

Advanced Graph Algorithms (II)

• Who should take– 對演算法有興趣– 對抽象思考有興趣– 培養基礎，以後有志於從事研究工作– 有興趣知道如何將理論應用到實際問題

• 生物資訊，網路搜尋，自然語言• 對任職於 Google, Facebook, Microsoft 有興趣者

• 課程網站（所有的 announcement 都放在此）– 我在研究院的網頁下面的教學網站– http://iasl.iis.sinica.edu.tw/hsu/#teach

成績計算方式• 小考（從作業內出題，作業不必繳交）

• 一次期末考筆試

• 一個程式 project

• 期末論文 presentation （視人數而定）

3/24

4/24

Biography

• 許聞廉 Wen-Lian Hsu 中研院資訊所特聘研究員• 1973 台大數學學士 • 1980 康乃爾 Operations Research 博士• 1980-89 美國西北大學工業工程系• 1989- 中央研究院資訊所

– 發展「自然輸入法」、許氏鍵盤• Research interests:

– Design of algorithms, artificial intelligence, natural language processing, bioinformatics, knowledge management

5/24

Scope

• Consecutive ones test• PQ-trees and PC-

trees• Planar graphs• Maximal Planar

Subgraph Algorithm• Chordal graphs• Interval graphs

• Applications – Sequence assembly– Motif discovery– de novo sequencing– Protein structure

prediction

6/24

A few Examples of Mathematical Induction

• Most algorithms we designed are “recursive.” • N! = N x (N-1)!

• The theoretical basis is “Mathematical Induction.”

7/24

A Hat Problem (I)

No strategy In the worst case, all men were shot.

Strategy 1 (with collaboration) In the worst case, half of the men will be shot.

N prisoners lined up in a row, each one can see the hats of all people in front of him. A person who guesses the color of his hat correctly can survive

8/24

A Hat Problem (II)

Design a strategy so that as few men will die as possible.

Strategy 1 (at least half can survive, probably ¾ will) Divide the men into two groups: odd-numbered and even-numbered. Each odd-numbered person should tell the person in front the correct color (since he can see it). As for the person himself, there is still ½ chance that he will survive)

9/24

A Hat Problem (III)Message Passing

Suppose we use 0 to indicate white hat and 1 for black hat

Let the original sequence be

Then the sequences each man will see are as follows

0 1 1 0 0 1 0 0 0 1 1 0 1 0 0 1 1 1

1 1 0 0 1 0 0 0 1 1 0 1 0 0 1 1 1

1 0 0 1 0 0 0 1 1 0 1 0 0 1 1 1

0 0 1 0 0 0 1 1 0 1 0 0 1 1 1

How do you let each man guess the right # (except the first one)? odd-evenness (or parity) of the # of 0’s and 1’s.

10/24

A Hat Problem (IV)

• If the current hat is 0, then moving to the next sequence will only change the parity of 0 (the parity of 1 stays the same)

• Everyone knows the parity of 0 and 1 for the sequence in front of him.

• If the 1st person says the parity of 1 for his sequence (either odd or even), then by checking whether the parity of 1 changes, the 2nd person knows his hat color

• By induction, everyone afterward can compute his hat color

0 1 1 0 0 1 0 0 0 1 1 0 1 0 0 1 1 1

1 1 0 0 1 0 0 0 1 1 0 1 0 0 1 1 1

1 0 0 1 0 0 0 1 1 0 1 0 0 1 1 1

0 0 1 0 0 0 1 1 0 1 0 0 1 1 1

11/24

Marriage Theorem

The following condition is both necessary and sufficient: Every set of r girls, 1 r n, like at least r boys.

There are n girls and n boys. Each girl has a list of boys she can marry. Assume a boy never rejects a girl’s offer. Under what condition can you find a perfect match?

Prove by induction (for the sufficient part, since necessity is clear). How do you reduce the problem size to a smaller one?

Easy case: There is a subset of k girls who like exactly k boys

By induction, can match these k girls with the k boys. Again, by induction, the remaining n-k girls can be matched to the n-k boys

Otherwise: we have: Every set of r girls, 1 r n, likes > r boys.

Marry a girl with a boy first, then for the remaining n-1 girls and n-1 boys, the condition still holds.

Sufficient: If the condition holds, then can find a perfect match

Necessary: If there is a perfect match, then the condition must hold.

12/24

Maximum Subsequence Sum Problem

• Given (possibly negative) integers A1, A2,…, AN,

find the maximum value of over all i, j

• For input –2, 11, -4, 13, -5, -2, the answer is 20

k=i j Ak

13/24

Algorithm 1: Brute-force method

• Given n integers A1, A2,…, AN, how many subsequences can you form?

• For example: 1, 2, 3, 4– The possible sums include:

• 1, 1+2, 1+2+3, 1+2+3+4• 2, 2+3, 2+3+4• 3, 3+4• 4

– Find the maximum in the above sums

– An O(N3) solution

14/24

Algorithm 2: A Bit Clever Algorithm

• By noting

= Aj +

• We can reuse partial computation in previous steps

k=i j Ak k=i j-1 Ak

15/24

Algorithm 2: A Bit Clever Algorithm

• Sum (i,j) = 0 for all i, j

• For i = 0 to n

For j = i to n

Sum (i,j) Sum (i, j-1) + Aj

end

end• An O(N2) algorithm

16/24

Algorithm 3: Divide and Conquer

• Divide Part:– Split the problem into two roughly equal

subproblems, which are then solved recursively.

• Conquer Part:– Patching together the two solutions of the

subproblems with small amount of additional work.

17/24

Algorithm 3: Divide and Conquer

• Maximum subsequence sum problem– Divide the sequence into two equal parts– The maximum subsequence sum can be

found in one of three places:• Entirely in the left half of the input• Entirely in the right half of the input• Crosses the middle and in both halves

18/24

Algorithm 3: Divide and Conquer

• In first half: 6

First Half Second Half

4 -3 5 -8 -1 2 6 -2

19/24

Algorithm 3: Divide and Conquer

• In second half: 8

First Half Second Half

4 -3 5 -8 -1 2 6 -2

20/24

Algorithm 3: Divide and Conquer

• Crosses middle: 11

First Half Second Half

4 -3 5 -2 -1 2 6 -2

21/24

Algorithm 3: Divide and Conquer

• When the maximum subsequence sum crosses the middle

• T(n) = 2T(n/2) + 2n/2• An O(n log n) algorithm

The maximum subsequence sum from the end

22/24

Algorithm 4: The most clever one

• An improvement over algorithm 2

• Clever observations:– No negative subsequence can possibly be a

prefix of the optimal subsequence

23/24

Algorithm 4: The most clever one

• Use induction. Keep two sums at each iteration and update them based on the three conditions below. Note that maxendSUM is the optimal subsequence sum from the right end

If maxendSUMn-1 + An 0, then maxendSUMn = 0

If maxendSUMn-1 + An > 0, then maxendSUMn maxendSUMn-1 + An

If maxendSUMn-1 + An > maxSUMn-1, then maxSUMn = maxendSUMn-1 + An

Else maxSUMn = maxendSUMn-1

maxSUMn-1 maxendSUMn-1

1 n - 1

maxSUMn-1 maxendSUMn-1 + An

1 n

When would adding An change the maxSUMn ?

24/24

Algorithm 4: The most clever one

• An O(N) algorithm– It takes constant time to update the two sums at each

iteration

• Example: Sequence: -2, 11, -4, 13, -5, -2Initially, maxendSUM0 = 0, maxSUM0 = 0i = 1: (a) maxendSUM1 = 0i = 2: (b) maxendSUM2 = 11, (c) maxSUM2 = 11i = 3: (b) maxendSUM3 = 7, (c) maxSUM3 = 11i = 4: (b) maxendSUM4 = 20, (c) maxSUM4 = 20i = 5: (b) maxendSUM5 = 15, (c ) maxSUM5 = 20i = 6: (b) maxendSUM6 = 13, (c ) maxSUM6 = 20

25/24

Common Computational Models

• Discrete algorithm– Probabilistic, approximation, on-line, randomized

• Non-linear programming (numerical)• Statistical

– Regression– Machine learning

• Neural net, SVM, Hidden Markov Model, Maximum entropy, Conditional random fields,

– Evolutionary• Genetic algorithm, particle swarm

• Areas: NLP, ASR, IR, IE, DM