ADVANCED ALGORITHMS LECTURE 3: DIVIDE & CONQUER

�1

LECTURE 3

ANNOUNCEMENTS

▸ Homework 0 grading

▸ Homework 1 posted; due on Monday Sep 10, 11:59 PM

▸ Start early! (Post comments/questions on canvas…)

▸ Unless immediate, argue why algorithm is correct

▸ Three steps — algorithm description, proof of correctness, complexity analysis

�2

LECTURE 3

LAST CLASS

�3

▸ Basics of data structures

▸ What data? What operations supported? Time/space complexity

▸ Prefix trees (aka tries)

▸ Connectivity in graphs via depth-first-search (DFS)

▸ Proved using induction on the length of the path

LECTURE 3

TODAY’S CLASS

▸ Divide and conquer

▸ Analysis: recurrences and how to solve them

�4

LECTURE 3

DIVIDE AND CONQUER

�5

LECTURE 3

DIVIDE AND CONQUER

�6

PROBLEM INSTANCE

Divide step

Solve sub-problems

FULL SOLUTIONCombine/“conquer”

LECTURE 3

WHEN AND WHY?

�7

▸ Problem “cleanly” divides into not-too-many sub-problems

▸ Solutions easily (efficiently) combined

▸ Leads to simple, efficient solutions

▸ Analysis easier — correctness by induction, complexity by recurrences

Sometimes difficult to see how D&C can be used…

LECTURE 3

BASIC: SORTING

�8

Basic question: given unsorted array A[0], …, A[n-1], re-order elements in increasing order (produce new array)

▸ Divide step?

▸ Conquer step

35 4 7 98 5 6 10

Find smallestelement

IMAM Aln DT

t 717 Fasort recursively

LECTURE 3

MERGING SORTED LISTS

�9

CyBY's BYE Bz

qfaoi

j is the smallest ett inan

B E Bik's EB the entire arrayu

Shift the array 134 tothe left repeated

qeeach taking time

7 8 9 nk log k5 6 10

LECTURE 3

OVERALL ALGORITHM — MERGE SORT

�10

▸ Divide A[] into k pieces each of size [n/k]

▸ Sort each one recursively

▸ Use merge procedure to create new sorted list

What is the overall time complexity?

Tfn k TLEdnt

rk arrays each takesFCI to sor

01

Let Tcm denote runningtime of procedureon

an instance of size m

LECTURE 3

BASIC “RECURRENCE”

�11

T(n) = k ⋅ T(n /k) + kn + n

T(n) = 2T(n /2) + 3nSetting k=2

Minor: is n divisible by k?

tlnl 4 TT t5n

Goyal come up witha closed form solution

I expticitfnotnik

THE n

LECTURE 3

ASIDE: HOW TO SOLVE RECURRENCES?

�12

▸ No silver bullet!

▸ Guess, prove by induction — sometimes proof gives “feedback”

▸ “Plug and chug” [Lehman, Leighton]

▸ Recurrence treekeep pluggingback

into equation see a

pattern pronebyinduction

LECTURE 3

PLUG AND CHUG

�13

T(n) = 2T(n /2) + 3n

TH I

Ent 2T ET n 3 n t 213 ng t 2T F 3nt3n t 4517

3nt3nt 4 3 1 2T Ig nt3nt3y1851

r steps

r 3N IT Ir 7 3mlogin n 1

Base case Lr t rzlogen

LECTURE 3

RECURRENCE TREE

�14

T nl Lt3n1 n

eternity tinitiation we do

Tt HzTs T u totwafaexha.sn

IT 11 c 3 nn

terms I3N logan n th

LECTURE 3

EXAMPLE 2: MULTIPLYING INTEGERS

�15

Question: given two n digit numbers, how efficiently can we multiply them?

A Az anb

gRftifunenians HTIsee.ee eini I i t

LECTURE 3

DIVIDE AND CONQUER

�16BA

a.az anxbibnLbn500 5 102 X Y Z W

A X 0000,20 4 X ion Y

B Z lo w

A B x t x.I.LII.IoIi 7Oln 0cm

conquer stepis 0h Tfn _4TH101in

LECTURE 3

RUNNING TIME?

�17

TIM 4 TIE c n

11hTfn Acn 4T I Hai

Cnt 41C Z 4T fD Cn then 47T

Cnt 2cn t life It 4T 8i

I negynentientta cn 4ttfITH i He

LECTURE 3

CAN WE DO BETTER?

�18

Reason to hope: we only need to find three quantities:

XZ, (XW + YZ), YW

21

LECTURE 3

CAN WE DO BETTER?

�19

Reason to hope: we only need to find three quantities:

XZ, (XW + YZ), YW

Kartsuba’s observation:

XW + YZ = (X + Y )(W + Z) − XZ − YW

LECTURE 3

OVERALL RUNNING TIME

�20

LECTURE 3

EVEN BETTER?

�21

▸ Can we do linear time?

▸ HW1 — will see something approaching linear

LECTURE 3

COMPUTING THE MEDIAN

�22

Question: given an unsorted list A[0], A[1], …, A[n-1], find the (n/2)th smallest element

Divide and conquer?

LECTURE 3

KEY INSIGHT: “APPROXIMATE MEDIAN” HELPS!

�23

LECTURE 3

MEDIAN OF MEDIANS IS APPROXIMATE MEDIAN!

�24