Unit 1. Sorting and Divide and Conquer

Lecture 1 Introduction to Algorithm and Sorting

What is an Algorithm?

• An algorithm is a computational procedure that takes some value, or a set of values, as input and produces some values, or a set of values, as output.

• So, it is a sequence of computational steps that transform the input into the output.

• Correction: sequence -> combination

Sequential and Parallel

• In sequential computation, an algorithm is a sequence of computational steps.

• However, it is not true in parallel computation.

• In this course, we study only algorithms in sequential computation.

Algorithms for Sorting

.'''such that sequence

input of }',...,','{n permutatioA :Output

}.,...,,{ numbers of sequenceA :Input

21

21

21

n

n

n

aaa

aaa

aaan

Insertion Sort, Merge Sort

6. 5, 4, 3, 2, 1, :Output

3. 1, 6, 4, 2, 5, :Input

e.g.,

Efficiency

• Running time from receiving the input to producing the output.

)log(

)( 2

nnO

nOInsert Sort

Merge Sort

Running time

.for )()(0

such that

0 and 0 constantsexist there

means ))(()(

0

0

nnncgnf

nc

ngOnf

function!

theofgrowth theisabout carereally what weTherefore,

.log

large,ly sufficient is When

bigger? is who,log and

22

1

22

1

nncnc

n

nncnc

keyiA

ii

iAiA

keyiAi

ji

jAkey

Alengthj

]1[

1

][]1[ do

][ and 0 while

1

][ do

][ to2for

Insertion SortAarray input

for.-end

;]1[

while;-end

;1

];[]1[

begin do

][ and 0 while

;1

];[

begin do

][ to2for

keyiA

ii

iAiA

keyiAi

ji

jAkey

Alengthj

3 1, ,6 5, 4, 2,

3 1, 6, 5, ,4 2,

3 1, 6, ,4 5, 2,

3 1, 6, 4, 5, ,2

3 1, 6, 4, ,2 5,

�

�

�

�

�

6 5, 4, ,3 2, 1,

6 5, ,3 4, 2, 1,

6 ,3 5, 4, 2, 1,

3 6, 5, 4, 2, 1,

�

�

�

�

3 6, 5, 4, 2, ,1

3 6, 5, 4, ,1 2,

3 6, 5, ,1 4, 2,

3 6, ,1 5, 4, 2,

3 ,1 6, 5, 4, 2,

�

�

�

�

�key

In the key outside of array.

How to calculate running time?

• Each “line” of pseudocode requires a constant time. (In RAM model)

for.-end

;]1[

while;-end

;1

];[]1[

begin do

][ and 0 ile wh

;1

];[

begin do

][ to2for

keyiA

ii

iAiA

keyiAi

ji

jAkey

Alengthj

This loop runs at mostj-1 times and each timeruns at most 3 lines.

This loop runs n-1 timesand each time runs at most4+3(j-1) lines.

2

)2)(1(31

))1(34()(2

nnn

jnTn

j

Remark on Running Time

• Running time is a function of input size.• In Turing machine model, the input size

of sorting is

ime.constant tin not runs "][" Hence,

logloglog 22212

keyiA

aaa n

Divide and Conquer

• Divide the problem into subproblems.

• Conquer the subproblems by solving them recursively.

• Combine the solutions to subproblems into the solution for original problem.

Merge Sort

end

);,1,(Sort-Merge

begin

ProgramMain

]....[subarray in elements sortswhich

procedure a be ),,(Sort-MergeLet

nA

rpA

rpA

Procedure

then.-end

);,,,(Merge

);,1,(Sort-Merge

);,,(Sort-Merge

;2/)(begin then

if

),,(Sort-Merge

rqpA

rqA

qpA

rpq

rp

rpA

];[][ do

to1for

];1[][ do

to1for

1];..1[ and 1]..1[array create

;

;1

),,,(Merge

2

1

21

2

1

jqAjR

nj

ipAiL

ni

nRnL

qrn

pqn

rqpA

;1 and ][][ else

1 and ][][hen t

][][ if do

tofor

;1

;1

;)1( ;)1( 21

jjjRkA

iiiLkA

jRiA

rpk

j

i

nRnL

5, 4, 3, 2, 1, :

; 8, ,6 4, 1, : ; 7, ,5 3, 2, :

4, 3, 2, 1, :

; 8, 6, ,4 1, : ; 7, ,5 3, 2, :

3, 2, 1, :

; 8, 6, ,4 1, : ; 7, 5, ,3 2, :

2, 1, :

; 8, 6, ,4 1, : ; 7, 5, 3, ,2 :

,1:

; 8, 6, 4, ,1 : ; 7, 5, 3, ,2 :

A

RL

A

RL

A

RL

A

RL

A

RL

Example

; 8, 6, 4, 1, : ; 7, 5, 3, 2, :

8, 7, 6, 5, 4, 3, 2, 1, :

; ,8 6, 4, 1, : ; 7, 5, 3, 2, :

7, 6, 5, 4, 3, 2, 1, :

; ,8 6, 4, 1, : ; ,7 5, 3, 2, :

6, 5, 4, 3, 2, ,1:

; 8, ,6 4, 1, : ; ,7 5, 3, 2, :

RL

A

RL

A

RL

A

RL

Procedure

then.-end

);,,,(Merge

);,1,(Sort-Merge

);,,(Sort-Merge

;2/)(begin then

if

),,(Sort-Merge

rqpA

rqA

qpA

rpq

rp

rpA

)(nT

)(n

)()2/(2)( nnTnT

)2/(nT

)2/(nT

))(()())(()(

.for )()()(0

such that

0 and 0 constantsexist there

means ))(()(

012

021

nfngngnf

nnngcnfngc

ncc

ngnf

Symmetry

How to Solve Recurrences

• Substitution method.

• Recursion-tree method.

• Master method.

Substitution Method

• Guess the form of the solution.

• Use math induction to find the constants and also show the solution works.

)).)(,max( (Take

lg

.)(lg

)2/lg()2/(2)(Then

).2/lg()2/()2/( Assume )2(

).()( ,For (1)

:Induction

.,0 constants somefor

)2/(2)()2/(2 :Note

)lg()( i.e., ,lg)( :Guess

02

2

2

00

012

21

nTcc

ncn

nccncn

ncnncnT

nncnT

nTnTnn

nncc

ncnTnTncnT

nnOnTncnnT

)).)(,2max( (Take

largely sufficientfor lg

largely sufficientfor )5.0(lg

)/lg21)(lg12/(2

2/lg2/2)(Then

.2/lg2/)2/( Assume )2(

).()( ,For (1)

:Induction

.,0 constants somefor

)2/(2)()2/(2 :Note

)lg()( i.e., ,lg)( :Guess

02

2

2

2

00

012

21

nTcc

nncn

nnccncn

ncnennc

ncnncnT

nncnT

nTnTnn

nncc

ncnTnTncnT

nnOnTncnnT

nen

exen

nn

nn

xn

/)lg2(1lg

)1:(note lg1lg

)/21lg()2/lg(

)12/lg(2/lg

/2

Remark

).lg()(then

)()2/(2)( If

nnOnT

nOnTnT

Recursion Tree)2( kT

)1(T

)2( 2kT

)2( 1kT)2( 1kT

)2( 2kT)2( 2kT )2( 2kT

)1(T)1(T)1(T

kc 22

kk cc 222 21

2

kk kcT 2)2( 2

Master Theorem

)).(()( then , largely sufficient

and 1constant afor )()/( if and

,0constant somefor )()( If

).lg()( then ),()( If

).()(then

,0constant somefor )()( If

./or / means / and ,1,1

constants where)()/()(Let

log

loglog

log

log

nfnTn

cncfbnaf

nnf

nnnTnnf

nnT

nOnf

bnbnbnba

nfbnaTnT

a

aa

a

a

b

bb

b

b

))(()())(()(

.for )()(0

such that

0 and 0 constantsexist there

means ))(()(

0

0

nfOngngnf

nnnfncg

nc

ngnf

Relationship

What we learnt in this lecture?

• How to calculate running time.

• How to solve recurrences.

• Insertion sort and Merge sort.

• Divide and conquer

• Lecture Notes give you key points in each lecture.

• You must read textbook after lectures in order to study well.