42
Data Structures for Data Structures for Media Media Introduction Introduction

Data Structures for Media Introduction. Dr. Minming Li Department of Computer Science Room :Y6426 Phone :27889538 Email: [email protected], [email protected]

Embed Size (px)

Citation preview

Page 1: Data Structures for Media Introduction. Dr. Minming Li Department of Computer Science Room :Y6426 Phone :27889538 Email: minmli@cs.cityu.edu.hk, mli000@cityu.edu.hk

Data Structures for MediaData Structures for MediaIntroductionIntroduction

Page 2: Data Structures for Media Introduction. Dr. Minming Li Department of Computer Science Room :Y6426 Phone :27889538 Email: minmli@cs.cityu.edu.hk, mli000@cityu.edu.hk

Dr. Minming Li

Department of Computer Science

Room Room :: Y6426Y6426

Phone Phone :: 2788953827889538

EmailEmail : : [email protected]@cs.cityu.edu.hk, ,

[email protected]@cityu.edu.hk

Mail boxMail box : : P16(P16(Outside CS General Office, Yellow zone, 6/FOutside CS General Office, Yellow zone, 6/F))

LecturerLecturer

Page 3: Data Structures for Media Introduction. Dr. Minming Li Department of Computer Science Room :Y6426 Phone :27889538 Email: minmli@cs.cityu.edu.hk, mli000@cityu.edu.hk

What you can find at the course webpage:What you can find at the course webpage:http://www.cs.cityu.edu.hk/~minmli/course.htmhttp://www.cs.cityu.edu.hk/~minmli/course.htm

Lecture SlidesLecture Slides

Ready at least 24 hours before the lectureReady at least 24 hours before the lecture

Tutorial ExercisesTutorial Exercises

Selected questions are to be done during tutorials or Selected questions are to be done during tutorials or as homeworkas homework

Assignments and solutionsAssignments and solutions

Announcement from the teacherAnnouncement from the teacher

Important ones will also be distributed through email Important ones will also be distributed through email

Tutorial Class Room changed to MMW-2478Tutorial Class Room changed to MMW-2478

Q&AQ&A

Course Web PageCourse Web Page

Page 4: Data Structures for Media Introduction. Dr. Minming Li Department of Computer Science Room :Y6426 Phone :27889538 Email: minmli@cs.cityu.edu.hk, mli000@cityu.edu.hk

Weiss M.Weiss M. Data Structures & Algorithm AnData Structures & Algorithm Analysis in C++.alysis in C++. 3rd Ed. 3rd Ed. Addison WesleyAddison Wesley (19 (1999)99)

Hanan Samet Hanan Samet The Design and Analysis of The Design and Analysis of Spatial Data StructuresSpatial Data Structures Addison WesleyAddison Wesley (1989)(1989)

Harvey M. Deitel, Paul J. DeitelHarvey M. Deitel, Paul J. Deitel. . Visual C+Visual C++.NET :How to Program +.NET :How to Program Prentice HallPrentice Hall (2 (2003)003)

Reference BooksReference Books

Page 5: Data Structures for Media Introduction. Dr. Minming Li Department of Computer Science Room :Y6426 Phone :27889538 Email: minmli@cs.cityu.edu.hk, mli000@cityu.edu.hk

Assessment PatternAssessment Pattern

Compulsory: exam mark >= 30Exam (70%)

2 Tests (7% +8%)

2 Assignments (6% + 9%)

Tutorial exercises (0%)

Page 6: Data Structures for Media Introduction. Dr. Minming Li Department of Computer Science Room :Y6426 Phone :27889538 Email: minmli@cs.cityu.edu.hk, mli000@cityu.edu.hk

Better not read the textbook ahead of timeBetter not read the textbook ahead of time Try to keep up with the lectureTry to keep up with the lecture Read the materials carefully after the lectureRead the materials carefully after the lecture Do the assignments on your ownDo the assignments on your own

You may discuss with each otherYou may discuss with each other You may study materials available on internetYou may study materials available on internet You may refer to any bookYou may refer to any book But the details should be entirely your workBut the details should be entirely your work

Respond in time if you have any suggestions Respond in time if you have any suggestions to the course or problems with the courseto the course or problems with the course

How to learn this course?How to learn this course?

Page 7: Data Structures for Media Introduction. Dr. Minming Li Department of Computer Science Room :Y6426 Phone :27889538 Email: minmli@cs.cityu.edu.hk, mli000@cityu.edu.hk

OVERVIEW OF CONTENTSOVERVIEW OF CONTENTS

Page 8: Data Structures for Media Introduction. Dr. Minming Li Department of Computer Science Room :Y6426 Phone :27889538 Email: minmli@cs.cityu.edu.hk, mli000@cityu.edu.hk

Why Data Structures?Why Data Structures? What to do when we have a large What to do when we have a large

amount of data to deal with?amount of data to deal with? Organize it in ways easy to understandOrganize it in ways easy to understand Space efficiencySpace efficiency Time efficiencyTime efficiency Easy to display and transformEasy to display and transform

OverviewOverview

Page 9: Data Structures for Media Introduction. Dr. Minming Li Department of Computer Science Room :Y6426 Phone :27889538 Email: minmli@cs.cityu.edu.hk, mli000@cityu.edu.hk

Linked Linked ListList

TreeTree StackStack QueueQueue HashingHashing

OverviewOverview

Data Data DataDataLabel

Page 10: Data Structures for Media Introduction. Dr. Minming Li Department of Computer Science Room :Y6426 Phone :27889538 Email: minmli@cs.cityu.edu.hk, mli000@cityu.edu.hk

OverviewOverview PART IPART I Program ComplexitiesProgram Complexities Abstract Data TypesAbstract Data Types

Linked listsLinked lists TreesTrees StacksStacks QueuesQueues HeapsHeaps Hash tablesHash tables

Page 11: Data Structures for Media Introduction. Dr. Minming Li Department of Computer Science Room :Y6426 Phone :27889538 Email: minmli@cs.cityu.edu.hk, mli000@cityu.edu.hk

OverviewOverview PART IIPART II Vectors and BitmapsVectors and Bitmaps

Quadtrees and OctreesQuadtrees and Octrees The handling of 2D and 3D dataThe handling of 2D and 3D data

Geometric StructuresGeometric Structures Spatial LayoutSpatial Layout Shape and AttributesShape and Attributes Connectivity of ComponentsConnectivity of Components

Page 12: Data Structures for Media Introduction. Dr. Minming Li Department of Computer Science Room :Y6426 Phone :27889538 Email: minmli@cs.cityu.edu.hk, mli000@cityu.edu.hk

Data Structures for MediaData Structures for MediaProgram ComplexitiesProgram Complexities

Page 13: Data Structures for Media Introduction. Dr. Minming Li Department of Computer Science Room :Y6426 Phone :27889538 Email: minmli@cs.cityu.edu.hk, mli000@cityu.edu.hk

What is an algorithm?What is an algorithm? A sequence of elementary computational steps that A sequence of elementary computational steps that transform the transform the inputinput into the into the outputoutput

What for?What for?A tool for solving well-specified A tool for solving well-specified computational problemscomputational problems, , e.g., Sorting, Matrix Multiplicatione.g., Sorting, Matrix Multiplication

What do we need to do with an algorithm?What do we need to do with an algorithm? Correctness Proof:Correctness Proof:

for every input instance, it halts with the correct outputfor every input instance, it halts with the correct output Performance Analysis:Performance Analysis:

How does the algorithm behave as the problem size gets How does the algorithm behave as the problem size gets largelarge

both in both in running timerunning time and and storage requirementstorage requirement

AlgorithmsAlgorithms

Page 14: Data Structures for Media Introduction. Dr. Minming Li Department of Computer Science Room :Y6426 Phone :27889538 Email: minmli@cs.cityu.edu.hk, mli000@cityu.edu.hk

Input : Input : <a<a00, a, a11, … , a, … , an-1n-1>>

Output: Output: A permutation (re-ordering) <a’A permutation (re-ordering) <a’00, a’, a’11, … , a’, … , a’n-1n-1> > ofof

the input sequence such that a’the input sequence such that a’0 0 a’ a’1 1 … … a’ a’n-1n-1

Example:Example:

<22, <22, 5151, 34, , 34, 4444, 67, , 67, 1111> => <> => <1111, 22, 34, , 22, 34, 4444, , 5151, 67>, 67>

A Sorting ProblemA Sorting Problem

Page 15: Data Structures for Media Introduction. Dr. Minming Li Department of Computer Science Room :Y6426 Phone :27889538 Email: minmli@cs.cityu.edu.hk, mli000@cityu.edu.hk

55, , 33, 1, 2, 6, 4, 1, 2, 6, 4

33, 5, 5, , 11, 2, 6, 4, 2, 6, 4

11, 3, 5, 3, 5, , 22, 6, 4, 6, 4

1, 1, 22, 3, 5, 3, 5, , 66, 4, 4

1, 2, 3, 5, 1, 2, 3, 5, 66, , 44

1, 2, 3, 1, 2, 3, 44, 5, 6, 5, 6

Insertion SortInsertion SortNote that when we are dealing with kth number, the first k-1 numbers are alrea

dy sorted

Page 16: Data Structures for Media Introduction. Dr. Minming Li Department of Computer Science Room :Y6426 Phone :27889538 Email: minmli@cs.cityu.edu.hk, mli000@cityu.edu.hk

To sort A[0,1,…,n-1] in placeTo sort A[0,1,…,n-1] in place Steps:Steps:

Pick element A[j]Pick element A[j] Move A[j-1,…,0] to the right unMove A[j-1,…,0] to the right un

til proper position for A[j] is foutil proper position for A[j] is foundnd

Insertion SortInsertion Sort

Currently sorted part

Currently unsorted

part

j0 … j+1 . . 11 3 5 3 5 22 6 4 6 4

1 3 1 3 55 5 6 4 5 6 4

1 1 33 3 5 6 4 3 5 6 4

1 1 22 3 5 6 4 3 5 6 4

55, , 33, 1, 2, 6, 4, 1, 2, 6, 4

33, 5, 5, , 11, 2, 6, 4, 2, 6, 4

11, 3, 5, 3, 5, , 22, 6, 4, 6, 4

1, 1, 22, 3, 5, 3, 5, , 66, 4, 4

1, 2, 3, 5, 1, 2, 3, 5, 66, , 44

1, 2, 3, 1, 2, 3, 44, 5, 6, 5, 6 Example11 3 5 3 5 22 6 4 6 4

Page 17: Data Structures for Media Introduction. Dr. Minming Li Department of Computer Science Room :Y6426 Phone :27889538 Email: minmli@cs.cityu.edu.hk, mli000@cityu.edu.hk

Insertion SortInsertion SortInsertion-Sort (A)Insertion-Sort (A) 1. for j=1 to n-11. for j=1 to n-1 2. 2. key=A[j]key=A[j] 3. 3. i=j-1i=j-1 4. 4. while i>=0 and A[i]>keywhile i>=0 and A[i]>key 5. 5. A[i+1]=A[i]A[i+1]=A[i] 6. 6. i=i-1i=i-1

7. 7. A[i+1]=keyA[i+1]=key

j=1j=1 5 5 33 1 2 6 4 1 2 6 4

j=2j=2 33 5 5 11 2 6 4 2 6 4

j=3j=3 11 3 5 3 5 22 6 4 6 4

j=4j=4 1 1 22 3 5 3 5 66 4 4

j=5j=5 1 2 3 5 1 2 3 5 66 44

1 2 3 1 2 3 44 5 6 5 6

A[0] A[1] A[2] A[3] A[4] A[5]

11 3 5 3 5 22 6 4 6 4

1 3 1 3 55 5 6 4 5 6 4

1 1 33 3 5 6 4 3 5 6 4

1 1 22 3 5 6 4 3 5 6 4

j=3

Page 18: Data Structures for Media Introduction. Dr. Minming Li Department of Computer Science Room :Y6426 Phone :27889538 Email: minmli@cs.cityu.edu.hk, mli000@cityu.edu.hk

We only consider algorithms with loopsWe only consider algorithms with loops Find a property as Find a property as loop invariantloop invariant

How to show something is loop invariant?How to show something is loop invariant? InitializationInitialization: :

It is true prior to the first iteration of the loopIt is true prior to the first iteration of the loop MaintenanceMaintenance: :

If it is true before an iteration, it remains true If it is true before an iteration, it remains true before the next iterationbefore the next iteration

TerminationTermination::

When the loop terminates, the invariant gives When the loop terminates, the invariant gives a useful property that helps to show the a useful property that helps to show the algorithm is correctalgorithm is correct

Correctness of AlgorithmCorrectness of Algorithm

Page 19: Data Structures for Media Introduction. Dr. Minming Li Department of Computer Science Room :Y6426 Phone :27889538 Email: minmli@cs.cityu.edu.hk, mli000@cityu.edu.hk

Correctness of Insertion SortCorrectness of Insertion Sort loop invariantloop invariant

At start of each iteration of At start of each iteration of forfor loop, A[0..j-1] consists of the el loop, A[0..j-1] consists of the elements originally in A[0..j-1] but in sorted orderements originally in A[0..j-1] but in sorted order

InitializationInitialization Before the first iteration, j=1. Before the first iteration, j=1.

=> A[0 .. j-1] contains only A[0]. => A[0 .. j-1] contains only A[0]. => Loop invariant holds prior to the first iteration.=> Loop invariant holds prior to the first iteration.

MaintenanceMaintenance In each iteration, the algorithm moves A[j-1],A[j-2],A[j-3] .. to tIn each iteration, the algorithm moves A[j-1],A[j-2],A[j-3] .. to t

he right until the proper position for A[j] is found. he right until the proper position for A[j] is found. Then A[j] is inserted. Then A[j] is inserted. => if the loop invariant is true before an iteration, it remains tr=> if the loop invariant is true before an iteration, it remains true before next iteration.ue before next iteration.

TerminationTermination The outer loop ends with j=n. The outer loop ends with j=n.

Substituting n for j in the loop invariant, we get “A[______] cSubstituting n for j in the loop invariant, we get “A[______] consists of the n sorted elements.”onsists of the n sorted elements.”

Page 20: Data Structures for Media Introduction. Dr. Minming Li Department of Computer Science Room :Y6426 Phone :27889538 Email: minmli@cs.cityu.edu.hk, mli000@cityu.edu.hk

The running time T(n) = c1*n+c2*(n-1)+c3*(n-1)+c4*(j=1..n-1 (tj+1))+c5*(j=1..n-1 tj)+c6*(j=1..n-1 tj)+c7*(n-1)

tj = no. of times that line 5,6 are executed, for each j.

Insertion-Sort(A)

1 for j = 1 to n-1

2 key = A[j]

3 i = j-1

4 while i >= 0 and A[i] > key

5 A[i+1] = A[i]

6 i = i - 1

7 A[i+1] = key

n

n-1

n-1

j=1..n-1 (tj+1)

j=1..n-1 tj

j=1..n-1 tj

n-1

c1

c2

c3

c4

c5

c6

c7

timesCost

c1, c2, .. = running time for executing line 1, line 2, etc.

Running time of Insertion Running time of Insertion SortSort

Page 21: Data Structures for Media Introduction. Dr. Minming Li Department of Computer Science Room :Y6426 Phone :27889538 Email: minmli@cs.cityu.edu.hk, mli000@cityu.edu.hk

T(n) = c1*n+c2*(n-1)+c3*(n-1)+c4*(j=1..n-1 (tj+1))+c5*(j=1..n-1 tj)+c6*(j=1..n-1 tj)+c7*(n-1)

Worse case:

Reversely sorted

inner loop body executed for all previous elements.

tj=j.

T(n) = c1*n+c2*(n-1)+c3*(n-1)+c4*(j=1..n-1 (j+1))+c5*(j=1..n-1 j)+c6*(j=1..n-1 j)+c7*(n-1)

T(n) = An2+Bn+C

T(n) = c1*n+c2*(n-1)+c3*(n-1)+c4*(j=1..n-1 (j+1))+c5*(j=1..n-1 j)+c6*(j=1..n-1 j)+c7*(n-1)

Note: j=1..n-1 j = n(n-1)/2 j=1..n-

1 (j+1) = (n+2)(n-1)/2

Analyzing Insertion SortAnalyzing Insertion Sort

Page 22: Data Structures for Media Introduction. Dr. Minming Li Department of Computer Science Room :Y6426 Phone :27889538 Email: minmli@cs.cityu.edu.hk, mli000@cityu.edu.hk

T(n)=T(n)=cc11*n+c*n+c22*(n-1)+c*(n-1)+c33*(n-1)+c*(n-1)+c44*(*(j=1..n-1j=1..n-1 (t (tjj+1))+c+1))+c55*(*(j=1..n-1j=1..n-1 t tjj)+c)+c66*(*(j=1..n-1j=1..n-1 t tjj)+c)+c77*(n-1)*(n-1)

Worst caseWorst case Reverse sorted Reverse sorted inner loop body executed for all previ inner loop body executed for all previous elements. So, tous elements. So, tjj=j.=j.

T(n) is quadratic (square): T(n)=AnT(n) is quadratic (square): T(n)=An22+Bn+C+Bn+C

Average caseAverage case Half elements in A[0..j-1] are less than A[j]. So, tHalf elements in A[0..j-1] are less than A[j]. So, t jj = j/2 = j/2 T(n) is also quadratic: T(n)=AnT(n) is also quadratic: T(n)=An22+Bn+C+Bn+C

Best caseBest case Already sorted Already sorted inner loop body never executed. So, t inner loop body never executed. So, t jj

=0.=0.

T(n) is linear: T(n)=An+BT(n) is linear: T(n)=An+B

Analyzing Insertion SortAnalyzing Insertion Sort

Page 23: Data Structures for Media Introduction. Dr. Minming Li Department of Computer Science Room :Y6426 Phone :27889538 Email: minmli@cs.cityu.edu.hk, mli000@cityu.edu.hk

(Usually) Worst case Analysis:(Usually) Worst case Analysis:T(n) = max time on any input of size nT(n) = max time on any input of size n

Knowing it gives us a guarantee about the upper Knowing it gives us a guarantee about the upper bound.bound.

In some cases, worst case occurs fairly oftenIn some cases, worst case occurs fairly often

((Sometimes) Average case Analysis:Sometimes) Average case Analysis:T(n) = average time over all inputs of size nT(n) = average time over all inputs of size n

Average case is often as bad as worst case.Average case is often as bad as worst case.There really exists “good” exampleThere really exists “good” example

(Rarely) Best case Analysis:(Rarely) Best case Analysis:Cheat with slow algorithm that works fast on some input.Cheat with slow algorithm that works fast on some input.Good only for showing bad lower bound.Good only for showing bad lower bound.(New) Smoothed Analysis(New) Smoothed AnalysisAverage in the local region instead of all inputsAverage in the local region instead of all inputs

Kinds of AnalysisKinds of Analysis

Page 24: Data Structures for Media Introduction. Dr. Minming Li Department of Computer Science Room :Y6426 Phone :27889538 Email: minmli@cs.cityu.edu.hk, mli000@cityu.edu.hk

Kinds of AnalysisKinds of Analysis

Worst Case: maximum valueWorst Case: maximum value Average Case: average valueAverage Case: average value Best Case: minimum valueBest Case: minimum value

Worst case

Average caseBest case

0 1 2 3 4 n

Page 25: Data Structures for Media Introduction. Dr. Minming Li Department of Computer Science Room :Y6426 Phone :27889538 Email: minmli@cs.cityu.edu.hk, mli000@cityu.edu.hk

Order of GrowthOrder of Growth

f(n)f(n) n=20n=20 n=40n=40 n=60n=60

LogLog22nn

Sqrt(n)Sqrt(n)

nn

n logn log22nn

nn22

nn44 12.96 sec

22nn

n!n!

5.32 * 10-6sec4.32 * 10-6sec 5.91 * 10-6sec

6.32 * 10-6sec4.47 * 10-6sec 7.75 * 10-6sec

40 * 10-6sec20 * 10-6sec 60 * 10-6sec

213 * 10-6sec86 * 10-6sec 354 * 10-6sec

1600 * 10-6sec400 * 10-6sec 3600 * 10-6sec

2.56 sec0.16 sec

12.73 days1.05 sec 36571 years

2.56 * 1034 years77147 years 2.64 * 1068 years

Examples:Examples:Running time of algorithm in microseconds(in term of data size n)

Algorithm A

Algorithm B

Algorithm C

Algorithm D

Algorithm E

Algorithm F

Algorithm G

Algorithm H

Page 26: Data Structures for Media Introduction. Dr. Minming Li Department of Computer Science Room :Y6426 Phone :27889538 Email: minmli@cs.cityu.edu.hk, mli000@cityu.edu.hk

11

mic

rose

con

ds

f(n)log2n

Sqrt n

n

nlog2n

n2

n4

2n

n!0

10000

20000

30000

40000

50000

60000

70000

80000

90000

1 3 5 7 9 13 15 17 19

n

Assume: an algorithm can solve a problem of size n in f(n) Assume: an algorithm can solve a problem of size n in f(n) microseconds (10microseconds (10-6-6 seconds). seconds).

Note: for example,Note: for example,For all f(n) in For all f(n) in (n(n44), ), the shapes of their curves are the shapes of their curves are nearlynearly the same as f(n)=n the same as f(n)=n44..

Order of GrowthOrder of Growth

Page 27: Data Structures for Media Introduction. Dr. Minming Li Department of Computer Science Room :Y6426 Phone :27889538 Email: minmli@cs.cityu.edu.hk, mli000@cityu.edu.hk

Asymptotic Tight Bound: f(n)= (g(n)) Intuitively like “=” f(n) grows as fast as g(n)

Asymptotic Upper Bound: f(n)= O (g(n)) Intuitively like “≤” f(n) grows not faster than g(n)

Asymptotic Lower Bound Ω f(n)= Ω(g(n)) Intuitively like “≥” f(n) grows not slower than g(n)

Asymptotic NotationAsymptotic Notation

Formal formulations:

(We’ll not go into details of these equations.)

O(g(n))O(g(n)) = { = {f(n): there exist positive f(n): there exist positive constants c and nconstants c and n00 such that 0 such that 0 f(n) f(n) cg(n) cg(n) for all n for all n n n0 0 }}

f(n)

cg(n)

n0 f(n) is O(g(n))

f(n)

c1g(n)

c2g(n)

n0 f(n) is (g(n))

(g(n))(g(n)) = { = {f(n): there exist positive f(n): there exist positive constants cconstants c11, c, c22, n, n00 such that such that

0 0 c c11g(n) g(n) f(n) f(n) c c22g(n) g(n)

for all n for all n n n0 0 }}

Page 28: Data Structures for Media Introduction. Dr. Minming Li Department of Computer Science Room :Y6426 Phone :27889538 Email: minmli@cs.cityu.edu.hk, mli000@cityu.edu.hk

Asymptotic NotationAsymptotic NotationNote that:Note that:Running time of Insertion Sort is Running time of Insertion Sort is (n(n22)) is incorrect.is incorrect.Why?Why?

Worst case running time of Insertion Sort is Worst case running time of Insertion Sort is (n(n22)) -- -- correct / incorrect*correct / incorrect*Best case running time of Insertion Sort is Best case running time of Insertion Sort is (n) (n) -- -- correct / incorrect*correct / incorrect*Running time of Insertion Sort is O(nRunning time of Insertion Sort is O(n22) ) -- -- correct / incorrect*correct / incorrect*

Asymptotic Tight Bound: Intuitively like “=”Asymptotic Upper Bound: Intuitively like “≤”Asymptotic Lower Bound Ω Intuitively like “≥” What about “<”?

o versus O : o means better e.g. nlogn=o(n2) means nlogn grows slower than n2

Page 29: Data Structures for Media Introduction. Dr. Minming Li Department of Computer Science Room :Y6426 Phone :27889538 Email: minmli@cs.cityu.edu.hk, mli000@cityu.edu.hk

Relationship between typical functionsRelationship between typical functions log n = o (n)log n = o (n) n = o (n log n)n = o (n log n) nncc = o (2 = o (2nn) where n) where ncc may be n may be n22, n, n44, etc., etc. If f(n)=n+log n, we call log n lower order termsIf f(n)=n+log n, we call log n lower order terms(You are not required to analyze, but remember these relations)(You are not required to analyze, but remember these relations)log n < sqrt(n) < n < nlog n < nlog n < sqrt(n) < n < nlog n < n22 < n < n44 < 2 < 2nn < n! < n!

Rule of combination (for positive function)Rule of combination (for positive function) f(n) = O (g(n)) and h(n) = O (k(n))f(n) = O (g(n)) and h(n) = O (k(n))

f(n)h(n) = O (g(n)k(n))f(n)h(n) = O (g(n)k(n)) f(n)+h(n) = O (g(n)+k(n))f(n)+h(n) = O (g(n)+k(n))

Asymptotic NotationAsymptotic Notation

Page 30: Data Structures for Media Introduction. Dr. Minming Li Department of Computer Science Room :Y6426 Phone :27889538 Email: minmli@cs.cityu.edu.hk, mli000@cityu.edu.hk

When calculating asymptotic running timeWhen calculating asymptotic running time Drop lower order termsDrop lower order terms Ignore leading constantsIgnore leading constants

Example 1: T(n) = AnExample 1: T(n) = An22+Bn+C+Bn+C AnAn22

T(n) = O(nT(n) = O(n22)) Example 2: T(n) = Anlogn+BnExample 2: T(n) = Anlogn+Bn22+Cn+D+Cn+D

BnBn22

T(n) = O(nT(n) = O(n22))

Remember: We can write T(n)=O(nRemember: We can write T(n)=O(n22); T(n)= ); T(n)= (n(n22), but not T(n) ), but not T(n) ≤ O(nO(n22););

Asymptotic NotationAsymptotic Notation

Page 31: Data Structures for Media Introduction. Dr. Minming Li Department of Computer Science Room :Y6426 Phone :27889538 Email: minmli@cs.cityu.edu.hk, mli000@cityu.edu.hk

Insertion-Sort(A)

1 for j = 1 to n-1

2 key = A[j]

3 i = j-1

4 while i >= 0 and A[i] > key

5 A[i+1] = A[i]

6 i = i – 1

7 A[i+1] = key

Very often the algorithm complexity can be observed directly from simple algorithms

O(n2)

There are 4 very useful rules for such Big-Oh analysis ...

Asymptotic PerformanceAsymptotic Performance

Page 32: Data Structures for Media Introduction. Dr. Minming Li Department of Computer Science Room :Y6426 Phone :27889538 Email: minmli@cs.cityu.edu.hk, mli000@cityu.edu.hk

General rules for Big-Oh Analysis:

Rule 1. FOR LOOPS

The running time of a for loop is at most the running time of the statements inside the for loop (including tests) times no. of iterations

Rule 2. NESTED FOR LOOPS

The total running time of a statement inside a group of nested loops is the running time of the statement multiplied by the product of the sizes of all the loops.

for (i=0;i<N;i++)for (j=0;j<N;j++)

k++; O(N2)

for (i=0;i<N;i++)a++; O(N)

Rule 3. CONSECUTIVE STATEMENTS

Count the maximum one.

for (i=0;i<N;i++)a++;

for (i=0;i<N;i++)for (j=0;j<N;j++)

k++;

O(N2)

Rule 4. IF / ELSE

For the fragment:If (condition)

S1 else

S2,

take the test + the maximum for S1 and S2.

Asymptotic PerformanceAsymptotic Performance

Page 33: Data Structures for Media Introduction. Dr. Minming Li Department of Computer Science Room :Y6426 Phone :27889538 Email: minmli@cs.cityu.edu.hk, mli000@cityu.edu.hk

Example of Big-Oh Analysis:

void function1(int n){ int i, j;

int x=0;

for (i=0;i<n;i++)x++;

for (i=0;i<n;i++)for (j=0;j<n;j++)

x++;}

This function is O(__)

void function2(int n){ int i;

int x=0;

for (i=0;i<n/2;i++)x++;

}

This function is O(__)

Asymptotic PerformanceAsymptotic Performance

Page 34: Data Structures for Media Introduction. Dr. Minming Li Department of Computer Science Room :Y6426 Phone :27889538 Email: minmli@cs.cityu.edu.hk, mli000@cityu.edu.hk

Example of Big-Oh Analysis:

void function4(int n){ int i;

int x=0;

for (i=0;i<10;i++)for (j=0;j<n/2;j++)

x--;}

void function3(int n){ int i;

int x=0;

if (n>10)for (i=0;i<n/2;i++)

x++;else{ for (i=0;i<n;i++)

for (j=0;j<n/2;j++)x--;

}}

This function is O(__)

This function is O(__)

Asymptotic PerformanceAsymptotic Performance

Page 35: Data Structures for Media Introduction. Dr. Minming Li Department of Computer Science Room :Y6426 Phone :27889538 Email: minmli@cs.cityu.edu.hk, mli000@cityu.edu.hk

Example of Big-Oh Analysis:

void function5(int n)

{ int i;

for (i=0;i<n;i++)

if (IsSignificantData(i))

SpecialTreatment(i);

}

This function is O(____)

Suppose IsSignificantData is O(n),SpecialTreatment is O(n log n)

Asymptotic PerformanceAsymptotic Performance

Page 36: Data Structures for Media Introduction. Dr. Minming Li Department of Computer Science Room :Y6426 Phone :27889538 Email: minmli@cs.cityu.edu.hk, mli000@cityu.edu.hk

RecursionRecursionint Power(int base,int pow){ if (pow==0) return 1;

else return base*Power(base,pow-1); }

ExampleExample3322=9=9Power(3,2)=3*Power(3,1)Power(3,2)=3*Power(3,1)Power(3,1)=3*Power(3,0)Power(3,1)=3*Power(3,0)Power(3,0)=1Power(3,0)=1

T(n): the number of multiplications needed to compute Power(3,T(n): the number of multiplications needed to compute Power(3,n)n)

T(n)=T(n-1)+1; T(0)=0T(n)=T(n-1)+1; T(0)=0T(n)=nT(n)=nRunning time of function Power(3,n) is O(n)Running time of function Power(3,n) is O(n)

Asymptotic Performance Asymptotic Performance

Page 37: Data Structures for Media Introduction. Dr. Minming Li Department of Computer Science Room :Y6426 Phone :27889538 Email: minmli@cs.cityu.edu.hk, mli000@cityu.edu.hk

Why recursion?Why recursion? Can’t we just use iteration (loop)?Can’t we just use iteration (loop)?

The reason for recursionThe reason for recursion Easy to program in some situationsEasy to program in some situations

DisadvantageDisadvantage More time and space requiredMore time and space required

Example:Example: Tower of Hanoi ProblemTower of Hanoi Problem

Asymptotic Performance Asymptotic Performance

Page 38: Data Structures for Media Introduction. Dr. Minming Li Department of Computer Science Room :Y6426 Phone :27889538 Email: minmli@cs.cityu.edu.hk, mli000@cityu.edu.hk

Given some rods for stacking disks.Given some rods for stacking disks.

Rules:

(1) The disks must be stacked in order of size.

(2) Each time move 1 disk.

The problem:The problem:Use fewest steps to move all diskUse fewest steps to move all disks from the source rod to the targes from the source rod to the target without violating the rules throt without violating the rules through the whole process (given one ugh the whole process (given one intermediate rod for buffering)?intermediate rod for buffering)?

a source rod an intermediate rod a target rod

Tower of HanoiTower of Hanoi

Page 39: Data Structures for Media Introduction. Dr. Minming Li Department of Computer Science Room :Y6426 Phone :27889538 Email: minmli@cs.cityu.edu.hk, mli000@cityu.edu.hk

Suppose you can manage the n-1 disksSuppose you can manage the n-1 disks How do you solve the n disks case?How do you solve the n disks case? A recursive solution:A recursive solution:

Step 1: Move the top n-1 disks from source rod to Step 1: Move the top n-1 disks from source rod to intermediate rod via target rodintermediate rod via target rod

Step 2: Move the largest disk from source rod to target Step 2: Move the largest disk from source rod to target rodrod

Step 3: Move the n-1 disks from intermediate rod to Step 3: Move the n-1 disks from intermediate rod to target rod via source rodtarget rod via source rod

Tower of HanoiTower of Hanoi

Page 40: Data Structures for Media Introduction. Dr. Minming Li Department of Computer Science Room :Y6426 Phone :27889538 Email: minmli@cs.cityu.edu.hk, mli000@cityu.edu.hk

void Towers (int n, int Source, int Target, int Interm) { if (n==1)

Console::Write(S“\nFrom {0} to {1}”, Source.ToString(), Target.ToString());else{ Towers(n-1, Source, Interm, Target); Towers(1, Source, Target, Interm); Towers(n-1, Interm, Target, Source);}

}

How many “Console::Write” are executed?T(n)=2T(n-1)+1T(n)=2T(n-1)+1

Tower of HanoiTower of Hanoi

Towers (3,’A’,’C’,’B’)

Towers (2,’A’,’B’,’C’)

Towers (1,’A’,’C’,’B’)

Towers (2,’B’,’C’,’A’)

Page 41: Data Structures for Media Introduction. Dr. Minming Li Department of Computer Science Room :Y6426 Phone :27889538 Email: minmli@cs.cityu.edu.hk, mli000@cityu.edu.hk

T(n)=T(n-1)+A; T(1)=1T(n)=T(n-1)+A; T(1)=1 T(n)=O(n)T(n)=O(n)

T(n)=T(n-1)+n; T(1)=1T(n)=T(n-1)+n; T(1)=1 T(n)=O(nT(n)=O(n22))

T(n)=2T(n/2) + n; T(1)=1T(n)=2T(n/2) + n; T(1)=1 T(n)=O(n log n)T(n)=O(n log n)

T(n)=2T(n-1)+1T(n)=2T(n-1)+1 T(n)=O(2T(n)=O(2nn))

More general form: T(n)=aT(n/b)+cnMore general form: T(n)=aT(n/b)+cn Master’s Theorem Master’s Theorem

(You are not required to know)(You are not required to know)

Recursive RelationRecursive Relation

Page 42: Data Structures for Media Introduction. Dr. Minming Li Department of Computer Science Room :Y6426 Phone :27889538 Email: minmli@cs.cityu.edu.hk, mli000@cityu.edu.hk

Introduction / Insertion sortIntroduction / Insertion sort Correctness of algorithmCorrectness of algorithm Worst /Average case analysisWorst /Average case analysis Order of growthOrder of growth Asymptotic PerformanceAsymptotic Performance

4 rules for asymptotic analysis4 rules for asymptotic analysis Recursive programsRecursive programs

SummarySummary