Algorithm Efficiency and Sorting

Data Structure & Algorithm

2

Measuring the Efficiency of Algorithms

• Analysis of algorithms– Provides tools for contrasting the efficiency of

different methods of solution• Time efficiency, space efficiency

3

The Execution Time of Algorithms

• Counting an algorithm's operations is a way to assess its time efficiency– An algorithm’s execution time is related to the

number of operations it requires– Example: Traversal of a linked list of n nodes

need n steps for reading the nodes or writing to the output screen

– Example: For loop with n data?

4

Algorithm Growth Rates

• An algorithm’s time requirements can be measured as a function of the problem size– Number of nodes in a linked list– Size of an array– Number of items in a stack

• Algorithm efficiency is typically a concern for large problems only

5

Algorithm Growth RatesFigure 9-1 Time requirements as a function of the problem size n

•Algorithm A requires time proportional to n2

•Algorithm B requires time proportional to n

6

Algorithm Growth Rates

•An algorithm’s growth rate–Enables the comparison of one algorithm with another–Algorithm A requires time proportional to n2

–Algorithm B requires time proportional to n–Algorithm B is faster than Algorithm A –n2 and n are growth-rate functions–Algorithm A is O(n2) - order n2

–Algorithm B is O(n) - order n

•Big O notation

•Also called complexity time

7

Big O Notation

• Big ‘O’ notation is denoted as

O(acc)O - order

acc - class of algorithm complexity that may consist of 1, logxn, n, n logxn, n2, …

8

Order-of-Magnitude Analysis and Big O Notation

• Order of growth of some common functions– O(1) < O(log2n) < O(n) < O(n * log2n) < O(n2)

< O(n3) < O(2n)

• Properties of growth-rate functions– O(n3 + 3n) is O(n3): ignore low-order terms– O(5 f(n)) = O(f(n)): ignore multiplicative

constant in the high-order term

9

Order-of-Magnitude Analysis and Big O Notation

• Worst-case analysis– A determination of the maximum amount of time that

an algorithm requires to solve problems of size n

• Average-case analysis– A determination of the average amount of time that an

algorithm requires to solve problems of size n

• Best-case analysis– A determination of the minimum amount of time that

an algorithm requires to solve problems of size n

10

Keeping Your Perspective

• Frequency of operations– When choosing an ADT’s implementation,

consider how frequently particular ADT operations occur in a given application

• Bad big O:– Frequent use - but smaller data – OK!

– Frequent use – bigger amount of data – Not OK!

– Bigger amount of data, not frequent use – OK!

11

Keeping Your Perspective

• If the problem size is always small, you can probably ignore an algorithm’s efficiency– Order-of-magnitude analysis focuses on large problems

• Weigh the trade-offs between an algorithm’s time requirements and its memory requirements

• Compare algorithms for both style and efficiency

12

Big O Notation

Notation Execution time / number of step

O(1) Constant. Independent of the input size, n.

O(logxn) Logarithmic increase.

13

Big O Notation

O(n) Linear increase.

Increase directly with the input size, n.

O(n logxn) log-linear increase

O(n2) Quadratic increase.

Practical for average input size, n.

O(n3) Cubic increase.

Practical for small input size, n.

O(2n) Exponential increase.

Increase too rapidly to be practical

14

Big O Notation

15

Big O Notation

16

Determine the complexity time of algorithm

• can be determined

- theoretically –

by calculation

- practically –

by experiment /implementation

17

Determine the complexity time of algorithm - practically

– Implement the algorithms in any programming language and run the programs

– Depend on the compiler, computer, data input and programming style.

18

Determine the complexity time of algorithm - theoretically

• The complexity time is related to the number of steps /operations.

• Complexity time can be determined by1. Count the number of steps and then find the

class of complexity.Or2. Find the complexity time for each steps and

then count the total.

19

Determine the number of steps

• The following algorithm is categorized as O(n).

int counter = 1;

int i = 0;

for (i = 1; i <= n; i++) {

cout << "Arahan cout kali ke " << counter << "\n";

counter++;

}

20

Determine the number of steps

Num statements

1 int counter = 1;

2 int i = 0;

3 i = 1

4 i <= n

5 i++

6 cout << "Arahan cout kali ke " << counter << "\n"

7 counter++

21

• Statement 3, 4 & 5 are the loop control and can be assumed as one statement.

Num Statements

1 int counter = 1;

2 int i = 0;

3 i = 1; i <= n; i++

6 cout << "Arahan cout kali ke " << counter << "\n"

7 counter++

Determine the number of steps

22

• statement 3, 6 & 7 are in the repetition structure.

• It can be expressed by summation series

= f(1) + f(2) + . . . + f(n) = n i = 1

n

f(i)

f(i) – statement executed in the loop

Determine the number of steps- summation series

23

• example:- if n = 5, i = 1

The statement that represented by f(i) will be repeated 5 time = n times = O (n)

= f(1) + f(2) + f(3) + f(4) + f(5) = 5 i = 1

5

f(i)

Determine the number of steps- summation series

24

Determine the number of steps- summation series

statements Number of stepsint counter = 1;

int i = 0;

i = 1; i= n; i++

cout << "Arahan cout kali ke " << counter << "\n"

counter++

= 1i=1

1f(i)

= 1i=1

1f(i)

= ni=1

nf(i)

.i=1

nf(i) = n . 1 = n

i=1

1f(i)

. i=1

nf(i) = n . 1 = n

i=1

1f(i)

25

Determine the number of steps- summation series

• Total steps:

1 + 1 + n + n + n = 2 + 3n

• Consider the largest factor.

• Algorithm complexity can be categorized as O(n)

26

Determine the number of steps- through actual counting

Line num

Statement Total steps

0

1

2

3

4

5

6

start

int counter = 1;

int i = 0

for (i = 1; i <= n; i++)

cout << "Arahan cout kali ke " << counter << "\n";

counter++;

End

-

1

1

n

n.1 = n

n.1 = n

-

27

Determine the number of steps- through actual counting

• Statements 1 and 2 executed only once, 0 (1)• Statement 3 linear execution, 0 (n)• Statements 4, 5, 6 executed only once, 0 (1) HOWEVER because those

statements is in a for loop, execution is linear, 0 (n)

• Algorithm complexity time:-

T(n) = 1 + 1 + n + n + n = 2 + 3n= O (n)

28

Shortcut formula for loop total steps

Total Steps = b-a+l b = loop final conditional valuea = loop initial conditional valuel = constant value for loop increment

Example – using the statement 3 (slide 38th)

b = na = 1l = 1Total steps = b – a + 1 = n – 1 + 1 = n

29

Penyataan Bilangan langkah

void contoh1 ( )

{

cout << “ Contoh kira langkah “;

}

0

0

1

0

Jumlah langkah 1

Jum bil langkah =1(nilai malar) ; masa kerumitan = O (1)

Penyataan Bilangan langkah

void contoh2 ( )

{

for (int a=1; a<=5; a++)

cout << “ Contoh kira langkah “;

}

0

0

5-1+1=5

5.1=5

0

Jumlah langkah 10

30

Penyataan Bilangan langkah

void contoh3 ( )

{

for (int a=1; a<=n; a++)

cout << “ Contoh kira langkah “;

}

0

0

n-1+1=n

n.1=n

0

Jumlah langkah 2n

Jum bil langkah =10(nilai malar) ; masa kerumitan = O (1)

Samb…

Jum bil langkah =2n(nilai yg b’gantung pd n) ; masa kerumitan = O (n)

31

Penyataan Bilangan langkah

void contoh4 ( )

{

for (int a=2; a<=n; a++)

cout << “ Contoh kira langkah “;

}

0

0

n-2+1=n-1

(n-1).1=n-1

0

Jumlah langkah 2(n-1)

Jum bil langkah =2(n-1)(nilai yg b’gantung pd n) ; masa kerumitan = O (n)

Penyataan Bilangan langkah

void contoh5 ( )

{

for (int a=1; a<=n-1; a++)

cout << “ Contoh kira langkah “;

}

0

0

n-1-1+1=n-1

(n-1).1=n-1

0

Jumlah langkah 2(n-1)

32

Penyataan Bilangan langkah

void contoh6 ( )

{

for (int a=1; a<=n; a++)

for (int b=1; b<=n; b++)

cout << “ Contoh kira langkah “;

}

0

0

n-1+1=n

n.(n-1+1)=n.n

n.n.1=n.n

0

Jumlah langkah n+2n2

Samb…

Jum bil langkah =n+2n2(nilai yg b’gantung pd n2) ; masa kerumitan = O (n2)

Jum bil langkah =2(n-1)(nilai yg b’gantung pd n) ; masa kerumitan = O (n)

33

• Count the number of steps and find the Big ‘O’ notation for the following algorithm

int counter = 1;

int i = 0;

int j = 1;

for (i = 3; i <= n; i = i * 3) {

while (j <= n) {

cout << "Arahan cout kali ke " << counter << "\n";

counter++;

j++;

}

}

Determine the number of steps - exercise

34

statements Number of stepsint counter = 1;

int i = 0;

int j = 1;

i = 3; i <= n; i = i * 3

j <= n

= 1i=1

1f(i)

= 1i=1

1f(i)

= 1i=1

1f(i)

= f(3) + f(9) + f(27) + … + f(n) = log3ni=3

nf(i)

.i=3

nf(i) = log3n . n

j=1

nf(i)

Determine the number of steps - solution

35

cout << "Arahan cout kali ke "

<< counter

<< "\n";

counter++;

j++;

.i=3

nf(i) . = log3n . n . 1

j=1

nf(i)

i=1

1f(i)

.i=3

nf(i) . = log3n . n . 1

j=1

nf(i)

i=1

1f(i)

Determine the number of steps - solution

.i=3

nf(i) . = log3n . n . 1

j=1

nf(i)

i=1

1f(i)

36

=> 1 + 1+ 1 + log3n + log3n . n + log3n . n . 1 + log3n . n . 1 + log3n . n . 1

=> 3 + log3n + log3n . n + log3n . n + log3n . n + log3n . n

=> 3 + log3n + 4n log3n

Total steps:

Determine the number of steps - solution

37

3 + log3n + 4n log3n• Consider the largest factor

(4n log3n)• and remove the coefficient

(n log3n) • In asymptotic classification, the base of the log can be

omitted as shown in this formula: logan = logbn / logba• Thus, log3n = log2n / log23 = log2n / 1.58…• Remove the coefficient 1/1.58.. • So we get the complexity time of the algorithm is

O(n log2n)

Determine the number of steps - solution