Algorithm Efficiency and Sorting

Algorithm Efficiency and Sorting

Data Structure & Algorithm

2

Measuring the Efficiency of Algorithms

• Analysis of algorithms– Provides tools for contrasting the efficiency of

different methods of solution• Time efficiency, space efficiency

3

The Execution Time of Algorithms

• Counting an algorithm's operations is a way to assess its time efficiency– An algorithm’s execution time is related to the

number of operations it requires– Example: Traversal of a linked list of n nodes

need n steps for reading the nodes or writing to the output screen

– Example: For loop with n data?

4

Algorithm Growth Rates

• An algorithm’s time requirements can be measured as a function of the problem size– Number of nodes in a linked list– Size of an array– Number of items in a stack

• Algorithm efficiency is typically a concern for large problems only

5

Algorithm Growth RatesFigure 9-1 Time requirements as a function of the problem size n

•Algorithm A requires time proportional to n2

•Algorithm B requires time proportional to n

6

Algorithm Growth Rates

•An algorithm’s growth rate–Enables the comparison of one algorithm with another–Algorithm A requires time proportional to n2

–Algorithm B requires time proportional to n–Algorithm B is faster than Algorithm A –n2 and n are growth-rate functions–Algorithm A is O(n2) - order n2

–Algorithm B is O(n) - order n

•Big O notation

•Also called complexity time

7

Big O Notation

• Big ‘O’ notation is denoted as

O(acc)O - order

acc - class of algorithm complexity that may consist of 1, logxn, n, n logxn, n2, …

8

Order-of-Magnitude Analysis and Big O Notation

• Order of growth of some common functions– O(1) < O(log2n) < O(n) < O(n * log2n) < O(n2)

< O(n3) < O(2n)

• Properties of growth-rate functions– O(n3 + 3n) is O(n3): ignore low-order terms– O(5 f(n)) = O(f(n)): ignore multiplicative

constant in the high-order term

9

Order-of-Magnitude Analysis and Big O Notation

• Worst-case analysis– A determination of the maximum amount of time that

an algorithm requires to solve problems of size n

• Average-case analysis– A determination of the average amount of time that an

algorithm requires to solve problems of size n

• Best-case analysis– A determination of the minimum amount of time that

an algorithm requires to solve problems of size n

10

Keeping Your Perspective

• Frequency of operations– When choosing an ADT’s implementation,

consider how frequently particular ADT operations occur in a given application

• Bad big O:– Frequent use - but smaller data – OK!

– Frequent use – bigger amount of data – Not OK!

– Bigger amount of data, not frequent use – OK!

11

Keeping Your Perspective

• If the problem size is always small, you can probably ignore an algorithm’s efficiency– Order-of-magnitude analysis focuses on large problems

• Weigh the trade-offs between an algorithm’s time requirements and its memory requirements

• Compare algorithms for both style and efficiency

12

Big O Notation

Notation Execution time / number of step

O(1) Constant. Independent of the input size, n.

O(logxn) Logarithmic increase.

13

Big O Notation

O(n) Linear increase.

Increase directly with the input size, n.

O(n logxn) log-linear increase

O(n2) Quadratic increase.

Practical for average input size, n.

O(n3) Cubic increase.

Practical for small input size, n.

O(2n) Exponential increase.

Increase too rapidly to be practical

14

Big O Notation

15

Big O Notation

16

Determine the complexity time of algorithm

• can be determined

- theoretically –

by calculation

- practically –

by experiment /implementation

17

Determine the complexity time of algorithm - practically

– Implement the algorithms in any programming language and run the programs

– Depend on the compiler, computer, data input and programming style.

18

Determine the complexity time of algorithm - theoretically

• The complexity time is related to the number of steps /operations.

• Complexity time can be determined by1. Count the number of steps and then find the

class of complexity.Or2. Find the complexity time for each steps and

then count the total.

19

Determine the number of steps

• The following algorithm is categorized as O(n).

int counter = 1;

int i = 0;

for (i = 1; i <= n; i++) {

cout << "Arahan cout kali ke " << counter << "\n";

counter++;

}

20


Num statements

1 int counter = 1;

2 int i = 0;

3 i = 1

4 i <= n

5 i++

6 cout << "Arahan cout kali ke " << counter << "\n"

7 counter++

21

• Statement 3, 4 & 5 are the loop control and can be assumed as one statement.

Num Statements

1 int counter = 1;

2 int i = 0;

3 i = 1; i <= n; i++

6 cout << "Arahan cout kali ke " << counter << "\n"

7 counter++


22

• statement 3, 6 & 7 are in the repetition structure.

• It can be expressed by summation series

= f(1) + f(2) + . . . + f(n) = n i = 1

n

f(i)

f(i) – statement executed in the loop

Determine the number of steps- summation series

23

• example:- if n = 5, i = 1

The statement that represented by f(i) will be repeated 5 time = n times = O (n)

= f(1) + f(2) + f(3) + f(4) + f(5) = 5 i = 1

5

f(i)


24


statements Number of stepsint counter = 1;

int i = 0;

i = 1; i= n; i++

cout << "Arahan cout kali ke " << counter << "\n"

counter++

= 1i=1

1f(i)

= 1i=1

1f(i)

= ni=1

nf(i)

.i=1

nf(i) = n . 1 = n

i=1

1f(i)

. i=1

nf(i) = n . 1 = n

i=1

1f(i)

25


• Total steps:

1 + 1 + n + n + n = 2 + 3n

• Consider the largest factor.

• Algorithm complexity can be categorized as O(n)

26

Determine the number of steps- through actual counting

Line num

Statement Total steps

0

1

2

3

4

5

6

start

int counter = 1;

int i = 0

for (i = 1; i <= n; i++)


counter++;

End

-

1

1

n

n.1 = n

n.1 = n

-

27

Determine the number of steps- through actual counting

• Statements 1 and 2 executed only once, 0 (1)• Statement 3 linear execution, 0 (n)• Statements 4, 5, 6 executed only once, 0 (1) HOWEVER because those

statements is in a for loop, execution is linear, 0 (n)

• Algorithm complexity time:-

T(n) = 1 + 1 + n + n + n = 2 + 3n= O (n)

28

Shortcut formula for loop total steps

Total Steps = b-a+l b = loop final conditional valuea = loop initial conditional valuel = constant value for loop increment

Example – using the statement 3 (slide 38th)

b = na = 1l = 1Total steps = b – a + 1 = n – 1 + 1 = n

29

Penyataan Bilangan langkah

void contoh1 ( )

{

cout << “ Contoh kira langkah “;

}

0

0

1

0

Jumlah langkah 1

Jum bil langkah =1(nilai malar) ; masa kerumitan = O (1)


void contoh2 ( )

{

for (int a=1; a<=5; a++)


}

0

0

5-1+1=5

5.1=5

0

Jumlah langkah 10

30


void contoh3 ( )

{

for (int a=1; a<=n; a++)


}

0

0

n-1+1=n

n.1=n

0

Jumlah langkah 2n

Jum bil langkah =10(nilai malar) ; masa kerumitan = O (1)

Samb…

Jum bil langkah =2n(nilai yg b’gantung pd n) ; masa kerumitan = O (n)

31


void contoh4 ( )

{



}

0

0

n-2+1=n-1

(n-1).1=n-1

0

Jumlah langkah 2(n-1)

Jum bil langkah =2(n-1)(nilai yg b’gantung pd n) ; masa kerumitan = O (n)


void contoh5 ( )

{

for (int a=1; a<=n-1; a++)


}

0

0

n-1-1+1=n-1

(n-1).1=n-1

0

Jumlah langkah 2(n-1)

32


void contoh6 ( )

{


for (int b=1; b<=n; b++)


}

0

0

n-1+1=n

n.(n-1+1)=n.n

n.n.1=n.n

0

Jumlah langkah n+2n2

Samb…

Jum bil langkah =n+2n2(nilai yg b’gantung pd n2) ; masa kerumitan = O (n2)

Jum bil langkah =2(n-1)(nilai yg b’gantung pd n) ; masa kerumitan = O (n)

33

• Count the number of steps and find the Big ‘O’ notation for the following algorithm

int counter = 1;

int i = 0;

int j = 1;

for (i = 3; i <= n; i = i * 3) {

while (j <= n) {


counter++;

j++;

}

}

Determine the number of steps - exercise

34

statements Number of stepsint counter = 1;

int i = 0;

int j = 1;

i = 3; i <= n; i = i * 3

j <= n

= 1i=1

1f(i)

= 1i=1

1f(i)

= 1i=1

1f(i)

= f(3) + f(9) + f(27) + … + f(n) = log3ni=3

nf(i)

.i=3

nf(i) = log3n . n

j=1

nf(i)

Determine the number of steps - solution

35

cout << "Arahan cout kali ke "

<< counter

<< "\n";

counter++;

j++;

.i=3

nf(i) . = log3n . n . 1

j=1

nf(i)

i=1

1f(i)

.i=3

nf(i) . = log3n . n . 1

j=1

nf(i)

i=1

1f(i)


.i=3

nf(i) . = log3n . n . 1

j=1

nf(i)

i=1

1f(i)

36

=> 1 + 1+ 1 + log3n + log3n . n + log3n . n . 1 + log3n . n . 1 + log3n . n . 1

=> 3 + log3n + log3n . n + log3n . n + log3n . n + log3n . n

=> 3 + log3n + 4n log3n

Total steps:


37

3 + log3n + 4n log3n• Consider the largest factor

(4n log3n)• and remove the coefficient

(n log3n) • In asymptotic classification, the base of the log can be

omitted as shown in this formula: logan = logbn / logba• Thus, log3n = log2n / log23 = log2n / 1.58…• Remove the coefficient 1/1.58.. • So we get the complexity time of the algorithm is

O(n log2n)


Documents

Algorithm Efficiency and Sorting