LeongHW, SoC, NUS (UIT2201: Algorithms) Page 1 Algorithms Readings: [SG] Ch. 2 & 3 Chapter Outline: 1.Chapter Goals 2.What are Algorithms 1.Real Life

LeongHW, SoC, NUS(UIT2201: Algorithms) Page 1

Algorithms

Readings: [SG] Ch. 2 & 3

Chapter Outline:1. Chapter Goals

2. What are Algorithms1. Real Life Examples (origami, recipes)

2. Definition

3. Example: A = B + C

3. Expressing Algorithms – Pseudo-Code

4. Simple Algorithms

5. Recursive Algorithms

6. Time Complexity of Algorithms


Algorithms

Computing devices are dumb How to explain to a dumb mechanical /

computing device how to solve a problem

How to solve a problem using “a small, basic set of primitive

instructions”.

Complexity of Solving Problems.


1. Goals of Algorithm Study

To develop framework for instructing computer to perform tasks

To introduce notion of algorithm as means of specifying how to solve a problem

To introduce and appreciate approaches for defining and solving very complex tasks in terms of simpler tasks;



2. What are Algorithms1. Real Life Examples (origami, recipes)

2. Definition of Algorithm

3. Example: A = B + C






2. Computer Science and Algorithms…

Computer Science can be considered…

as study of algorithms including their formal properties Their hardware and software realisations Their applications to diverse areas


Algorithms: Real Life Examples

Many Real-Life Examples Cooking: Recipe for preparing a dish Origami: The Art of Paper Folding Directions: How to go to Changi Airport

Remember: Framework: How to give instructions; Alg: The actual step-by-step instructions; Abstraction: Decomposing / Simplifying


Real Life Example: Cooking

Framework: “Cooking or Recipe language”

Algorithm: “Recipe for a Dish”

(step by step instructions on how to cook a dish)

Problem Decomposition Preparing Ingredients; Preparing Sauce;


Real Life Example: Origami

Framework: “Origami or Paper-Folding language”

Algorithm: “Sequence of Paper-Folding Instructions”

(step by step instructions for each fold)

Problem Decomposition Start with a Bird Base; … Finish the Head; Finish the Legs; Finish the Tail;


Real Life Examples: Issues

Problems/Difficulties: Imprecise Instructions; Job can often be done even if instructions are

not followed precisely Modifications may be done by the person

following the instructions;

But, NOT for a Computer Needs to told PRECISELY what to do;

Instructions must be PRECISE; Cannot be vague or ambiguous


Definition of Algorithm:

An algorithm for solving a problem “a finite sequence of unambiguous, executable

steps or instructions, which, if followed would ultimately terminate and give the solution of the problem”.

Note the keywords: Finite set of steps; Unambiguous; Executable; Terminates;


Algorithm Examples?

Problem 1: What is the largest integerINPUT: All the integers { … -2, -1, 0, 1, 2, … }

OUTPUT: The largest integer

Algorithm: Arrange all the integers in a list in decreasing order; MAX = first number in the list; Print out MAX;

WHY is the above NOT an Algorithm? (Hint: How many integers are there?)

Problem 2: Who is the tallest women in the world? Algorithm: Tutorial...


Example: Adding two (n-digit) numbers

Input: Two positive m-digit decimal numbers (a and b)

am, am-1, …., a1

bm, bm-1, …., b1

Output: The sum c = a + b

cm+1, cm, cm-1, …., c1

(Note: In the textbook, it was am-1,…,a1,a0 …)


How to “derive” the algorithm

Adding is something we all know done it a thousand times, know it “by heart”

How do we give the algorithm? A step-by-step instruction to a dumb machine

Try an example: 3 4 9 2

8 1 5 7

“Imagine you looking at yourself solving it”


Step 1: Set the value of carry to 0

Step 2: Set the value of i to 1.

Step 3: Repeat steps 4 through 6 until the value of i is > m.

Step 4: Add ai and bi to the current value of carry, to get xi.

Step 5: If xi < 10 then

let ci=xi and reset carry to 0.

Else (* i.e. xi 10 *)

let ci=xi - 10 and reset carry to 1.

Step 6: Increase the value of i by 1.

Step 7: Set cm+1 to the value of carry.

Step 8: Print the final answer cm+1, cm, …., c1

Step 9: Stop.

Algorithm: Finding sum of A & B


Chapter Outline:1. Chapter Goals2. What are Algorithms3. Expressing Algorithms – Pseudo-Code

1. Communicating Alg to computer2. Pseudo-Code3. Primitive Operations and examples4. Variables and Arrays5. Algorithm C=A+B in pseudo-code

4. Simple Algorithms5. Recursive Algorithms6. Time Complexity of Algorithms


3. Expressing Algorithms to Computer

To communicate algorithm to computer Need way to “represent” the algorithm Cannot use English

Can use computer language machine language and programming languages (Java, Pascal, C) But, these are too tedious (&technical)

Use Pseudo-Code instead…


Pseudo-Code to express Algorithms

Pseudo-Code Mixture of computer language and English

Somewhere in between precise enough to describe what is meant without

being too tediuos

Examples: Let c be 0; Sort the list of numbers in increasing order;

Need to know both syntax – representation semantics – meaning


Primitive Operations

To describe an algorithm, we need some well-defined programming primitives Assignment primitive:

assignment statement, input/output statements

Conditional primitive: if statement case statement

Looping (iterative) primitive: for loop, while loop,

Statements are executed one-by-one


Conditional Primitives…

if statement to take different actions based on condition

Syntaxif (condition) then (Step A) else (Step B)endif

if (condition) then (Step A)endif

Semantics

condition?

Step B

true false

Step A


Examples -- (if statement)…

Let mark be the total-mark obtainedif (mark < 40) then (print “Student fail”) else (print “Student pass”)endif…

read in mark (*from the terminal*)if (mark < 40) then (Grade “F”) else if (mark < 50) then (Grade “D”) else if (mark < 60) then (Grade “C”) else if (mark < 70) then (Grade “B”) else if (mark < 80) then (Grade “A”);endifprint “Student grade is”, Grade…


the while-loop loop a “variable”

number of times

Syntaxwhile (condition) do

(some sequence

of statements)

endwhile

Semantics…

Looping Primitive – while-loop

condition?

Some sequenceof statements;

true

false


First, the for-loop loop a “fixed” or (pre-

determined) number of times

Syntaxfor j a to b do

(some sequence

of statements)

endfor

Semantics…

Looping Primitive – for-loop

j a;

(j <= b)?

Some sequenceof statements;

j j+1;

false

true


“Exercising the alg”: for and while

for j 1 to 4 do print 2*j;endforprint “--- Done ---”

Output: 2 4 6 8--- Done ---

j 1;while (j <= 4) do print 2*j; j j + 1;endwhileprint “--- Done ---”

Output: 2 4 6 8--- Done ---


Variables and Arrays…

In the computer, each variable is assigned a storage “box” can store one number at any time eg: sum, j, carry

Arrays: Often deal with many numbers Such as A1, A2, A3, … , A100

Store as an “array” A[1], A[2], … , A[100] we treat each of them as a variable, each is assigned a storage “box”


Algorithm: A = B + C (in pseudo-code)

We can re-write the C=A+B algorithm as follows:

Alg. to Compute C = A + B: (*sum two big numbers*)carry 0;for i 1 to m do

x[i] a[i] + b[i] + carry ;if (x[i] < 10) then ( c[i] x[i]; carry 0; )

else ( c[i] x[i] – 10; carry 1; )endfor;c[m+1] carry;

Print c[m+1], c[m], …., c[1]



2. What are Algorithms


4. Simple Algorithms1. Simple iterative algorithms

Computing Sum, Find Max/Min

2. Modular Program Design

3. Divisibility and Prime Numbers




Simple iterative algorithm: Sum

Given: Given: List of numbers: A1, A2, A3, …., An

Output: To compute the sum of the numbers

Note: Store numbers in array A[1], A[2], … , A[n]

Sum(A, n);begin Sum_sf 0; k 1; while (k <= n) do Sum_sf Sum_sf + A[k]; k k + 1; endwhile Sum Sum_sf; Print “Sum is”, Sumend;


Exercising Algorithm Sum:

A[1] A[2] A[3] A[4] A[5] A[6] n=6 2 5 10 3 12 24

k Sum-sf Sum ? 0 ? 1 2 ? 2 7 ? 3 17 ? 4 20 ? 5 32 ? 6 56 ? 6 56 56

Sum is 56

Input:

Processing:

Output:


Algorithm for Sum (with for-loop)

We can also use a while-loop instead of a for loop.

HW: (a) Note the differences… (b) Modify it to compute the average?

Sum(A, n);(* Find the sum of A1, A2,…, An. *)begin Sum_sf 0; for k 1 to n do Sum_sf Sum_sf + A[k]; endfor Sum Sum_sf; Print “Sum is”, Sumend;


Remarks about the iterative algorithm…

Note the three stages:1. Initialization

Set some values at the beginning

2. Iteration This is the KEY STEP Where most of work is done

3. Post-Processing or Cleanup

Can use this setup for other problems Calculating average, sum-of-squares Finding max, min; Searching for a number,


Modular Program Design

Software are complex HUGE (millions of lines of code) eg: Linux, Outlook COMPLEX; eg: Flight simulator

Idea: Divide-and-Conquer Method Complex tasks can be divided and each part solved

separately and combined later.

Modular Program Design Divide big programs into smaller modules The smaller parts are

called modules, subroutines, or procedures Design, implement, and test separately

Modularity, Abstraction, Division of Labour Simplifies process of writing alg/programs


Simple Algorithms: Prime Numbers

Algorithm to determine if n is primeGiven: A positive integer n

Question: Is n a prime number?

What do we know?“A number n is prime iffn has no positive factors except 1 and n”

Idea: Express it “algorithmically”“n is not divisible by any positive number k such that 1 < k < n.” or k=2,3,4,…,(n-1)

What we already have: A module Divisible(m,n) to check divisibility We can use it as a primitive


Pseudo-Code for Prime:

Note: Prime uses the module Divisible(k,n) Exercise it with:

Prime (5); Prime (4); Prime (55); Prime (41);

Prime(n)(* To determine if n is prime *)begin for k 2 to (n-1) do if Divisible(k,n) then (Print “False” & exit) else (* Do nothing *) endfor Print “True” & Stopend;


A Simple Module: Divisibility

A module (procedure) for divisibilityGiven: Two positive integers m and n

Question: Is n divisible by m?

or Algorithm to compute Divisible(m,n)

Algorithm Idea:“A positive integer n is divisible by m iff for some positive integer k n, m*k = n.”

mby divisiblenot isn if False,

mby divisible isn if True,),( {nmDivisible


Module for Divisibility:

Algorithm (in pseudo-code)

Exercise it with: Divisible (3, 9); Divisible (3, 5); Divisible (3,101);

Divisible(m,n)(* To compute if n is divisible by m *)begin D false; (* assume false, first *) for k 1 to n do if (m*k = n) then (Dtrue; exit-loop) else (* Do nothing *) endfor Divisible D; (* true or false *)end;







6. Time Complexity of Algorithms1. Sequential search algorithm

2. Binary search algorithm

3. Analysis of Time Complexity

7. Summary…


Search: sequential search algorithm

Given: Given: List of numbers: A1, A2, A3, …., An and a query number x;

Question: Search for x in the list;

Sequential-Search(A, n, x);(* Search for x in A1, A2,…, An. *)begin for k 1 to n do if (x = A[k]) then (Print “Yes”; Stop) else (* do nothing *) endfor Print “No”; Stopend;


Remarks on Sequential Search Alg

Analogy: House-to-house search…

How fast is it to search for x? How many comparisons do we need? Example: 13, 38, 19, 74, 76, 14, 12, 38, 22, 55

When x=14, need 6 comparisons When x=13, need only 1 comparison BEST CASE When x=55, need 10 comparisons WORST CASE When x=5, need 10 comparisons WORST CASE

In general, given n numbers, A1,…,An

Best Case: 1 comparison Worst Case: n comparisons Average Case: (n+1)/2 comparisons


Binary Search

If the List is sorted, that is

A1 A2 A3 …. An

Then, we can do better, actually a lot better….


Binary Search

1, 4, 9, 11, 14, 43, 78


Binary Search (How fast is it?)

Imagine n=100 After 1 step, size is 50 After 2 steps, size is 25 After 3 steps, size is 12 After 4 steps, size is 6 After 5 steps, size is 3 After 6 steps, size is 1 DONE!!

What if n=1000? 1,000,000?


Binary Search Algorithm

Input: Sorted List A1 A2 A3 …. An

A number x.

Question: Is x in the list?

Binary-Search(A,n,x);

1. First 1

Last n

2. While ( First Last ) do

mid (first+last) / 2 If ( x = A[mid] )

then (output Yes and Stop)

else If ( x < A[mid] )

then Last mid-1

else If ( x > A[mid] ) then First mid+1

EndWhile

3. Output False and Stop

4. End


Time Complexity Analysis

Sequential Search (Alg): Worst Case: n comparisons Best Case: 1 comparison Avg Case: n/2 comparisons

Binary Search (Alg): Worst Case: log2 n comparisons Best Case: 1 comparison Avg Case: about log2 n comparisons

How to get the Average Case? using mathematical analysis…


Complexity of Algorithm…

Logarithmic Time Algorithm Binary Search

A Linear Time Algorithm Algorithm Sum(A,n) -- O(n) time Algorithm Sequential-Search(A,n,x) – O(n) time

A Quadratic Time Algorithm Simple Median-Find (T2-Q3)

An Exponential Time Algorithm All-Subsets(A,n)






5. Recursive Algorithms1. Recursion – the idea

2. Fibonacci sequence

3. Tower of Hanoi



5. Recursion

A problem solving method of “decomposing bigger problems into smaller sub-problems that are identical to itself.”

General Idea: Solve simplest (smallest) cases DIRECTLY

usually these are very easy to solve Solve bigger problems using smaller sub-problems

that are identical to itself (but smaller and simpler)

Abstraction: To solve a given problem, we first assume that we

ALREADY know how to solve it for smaller instances!!

Dictionary definition:recursion

see recursion


Example: Fibonacci Numbers… Definition of Fibonacci numbers

1. F1 = 1,

2. F2 = 1,

3. for n>2, Fn = Fn-1 + Fn-2

Problem: Compute Fn for any n. The above is a recursive definition.

Fn is computed in-terms of itself actually, smaller copies of itself – Fn-1 and Fn-2

Actually, Not difficult:F3 = 1 + 1 = 2 F6 = 5 + 3 = 8 F9 = 21 + 13 = 34

F4 = 2 + 1 = 3 F7 = 8 + 5 = 13 F10 = 34 + 21 = 55

F5 = 3 + 2 = 5 F8 = 13 + 8 = 21F11 = 55 + 34 = 89

1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, …


Fibonacci Numbers: Recursive alg

The above is a recursive algorithm

It is simple to understand and elegant!

But, very SLOW

Fibonacci(n) (* Recursive, SLOW *)begin if (n=1) or (n=2) then Fibonacci(n) 1 (*simple case*) else Fibonacci(n) Fibonacci(n-1) + Fibonacci(n-2) endifend;


Recursive Fibonacci Alg -- Remarks

How slow is it? Eg: To compute F(6)…

F(5)

F(4) F(3)

F(3) F(2)

F(2) F(1)

F(2) F(1)

F(4)

F(3) F(2)

F(2) F(1)

F(6)

HW: Can we compute it faster?


Given: Three Pegs A, B and CPeg A initially has n disks, different size, stacked up,

larger disks are below smaller disks

Problem: to move the n disks to Peg C, subject to1. Can move only one disk at a time

2. Smaller disk should be above larger disk

3. Can use other peg as intermediate

Example: Tower of Hanoi

A B C A B C


Tower of Hanoi

How to Solve: Strategy… Generalize first: Consider n disks for all n 1 Our example is only the case when n=4

Look at small instances… How about n=1

Of course, just “Move disk 1 from A to C”

How about n=2?1. “Move disk 1 from A to B”

2. “Move disk 2 from A to C”

3. “Move disk 1 from B to C”


Tower of Hanoi (Solution!)

General Method: First, move first (n-1) disks from A to B Now, can move largest disk from A to C Then, move first (n-1) disks from B to C

Try this method for n=3

1. “Move disk 1 from A to C”2. “Move disk 2 from A to B”3. “Move disk 1 from C to B”

4. “Move disk 3 from A to C”

5. “Move disk 1 from B to A”6. “Move disk 1 from B to C”7. “Move disk 1 from A to C”


Algorithm for Towel of Hanoi (recursive)

Recursive Algorithm when (n=1), we have simple case Else (decompose problem and make recursive-calls)

Hanoi(n, A, B, C);(* Move n disks from A to C via B *)begin if (n=1) then “Move top disk from A to C” else (* when n>1 *) Hanoi (n-1, A, C, B); “Move top disk from A to C” Hanoi (n-1, B, C, A); endifend;


Characteristics of Algorithm

Correctness

Complexity --- time, space (memory), etc

Ease of understanding

Ease of coding

Ease of maintainence


If you are new to algorithms read the textbook try out the algorithms do the exercises

… The End …

Documents

LeongHW, SoC, NUS (UIT2201: Algorithms) Page 1 Algorithms Readings: [SG] Ch. 2 & 3 Chapter Outline: 1.Chapter Goals 2.What are Algorithms 1.Real Life