By: M.Sultan ZiaAssistant Professor (DCS),CIIT, Sahiwal.

Lecture # 01

TODAY

What is an Algorithm?

And How do we analyze one?

2CIIT, Sahiwal

CIIT, Sahiwal 3

ALGORITHMS Informally,

A tool for solving a well-specified computational problem.

Example: sortinginput: A sequence of numbers.output: An ordered permutation of the input.issues: correctness, efficiency, storage, etc.

4CIIT, Sahiwal

AlgorithmInput Output

STRENGTHENING THE INFORMAL DEFINITION

An algorithm is a finite sequence of unambiguous instructions for solving a well-specified computational problem.

Important Features: Finiteness. Definiteness. Input. ≥ 0 Output. ≥ 1 Effectiveness.

5CIIT, Sahiwal

ALGORITHM – IN FORMAL TERMS…

In terms of mathematical models of computational platforms (general-purpose computers).

One definition – Turing Machine that always halts. Other definitions are possible (e.g. Lambda Calculus.) Mathematical basis is necessary to answer questions

such as: Is a problem solvable? (Does an algorithm exist?) Complexity classes of problems. (Is an efficient algorithm

possible?)

Interested in learning more? Take AAA…

6CIIT, Sahiwal

EXAMPLE OF COMPUTATIONAL PROBLEM: SORTING

Statement of problem: Input: A sequence of n numbers <a1, a2, …, an>

Output: A reordering of the input sequence <a´1, a´2, …, a´n> so that a´i ≤ a´j whenever i < j

Instance: The sequence <5, 3, 2, 8, 3>

Algorithms: Selection sort Insertion sort Merge sort (many others)

7CIIT, Sahiwal

Sorting Searching Shortest paths in a graph Minimum spanning tree Primality testing Traveling salesman problem Knapsack problem Chess Towers of Hanoi Program termination

8CIIT, Sahiwal

Some of these problems don’t have efficient algorithms, or algorithms at all!

SOME WELL-KNOWN COMPUTATIONAL PROBLEMS

The theoretical study of computer program’s Performance, and Resource Usage.

Or

Abstract/Mathematical Study/Comparison of Algorithms

9CIIT, Sahiwal

ANALYSIS OF ALGORITHMS

What is more Important than Performance ???

10CIIT, Sahiwal

ANALYSIS OF ALGORITHMS

Performance correlated with user friendliness Real time constraints Performance measure the line b/w what is

feasible and what is not Algorithms gives you a language to talk about

program’s behavior Performance is like money Its Fun Algorithm Designing Capabilities

11CIIT, Sahiwal

WHY STUDY ALGORITHMS AND PERFORMANCE

Sequential SearchO (n)

• A sequential search of a list/array begins at the beginning of the list/array and continues until the item is found or the entire list/array has been searched

12CIIT, Sahiwal

Sequential Search

bool LinSearch(double x[ ], int n, double item){

for(int i=0;i<n;i++){if(x[i]==item) return true;else return false;

}return false;

}

13CIIT, Sahiwal

Search AlgorithmsSuppose that there are n elements in the array. The following expression gives the average number of comparisons:

It is known that

Therefore, the following expression gives the average number of comparisons made by the sequential search in the successful case:

14CIIT, Sahiwal

Search Algorithms

15CIIT, Sahiwal

Binary SearchO(log2n)

• A binary search looks for an item in a list using a divide-and-conquer strategy

16CIIT, Sahiwal

Binary Search– Binary search algorithm assumes that the items in

the array being searched are sorted

– The algorithm begins at the middle of the array in a binary search

– If the item for which we are searching is less than the item in the middle, we know that the item won’t be in the second half of the array

– Once again we examine the “middle” element

– The process continues with each comparison cutting in half the portion of the array where the item might be

17CIIT, Sahiwal

Binary Search

18CIIT, Sahiwal

Binary Search: middle element

left + right

2mid =

19CIIT, Sahiwal

Binary Search

bool BinSearch (double list[ ], int n, double item, int&index) {

int left=0;int right=n-1;int mid;while(left<=right){

mid=(left+right)/2;if(item> list [mid]) { left=mid+1; }else if(item< list [mid]) {right=mid-1;}else{

item= list [mid];index=mid;return true; }

}// while return false; }

20CIIT, Sahiwal

Binary Search: Example

21CIIT, Sahiwal

BASIC ISSUES RELATED TO ALGORITHMS

How to design algorithms

How to express algorithms

Proving correctness

Efficiency (or complexity) analysis Theoretical analysis

Empirical analysis

Optimality 22CIIT, Sahiwal

BUBBLE SORT ALGORITHM

for i = 1 to n

for j = 1 to n-1

if A[j] < A[j+1]

swap(A,i,j)

23CIIT, Sahiwal

for(int x=0; x<n; x++)

for(int y=0; y<n-1; y++)

if(array[y]>array[y+1])

{

int temp = array[y+1]; array[y+1] = array[y];

array[y] = temp;

}

SELECTION SORT

Input: array a[1],…,a[n] Output: array a sorted in non-decreasing order

Algorithm:

24CIIT, Sahiwal

for i=1 to n swap a[i] with smallest of a[i],…,a[n]

INSERTION SORTvoid insertionSort(int numbers[], int array_size){

int i, j, index;for(i=1; i < array_size; i++){

index = numbers[i]; j = i;while ((j > 0) && (numbers[j-1] > index)){

numbers[j] = numbers[j-1]; j = j - 1; }numbers[j] = index;

}}

} 25CIIT, Sahiwal

BASIC ISSUES RELATED TO ALGORITHMS

How to design algorithms

How to express algorithms

Proving correctness

Efficiency (or complexity) analysis Theoretical analysis

Empirical analysis

Optimality 26CIIT, Sahiwal

ALGORITHM ANALYSIS Determining performance characteristics.

(Predicting the resource requirements.) Time, memory, communication bandwidth etc. Computation time (running time) is of primary

concern. Why analyze algorithms?

Choose the most efficient of several possible algorithms for the same problem.

Is the best possible running time for a problem reasonably finite for practical purposes?

Is the algorithm optimal (best in some sense)? – Is something better possible?

27CIIT, Sahiwal

EFFICIENCY OF ALGORITHMS

What resources are required to accomplish the task

How one algorithm compares with other algorithms

28CIIT, Sahiwal

EFFICIENCY AND COMPLEXITY

Efficiency How much time or space is required Measured in terms of common basic operations

Complexity How efficiency varies with the size of the task Expressed in terms of standard functions of n E.g. O(n), O(n2), O(log n), O(n log n)

29CIIT, Sahiwal

EUCLID’S ALGORITHM Problem: Find gcd(m,n), the greatest common divisor of two

nonnegative, not both zero integers m and n

Examples: gcd(60,24) = 12, gcd(60,0) = 60, gcd(0,0) = ?

Euclid’s algorithm is based on repeated application of equality

gcd(m,n) = gcd(n, m mod n)

until the second number becomes 0, which makes the problem trivial.

Example: gcd(60,24) = gcd(24,12) = gcd(12,0) = 1230CIIT, Sahiwal

31CIIT, Sahiwal

((( )))