11/13/2014

1

ECEG-5501Introduction to Algorithm

Analysis & Design

What is Algorithm?

• a sequence of unambiguous instructions for solving a problem, i.e. for obtaining a required output for any legitimate input in a finite amount of time.

Introduction

What is a problem?

• Roughly, a problem specifies what set of outputs is desired for each given set of inputs. Example: The Sorting Problem

• •A problem instance is just a specific set of inputs.•• •Solving a problem instance consists of specifying a

procedure for converting the inputs to an output of the desired form (called a solution).

• •An algorithm that is guaranteed to result in a solution for every instance is said to be correct.

• •Note that a given instance may have either no solutions or more than one solution.

Introduction

11/13/2014

2

The First AlgorithmIntroduction

Simple factoring algorithm

Input: 2 integers m,nOutput: Largest integer that divides both without a remainderAlgorithm 1:

1. factorize m=m1Xm2X…XmP2. factorize n=n1Xn2X…Xnk3. Identify the common factors,

multiply the result and return

Euclid’s Algorithm Euclid(m,n){

while n does not divide mr m%n;m nn rendreturn n

}

while n ≠0 dor ← m mod nm ← nn ← r

return m

What is a program?

• A program is the expression of an algorithm in a programming language

• A set of instructions which the computer will follow to solve a problem

Introduction

Important points

• Non-ambiguity

• Range of inputs

• Several algorithms for solving the same problem

Introduction

11/13/2014

3

How Algorithm is Specified?

• An algorithm is specified:

In words (in a free and also a step-by-step form)

In pseudocode, i.e. a mixture of a natural language and programming language-like construct

Introduction

if – then

if – then – else

for – do

while – do

repeat – until

Pseudo code

C

S S1

C

next statement

false

true

true

S2

next statement

Introduction

if – then if – then – else

Pseudo code

Introduction

for k 1 to n do

.

.

.

done n times

11/13/2014

4

Pseudo code

Introduction

C

S

next statement

false

true

while – do

Pseudo code

Introduction

C

S

next statement

false

true

repeat – until

Steps Problem Solving using Algorithm

Introduction

Understand the problem

Decide on computational means

Exact vs approximate solving,

Algorithm design technique

Design an algorithm

Prove correctness

Analyze the algorithm

Code the algorithm

11/13/2014

5

Fundamentals of Algorithmic Problem Solving

• What does it mean to understand the problem?– Read the problem’s description carefully

– Do a few small examples by hand

– Think about special cases

• Capabilities of the Computational Device

- Random access machines

- Sequential algorithms

- parallel algorithms

- exact vs Approximate algorithms

Introduction

Fundamentals of Algorithmic Problem Solving

• Algorithm Design Techniques

Build a computational model of the solving process

• Prove correctness

Correct output for every legitimate input in finite time

By Mathematical induction

Introduction

Fundamentals of Algorithmic Problem Solving

• Analyze the algorithm

Efficiency: time and space

Simplicity

Generality:

range of inputs, it accepts

Problems the algorithm solve

Optimality

(What is the minimum amount of effort any algorithm will need to exert

to solve the problem?)

Introduction

11/13/2014

6

Fundamentals of Algorithmic Problem Solving

• Analyze the algorithm recognize limitations of various algorithms for

solving a problem

understand relationship between problem size and running time

how to analyze an algorithm's running time without coding it

• CodingHow the objects and operations in the algorithm are

represented in the chosen programming language?

Introduction

Important Problem Types

• Sorting

• Searching

• String Processing

• Combinatorial Problems

• Geometric Problems

• Numerical Problems

Introduction

Important Problem Types

• Sorting rearrange items of a given list in a specific order, based on some key

• Searching find a given search key in a given set

• String Processing String is a sequence of characters from alphabet

search a word in a text (string matching)

• Graph Problems Collections of vertexes some of them connected by line segments

includes graph traversal problems, shortest-path algorithms, traveling salesman problem, graph coloring problems (event scheduling)

Introduction

11/13/2014

7

Important Problem Types

• Combinatorial ProblemsUse permutation, combination, subset that satisfies certain

constraints(either maximize a value or minimize a cost)

e.g. traveling salesman

• Numerical Problems solve equations & systems of equations, evaluate

functions, etc.[ difficulty: round-off error]

• Geometric problems Deals with geometric objects such as points, lines and

polygon

E.g. Closest pair problem

Introduction

Fundamental Data Structures

• Linear Data Structures

Array

Linked list

Stack

Queue

Introduction

Fundamental Data Structures

• Non Linear Data Structures Graphs G=<V, E>

-graph representation: adjacency lists,

adjacency matrix

- weighted graph

Trees : connected graph without cyclesRooted trees

Ordered trees

Binary trees

Tree representation: as graphs; binary nodes

Introduction

11/13/2014

8

Fundamental Data Structures

• Sets, Bags, Dictionaries

Set: unordered collection of distinct elements

Operations: membership, union, intersection

Representation: bit string; linear structure

Bag: unordered collection, elements may be repeated

Dictionary: a bag with operations search, add, delete

Introduction

Algorithm classification

• By types of problems: Study sorting algorithms, then searching algorithms, etc.

• By design technique: Study algorithms by design strategy. A general approach to solving problems Examples of design strategies include:

Brute forceDivide-and-conquerThe greedy methodDynamic programming

Introduction

Example: ALGORITHM Input / Output

• Example: ALGORITHM Euclid (m, n)

// compute GCD(m, n) by Euclid’s algorithm

// input: two non-negative, not-both-zero integers m and n

// output: Greatest Common Divisor of m and n

while n 0 do

r m mod n

m n

n r

return m

Introduction

Input : m and n

Output: the greatest common divisor of m and n

Time : ?

11/13/2014

9

Example: ALGORITHM Input/Output

• Example: sequential search Algorithm sequential search (A[0..n-1], K)

// searches for a given value in a given array by sequential search input: an array A[0..n-and //a search key K output: returns the index of the first element of A that matches K or //-1 if there are no matching elements

i 0

while i n and A[i] K do

i i + 1

if i n return i

else return -1

Introduction

Input: A[0..n-1], K output: j such that A[j] = K or –1

Time: ?

• End of Slide Show

Introduction