Summer training on

Programming Methodology with Logic and

Algorithm Designing

Informal definition of an algorithm used in a computer

3

History of Algorithm

An algorithm, named for the ninth century Persian Mathematician al-Khowarizmi, is simply a set of rules for carrying out some calculations, either by hand or, more usually, on a machine.

Informally, an algorithm is any well defined computational procedure that takes some value or set of values as output.

30’s: Mathematical notion of algorithms

Church, Turing, Post

recursive functions, Λ-calculus, Turing-machines

Church, Gödel

algorithmic and logical undecidability

50’s, 60’s: Computers

the significance of running time

simple and complex problems

sorting

searching

arithmetic

…

Travelling Salesman

matching

network flows

factoring

…

late 60’s-80’s: Complexity theory

P=NP?

Time, space, information complexity

Polynomial hierarchy

Nondeterminism, good characteriztion, completeness

Randomization, parallelism

Classification of many real-life problems into P vs. NP-complete

90’s: Increasing sophistication upper and lower bounds on complexity

algorithms negative results

factoringvolume computationsemidefinite optimization

topologyalgebraic geometrycoding theory

Higlights of the 90’s:

Approximation algorithms

positive and negative results

Probabilistic algorithms

Markov chains, high concentration, nibble methods, phase transitions

Pseudorandom number generators

from art to science: theory and constructions

Randomized algorithms (making coin flips):

Algorithms and probability

Algorithms with stochastic input:

difficult to analyze

even more difficult to analyze

important applications (primality testing, integration, optimization, volume computation, simulation)

even more important applications

Difficulty: after a few iterations, complicated functions of the original random variables arise.

In our opinion, the notion of algorithm cannot be rigorously defined in full generality, at least for the time being. The reason is that the notion is expanding.

Andreas Blass suggested a better analogy: algorithms to numbers.

Many kinds of numbers have been introduced throughout history: positive integers, natural numbers, rationals, reals, complex numbers, quaternions, infinite cardinals, infinite ordinals, etc.

Similarly many kinds of algorithms have been introduced.

Classical sequential algorithms, in use from antiquity, we have now parallel, interactive, distributed, real-time, analog, hybrid, quantum, etc. Algorithms.

The only algorithms in use from antiquity to the 1950s.

“Algorithms compute in steps of bounded complexity”, wrote Andrei Kol mogorov in 1953.

From 3000 BC to 2000 AD

Perspective

Algorithms we can find everywhere. They have been developed to easy our daily life. It began with calculating algorithms and is today going to artificial intelligence and molecular biology. Google is one of the actual product of this development, knowledge based databases and neuronal networks will lead to the next...

Another Defination of Algorithm

“An algorithm is a sequence of unambiguous instructions for solving a problem, i.e., for obtaining a required output for any legitimate input in a finite amount of time [Levitin, p. 3].”

The word "algorithm" has an interesting history

algorism (noun), algorithm (noun), algorist (noun): these words come from the now-quite-distorted name of a person, Ja'far Mohammed Ben Musa, who was known as al-Khowarazmi, meaning "the man from Khwarazm."

Any use or manipulation of Arabic numerals - especially a pattern used to add, subtract, multiply, etc. - was known as an algorism.

The current form algorithm exhibits what the Oxford English Dictionary calls a "pseudo-etymological perversion": it got confused with the word arithmetic.

Current dictionaries still list the older form algorism in the sense of "the decimal or Arabic system of numeration."

A calendar is nothing but an algorithmic way of assigning - in this case Hebrew – dates.

What Kind of Problems solved by Algorithms?

Anything solved by a computer is done using an algorithm.

The halting problem is a famous problem that *cannot* be solved with an algorithm.

“algorithmic problem-solving”; this means exploiting what has been learned from the experience of developing computer software over the last 50 years that has helped us to hone our problem-solving skills.

Algorithms are the end-product of the problem-solving process and it is imperative that they are made explicit and are carefully studied.

Correct-by-construction is a systematic way of designing algorithms using the specification as a blueprint; it is a pre-hoc design process quite different from the post-hoc process used to “debug” computer software

Roland Backhouse’s book on Algorithmic Problem Solving introduces the principles underlying correct-by-construction design techniques.

Problem solving is not easy. Don’t confuse problem solving with the verification of existing solutions; construction is much harder than verification!

If the problem is that of computations, being "unambiguous" usually means that a human of average intelligence must be able (if only in principle) to follow the instructions with pencil and paper.

A discerning robot must be able to perform the job as well this links the idea of algorithm to programming.

However, quite often, the concept of algorithm is thought to be distinct from that of program.

Problems Listing

Sorting: Sorting provides a good example of an algorithm that is very frequently used by computer scientists.

Shortest Path: The most real worl problems as networking, traffic problem or hacking.

Approximate algorithms: NP-hard problems, even the most advanced algorithm, with the most advanced heuristics, on the fastest computers which is too slow.

Since the fastest known algorithm for solving this problem is too slow – and many believe this will always be true – programmers look for sufficiently fast algorithms that give good, but not optimal solutions.

Random Algorithms: OS desining, interrupt handling, password cracking, cryptography, etc.

Compression: Data compression, cache memory, mailing, sharing, etc.

Maximum Flow: The US wanted to know how quickly the Soviet Union could get supplies through its rail network to its satellite states in Eastern Europe.

Pattern matching: Sequence comparison, anti-virus development, hacking, DNA problems and plagiarism detection, Bioinformatics, medical,etc.

Circuit diagrams. Compilation (Directed Acyclic graphs) Maps. Very compact as graphs. Network flow problems. Decision trees for expert systems. Dynamic Problems

Greedy algortihm Branch and Bound algorithms Divide and Conquer algorithms Recursion algorithms Backtracking algorithms And many more......