Lecture Preliminaries

Lecture 1: Preliminaries*Lecture 1PreliminariesCOMP 523: Advanced Algorithmic TechniquesLecturer: Dariusz Kowalski

Lecture 1: Preliminaries

Lecture 1: Preliminaries*AnnouncementsTextbook: J. Kleinberg, E. Tardos. Algorithm Design. Addison-Wesley, 2005Lectures: Wednesday 9-11, Friday 15-16 in 3.10Tutorials: Thursday 11-12 in 1.01 Ashton BldgEleni Akrida (akridel@hotmail com) Office hours (room 3.11, Ashton Bldg): Wednesday 12-13Friday 14-15Info and contact:http://www.csc.liv.ac.uk/~darek/[email protected]


Assessment and feedbackAssessment: Exam (75%) and 2 tasks (25% total)Feedback: Presentation and discussion of solutions after assignment deadlineIn case of specific questions, individual appointments Lecture 1: Preliminaries*


Lecture 1: Preliminaries*Algorithms - what does it mean?Algorithm - a list/structure of instructions, which are carried out in a fixed order to find the answer to a questionto calculateAl-Khwarizmi (Muhammed Ibn Musa) - Arabian mathematician who introduced the decimal positional number system to the Western world, the author of AlgebraThis module: techniques to design algorithms for problems concerning data commonly processed by computers (written in hard discs, propagated in networks, entered by user or external device, etc.)


Lecture 1: Preliminaries*Types of algorithmsConstructive vs. Non-constructiveDiscrete vs. NumericalDeterministic vs. non-deterministic (e.g., Randomized, Quantum)Sequential vs. Concurrent vs. DNA vs. Exact vs. Approximateand many others


Lecture 1: Preliminaries*ExampleProblem: Find if a given word occurs in a given textInput: Text and word represented as listsOutput: Word found or not

onceuponatona


Lecture 1: Preliminaries*Example - algorithmNave solution (exhaustive search):Intuition:Check letter after letter, starting from the beginning of the lists, if the corresponding letters are equalIf some corresponding letters are not equal, re-start comparing from the second letter of the text (and the first letter of the word)Etc. from the 3rd letter, 4th letter, until the end of text or successful comparisonImplementation:Initiate three pointers: blue_pointer and black_pointer at the beginning of blue_list yellow_pointer at the beginning of yellow_listInitiate Boolean variable successful into false


Lecture 1: Preliminaries*Nave solution cont.Repeat stop := falseMove blue_pointer to the next element Set black_pointer to blue_pointerMove yellow_pointer to the first element of yellow_listRepeatMove black_pointer and yellow_pointer to the next elements in corresponding listsIf values pointed by black_pointer and yellow_pointer are different then stop := trueuntil yellow_pointer or black_pointer is at the end of its list or stopIf yellow_pointer is at the end of yellow_list and not stop then successful := trueuntil successful or blue_pointer is at the end of blue_listOutput: successful


Lecture 1: Preliminaries*Example - how it worksonceuponatonaRepeatstop := false, move blue_pointer to the next element, set black_pointer to blue_pointer, move yellow_pointer to the first element of yellow_listRepeatMove black_pointer and yellow_pointer to the next elements in corresponding listsIf values pointed by black_pointer and yellow_pointer are different then stop := trueuntil yellow_pointer or black_pointer is at the end of its list or stopIf yellow_pointer is at the end of yellow_list and not stop then successful := trueuntil successful or blue_pointer is at the end of blue_list


Lecture 1: Preliminaries*What we expect from algorithms?Correctness: computes desired outputTermination: stops eventually (or with high probability)Efficiency: with respect toPerformance measureTime (or total number of computation steps)Size of memory usedNumber of messages sent Methods of measuringWorst-caseAverage-caseSmoothed (a subset of possible inputs, specific distribution of inputs, etc.)Competitive (comparing to the best solution for particular input)ExpectedAnalysis depends on Model!


Lecture 1: Preliminaries*Example - correctnessSuccessful is set to true iff after some execution of the internal loop the yellow_pointer is at the end of the yellow_listyellow_pointer is at the end of the yellow_list iff it came through the whole yellow_list without coming back to the beginning it happens if the values pointed by black_pointer and yellow_pointer have been the same during all checks of internal loop in the current run (thus a copy of the word exists in the text)Repeatstop := false, move blue_pointer to the next element, set black_pointer to blue_pointer, move yellow_pointer to the first element of yellow_listRepeatMove black_pointer and yellow_pointer to the next elements in corresponding listsIf values pointed by black_pointer and yellow_pointer are different then stop := trueuntil yellow_pointer or black_pointer is at the end of its list or stopIf yellow_pointer is at the end of yellow_list and not stop then successful := trueuntil successful or blue_pointer is at the end of blue_list


Lecture 1: Preliminaries*Example - terminationExternal Loop Invariant: At the beginning of the external loop, blue_pointer advances, and it has a finite number of advances to take before reaching the end of the textInternal Loop Invariant:Each run of the internal loop finishes eventually;It happens because yellow_pointer keeps advancing every iteration (first line of the loop), unless stop condition becomes true or it reaches the end of the word (or black_ pointer reaches the end of the text)Repeatstop := false, move blue_pointer to the next element, set black_pointer to blue_pointer, move yellow_pointer to the first element of yellow_listRepeatMove black_pointer and yellow_pointer to the next elements in corresponding listsIf values pointed by black_pointer and yellow_pointer are different then stop := trueuntil yellow_pointer or black_pointer is at the end of its list or stopIf yellow_pointer is at the end of yellow_list and not stop then successful := trueuntil successful or blue_pointer is at the end of blue_list


Lecture 1: Preliminaries*Example - efficiencyComplexity measures: time, size of additional memorySize of additional memory:3 single pointers and 2 binary variablesTime complexity:Worst-case: if the sequence is not included in the text then time is (almost) proportional to the multiplication of sizes of blue and yellow lists, e.g., text: aab repeated n times word: aaa1, 2 or 3 internal loop runs, 3n times, gives about 6n runs


Lecture 1: Preliminaries*Example - efficiency cont.Time complexity (cont.):Average case: text or sequence are randomly selected;average short word should be find quickly in the beginning of an average long text, in time proportional to the squared length of the word (exercise)Smoothed analysis: text must be randomly selected from reasonable set of texts (usually complicated analysis, depends on the family of texts - dictionary)


Lecture 1: Preliminaries*Efficiency - asymptotic notationn - integer, size of data, f() - function over integersComplexity O(f(n)) : there is a positive constant c such that for every positive integer n complexity is at most cf(n)Complexity (f(n)) : there is a positive constant c such that for every positive integer n complexity is at least cf(n)Complexity o(f(n)) : complexity divided by f(n) comes to 0 with n going to infinity (strictly smaller than f(n))Complexity (f(n)) : complexity divided by f(n) comes to with n going to infinity (strictly bigger than f(n))


Lecture 1: Preliminaries*Example - asymptotic complexityWorst case time: If size of text is n and size of word is n/2 :O((n/2)(n/2)) = O(n2) If size of text is m and size of sequence is k :O((m-k)k)


Lecture 1: Preliminaries*Examples5n3+100 = O(n3) , 5n3+100 O(n2)5n3+100 = (n3) , 5n3+100 (n4)log n = o(na) for any positive constant ana = o(cn)for any positive constants a,clog (4n) = log n + log 4 = O(log n)log (n4)= 4 log n = O(log n)(4n)3 = 64n3 = O(n3)(n4)3 = n12 = (n4) (3n)4 = 81n = (3n)Logarithms are to the base 2


Lecture 1: Preliminaries*Symmetric properties and tight boundIf f(n) = O(g(n)) then g(n) = (f(n)) and vice versaIf f(n) = o(g(n)) then g(n) = (f(n)) and vice versa

Definition:If f(n) = O(g(n)) and f(n) = (g(n)) thenf(n) = (g(n)) and g(n) = (f(n))

Example:9n2 + n +7 = (3n2) = (n2)


Lecture 1: Preliminaries*Transitive properties (order)If f(n) = O(g(n)) and g(n) = O(h(n)) thenf(n) = O(h(n))If f(n) = (g(n)) and g(n) = (h(n)) thenf(n) = (h(n))If f(n) = (g(n)) and g(n) = (h(n)) thenf(n) = (h(n))


Lecture 1: Preliminaries*Sum and maximumf1(n) + + fa(n) = (max(f1(n), , fa(n))) for any constant positive integer aExample:3n2 + n +7 = (3n2) = (n2)If the range of the summation index is not constant:in i = n(n+1)/2 = (n2) (max{1, , n}) = (n)


Lecture 1: Preliminaries*Logarithmic time O(log n)Reduce input to any fraction of it in constant timeExample: searching if a given element x is in a sorted array


Lecture 1: Preliminaries*Linear time O(n)It is sufficient to look at each element of the input constant number of timesExample: finding maximum in an (unsorted) array


Lecture 1: Preliminaries*Time O(n log n)Split the input into two pieces of the similar size and merge both solutions in linear timeExample: sorting an array (split into halves, sort each part separately, merge sorted parts in linear time)


Lecture 1: Preliminaries*Quadratic time O(n2)It is enough to consider pairs of elementsExamples: finding the closest pair of points located in a planesorting by comparison of subsequent elements (insertion sort)


Lecture 1: Preliminaries*Polynomial time O(na)Algorithm has many nested loopsExample: are there any two disjoint sets among given family of n sets, each of at most n elements (time O(n3))


Lecture 1: Preliminaries*Exponential time O(cn)Algorithm considers many subsets of the inputExample: exhaustive search to find the largest clique contained in a given graph (time O(2n))


Lecture 1: Preliminaries*Beyond exponential time: O(n!), O(nn), Algorithm searching in a large spaceExample: search performed in the set of all permutations (time O(n!))


Lecture 1: Preliminaries*GraphsSet of nodes |V| = nSet of edges |E| = mUndirected edges/graph: pairs of nodes {v,w}Directed edges/graph: pairs of nodes (v,w)Set of neighbors of v : set of nodes connected by an edge with v (directed: in-neighbors, out-neighbors)Degree of a node: number of its neighborsPath: a sequence of (non-repeating) nodes such that every two consecutive nodes constitute an edgeLength of a path: number of nodes minus 1Distance between two nodes: the length of the shortest path between these nodesDiameter: the longest distance in the graph


Lecture 1: Preliminaries*Lines, cycles, trees, cliquesLineCycleCliqueTree


Lecture 1: Preliminaries*ConclusionsAlgorithm: list/structure of instructionsAlgorithmic methods for problems arising from computer and communication applicationsGuarantee correctness, termination and efficiencyModel is important!Asymptotic analysis of efficiency


Lecture 1: Preliminaries*Textbook and exercisesREADING:Chapter 2, up to section 2.4

OBLIGATORY EXERCISES: Exercises 1,2,3 page 67 Solved exercises 1,2 pages 65,66

OPTIONAL:Exercises 3,4,5,6 pages 67,68,69Exercises 7,8 pages 69,70


******************************

Documents

Lecture Preliminaries