Coding Project - Projet de Programmation - MINFO1302Preface Pythonrecalls Problems&Algorithms Big-ONotation Recursion Sorting BruteForceSearch&Backtracking Graphs FundamentalProblemsonGraphs

Coding ProjectProjet de Programmation - MINFO1302

Daniele Catanzaro

Center for Operations Research and Econometrics (CORE)Louvain School of Management - Université Catholique de Louvain

https://perso.uclouvain.be/daniele.catanzaro/Index.html



Preface Python recalls Problems & Algorithms Big-O Notation Recursion Sorting Brute Force Search & Backtracking Graphs Fundamental Problems on GraphsCODINGPROJECT

Daniele Catanzaro | COREUniversité Catholique de Louvain

PrefaceThis course introduces to the foundations of data structures in computer science as well as to the design, anal-ysis, and implementation of the algorithms working on them.The desired pedagogical goals of this course include:

• learning how to model real life problems by means of data structures;• developing problem solving skills, by learning how to design algorithms;• learning how to code an algorithm by using a modern programming language such as Python;• learning how to analyze the performances of an algorithm, both theoretically and experimentally, as wellas to recognize common trade-offs between competing algorithmic strategies.

1 | 268






PrefaceWe will start by providing you with some brief recalls of Python programming (∼4h). We will then present thefollowing topics in the subsequent lectures (∼4h each):

• Problems and Algorithms• Algorithm Analysis and Big-O Notation• Recursion• Sorting• Brute Force Search & Backtracking• Graphs• Fundamental Problems on Graphs

2 | 268






Course Material• Main source: blackboard lectures.The slides are meant to complement the blackboard lectures. We strongly invite you to integrate theblackboard lectures with (at least one of) the following reference textbooks, in the following order ofimportance:

• Michael T. Goodrich, Roberto Tamassia, and Michael H. Goldwasser. Data Structures and Algo-rithms in Python. John Wiley and Sons, 2013.

• T. H. Cormen, C. E. Leiserson, R. L. Rivest, and C. Stein. Introduction to Algorithms. The MIT Press,2009. (A french version of this text is also available: T. H. Cormen, C. E. Leiserson, R. L. Rivest, andC. Stein. Algorithmique. Dudon, 2009.)

• A. Aho and J. Ullman. Foundations of Computer Science. W. H. Freeman, 1992.• Donald Ervin Knuth. The Art of Computer Programming. Addison-Wesley, 2011.

• The course includes also a number of exercise sessions, provided by the assistants.

3 | 268






The Lecturer• Specialties: Discrete Optimization & Theoretical Computer Science.

• Affiliation: Center for Operations Research and Econometrics (CORE), Université Catholique de Louvain.

• Address• Campus LLN: Voie du Roman Pays 34, L1.03.01, B-1348, Louvain-la-Neuve. Floor 0, Bureau: B-003.• Campus Mons: Chaussée de Binche 151, M1.01.01, 7000 Mons, Belgium. Floor 3, Bureau: B-302.

• Webpage: http://perso.uclouvain.be/daniele.catanzaro/Index.html.

• Office Hours: Every Wednesday from 14:00 to 16:30.You may book an appointment either “on the fly” just after the lecture or by sending a simple email [email protected].

4 | 268




http://perso.uclouvain.be/daniele.catanzaro/Index.html

[email protected]



The ExamThe evaluation of this course takes place only once per year and focuses in testing the algorithmic know-howof the student as well as her/his coding skills. To this end, each student is requested to

• Develop a Python Project according to the modalities specified by the lecturer.• The project usually accounts for (at most) 8 points of the final grade, but this scoring may changefor year to year in function of the type of project to address.

• The project ismandatory and no student can access to the subsequent part of the evaluation with-out delivering it within the specified deadline.

• The description and the terms of the projectmay change from year to year.

• Discuss the project in front of the lecturer. In this case, the lecturer will ask questions to (i) check the au-thorship of the code as well as (ii) the abovementioned algorithmic know-how. This part usually accountsfor (at most) 12 points of the final grade, but it can be adjusted in function of the scoring of the project.

5 | 268






Final RemarkThis course is a standard 5 ECTS one, hence it is designed to be delivered throughout 30 hours of frontalteaching. This limited amount of time does not allow the proper covering of classic topics such as Linked Lists,Stacks, Queues, Priority Queues, Heaps, Hash-Tables, and Trees. The reader interested in such topics is referredto more comprehensive sources such as any of the alternative textbooks mentioned in the previous slides.Because many fundamental data structures cannot be covered in this course, several algorithms described inthe subsequent slides have been appropriately modified so as to be self-contained. Also, many formal proofsof correctness have been removed so as to gain in terms of a better coverage of the topics threaten.This set of slides is in a permanent revision for the removal of typos and other small coding errors. If you shouldsee any of them, please, do not hesitate to contact the lecturer.

6 | 268





Python Recalls

7 | 268



Python RecallsPython was originally developed by Guido van Rossum in the early 1990s, and has since become a prominentlyused language in industry and education. The second major version of the language, Python 2, was releasedin 2000, and the third major version, Python 3, released in 2008. We note that there are significant incompati-bilities between Python 2 and Python 3.This course will always refers to Python 3 (more specifically,Python 3.1 or later). The latest version of the lan-guage is freely available at www.python.org, along with documentation and tutorials.This is not a basic programming course. The participants to this course are supposed to have already priorprogramming experience in Python. Hence, in this chapter we will just limit to provide some brief recalls of thePython programming language.Colaboratory is a nice reference IDE to start coding in Python. Please take your time to be familiar with it,because this may prove useful to improve your coding skills.Important: Once acquaintedwith Colaboratory, make an effort to learn coding by just using a simple text editorand the command line. This will be requested during the examination session.

8 | 268




www.python.org

https://colab.research.google.com/drive/1TCxxWFykTyNK3Cse6EPukKaaeQqvil5W



Python RecallsPython is formally an interpreted language. Commands are executed through a piece of software known as thePython interpreter. The interpreter

• receives a command• evaluates that command• reports the result of the command.

While the interpreter can be used interactively (especially when debugging), a programmer typically defines aseries of commands in advance and saves those commands in a plain text file known as source code or a script.For Python, source code is conventionally stored in a file named with the “.py” suffix (e.g., demo.py).Onmost operating systems, the Python interpreter can be started by typing python from the command line. Bydefault, the interpreter starts in interactive mode with a clean workspace. Commands from a predefined scriptsaved in a file (e.g., demo.py) are executed by invoking the interpreter with the filename as an argument (e.g.,python demo.py), or using an additional -i flag in order to execute a script and then enter interactive mode(e.g., python -i demo.py).

9 | 268






A Simple CodeIn the spirit of a brief recall to Python, let’s consider the following program that computes the Grade-PointAverage (GPA) for a student based on US letter grades that are entered by a user. Many of the techniquesdemonstrated in this example will be discussed in the remainder of this chapter. At this point, we draw attentionto a few high-level issues, by starting from a recall of the Python reserved words:

True False None as continue assert defbreak del and class elif for importnonlocal raise with else from in notreturn yield except global is or tryfinally if lambda pass while

Table 1: A listing of reserved words in Python. These names cannot be used as identifiers.

10 | 268






A Simple CodePython’s syntax relies heavily on the use of whitespace. Individual statements are typically concluded with anewline character, although a command can extend to another line, either with a concluding backslash char-acter \, or if an opening delimiter has not yet been closed, such as the { character.Whitespace is also key in delimiting the bodies of control structures in Python. Specifically, a block of code isindented to designate it as the body of a control structure, and nested control structures use increasing amountsof indentation.In the following code, the body of the while loop consists of the subsequent 8 lines, including a nested condi-tional structure. Comments are annotations provided for human readers, yet ignored by the Python interpreter.The primary syntax for comments in Python is based on use of the# character, which designates the remainderof the line as a comment.

11 | 268






A Simple Code1 pr i n t ( ”Welcome to the GPA ca l cu l a to r ” )2 p r i n t ( ” Please enter a l l your l e t t e r grades , one per l i ne . ” )3 p r i n t ( ” Enter a blank l i ne to designate the end . ” )4 # map from l e t t e r grade to point value5 points = { ’A+ ’ : 4 . 0 , ’A ’ : 4 . 0 , ’A− ’ :3 .67 , ’B+ ’ :3 .33 , ’B ’ : 3 . 0 , ’B− ’ :2 .67 , ’C+ ’ :2 .33 , ’C ’ : 2 . 0 , ’C ’

:1 .67 , ’D+ ’ :1 .33 , ’D ’ : 1 . 0 , ’ F ’ : 0 . 0 }6 num_courses = 07 to ta l _po in t s = 08 done = False9 while not done :10 grade = input ( ) # read l i ne from user11 i f grade == ’ ’ : # empty l i ne was entered12 done = True13 e l i f grade not in points : # unrecognized grade entered14 pr i n t ( ”Unknown grade ’ { 0 } ’ being ignored ” . format ( grade ) )15 else :16 num_courses += 117 to ta l _po in t s += points [ grade ]18 i f num_courses > 0: # avoid d i v i s i on by zero19 pr i n t ( ’ Your GPA i s { 0 : . 3 } ’ . format ( to t a l _po in t s / num_courses ) )

Source code

12 | 268




https://colab.research.google.com/drive/1TCxxWFykTyNK3Cse6EPukKaaeQqvil5W#scrollTo=CNHLJLRqEN-P&line=1&uniqifier=1



Python Recalls

Class Descriptionbool Bolean valueint integer value (arbitrary magnitude)float floating-point numberlist mutable sequence of objectstuple immutable sequence of objectsstr character stringset unordered set of distinct objectsdict associative mapping (dictionary)

Table 2: Commonly used built-in classes for Python.

The bool class is used to manipulate logical (or more precisely, boolean) values. The only two instances ofthat class are expressed as the literals True and False. Python allows the creation of a Boolean value from anonboolean type using the syntax bool(foo) for value foo. The interpretation depends upon the nonbooleantype. Numbers evaluate to False if zero, and True if nonzero. Sequences and other container types, such asstrings and lists, evaluate to False if empty and True if nonempty. An important application of this interpretationis the use of a nonboolean value as a condition in a control structure.

13 | 268






Python Recalls: Basic data typesThe int and float classes are the primary numeric types in Python.The int class is designed to represent integer values with arbitrary magnitude. Unlike Java and C++, whichsupport different integral types with different precisions (e.g., int, short, long), Python automatically choosesthe internal representation for an integer based upon the magnitude of its value.Typical literals for integers include 0, 137, and −23. In some contexts, it is convenient to express an integralvalue using binary, octal, or hexadecimal. That can be done by using a prefix of the number 0 and then acharacter to describe the base. Example of such literals are respectively 0b1011, 0o52, and 0x7f.The integer constructor, int(), returns value 0 by default. But this constructor can be used to construct an integervaluebasedupon an existing value of another type. For example, if f represents a floating-point value, the syntaxint(f) gives the truncated value of f.For example,

1 i n t ( 3 . 14 )

and

1 i n t ( 3 . 99 )

14 | 268






Python Recalls: Basic data typesgive the value 3, while

1 i n t ( −3.9)

gives the value −3.The constructor can also be used to parse a string that is presumed to represent an integral value (such as oneentered by a user). If s represents a string, then int(s) gives the integral value that string represents.For example, the expression

1 i n t ( 137 )

gives the integer value 137. If an invalid string is given as a parameter, as in

1 i n t ( he l lo )

15 | 268






Python Recalls: Basic data typesa ValueError is raised.By default, the stringmust use base 10. If conversion from a different base is desired, that base can be indicatedas a second, optional, parameter. For example, the expression

1 i n t ( 7 f , 16)

gives the integer 127 which is the decimal correspondent to the hexadecimal 7f.

16 | 268






Python Recalls: Basic data typesThe float class is the sole floating-point type in Python, using a fixed-precision representation. Its precision ismore akin to a double in Java or C++, rather than those languages’ float type.The floating-point equivalent of an integral number can be expressed directly e.g., as 2.0. Technically, thetrailing zero is optional, so some programmers might use the expression 2. to designate this floating-pointliteral. One other form of literal for floating-point values uses scientific notation. For example, the literal

1 6.022e23

represents the mathematical value 6.022 × 10.The constructor form of float() returns 0.0. When given a parameter, the constructor attempts to return theequivalent floating-point value. For example, the call

1 f l o a t ( 2 )

returns the floating-point value 2.0. If the parameter to the constructor is a string, as with float( 3.14 ), it attemptsto parse that string as a floating-point value, raising a ValueError as an exception.

17 | 268






Python Recalls: Basic data typesThe list, tuple, and str classes are sequence types in Python, representing a collection of values in which theorder is significant. The list class is the most general, representing a sequence of arbitrary objects (akin to an“array” in other languages).The tuple class is an immutable version of the list class, benefiting from a stream-lined internal representation.The str class is specially designed for representing an immutable sequence of text characters. We note thatPython does not have a separate class for characters; they are just strings with length one.Let’s study them more in detail.

18 | 268






Python Recalls: Basic data typesA list instance stores a sequence of objects. A list is a referential structure, as it technically stores a sequenceof references to its elements. Elements of a list may be arbitrary objects (including the None object). Listsare array-based sequences and are zero-indexed, thus a list of length n has elements indexed from 0 to n − 1inclusive.For example, the Python’s internal representation of a list of integers, instantiated as

1 prime=[2 ,3 ,5 ,7 ,11 ,13 ,17 ,19 ,23 ,31]

is the following:

3 4 5 6 70 1 2 1098

primes:

13 19 23 29 317532 11 17

19 | 268






Python Recalls: Basic data typesLists are perhaps the most used container type in Python and they will be extremely central to our study of datastructures and algorithms. They havemany valuable behaviors, including the ability to dynamically expand andcontract their capacities as needed. In this chapter, we will discuss only the most basic properties of lists.Python uses the characters [ ] as delimiters for a list literal, with [ ] itself being an empty list. As another example,

[red, green, blue]

is a list containing three string instances. The contents of a list literal need not be expressed as literals; if iden-tifiers a and b have been established, then syntax [a, b] is legitimate.The list( ) constructor gives an empty list by default. However, the constructor will accept any parameter that isof an iterable type. We will discuss iteration further in Section 1.8, but examples of iterable types include all ofthe standard container types (e.g., strings, list, tuples, sets, dictionaries). For example, the syntax

1 l i s t ( ” he l lo ” )

gives a list of individual characters, [ ’h’ , ’e’ , ’l’ , ’l’ , ’o’ ]. Because an existing list is itself iterable, the syntaxbackup = list(data) can be used to construct a new list instance referencing the same contents as the original.

20 | 268






Python Recalls: Basic data typesThe tuple class provides an immutable version of a sequence, and therefore its instances have an internal rep-resentation that may be more streamlined than that of a list.While Python uses the [ ] characters to delimit a list, parentheses delimit a tuple, with () being an empty tuple.There is one important subtlety. To express a tuple of length one as a literal, a comma must be placed after theelement, but within the parentheses. For example, (17,) is a one-element tuple. The reason for this requirementis that, without the trailing comma, the expression (17) is viewed as a simple parenthesized numeric expression.

21 | 268






Python Recalls: Basic data typesPython’s str class is specifically designed to efficiently represent an immutable sequence of characters, basedupon the Unicode international character set.String literals can be enclosed in single quotes, as in

1 ’ he l lo ’

or double quotes, as in

1 ” he l lo ”

22 | 268






Python Recalls: Basic data typesThis choice is convenient, especially when using another of the quotation characters as an actual character inthe sequence, as in ”Don’t worry”. Alternatively, the quote delimiter can be designated using a backslash as aso-called escape character, as in “Don\’ t worry”.Because the backslash has this purpose, the backslash must itself be escaped to occur as a natural characterof the string literal. This is done by adding two consecutive backslashes. Other commonly escaped charactersare \n for newline and \t for tab. Unicode characters can be included, such as 20\u20AC for the string 20.Python also supports using the delimiter ”’ or ””” to begin and end a string literal. The advantage of such triple-quoted strings is that newline characters can be embedded naturally (rather than escaped as “\n”). This cangreatly improve the readability of long, multiline strings in a source code:

1 pr i n t ( ” ” ”Welcome to the GPA ca l cu l a to r2 Please enter a l l your l e t t e r grades , one per l i ne .3 Enter a blank l i ne to designate the end . ” ” ” )

23 | 268






Python Recalls: Basic data typesPython’s set class represents the mathematical notion of a set, namely a collection of elements, without dupli-cates, and without an inherent order to those elements. The major advantage of using a set, as opposed to alist, is that it has a highly optimizedmethod for checking whether a specific element is contained in the set. Thisis based on a data structure known as a hash table.However, there are two important restrictions due to the algorithmic underpinnings. The first is that the set doesnot maintain the elements in any particular order. The second is that only instances of immutable types can beadded to a Python set. Therefore, objects such as integers, floating-point numbers, and character strings areeligible to be elements of a set. It is possible to maintain a set of tuples, but not a set of lists or a set of sets, aslists and sets are mutable.The frozenset class is an immutable form of the set type, so it is legal to have a set of frozensets.Python uses curly braces { and } as delimiters for a set, for example, as

1 {17 }

or

1 { ” red ” , ” green ” , ” blue ” }

24 | 268






Python Recalls: Basic data typesThe exception to this rule is that { } does not represent an empty set; for historical reasons, it represents anempty dictionary (see next paragraph). Instead, the constructor syntax set() gives an empty set. If an iterableparameter is sent to the constructor, then the set of distinct elements is gived. For example,

1 set ( ” he l lo ” )

gives { h , e , l , o }.

25 | 268






Python Recalls: Basic data typesPython’s dict class represents a dictionary, or mapping, from a set of distinct keys to associated values. For ex-ample, a dictionarymightmap fromunique student ID numbers, to larger student records (such as the student’sname, address, and course grades).Python implements a dict using an almost identical approach to that of a set, but with storage of the associatedvalues.A dictionary literal also uses curly braces, and because dictionaries were introduced in Python prior to sets, theliteral form { } gives an empty dictionary. A nonempty dictionary is expressed using a comma-separated seriesof key:value pairs. For example, the dictionary

1 { ”ga” : ” I r i s h ” , ”de” : ”German” }

maps ga to Irish and de to German. The constructor for the dict class accepts an existing mapping as a param-eter, in which case it creates a new dictionary with identical associations as the existing one. Alternatively, theconstructor accepts a sequence of key-value pairs as a parameter, as in

1 pa i r s = [ ( ”ga” , ” I r i s h ” ) , ( ”de” , ”German” ) ]2 d i c t ( pa i r s )

26 | 268






Python Recalls: Operators1. Logical Operators: Python supports the following keyword operators for Boolean values:

• and: conditional and;• or: conditional or;• not: unary negation.

The and and or operators short-circuit, in that they do not evaluate the second operand if the result canbe determined based on the value of the first operand. This feature is useful when constructing Booleanexpressions in which we first test that a certain condition holds (such as a reference not being None),and then test a condition that could have otherwise generated an error condition had the prior test notsucceeded.

2. Comparison Operations: Python supports the following operators to test two notions of equality:• is: same identity;• is not: different identity;• ==: equivalent;• !=: not equivalent.

27 | 268






Python Recalls: OperatorsThe expression a is b evaluates to True, precisely when identifiers a and b are aliases for the same object.The expression a == b tests a more general notion of equivalence. If identifiers a and b refer to the sameobject, then a == b should also evaluate to True. Yet a == b also evaluates to True when the identifiersrefer to different objects that happen to have values that are deemed equivalent.The precise notion of equivalence depends on the data type. For example, two strings are consideredequivalent if they match character for character. Two sets are equivalent if they have the same contents,irrespective of order. In most programming situations, the equivalence tests == and != are the appropri-ate operators; use of is and is not should be reserved for situations in which it is necessary to detect truealiasing.Data types may define a natural order via the following operators:

• <: less than;• <=: less than or equal to;• > greater than;• >=: greater than or equal to.

These operators have expected behavior for numeric types, and are defined lexicographically, and case-sensitively, for strings. An exception is raised if operands have incomparable types, as with 5 < hello.

3. Arithmetics Operations: Python supports the following arithmetic operators:

28 | 268






Python Recalls: Operators• +: addition;• -: subtraction;• *: multiplication;• /: true division;• //: integer division;• %: modulo operator.

The use of addition, subtraction, and multiplication is straightforward, noting that if both operands havetype int, then the result is an int as well; if one or both operands have type float, the result will be a float.

Python takes more care in its treatment of division. We first consider the case in which both operandshave type int, for example, the quantity 27 divided by 4. In mathematical notation, 27

4 = 6 34 = 6.75.

In Python, the / operator designates true division, returning the floating-point result of the computation.Thus, 27 / 4 results in the float value 6.75.

Python supports the pair of operators // and % to perform the integral calculations, with expression 27 //4 evaluating to int value 6 (the mathematical floor of the quotient), and expression 27 % 4 evaluating toint value 3, i.e., the remainder of the integer division.

29 | 268






Python Recalls: OperatorsPython carefully extends the semantics of // and % to cases where one or both operands are negative.For the sake of notation, let us assume that variables n and m represent respectively the dividend anddivisor of a quotient n

mand that q = n//m and r = n%m. Python guarantees that q ∗ m + r will equal n.

We already saw an example of this identity with positive operands, as 6 ∗ 4 + 3 = 27. When the divisor mis positive, Python further guarantees that 0 ≤ r < m. As a consequence, we find that −27 // 4 evaluatesto −7 and −27 % 4 evaluates to 1, as (7) ∗ 4 + 1 = 27. When the divisor is negative, Python guaranteesthatm < r ≤ 0. As an example, 27//−4 is −7 and 27%−4 is −1, satisfying the identity 27 = (7) ∗ (4) + (1).

The conventions for the // and % operators are even extended to floating-point operands, with the ex-pression q = n // m being the integral floor of the quotient, and r = n % m being the “remainder” toensure that q ∗ m + r equals n. For example, 8.2//3.14 evaluates to 2.0 and 8.2 % 3.14 evaluates to 1.92,as 2.0 ∗ 3.14 + 1.92 = 8.2.

4. Sequence Operations: Each of Python’s built-in sequence types (i.e., str, tuple, and list) support the fol-lowing operator syntaxes:

• s[j]: element at index j;• s[start:stop]: slice including the interval [start,stop);• s[start:stop:step]: slice including indexes start, start + step, start + 2 step,…, up to | but not equalingto | stop;

30 | 268






Python Recalls: Operators• s+t: concatenation of the sequences s and t;• k*s: shorthand for s + s + s + . . . (ktimes);• val in s: containment check• val not in s: non-containment check

Python relies on zero-indexing of sequences, thus a sequence of length n has elements indexed from 0to n − 1 inclusive. Python also supports the use of negative indexes, which denote a distance from theend of the sequence; index −1 denotes the last element, index −2 the second to last, and so on.

Python uses a slicing notation to describe subsequences of a sequence. Slices are described as half-open intervals, with a start index that is included, and a stop index that is excluded. For example, thesyntax data[3:8] denotes a subsequence including the five indexes: 3, 4, 5, 6, 7. An optional ”step” value,possibly negative, canbe indicated as a third parameter of the slice. If a start index or stop index is omittedin the slicing notation, it is presumed to designate the respective extreme of the original sequence.

Because lists are mutable, the syntax s[j] = val can be used to replace an element at a given index. Listsalso support a syntax, del s[j], that removes the designated element from the list. Slice notation can alsobe used to replace or delete a sublist.

31 | 268






Python Recalls: OperatorsThe notation val in s can be used for any of the sequences to see if there is an element equivalent to val inthe sequence. For strings, this syntax can be used to check for a single character or for a larger substring,as with ’amp’ in ’example’.

All sequences define comparison operations based on lexicographic order, per-forming an element byelement comparison until the first difference is found. For example, [5, 6, 9] < [5, 7] because of the entriesat index 1. Therefore, the following operations are supported by sequence types:

• s==t: equivalent (element by element);• s != t: not equivalent;• s<t: lexicographically less than;• s<=t: lexicographically less than or equal to;• s>t: lexicographically greater than;• s>=t: lexicographically greater than or equal to.

5. Operators for Sets and Dictionaries Sets and frozensets support the following operators:• key in s: containment check;• key not in s: non-containment check;

32 | 268






Python Recalls: Operators• s1 == s2: s1 is equivalent to s2;• s1 != s2: s1 is not equivalent to s2;• s1 <= s2: s1 is subset of s2;• s1 < s2: s1 is proper subset of s2;• s1 >= s2: s1 is superset of s2;• s1 > s2: s1 is proper superset of s2;• s1 | s2: the union of s1 and s2;• s1 & s2: the intersection of s1 and s2;• s1 − s2: the set of elements in s1 but not s2;• s1 ∧ s2: the set of elements in precisely one of s1 or s2.

33 | 268






Python Recalls: OperatorsNote well that sets do not guarantee a particular order of their elements, so the comparison operators,such as <, are not lexicographic; rather, they are based on the mathematical notion of a subset. As aresult, the comparison operators define partial order, but not a total order, as disjoint sets are neither“less than”, “equal to”, or “greater than” each other.

Sets also support many fundamental behaviors through named methods (e.g., add, remove); refers tothe set standard library for more information.

Dictionaries, like sets, do not maintain a well-defined order on their elements. Furthermore, the conceptof a subset is not typically meaningful for dictionaries, so the dict class does not support operators suchas <. Dictionaries support the notion of equivalence, with d1 == d2 if the two dictionaries contain thesame set of keyvalue pairs. The most widely used behavior of dictionaries is accessing a value associatedwith a particular key k with the indexing syntax, d[k]. The supported operators are as follows:

• d[key]: value associated with given key;• d[key] = value: set (or reset) the value associated with given key;• del d[key]: remove key and its associated value from dictionary;• key in d: containment check;

34 | 268






Python Recalls: Operators• key not in d: non-containment check;• d1 == d2: d1 is equivalent to d2;• d1 != d2: d1 is not equivalent to d2.

6. Extended Assignment Operators: Python supports an extended assignment operator for most binaryoperators, for example, allowing a syntax such as count += 5. By default, this is a shorthand for the moreverbose count = count + 5. For an immutable type, such as a number or a string, one should not presumethat this syntax changes the value of the existing object, but instead that it will reassign the identifier toa newly constructed value. However, it is possible for a type to redefine such semantics to mutate theobject, as the list class does for the += operator.

• alpha = [1, 2, 3]• beta = alpha an alias for alpha• beta += [4, 5] extends the original list with two more elements• beta = beta + [6, 7] reassigns beta to a new list [1, 2, 3, 4, 5, 6, 7]• print(alpha) will be [1, 2, 3, 4, 5]

This example demonstrates the subtle difference between the list semantics for the syntax beta += fooversus beta = beta + foo.

35 | 268






Python Recalls: Conditionals and Loops• Conditional constructs (also known as if statements) provide a way to execute a chosen block of codebased on the run-time evaluation of one or more Boolean expressions. In Python, the most general formof a conditional is written as follows:

1 i f F i r s tCondi t ion :2 Firs tBody3 e l i f SecondCondition :4 SecondBody5 e l i f ThirdCondit ion :6 ThirdBody7 else :8 FourthBody

36 | 268






Python Recalls: Conditionals and LoopsEach condition is a Boolean expression, and each body contains one or more commands that are to beexecuted conditionally. If the first condition succeeds, the first bodywill be executed; no other conditionsor bodies are evaluated in that case. If the first condition fails, then theprocess continues in similarmannerwith the evaluation of the second condition.

The execution of this overall construct will cause precisely one of the bodies to be executed. There maybe any number of elif clauses (including zero), and the final else clause is optional. Nonboolean typesmaybe evaluated as Booleans with intuitive meanings. For example, if response is a string that was enteredby a user, and we want to condition a behavior on this being a nonempty string, we may write

1 i f response :

as a shorthand for the equivalent

1 i f response != ’ ’ :

As a simple example, a robot controller might have the following logic:

37 | 268






Python Recalls: Conditionals and Loops

1 i f door_is_closed :2 open_door ( )3 advance ( )

Notice that the final command, advance( ), is not indented and therefore not part of the conditional body.It will be executed unconditionally (although after opening a closed door).

We may nest one control structure within another, relying on indentation to make clear the extent of thevarious bodies. Revisiting our robot example, here is a more complex control that accounts for unlockinga closed door.

1 i f door_is_closed :2 i f door_is_locked :3 unlock_door ( )4 open_door ( )5 advance ( )

• Loop constructs divide into the while loop and the for loop. The while loop obeys to the following con-struct:

38 | 268







1 while condit ion :2 body

The for loop obeys to the following construct:

1 fo r element in i t e r ab le :2 body

In both cases, condition can be an arbitrary Boolean expression, and body can be an arbitrary block ofcode (including nested control structures). The execution of a while loop begins with a test of the Booleancondition. If that condition evaluates to True, the body of the loop is performed. After each executionof the body, the loop condition is retested, and if it evaluates to True, another iteration of the body isperformed. When the conditional test evaluates to False (assuming it ever does), the loop is exited andthe flow of control continues just beyond the body of the loop. For example,

1 j=02 while j <len ( data ) and data [ j ] != ’X ’3 j+=1

39 | 268






Python Recalls: Conditionals and LoopsThe len function returns the length of a sequence such as a list or string. The correctness of this looprelies on the short-circuiting behavior of the and operator. We intentionally test j < len(data) to ensurethat j is a valid index, prior to accessing element data[j]. Had we written that compound condition withthe opposite order, the evaluation of data[j] would eventually raise an IndexError when X is not found.

As written, when this loop terminates, variable j’s value will be the index of the leftmost occurrence of X, if found, or otherwise the length of the sequence (which is recognizable as an invalid index to indicatefailure of the search). It is worth noting that this code behaves correctly, even in the special case when thelist is empty, as the condition j < len(data) will initially fail and the body of the loop will never be executed.

The equivalent for loop would look like the following:

1 fo r j in range ( len ( data ) ) :2 i f data [ j ] != ’X ’ :3 break

40 | 268






Python Recalls: Conditionals and LoopsThe range() function returns a sequence of numbers, starting from 0 by default, and increments by 1 (bydefault), and ends at a specified number. For example, if len(data)=6, then the range function returns thevalues from 0 to 5.

It is possible to specify the starting value by adding a parameter. For example range(2, 6), means valuesfrom 2 to 6 (but not including 6).

1 fo r j in range (2 ,6 ) :2 p r i n t ( j )

The range() function defaults to increment the sequence by 1. However, it is possible to specify theincrement value by adding a third parameter: range(2, 30, 3):

1 fo r j in range (2 ,30 ,3 ) :2 p r i n t ( j )

The else keyword in a for loop specifies a block of code to be executed when the loop is finished:

41 | 268







1 fo r j in range (2 ,30 ,3 ) :2 p r i n t ( j )3 else :4 p r i n t ( ” F i n a l l y f in i shed ! ” )

Any iterable construct can replace the range function. For example,

1 adj = [ ” red ” , ” big ” , ” t a s t y ” ]2 f r u i t s = [ ” apple ” , ”banana” , ” cherry ” ]34 fo r x in adj :5 fo r y in f r u i t s :6 p r i n t ( x , y )

About the break keyword, Python supports a break statement that immediately terminate a while or forloop when executed within its body. More formally, if applied within nested control structures, it causesthe termination of the most immediately enclosing loop.

42 | 268






Python Recalls: FunctionsA function is a block of code that only runs when it is called and that both may (or may not) receive data (alsoknown as parameters) when called and may (or may not) return data as a result. An example is the following:

1 def my_function ( ) :2 p r i n t ( ” Hel lo from a funct ion ” )34 my_function ( )

Another is

1 def count ( data , ta rget ) :2 n=03 fo r item in data :4 i f item == target :5 n+=16 return n78 A_chars_number = count ( [ ’X ’ , ’A ’ , ’Y ’ , ’ Z ’ , ’A ’ ] , ’A ’ )

43 | 268






Python Recalls: FunctionsThe first line, beginningwith the keyworddef, serves as the function’s signature. This establishes a new identifieras the name of the function (count, in this example), and it establishes the number of parameters that it expects,as well as names identifying those parameters (data and target, in this example).Unlike Java and C++, Python is a dynamically typed language, and therefore a Python signature does not desig-nate the types of those parameters, nor the type (if any) of a return value. Those expectations should be statedin the function’s documentation and can be enforced within the body of the function, but misuse of a functionwill only be detected at run-time.The remainder of the function definition is known as the body of the function. As is the case with control struc-tures in Python, the body of a function is typically expressed as an indented block of code. Each time a functionis called, Python creates a dedicated activation record that stores information relevant to the current call. Thisactivation record includes what is known as a namespace to manage all identifiers that have local scope withinthe current call. The namespace includes the function’s parameters and any other identifiers that are definedlocally within the body of the function. An identifier in the local scope of the function caller has no relation toany identifier with the same name in the caller’s scope (although identifiers in different scopes may be aliasesto the same object). In our first example, the identifier n has scope that is local to the function call, as does theidentifier item, which is established as the loop variable.

44 | 268






Python Recalls: FunctionsA return statement is used within the body of a function to indicate that the function should immediately ceaseexecution, and that an expressed value should be returned to the caller. If a return statement is executedwithoutan explicit argument, the None value is automatically returned.Likewise, None will be returned if the flow of control ever reaches the end of a function body without havingexecuted a return statement. Often, a return statementwill be the final commandwithin thebodyof the function,as was the case in our earlier example of a count function. However, there can be multiple return statementsin the same function, with conditional logic controlling which such command is executed, if any. As a furtherexample, consider the following function that tests if a value exists in a sequence.

1 def contains ( data , ta rget ) :2 fo r item in data :3 i f item == target :4 return True5 return False

45 | 268






Python Recalls: FunctionsIf the conditional within the loop body is ever satisfied, the return True statement is executed and the functionimmediately ends, with True designating that the target value was found. Conversely, if the for loop reaches itsconclusion without ever finding the match, the final return False statement will be executed.

Some parameters can be set to a default value. For example,

1 def my_function ( country = ”Norway” ) :2 p r i n t ( ” I am from ” + country )34 my_function ( ”Sweden” )5 my_function ( ” Ind ia ” )6 my_function ( )7 my_function ( ” B r a z i l ” )

The names of the functions should be assigned wisely so as to not ”clash” with other important ones, such as

• Input/Output: print, input, andopen. Character Encoding: ord and chr relate characters and their integercode points. For example, ord( A ) is 65 and chr(65) is A.

• Mathematics: abs, div, mod, pow, round, and sum, provide common mathematical functionality;

46 | 268




https://en.wikipedia.org/wiki/ASCII




Python Recalls: Functions• Ordering: max and min apply to any data type that supports a notion of comparison, or to any collectionof such values. Likewise, sorted can be used to give an ordered list of elements drawn from any existingcollection.

• Collections/Iterations: range generates a new sequence of numbers; len reports the length of any exist-ing collection; functions reversed, all, any, and map operate on arbitrary iterations as well; iter and nextprovide a general framework for iteration through elements of a collection.

47 | 268






Python Recalls: Console Input and OutputThe built-in function, print, is used to generate standard output to the console. In its simplest form, it printsan arbitrary sequence of arguments, separated by spaces, and followed by a trailing newline character. Forexample, the command

1 pr i n t ( maroon , 5)

outputs the string ’maroon 5\’. Note that arguments need not be string instances. A nonstring argument x willbe displayed as str(x).Without any arguments, the command print() outputs a single newline character. The print function can becustomized through the use of the following keyword parameters:

• By default, the print function inserts a separating space into the output between each pair of arguments.The separator can be customized by providing a desired separating string as a keyword parameter, sep.For example, colon- separated output can be given as

1 pr i n t ( a , b , c , sep= ’ : ’ )

The separating string needs not to be a single character; it can be a longer string, and it can be the emptystring, sep=”, by causing successive arguments to be directly concatenated.

48 | 268






Python Recalls: Console Input and Output• By default, a trailing newline is output after the final argument. An alternative trailing string can be des-ignated e.g., as follows:

1 pr i n t ( a , b , c , end=” ” )

1 p r i n t ( a , b , c , end=” \ t ” )

• By default, the print function sends its output to the standard console. However, output can be directedto a file by indicating an output file stream using file as a keyword parameter.

49 | 268






Python Recalls: Console Input and OutputThe primarymeans for acquiring information from the user console is a built-in function named input. This func-tion displays a prompt, if given as an optional parameter, and then waits until the user enters some sequenceof characters followed by the return key. The formal return value of the function is the string of characters thatwere entered strictly before the return key (i.e., no newline character exists in the returned string).

When reading a numeric value from the user, a programmer must use the input function to get the string ofcharacters, and then use the int or float syntax to construct the numeric value that character string represents.That is, if a call to response = input() reports that the user entered the characters, 2013 , the syntax int(response)could be used to give the integer value 2013. It is quite common to combine these operations with a syntaxsuch as

1 year = i n t ( input ( ’ In what year were you born? ’ ) )

if we assume that the user will enter an appropriate response.

50 | 268






Python Recalls: Console Input and OutputBecause input returns a string as its result, the use of that function can be combined with the existing function-ality of the string class. For example, if the user enters multiple pieces of information on the same line, it iscommon to call the split method on the result, as in

1 reply = input ( ’ Enter x and y , separated by spaces : ’ )2 pieces = reply . s p l i t ( ) # re turns a l i s t of s t r i ngs separated by spaces3 x = f l o a t ( pieces [ 0 ] )4 y = f l o a t ( pieces [ 1 ] )

51 | 268






Python Recalls: Console Input and OutputHere is a simple, but complete, program that demonstrates the use of the input and print functions.

1 age = i n t ( input ( ’ Enter your age in years : ’ ) )2 max_heart_rate = 206.9 − (0.67*age )3 p r i n t ( ’ Your ta rget fa t −burning heart ra te i s ’ , max_heart_rate )

52 | 268






Python Recalls: FilesFiles are typically accessed in Python beginning with a call to a built-in function, named open, that returns aproxy for interactions with the underlying file. For example, the command

1 fp = open ( ’ sample . t x t ’ )

attempts to open a file named sample.txt, returning a proxy that allows read-only access to the text file.The open function accepts an optional second parameter that determines the access mode. The default modeis ’r’ for reading. Other common modes are ’w’ for writing to the file (causing any existing file with that name tobe overwritten), or ’a’ for appending to the end of an existing file.

53 | 268






Python Recalls: FilesWhen processing a file, the proxy maintains a current position within the file as an offset from the beginning,measured in number of bytes. When opening a file with mode ‘r’ or ‘w’, the position is initially 0; if opened inappend mode, ‘a’, the position is initially at the end of the file. The syntax

1 fp . close ( )

closes the file associated with proxy fp, ensuring that any written contents are saved. A summary of methodsfor reading and writing a file is given in the following table:

Calling Syntax Descriptionfp.read( ) Return the (remaining) contents of a readable file as a string.fp.readline( ) Return (remainder of ) the current line of a readable file as a string.fp.readlines( ) Return all (remaining) lines of a readable file as a list of strings.for line in fp: Iterate all (remaining) lines of a readable file.fp.write(string) Write given string at current position of the writable file.print(..., file=fp) Redirect output of print function to the file.

Table 3: Behaviors for interacting with a text file via a file proxy (named fp).

54 | 268






Python Recalls: FilesExample: Assume we have a file, named demofile.txt, located in the same folder as our Python program.

1 # demofile . t x t2 #3 # Hello ! Welcome to demofile . t x t4 # This f i l e i s fo r tes t i ng purposes .5 # Good Luck !

To open the file, use the built-in open() function:

1 f = open ( ” demofile . t x t ” , ” r ” )2 p r i n t ( f . read ( ) )3 f . c lose ( )

By default the read() method returns the whole text, but you can also specify how many characters you want toreturn:

1 f = open ( ” demofile . t x t ” , ” r ” )2 p r i n t ( f . read ( 5 ) )3 f . c lose ( )

55 | 268






Python Recalls: FilesIn this case the output will be just Hello.You can return one line by using the readline() method:

1 f = open ( ” demofile . t x t ” , ” r ” )2 p r i n t ( f . readl ine ( ) )3 f . c lose ( )

In this case the output is:

• Hello! Welcome to demofile.txt

56 | 268






Python Recalls: FilesYou can also read two lines:

1 f = open ( ” demofile . t x t ” , ” r ” )2 p r i n t ( f . readl ine ( ) )3 p r i n t ( f . readl ine ( ) )4 f . c lose ( )

In this case the output is:

• Hello! Welcome to demofile.txt• This file is for testing purposes.

57 | 268






Python Recalls: FilesFinally, you can loop through the file line by line:

1 f = open ( ” demofile . t x t ” , ” r ” )2 fo r x in f :3 p r i n t ( x )4 f . c lose ( )

The output will be then:

• Hello! Welcome to demofile.txt• This file is for testing purposes.• Good Luck!

58 | 268






Python Recalls: FilesTo write to an existing file, you must add a parameter to the open() function:

• ”a” - Append - to append supplementary information at the end of the file• ”w” - Write - to overwrite any previous existing content in the file

Example: we open the file ”demofile.txt” and append the string ”Now the file has more content!” to the end ofthe file:

1 f = open ( ” demofile . t x t ” , ” a ” )2 f . wr i te ( ”Now the f i l e has more content ! ” )3 f . c lose ( )45 #open and read the f i l e a f t e r the appending :6 f = open ( ” demofile . t x t ” , ” r ” )7 p r i n t ( f . read ( ) )

59 | 268






Python Recalls: FilesNow in this example we open ”demofile.txt” and completely overwrite its content:

1 f = open ( ” demofile3 . t x t ” , ”w” )2 f . wr i te ( ”Woops! I have deleted the content ! ” )3 f . c lose ( )45 #open and read the f i l e a f t e r the appending :6 f = open ( ” demofile3 . t x t ” , ” r ” )7 p r i n t ( f . read ( ) )

60 | 268






Python Recalls: IteratorsThe for-loop syntax begins as:

1 fo r element in i t e r ab le :

We noted that there are many types of objects in Python that qualify as being iterable. Basic container types,such as list, tuple, and set, qualify as iterable types. Furthermore, a string can give an iteration of its characters,a dictionary can give an iteration of its keys, and a file can give an iteration of its lines. User- defined types mayalso support iteration. In Python, the mechanism for iteration is based upon the following conventions:

• An iterator is an object that manages an iteration through a series of values. If variable, i, identifies an iter-ator object, then each call to the built-in function, next(i), gives a subsequent element from the underlyingseries, with a StopIteration exception raised to indicate that there are no further elements.

• An iterable is an object, obj, that gives an iterator via the syntax iter(obj).

61 | 268






Python Recalls: IteratorsBy these definitions, an instance of a list is an iterable, but not itself an iterator. With data = [1, 2, 4, 8], it isnot legal to call next(data). However, an iterator object can be gived with syntax, i = iter(data), and then eachsubsequent call to next(i) will return an element of that list.The for-loop syntax in Python simply automates this process, creating an iterator for the give iterable, and thenrepeatedly calling for the next element until catching the StopIteration exception.More generally, it is possible to createmultiple iterators based upon the same iterable object, with each iteratormaintaining its own state of progress. However, iterators typically maintain their state with indirect referenceback to the original collection of elements.For example, calling iter(data) on a list instance gives an instance of the list iterator class. That iterator does notstore its own copy of the list of elements. Instead, it maintains a current index into the original list, representingthe next element to be reported. Therefore, if the contents of the original list are modified after the iterator isconstructed, but before the iteration is complete, the iterator will be reporting the updated contents of the list.Python also supports functions and classes that give an implicit iterable series of values, that is, without con-structing a data structure to store all of its values at once. For example, the call range(1000000) does not returna list of numbers; it returns a range object that is iterable. This object generates the million values one at a time,and only as needed. Such a lazy evaluation technique has great advantage. In the case of range, it allows a loopof the form, for j in range(1000000):, to execute without setting aside memory for storing one million values.

62 | 268






Python Recalls: IteratorsAlso, if such a loop were to be interrupted in some fashion, no time will have been spent computing unusedvalues of the range.We see lazy evaluation used in many of Python’s libraries. For example, the dictionary class supports methodskeys(), values(), and items(), which respectively give a “view” of all keys, values, or (key,value) pairs within adictionary. None of these methods gives an explicit list of results. Instead, the views that are gived are iterableobjects based upon the actual contents of the dictionary.Anexplicit list of values fromsuch an iteration canbe immediately constructedby calling the list class constructorwith the iteration as a parameter. For example, the syntax list(range(1000)) gives a list instance with values from0 to 999, while the syntax list(d.values()) gives a list that has elements based upon the current values of dictionaryd. We can similarly construct a tuple or set instance based upon a given iterable.

63 | 268






Python Recalls: Language shortcutsIn some circumstances we might be willing to write something like this:

1 i f n>=0:2 param=n3 else :4 param=−n5 re su l t=foo (param ) # c a l l the funct ion foo with the parameter ‘ ‘ param ’ ’

We can compress these instructions in just one line:

1 re su l t = foo ( n i f n>=0 else −n )

64 | 268






Python Recalls: Language shortcutsIn other circumstances we might be willing to write something like this:

1 squares = [ ]2 fo r k in range (1 ,n+1) :3 squares . append ( k*k )

We can compress these instructions in just one line:

1 squares = [ k*k fo r k in range (1 ,n+1) ]

Example: suppose we want the list of factors for the numbers in the range 1…n+1. Then we can write

1 fac to r s = [ k fo r k in range (1 ,n+1) i f n % k ==0]

65 | 268






Python Recalls: Language shortcutsConsider this piece of code:

1 numbers = [2 . 5 , 3 , 4 , −5]2 va l=03 fo r k in numbers :4 va l+=k5 pr i n t ( va l )

Python offers a built in function to perform such a task, i.e., the function sum, whose syntax is the following:1 sum( i te rab le , s t a r t )

Iterable must have numbers as items; start is optional and if present is added to the sum of items of the iterable.If omitted, the default value of start is 0. The function start and items of the given iterable from left to right. Byusing this function, the above code would look like the following:

1 numbers = [2 . 5 , 3 , 4 , −5]2 va l = sum( numbers )3 p r i n t ( va l )

66 | 268






Python Recalls: 2D ArraysPython does not have a built-in support for arrays; lists are generally used tomimic arrays. Amono-dimensionalarray is therefore just a list, e.g.,

1 numbers = [2 . 5 , 3 , 4 , −5]

A two dimensional array (2D array) is an array within an array (or an array of arrays, if you like it). In a 2D arraythe position of a data element is referred by two indices instead of one. So it represents a table with n rows ofdata, each of which is an array not necessarily having the same length than the others. For example, considerthe action of sampling the temperature of a target place four times a day, every day. Sometimes, the samplerinstrument may fail to record data and provide e.g., the following information:

Day 1 − 11 12 5 2Day 2 − 15 6 10Day 3 − 10 8 12 5Day 4 − 12 15 8 6

Such data can be encoded in Python by means of the following 2D array:

67 | 268






Python Recalls: 2D Arrays

1 T = [2 [11 , 12 , 5 , 2 ] ,3 [15 , 6 ,10] ,4 [10 , 8 , 12 , 5 ] ,5 [12 ,15 ,8 ,6]6 ]

Suppose that you want to know the values of the temperature sampled the third day. You can carry out this taskas follows:

1 pr i n t ( T [ 2 ] )

Suppose now that you want to know the third value of the temperature sampled the second day. Then, youwrite

1 pr i n t ( T [ 1 ] [ 2 ] )

68 | 268






Python Recalls: 2D ArraysSuppose that you want just to print out the table T. You can carry out this task as follows:

1 T = [2 [11 , 12 , 5 , 2 ] ,3 [15 , 6 ,10] ,4 [10 , 8 , 12 , 5 ] ,5 [12 ,15 ,8 ,6]6 ]78 fo r row in T :9 fo r col in row :10 pr i n t ( col , end = ” ” )11 p r i n t ( )

The instruction

1 pr i n t ( col , end = ” ” )

write the (row,col)-th element of T followed by an empty space.

69 | 268






Python Recalls: 2D ArraysThe instruction

1 pr i n t ( )

print a new line once that all of the entries of a given row have been printed out.

70 | 268






Python Recalls: 2D ArraysNow, suppose that we want to modify T. For example assume that we want to change the first element of thesecond row by setting 3 instead of 15. Then, we write

1 T = [2 [11 , 12 , 5 , 2 ] ,3 [15 , 6 ,10] ,4 [10 , 8 , 12 , 5 ] ,5 [12 ,15 ,8 ,6]6 ]78 def P r in tT ( ) :9 fo r row in T :10 fo r col in row :11 pr i n t ( col , end = ” ” )12 p r i n t ( )1314 Pr in tT ( )1516 T [1 ] [0 ]=31718 pr i n t ( ” \n\n” )19

71 | 268






Python Recalls: 2D Arrays20 Pr in tT ( )

Then, the output will be

11 12 5 215 6 1010 8 12 512 15 8 6

11 12 5 23 6 1010 8 12 512 15 8 6

Suppose that we want to modify T by adding a new value to the second row, say 21. Then, we write

72 | 268







1 T = [2 [11 , 12 , 5 , 2 ] ,3 [15 , 6 ,10] ,4 [10 , 8 , 12 , 5 ] ,5 [12 ,15 ,8 ,6]6 ]78 def P r in tT ( ) :9 fo r row in T :10 fo r col in row :11 pr i n t ( col , end = ” ” )12 p r i n t ( )1314 T [ 1 ] . append(21 )15 Pr in tT ( )

Then, the output will be

73 | 268







11 12 5 23 6 10 2110 8 12 512 15 8 6

Note that now the 2D array is amatrix.We don’t really like the way in which T is shown. We would like to have it formatted properly. In this case wecan modify the function PrintT() as follows:

74 | 268







1 T = [2 [11 , 12 , 5 , 2 ] ,3 [15 , 6 ,10] ,4 [10 , 8 , 12 , 5 ] ,5 [12 ,15 ,8 ,6]6 ]78 def P r in tT ( ) :9 fo r row in T :10 fo r col in row :11 pr i n t ( ”%6.0 f ” %col , end=” ” )12 p r i n t ( )1314 T [ 1 ] . append(21 )15 Pr in tT ( )

The line

1 pr i n t ( ”%6.0 f ” %col , end=” ” )

75 | 268






Python Recalls: 2D Arraysaligns each entry of T in this way:

11 12 5 215 6 10 2110 8 12 512 15 8 6

If you want to add a given row in T and put it after the second one, you may do as follows:

1 T . i n s e r t (2 , [0 ,5 ,11 ,13 ] )

If you want to remove the second row of T, instead, you may do as follows:

1 del ( T [ 1 ] )

Standard insertion and deletion operations with lists allow to modify a given 2D array (or matrix).Suppose that we want to add a column to T having as generic entry -1. Then, we may write as follows:

76 | 268







1 fo r row in T :2 row . i n s e r t (2 , −1)

What should we write to remove the second column from T?

77 | 268






Exercises1. Write a short Python function, IsMultiple(n, m), that takes two integer values and returns True if n is a

multiple of m, that is, n = m*i for some integer i, and False otherwise.2. Write a short Python function, IsEven(k), that takes an integer value and returns True if k is even, and False

otherwise. However, your function cannot use the multiplication, modulo, or division operators.3. Write a short Python function, minmax(data), that takes a sequence of one or more numbers, and returns

the smallest and largest numbers, in the form of a tuple of length two. Do not use the built-in functionsmin or max in implementing your solution.

4. Write a short Python function that takes a positive integer n and returns the sum of the squares of all thepositive integers smaller than n.

5. Give a single command that computes the sum from the previous exercise relying on Python’s compre-hension syntax and the built-in sum function.

6. Write a short Python function that takes a positive integer n and returns the sum of the squares of all theodd positive integers smaller than n.

7. Give a single command that computes the sum from the previous exercise, relying on Python’s compre-hension syntax and the built-in sum function.

8. Python allows negative integers to be used as indices into a sequence, such as a string. If string s haslength n, and expression s[k] is used for index n ≤ k < 0, what is the equivalent index j ≥ 0 such that s[j]references the same element?

78 | 268






Exercises9. What parameters should be sent to the range constructor, to give a range with values 50, 60, 70, 80?10. What parameters should be sent to the range constructor, to give a range with values 8, 6, 4, 2, 0, −2, −4,

−6, −8?11. Demonstrate how to use Python’s list comprehension syntax to give the list [1, 2, 4, 8, 16, 32, 64, 128,

256].12. Python’s random module includes a function choice(data) that returns a random element from a non-

empty sequence. The random module includes a more basic function randrange, with parameterizationsimilar to the built-in range function, that return a random choice from the given range. Using only therandrange function, implement your own version of the choice function.

13. Write a pseudo-code description of a function that reverses a list of n integers, so that the numbers arelisted in the opposite order than they were before, and compare this method to an equivalent Pythonfunction for doing the same thing.

14. Write a short Python function that takes a sequence of integer values and determines if there is a distinctpair of numbers in the sequence whose product is odd.

15. Write a Python function that takes a sequence of numbers and determines if all the numbers are differentfrom each other (that is, they are distinct).

16. Demonstrate how to use Python’s list comprehension syntax to give the list [0, 2, 6, 12, 20, 30, 42, 56, 72,90].

79 | 268






Exercises17. Demonstrate how to use Python’s list comprehension syntax to give the list[ a , b , c ,…, z], but without

having to type all 26 such characters literally.18. Write a short Python program that takes two arrays a and b of length n storing int values, and returns the

dot product of a and b. That is, it returns an array c of length n such that c[i] = a[i]*b[i], for i = 0,…,n−1.

80 | 268





Problems & Algorithms

81 | 268



Problems & AlgorithmsA problem is a precise question about a specific system (i.e., a part of the universe) under study. A problemis said to be computable if the mentioned question, the corresponding answer, and the list of instructionsnecessary to answer the question can be encoded inside a computer.Example of a computable problem: Given a list of numbers a1, a2, . . . , an, find a reorganization of the entriesof such list, say a′

1, a′2, . . . , a′

n, such that a′1 ≤ a′

2 ≤ . . . ≤ a′n.

The system here is the given list of numbers and constitutes the input of the problem; note that the list can betrivially encoded in a computer (e.g., by using a programming language such as Python). The question is askingabout the existence of a reorganization of the entries of the input list so as to satisfy a specific property: the newlist must be sorted in non-descending order. In other words, the question is predicating (i.e., is precisely tellingus) about characteristic the output (i.e., the sorted list) must satisfy in order to be considered as an answer. Notethat, likewise the input, also the output can be encoded in a computer. The answer to this question could beobtained e.g., by (i) computing all of the permutations of the given list and (ii) storing the one whose entriesare non-descending ordered. Thus, because the input, the output and the instructions necessary to obtain theoutput are encodable in a computer, we can deduce that this problem definitely is computable.

82 | 268






Problems & AlgorithmsExample of a non-computable problem: Will that person ever love me?The system here is represented by the target person. However, the question is not well defined; in particular,it is not predicating about the specific characteristic or feature we should look for to compute the output. It isalso quite unclear how to encode both the input and the output in a computer. In this scenario, it is thereforenot possible to develop a list of instructions that enable us to solve the problem.

83 | 268






Problems & AlgorithmsComputer science is the study of problems that both are and are not computable.We say that we have an instance of a given problem, when the numbers of the list are specified.Example: Given a list of numbers 31, 41, 59, 26, 41, 58, 1, find a reorganization of the entries of such list, saya′

1, a′2, . . . , a′

n, such that a′1 ≤ a′

2 ≤ . . . ≤ a′n.

An algorithm is a precise sequence of instructions that operates on the given input to produce the desiredoutput. An algorithm is said to be correct if the output corresponds to the expected one for each possibleinput instance.We say that a correct algorithm solves the given problem.

84 | 268






Problems & AlgorithmsExercise. Given two string, say s1 and s2, consider the problem of detecting whether s1 is the anagram of s2,i.e., if s2 is simply a rearrangement of s1.For example, ’heart’ and ’earth’ are anagrams. The strings ’python’ and ’typhon’ are anagrams as well.For the sake of simplicity, assume that s1 and s2 are of equal length and that they are made up of symbols fromthe set of 26 lowercase alphabetic characters.Write a boolean function that takes s1 and s2 as input and returns True if s1 and s2 are anagrams and Falseotherwise.

85 | 268






Problems & AlgorithmsFirst solution approach. We check whether each character in the first string occurs in the second one. If it ispossible to “checkoff” each character, then the two strings must be anagrams.Checking off a character will be accomplished by replacing it with the special Python value None.Note that, as strings in Python are immutable, the first step in the process will be to convert the second string toa list. Each character from the first string can be checked against the characters in the list and if found, checkedoff by replacement.

86 | 268







1 def AnagramAlgorithm1 ( s1 , s2 ) :2 a _ l i s t = l i s t ( s2 )3 pos1 = 04 s t i l l _ o k = True5 while pos1 < len ( s1 ) and s t i l l _ o k :6 pos2 = 07 found = False8 while pos2 < len ( a _ l i s t ) and not found :9 i f s1 [ pos1 ] == a _ l i s t [ pos2 ] :10 found = True11 else :12 pos2 = pos2 + 113 i f found :14 a _ l i s t [ pos2 ] = None15 else :16 s t i l l _ o k = False17 pos1 = pos1 + 118 return s t i l l _ o k1920 pr i n t ( AnagramAlgorithm1 ( ’ abcd ’ , ’ dcba ’ ) )21

87 | 268






Problems & AlgorithmsSecond solution approach. The strings s1 and s2 are anagrams only if they consist of exactly the same charac-ters. So, if we begin by sorting each string alphabetically, from a to z, we will end up with the same string if theoriginal two strings are anagrams. The code below shows this solution. Again, in Python we can use the built-insort method on lists by simply converting each string to a list at the start.

1 def AnagramAlgorithm2 ( s1 , s2 ) :2 a _ l i s t 1 = l i s t ( s1 )3 a _ l i s t 2 = l i s t ( s2 )4 a _ l i s t 1 . so r t ( )5 a _ l i s t 2 . so r t ( )6 pos = 07 matches = True8 while pos < len ( s1 ) and matches :9 i f a _ l i s t 1 [ pos ] == a _ l i s t 2 [ pos ] :10 pos = pos + 111 else :12 matches = False13 return matches1415 pr i n t ( AnagramAlgorithm2 ( ’ abcde ’ , ’ edcba ’ ) )

88 | 268






Problems & AlgorithmsThird solution approach. Abrute force technique for solving a problem typically tries to exhaust all possibilities.For the anagram detection problem, we can simply generate a list of all possible strings using the charactersfrom s1 and then see if s2 occurs.There is a difficulty with this approach. When generating all possible strings from s1, there are n possible firstcharacters, n − 1 possible characters for the second position, n − 2 for the third, and so on. The total number ofcandidate strings is n(n1)(n2) · · · 3 ∗ 2 ∗ 1, which is exactly n!. Although some of the strings may be duplicates,the program cannot know this ahead of time and so it will still generate n! different strings.For the moment we do not show the source code of this solution approach, because it requires nontrivial algo-rithmic techniques such as recursion. You will learn how to develop it later in this course.

89 | 268






Problems & AlgorithmsFourth solution approach. Any two anagrams will have the same number of a’s, the same number of b’s, thesame number of c’s, and so on. In order to decide whether two strings are anagrams, wemay count the numberof times each character occurs. Since there are 26 possible characters, we can use a list of 26 counters, one foreach possible character. Each time we see a particular character, we will increment the counter at that position.In the end, if the two lists of counters are identical, the strings must be anagrams.

90 | 268







1 def AnagramAlgorithm4 ( s1 , s2 ) :2 c1 = [ 0 ] * 263 c2 = [0 ] * 264 base = ord ( ’ a ’ )5 fo r i in range ( len ( s1 ) ) :6 c1 [ ord ( s1 [ i ] ) − base ] += 17 c2 [ ord ( s2 [ i ] ) − base ] += 18 s t i l l _ o k = True9 fo r i in range ( len ( s1 ) ) :10 i f c1 [ i ]−c2 [ i ] != 0:11 s t i l l _ o k = False12 break13 return s t i l l _ o k14 pr i n t ( AnagramAlgorithm2 ( ’ abcde ’ , ’ edcba ’ ) )

91 | 268






Problems & AlgorithmsThis example shows that there may exist several possible distinct correct algorithms to solve a given problem.A question that naturally arises is: ”Which algorithm is better?”A possible approach to answer this question consists in understanding how many resources an algorithm mayuse. These resourses generally are:

• memory• CPUs• running time

Memory and CPUs are very important but somehow negligible if noncritical for a specific application. At theend of the day, we can always add more RAM, more storage and more CPUs if necessary.The story is differentwith the running time: Time is an intrinsic property of an algorithm that cannot be shortenedat will. Because algorithms should run as fast as possible, the running time appears to be the most criticalresource we should look at.Let’s compare the runtime of the previous three algorithms by executing the following code:Source code

92 | 268




https://colab.research.google.com/drive/1TCxxWFykTyNK3Cse6EPukKaaeQqvil5W#scrollTo=xN7xiz9wDGwk&line=62&uniqifier=1



Problems & AlgorithmsThe output on my machine is the following:

Executing algorithm 1. Answer : True . Required seconds : 0.0000079Executing algorithm 2. Answer : True . Required seconds : 0.0000048Executing algorithm 4. Answer : True . Required seconds : 0.0000072

If we run the code again we get


If we run the code once again we get


So which are the correct runtime values of these algorithms? Why do we see changes at each execution? Whyjust referring to the running time of my machine and not to the one of any other laptop or workstation?

93 | 268






Problems & AlgorithmsIn general, the running time of an algorithm

• is influenced by the scheduler of a given Operating System (OS) – this is why you may see changes onmultiple runs;

• may vary sensibly in function of the programming language used, compiler used, and OS used;• may dramatically change from computer to computer (because of the processor type, clock rate, memoryand disk latencies);

• generally increases with the input size and it may also vary for different inputs of the same size.

All other factors being equal, the running time of the same algorithm on the same input data will be smaller ifthe computer has, say, a much faster processor or if the implementation is done in a program compiled intonative machine code instead of an interpreted implementation.

94 | 268






Problems & AlgorithmsUnderstanding how to measure the running time of an algorithm therefore appears to be a very hard task.We would like to identify an ideal measure that

• may enable us to characterize an algorithm’s running time as a function of the input size;• may take into account any possible input instance of the specific problem under study;• may not incur noise or dependencies related to the specific hardware and software used to run that al-gorithm;

• may beperformedby studying a high-level description of the algorithm, i.e., without having to implementit.

But how to find it?

95 | 268






Problems & Algorithms: Focusing on the Worst-Case InputFirst things first.Let’s observe that an algorithmmay run faster on some inputs than it does on others of the same size. Thus, wemay wish to express the running time of an algorithm as the function of the input size obtained by taking theaverage over all possible inputs of the same size.Unfortunately, such an average-case analysis is typically quite challenging, as it usually requires to compute theexpected running times based on a given input distribution, which usually involves sophisticated probabilitytheory. Therefore, we will characterize running times in terms of the worst case, as a function of the input size,n, of the algorithm.Worst-case analysis is much easier than average-case analysis, as it requires only the ability to identify the worst-case input, which is often simple. Also, this approach typically leads to better algorithms. Making the standardof success for an algorithm to perform well in the worst case necessarily requires that it will do well on everyinput.

96 | 268






Problems & Algorithms: Counting Primitive OperationsIn order to analyze the running time of an algorithm without carrying out any experiment, we may perform adirect analysis on a high-level description of the algorithm (e.g., in the form of an actual code fragment). Forexample, consider the following code:

1 def MyDummyFunction ( myString ) :2 p r i n t ( myString )3 a = 14 b = 35 c = a+b6 return c

Now, we may define a set of primitive operations such as

• Assigning an identifier to an object• Determining the object associated with an identifier• Performing an arithmetic operation (for example, adding two numbers)• Comparing two numbers• Accessing a single element of a Python list by index• Calling a function (excluding operations executed within the function)

97 | 268






Problems & Algorithms: Counting Primitive Operations• Returning from a function.

These operations correspond to low-level instructions having an execution time that is approximatively constanton any hardware. Hence, instead of trying to determine the specific execution time of each primitive operationon a specific hardware, wemay simply count howmany primitive operations are executed within that algorithmand use this information as a measure of its overall running time.

98 | 268






Problems & Algorithms: Counting Primitive OperationsConsider again this code:

1 def MyDummyFunction ( myString ) :2 p r i n t ( myString )3 a = 14 b = 35 c = a+b6 return c

Here, all of the instructions are primitive and we can count a total of five primitives. Should we count them thisway?No: because we want a measure as universal as possible (i.e., independent of any factors that may influenceit) we generally use the asymptotic notation, also referred to as the “big-O” notation. This type of notationexpresses the running time of an algorithm as a function f of the input size n, i.e., f(n). Let’s define it.

99 | 268





Big-O Notation

100 | 268



Problems & Algorithms: Big-O notationLet f(n) and g(n) be functions mapping positive integers to positive real numbers. We say that f(n) isO(g(n))(reads “f(n) is order of g(n)”) if there is a real constant c > 0 and an integer constant n0 ≥ 1 such that

f(n) ≤ c · g(n) ∀ n ≥ n0.

Input Size

Runnin

g T

ime

cg(n)

f(n)

n0

101 | 268






Problems & Algorithms: Big-O notationThe big-O notation allows us to say that, up to a constant factor, a given function f(n) grows up in a way sloweror at most equal to another function g(n) and as n grows toward infinity.Example 1. The function f(n) = 8n + 5 is O(n).Proof. By definition, we need to find a real constant c > 0 and an integer constant n0 ≥ 1 such that 8n + 5 ≤ cnfor every integer n ≥ n0. It is easy to see that the values c = 9 and n0 = 5 satisfy the mentioned condition.Note that the choice c = 9 and n0 = 5 constitutes just one of infinitely many choices available; in fact, any othercombination between c and n0 for which 8n + 5 ≤ cn is satisfied is feasible for us. For example, we could havepicked c = 13 and n0 = 1.Example 2. The function f(n) = 2n3 + n2 is O(n3).Proof. As before, to prove the statement it is sufficient to show a constant c and n0 for which the definitionholds. We can take, for example, c = 3 and n0 = 0. Indeed,

∀ n ≥ n0, f(n) ≤ cg(n) ⇔ 2n3 + n2 ≤ 3n3

⇔ 2n3 + n2 − 2n3 ≤ 3n3 − 2n3

⇔ n2 ≤ n3

102 | 268






Problems & Algorithms: Big-O notationwhich always holds true: the square of a positive integer is always smaller than its cube.

The Big-O notation suggests a possible way to determine the running-time of an algorithm, which consists indetermining the number of primitives instruction f(n) executed by the algorithm when dealing with an inputof size n and then finding another function g(n) as close as possible to f(n) and such that f(n) is O(g(n)).How to do that?The definition ofO(·) allows us to identify several combination rules that enable us to simplify the determinationof the number of primitives instruction f(n) executed by an algorithm.In particular, we have the following mathematical rules:

1. Suppression of the multiplicative constants: A function f(n) = c · h(n), where c is a positive integerconstant is in O(h(n)). To prove it, it is sufficient to take n0 = 0 and c = c. For example, 3n2 ∈ O(n2).

2. Most significant term: in the context of the Big-O notation, given a sum of functions, we can just look atthe most significant term. For example, f(n) = n + log(n) is O(n). When dealing with polynomials, wecan just take the monomial of higher degree. For example, 3n2 + 2n + 3 is O(3n2), hence O(n2) for thesuppression of the multiplicative constants.

103 | 268






Problems & Algorithms: Big-O notationIn addition, the following rules enable us to analyze a program step-by-step:

• Elementary Instructions: The standard operations +, −, ∗, /, the assignment operations :=, read, print,and elementary tests (e.g., i = j) are O(1), i.e., constant.

The following are standard operations that take a constant time O(1).

Example1 pr i n t ( myString )

Example1 a=1

Example1 c = a * b

104 | 268






Problems & Algorithms: Big-O notation• Sequences. If I1 is a sequence of instruction inO(f1(n)) and I2 is a sequence of instructions inO(f2(n)),then

1 I12 I2

is O(f1(n) + f2(n)).

The following example shows a sequence of instructions that take O(1):

1 pr i n t ( myString )2 a=13 b=44 c = a * b

105 | 268






Problems & Algorithms: Big-O notation• Loop. If I is a sequence of instructions in O(f(n)) and T is a test in O(g(m)), then the loop

1 while ( T ) :2 I

is O(l × (f(n) + g(m))), where l is the number of times in which the loop is executed.

The following example shows a sequence of instructions that take O(n):

1 i = 12 while i < n :3 p r i n t ( i )4 i += 1

106 | 268






Problems & Algorithms: Big-O notation• Test. If I1 is a sequence of instructions in O(f1(n)), I2 is a sequence of instructions in O(f2(n)) and T isa test in O(g(m)), then the test

1 i f T :2 I13 else :4 I2

is in O(max{f1(n), f2(n)} + g(m)).

The following example shows a sequence of instructions that take O(1):

1 a = 2002 b = 333 i f b > a :4 p r i n t ( ”b i s greater than a ” )5 else :6 p r i n t ( ” a i s greater than or equal to b” )

107 | 268






Problems & Algorithms: Big-O notationExercise: Compute the running-time of this algorithm

1 i = 12 n = input ( ” give me a number : ” )3 while ( i <= i n t ( n ) ) :4 j =15 while ( j <= i n t ( n ) ) :6 i f i== j :7 p r i n t ( i , j )8 j= j +19 i= i +1

108 | 268








Let’s start by analyzing the content of the most internal loop j ≤ n. This loop contains an if in the body whichin turn contains a print instruction. The print instruction trivially is O(1). The test i = j is also O(1). For the“test rule” we have then that the if block is O(1 + 1) ∼ O(2) ∼ O(1) for the suppression of the multiplicativeconstants.

109 | 268








The if instruction is followed by the assignment j = j+1, which isO(1). By the “sequence of instruction rule” wethat that the if block plus the assignment are O(1). The overall loop then executes n times an O(1) instructionset. The test of the loop is O(1). Thus, for the “Rule of the Loop” the overall loop is O(n).

110 | 268








Let’s now have a look to the most external while loop. In this body we have:

• an O(1) instruction (j = 1)• an O(n) while loop• an O(1) instruction (i = i + 1).

Then, for the “sequence of instruction rule” we have that the body of the loop takesO(1 + n + 1) ∼ O(n + 2) ∼O(n). Now, the test of the loop (i.e., i <= int(n)) is O(1). and the loop itself is repeated n times. Thus, for the“rule of the loop” the overall loop takes O(n × (n + 1)) ∼ O(n2 + n) ∼ O(n2).

111 | 268








Finally, the algorithm contains two further O(1) instructions, i.e., the first and the second. Then, by the “se-quence of instruction rule” the algorithm is O(2 + n2) ∼ O(n2).

112 | 268






Problems & Algorithms: Big-O notationImportant: The typical running-time (also called order of complexity) of an algorithm can be

• Constant, i.e., O(1);• Logarithmic, i.e., O(log n);• Linear, i.e., O(n);• n log n, i.e., O(n log n);• Quadratic, i.e., O(n2);• Cubic, i.e., O(n3);• Polynomial, i.e., O(nα), for some fixed scalar α > 1;• Exponential, i.e., O(cn), for some fixed scalar α > 1;• Factorial, i.e., O(n!);• Double Factorial, i.e., O(n!!);• n-Power n, i.e., O(nn);

113 | 268






Problems & Algorithms: Big-O notationLet’s see a couple of examples of dummy algorithms characterized by (some of) the above order of complexity.Specifically, a dummy example of an algorithm with a constant order of complexity is the following:

1 def MyMin ( a , b ) :2 i f a<=b:3 return a4 return b5 pr i n t ( ” the minimum between 13 and 7 i s : ” , MyMin(13 ,7 ) )

114 | 268






Problems & Algorithms: Big-O notationAn example of an algorithm with a linear order of complexity is the following:

1 MyList =[15 ,7 ,21 ,4]23 def FindMin ( ) :4 mymin=100000000000 #A very big number5 fo r i in MyList :6 i f i < mymin :7 mymin= i8 return mymin910 pr i n t ( ” the minimum of MyList i s : ” , FindMin ( ) )

115 | 268






Problems & Algorithms: Big-O notationAn example of an algorithm with a quadratic order of complexity is the following:

1 MyList =[15 ,7 ,21 ,4]23 def T r u l l a l l a ( ) :4 fo r i in MyList :5 fo r j in MyList :6 i f ( i < j ) :7 p r i n t ( ” i : ” , i , ” \ t j : ” , j )8 return910 T r u l l a l l a ( )

We will see, in a couple of slides, some important examples of algorithms with a quadratic order of complexity.

116 | 268






Problems & Algorithms: Big-O notationAn example of an algorithm with a factorial order of complexity is the following:

1 def f ibonacc i ( n ) :2 i f n <= 1:3 return n4 return f ibonacc i ( n−1) + f ibonacc i ( n−2)

This example uses the fundamental concept of recursion. We will discuss it later on in this course.

117 | 268






Problems & Algorithms: Big-O notationTo have an idea of the practical behavior of (some of) these orders of complexities, look at this picture:

Hence, when dealing with practical problems characterized by a large input size, we want to search for algo-rithms that run as fast as possible, i.e., that are characterized by order of complexities as low as possible.

118 | 268






Problems & Algorithms: ExercisesExercise: Compute the running-time of this algorithm

1 i=12 j=13 n = i n t ( input ( ” give me a number : ” ) )4 while ( i <= n ) :5 p r i n t ( i , j )6 j= j +17 i= i +1

119 | 268






Problems & Algorithms: ExercisesExercise: Compute the running-time of this algorithm

1 n = i n t ( input ( ” give me a number : ” ) )2 i f ( n=1) :3 p r i n t ( n )4 else :5 i =16 while ( i <= n ) :7 i= i +1

120 | 268






Problems & Algorithms: Exercises1. Given a list of integers L and an integer k, write an algorithm that checks whether k is in L.2. Given a list of integers L, sorted in ascending order, and an integer k, write an algorithm that checks

whether k is in L. In particular, show how you can exploit the fact that L is sorted to speed up computa-tions.

3. Given a list of integers L, write an algorithm that returns the sum of the elements of L.4. Given a list of integers L, write an algorithm that rotates k times its elements. For example, if L =

[10, 20, 30, 40, 50, 60], by rotating of 2 position the elements of L provides [30, 40, 50, 60, 10, 20].5. Given a string s of length n, write an algorithm that shifts of k positions each character of s elements. For

example, if s =”abc”, by rotating of 2 position each character of s provides ”cdf”.6. Given a list of positive and negative integers L, find a contiguous sublist whose sum of elements is maxi-

mum.

121 | 268





Recursion

122 | 268



RecursionOne way to describe repetition within a computer program is the use of loops, such as Python’s while-loop andfor-loop constructs previously described.An entirely different way to achieve repetition is through a process known as recursion. Specifically, recursion(or autopoiesis) — from Greek αυτω (i.e., auto, “in itself”) and ποίησις (poiesis), i.e., “creation or production” —refers to the ability of defining something (e.g., an entity, a process or a function) in terms of itself. In computerscience, recursion refers either to a technique by which a function makes one or more calls to itself duringexecution, or to a data structure that relies upon smaller instances of the very same type of structure in itsrepresentation.There are many examples of recursion in art and nature. For example, fractal patterns are naturally recursive. Aphysical example of recursion used in art is in the Russian Matryoshka dolls. Each doll is either made of solidwood, or is hollow and contains another Matryoshka doll inside it.In computing, recursion provides an elegant and powerful alternative to perform repetitive tasks. In fact, a fewprogramming languages (e.g., Scheme, Smalltalk) do not explicitly support looping constructs and instead relydirectly on recursion to express repetition. Most modern programming languages support functional recursionusing the identical mechanism that is used to support traditional forms of function calls. When one invocationof the function make a recursive call, that invocation is suspended until the recursive call completes.

123 | 268






RecursionRecursion is an important technique in the study of data structures and algorithms. We will often use it in thiscourse. Here, we introduce it by starting with a classical example. Specifically, consider the definition of thefactorial of a positive integer n:

n! ={

1 if n = 01 · 2 · 3 . . . · (n − 2) · (n − 1) · n if n ≥ 1.

For example 5! = 5 · 4 · 3 · 2 · 1 = 120. The factorial function is one of the most important one in combinatoricsand computer science because it encodes the number of ways in which n distinct items can be arranged intoa sequence (i.e., the number of permutations of n items). For example, the three characters a, b, and c can bearranged in 3! = 3 · 2 · 1 = 6 ways: abc, acb, bac, bca, cab, and cba.It is worth noting that there exists a natural alternative definition of the factorial function which is recursive.Specifically, first observe that 5! = 5 · (4 · 3 · 2 · 1) = 5 · 4!. Then, for a positive integer n, we can define n! to ben · (n1)!. This recursive definition can be formalized as follows:

n! ={

1 if n = 0n · (n − 1)! if n ≥ 1.

124 | 268






RecursionThe first condition is called base case and is defined non-recursively in terms of fixed quantities. The base casehas the fundamental function of stopping the loop.The second condition is called recursive step and is defined by appealing to the definition of the function beingdefined.Observe that in the recursive step the appealing is performed on a argument that has “size” smaller than theprevious one. Hence, the recursive step has the fundamental function of decomposing the problem (in thiscase the definition of the recursive function) into subproblems of smaller sizes.Let’s see how can we implement the recursive factorial in Python.

1 def f a c t o r i a l ( n ) :2 i f n==0:3 return 14 else :5 return n* f a c t o r i a l ( n−1)

125 | 268






RecursionNote that this function does not use any explicit loops. Repetition is provided by the repeated recursive invo-cations of the function itself.There is no circularity in this definition, because each time the function is invoked, its argument is smaller byone, and when a base case is reached, no further recursive calls are made.We illustrate the execution of a recursive function using a recursion trace. Each entry of the trace corresponds toa recursive call. Each new recursive function call is indicated by a downward arrow to a new invocation. Whenthe function returns, an arrow showing this return is drawn and the return valuemay be indicated alongside thisarrow. Then, an example of such a trace for the factorial function is the following:

return 4 * 6 = 24

factorial(4)

factorial(1)

factorial(0)

factorial(3)

factorial(2)

return 1

return 1 * 1 = 1

return 2 * 1 = 2

return 3 * 2 = 6

126 | 268






RecursionA recursion trace closely mirrors the programming language’s execution of the recursion. In Python, each timea function (recursive or otherwise) is called, a structure known as an activation record or frame is created to storeinformation about the progress of that invocation of the function. This activation record includes a namespacefor storing the function call’s parameters and local variables, and information about which command in thebody of the function is currently executing.When the execution of a function leads to a nested function call, the execution of the former call is suspendedand its activation record stores the place in the source code at which the flow of control should continue uponreturn of the nested call. This process is used both in the standard case of one function calling a differentfunction, or in the recursive case in which a function invokes itself. The key point is that there is a differentactivation record for each active call.

127 | 268






RecursionIn order to learn to mastering recursion, we will now review a number of typical problems that can be tackledrecursively. The first of them is the problem of computing the sum of the first n number of a list S.

1 S=[4 , 3 , 6 , 2 , 8]2 def l inear_sum ( S , n ) :3 i f n == 0:4 return 05 else :6 return linear_sum ( S , n−1) + S [ n−1]

return 15 + S[4] = 15 + 8 = 23

linear sum(S, 5)

linear sum(S, 4)

linear sum(S, 3)

linear sum(S, 2)

linear sum(S, 1)

linear sum(S, 0)

return 0

return 0 + S[0] = 0 + 4 = 4

return 4 + S[1] = 4 + 3 = 7

return 7 + S[2] = 7 + 6 = 13

return 13 + S[3] = 13 + 2 = 15

128 | 268






RecursionWrite a recursive function to reverse a list L of n numbers:

1 def reverse ( L , s t a r t , stop ) :2 i f s t a r t < stop − 1: # i f a t l e a s t 2 elements :3 temp = L [ s t a r t ] # swap4 L [ s t a r t ] = L [ stop −1] # the f i r s t5 L [ stop −1] = temp # and l a s t6 reverse ( L , s t a r t +1, stop −1) # recur on res t

6

3 6 2 8 9 5

5 9 8 2 6 3 4

5 3 6 2 8 9 4

5 9 8 2 6 3 4

5 9 6 2 8 3 4

50 1 2 3 4

4

129 | 268






RecursionWrite a recursive function to compute the xn, for some integer n > 0:

1 def power ( x , n ) :2 i f n == 0:3 return 14 else :5 return x * power ( x , n−1)

130 | 268






Recursion: Binary SearchBinary search is a classical (and yet one of the most important) recursive algorithms used to efficiently locate atarget value within a sorted sequence of n elements. Its computational complexity is the main reason why weoften store data in sorted order.When a list L is unsorted, the standard approach to search for a target value is to use a loop to examine ev-ery element, until either finding the target or exhausting the data set. This is known as the sequential searchalgorithm. This algorithm runs inO(n) time (i.e., linear time) since every element is inspected in the worst case.When the list L is sorted and indexable, a much more efficient algorithm algorithm is possible. Specifically, forany index j, we know that all the values stored at indices 0, . . . , j1 are less than or equal to the value at index j,and all the values stored at indices j + 1, . . . , n1 are greater than or equal to that at index j.This observation allows us to quickly “home in” on a search target using a variant of the children’s game “high-low”: we call an element of the list a candidate if, at the current stage of the search, we cannot rule out that thisitem matches the target. The algorithm maintains two parameters, low and high, such that all the candidateentries have index at least low and at most high. Initially, low = 0 and high = n − 1. We then compare the targetvalue to the median candidate, that is, the item data[mid] with index

mid = ⌊(low + high)/2⌋.

131 | 268






Recursion: Binary SearchWe consider three cases

1. If the target equals L[mid], then we have found the item we are looking for, and the search terminatessuccessfully.

2. If target < L[mid], then we recur on the first half of the list, that is, on the interval of indices from low tomid − 1.

3. If target > L[mid], then we recur on the second half of the list, that is, on the interval of indices from mid+ 1 to high.

An unsuccessful search occurs if low > high, as the interval [low, high] is empty.This idea is core of the binary search algorithm. Let’s see its implementation in Python.

132 | 268






Recursion: Binary Search

1 # Return True i f ta rget i s found in indicated port ion of a Python l i s t . False otherwise2 def binary_search ( L , target , low , high ) :3 i f low > high :4 return False # i n t e r v a l i s empty ; no match5 else :6 mid = ( low + high ) / / 27 i f ta rget == L [mid ] : # found a match8 return True9 e l i f ta rget < L [mid ] :10 # recur on the port ion l e f t of the middle11 return binary_search ( L , target , low , mid − 1)12 else :13 # recur on the port ion r igh t of the middle14 return binary_search ( L , target , mid + 1 , high )

Whereas sequential search runs in O(n) time, the more efficient binary search runs in O(log n) time. This is asignificant improvement, given that if n is one billion, log n is only 30.

133 | 268






Recursion: Binary SearchFor example, consider the following list L of numbers and suppose that you want to check whether 22 is con-tained in it.

mid

high

highlow

low mid

low mid

low=mid=high

high

14 19 22 25 27 28 33 37

6 7 8 9 10 11 12 13 14 15

7542 98

92 4 5 7 8 12 14 17

37332827252219

92 4 5 7 8 12 14 17 19 22 25 27 28 33 37

19 22 25 27 28 33 37

50 1 2 3 4

171412

92 4 5 7 8 12 17

134 | 268






Recursion: The Tower of HanoiIn 1883 the French mathematician Édouard Lucas designed a nice puzzle which appeared in the third volumeof “Récréations mathématiques” in 1892. The puzzle was named “The Tower of Hanoi” and was introduced asa problem posed by one of his friends, N. Claus de Siam (anagram of Lucas d’Amiens — Amiens was the city inwhich Lucas born) professor at the college Li-Sou-Stian (anagram of Saint Louis, lyceum where Lucas taught)[Wikipedia].The problem arises from a legend about an Indian temple in Kashi Vishwanath which contains a large roomwith three time-worn posts in it, surrounded by 64 golden disks. Brahmin priests, acting out the command ofan ancient prophecy, have been moving these disks in accordance with the immutable rules of Brahma sincethat time. According to the legend, when the last move of the puzzle is completed, the world will end.If the legend were true, and if the priests were able tomove disks at a rate of one per second, using the smallestnumber of moves it would take them 2641 seconds or roughly 585 billion years to finish, which is about 42 timesthe current age of the universe.

135 | 268






Recursion: The Tower of HanoiThe puzzle proposed by Lucas can be simplified as follows. Consider 3 rods and 4 of disks of different sizes,which can slide onto any rod. The puzzle starts with the disks in a neat stack in ascending order of size on onerod, the smallest at the top, thus making a conical shape. The objective of the puzzle is to move the entire stackto another rod, obeying the following simple rules:

1. Only one disk can be moved at a time.2. Each move consists of taking the upper disk from one of the stacks and placing it on top of another stack

or on an empty rod.3. No larger disk may be placed on top of a smaller disk.

136 | 268






Recursion: The Tower of HanoiLet’s see an implementation in Python of a possible solution algorithm when assuming just 3 rods and 3 disks.

1 def move( n , source , au x i l i a r y , ta rget ) :2 i f n == 1:3 p r i n t ( source , ” −> ” , ta rget )4 else :5 move( n − 1 , source , target , a u x i l i a r y )6 p r i n t ( source , ” −> ” , ta rget )7 move( n − 1 , aux i l i a r y , source , ta rget )89 move(3 , ’A ’ , ’B ’ , ’C ’ )

whose output is:

1 A −> C2 A −> B3 C −> B4 A −> C5 B −> A6 B −> C7 A −> C

137 | 268






Exercises1. Describe a recursive function for converting a string of digits into the integer it represents. For example,

”13531” represents the integer 13531.2. Write a short recursive Python function that finds theminimumandmaximumvalues in a sequencewithout

using any loops.3. Give a recursive algorithm to compute the product of two positive integers, m and n, using only addition

and subtraction.4. Write a recursive function that will output all the subsets of a set of n elements (without repeating any

subsets).5. Write a short recursive Python function that takes a character string s and outputs its reverse. For example,

the reverse of ”pots&pans” would be ”snap&stop”.6. Write a short recursive Python function that determines if a string s is a palindrome, that is, it is equal to

its reverse. For example, ”racecar”, ”anna”, ”gohangasalamiimalasagnahog” are palindromes.7. Use recursion to write a Python function for determining if a string s has more vowels than consonants.8. Write a short recursive Python function that rearranges a list of positive integers so that all the even num-

bers appear before all the odd ones.

138 | 268





Sorting

139 | 268



SortingSorting is the process of rearranging data in such a way that they appear either in ascending or descendingorder with respect to a specific ordering criterion. This operation is fundamental in a multitude of situations(think, e.g., in a typical situation in which you want to order a given column of an excel file according to somecriterion; or in a way you just wish to order by name the passengers of a given flight). A sorting is typically usedto speed up the subsequent manipulations of data in a given data structure.There exist a very large literature on sorting algorithms. Here we will just focus on “Bubble-Sort” and “MergeSort” and for the sake of ease we will learn to apply them on the elements of a given list L.

140 | 268






SortingAs mentioned before, the most common sorting algorithms use a specific criterion to ordering data. For exam-ple, suppose you have the following list of integers

1 L=[85 ,24 ,63 ,45 ,17 ,31 ,96 ,50]

and that you want to sort it in ascending order. In this case, the implicit criterion used is the fact that a givennumber i in the list L must precede another number j if i ≤ j.A situation in which the criterion becomes more explicit is the following: your list L contains strings, e.g.,

1 L=[ ”Rome” , ” Brusse ls ” , ” Pa r i s ” , ” Zur ich ” , ”New York ” , ” Adelaide ” ]

and you want to order these cities in such a way that the following ordering criterion is satisfied:ordering criterion: A city i in L must precede a city j if the first letter of i is lexicographically smaller than orequal to the corresponding one of j.

141 | 268






SortingThese examples suggest a fundamental idea to sort the corresponding list L, which consists in

1. fixing any pair of elements i and j in L;2. comparing i and j in function of the given ordering criterion;3. swapping the position of i and j in L if the comparison should highlight that i > j.

Example. Pick i = 24 and j = 17 in the following list L

1 L=[85 ,24 ,63 ,45 ,17 ,31 ,96 ,50]

because j = 17 < 24 = i then we have to swap their respective positions by obtaining the following new list

1 L=[85 ,24 ,63 ,45 ,17 ,31 ,96 ,50]

We should iterate this operation until any pair of elements i and j in L satisfies i ≤ j.Observation 1: if it should occur that i ≤ j, then we do NOT perform any swap operation.Observation 2: When dealing with the list of string in the second example, i.e.,

142 | 268






Sorting

1 L=[ ”Rome” , ” Brusse ls ” , ” Pa r i s ” , ” Zur ich ” , ”New York ” , ” Adelaide ” ]

some built in functions of Python may prove very useful to sorting purposes. Specifically, recall that ord and chrrelate characters and their integer code points. For example, ord( A ) is 65 and chr(65) is A.Then, we should compare any pair of first letters of cities i and j in L and swap the respective city positions if

ord(L[i][0]) > ord(L[j][0]).

Observation 3: if we must iterate the swap operation until any pair of elements i and j in L satisfies i ≤ j thendoes this means that we need to enumerate all of the permutations of L?This is not a trivial question: listing the permutations of a list of n elements is O(n!). This means e.g., that ifn = 50 then we need to enumerate

30414093201713378043612608166064768844377641568960512000000000000

143 | 268







Sortingpermutations, which is intractable.The following Source code provide you with of an idea of what intractability means.

144 | 268




http://perso.uclouvain.be/daniele.catanzaro/SupportingMaterial/UCL/Teaching/CodingProject/Codes/ch04/Permutation.py



Sorting: Bubble-SortLuckily, we do not need to enumerate all of the permutation of L to sort its elements. In fact, when dealing e.g.,with positive integers (or any other thing that can be reduced to common numbers) the natural order definedby using the < operator enjoys the following properties:

• the irreflexive property, i.e. k ≮ k, for any positive integer k;• the transitive property, i.e., if k1 < k2 and k2 < k3, then k1 < k3, for any positive integers k1, k2, k3.

The transitive property is important as it allows us to infer the outcome of certain comparisonswithout perform-ing those comparisons, thereby leading to more efficient algorithms. The following algorithm exploits theseproperties to sort a given input list:

145 | 268






Sorting: Bubble-Sort

1 L=[21 ,13 ,17 ,4 ,9 ,21]23 def BubbleSort ( l i s t ) :4 fo r i in range ( len ( l i s t ) −1) :5 fo r j in range ( i +1, len ( l i s t ) ) :6 i f l i s t [ j ] < l i s t [ i ] :7 temp = l i s t [ j ]8 l i s t [ j ] = l i s t [ i ]9 l i s t [ i ] = temp10 return l i s t1112 pr i n t ( ” Input l i s t : ” , L )13 p r i n t ( ” Sorted l i s t : ” , BubbleSort ( L ) )

Let’s analyze this code. The outer loop runs over all the elements i of L that go from position 0 to position n − 1of the list. During each iteration the algorithm compares the value of i with the value of each element j of Llocated from position i + 1 till position n. If it occurs that i > j, then a swap of i with j is performed. The twoloops combined ensure that at the end of the function BubbleSort the new list L′ satisfies L′[i] ≤ L′[j] for alli < j ∈ 1 . . . n. The computational cost of Bubble-Sort is O(n2).

146 | 268






Sorting: Bubble-SortBubble-Sort is a classic example of internal sorting: all of the elements can be loaded into memory and thesorting is performed in memory. However, in some circumstances n is so big that loading the whole elementsinmemory is just impossible. In these situations we need to use superior algorithms able to perform an externalsorting, i.e., algorithms able to sort chunks of data at time. An example of these algorithms isMerge-sort.

147 | 268






Sorting: Merge-sortMerge-sort is a recursive algorithm that applies a divide et impera approach. This approach consists of thefollowing three steps:

1. Divide: If the input size is smaller than a certain threshold (say, one or two elements), solve the problemdirectly using a straightforward method and return the solution so obtained. Otherwise, divide the inputdata into two or more disjoint subsets.

2. Impera: Recursively solve the subproblems associated with the subsets.3. Combine: Take the solutions to the subproblems andmerge them into a solution to the original problem.

We will first describe Merge-sort at a high level and then we will present the relative implementation.

148 | 268






Sorting: Merge-sortIn order to sort a sequence L of n elements (integers) by using the divide et impera approach, Merge-sortproceeds as follows:

1. Divide: If L has zero or one element, return L immediately as L is already sorted. Otherwise L has atleast two elements; then, remove all the elements from L and put them into two lists, say L1 and L2,each containing about half of the elements of L; that is, L1 contains the first ⌊n/2⌋ elements of L, and L2contains the remaining ⌈n/2⌉ elements.

2. Impera: Recursively sort sequences L1 and L2.3. Combine: Put back the elements into L by merging the sorted list L1 and L2.

149 | 268






Sorting: Merge-sortWe can visualize an execution of Merge-sort by means of the following image.

45

85

5031

24 17 3163 45 96 50

45632485 17 31 96 50

17 31 96 5085 24 63 45

85 63 17 9624

150 | 268






Sorting: Merge-sortEach sublist represents a recursive invocation (or call) of Merge-sort. The bottom-most lists are associated withindividual elements of L, i.e., instances of the algorithm that make no recursive calls.

31 50

24

96

85 17 3145 63 50 96

17 31 50 9624 45 63 85

45312417 50 63 85 96

85 63 1724 45

The two figures above summarize an execution of Merge-sort, by showing the input and the output sequencesprocessed during the recursion.

151 | 268






Sorting: Merge-sortThe following sequence of figures helps to understand step-by-step the overall sorting process.

9645632485 17 31 50

152 | 268







31 96 50

85 24 63 45

17

153 | 268







50

4563

85

17 31 96

24

154 | 268







4563

17

24

85

31 96 50

155 | 268







45

50

85

63

24

17 31 96

156 | 268







24

63

50963117

85

45

157 | 268







24 85 63

50963117

45

158 | 268







63 45

17 31 96 50

24 85

159 | 268







45

63

17 31 96 50

24 85

160 | 268







63

45

17 31 96 50

24 85

161 | 268







45 63

17 31 96 50

24 85

162 | 268







17 31 96 50

24 6345 85

163 | 268







24

17 31 96 50

856345

164 | 268







17 31 50 96

85634524

165 | 268







63 85 96501724 3145

166 | 268







45312417 50 63 85 96

167 | 268






Sorting: Merge-sortThe figures also help us to analyze the running time of the algorithm. In particular, since the size of the inputsequence roughly halves at each recursive call of Merge-sort, all of the terminal instances are reached in exactlylog2 n calls. So we can expect an overall computational cost of at least O(log n).Now, the problem is to understandwhat happens during themergingprocess, i.e., what happens in the functionthat is responsible for the subtask of merging two previously sorted sequences, L1 and L2, with the outputcopied into L. Let’s analyze this code.

1 def merge ( L1 , L2 , L ) :2 i = j = 03 while i + j < len ( L ) :4 i f j == len ( L2 ) or ( i < len ( L1 ) and L1 [ i ] < L2 [ j ] ) :5 L [ i+ j ] = L1 [ i ] # copy the i t h element of L1 as next item of L6 i += 17 else :8 L [ i+ j ] = L2 [ j ] # copy the j t h element of L2 as next item of L9 j += 1

168 | 268






Sorting: Merge-sortWe copy one element during each pass of the while loop, conditionally determining whether the next elementshould be taken from L1 or L2. We illustrate a step of the merge process in the following figure.

S1

S

S2

0 1 2 3 4 65

2518 19 229 10

92

j

i

i + j

3

1 2 3 4 65

11 12 142 5 8

0

15

0 1 2 3 4 6 7 8 9 105 11 12 13

83 5

L1

L2

L

169 | 268






Sorting: Merge-sortDuring the process, index i represents the number of elements of L1 that have been copied to L, while indexj represents the number of elements of L2 that have been copied to L. Assuming that L1 and L2 both haveat least one uncopied element, we copy the smaller of the two elements being considered. Since i + j objectshave been previously copied, the next element is placed in L[i + j]. (For example, when i + j is 0, the nextelement is copied to L[0]). If we reach the end of one of the sequences, we must copy the next element fromthe other.

S

S1

S2

5

0 1 2 3 4 65

2518 19 223 10

92

i

j

i + j

10

9

1 2 3 4 65

11 12 142 5 8

0

15

0 1 2 3 4 6 7 8 9 105 11 12 13

83

L1

L2

L

170 | 268






Sorting: Merge-sortThe computational complexity of the merge function isO(n) in the worst case. This cost must be payed at eachmerging step of the recursion (in particular at the steps located at the same level of the bifurcation), leading toan overall computational complexity O(n log n).

Height Time per level

Total time: O(n logn)

O(n)

O(n)

O(logn)

O(n)

n

n/2

n/4n/4n/4n/4

n/2

171 | 268






The Algortithm Merge-sort1 def merge ( L1 , L2 , L ) :2 i = j = 03 while i + j < len ( L ) :4 i f j == len ( L2 ) or ( i < len ( L1 ) and L1 [ i ] < L2 [ j ] ) :5 L [ i+ j ] = L1 [ i ] # copy the i t h element of L1 as next item of L6 i += 17 else :8 L [ i+ j ] = L2 [ j ] # copy the j t h element of L2 as next item of L9 j += 11011 def merge_sort ( L ) :12 n = len ( L )13 i f n < 2:14 return # the l i s t i s already sorted1516 mid = n / / 2 # otherwise div ide :17 L1 = L [ 0 :mid ] # copy of f i r s t ha l f18 L2 = L [mid : n ] # copy of second ha l f19 merge_sort ( L1 ) # conquer : so r t copy of f i r s t ha l f20 merge_sort ( L2 ) # conquer : so r t copy of second ha l f21 merge ( L1 , L2 , L ) # merge sorted halves back in to L

172 | 268





Brute Force Search & Backtracking

173 | 268



Brute Force Search & Backtracking: Crockrock’s LabyrinthThe brute-force search or exhaustive search is a search paradigm that consists in systematically enumeratingall of the possible candidate solutions to a given problem and checking whether each candidate satisfies theproblem’s statement or not.An example of an application of the brute-force search is provided by the following Knapsack Problem (KP):Given apositive constantB anda setN ofn ≥ 3 items, eachhavingboth a value vali anda costwti, i = 1, . . . , n,the (KP) is to find the subset of N having maximum value (i.e., maximum sum of the values associated to theinvolved items) and such that the sum of the corresponding costs is smaller than or equal to B.In the KP, the set of the possible candidate solutions is given by the power-set of N , i.e., the set of its possiblesubsets. Then, a brute-force search algorithm enumerates all of the possible subsets in the power-set of N ,eliminates all of those that do not comply with the capacity constraint, and record the one having maximumweight.A possible python implementation of a brute-force search algorithm for the KP is the following:

174 | 268






Brute Force Search & Backtracking: Crockrock’s Labyrinth

1 def knapSack (B , wt , val , n ) :2 # Base Case3 i f n == 0 or B == 0 :4 return 056 # I f weight of the nth item i s more than Knapsack of capaci ty7 # W, then t h i s item cannot be included in the optimal so lu t ion8 i f ( wt [ n−1] > B) :9 return knapSack (B , wt , val , n−1)1011 # return the maximum of two cases :12 # ( 1 ) nth item included13 # ( 2 ) not included14 else :15 return max( va l [ n−1] + knapSack (B−wt [ n−1] , wt , val , n−1) , knapSack (B , wt , val , n−1) )1617 va l = [50 , 150 , 100]18 wt = [10 , 20 , 30]19 B = 5520 n = len ( va l )21 p r i n t ( knapSack (B , wt , val , n ) )

175 | 268






Brute Force Search & Backtracking: Crockrock’s LabyrinthAn alternative search paradigm is Backtracking. This paradigm is generally used to find some (or possibly allof the) solutions to problems that involve constraint satisfaction features. This paradigm incrementally buildscandidate solutions to a given constraint satisfaction problem, and abandons a candidate solution under con-struction (i.e., the paradigm “backtracks”) as soon as it determines that this candidate does not lead to a validsolution. An example of backtracking is the following classical labyrinth problem

176 | 268






Brute Force Search & Backtracking: Crockrock’s LabyrinthCrockRock is a little cute dragon lost in a labyrithm. We need to help him to find the way out.

As a first step, let’s see how we can encode a simple labyrinth.

177 | 268






Brute Force Search & Backtracking: Crockrock’s LabyrinthOne way to do that is as follows.

1 G=[ [ ”−” , ”−” , ”−” , ” | ” , ”−” , ”−” , ” | ” , ” | ” , ” | ” , ”−” , ”−” , ”−” , ”−” ] ,2 [ ”−” , ” | ” , ”−” , ”−” , ”−” , ” | ” , ”−” , ”−” , ”−” , ”−” , ” | ” , ” | ” , ”−” ] ,3 [ ” | ” , ” | ” , ” | ” , ” | ” , ”−” , ” | ” , ”−” , ” | ” , ”−” , ” | ” , ”−” , ” | ” , ” | ” ] ,4 [ ”−” , ”−” , ”−” , ” | ” , ”−” , ”−” , ”−” , ” | ” , ”−” , ” | ” , ”−” , ”−” , ”−” ] ,5 [ ”−” , ” | ” , ”−” , ” | ” , ” | ” , ” | ” , ” | ” , ”−” , ”−” , ”−” , ” | ” , ” | ” , ”−” ] ,6 [ ”−” , ” | ” , ”−” , ”−” , ”−” , ”−” , ”−” , ”−” , ” | ” , ”−” , ”−” , ”−” , ”−” ] ,7 [ ”−” , ” | ” , ” | ” , ” | ” , ” | ” , ” | ” , ” | ” , ” | ” , ” | ” , ” | ” , ” | ” , ” | ” , ” | ” ] ,8 [ ”−” , ”−” , ”−” , ”−” , ”−” , ”−” , ”−” , ”−” , ”−” , ”−” , ”−” , ”−” , ”−” ] ]

The following simple function can be used to print out G on the screen:

1 def P r in t Laby r in th ( ) :2 p r i n t ( ” ” )3 fo r i in range (0 , len (G) ) :4 fo r j in range (0 , len (G[ 0 ] ) ) :5 p r i n t (G[ i ] [ j ] , end= ’ ’ )6 p r i n t ( ” ” )

178 | 268






Brute Force Search & Backtracking: Crockrock’s LabyrinthThe following function can be used to determine whether a specific entry (i,j) of G is valid, i.e., whether it canbe crossed by CrockRock because not blocked or inside the boundaries of the labyrint:

1 def va l id ( i , j ) :2 r e su l t=False3 #A loca t ion i s va l id i f the c e l l i s in the bounds of the matr ix4 i f i >=0 and i <len (G) and j >=0 and j <len (G[ 0 ] ) :5 #and i f c e l l i s not blocked and not prev ious ly v i s i t ed6 i f G[ i ] [ j ]== ”−” :7 r e su l t=True8 return r e su l t

The following function constitutes the engine of the search algorithm: it recursively explores all of the possibleentries (i,j) of G until either the exit is found or a failure is acheived:

179 | 268






Brute Force Search & Backtracking: Crockrock’s Labyrinth

1 def explore ( i , j ) :2 exi t_found = False3 i f va l id ( i , j ) == True :4 G[ i ] [ j ] = ” x ” # we use the symbol ’ x ’ to mark the entry of G as v i s i t ed5 # I f we ar r i ved to the bottom−r igh t corner of G6 i f i == len (G)−1 and j == len (G[ 0 ] ) −1:7 exi t_found = True #we found the way out ! ! ! Hurraa !8 else :9 exi t_found = explore ( i +1, j ) #We desperately search down10 i f exi t_found == False : #and i f we f a i l ed we desperately search r igh t11 exi t_found = explore ( i , j +1)12 i f exi t_found == False : #and i f we f a i l ed we desperately search up13 exit_found = explore ( i −1, j )14 i f exi t_found == False : #and i f we f a i l ed we desperately search l e f t15 exi t_found = explore ( i , j −1)16 # i f we succeeded , we mark ( i , j ) with ”8” , meaning tha t t h i s entry i s part of the way out .17 i f exi t_found == True :18 G[ i ] [ j ] = ”8”1920 return exi t_found

180 | 268






Brute Force Search & Backtracking: Crockrock’s LabyrinthFinally, these few commands set up and carry out the exploration:

1 pr i n t ( ” Crockrock ’ s Labyr i th ” )2 p r i n t ( ” p r in t ing the l aby r i t h . . . ” )3 P r in t Laby r in th ( )4 p r i n t ( ” Crockrock i s t r y i ng to f ind ing a way out . . . ” )5 i f explore (0 , 0) == True :6 p r i n t ( ” I found a way out ! ! fo l low my paws 8 . . . ” )7 else :8 p r i n t ( ”No way out found : ( ” )9 P r in t Laby r in th ( )

Source code

181 | 268




http://perso.uclouvain.be/daniele.catanzaro/SupportingMaterial/UCL/Teaching/CodingProject/Codes/ch04/CrockRock.py



Brute Force Search & Backtracking: The N-Queen ProblemFixed a positive integer n ≥ 4, the n-queen problem consists in placing n chess queens on a n × n chessboardso that no two queens threaten each other, i.e., the two queens do not share the same row, column, or diagonal.For example, if n = 8, then the following placement is a feasible one:

Note that a solution to the n-queen problem certainly exists for all natural numbers n ≥ 4. The problem there-fore is to find it. To this end, we shall develop an algorithm to solve this problem.

182 | 268






Brute Force Search & Backtracking: The N-Queen ProblemWe start by considering the problem of encoding a chessboard. One way to do so consists of assigning eachqueen to one specific row of the chessboard. Then, we can encode the chessboard as a list whose i-th entryrepresents the j-th column of the chessboard at which the i-th queen is placed. For example, if the list is [0,4, 7, 5, 2, 6, 1, 3] then the queen 0 is placed in the position (1,1) of the chessboard, the subsequent queen isplaced in position (2,5), and so on and so forth. A possible backtracking algorithm to solve n-queen problemthen reads as follows:

183 | 268






Brute Force Search & Backtracking: The N-Queen Problem1 def Feasible ( ChessBoad , k ) :2 i = 03 while i < k :4 i f ChessBoad [ k ] == ChessBoad [ i ] or abs ( ChessBoad [ i ] − ChessBoad [ k ] ) == abs ( i − k ) :5 return False6 i += 17 return True89 def NQueens ( ChessBoad , kQueen , n ) :10 i f kQueen == n :11 pr i n t ( ChessBoad )12 return13 i = 014 while i < n :15 ChessBoad [kQueen ] = i16 i f Feasible ( ChessBoad , kQueen ) :17 NQueens ( ChessBoad , kQueen + 1 , n )18 i += 119 n=8 #number of queens20 ChessBoad = [0 ] * n21 NQueens ( ChessBoad , 0 , n )

184 | 268






Brute Force Search & Backtracking: Exercises1. Given a list L of n numbers, write an algorithm that print the duplicate element of L.2. Given a list L of n numbers, write an algorithm that find the minimum, the maximum, and all of the num-

bers that are between the minimum and the maximum and that are absent in L.3. Given a list L of n numbers and a value v, write an algorithm that find (if any) a pair of elements whose

sum is equal to v.4. Given a list L of n numbers and a value v, write an algorithm that find (if any) a pair of elements whose

sum is the closest to v.5. Given two lists L1 and L2 of n andm numbers, respectively, and a value v, write an algorithm that find (if

any) a pair of elements x and y whose sum is v and such that x ∈ L1 and y ∈ L2.6. Given a list L of n integers and a value v, write an algorithm that find (if any) a triplet that sum to 0.7. Given a list L of n integers, write an algorithm that find (if any) a triplet of integers x, y, z ∈ L such that

x + y = z.

185 | 268





Graphs

186 | 268



GraphsA Graph G = (V, E) consists of a finite nonempty set V of nodes (or vertices) and a finite set E of edges. Withevery edge, an unordered pair of nodes, called its endnodes and usually denoted as (u, v), with u, v ∈ V , isassociated and we say that an edge is incident to its endnodes.

bc

bc

bc

bc bc

bcbc

bc

bc

bc

187 | 268






GraphsWhen an ordering is explicitly given on the pair of endnodes (u, v), u, v ∈ V , we say that such pair (u, v) definesan arc and we say that G = (V, E) is directed. To remark this fact, we replace the letter E (i.e., the edge-set)with the letter A (i.e., the arc-set). We say that G = (V, E) is undirected otherwise.

u v z

x y

188 | 268






GraphsThe origin of graph theory dates back to the 18th century, in the Prussian town of Königsberg. The river Pregelflowed through the town, creating two islands. The city and the islands were connected by seven bridges asshown in this figure.

The inhabitants of Königsberg were moved by the following question: is it possible to take a walk through thetown, by visiting each area of the town and crossing each bridge only once? Every bridge must have beencrossed completely, i.e. it is not allowed to walk halfway onto a bridge and then turn around and later cross theother half from the other side. The walk need not start and end at the same spot.

189 | 268






GraphsLeonhard Euler solved the problem in 1735 by proving that this walk could not exist. Euler found out thatthe choice of a route inside each land area was irrelevant: the only real important thing was the order of thesequence in which the bridges were crossed. Euler had formulated an abstraction of the problem, eliminatingunnecessary facts and focussing on the land areas and the bridges connecting them. Specifically, if we see a”land area” as a node and each bridge as an edge, we have ”reduced” the problem to a graph.

By means of this abstraction process, Euler set the ground to the foundations of graph theory.190 | 268






GraphsGraphs are used to model many domains and functions, including mapping, transportation, computer net-works, social networks, distribution networks, banking, and electrical engineering just to cite a few.

Example: Social networks. Social graphs draw edges between you and the people, places and things youinteract with online. Anything that has properties to store data is a node. Every connection or relationship is anedge: e.g., a user posting a photo, a profile update, a change in relationship status, the liking a friend’s photo,and so on and so forth.

191 | 268






GraphsExample: GoogleMaps. GoogleMaps is a classic tool that enables us to find the best connection that fits somepredefined criteria e.g. speed, safety, fuel and so on or set of criteria e.g procedures, routes. The idea at thecore of Google Maps is to find the shortest path from one node to every other as in a single source shortestpath variant, from your current location to every other destination you might be interested in going to on themap.

192 | 268






GraphsExample: Flight Networks. Computing the best possible connections and fuel consumption in flight networksis crucial for a flight company. The nodes in a flight network are places of departure and destination, airports,aircrafts, cargo weights. The flight trajectories between airports are the edges. Entities such as flights can haveproperties such as fuel usage or crew pairing, which can be graphs themselves.

193 | 268






GraphsExample: Protein Interaction Networks. Understanding how genes and proteins interact inside a cell is crucialfor medical reasons. In a Protein Interaction Network each protein represents a node and there exists an edgebetween two nodes if two proteins interact among them.

194 | 268






Graphs: Fundamental Notation & DefinitionsThe following notation and definitions are fundamental in graph theory:

• The number of nodes of G is called the order of G.

• A node that is not incident to any edge is called isolated. Two nodes that are joined by an edge are calledadjacent or neighbor. The neighborhood of a node u is the set of its neighbors.

• The set of edges having a node u ∈ V as one of their endnodes is denoted by δ(u). The number |δ(u)|denotes the degree of the node u ∈ V .

• Consider a graph G = (V, E). Then, we say that G is weighted if there exists a function w : E → R thatassociates a real number to each edge of G. In this situation, we say that wij is the weight associated tothe edge (i, j) ∈ E.

195 | 268






Graphs: Fundamental Notation & Definitions• More generally, if S ⊆ V , then δ(S) denotes the set of edges with one endnode in S and the otherendnode in V \ S. Any edge of the form δ(S), where ∅ = S = V is called a cut.

2 4

1 5 7

3 6

• If s and t are two different nodes of G, then an edgeset F ⊆ E is called a (s, t)-cut if there exists a nodeset S ⊆ V , with s ∈ S, t /∈ S, such that F = δ(S).

196 | 268






Graphs: Fundamental Notation & Definitions• A simple graph is called complete is every two of its nodes are joined by an edge. The complete graphof order n is denoted by Kn.

1

2

34

5

6

7

8 9

10

197 | 268






Graphs: Fundamental Notation & Definitions• A graph G whose nodeset V can be partitioned into two nonempty disjoint sets V1, V2 with V1 ∪ V2 = Vis called bipartite.

u4

u3

v3

u2

v2

u1

v1

u0

v0

V1 V2

When |V | = m + n, |V1| = m, and |V2| = n, we denote this bipartite graph as Km,n.

198 | 268






Graphs: Fundamental Notation & Definitions• The complete bipartite graph having |V1| = m and |V2| = n is denoted asKm,n. The complete bipartitegraph K1,n is called a star and the star K1,3 is called a claw.

• Given a graph G, the complement of G, denoted by G, is the simple graph which has the same nodesetas G and in which two nodes are adjacent if and only if they are nonadjacent in G.

• A graph is planar, if it can be drawn in the plane in such a way that no two edges intersect, except possiblyin their endpoints.

• A directed graph (or digraph)D = (V, A) consists of a finite nonempty set V of nodes and a setA of arcs.

• With every arc a, an ordered pair (u, v) of nodes, called its endnodes, is associated; u is the initialendnode and v is the terminal endnode.

• If u ∈ V then the set of arcs having u as initial (terminal) node is denoted by δ+(u) (δ+(u)). We setδ(u) = δ+(u) ∪ δ−(u). The numbers |δ+(u)|, |δ−(u)|, and |δ(u)| are called the outdegree, indegree, anddegree of u, respectively.

• For any set W ⊆ V , ∅ = W = V , we set δ+(W ) = {(i, j) ∈ A : i ∈ W, j /∈ W }; δ−(W ) = δ+(V \ W ),and δ(W ) = δ+(W ) ∪ δ−(W ).

199 | 268






Graphs: Fundamental Notation & Definitions• If s and t are two different nodes of a digraph D = (V, A), then an arcset F ⊆ A is called an (s, t)-cut in

D if there is a node set W with s ∈ W , t /∈ W , such that F = δ+(W ).

• In a graph (or a digraph), a walk is a finite and alternating sequence

W = v0, e1, v1, e2, v2, . . . , ek, vk,

for some k ≥ 0, of nodes and edges (or arcs) beginning and ending with a node, and in which each nodevi is followed by an edge (or arc) ej .

• An edge (arc) connecting two nodes of a walk but not contained in the walk is called a cord of the walk.

• A walk in which all nodes (edges or arcs) are distinct is called a path. A path in a directed graph is calleda directed path or dipath.

• Two nodes s, t of a graphG are said to be connected ifG contains an (s, t)-path, i.e., a path beginning ins and ending in t or vice versa.

• G is called connected if every two nodes of G are connected.

200 | 268






Graphs: Fundamental Notation & Definitions• A digraph D is strongly connected if for every two nodes s, t of D, there are an (s, t)-dipath and a (t, s)-dipath in D.

1

2 3

201 | 268






Graphs: Fundamental Notation & Definitions• A digraph D is weakly connected if for every two nodes s, t of D, there is either an (s, t)-dipath or a

(t, s)-dipath in D.

1 4

2 3 5 6

202 | 268






Graphs: Very Important Definitions• A walk is called closed if it has nonzero length and its origin and terminus are identical. A closed walk inwhich the origin, all internal nodes, and all edges are different is called a circuit or cycle.

• Awalk that traverses every edge (arc) of a graph (digraph) exactly once is called an Eulerian trail. A closedEulerian trail is called an Eulerian tour. An Eulerian graph is a graph that contains an Eulerian tour.

• A circuit of length n in a graph of order n is called a Hamiltonian circuit. A graph G that contains aHamiltonian circuit is called Hamiltonian.

203 | 268






Graphs: Very Important Definitions• A forest is an edgeset in a graph which does not contain a circuit.

1

2 3 4

• A connected forest is called a tree. In other words, a tree is an acyclic graph.

1

2 3 4

204 | 268






Graphs: Very Important Definitions• Let T = (V, E) a finite tree of order n. Then, the following statements are equivalent:

• T is connected and has no circuits;

• T has n − 1 edges and has no circuits;

• T is connected and has n − 1 edges;

• T is connected and the removal of any one edge makes T disconnected;

• Any two nodes of T are connected by exactly one path;

• T has no circuits and adding one edge in T gives rise to a cycle.

205 | 268






Graphs: Very Important Definitions• A spanning tree of a graph G of order n is a tree containing all of the n nodes of G.

2 4

1 5 7

3 6

206 | 268






Graphs: ImplementationThere are two common ways of representing a graph G = (V, A):

• by means of an Adjacency Matrix (AM);

• by means of an Adjacency List (AL).

An AM is a two-dimensional matrix M of size |V | × |V |. The generic entry mij of the matrix is such that

aij ={

1 if the arc (i, j) ∈ A;0 otherwise, ∀ i, j ∈ V. (1)

For example, the AM corresponding to the following graph is

1 4

2 3M =

0 1 1 01 0 1 01 1 0 10 0 1 0

207 | 268






Graphs: ImplementationFrom an implementation point of view, these AM can be encoded as

1 M=[ [0 ,1 ,1 ,0 ] ,2 [1 ,0 ,1 ,0 ] ,3 [1 ,1 ,0 ,1 ] ,4 [0 , 0 , 1 , 0 ] ]

If the graph is weighted, the weights can be used in place of 1s:

1 4

2 33

4

72 M =

0 3 7 03 0 4 07 4 0 20 0 2 0

As before, this AM can be encoded as

208 | 268






Graphs: Implementation

1 M=[ [0 ,3 ,7 ,0 ] ,2 [3 ,0 ,4 ,0 ] ,3 [7 ,4 ,0 ,2 ] ,4 [0 , 0 , 2 , 0 ] ]

More in general, a generic code to encode a graph is the following.

1 G = [ ]23 def CreateEmptyGraph ( order ) :4 fo r i in range ( order ) :5 G. append ( [ 0 fo r i in range ( order ) ] )

209 | 268







1 def addEdge ( v1 , v2 ) :2 i f v1 == v2 :3 p r i n t ( ”Same node %d and %d” % ( v1 , v2 ) )4 G[ v1 ] [ v2 ] = 15 G[ v2 ] [ v1 ] = 167 def removeEdge ( v1 , v2 ) :8 i f s e l f .G[ v1 ] [ v2 ] == 0:9 p r i n t ( ”No edge between %d and %d” % ( v1 , v2 ) )10 return11 G[ v1 ] [ v2 ] = 012 G[ v2 ] [ v1 ] = 0

We can further add the following functions:

210 | 268







1 def containsEdge ( v1 , v2 ) :2 return True i f G[ v1 ] [ v2 ] > 0 else False34 def PrintGraph ( ) :5 fo r row in G:6 fo r va l in row :7 p r i n t ( ’ { : 4 } ’ . format ( va l ) ) ,8 p r i n t

Source code AM.pyObservations:

1. The space complexity of an AM is O(n2) because a two-dimensional array (list) must be created.

2. Querying if there exists an edge between the nodes i and j takes O(1).

3. Removing the edge between the nodes i and j takes O(1).

4. Adding a new node takes O(n2) because a new AM must be created from scratch.

211 | 268




https://colab.research.google.com/drive/1TCxxWFykTyNK3Cse6EPukKaaeQqvil5W#scrollTo=KGWXc-MCF-fH&line=28&uniqifier=1



Graphs: Implementation5. In huge graphs, each node may be connected just to a few nodes. As a result, in the AM most of the

space is constituted by ”0”. In such situation we say that the AM is sparse. Encoding sparse matrices byAMs is quite inefficient because most of the space contain no information.

A more efficient way to encode a graph is provided by an Adjacency List (AL). An AL can be thought as a sortof list of lists. In particular, the i-th element of the list corresponds to the i-th node of the graph and containsthe list of the nodes that are adjacent to i together with their optional weights.

1

2 5 3

4 6 7

2

3

3

11

1

11

1 graph = {2 ’1 ’ : { ’ 2 ’ : 2 , ’ 3 ’ : 1 , ’ 5 ’ : 3 } ,3 ’2 ’ : { ’ 1 ’ : 2 , ’ 4 ’ : 1 , ’ 5 ’ : 3 } ,4 ’3 ’ : { ’ 1 ’ : 1 , ’ 6 ’ : 1 , ’ 7 ’ : 5 } ,5 ’4 ’ : { ’ 2 ’ : 1 , ’ 5 ’ : 1 } ,6 ’5 ’ : { ’ 1 ’ : 3 , ’ 2 ’ : 3 , ’ 4 ’ : 1 } ,7 ’6 ’ : { ’ 3 ’ : 1 } ,8 ’7 ’ : { ’ 3 ’ : 1}9 }

212 | 268






Graphs: ImplementationMore in general, we can declare the graph as follow:

1 from co l l ec t i ons import de fau l td i c t23 graph = de fau l td i c t ( l i s t )45 def addArc ( graph , u , v ) :6 graph [ u ] . append ( v )78 def deleteArc ( graph , u , v ) :9 graph [ u ] . remove ( v )

We can then add our edges as follows

213 | 268







1 addArc ( graph , ’ u ’ , ’ v ’ )2 addArc ( graph , ’ u ’ , ’w ’ )3 addArc ( graph , ’ v ’ , ’ u ’ )4 addArc ( graph , ’ v ’ , ’w ’ )5 addArc ( graph , ’w ’ , ’ u ’ )6 addArc ( graph , ’w ’ , ’ v ’ )7 addArc ( graph , ’w ’ , ’ z ’ )8 addArc ( graph , ’ z ’ , ’w ’ )

At this point, we could be willing to create a list containing all of the edges. We can do that by means of thefollowing function:

1 def generate_edges ( graph ) :2 edges = [ ]3 fo r node in graph :4 fo r neighbor in graph [node ] :5 edges . append ( ( node , neighbor ) )6 return edges

214 | 268






Graphs: Implementationand finally print the edge list:

1 pr i n t ( generate_edges ( graph ) )

Source code AL.py

215 | 268




http://perso.uclouvain.be/daniele.catanzaro/SupportingMaterial/UCL/Teaching/CodingProject/Codes/ch14/AL.py



Converting an AL into an AMSuppose that a graph G is given in form of an AL, as shown in this fragment of code:

1 mygraph = {2 ’1 ’ : { ’ 2 ’ : 2 , ’ 3 ’ : 1 , ’ 5 ’ : 3 } ,3 ’2 ’ : { ’ 1 ’ : 2 , ’ 4 ’ : 1 , ’ 5 ’ : 3 } ,4 ’3 ’ : { ’ 1 ’ : 1 , ’ 6 ’ : 1 , ’ 7 ’ : 1 } ,5 ’4 ’ : { ’ 2 ’ : 1 , ’ 5 ’ : 1 } ,6 ’5 ’ : { ’ 1 ’ : 3 , ’ 2 ’ : 3 , ’ 4 ’ : 1 } ,7 ’6 ’ : { ’ 3 ’ : 1 } ,8 ’7 ’ : { ’ 3 ’ : 1}9 }

Suppose also that for some reason you want to translate mygraph into an AM. Then, we can do as follows:

216 | 268






Converting an AL into an AM

1 def TranslateALtoAM ( graph ) :2 G = [ ] #The AM of the graph3 order = len ( graph )4 fo r i in range ( order ) :5 G. append ( [ 0 fo r i in range ( order ) ] )6 edges = [ ( cost , from_node , to_node ) \7 fo r from_node in graph \8 fo r to_node , cost in graph [ from_node ] . items ( ) ]9 fo r e in edges :10 u , v , w = ord ( e [ 1 ] ) −48, ord ( e [ 2 ] ) −48, e [ 0 ]11 G[ u ] [ v ]=w12 return G1314 pr i n t ( TranslateALtoAM (mygraph ) )

As an exercize, write a python code that translates a given AM into the correspondent AL.

217 | 268






Graph TraversalGreek mythology tells of an elaborate labyrinth that was built to house the monstrous Minotaur, which was partbull and part man. This labyrinth was so complex that neither beast nor human could escape it. No human, thatis, until the Greek hero, Theseus, with the help of the king’s daughter, Ariadne, decided to implement a graphtraversal algorithm.Theseus fastened a ball of thread to the door of the labyrinth and unwound it as he traversed the twistingpassages in search of themonster. Theseus obviously knew about good algorithm design, for, after finding anddefeating the beast, Theseus easily followed the string back out of the labyrinth to the loving arms of Ariadne.

218 | 268






Graph TraversalFormally, a traversal is a systematic procedure for exploring a graph by examining all of its nodes and edges. Atraversal is efficient if it visits all the nodes and edges in time proportional to their number, that is, in linear time.Graph traversal algorithms are key to answeringmany fundamental questions about graphs involving the notionof reachability, that is, in determining how to travel from one node to another while following paths of a graph.Interesting problems that deal with reachability in a graph G include e.g., the following:

• Computing a path from node u to node v, or reporting that no such path exists.

• Given a start node s of G, computing, for every node v of G, a path with the minimum number of edgesbetween s and v, or reporting that no such path exists.

• Testing whether G is connected.

• Computing a spanning tree of G, if G is connected.

• Computing the connected components of G.

• Computing a cycle in G, or reporting that G has no cycles.

219 | 268






Graph Traversal• Computing a directed path from node u to node v, or reporting that no such path exists.

• Determine whether G is acyclic.

The literature provides us with two fundamental algorithms for traversing or searching a graph, namely:

• the Depth-First Search (DFS);

• the Breadth-First Search (BFS).

They mainly differ for the way in which a given graph G is explored. Let’s see them in detail.

220 | 268






Graph TraversalThe DFS algorithm is useful for testing a number of properties of graphs, including e.g., whether there is a pathfrom one node to another and whether or not a graph is connected.The DFS algorithm in a graphG is analogous to wandering in a labyrinth with a string and a can of paint withoutgetting lost. We begin at the root node of G, i.e., some specific starting node s in G that we have arbitrarilychosen.We set s as our current node u and we mark it as “visited”. Then, we traverse G by considering an (arbitrary)edge (u, v) incident to the current node u. If the edge (u, v) leads us to a node v that is already visited, weignore that edge. Otherwise, if (u, v) leads to an unvisited node v, then we unroll our string, and go to v. Wethen mark v as “visited”, and make it the current node u, repeating the computation above.Eventually, we will get to a ”dead end”, that is, a current node v such that all the edges incident to v lead tonodes already visited. To get out of this impasse, we roll our string back up, backtracking along the edge thatbrought us to v, going back to a previously visited node u. We then make u our current node and repeat thecomputation above for any edges incident to u that we have not yet considered. If all of u’s incident edgeslead to visited nodes, then we again roll up our string and backtrack to the node we came from to get to u, andrepeat the procedure at that node.

221 | 268






Graph TraversalThus, we continue to backtrack along the path that we have traced so far until we find a node that has yet unex-plored edges, take one such edge, and continue the traversal. The process terminates when our backtrackingleads us back to the start node s, and there are no more unexplored edges incident to s.Now, suppose that we have given the following graph and that we want to traverse it.

1

2 5 3

4 6 7

222 | 268






Graph TraversalThen, a possible implementation of the DFS algorithm is the following:

1 def DFS(node , v i s i t ed ) :2 p r i n t ( node )3 v i s i t ed . append (node )4 fo r neighbor in graph [node ] :5 i f neighbor not in v i s i t ed :6 DFS( neighbor , v i s i t ed )

Which output this code gives when calling DFS(’A’,[]) on the graph below?

1

2 5 3

4 6 7

223 | 268






Graphs: Fundamental Properties of the DFS algorithmThere are a number of observations that we can make about the DFS algorithm, many of which derive from theway the DFS algorithm partitions the edges of a graph G into groups. We mention the most significant ones.

• Let G be an undirected graph on which a DFS traversal starting at a node s has been performed. Thenthe traversal visits all nodes in the connected component of s, and the discovery edges form a spanningtree of the connected component of s.

• Let G be a directed graph. The DFS algorithm on G starting at a node s visits all the nodes of G that arereachable from s. Also, the DFS tree contains directed paths from s to every node reachable from s.

• Let G be an undirected graph with n nodes and m edges. A DFS traversal of G can be performed inO(n + m) time, and can be used to solve the following problems in O(n + m) time:

• Computing a path between two given nodes of G, if one exists.

• Testing whether G is connected.

• Computing a spanning tree of G, if G is connected.

• Computing the connected components of G.

• Computing a cycle in G, or reporting that G has no cycles.224 | 268






Graphs: Fundamental Properties of the DFS algorithm• IfG is a directed graph with n nodes andm edges, the DFS algorithm can be still performed inO(n + m)time, and can be used to solve the following problems in O(n + m) time:

• Computing a directed path between two given nodes of G, if one exists.

• Computing the set of nodes of G that are reachable from a given node s.

• Testing whether G is strongly connected.

• Computing a directed cycle in G, or reporting that G is acyclic.

• Computing the transitive closure ofG, i.e., a directed graphG′ having the same set of nodes ofG,and arc (u, v), whenever G has a directed path from u to v (including the case where (u, v) is anedge of G).

We will see later some implementations of these problems.

225 | 268






The BFS algorithmThe advancing and backtracking of the DFS algorithm, as previously described, defines a traversal that couldbe physically traced by a single person exploring a graph. In this section, we consider another algorithm fortraversing a connected component of a graph, known as a Breadth-First Search (BFS).The BFS algorithm is more akin to sending out, in all directions, many explorers who collectively traverse agraph in coordinated fashion. A BFS proceeds in rounds and subdivides the nodes into levels. BFS starts atnode s, which is at level 0. In the first round, we mark as visited all of the nodes adjacent to the starting node s.These nodes are one step away from ts and are placed into level 1.In the second round, we allow all explorers to go two steps (i.e., edges) away from the starting node s. Thesenew nodes, which are adjacent to level 1 nodes and not previously assigned to a level, are placed into level 2and marked as visited. This process continues in similar fashion, terminating when no new nodes are found ina level.A Python implementation of the BFS algorithm is given below.

226 | 268






The BFS algorithm

1 def BFS ( s t a r t ) :2 v i s i t ed = [ ]3 queue = [ s t a r t ]4 while queue :5 node = queue .pop( 0 )6 i f node not in v i s i t ed :7 p r i n t ( node )8 v i s i t ed . append (node )9 neighbors = graph [node ]10 fo r neighbor in neighbors :11 queue . append ( neighbor )

Which output this code gives when calling BFS(’A’) on the graph below?

1

2 5 3

4 6 7

227 | 268






The BFS algorithm

Observations:Let G be an undirected or directed graph on which a BFS algorithm starting from node s hasbeen performed. Then

• The algorithm visits all of the nodes of G that are reachable from s.

• For each node v at level i, the path of the BFS tree T between s and v has i edges, and any other path ofG from s to v has at least i edges.

• If (u, v) is an edge that is not in the BFS tree, then the level number of v can be at most 1 greater than thelevel number of u.

• If G has n nodes and m edges, a BFS traversal of G takes O(n + m) time.

228 | 268






Finding a path (if any) between a starting node and an end node1 def f ind_path ( graph , s t a r t , end , path = [ ] ) :2 path = path + [ s t a r t ]3 i f s t a r t == end :4 return path5 fo r node in graph [ s t a r t ] :6 i f node not in path :7 newpath = f ind_path ( graph , node , end , path )8 i f newpath :9 return newpath10 return None1112 pr i n t ( f ind_path ( graph , 4 , 7 , [ ] ) )

Which output this code gives when called on the graph below?

1

2 5 3

4 6 7

229 | 268






Finding all paths (if any) between a starting node and an end node1 def f i nd_a l l _pa ths ( graph , s t a r t , end , path = [ ] ) :2 path = path + [ s t a r t ]3 i f s t a r t == end :4 return [ path ]5 paths = [ ]6 fo r node in graph [ s t a r t ] :7 i f node not in path :8 newpaths = f i nd_a l l _pa ths ( graph , node , end , path )9 fo r newpath in newpaths :10 paths . append ( newpath )11 return paths12 pr i n t ( f i nd_a l l _pa ths ( graph ,4 , 7 ) )

Which output this code gives when called on the graph below?

1

2 5 3

4 6 7

230 | 268






Detecting cycles in a graph1 def CheckForCycle ( graph , root , current_node , cyc le ) :2 cyc le . append ( current_node )3 fo r neighbor in graph [ current_node ] :4 i f neighbor not in cyc le :5 i f ( CheckForCycle ( graph , root , neighbor , cyc le ) ) :6 return True7 else :8 del cyc le [ −1]9 e l i f root==neighbor and len ( cyc le ) >=3:10 cyc le . append ( neighbor )11 return True12 return False

231 | 268






Detecting cycles in a graph1 def I sCy c l i c ( graph ) :2 cyc l ic_graph=False3 fo r root in range ( len ( graph ) ) :4 p r i n t ( ” searching fo r a cyc le s t a r t i ng from ” , root )5 cyc le = [ ]6 response = CheckForCycle ( graph , root , root , cyc le )7 i f response == True :8 p r i n t ( ” cyc le found : ” , cyc le )9 cyc l ic_graph = True ;10 else :11 p r i n t ( ”no cyc le detected ! ” )12 i f cyc l ic_graph == True :13 pr i n t ( ”The graph i s c y c l i c ! ” )14 else :15 p r i n t ( ” the graph i s not c y c l i c ! ” )16 return cyc l ic_graph

232 | 268






Detecting cycles in a graphWhich output this code gives when called on the graph below? How would you modify this code to check thepresence of cycles of length at least K ≥ 4?

1

2 5 3

4 6 7

233 | 268






Detecting cycles in a graphA full implementation of the before mentioned algorithms can be found here:Source codeExercises

• Write an algorithm to check whether a given graph is connected.• The diameter D of a graph is the length of the shortest path between the most distanced nodes. Todetermine the diameter of a graph, first find the shortest path between each pair of nodes. The greatestlength of any of these paths is the diameter of the graph. Write an algorithm to find the diameter of agiven graph G.

234 | 268




https://colab.research.google.com/drive/1TCxxWFykTyNK3Cse6EPukKaaeQqvil5W#scrollTo=0qJVipZDHQuF&line=109&uniqifier=1


Fundamental Problems on Graphs

235 | 268



Fundamental Problems on GraphsThe Minimum Spanning Tree Problem (MSTP)Given a weighted undirected connected graph G = (V, E), having a non-negative weight wij on each edge(i, j) ∈ E, the MSTP consists in finding a spanning tree T for G whose overall length (i.e., the sum of its edgeweights) is minimum.

2 4

1 5 7

3 6

236 | 268






Fundamental Problems on GraphsMinimum spanning trees have direct applications in the design of networks, including (but not limited to) com-puter networks, telecommunications networks, transportation networks, water supply networks, supply chainnetworks, and electrical grids (which they were first invented for).For example, consider a telecommunications company trying to lay cable in a new neighborhood. If it is con-strained to bury the cable only along certain paths (e.g. roads), then there would be a graph containing thepoints (e.g. houses) connected by those paths. Some of the paths might be more expensive, because theyare longer, or require the cable to be buried deeper; these paths would be represented by edges with largerweights. Currency is an acceptable unit for edge weight – there is no requirement for edge lengths to obeynormal rules of geometry such as the triangle inequality. A spanning tree for that graph would be a subset ofthose paths that has no cycles but still connects every house. Note that there might be several spanning treespossible. A minimum spanning tree would be one with the lowest total cost, hence the least expensive set ofpaths for laying the cable.An algorithm for theMST is frequently invokedby algorithms for other problems (see e.g., theChristofides algo-rithm for approximating the traveling salesman problem, or the approximation of the minimum-cost weightedperfect matching). Other practical applications based on minimal spanning trees include:

• Taxonomy• Cluster analysis (e.g., hierarchical clustering)

237 | 268






Fundamental Problems on Graphs• Constructing trees for broadcasting in computer networks• Image registration and segmentation• Curvilinear feature extraction in computer vision• Handwriting recognition of mathematical expressions• Circuit design• Regionalisation of socio-geographic areas and the grouping of areas into homogeneous, contiguousregions.

• Topological observability in power systems• Measuring homogeneity of two-dimensional materials• Minimax process control• Description of financial markets. In particular, a correlation matrix can be created by calculating a coef-ficient of correlation between any two stocks; this matrix can be represented topologically as a complexnetwork and a minimum spanning tree can be constructed to visualize relationships.

238 | 268






Fundamental Problems on GraphsRecall:

• A spanning tree of a graph G of order n is a tree containing all of the n nodes of G.• Let T = (V, E) a finite tree of order n. Then, the following statements are equivalent:

• T is connected and has no circuits;

• T has n − 1 edges and has no circuits;

• T is connected and has n − 1 edges;

• T is connected and the removal of any one edge makes T disconnected;

• Any two nodes of T are connected by exactly one path;

• T has no circuits and adding one edge in T gives rise to a cycle.

239 | 268






Fundamental Problems on GraphsFundamental property of the MSTP: The Cut-Optimality conditionProposition 1Given a weighted undirected graph G = (V, E), consider a bipartition of the set V into the subsets S and S′.Then, any MST of G must contain the edge (i, j) ∈ E such that i ∈ S, j ∈ S′, and wij is minimum.

240 | 268






Fundamental Problems on GraphsProof.By contradiction, suppose that the edge (i, j) hasminimal weight, it such that i ∈ S and j ∈ S′, but it is not in theMST. Then, because the MST must be connected, another edge of E, say (k, l), must link S and S′. To displaythis situation, suppose that (i, j) = (3, 6), (k, l) = (2, 4), S is the red-circled nodeset and S′ is the gray-circlednodeset.

2 4

1 5 7

3 6

Let’s add (i, j) to our the tree and observe that now a cycle arises due to the simultaneous presence of (i, j)and (k, l). By removing edge (k, l) we obtain a tree with a shorter length than the previous tree as the weightof (i, j) is smaller than the one of (k, l). This fact contradicts the hypothesis that the initial tree was a MST of thegraph and the statement follows.

241 | 268






Fundamental Problems on GraphsThe following algorithm makes operational the cut-optimality condition:Prim’s algorithm for the MSTP (1957)1. Create a set T containing a single node, chosen arbitrarily from the graph;2. Create a set Q containing all the remaining nodes;3. Loop until T = V :

• Find the edge e of minimum weight connecting any of the nodes in T with any node in Q;• Mark e as belonging to the MST;• Add to T the node in Q which is incident to e.

Prim’s algorithm optimally solve the MSTP and has a computational cost O(|E|log|V |).Let’s see how to implement Prim’s algorithm in Python.

242 | 268






Fundamental Problems on Graphs

1 def Prim ( graph , start ing_node ) :2 MST , MSTLen = de fau l td i c t ( set ) , 03 T = [ start ing_node ]4 Q = [node fo r node in graph i f node!= start ing_node ]5 while len ( T ) < len ( graph ) :6 edges = [ ( cost , from_node , to_node ) \7 fo r from_node in T \8 fo r to_node , cost in graph [ from_node ] . items ( ) i f to_node in Q]9 min_weight_edge = 100000000 # i n f i n i t y10 min_pos_edge = −111 fo r e in edges :12 i f e [ 0 ] < min_weight_edge :13 min_weight_edge , min_pos_edge = e [ 0 ] , e14 T . append (min_pos_edge [ 2 ] )15 Q. remove (min_pos_edge [ 2 ] )16 MST[min_pos_edge [ 1 ] ] . add (min_pos_edge [ 2 ] )17 MSTLen += min_pos_edge [ 0 ]18 p r i n t ( ” Overal l MST length : ” , MSTLen )19 return MST2021 pr i n t ( Prim ( graph , ’ 1 ’ ) )

243 | 268






Fundamental Problems on GraphsWhich output this code gives when called on the graph below?

1

2 5 3

4 6 7

2

3

3

11

1

11

Source code Prim

244 | 268




https://colab.research.google.com/drive/1TCxxWFykTyNK3Cse6EPukKaaeQqvil5W#scrollTo=UI3LjVRKSx3z&line=1&uniqifier=1



Fundamental Problems on GraphsThe Shortest Path Problem (SPP)In some situations, we might be interested in finding the fastest way to travel cross-country or the fastest wayto go from a point A to a point B. If a (weighted) graph is used to map the roads between cities, this problemtranslates into finding the the shortest-path connecting a node to another one.

245 | 268






Fundamental Problems on GraphsThe Shortest Path Problem (SPP)LetG be a weighted graph of order n > 0 and let p be a path inG. The length (or weight) of p is defined as thesum of the weights of the edges belonging to p. In other words, if p = ((v0, v1), (v1, v2), ..., (vk1, vk)), then thelength of p, denoted as w(p), is defined as

w(p) =k−1∑i=0

wi,i+1

Given any two vertices of G, say u and v, the Shortest Path Problem (SPP) asks to find the shortest path (if itexists) between u and v.

246 | 268






Fundamental Problems on GraphsSuppose that your graph is simply a complete graph, such as the following one. How many paths should youpotentially consider to find the solution?

1

2

34

5

6

7

8 9

10

247 | 268






Fundamental Problems on GraphsLet’s give a look to the number of nodes that we could potentially insert between the source and the destination.We could insert 0 nodes (i.e., we go directly from node i to node j) or 1 node (i.e., we pass by an intermediatepoint), or 2 nodes and so on and so forth. Note that if we insert a number of nodes greater or equal to 2 then wehave to consider the permutations of those nodes For example, if i = 1 and j = 5, one path could be (1, 2, 3, 5)and another one could be (1, 3, 2, 5).

Recall that the binomial coefficient(

nk

)provides the number of subsets� �of k elements that can be extracted

from a set of n elements. Then, we have that(n − 2

0

)+

(n − 2

1

)1! +

(n − 2

2

)2! + . . .

(n − 2n − 2

)(n − 2)!

n-2 because the source and the destination are fixed; 1!, 2! and so on because if we insert between the sourceand thedestination a number of nodesgreater thanor equal to 2 thenwehave also to consider thepermutationsof those nodes. Since by definition 0!=1!=1 the above formula can be rewritten as:

n−2∑k=0

(n − 2

k

)k!

248 | 268






Fundamental Problems on GraphsThis number increase exponentially in function of n. Enumerating all of these paths is therefore intractable. Weneed however to find a solution to this problem as it is fundamental in several applications such as

• Maps• Robot navigation• Texture mapping• Typesetting in TeX• Urban traffic planning• Air traffic planning• Optimal pipelining of VLSI chip• Subroutine in advanced algorithms• Telemarketer operator scheduling• Routing of telecommunications messages• Approximating piecewise linear functions• Network routing protocols (OSPF, BGP, RIP)

249 | 268






Fundamental Problems on Graphs• Exploiting arbitrage opportunities in currency exchange• Optimal truck routing through given traffic congestion pattern

The SPP is in general intractable, i.e., we do not knowpolynomial-time algorithms to solve any possible instanceof the problem. In some particular cases, however, we do know them. Specifically, the literature distinguishesbetween several versions of the SPP, the most important of which include

• source-target (s − t) with non-negative edge weights; (solvable by Dijkstra’s Algorithm)• single source to all remaining vertices and non-negative edge weights; (solvable by Dijkstra’s Algorithm)• single source to all remaining vertices, arbitrary edgeweights andnonegative cycles; (solvablebyBellman-Ford’s Algorithm)

• all-pairs, arbitrary edge weights, and no negative cycles; (solvable by Floyd-Warshall’s Algorithm)• source-target (s − t) with arbitrary edge weights and negative cycles (HARD, i.e., generally intractable).

250 | 268






Fundamental Problems on GraphsIn order to study these cases, we will assume from now on that:

• the graph is directed;• the graph contains a directed path from node s to every other node;�• the graph has non-negative weights.

251 | 268






The SPP: Dijkstra’s AlgorithmThe first case that we will consider is the case single-source shortest path problem with non-negative edgeweights. This case can be solved via the fundamental Dijkstra’s Algorithm.The main idea at the core of this algorithm is to perform a “weighted” BFS starting from the given source nodes. The algorithm iteratively grows a “cloud” of nodes out of s, with the nodes entering the cloud in order of theirdistances from s. Thus, at each iteration, the next vertex chosen is the vertex outside the cloud that is closest tos. The algorithm terminates when no more nodes are outside the cloud (or when those outside the cloud arenot connected to those within the cloud), at which point we have a shortest path from s to every node ofG thatis reachable from s.Let us define a label DistanceToVertex[v] for each node v ∈ V , which we use to approximate the distanceinG from s to v. The meaning of these labels is that DistanceToVertex[v]will always store the length of thebest path we have found so far from s to v. Initially, DistanceToVertex[s] = 0 and DistanceToVertex[v]= ∞, for each v = s. We also define the set S, which is our “cloud” of nodes, to initially be the empty set. Ateach iteration of the algorithm, we select a node u not in S with smallest DistanceToVertex[u] label, andwe pull u into S. In the very first iteration we will, of course, pull s into S. Once a new node u is pulled into S,we then update the label DistanceToVertex[v] of each node v that is adjacent to u and is outside of S, toreflect the fact that there may be a new and better way to get to v via u. This update operation is known as arelaxation procedure, for it takes an old estimate and checks if it can be improved to get closer to its true value.The specific edge relaxation operation is as follows:

252 | 268






The SPP: Dijkstra’s Algorithm

1 i f DistanceToVertex [ u ] + w(u , v ) < DistanceToVertex [ v ] :2 DistanceToVertex [ v ] = DistanceToVertex [ u]+w(u , v )

Let’s see the python code. To this end, assume that the graph is given as follows

1 mygraph = {2 ’1 ’ : { ’ 2 ’ : 2 , ’ 3 ’ : 1 , ’ 5 ’ : 3 } ,3 ’2 ’ : { ’ 1 ’ : 2 , ’ 4 ’ : 1 , ’ 5 ’ : 3 } ,4 ’3 ’ : { ’ 1 ’ : 1 , ’ 6 ’ : 1 , ’ 7 ’ : 1 } ,5 ’4 ’ : { ’ 2 ’ : 1 , ’ 5 ’ : 1 } ,6 ’5 ’ : { ’ 1 ’ : 3 , ’ 2 ’ : 3 , ’ 4 ’ : 1 } ,7 ’6 ’ : { ’ 3 ’ : 1 } ,8 ’7 ’ : { ’ 3 ’ : 1}9 }

and translated into an AM called G.

253 | 268






The SPP: Dijkstra’s Algorithm1 def D i j k s t r a (G, source_node ) :2 DistanceToVertex , PredecessorL is t = [ i n f i n i t y ]* len (G) , [− i n f i n i t y ]* len (G)3 S , DistanceToVertex [ source_node ] = [ source_node ] , 04 T=[k fo r k in range ( len (G) ) i f k !=source_node ]5 fo r k in T :6 i f G[ source_node ] [ k ] != i n f i n i t y :7 PredecessorL is t [ k ] , DistanceToVertex [ k ] = source_node , G[ source_node ] [ k ]8 while T :9 mymin , j = i n f i n i t y , − i n f i n i t y10 fo r k in T :11 i f DistanceToVertex [ k ] < mymin :12 mymin , j = DistanceToVertex [ k ] , k13 i f mymin== i n f i n i t y :14 p r i n t ( ”G i s disconnected ! Aborting . ” )15 e x i t ( 0 )16 else :17 T . remove ( j )18 S . append ( j )19 fo r k in T :20 i f G[ j ] [ k ] != i n f i n i t y and DistanceToVertex [ k]>DistanceToVertex [ j ]+G[ j ] [ k ] :21 DistanceToVertex [ k ] , PredecessorL is t [ k ] = DistanceToVertex [ j ]+G[ j ] [ k ] , j22 p r i n t ( PredecessorL is t )

254 | 268






The SPP: Dijkstra’s AlgorithmDijkstra’s Algorithm is O(n2).Which output Dijkstra’s Algorithm provides when called with the command Dijkstra(G, 0) on the input graphbelow?

1

2 5 3

4 6 7

2

3

3

11

1

11

Source code Dijkstra

255 | 268




https://colab.research.google.com/drive/1TCxxWFykTyNK3Cse6EPukKaaeQqvil5W#scrollTo=N5C5PIJXS0nn&line=11&uniqifier=1



The SPP: Dijkstra’s AlgorithmThe output is

1 [− in f , 0 , 0 , 1 , 0 , 2 , 2]

which corresponds to the following tree

1

2 5 3

4 6 7

23

1

1

11

256 | 268






The SPP: Dijkstra’s AlgorithmConsider the following input graph:

1

2 5 3

4 6 7

2

3

3

11

1

11

How do you explain the difference between the solutions provided by Prim and Dijkstra’s Algorithms?

1

2 5 3

4 6 7

2

11

1

11

Solution of Prim’s Algorithm

1

2 5 3

4 6 7

23

1

1

11

Solution of Dijkstra’s Algorithm257 | 268






The SPP: Dijkstra’s AlgorithmObservation: Sometimes, something weird happens with Dijkstra’s algorithm. For example, which is the short-est 1-3 path?

2

1

3

4

3

−2

Dijkstra’s algorithm would tell us that the shortest 1-3 path is (1,3) with a total weight of 3. But we know that theshortest path is (1,2,3) with a total weight of 2. Why does this occur? Because Dijkstra’s algorithm assumes thatthe length of a path can only either increase or remain at most equal as much as we advance.

258 | 268






The SPP: Dijkstra’s AlgorithmObservation: Sometimes, something weird happens with Dijkstra’s algorithm...For example, which is the short-est 1-3 path?

2

1

3

4

3

−2

Note that adding +2 to each edge does not help: if we add +2 we shortest path that we get is not the shortestpath on the original problem! Dijkstra’s algorithm fails because the fundamental assumption about nonde-creasing path-length is not respected when dealing with negative edge weights.

259 | 268






The SPP: Dijkstra’s AlgorithmNegative weights may often occur in practice. One example, is the following:Arbitrage. Given a number of currencies, say (dollar, euro, pound, yen) and their exchange rates, what is bestway to convert 1000 euro into other currencies and then back to euro?We can formulate this problem as a problem on a graph; Specifically:

• Each node is a currency;• Each edge is a transaction with weight equal to the exchange rate.

e

$ £

y

0.9

1.4

1

1

1

1

260 | 268






The SPP: Dijkstra’s AlgorithmWe want to find a cycle whose product of weights is greater than or equal to 1. Alternatively, we want the paththat maximizes the product of weights. The product is unfortunately nonlinear. However, if we replace eachrate wij by − log wij then the multiplication turns into an addition. So the problem can be transformed into ashortest path problem with positive and negative edge weights.

261 | 268






The SPP: Floyd-Warshall’s AlgorithmWhen dealing with arcs with negative weights two main situations may arise, namely:

• the network does not contain negative cycles, i.e., a cycle whose sum of the edge weights is negative(see e.g., figure below on the left);

• the network contains negative cycles (see e.g., figure below on the right);

2

1

3

4

3

−2

2

1

3

4

3

−8

In the first case (no negative cycles) we can still solve easily theproblembyusing either Bellman-Ford’s algorithmor Floyd-Warshall’s algorithm. Specifically, Bellman-Ford’s algorithm finds the shortest path tree from a sourceto all destinations. Floyd-Warshall’s algorithm finds the shortest paths from every node to every other node.Both algorithms can be applied when Dijkstra’s one cannot but they are a computationally more expensive, i.e.,the algorithms may take more time to find the solution.

262 | 268






The SPP: Floyd-Warshall’s AlgorithmWe shall focus on Floyd-Warshall’s algorithm. This algorithm is based on the following fundamental property.For every pair of vertices i, j ∈ V , let d(i, j) represents the length of some directed walk from node i to node jsatisfying the following condition:

d(i, j) ≤ wij (i, j) ∈ A, i = j.

�Then, the values d(i, j), for all i, j ∈ A, i = j, represent all-pairs shortest path distances if and only if theysatisfy the following optimality condition:

d(i, j) ≤ d(i, k) + d(k, j) i, j, k ∈ A, i = j = k.

Proof.(Necessity) Suppose by contradiction that for some distinct triplet i, j, k it holds that d(i, k) + d(k, j) < d(i, j).The union of the shortest paths from node i to node k and from node k to node j is a directed walk of lengthd(i, k) + d(k, j). This directed walk decomposes into a directed path p from node i to node j and a somedirected cycles.Since each directed cycle in the network has non-negative length, then the length of the path p is at mostd(i, k) + d(k, j) < d(i, j), contradicting the optimality of d(i, j).

263 | 268






The SPP: Floyd-Warshall’s Algorithm(Sufficiency). Let p be a directed path of length d(i, j) consisting of the sequence of nodes i = i1, i2, . . . , ik = j.The condition

d(i, j) ≤ d(i, k) + d(k, j) i, j, k ∈ A, i = j = k

implies that d(i, j) = d(i1, ik) ≤ d(i1, i2) + d(i2, ik) ≤ wi1i2 + d(i2, ik). Note also that d(i2, ik) ≤ wi2i3 +d(i3, ik). Hence, iteratively, we have that d(ik−1, ik) ≤ wik−1,ik

+ d(i3, ik). These conditions in turn imply that

d(i, j) ≤ wi1i2 + wi2i3 + . . . + wik−1,ik=

∑(i,j)∈p

wij .

Hence, d(i, j) is a lower bound on the length of any directed path from node i to node j. However, note thatby assumption d(i, j) is also an upper bound on the shortest path length from node i to node j. Hence, d(i, j)must be the shortest path length between these vertices.

264 | 268






The SPP: Floyd-Warshall’s AlgorithmThe above proposition suggests a possible algorithm (the Floyd-Warshall’s algorithm) to find all-pairs shortestpaths. The algorithm is based on the following inductive argument.Let d(i, j)k be the length of the shortest path from node i to node j subject to the condition to use only vertices1, 2, 3, ...k−1 as internal vertices. The algorithmfirst compute d(i, j)1 for all distinct pairs i, j ∈ V . Subsequently,the algorithm computes d(i, j)2 by using the following formula:

d(i, j)2 = min{d(i, j)1, d(i, k)1 + d(k, j)1}

i.e., it checks whether it is shorter to reach j from i by passing by node k. At the (k + 1)-th step the algorithmcomputes �

d(i, j)k+1 = min{d(i, j)k, d(i, k)k + d(k, j)k}

and stops once that k = n + 1.The algorithm reads as follows

265 | 268






The SPP: Floyd-Warshall’s Algorithm

1 def FloydWarshall ( graph ) :2 d i s t = map( lambda i : map( lambda j : j , i ) , graph )3 fo r k in range ( len ( graph ) ) :4 fo r i in range ( len ( graph ) ) :5 fo r j in range ( len ( graph ) ) :6 d i s t [ i ] [ j ] = min ( d i s t [ i ] [ j ] , d i s t [ i ] [ k ]+ d i s t [ k ] [ j ] )7 return d i s t

This algorithm is O(n3).

266 | 268






The SPP: Floyd-Warshall’s AlgorithmWhat if all weights are generic and the network contains negative cycles?In this situation solving the shortest path problem becomes much harder. This because the algorithms forshortest paths do not enumerates paths but directed walks. When negative cycles are absent the algorithmsconstruct the shortest path by applying criteria that enable us to remove potential cycles from the walk withoutincreasing the length of the path.When dealing with negative cycles we do not know any a priori criterion to remove cycles from a walk withoutincreasing the length of the path. Hence, the algorithm would enter the cycle and loop until the end.Learning how to solve the SPP with negative cycles is out of the scope of the present course and will be studiedin the course “Optimization”.

267 | 268






Brute Force Search on Graphs: Exercises1. Given a graph G = (V, E), write an algorithm to find an Hamiltonian path in G.2. Given a graph G = (V, E), write an algorithm to list all of the Hamiltonian path in G.3. Given a graph G = (V, E), write an algorithm to find an Hamiltonian circuit in G.4. Given a graph G = (V, E), write an algorithm to list all of the Hamiltonian circuit in G.5. Given a weighted graph G = (V, E), write an algorithm to find the shorted Hamiltonian path in G.6. Given a weighted graph G = (V, E), write an algorithm to find the longest Hamiltonian path in G.7. Given a weighted graph G = (V, E), write an algorithm to find the shortest Hamiltonian circuit in G.8. Given a weighted graph G = (V, E), write an algorithm to find the longest Hamiltonian circuit in G.

268 | 268




Documents

Coding Project - Projet de Programmation - MINFO1302Preface Pythonrecalls Problems&Algorithms Big-ONotation Recursion Sorting BruteForceSearch&Backtracking Graphs FundamentalProblemsonGraphs