Outline Search Algorithms
Sort Algorithms
Module 9 Search and Sort Algorithms
Tamer Abdou, PhD
IND 830 - Python Programming for Data Science
Data Science Laboratory
Ryerson University
Tamer Abdou, PhD Module 9 Search and Sort Algorithms 1/26
Outline Search Algorithms
Sort Algorithms
Outline
1 Search Algorithms
2 Sort Algorithms
Tamer Abdou, PhD Module 9 Search and Sort Algorithms 2/26
Outline Linear Search
Search Algorithms Binary Search
Sort Algorithms
Search Algorithms
1 Search Algorithms Linear Search Binary Search
2 Sort Algorithms
Tamer Abdou, PhD Module 9 Search and Sort Algorithms 3/26
Outline Linear Search
Search Algorithms Binary Search
Sort Algorithms
Linear Search: Algorithm Design
• Two inputs are needed: an item and a list
• A loop is constructed to iterate through all the items inside the list
• On each pass, the given item is compared to the current item in the list
• The output is either the position of the matched item or −1 if the item has no match
Tamer Abdou, PhD Module 9 Search and Sort Algorithms 4/26
Outline Linear Search
Search Algorithms Binary Search
Sort Algorithms
Example 9.1 (Script)
1 def linearSearch(target, lyst): 2 #Returns the position of the target item if found,or -1 otherwise. 3 position = 0 4 while position < len(lyst): 5 if target == lyst[position]: 6 return position 7 position += 1 8 return -1 9
10 #Best-Case Scenario, O(1) 11 firstLinearSearch = linearSearch(4, [4, 5, 8, 4, 9, 3, 6, 2]) 12 print(firstLinearSearch) 13 #Average-Case Scenario, O(n) 14 secondLinearSearch = linearSearch(9, [4, 5, 8, 4, 9, 3, 6, 2]) 15 print(secondLinearSearch) 16 #Worst-Case Scenario, O(n) 17 thirdLinearSearch = linearSearch(2, [4, 5, 8, 4, 9, 3, 6, 2]) 18 print(thirdLinearSearch)
Tamer Abdou, PhD Module 9 Search and Sort Algorithms 5/26
OutlineLinear Search
Search AlgorithmsBinary Search
Sort Algorithms
Example 9.1 (Control Flow)module
Shebang line: not specifiedEncoding: not specified
def
linearSearch(target, lyst)
Returns the position of the target item if found,or -1 otherwise.
position = 0
while
position < len(lyst)
Ytarget == lyst[position]
position
position += 1
-1
Best-Case Scenario, O(1)
firstLinearSearch = linearSearch(4, [4, 5, 8, 4, 9, 3, 6, 2])print(firstLinearSearch)
Average-Case Scenario, O(n)
secondLinearSearch = linearSearch(9, [4, 5, 8, 4, 9, 3, 6, 2])print(secondLinearSearch)
Worst-Case Scenario, O(n)
thirdLinearSearch = linearSearch(2, [4, 5, 8, 4, 9, 3, 6, 2])print(thirdLinearSearch)
Tamer Abdou, PhD Module 9 Search and Sort Algorithms 6/26
OutlineLinear Search
Search AlgorithmsBinary Search
Sort Algorithms
Example 9.1 (Program Execution)(drag lower right corner to resize code editor)
Objects
functionlinearSearch(target, lyst)
list0 1 2 3 4 5 6 7
4 5 8 4 9 3 6 2
list0 1 2 3 4 5 6 7
4 5 8 4 9 3 6 2
list0 1 2 3 4 5 6 7
4 5 8 4 9 3 6 2
This mode is experimental. Use the regular Python Tutor to get live help and use more features.
Write code in Python 3.6Python 3.6Python 3.6Python 3.6Python 3.6Python 3.6Python 3.6Python 3.6
line that has just executed next line to execute
Done running (58 steps)
Help improve this tool by clicking whenever you learn something:
Print output (drag lower right corner to resize)
Frames
Global frame
linearSearch
firstLinearSearch 0
secondLinearSearch 4
thirdLinearSearch 7
linearSearch
target 4
lyst
position 0
Returnvalue 0
linearSearch
target 9
lyst
position 4
Returnvalue 4
linearSearch
target 2
lyst
position 7
Returnvalue 7
047
def linearSearch(target, lyst): #Returns the position of the target item if found,or -1 otherwise. position = 0 while position < len(lyst): if target == lyst[position]: return position position += 1 return -1
#Best-Case Scenario, O(1)firstLinearSearch = linearSearch(4, [4, 5, 8, 4, 9, 3, 6, 2])print(firstLinearSearch)#Average-Case Scenario, O(n)secondLinearSearch = linearSearch(9, [4, 5, 8, 4, 9, 3, 6, 2])print(secondLinearSearch)#Worst-Case Scenario, O(n)thirdLinearSearch = linearSearch(2, [4, 5, 8, 4, 9, 3, 6, 2])print(thirdLinearSearch)
123456789
101112131415161718
Tamer Abdou, PhD Module 9 Search and Sort Algorithms 7/26
Outline Linear Search
Search Algorithms Binary Search
Sort Algorithms
Binary Search: Algorithm Design
• Two inputs are needed: an item and a list
• The items in the list should be sorted in an ascending order
• The given item (target) is compared to the middle position item in the list
• Three-way selection statement: 1 The target matches the middle position item: Return the position
2 The target is less than the middle position item: BinarySearch the portion of the list before the middle position
3 The target is greater than the middle position item: BinarySearch the portion of the list after the middle position
• The algorithm stops when the target is found
8/26 Tamer Abdou, PhD Module 9 Search and Sort Algorithms
Outline Linear Search
Search Algorithms Binary Search
Sort Algorithms
Example 9.2 (Script)
1 def binarySearch(target, lyst): 2 lyst.sort() 3 left = 0 4 right = len(lyst) - 1 5 while left <= right: 6 midpoint = (left + right) // 2 7 if target == lyst[midpoint]: 8 return midpoint 9 elif target < lyst[midpoint]:
10 right = midpoint - 1 11 else: 12 left = midpoint + 1 13 return -1 14 15 #Best-Case Scenario 16 firstBinarySearch = binarySearch(5, [2, 3, 4, 5, 8, 9, 10, 14]) 17 print(firstBinarySearch) 18 19 #Average-Case Scenario 20 secondBinarySearch = binarySearch(3, [2, 3, 4, 5, 8, 9, 10, 14]) 21 print(secondBinarySearch) 22 23 #Worst-Case Scenario, O(log2(n)) 24 thirdBinarySearch = binarySearch(17, [2, 3, 4, 5, 8, 9, 10, 14]) 25 print(thirdBinarySearch)
Tamer Abdou, PhD Module 9 Search and Sort Algorithms 9/26
OutlineLinear Search
Search AlgorithmsBinary Search
Sort Algorithms
Example 9.2 (Control Flow)module
Shebang line: not specifiedEncoding: not specified
def
binarySearch(target, lyst)
lyst.sort()left = 0right = len(lyst) - 1
while
left <= right
midpoint = (left + right) // 2
Ytarget == lyst[midpoint]
Ytarget < lyst[midpoint]
left = midpoint + 1 right = midpoint - 1
midpoint
-1
Best-Case Scenario
firstBinarySearch = binarySearch(5, [2, 3, 4, 5, 8, 9, 10, 14])print(firstBinarySearch)
Average-Case Scenario
secondBinarySearch = binarySearch(3, [2, 3, 4, 5, 8, 9, 10, 14])print(secondBinarySearch)
Worst-Case Scenario, O(log2(n))
thirdBinarySearch = binarySearch(17, [2, 3, 4, 5, 8, 9, 10, 14])print(thirdBinarySearch)
Tamer Abdou, PhD Module 9 Search and Sort Algorithms 10/26
OutlineLinear Search
Search AlgorithmsBinary Search
Sort Algorithms
Example 9.2 (Program Execution)(drag lower right corner to resize code editor)
Objects
functionbinarySearch(target, lyst)
list0 1 2 3 4 5 6 7
2 3 4 5 8 9 10 14
list0 1 2 3 4 5 6 7
2 3 4 5 8 9 10 14
list0 1 2 3 4 5 6 7
2 3 4 5 8 9 10 14
This mode is experimental. Use the regular Python Tutor to get live help and use more features.
Write code in Python 3.6Python 3.6Python 3.6Python 3.6Python 3.6Python 3.6Python 3.6Python 3.6
line that has just executed
Print output (drag lower right corner to resize)
Frames
Global frame
binarySearch
firstBinarySearch 3
secondBinarySearch 1
thirdBinarySearch -1
binarySearch
target 5
lyst
left 0
right 7
midpoint 3
Returnvalue 3
binarySearch
target 3
lyst
left 0
right 2
midpoint 1
Returnvalue 1
binarySearch
target 17
lyst
left 8
right 7
midpoint 7
Returnvalue -1
31-1
def binarySearch(target, lyst): lyst.sort() left = 0 right = len(lyst) - 1 while left <= right: midpoint = (left + right) // 2 if target == lyst[midpoint]: return midpoint elif target < lyst[midpoint]: right = midpoint - 1 else: left = midpoint + 1 return -1
#Best-Case Scenario, O(1)firstBinarySearch = binarySearch(5, [2, 3, 4, 5, 8, 9, 10, 14])print(firstBinarySearch)
#Average-Case Scenario, O(log2(n))secondBinarySearch = binarySearch(3, [2, 3, 4, 5, 8, 9, 10, 14])print(secondBinarySearch)
#Worst-Case Scenario, O(log2(n))thirdBinarySearch = binarySearch(17, [2, 3, 4, 5, 8, 9, 10, 14])print(thirdBinarySearch)
123456789
10111213141516171819202122232425
Tamer Abdou, PhD Module 9 Search and Sort Algorithms 11/26
Outline Selection Sort Search Algorithms Bubble Sort
Sort Algorithms Insertion Sort
Sort Algorithms
1 Search Algorithms
2 Sort Algorithms Selection Sort Bubble Sort Insertion Sort
Tamer Abdou, PhD Module 9 Search and Sort Algorithms 12/26
Outline Selection Sort Search Algorithms Bubble Sort
Sort Algorithms Insertion Sort
Selection Sort: Algorithm Design
• Only one input is required; either an ordered or unordered list
• A nested loop is required:
Outer loop: Traverses the list from the beginning, and on each pass, reorders two elements if needed
Inner loop: Compares the value of the engaged element with the values of the remaining elements in the list, and returns the position of the element with the least value
Tamer Abdou, PhD Module 9 Search and Sort Algorithms 13/26
Outline Selection Sort Search Algorithms Bubble Sort
Sort Algorithms Insertion Sort
Example 9.3 (Script)
1 def selectionSort(lyst): 2 i = 0 3 while i < len(lyst) - 1: # Do n - 1 searches 4 minIndex = i # for the smallest 5 j = i + 1 6 while j < len(lyst): # Start another search 7 if lyst[minIndex] > lyst[j]: 8 minIndex = j 9 j += 1
10 if minIndex != i: # Swap if needed 11 lyst[minIndex], lyst[i] = lyst[i], lyst[minIndex] 12 i += 1 13 return(lyst) 14
15 #The complexity in all cases is O(n^2) 16 selectionSort = selectionSort([5, 6, 8, 3, 2, 1, 9, 4]) 17 print(selectionSort)
Tamer Abdou, PhD Module 9 Search and Sort Algorithms 14/26
Outline Selection SortSearch Algorithms Bubble Sort
Sort Algorithms Insertion Sort
Example 9.3 (Control Flow)module
Shebang line: not specifiedEncoding: not specified
def
selectionSort(lyst)
i = 0
while
i < len(lyst) - 1 Do n - 1 searches
minIndex = ij = i + 1
for the smallest
while
j < len(lyst) Start another search
Ylyst[minIndex] > lyst[j]
minIndex = j
j += 1
YminIndex != i Swap if needed
lyst[minIndex], lyst[i] = lyst[i], lyst[minIndex]
i += 1
(lyst)
The complexity in all cases is O(n^2)
selectionSort = selectionSort([5, 6, 8, 3, 2, 1, 9, 4])print(selectionSort)
Tamer Abdou, PhD Module 9 Search and Sort Algorithms 15/26
Outline Selection SortSearch Algorithms Bubble Sort
Sort Algorithms Insertion Sort
Example 9.3 (Program Execution)
(drag lower right corner to resize code editor)
Objects
list0 1 2 3 4 5 6 7
1 2 3 4 5 6 8 9
This mode is experimental. Use the regular Python Tutor to get live help and use more features.
Write code in Python 3.6Python 3.6Python 3.6Python 3.6Python 3.6Python 3.6Python 3.6Python 3.6
line that has just executed next line to execute
Done running (151 steps)
Help improve this tool by clicking whenever you learn something:
Print output (drag lower right corner to resize)
Frames
Global frame
selectionSort
selectionSort
lyst
i 7
minIndex 7
j 8
Returnvalue
Click the button above to create a permanent link to your visualization (video demo).
This live programming mode of Python Tutor (code on GitHub) supports three languages: Python 2.7 and 3.6 withlimited module imports, and JavaScript running in Node.js v6.0.0 with limited support for ES6. Try the regularPython Tutor visualizer for additional language support.
Privacy Policy: By using Python Tutor, your visualized code, options, user interactions, text chats, and IP addressare logged on our server and may be analyzed for research purposes. Nearly all web services collect this basicinformation from users in their server logs. However, Python Tutor does not collect any personally identifiableinformation from its users. It uses Google Analytics for website analytics.
[1, 2, 3, 4, 5, 6, 8, 9]
def selectionSort(lyst): i = 0 while i < len(lyst) - 1: # Do n - 1 searches minIndex = i # for the smallest j = i + 1 while j < len(lyst): # Start another search if lyst[minIndex] > lyst[j]: minIndex = j j += 1 if minIndex != i: # Swap if needed lyst[minIndex], lyst[i] = lyst[i], lyst[minIndex] i += 1 return(lyst)
#The complexity in all cases is O(n^2)selectionSort = selectionSort([5, 6, 8, 3, 2, 1, 9, 4]) print(selectionSort)
1234567891011121314151617
Tamer Abdou, PhD Module 9 Search and Sort Algorithms 16/26
Outline Selection Sort Search Algorithms Bubble Sort
Sort Algorithms Insertion Sort
Bubble Sort: Algorithm Design
• One input is needed; either an ordered or unordered list, and on each pass it leaves
• A nested loop is required:
Outer loop: Traverses the pairs in the given list, and on each pass it leaves one element out (bubbles it up)
Inner loop: Compares the values of the engaged pairs and swaps the values if they are out of order
Tamer Abdou, PhD Module 9 Search and Sort Algorithms 17/26
Outline Selection Sort Search Algorithms Bubble Sort
Sort Algorithms Insertion Sort
Example 9.4 (Script)
1 def bubbleSort(lyst): 2 n = len(lyst) 3 while n > 1: # Do n - 1 bubbles 4 i = 1 # Start each bubble 5 while i < n: 6 if lyst[i] < lyst[i - 1]: # Swap if needed 7 lyst[i], lyst[i - 1] = lyst[i-1], lyst[i] 8 i += 1 9 n -= 1
10 return(lyst) 11
12 #On average, the behavior is O(n^2) 13 bubbleSort = bubbleSort([5, 6, 8, 3, 2, 1, 9, 4]) 14 print(bubbleSort)
Tamer Abdou, PhD Module 9 Search and Sort Algorithms 18/26
Outline Selection SortSearch Algorithms Bubble Sort
Sort Algorithms Insertion Sort
Example 9.4 (Control Flow)module
Shebang line: not specifiedEncoding: not specified
def
bubbleSort(lyst)
n = len(lyst)
while
n > 1 Do n - 1 bubbles
i = 1 Start each bubble
while
i < n
lyst[i] < lyst[i - 1]n
Swap if needed
lyst[i], lyst[i - 1] = lyst[i-1], lyst[i]
i += 1
n -= 1
(lyst)
On average, the behavior is O(n^2)
bubbleSort = bubbleSort([5, 6, 8, 3, 2, 1, 9, 4])print(bubbleSort)
Tamer Abdou, PhD Module 9 Search and Sort Algorithms 19/26
Outline Selection SortSearch Algorithms Bubble Sort
Sort Algorithms Insertion Sort
Example 9.4 (Program Execution)
(drag lower right corner to resize code editor)
Objects
list0 1 2 3 4 5 6 7
1 2 3 4 5 6 8 9
This mode is experimental. Use the regular Python Tutor to get live help and use more features.
Write code in Python 3.6Python 3.6Python 3.6Python 3.6Python 3.6Python 3.6Python 3.6Python 3.6
line that has just executed next line to execute
Done running (136 steps)
Help improve this tool by clicking whenever you learn something:
Print output (drag lower right corner to resize)
Frames
Global frame
bubbleSort
bubbleSort
lyst
n 1
i 2
Returnvalue
Click the button above to create a permanent link to your visualization (video demo).
This live programming mode of Python Tutor (code on GitHub) supports three languages: Python 2.7 and 3.6 withlimited module imports, and JavaScript running in Node.js v6.0.0 with limited support for ES6. Try the regularPython Tutor visualizer for additional language support.
Privacy Policy: By using Python Tutor, your visualized code, options, user interactions, text chats, and IP addressare logged on our server and may be analyzed for research purposes. Nearly all web services collect this basicinformation from users in their server logs. However, Python Tutor does not collect any personally identifiableinformation from its users. It uses Google Analytics for website analytics.
Terms of Service: The Python Tutor service is provided for free on an as-is basis. Use this service at your own risk.Do not use it to share confidential information. The developers of Python Tutor are not responsible for the chatmessages or behaviors of any of the users on this website. We are also not responsible for any damages caused byusing this website. Finally, it is your responsibility to follow appropriate academic integrity standards.
[1, 2, 3, 4, 5, 6, 8, 9]
def bubbleSort(lyst): n = len(lyst) while n > 1: # Do n - 1 bubbles i = 1 # Start each bubble while i < n: if lyst[i] < lyst[i - 1]: # Swap if needed lyst[i], lyst[i - 1] = lyst[i-1], lyst[i] i += 1 n -= 1 return(lyst)
#On average, the behavior is O(n^2)bubbleSort = bubbleSort([5, 6, 8, 3, 2, 1, 9, 4]) print(bubbleSort)
123456789
1011121314
Tamer Abdou, PhD Module 9 Search and Sort Algorithms 20/26
Outline Selection Sort Search Algorithms Bubble Sort
Sort Algorithms Insertion Sort
Insertion Sort - Algorithm Design
• One input is needed: either an ordered or unordered list
• A nested loop is required:
Outer loop: Traverses the list from left to right starting with the second item
Inner loop: Traverses the list from right to left starting with the engaged item. Inserts/updates the prior item with the value of the engaged item, if they are out of order
Tamer Abdou, PhD Module 9 Search and Sort Algorithms 21/26
Outline Selection Sort Search Algorithms Bubble Sort
Sort Algorithms Insertion Sort
Example 9.5 (Script)
1 def insertionSort(lyst): 2 i = 1 3 while i < len(lyst): 4 itemToInsert = lyst[i] 5 j = i - 1 6 while j >= 0: 7 if itemToInsert < lyst[j]: 8 lyst[j + 1] = lyst[j] 9 j -= 1
10 else: 11 break 12 lyst[j + 1] = itemToInsert 13 i += 1 14 return(lost) 15
16 #On average, the complexity is O(n^2) 17 insertionSort = insertionSort([5, 6, 8, 3, 2, 1, 9, 4]) 18 print(insertionSort)
Tamer Abdou, PhD Module 9 Search and Sort Algorithms 22/26
Outline Selection SortSearch Algorithms Bubble Sort
Sort Algorithms Insertion Sort
Example 9.5 (Control Flow)module
Shebang line: not specifiedEncoding: not specified
def
insertionSort(lyst)
i = 1
while
i < len(lyst)
itemToInsert = lyst[i]j = i - 1
while
j >= 0
itemToInsert < lyst[j]n
break lyst[j + 1] = lyst[j]j -= 1
lyst[j + 1] = itemToInserti += 1
(lyst)
On average, the complexity is O(n^2)
insertionSort = insertionSort([5, 6, 8, 3, 2, 1, 9, 4])print(insertionSort)
Tamer Abdou, PhD Module 9 Search and Sort Algorithms 23/26
Outline Selection SortSearch Algorithms Bubble Sort
Sort Algorithms Insertion Sort
Example 9.5 (Program Execution)
(drag lower right corner to resize code editor)
Objects
list0 1 2 3 4 5 6 7
1 2 3 4 5 6 8 9
This mode is experimental. Use the regular Python Tutor to get live help and use more features.
Write code in Python 3.6Python 3.6Python 3.6Python 3.6Python 3.6Python 3.6Python 3.6Python 3.6
line that has just executed next line to execute
Done running (122 steps)
Help improve this tool by clicking whenever you learn something:
Print output (drag lower right corner to resize)
Frames
Global frame
insertionSort
insertionSort
lyst
i 8
itemToInsert 4
j 2
Returnvalue
Click the button above to create a permanent link to your visualization (video demo).
This live programming mode of Python Tutor (code on GitHub) supports three languages: Python 2.7 and 3.6 withlimited module imports, and JavaScript running in Node.js v6.0.0 with limited support for ES6. Try the regularPython Tutor visualizer for additional language support.
Privacy Policy: By using Python Tutor, your visualized code, options, user interactions, text chats, and IP addressare logged on our server and may be analyzed for research purposes. Nearly all web services collect this basicinformation from users in their server logs. However, Python Tutor does not collect any personally identifiable
[1, 2, 3, 4, 5, 6, 8, 9]
def insertionSort(lyst): i = 1 while i < len(lyst): itemToInsert = lyst[i] j = i - 1 while j >= 0: if itemToInsert < lyst[j]: lyst[j + 1] = lyst[j] j -= 1 else: break lyst[j + 1] = itemToInsert i += 1 return(lyst)
#On average, the complexity is O(n^2)insertionSort = insertionSort([5, 6, 8, 3, 2, 1, 9, 4]) print(insertionSort)
123456789101112131415161718
Tamer Abdou, PhD Module 9 Search and Sort Algorithms 24/26
Outline Selection SortSearch Algorithms Bubble Sort
Sort Algorithms Insertion Sort
Algorithm Performance
Figure 9.1: Big-O Computational Complexity Functions
Table 9.1: Types of Performance
Best Average WorstAlgorithm
Case Case Case
Linear Search O(1) O(n) O(n)
Binary Search O(1) O(log2n) O(log2n)
Selection Sort O(n2) O(n2) O(n2)
Bubble Sort O(n) O(n2) O(N2)
Insertion Sort O(n) O(n2) O(n2)
25/26Tamer Abdou, PhD Module 9 Search and Sort Algorithms
Outline Selection Sort Search Algorithms Bubble Sort
Sort Algorithms Insertion Sort
References
• Lambert, K. A. (2010). Searching, sorting, and complexity analysis. In Fundamentals of Python: From first programs through data structures (pp. 431-459). Boston, MA: Cengage Learning
• Python Software Foundation. (2020). Python 3.7.8 documentation. Retrieved from https://docs.python.org/3.7/index.html.
Tamer Abdou, PhD Module 9 Search and Sort Algorithms 26/26