Sorting algorithms

Algorithms

Sorting

Vicente García Díaz – [email protected]

University of Oviedo, 2013

mailto:[email protected]

Table of contents

1. Basic concepts

2. Sorting algorithms ▫ Simple algorithms

Sorting by direct exchange

Sorting by insertion

Sorting by selection

▫ Sophisticated algorithms

Sorting by direct exchange

Sorting by insertion

Sorting by selection

▫ Constrained algorithms

Radix

▫ External algorithms

2

Sorting

• Given a set of n elements a1, a2, a3, …, an and an order relation (≤) the problem of sorting is to sort these elements increasingly

• What can influence the sorting? ▫ The type ▫ The size ▫ The device on which they are

• Integers stored in a vector (simplification)

4

Basic concepts

Introduction

• In practice, the problems tend to be much more complex ▫ However they can be reduced to the same problem

that the integers (using registry keys, indexes, etc.)

• In essence it is the same to sort numbers or listed streets, houses, cars or any numerically quantifiable property

5

Sorting integers

Basic concepts

1. Total number of steps

▫ Complexity

2. Number of comparisons performed

3. Number of interchanges performed

▫ Much more expensive than the comparison

4. Stability

5. Type of memory used

▫ Internal algorithms

▫ External algorithms

6

Classification criteria for algorithms

Basic concepts

• We will sort arrays of integers using the Java programming language

• We will make use of utility methods to facilitate the work and make the code more readable

7

Preliminary considerations (I)

Basic concepts

8

Preliminary considerations (II)

Basic concepts

Bubble

• Description ▫ Based on the successive

comparison and exchange of adjacent elements

▫ At each step, each item is compared with the previous one and in case of being disordered, they are exchanged

▫ In the first step, the smallest element will be placed in the leftmost position, and so on…

10

Sorting algorithms Simple algorithms Sophisticated algorithms Constrained algorithms External algorithms

Sorting by direct exchange Sorting by insertion Sorting by selection

1

2

3

7

8

5 4

6

Bubble

• Initial vector:

11

Sorting algorithms

4 5 6 1 3 2 7 8

1 1 4 5 6 2 3 7 8

2 1 2 4 5 6 3 7 8

3 1 2 3 4 5 6 7 8

4 1 2 3 4 5 6 7 8

5 1 2 3 4 5 6 7 8

6 1 2 3 4 5 6 7 8

7 1 2 3 4 5 6 7 8

Simple algorithms Sophisticated algorithms Constrained algorithms External algorithms


Bubble

12

Sorting algorithms

Best case: O(n2)

Worst case: O(n2)

Average case: O(n2)

• High number of exchanges • High number of comparisons



Bubble with sentinel

• Description ▫ Observing the previous result

you can see that the last steps are repeated unnecessarily

▫ To avoid this you can enter a stop condition when the vector is sorted

If you make a pass and you do not make any exchange it means that it is already sorted

13

1

2

3

7

8

5 4

6




• Initial vector:

14

Sorting algorithms

4 5 6 1 3 2 7 8

1 1 4 5 6 2 3 7 8

2 1 2 4 5 6 3 7 8

3 1 2 3 4 5 6 7 8

4 1 2 3 4 5 6 7 8




15

Best case: O(n)

Worst case: O(n2)

Average case: O(n2)

• Complexity whether the input would be 8 1 2 3 4 5 6 7?




• Initial vector:

16

Sorting algorithms

8 1 2 3 4 5 6 7

1 1 8 2 3 4 5 6 7

2 1 2 8 3 4 5 6 7

3 1 2 3 8 4 5 6 7

4 1 2 3 4 8 5 6 7

5 1 2 3 4 5 8 6 7

6 1 2 3 4 5 6 8 7

7 1 2 3 4 5 6 7 8



• Description ▫ Used to prevent problems when

the vector is almost sorted

▫ It operates in both directions at the same time

17

1

2

3

7

8

5 4

6

Bidirectional bubble



• Initial vector:

18

8 1 2 3 4 5 6 7

1 1 2 3 4 5 6 7 8

2 1 2 3 4 5 6 7 8




19

Best case: O(n)

Worst case: O(n2)

Average case: O(n2)

• It still has a high number of comparisons and

exchanges, just as the other methods of bubble




• Description ▫ The vector is divided into two

“virtual” parts: an ordered and an unordered sequence

▫ At each step we take the first element of the unordered sequence and insert it into the corresponding place of the ordered sequence

20

1

2

3 7

8 5

4

6

Sorting algorithms

Direct insertion

Ordered sequence

Unordered sequence



• Initial vector:

21

4 5 6 1 3 2 7 8

1 4 5 6 1 3 2 7 8

Direct insertion

Sorting algorithms

Ordered vector Unordered vector

2 4 5 6

4 1 3 4 5 6

5 1 2 3 4 5 6

6 1 2 3 4 5 6 7

7 1 2 3 4 5 6 7 8

3 1 4 5 6

1 3 2 7 8

3 2 7 8

2 7 8

7 8

8



Direct insertion

• Initial vector:

22

Sorting algorithms

4 5 6 1 3 2 7 8

1 4 5 6 1 3 2 7 8

2 4 5 6 1 3 2 7 8

3 1 4 5 6 3 2 7 8

4 1 3 4 5 6 2 7 8

5 1 2 3 4 5 6 7 8

6 1 2 3 4 5 6 7 8

7 1 2 3 4 5 6 7 8



23

Best case: O(n)

Worst case: O(n2)

Average case: O(n2)

• High number of exchanges • Less comparisons than the bubble method

Direct insertion



• Description ▫ The direct insertion can be

improved taking into account that the sequence in which the elements are inserted is already sorted

▫ We use binary search to quickly locate the suitable position for the insertion

▫ *** The binary search was discussed in the introduction to Algorithms

24

1

2

3 7

8 5

4

6

Sorting algorithms

Binary insertion

Ordered sequence

Unordered sequence



Binary insertion

• Initial vector:

25

4 5 6 1 3 2 7 8

1 4 5 6 1 3 2 7 8

2 4 5 6 1 3 2 7 8

3 1 4 5 6 3 2 7 8

4 1 3 4 5 6 2 7 8

5 1 2 3 4 5 6 7 8

6 1 2 3 4 5 6 7 8

7 1 2 3 4 5 6 7 8

Sorting algorithms

The result is exactly the same as with the direct insertion



26

Best case: O(n logn)

Worst case: O(n2)

Average case: O(n2)

• Less comparisons than the direct insertion • High number of exchanges • THE INSERTION MOVING ELEMENTS ONE POSITION

IS NOT ECONOMIC

Binary insertion



2

• Description ▫ It consists on selecting the

smallest element and exchange it with the first element

▫ Repeat the process with the remaining elements until there is only one element (the greatest of all)

27

1 3 7 5 4

Sorting algorithms

Direct selection

2 3



Direct selection

• Initial vector:

28

4 5 6 1 3 2 7 8

Sorting algorithms

1 1 5 6 4 3 2 7 8

2 1 2 6 4 3 5 7 8

3 1 2 3 4 6 5 7 8

4 1 2 3 4 6 5 7 8

5 1 2 3 4 5 6 7 8

6 1 2 3 4 5 6 7 8

7 1 2 3 4 5 6 7 8



29

Best case: O(n2)

Worst case: O(n2)

Average case: O(n2)

• The number of exchanges is minimum • It is predictable: the number of exchanges and

comparisons depends on n • The number of comparisons is very high

Direct selection



• Description ▫ The idea is to divide the vector

of elements in smaller virtual vectors

▫ Each subvector is sorted using an insertion sort algorithm

▫ Each element of the subvectors is positioned at a fixed distance which decreases in each iteration (increments sequence)

30

1

2

3 7

8 5

4

6

Sorting algorithms

ShellSort

Ordered sequence

Unordered sequence



• Increments sequence: {1, 2, 4, 8} (example)

31

36 20 11 13 28 14 23 15 59 98 17 70 65 41 42 83

ShellSort

Sorting algorithms

28 14 11 13 36 20 17 15 59 41 23 70 65 98 42 83

11 13 14 15 17 20 23 28 36 41 42 59 65 70 83 98

11 13 17 14 23 15 28 20 36 41 42 70 59 83 65 98

Iteration 1 8 sublists of size 2 (increment of 8)




59 20 17 13 28 14 23 83 36 98 11 70 65 41 42 15

**The elements of the sequence of increments should not be multiples of each other



ShellSort

• Initial vector: with increments sequence {1, 3, 7}

32

4 5 6 1 3 2 7 8

1 4 5 6 1 3 2 7 8

2 1 3 2 4 5 6 7 8

3 1 2 3 4 5 6 7 8

Sorting algorithms

K=7

K=3

K=1



33

• Can be optimized by distributing the processing of each subvector

across multiple processors

ShellSort

Sorting algorithms

The complexity depends on the sequence of increments

We do not know the sequence that works best Knuth recommends: hk = 3hk-1 + 1 1, 4, 13, 40, … hk = 2hk-1 + 1 1, 3, 7, 15, 31, … O(n1.2) h1 = 1



ShellSort

• Principle on which relies ▫ THEOREM: If a sequence is sorted in n by n and

subsequently ordered at intervals of j by j, it will also be ordered in n by n

• The intervals should be decreasing

• The last interval must be 1

• The interval values should not be multiples of each other (for efficiency)

34



35

Sorting algorithms

ShellSort

Direct insertion

ShellSort

VS



• Description ▫ The idea is to create a heap of

maximums in the vector ▫ Subsequently, the first element is

chosen (maximum) and exchanged with the final element of the vector (sorted area)

▫ The heap is reorganized and we again choose the first element (maximum) to place it in the last unsorted position of the vector The process is repeated until all the

elements of the unsorted vector are placed in the sorted vector

36

Sorting algorithms

HeapSort

1

7 6

5

3 4

9

2

5

1

4

2 3

4

Unsorted area of the vector

Sorted area of the vector



HeapSort

• Idea of the algorithm CREATE HEAP OF MAXIMUMS IN THE FIRST PART OF VECTOR O(n)

GET THE ROOT ELEMENT AND EXCHANGE IT WITH THE LAST ELEMENT OF THE VECTOR (LAST POSITION) O(1)

REPEAT UNTIL THE HEAP IS EMPTY O(n)

REORGANIZE THE HEAP OF MAXIMUMS O(log n)

GET THE ROOT ELEMENT AND EXCHANGE IT WITH THE NEXT UNSORTED ELEMENT OF THE VECTOR O(1)

37

Sorting algorithms

First part

Seco

nd p

art



HeapSort

• Initial vector:

1. Create heap of maximums

38

4 5 6 1 3 2 7 8

Sorting algorithms

4

5 6

1 3 2 7

8 4

5 6

8 3 2 7

1

4

5 7

8 3 2 6

1

4

8 7

5 3 2 6

1 8

5 7

4 3 2 6

1

sinks 1 sinks 6 sinks 5

sinks 4

8 5 7 4 3 2 6 1



HeapSort

• Initial vector:

2. Exchange and restructure the heap

39

8 5 7 4 3 2 6 1

Sorting algorithms

1 5 7 4 3 2 6 8

1

5 7

4 3 2 6

7

5 6

4 3 2 1

sinks 1

7 5 6 4 3 2 1 8



HeapSort

• Initial vector:


40

7 5 6 4 3 2 1 8

Sorting algorithms

1 5 6 4 3 2 7 8

1

5 6

4 3 2

6

5 2

4 3 1

sinks 1

6 5 2 4 3 1 7 8



HeapSort

• Initial vector:


41

6 5 2 4 3 1 7 8

Sorting algorithms

1 5 2 4 3 6 7 8

1

5 2

4 3

5

4 2

1 3

sinks 1

5 4 2 1 3 6 7 8



HeapSort

• Initial vector:


42

5 4 2 1 3 6 7 8

Sorting algorithms

3 4 2 1 5 6 7 8

3

4 2

1

4

3 2

1

sinks 3

4 3 2 1 5 6 7 8



HeapSort

• Initial vector:


43

4 3 2 1 5 6 7 8

Sorting algorithms

1 3 2 4 5 6 7 8

1

3 2

3

1 2

sinks 1

3 1 2 4 5 6 7 8



HeapSort

• Initial vector:


44

3 1 2 4 5 6 7 8

Sorting algorithms

2 1 3 4 5 6 7 8

2

1

2

1

sinks 2

2 1 3 4 5 6 7 8



HeapSort

• Initial vector:


45

2 1 3 4 5 6 7 8

Sorting algorithms

1 2 3 4 5 6 7 8

1

1 2 3 4 5 6 7 8

1 sinks 1



46

• Not recommended for small vectors • Fast for large vectors

HeapSort

Sorting algorithms


Worst case: O(n logn)

Average case: O(n logn)



HeapSort

• Peculiarities ▫ First all big elements are moved to the left

▫ Then they are moved to the right

▫ The best case does not have to be the ordered vector

▫ The worst case does not have to be the unordered vector

47



• Description ▫ It is based on partitioning ▫ When you partition an item,

that item will be in its corresponding position Then, the idea is to partitionate

all the elements

▫ You have to choose a good element to partitionate (pivot) so that it is in the center and it creates a tree (if would be on one corner it will create a list)

▫ The implementation is recursive

48

Sorting algorithms

QuickSort Partitioned element

1 2 3 7 4 6

3 1 2

1 3

6 7

7



QuickSort

• Criteria for choosing a good pivot ▫ The median

It is the ideal solution but it is very expensive to compute ▫ The first element

It is "cheap" but it is a bad choice ▫ The last element

Occurs as with the first element ▫ A random element

Statistically it is not the worst choice, but has computational cost ▫ The central element

Statistically it is not a bad choice ▫ A compromise solution (median-of-3)

We obtain a sample of 3 elements and calculate the median It does not guarantee anything but it can be a good indicator We chose the first element, the last element and the center element We order the elements, and we assume that the median is the element which is

in the center

49

Sorting algorithms

1 3 8 7 2 4 6 5



QuickSort

• Idea of the algorithm REPEAT UNTIL ALL THE ELEMENTS ARE SORTED O(log n) …O(n)

CHOOSE A PIVOT Using median-of-3 is O(1)

PARTITIONING THE PIVOT THROUGH A PARTITIONING STRATEGY Typical case O(n)

50

First part

Second part



QuickSort

• Initial vector:

51

4 5 6 1 3 2 7 8

1 4 5 6 1 3 2 7 8

Sorting algorithms

1 1 5 6 4 3 2 7 8

1 1 5 6 8 3 2 7 4 Pivot

j i

Choice of pivot

1 1 5 6 8 3 2 7 4 j i

Partitioning strategy E

xch

ange

1

1 1 2 6 8 3 5 7 4

1 1 2 6 8 3 5 7 4 j i

Ex

chan

ge 2

1 1 2 3 8 6 5 7 4

1 1 2 3 8 6 5 7 4 j i

Posi

tion

ing

1 1 2 3 4 6 5 7 8

Search for an element[i] > pivot and an element[j] < pivot and interchange them i is moved to the right and j is moved to the left displacing elements element[j] < pivot to the left and elements element[i] > pivot to the right (Ends when j and i se are crossed, determining i the final position)



2 5 8 7 6

2 5 6 7 8

QuickSort

• Initial vector:

52

4 5 6 1 3 2 7 8

Sorting algorithms

2 1 2 3

1 2 3 4 6 5 7 8

Choice of pivot

2 1 2 3

Partitioning strategy

2 1 2 3

Being 3 items or less, the elements are already sorted when searching the median of 3 (ends the recursion in that branch)

2 6 5 7 8

Choice of pivot

Partitioning strategy

j i

Pivot 2 5 8 7 6 j i

Posi

tion

ing

2 5 6 7 8



3 7 8

QuickSort

• Initial vector:

53

4 5 6 1 3 2 7 8

Sorting algorithms

3 5

3 7 8

Choice of pivot

Choice of pivot

5 6 7 8

3 7 8

Being 3 items or less, the elements are already sorted when searching the median of 3 (ends the recursion in that branch)

There is only one element, then there is no need to do the median of 3 because the element is already sorted (ends the recursion in that branch)



QuickSort

• Recursive calls performed

54

4 5 6 1 3 2 7 8

Sorting algorithms

1 1 2 3 4 6 5 7 8

2 5 6 7 8 2 1 2 3

3 5 3 7 8



55


Worst case: O(n2)

Average case: O(n logn)

• Very fast for values of n > 20 • Optimizable distributing the processing of each

partition in different threads in parallel

QuickSort



56

Sorting algorithms

Comparison of algorithms (I)

n = 256 Sorted Random Inverse

Bubble 540 1026 1492

Bubble with sent. 5 1104 1645

Bidirect. bubble 5 961 1619

Direct insertion 12 366 704

Binary insertion 56 373 662

Direct selection 489 509 695

ShellSort 58 127 157

HeapSort 116 110 104

QuickSort 31 60 37


Bubble 2165 4054 5931








QuickSort 69 146 79

• Data consists of only one key

57

Sorting algorithms

Comparison of algorithms (II)


Bubble 540 1026 1492








QuickSort 31 60 37


Bubble 610 3212 5599








QuickSort 55 137 75

• Data consists of only one key VS data with a size 7 times the key size

Radix

• Description ▫ It's actually a external sorting

method ▫ It is not COMPARATIVE ▫ It can be used to sort sequences of

digits that support a lexicographic order (words, numbers, dates, ...)

▫ It consists of defining K queues[0, k-1], being k the possible values that can take each of the digits that make up the sequence

▫ Then the elements are distributed in the queues carrying different passes (ordered from the least significant digit to the most significant)

58

1 2 3

0 8 5

4 6

Sorting algorithms

Q0 Q1 Q9 …


Radix

• Initial vector:

▫ Natural numbers in base 10 10 queues

59

Sorting algorithms

48 5 86 1 3 2 25 8 0 23 27 42

0 1 2 3 4 5 6 7 8 9 Queues

First pass (least significant digit -> 0)

0 1 2, 42 3, 23 5, 25 86 27 48, 8

0 1 2 42 3 23 5 25 86 27 48 8

Second last (next least significant digit -> 1)

0, 1, 2, 3, 5, 8

23, 25, 27

42, 48 86

0 1 2 3 5 8 23 25 27 42 48 86


Radix

60

Best case: O(k * n) = O(n) Worst case: O(k * n) = O(n) Average case: O(k * n) = O(n)

• Needs external data structures • Not very good if many digits in each element • It is very efficient

Sorting algorithms

**K is the number of digits of each element


• They are based on the principle of DIVIDE AND CONQUER

• They use external memory ▫ The slowest of them is like Quicksort

• They use the same basic principles that internal sorting algorithms

• Different algorithms: ▫ Direct mixture ▫ Natural mixture ▫ Balanced mixture ▫ Polyphasic mixture

61

Sorting algorithms

External algorithms


Bibliography

62

JUAN RAMÓN PÉREZ PÉREZ; (2008) Introducción al diseño y análisis de algoritmos en Java. Issue 50. ISBN: 8469105957, 9788469105955 (Spanish)

Technology

Sorting algorithms