Sorting in Linear Time

Srichand Pendyala

The next few minutes...

1. Lower Bounds for Sorting2. Comparison based sorting3. Counting Sort4. Radix Sort5. Bucket Sort

Comparison Sorting

Comparison sorting uses comparisons of the following forms to decide the relative

order of two elements in a list

a < ba == ba > b

Decision Tree

1:2

2:3 1:3

<1,2,3> 1:3 2:3<2,1,3>

<1,3,2> <3,1,2> <2,3,1> <3,2,1>

More precisely, it can be proved, that the lower bound on comparison based sorting algorithms is Ω(n log n) comparisons in the

worst case.

Lower Bound on Comparison sorting

You want the proof? You can't han...

Determine the lower bound on the heights of decision trees in which each permutation

appears as a reachable leaf.

n! <= l <= 2h h >= lg(n!)

lg(n!) = Ω(n lg n)

Linear Ordering Sort

Linear ordering? Didn't we just prove that sorting takes Ω(n log n) ?

Remember: That's true for comparison based sorting algorithms * only *!

Linear sorts in general

In general linear sorting algorithms use operations other than comparison to

determine the sorted order of the elements.

Quotation

“The first person to sleep during my presentation would be ... me”

- Srichand Pendyala, 2003.

To stay awake: I've got 3 pictures, see if you can connect them! And no, it doesn't

have anything to do with sorting.

Counting Sort

● Counting sort assumes that each of the n input elements in an integer in the range 0 to k.

● Determine, for each input element x, the number of elements less than x.

● So, if there are 41 elements less than x then x is at the 42nd position.

Counting SortCounting-Sort(A,B,k)

for i ←0 to k do C[i] ← 0

for j ← 1 to length[A]do C[A[j]] ← C[A[j]] + 1

for i ← to kdo C[i] ← C[i] + C[i-1]

for j ← length[A] downto 1do B[C[A[j]]] ← A[j]

C[A[j]] ← C[A[j]] -1

# Initialize C to 0s

# C now contains number of elements equal to i

# C now contains the number of elements less than or equal to i

# Rearrange into output array

Counting Sort

An example

Initial array: [5, 1, 4, 6, 2]C = [1, 1, 0, 1, 1, 1]

Making it cumulativeC = [1, 2, 2, 3, 4, 5]

B = [N 2 N N N], C = [1 1 2 3 4 5] B = [N 2 N N 6], C = [1 1 2 3 4 4]B = [N 2 4 N 6], C = [1 1 2 2 4 4]B = [1 2 4 N 6], C = [0 1 2 2 4 4]B = [1 2 4 5 6], C = [0 1 2 2 3 4]

More on Counting sort

Counting sort is stable, ie numbers with the same value appear in the same order in the

output array.

Counting sort's stability is the basis for Radix sort's correctness.

Radix SortRadix sort is (was?) used by card-sorting

machines.

Sorting is based on a counting sort through each digit of the numbers.

Numbers are placed into one of 10 bins, for each of the 10 possible values of each digit

Radix sort sorts the least significant digits first. Sorting by most significant digits

requires recursive sorting in addition to keeping 9/10ths of the numbers aside.

Radix Sort

Radix-Sort(A, d)

for i ← 1 to ddo use a stable sort to sort array A on digit i

Radix Sort

Example:

A = [101, 512, 123, 521, 23, 989]A = [101, 521, 512, 123, 23, 989]A = [101, 512, 521, 123, 23, 989]A = [23, 101, 123, 512, 521, 989]

Radix Sort

How does Radix Sort compare to, say, Merge Sort?

Radix sort, which uses counting sort as the intermediate stable sort, does not sort in

place.

Bucket Sort

Bucket sort is able to sort in linear time, because it assumes something about its input, just like counting sort. It assumes

that the input array is uniformly distributed in the range [0,1)

Not stable!

It divides the interval [0,1) into n equal sized subintervals, or buckets and then distribute the n input numbers into the

buckets.

Bucket Sort

Bucket-Sort(A)n ← length[A]for i ← 1 to n

do insert A[i] into list B[[nA[i]]

for i ← 0 to n-1do sort list B[i] with insertion sort

concatenate the lists B[0], B[1], ..., B[n-1] together in order

Bucket Sort

.78

.17

.39

.26

.72

.94

.21

.12

.23

.68

.21 -> .23 -> .26

.39

/

/

.68

.72 -> .78

/

.12 -> .17

/

.94

Summary

• Comparison based sorting algorithms, and why they sort of suck.

• Linear ordering and how its sort of possible.

• Counting sort

• Radix Sort

• Bucket Sort

Questions?

“I do not promise an answer to your question. I only merely offer you the

hypothesis that I may have understood it.”- Anonymous, (Not sure when)

(Did he mean it the question, or it the answer?)

The pictures?

Any ideas?

The pictures?

Any ideas?

Tupper's Self Referential Formula

Thank you!

“A captive audience is just as good as a captivated one.” - Srichand Pendyala, 2006