As simple as sorting

OutlineIntroduction

nlogn lower boundSorting in linear time

Conclusion

As simple as sorting

Wojciech Kazana

INRIA Saclay, ENS de Cachan

”Gouter des doctorants”, Cachan, June 26th 2012

1 / 22 Wojciech Kazana As simple as sorting

OutlineIntroduction


Conclusion

1 Introduction

2 nlogn lower bound

3 Sorting in linear time

4 Conclusion


OutlineIntroduction


Conclusion

1 Introduction

2 nlogn lower bound


4 Conclusion


OutlineIntroduction


Conclusion

The problem of sorting can be formalized in the following way:INPUT:A list of elements a1, a2, . . . , an belonging to some domain A.OUTPUT:A sorted (with respect to some order ≤ over A) listb1 ≤ b2 ≤ . . . ≤ bn such that there exists a mappingf : [1, n] → [1, n] such that for each i ∈ [1, n] we have bf (i) = ai .


OutlineIntroduction


Conclusion

Example:Consider list L = {73, 16, 323, 1, 64, 2, 0} of natural numbers (N)and a less-or-equal order ≤ over N.Then the solution to our instance is the following list:{0, 1, 2, 16, 64, 73, 323}.


OutlineIntroduction


Conclusion

In our case we assume that the input is always a list (sequence) ofnatural numbers.


OutlineIntroduction


Conclusion

In our case we assume that the input is always a list (sequence) ofnatural numbers.We will be interested in the complexity of an algorithm sorting agiven list.


OutlineIntroduction


Conclusion

There are thousands of examples showing that sorting is a real lifeproblem. To mention at least one:


OutlineIntroduction


Conclusion

There are thousands of examples showing that sorting is a real lifeproblem. To mention at least one:You perform a Google search. You want “the most interesting”pages to appear first and then (sometimes) the rest might beuseful too. Google sorts the pages for you with respect to theorder “interesting”.


OutlineIntroduction


Conclusion

1 Introduction

2 nlogn lower bound


4 Conclusion


OutlineIntroduction


Conclusion

As an input we get the sequence a1, . . . , an.


OutlineIntroduction


Conclusion

As an input we get the sequence a1, . . . , an.For simplicity assume that for i 6= j we have ai 6= aj .


OutlineIntroduction


Conclusion

As an input we get the sequence a1, . . . , an.For simplicity assume that for i 6= j we have ai 6= aj .How fast can we sort the list?


OutlineIntroduction


Conclusion

Sorting the input list can be viewed as: find a permutation p ofinterval [1, n] such that ap(1) ≤ ap(2) ≤ . . . ≤ ap(n). (In a sense, wethink about the place of each ai in the final order.)


OutlineIntroduction


Conclusion

Sorting the input list can be viewed as: find a permutation p ofinterval [1, n] such that ap(1) ≤ ap(2) ≤ . . . ≤ ap(n). (In a sense, wethink about the place of each ai in the final order.)Each comparison test of the form ?(ai < aj)? divide our “statespace” into two disjoint worlds: one with p(i) < p(j) and the otherwith p(j) < p(i).


OutlineIntroduction


Conclusion

Sorting the input list can be viewed as: find a permutation p ofinterval [1, n] such that ap(1) ≤ ap(2) ≤ . . . ≤ ap(n). (In a sense, wethink about the place of each ai in the final order.)Each comparison test of the form ?(ai < aj)? divide our “statespace” into two disjoint worlds: one with p(i) < p(j) and the otherwith p(j) < p(i).This quickly brings us to decision trees:

nodes are comparison tests,

leafs are different permutations of [1, n].


OutlineIntroduction


Conclusion

We are dealing with binary decision trees with:

nodes are comparison tests,

leafs are different permutations of [1, n].


OutlineIntroduction


Conclusion

How many leafs are there?

At least how many comparisons does our algorithm need?


OutlineIntroduction


Conclusion

How many leafs are there? n!

At least how many comparisons does our algorithm need?Length of the longest path.


OutlineIntroduction


Conclusion



What is the length of the longest path?


OutlineIntroduction


Conclusion



What is the length of the longest path? log(n!)


OutlineIntroduction


Conclusion

So for n elements in the input we need at least:log(n!) = nlogn − O(n) + O(logn) = O(nlogn) comparisons.


OutlineIntroduction


Conclusion

So for n elements in the input we need at least:log(n!) = nlogn − O(n) + O(logn) = O(nlogn) comparisons.And we cannot hope for anything better...


OutlineIntroduction


Conclusion

1 Introduction

2 nlogn lower bound


4 Conclusion


OutlineIntroduction


Conclusion

Example:Consider list L = {73, 16, 323, 1, 64, 2, 0} of natural numbers (N)and a less-or-equal order ≤ over N.Then the solution to our instance is the following list:{0, 1, 2, 16, 64, 73, 323}.


OutlineIntroduction


Conclusion

Example:Consider list L ={73123123, 16123123, 323123123, 1123123, 64123123, 2123123, 123123}of natural numbers (N) and a less-or-equal order ≤ over N.Then the solution to our instance is the following list:{123123, 1123123, 2123123, 16123123, 64123123, 73123123, 323123123}.But does it make sense to count the comparisons in this case?


OutlineIntroduction


Conclusion

It is hard to say. But when we think about strings (or largeintegers), by the size of the input {l1, . . . , ln} of n words we usuallymean

∑1≤i≤n |li | and not just n.


OutlineIntroduction


Conclusion

It is hard to say. But when we think about strings (or largeintegers), by the size of the input {l1, . . . , ln} of n words we usuallymean

∑1≤i≤n |li | and not just n.

And this changes everything!


OutlineIntroduction


Conclusion

Etienne Grandjean in ’96 (in a very formal way), but also Aho,Hopcroft and Ullman some time earlier have shown a linear timealgorithm for sorting strings of arbitrary lengths.


OutlineIntroduction


Conclusion

Etienne Grandjean in ’96 (in a very formal way), but also Aho,Hopcroft and Ullman some time earlier have shown a linear timealgorithm for sorting strings of arbitrary lengths.But here linear means linear in the size of the sum of the lengths

of input words.


OutlineIntroduction


Conclusion

1 Introduction

2 nlogn lower bound


4 Conclusion


OutlineIntroduction


Conclusion

As for the conclusions:

comparison model is very nice, but how does it fit into thereal life?


OutlineIntroduction


Conclusion



which number is bigger: 23415864587156425629 or234158645871564255629?


OutlineIntroduction


Conclusion




the linear time sorting algorithms sort the inputlexicographically, the order is not arbitrary!


OutlineIntroduction


Conclusion




the linear time sorting algorithms sort the inputlexicographically, the order is not arbitrary!

there is no magic! As Socrates said: to understand each otherwe need to agree on the same meaning of words. The nlogn

lower bound is real, it just does not fit to the other model.


OutlineIntroduction


Conclusion

Some “fun” (and maybe also not that fun) facts:

it started with (quadratic in terms of # comparisons)bubble-sort and insertion-sort,

the breakthrough was Shell-sort (Shell ’59, Frank and Lazarus’60, Pratt ’71) with complexities dropping from O(n2),

through O(n32 ), to O(nlog2n),

we have plenty of nlogn sorting algorithms now, likemerge-sort, heap-sort and quick-sort.

and there is always linear bucket-sort or radix-sort for inputfrom small domain.


OutlineIntroduction


Conclusion

Thank You!


OutlineIntroduction


Conclusion

Thank You!

Any questions?


Documents

As simple as sorting