Algorithms: Sorting. Rand Sort. Compare-and-exchange. Merge Sort. Quick Sort. Odd-even merge sort....

Algorithms: Sorting

. Rand Sort

. Compare-and-exchange

. Merge Sort

. Quick Sort

. Odd-even merge sort

. Bitonic merge sort

Rank Sort

for(i=0 ; i<n ; i++){ x=0; for(j=0 ; j<n ; j++) if( a[i]>a[j]) x++; b[x] = a[i];}

O(n2)

Rank Sort

Using n processors

forall(i=0 ; i<n ; i++){ x=0; for(j=0 ; j<n ; j++) if(a[i] > a[j] ) x++: b[x] = a[i];}

O(n)

Rank Sort

Using n2 processors O(1)

Rank Sort

Compare-and-Exchange

if(A > B) { temp = A; A = B; B = temp;}

Compare-and-Exchange

Process P1send(&A, P2);recv(&A, P2);

Process P2recv(&A, P1);if(A > B){ send(&B, P1); B = A;}else send(&A, P1);

Compare-and-Exchange

Process P1send(&A, P2);recv(&B, P2);if(A > B) A = B;

Process P2recv(&A, P1);send(&B, P1);if(A > B) B = A;

Compare-and-Exchange

Data partitioning

Compare-and-Exchange

Data partitioning

Bubble Sort and Odd-even Transposition Sort

Bubble Sort and Odd-even Transposition Sort

for(i=n-1 ; i > 0 ; i++) for(j=0 ; j < i ; j++){ k = j+1; if( a[j] < a[k]) { temp = a[j]; a[i] = a[k]; a[k] = temp; } }

O(n2)

Bubble Sort and Odd-even Transposition Sort

Parallel Code

Bubble Sort and Odd-even Transposition Sort

Bubble Sort and Odd-even Transposition Sort

Pi,i=0,2,4,6,...,n-2(even)

recv(&A, Pi+1);

send(&B, Pi+1);

if(A > B) B = A;

Pi,i=1,3,5,..,n-1(odd)

send(&A,Pi-1);

recv(&B, Pi-1);

if(A > B) A = B;

Bubble Sort and Odd-even Transposition Sort

Pi,i=1,3,5,...,n-3(odd)

send(&A, Pi+1);

recv(&B, Pi+1);

if(A > B) A = B;

Pi,i=2,4,6,...,n-2(even)

recv(&A, Pi-1);

send(&B, Pi-1);

if(A > B) B = A;

Bubble Sort and Odd-even Transposition Sort

Pi,i=0,2,4,...,n-1(odd)send(&A, Pi-1);recv(&B, Pi-1);if(A > B) A=B;if(i<=n-3){ send(&A, Pi+1); recv(&B, Pi+1); if(A < B) A=B;}

Pi,i=0,2,4,...,n-2(even)recv(&A, Pi+1);send(&B, Pi+1);if(A > B) B = A;if(i >= 2){ recv(&A, Pi-1); send(&B, Pi-1); if(A > B) B=A;}

Bubble Sort and Odd-even Transposition Sort

Two-Dimension Sorting

Bubble Sort and Odd-even Transposition Sort

Odd Phase, the following actios are done:

Each row of numbers is sorted independently, in alternative directions:

Even rows: The smallest number of each column is placed at rightmost end

and largest number at the leftmost end

Odd rows: The smallest number of each column is placed at the leftmost end

and the largest number at the rightmost end.

In even phase, the following actions are done:

Each column of numbers is sorted independently, placing the smallest number of each column at the leftmost end and the largest number at the rightmost end.

)1(log nn

Bubble Sort and Odd-even Transposition Sort

Bubble Sort and Odd-even Transposition Sort

Merge Sort

Merge Sort

Communication (division phase)

datastartup tnt )2/(

datastartup tnt )4/(

datastartup tnt )8/(

Communication at each stepProcessor communication

P0->P4

P0->P2;P4->P6

P0->P1;P2->P3;P4->P5;P6->P7

Merge Sort

Communication (merge phase)

datastartup tnt )8/(

datastartup tnt )4/(

datastartup tnt )2/(

Communication at each stepProcessor communication

P0->P4

P0->P2;P4->P6

P0->P1;P2->P3;P4->P5;P6->P7

Merge Sort

Communication

datastartupcomm

datastartupdatastartupdatastartupcomm

nttpt

tnttnttntt

2)(log2

....))8/()4/()2/((2

Merge Sort

Computation

1compt

3compt

7compt P0

P0;P4

P0;P2;P4;P6

p

i

icompt

log

1

)12(

The parallel computational time complexity is O(p) using p processorsand one number in each processor

Quick Sort

quicksort(list, start, end)

{

if(start < end)

partition(list, start, end pivot);

quicksort(list, start, pivot-1);

quicksort(list, pivot-1, end);

}

Quick Sort

Quick Sort

Quick Sort

Computation

nnnnntcomp 2....8/4/2/

Communication

datastartup

datastartupdatastartupdatastartupcomm

nttp

tnttnttntt

)(log

...))8/(())4/(())2/((

Quick Sort

Implementation

Quick Sort on Hypercube

Complete list in one processor

1st step: 000 -> 100 (numbers greater than a pivot, say p1)

2nd step: 000 -> 010 (numbers greater than a pivot, say p2)

100 -> 110 (numbers greater than a pivot, say p3)

3rd step: 000 -> 001 (numbers greater than a pivot, say p4)

010 -> 011 (numbers greater than a pivot, say p5)

100 -> 101 (numbers greater than a pviot, say p6)

110 -> 111 (numbers greater than a pivot, say p7)

Quick Sort on Hypercube

Quick Sort on Hypercube

Number initially distributed across all processors

1. one processor(say P0) selects (or computers) a suitable pivot and broadcast this to all others in the cube

2. The processors in the lower subcube send their numbers, which are greater than the pivot, to their partner processor in the upper subcube. The processors in the upper subcube send their numbers, which are equal to or less than the pivot, to their partner processor in the lower cube.

3. Each processor concatenates the list received with what remains of its own list.

Quick Sort on Hypercube

Quick Sort on Hypercube

Quick Sort on Hypercube

1. Each processor sorts its list sequentially.

2. one processor(say P0) selects (or computers) a suitable pivot

and broadcast this to all others in the cube

3. The processors in the lower subcube send their numbers, which are

greater than the pivot, to their partner processor in the upper subcube.

The processors in the upper subcube sned their numbers, which are equal

to or less than the pivot, to their partner processor in the lower cube.

4. Each processor merger the list received with its own to obtain a sorted list.

Quick Sort on Hypercube

Quick Sort on Hypercube

Computation-Pivot Selection O(1) : the (n/2p)th number

Communication-Pivot broadcast

Computation-Data split if the numbers are sorted and there are x numbers, the split operation can be done in logx steps. (same as binary search)

Communication-Data from split

Computation-Data Merge to merge two sorted lists into one sorted list requires x steps if the biggest list has x numbers

)(2

)1(

2

)1()(

1

0

datastartup

d

i

ttdd

ddid

datastartup tx

t2

Odd-even Merge Sort

1. The elements with odd indices of each sequence-that is, A1, A3,A5,…,An-1, and B1, B3,B5,…,Bn-1---are merged into one sorted list, C1, C2, C3,…,Cn

2. The elements with even indices of each sequence---that is A2,A4,A6,…,An, and B2,B4,B6,…,Bn-2---are merged into one sorted list, D1, D2, D3,…,Dn.

3. The final sorted list, E1, E2,…,E2n, is obtained by the following: E2i=min{Ci+1, Di} E2i+1=max{Ci+1,Di} for 1<=i<=n-1

Odd-even Merge Sort

Odd-even Merge Sort

Bitonic Merge Sort

Bitonic sequence

A0<A1<A2<A3<…,Ai-1<Ai>Ai+1>Ai+2>…..>An

3,5,8,9,7,4,2,1

Bitonic Merge Sort

Bitonic Merge Sort

Bitonic Merge Sort

Bitonic Merge Sort

Phase 1(step1) Covert pairs of numbers into increasing/decreasing sequences

and hence into 4-bit bitonic sequences

Phase 2(step2/3) Split each 4-bit bitonic sequence into two 2-bit bitonic

sequences, higher sequence at center.

Sort each 4-bit bitonic sequence increasing/decreasing

sequences and merge into 8-bit bitonic sequence.

Phase 3(step4/5/6) Sort 8-bit bitonic sequence

Bitonic Merge Sort

Bitonic Merge Sort