Parallel FFT-algorithms - TUM€¦ · Parallel FFT-algorithms Shmeleva Yulia Saint-Petersburg State...

Parallel FFT-algorithms

Shmeleva YuliaSaint-Petersburg State University

Department of Computational Physics

Joint Advanced Student School 2009

Jass 2009, Shmeleva Yulia, SPbSU

Outline

• Motivation• Mathematical theory• Serial FFT• Parallel FFT

• Binary-exchange algorithm • Transpose algorithm

• Conclusion

Motivation

• Linear partial differential equations• Waveform analysis• Convolution and correlation• Digital signal processing • Image filtering

Continuous Fourier Transform

• (Forward) Fourier Transform

,2∫+∞

∞−

= dth(t)еH(f) πift

• (Inverse) Fourier Transform

,)()( 2∫+∞

∞−

−= dfefHth iftπ

where 1−=i

Discrete Fourier transform

• Finite time series , sampled at an interval Δ

][][][ Δ== khthkh k

1-N ..., ,1 ,0=k

>−=< ]1[..., ],1[],0[ Nhhhh

• The discrete (forward) Fourier transform

1-N ..., ,1 ,0=j

>−=< ]1[..., ],1[ ],0[ NHHHH

,][][1

/2∑−

NikjekhjH πwhere

• Let NiN eW /2π=

• The discrete (forward) Fourier transform

∑−

0 ][][

jkNWkhjH

• The discrete (inverse) Fourier transform

∑−

0 ][1][

jkNWjH

Each H[j] requires N multiplications

The entire sequence H needs an order of

N² operations!

DFT property of symmetry

Recall πi/NN eW 2 ⇒= 1 ,1W 2/

N −== NN

If we associate ][][ jHkh ⇔

then ][][ jHkh −⇔−

H[j]][ lkWlkh −⇔+

][][Wlk ljHkh +⇔

Serial FFT (Cooley and Tukey, 1965)

Assume that dN 2=

∑ ∑−

++=12/

2N 1]Wk2[WW]2[][

kj hkhjH

∑ ∑−

+−=+12/

2kjN 1]Wk2[W[2k]W]2/[

khhNjH

Serial FFT

Butterfly

Nj bWa +

Nj bWa −

Serial FFT (decomposition)

h[7] h[6] h[5] h[4] h[3] h[2] h[1] h[0]

h[6] h[4] h[2] h[0] h[7] h[5] h[3] h[1]

h[4] h[0] h[6] h[2] h[7] h[3]h[5] h[1]

h[0] h[4] h[2] h[6] h[5]h[1] h[3] h[7]

1=m 3=m2=m0

4NWh[6]

• iterationsN2log

• During each iteration – N operations

Serial FFT

N² operationsOriginal DFT:

operationsFFT: NN 2log

Compare!

Serial FFT (decomposition)

h[7] h[6] h[5] h[4] h[3] h[2] h[1] h[0]

h[6] h[4] h[2] h[0] h[7] h[5] h[3] h[1]

h[4] h[0] h[6] h[2] h[7] h[3]h[5] h[1]

h[0] h[4] h[2] h[6] h[5]h[1] h[3] h[7]

Serial FFT• Bit reversal sorting algorithm

Original sequence Rearranged sequence

decimal binary binary decimal0 000 000 01 001 100 42 010 010 23 011 110 64 100 001 15 101 101 56 110 011 37 111 111 7

Definitions

• Bisection width –minimum number of communication links that can be removed to break a network into two equal sized disconnected networks

bisection width is 1

Definitions

• Speedup –measure that gives the relative benefit of solving a problem in parallel

Definitions

• Efficiency –measure of the fraction of time for which a processing element is usefully employed

Definitions

• Scalability –measure of capacity to increase speedup in proportion to the number of processing elementsin order to maintain efficiency fixed.

Definitions

• Problem size –number of basic computation steps in the best sequential algorithm to solve the problem on a single processing element

Definitions

• Isoefficiency function –function which dictates the growth rate of problem size required to keep the efficiency fixed as a number of processors increases.

Parallel FFT

1. The Binary-Exchange algorithm

2. The transpose algorithm

Binary-Exchange algorithm

• Full bandwidth network• p parallel processes• bisection width is an order of p

Example: hypercube network

• Simple mapping:

One task ↔ one processp = N

• Assume thatrN 2=

5P4P3P2P1P

1=m 2=m

5P4P3P2P1P

1=m 2=m

5P4P3P2P1P

1=m 2=m

• Another mapping:multiple tasks ↔ one process

• Assume that dr pN 2 , 2 ==

• Partition the sequence into blocks of N/p elements one block ↔ one process

1=m 3=m2=m

)3 ,2( 4 , 8 ==== rdpN• Consider

1=m 3=m2=m

)3 ,2( 4 , 8 ==== rdpN• Consider

1=m 3=m2=m

)3 ,2( 4 , 8 ==== rdpN• Consider

First d iterations N/p words of data exchangepd 2log=

Last (r-d) iterationsNr 2log=

No interprocess interaction

• Denote:

st ─ the startup time for the data transfer

wt ─ the per-word transfer time

─ computation timect

• The parallel run time

ppNtptN

pNtT wscp 222 logloglog ++=

• Speedup

p)N/t(tp)p/t(tNNNpN

Scwcsp

logloglogloglog

• Efficiency

)log/()log()log/()log(11

2222 NtptNNtpptE

cwcs ++=

• Scalability

ttKppW wKt

⎭⎬⎫

⎩⎨⎧

,log ,log ,logmax 2t/

• Assume that

25 ,4 ,2 === swc ttt

• Assume that

256 ,25 ,4 ,2 ==== pttt swc

• Limited bandwidth network• p parallel processes• bisection width is less then Θ(p)

Example: a mesh interconnection network

• Mapping:

pp ×N tasks ↔ processes

• Assume thatrN 2=dp 2=

p2log steps ─communicating processes are in the same row

p2log steps ─communicating processes are in the same column

pNtptN

pNtT wscp 2loglog 22 ++=

• Speedup

p)N/t(tp)p/t(tNNNpNS

cwcs 2logloglog

• Efficiency

)log/()(2)log/()log(11

222 NtptNNtpptE

cwcs ++=

• Scalability

ttKppW pKt

⎭⎬⎫

⎩⎨⎧

,2 ,log ,logmax )t/(222

Transpose algorithm

• Two-dimensional transpose algorithm

sequence of size N → NN × array

Transpose algorithm

1. A -point FFT is computed for each column of the initial array

2. The array is transposed

3. A -point FFT for each column of the transposed array

Transpose algorithm

• Simple mappingone column ↔ one process

• Full bandwidth network• bisection width is an order of p• example: hypercube

Transpose algorithm

Transpose

Transpose algorithm

• Another mapping:Several columns ↔ one process

• Partition the array into blocks of rows one block ↔ one process

Transpose algorithm

• Speedup pNtptN

pNtT wscp +−+= )1(log 2

NttpttNNNpN

Scwcs )/()/(log

Transpose algorithm

• Efficiency

• Scalability

)log( 22 ppW Θ=

)log/()log/()(11

222 NttNNtpt

Ecwcs ++

Transpose algorithm

Compare two algorithms!

pNtptN

Binary-exchangealgorithm

Transposealgorithm

ppNtptN

Transpose algorithm

Compare two algorithms!

pNtptN

Binary-exchangealgorithm

Transposealgorithm

ppNtptN

Transpose algorithm

• Three-dimensional transpose algorithm

sequence of size N → 333 NNN ×× array

• Mapping

333 NNN ×× ↔ pp × mesh ofarray processes

Transpose algorithm

1. A -point FFT along the z-axis2. Each of the cross-sections along the

y-z plane is transposed3. A -point FFT along the z-axis4. Each of the cross-sections along the

x-z plane is transposed5. A -point FFT along the z-axis

33 NN ×

Transpose algorithm• The transposition phases in the tree-dimensional

transpose algorithm

Transpose algorithm

• Speedup

NttppttNNNpNS

cwcs )/(2)1()/(2loglog

+−+≈

pNt)p(tN

pNtT wscp 212log 2 +−+=

Comparison of described algorithms

• Assume that 64 ,25 ,4 ,2 ==== pttt swc

Conclusion

• Binary-exchange algorithm• high communication bandwidth • shared memory (Open MP)

• Transpose algorithm• limited communication bandwidth • distributed memory (MPI)

Parallel FFT software

• Parallel Engineering and Scientific Subroutine Library (PESSL)

• “Fastest Fourier Transform in the West.” (FFTW)

• Intel® Math Kernel Library (Intel® MKL)

Acknowledgments

Sergei Andreevitch Nemnyugin

Sergei Yurievitch Slavyanov

Roman Kuralev

Thank you for attention!

Parallel FFT-algorithms - TUM€¦ · Parallel FFT-algorithms Shmeleva Yulia Saint-Petersburg State...

Documents

Parallel Systolic FFT Architectures

FFT Report

Pipelined Parallel FFT Architectures Via

CSA-1 2 MHz FFT Spectrum Analyzer FFT Primer

Fft Correlation

Optimization of a parallel 3d-FFT with non-blocking ... · Optimization of a parallel 3d-FFT with non-blocking collective operations Torsten Hoeer , Gilles ZØrah Chair of Computer

Parallel FFT Libraries - 2DECOMP · Year of Presentation: 2011. Abstract The focus of this project is the area of the fast Fourier transform (FFT) libraries (FFTW 3.3, P3DFFT 2.4

A fast parallel Poisson solver on irregular domains · Peter Arbenz A fast parallel Poisson solver on irregular domains Woudschouten 2010 19 / 38. 3-dimensional FFT Let X be a 3-dimensional

4.4 The FFT and MATLAB - Kennesaw State Universityksuweb.kennesaw.edu/~plaval/math4490/fall2017/harmonic_FFT-MATLAB.pdf4.4. THE FFT AND MATLAB 69 4.4 The FFT and MATLAB 4.4.1 The FFT

Comparative Study Of Fpga Implementation Of Parallel 1-D ... · signal flow graph of 16 point DIT FFT. Comparative Study Of FPGA Implementation Of Parallel 1-D FFT Algorithm Using

FFT descreening

A simple and efficient parallel FFT algorithm using the BSP model ⋆

World’s Fastest FFT Architectures: Breaking the Barrier of ...1300794/...Index Terms—Fast Fourier transform (FFT), fully parallel, pipelined architecture. I. INTRODUCTION W ITH

MLFMA-FFT PARALLEL ALGORITHM FOR THE SO ... In Electromagnetics Research, Vol. 105, 15{30, 2010 MLFMA-FFT PARALLEL ALGORITHM FOR THE SO-LUTION OF LARGE-SCALE PROBLEMS IN ELECTRO-MAGNETICS

FreeTVing TELENAVing · dvd tv cpv35 dvd fft-189 × ― ckv36 hdd fft-193 × ms-193 btn-n17 ckv36 fft-200 ― ckv36 hdd fft-205 × ms-205 btn-n23 hdd fft-205 × ms-205 × btn-n23

A Hybrid GPU/CPU FFT Library for Large FFT Problemsxli/publications/chen2013hybrid.pdf · A Hybrid GPU/CPU FFT Library for Large FFT Problems Shuo Chen ... a hybrid parallel framework

15.Pipelined Parallel FFT Architectures via Folding Transformation (1)

Boundary element method for normal non-adhesive and ... · parallel calculation on a graphics processing unit. Using the FFT, Eq. (7) is calculated as 0 0, IFFT FFT FFT kck E uKp

CS 6068 Parallel Computing Fall 2013 Lecture 10 – Nov 18 The Parallel FFT Prof. Fred Annexstein @proffreda fred.annexstein@uc.edu Office Hours: 10-11 MWF

Inside the FFT Black Box: Serial & Parallel Fast Fourier Transform Algorithms