Chapter 12

Fast Fourier Transform

1. Metropolis algorithm for Monte Carlo

2. Simplex method for linear programming

3. Krylov subspace iteration (CG)

4. Decomposition approach to matrix computation (LU, Singular value)

5. The Fortran compiler

6. QR algorithm for eigenvalues

7. Quick sort

8. Fast Fourier transform

9. Integer relation detection

10. Fast multipole

Definition of Fourier Transform

2

2

( ) ( ) , 1

( ) ( ) , 2

i f t

i f t

H f h t e dt i

h t H f e df f

Convolution, Correlation, and Power

( ) ( ) ( ) ( ) ( ) ( )g h t g h t d G f H f

[ * ]( ) ( ) ( ) ( ) ( )g h t g h t d G f H f

Autocorrelation if g = h. Autocorrelation is equal to power spectrum |G(f)|2 in frequency space.

2 2| ( ) | | ( ) |h t dt H f df

Total power:

Sampling Theorem• Let Δ be the spacing in time domain, with hn=h(nΔ), n = …,-2,-1,0,1,2,…, the sampled value of continuous function h(t). Let

fc=1/(2Δ) [Nyquist critical frequency]. Then if H( f ) = 0 for all | f | ≥ fc, then the function h(t) is completely determined by hn. sin 2 ( )

( )( )

cn

n

f t nh t h

t n

AliasingFigure 12.1.1. The continuous function shown in (a) is nonzero only for a finite interval of time T. It follows that its Fourier transform, whose modulus is shown schematically in (b), is not bandwidth limited but has finite amplitude for all frequencies. If the original function is sampled with a sampling interval Δ, as in (a), then the Fourier transform (c) is defined only between plus and minus the Nyquist critical frequency. Power outside that range is folded over or “aliased” into the range. The effect can be eliminated only by low-pass filtering the original function before sampling.

From Continuous to Discrete

• Sample time at interval Δ for N points (N even), tk=kΔ, k = 0, 1, 2, …, N-1.

• Frequency takes values at fn=n/(NΔ), n=-N/2,-N/2+1, …, 0, 1, 2,…,N/2-1.

• Then 12 2

0

21

0

( ) ( ) n n k

Ni f t i f t

n kk

nkN iN

kk

H f h t e dt h e

h e

Discrete Fourier Transform

• Definition

• Some propertiesF is symmetric, FT=F

(FT)* F = N I

F-1=F*/N (inverse transform is obtained by replacing i by –i, and dividing by N)

2 21

0

1 2 4

, ,

1 1 1 1

1 1 1 11 , ,

1 1 1 1 1 1

1 1

nk nkN i iN N

n k N N nkk

H h e or H F h F e

i iF F F

i i

Basic Idea of FFT

• Where HN/2,e is the discrete Fourier transform of N/2 points formed from even set of points, and HN/2,o similar but from odd set of points. This calculation is performed recursively.

12 /

0

2 / 2 /

/ 2 1 / 2 12 /( / 2) 2 / 2 /( / 2)

2 2 10 0

/ 2, / 2,

( )

NN i jk Nk j

j

i jk N i jk Nj j

j even j odd

N Ni jk N i N k i jk N

j jj j

N e k N ok N k

H h e

h e h e

h e e h e

H H

Example for N=8

2 3 4 5 6 70 1 2 3 4 5 6 7

2 4 6 2 4 60 2 4 6 1 3 5 7

4 2 4 4 2 40 4 2 6 1 5 3 7

2 /8

( ) ( ) ( ) ( )

k

i k

H h h h h h h h h

h h h h h h h h

h h h h h h h h

e

(A) The order of input data need to be rearranged (according to binary bit-reversed pattern.

(B) Values for all k can be evaluated in place. No additional memory is needed.

Bit Reversal

Example of FFT

x0

x1

x2

x3

x4

x5

x6

x7

x0

x4

x2

x6

x1

x5

x3

x7

Swap data according to bit reversal

Spacing =1

x0–x4

x2+x6

x2–x6

x1+x5

x1–x5

x3+x7

x3–x7

x0+x4

4= eik 2= eik/2

Spacing =2

x0+x4+x2+x6

x0-x4+i(x2-x6)

x0+x4-(x2+x6)

x0-x4-i(x2-x6)

x1+x5+x3+x7

x1-x5+i(x3-x7)

x1+x5-(x3+x7)

x1-x5-i(x3-x7)

= eik/4

Spacing =4

x0+x4+x2+x6+x1+x5+x3+x7

x0-x4+i(x2-x6)+ei/4 (x1-x5+i(x3-x7))

x0+x4-x2-x6+i(x1+x5-x3-x7)

x0-x4-i(x2-x6)+ ei3/4(x1-x5-i(x3-x7))

x0+x4+x2+x6-(x1+x5+x3+x7)

x0-x4+i(x2-x6)-ei/4 (x1-x5+i(x3-x7))

x0+x4-x2-x6-i(x1+x5-x3-x7)

x0-x4-i(x2-x6)- ei3/4(x1-x5-i(x3-x7))

FT of x in place

F2

F4

F8

Cooley-Tukey bit reversal FFT program

FFT runs in O(N log N)

Input Output

Wavelets

• Fourier transform is local in frequency domain and nonlocal in time

• Wavelet transforms are generalization that is local in both

• Discrete wavelet transform is some kind of matrix transform y = Fx, where FTF=I

• Wavelets are used in data compression and efficient representation of functions

Daubechies Wavelet Filter

The coefficients ci are determined by requirements of orthogonality (FTF=I), and certain “vanishing moments”.

F =

Discrete Wavelet Transform

F

F

F

Apply F to the upper half of the vector only

Suggested Reading and Software

• For a in-depth discussion of FFT algorithms, see C van Loan, “Computational Frameworks for the Fast Fourier Transform”

• For state-of-the-art free software, use FFTW at http://www.fftw.org/

Problem set 8• In the mode-coupling theory of heat transport through materials, one

need to solve a set of coupled nonlinear integral-differential equations numerically as follows:

• (a) Transform the first equation into frequency domain and solve (algebraically) g in terms of . (b) Describe a procedure to solve the system iteratively using FFT.

2

0

mod

( ) ( ) ( ) ( ) 0, 0,1, 2,..., 1

( ) ( ) ( )

(0) 1, (0) 0

( ) 0 and ( ) 0 for 0

t

k k k k k

k i ji j k N

k k

k k

g t t s g s ds g t k N

t g t g t

g g

g t t t

Where k2 are given, and

gk(t) and k(t) are unknown real functions. Dot means time derivative.