Upload
others
View
2
Download
0
Embed Size (px)
Citation preview
Remarks on History of FFT and Related Algorithms
Radomir S. Stankovic, Miomir S. Stankovic
Dept. of Computer Science, Faculty of Electronics, 18 000 Nis, Serbia
Why to Discuss History of FFT Treatise by Gauss in 1805
Gauss, C.F., ”Nachlass – Teoria interpolationis methodo nova tractata”, 265-303, in Carl Friedrich Gauss, Werke, Band 3, Gottingen, Konglichen Gesellschaft der Wissenschafteln, 1866.
Asteroid Juno
Discovered 1 September 1804 by Karl L. Harding
Referenced but Unnoticed
Burkhardt, H., ”Trigonometische interpolation”, in ch 9, Encyklopaedie der Mathematischen Wissenschaften, Vol. 2, Part 1, 1st half, 643-693, Leipzig, B.G. Teubner, 1899-1916
Burkhardt, H., 1904
Goldstine, H.H., 1977
Goldstine, H.H., A History of Numerical Analysis from the 16th Through the 19th Century, Springer, 1977, 249-253
In 2008, question about developments after Colley-Tukey FFT
Danielson, G.C. 1942 Lanczos, C. 1942
Prehistory of FFT
Gauss, C.F. 1805 Carlini, F. 1828 Smith, A. 1846 Everett, J.D. 1860 Runge, C. 1903 Stumpff, K. 1939
Thomas, L.H. 1948 Good, I.J. 1958
Any composite integer 12
4,8,16,32
12
2nk
2nk,3nk
2n
Any integer with relatively prime factors
Any integer with relatively prime factors
Cooley-Tukey FFT Cooley, J.W., Tukey, J.W., "An algorithm for the ma- chine calculation of complex Fourier series", Math. of Comput., Vol. 19, April 1965, 297- 301.
Reference to Runge and Konig, 1903, 1905, 1924
Cooley Tukey
Garwin
Motivations for FFT President Kennedy's Scientific Advisory Committee, 1963
Bill Dorn Manager of the Computing Center at the IBM T.J. Watson Research Center
John W. Tukey Professor of Mathematics, Princeton University Dick Garwin Columbia University IBM Watson Research Center, New York City
James W. Cooley
Three-dimensional Fourier transform of spin orientations of He3 Spectral analyses of data from seismic sensing devices
Reaction to FFT Runge, C., "Uber die Zerlegung empirisch periodischer Funktionen in Sinnus-wellen", Zeit. fÄur Math, und Physik, Vol. 48, 443-456, 1903. Runge, C., Zeit fur Math, und Physik, Vol. 53, 1-17, 1905. Runge, C., Konig, H., "Die Grundlehren der Mathematischen Wissenschaften", Vorlesungen Äuber Numerisches Rechnen, Vol. 11 Berlin, Julius Springer,1924.
Runge Konig
Runge-Konig Algorithm Cooley-Tukey FFT – a generalization to an arbitrary non-prime N
Due to symmetry of sinus and cosinus 2N transform Two N transforms with a bit > of operations
Repeated to sequences of the length 2m
m = log2 N fields with 2m+1 of the length 2m-1
N/2 fields with sequences of 2 elements
# of operations O(Nlog2N)
Algorithm by Rudnick
Rudnick, P., "Note on the calculation of Fourier series", Math, of Comput., Vol. 20, July 1966, 429-430.
Danielson, G.C., Lanczos, C., "Some improvements in practical Fourier analysis and their application to X- ray scattering from liquids", J. Franklin Inst., Vol. 233, April 1942, 365-380, and 435-452.
Refering to N = 2n # of operations N log2 N
Runge and Konig Scripps Institution of Oceanography, La Jolla, California
Stumpff, K., Tafeln und Aufgaben zur Harmonischer Analyse und Periodogrammrechnung, Berlin: Julius Springer, 1939.
Thomas, a generalization of work by Stumpff
Thomas, L.H., "Using a computer to solve problems in physics", in Application of Digital Computers, Boston, Mass., Ginn, 1963.
Algorithm by Thomas
1D transform replaced by multi-dimensional transforms
Thomas
Llewellyn Thomas, Columbia University
Algorithm by Good
A generalization of the algorithm by Thomas
nrrN ...1= ri –mutually prime
i
n
iRR
1=⊗= nCCR ...1=
ij
n
ji CC1=
⊗=
≠=
=.,,,
jiji
i
iij I
RC
Transform size
Kronecker
Good-Thomas and FFT
1. In the Good algorithm factors of N must be mutually prime numbers. 2. In this case, steps are purely multidimensional transform without twiddle factors. 3. The correspondence between single and multiple dimensional indexing are completely different.
Good
Mapping of Indices N = r1r2
Cooley-Tukey Indices of the array in lexicographic order
n = r2n1 + n0
1D 2D
k = r1k1 + k0
Good r1 and r2 mutually prime
k - Ruritanian correspondence
n - Sino representation n = r1n0 + r2n1 mod N n0 = k mod r1 (0≤ n0 < r1) n1 = k mod r2 (0≤ n1 < r2)
DFT = wNkn
k = r2pk0 + r1qk1 mod N (0 ≤ k < N) p, q – solutions for r2p = 1 mod r1 (p < r1) r1q = 1 mod r2 (q < r2)
10
1
01
2
0
1
1
2
2110 nkr
nkr
knr
knr
rknN
rknN
knN wwwwwww ===
1-D into 2-D by keeping lexicographic ordering among indices of the two-dimensional array
Cooley-Tukey and Good-Thomas N = 24, r1 = 8, r2 = 3
n = r2n1 + n0 = 3n1+ n0
k = r1k1 + k0 = 8k1+ k0
0 1 2
3 4 5
6 7 8
9 10 11
12 13 14
15 16 17
18 19 20
21 22 23
0 1 2
0 1 2 3 4 5 6 7
0 8 16
1 9 17
2 10 18
3 11 19
4 12 20
5 13 21
6 14 22
7 15 23
0 1 2
0 1 2 3 4 5 6 7
n1
n0
k0
k1
0 8 16
3 11 19
6 14 22
9 17 1
12 20 4
15 23 7
18 2 10
21 5 13
0 1 2
0 1 2 3 4 5 6 7
0 16 8
9 1 17
18 10 2
3 19 11
12 4 20
21 13 5
6 22 14
15 7 23
0 1 2
0 1 2 3 4 5 6 7
n1
n0
k0
k1
n = r1n0 + r2n1 = 8n0 + 3n1 mod 24
k = r2pk0 + r1qk1 = 9k0 + 16k1 mod 24 Ruritanian correspondence
Sino representation
Matrix Representation of Good - FFT )()2()1( ... rMMMM ⊗⊗⊗=
)( )(,
)( ννjim=M∏
=
=r
nknk mm1
)(,,
ν
ννν
1,...,1,0 −= νpi1,...,1,0 −= νqj
)()2()1()()2()1( ...... rr AAAMMMM =⊗⊗⊗=
)( )(,
)( ννnka=A
),(...),(),( 12312)(,
)(, 1 −= rrnknk nknknkma
rδδδνν
1121121 0,...,0,0,...,0,0 −+−+−+ <≤<≤<≤<≤<≤ ννννν qkqkpkpkpk rrrr
rrrrr qnqnpnpnpn <≤<≤<≤<≤<≤ +−−++ 0,...,0,0,...,0,0 112211 νννν
Good FFT N = 12 = 3×4 From Chinese remainder theorem
)0,2(8)0,1(4)0,0(0
→→→
)1,0(9)1,2(5
)1,1(1
→→→
)2,1(10)2,0(6)2,2(2
→→→
)3,2(11)3,1(7)3,0(3
→→→
DFT(12) = A(1)A(2)
=
wwww
wwww
wwww
wwww
2
2
2
2
2
2
2
2
1000000000000100000000000010000000000001
1000000000000100000000000010000000000001111000000000000111000000000000111000000000000111
)1(A
−−−−
−−−−
−−−−
−−−−
−−
=
iiii
ii
iiii
ii
110000000000001100000000000011111100000000000011110000000000001111
110000000000001100000000000011111100000000000011110000000000001111
)2(A
Good FFT Alterantive
If M(ν) square matrices with pν = qν, then )()2()1()()2()1( ...... rr CCCMMMM =⊗⊗⊗=
rrr IIMC ⊗⊗⊗= ...2
)1()1(
rrr IMIC ⊗⊗⊗= ...)2()2(1
)()( ...21
rrr
r MIIC ⊗⊗⊗=
.
.
.
⊗
⊗ ⊗ ( ( ) ) .
=
=
)1(M )2(M
)1(M )1(I )1(I )2(M
Good – Thomas FFT N = 12 = 3×4
=
wwww
wwww
wwww
wwww
0000001000000000010000000000100000000001
0000001000000000010000000000100000000001100010001000010001000100001000100010000100010001
2
2
2
2
2
2
2
2
)1(C
−−−−
−−
−−−−
−−
−−−−
−−
=
ii
ii
ii
ii
ii
ii
1100000000111100000000
11000000001111000000000000110000000011110000000011000000001111000000000000110000000011110000000011000000001111
)2(C
Matrix Interpretation of FFT F – DFT matrix F = PQR
R =
...
Q =
permutation
P =
1 1
1
1
block diagonal block matrix
This image cannot currently be displayed.
),(),( 121 cfrfcrcfrc ++→+
Theilheimer,1969 N = r1r2 Sande-Tukey
F = RTQPT
Cooly-Tukey Theilheimer
Theilheimer and Glassman
−
−
−
=
=
42
34
531
42
12345
2424
333
4242
54321
01000001
0000001101010010101
110000110000001100001100000011000011
100000001000000010010000000100000001
111
11111
1111111
wwww
wwwww
wwwwwwwwwwwwwwwwwwwww
R
=
=
3
3
3
45
24
3
42
21
12345
2424
333
4242
54321
001000001000001100100010010001001
10000001110000001
1000000111
111
11111
1111111
ww
w
wwww
www
ww
wwwwwwwwwwwwwwwwwwwww
R
Rader-Brenner Algorithm DFT by FFT requires multiplication of complex numbers z = x + iy Multiplication simplified if Re(z) = x = 0 or Im(z) = y = 0
Rader-Brenner algorithm = FFT with Mul(z) Mul(x) or Mul(y)
Rader
C. M. Rader and N. M. Brenner, "A new principle for fast Fourier transform," IEEE Trans. Acoust., Speech, Signal Processing, Vol. 24, 1976, 264-265
Brenner
Rader-Brenner Algorithm (Theory) { })(nx mN 2=
{ })(nb{ })(nc
)2()( nxnb =
Qnxnxnc +−−+= )12()12()(12/,...,1,0 −= Nn
∑−
=
+=1
0)12(2 N
nnx
NQ
)0()0()0( CBX +=
)()2(eccos21)()( kC
NkjkBkX π
−= 2/,0 Nk ≠
)0()0()2/( CBNX −=
R = DFT(r)
When cosec large, small error
huge error in the Out
1976
Rader and his Paper Rader, C.M., ”Discrete Fourier transforms when the number of data samples is prime”, Proc. IEEE, Vol.5, No.6, June 1968, 1107-1108.
If N = prime, then
2. DFT = cyclic convolution of the lenght N-1
1. Reordering of indices Multiplication of indices mod N
Addition of indices mod (N-1)
Rader
Winograd Algorithm
Winograd = Method by Rader with cyclic convolution implemented using convolution theorem performed with minimum number of multiplications
Min # of multiplication to calculate product of two polynomials
This image cannot currently be displayed.
)(mod)()()( xPxHxXxY = over a field F is 2N - k, where N = deg P(x)
Multiplication with constants in F is not counted
k - # of irreducible factors of P(x)
Winograd
Matix Form of Winograd Algorithm Coefficients of X(x), H(x) and Y(x) represented as vectors
[ ]TNxx )1(,...),0( −=x[ ]TNhh )1(,...),0( −=h[ ]TNyy )1(,...),0( −=y
Algorithm with min # of multiplications
)( AxBhCy ⋅=
x(0) x(1)
x(N-1)
h(N-1)
1
2
2N-k
h(0) h(1)
A
B
C
y(N-1)
y(0) y(1)
A, B = DFT C=DFT-1
Hadamard Calculation of cyclic convolution
Prime Factors Algorithms
1. 1D-DFT nD-DFT by Good algorithm
2. Transforms of small dimensions by Winograd algorithm
Good-Winograd algorithms
Winograd-Fourier algorithms (WFTA)
Different convolution of algorithms of small dimensions by Chinese remainder theorem and Kronecker product representable matrices
Algorithms for Walsh and Haar
⊗
⊗ ⊗ ( ( ) ) .
=
=
Good-Thomas Algorithms
=
wwww
wwww
wwww
wwww
2
2
2
2
2
2
2
2
1000000000000100000000000010000000000001
1000000000000100000000000010000000000001111000000000000111000000000000111000000000000111
)1(A
Different Matrix Representations
Decision Diagram Methods
FFT over DD
Exploits Properties
Transform R FFT
Decision diagrams
Decision trees
Function f
Decision diagrams
Functions and Decision Trees
x3
x1
x2
S
S S
S S S S
f
0
0 0
0 0 0 0
1
1 1
1 1 1 1
0 0 00 0 10 1 00 1 11 0 01 0 11 1 01 1 1
x x x1 2 3 ff(000) Sf(000)f(001) Sf(001)f(010) Sf(010)f(011) Sf(011)f(100) Sf(100)f(101)
Sf(101)f(110)Sf(110)f(111)Sf(111)
Decompositionrules
G C= 2 × × C C2 2
BDD n = 3
S S
S S S S
1 0 0 1 0 1 1 1
x2 x2
x3 x3 x3 x3x3 x3 x3 x3
x2 x2
_ _
_ _ _ _
f
x1 x1
_ S S
S S
S S
1 0
x1
x2x2
x3 x3
x3 x3
x2 x2
x1
_
_ _
_ _
)7()6()5()4()3()2()1()0(
321321321321
321321321321
fxxxfxxxfxxxfxxxfxxxfxxxfxxxfxxxf
⊕⊕⊕⊕⊕⊕⊕=
321321
21321
xxxxxxxxxxxf
⊕⊕⊕=
Reduction Rules
Bit-level
1. Delete nodes whose both outgoing edges point to the same node. The input edge of the deleted node connect to the corresponding successing node.
2. Share isomorphic subtrees.
BDD
S
S Sxixi
_
g gg
x ⊕ x = 1 _
x ∨ x = x
DD-Methods
⊗
⊗ ⊗ ( ( ) ) .
=
=
Calculations with sub-functions
At each node perform the basic transform matrix
Spectral Transforms over DD
Reed-Muller Workshop September 1993, Hamburg, Germany
Reed-Muller
Transform
Walsh
This image cannot currently be displayed.
−−−−
=)1()1()1()1(
)(nnnn
nRR0R
R
−−−−−
=)1()1()1()1(
)(nnnn
nWWWW
W
Walsh and Haar over DD
Haar Walhs
BDD into WDD
1+11 -1
20=
1 -1 1 -1
x3
x2 x2
x1x1
x2 x2
x3
x3x3_
_ _
_
_
S
S S
S S
f
-1 -1-1+1
-2 0=
1 -1-1-(-1)
0 2= -1+1
-1 -1 0 -2=
2020
-2 0-2 0
0040
+
-=
0202
0 -2 0 -2
0004
+
-=
00400040
00040004
0 0 4 4 0 0 4-4
+
-
= MTBDD
WDD
MTBDD WDD
Size
5 4
DD
WW
W
W
0 -44
1
1-2x2
1-2x3
1 (2-2 )x3⋅1 (2-2 )x3⋅
f
1
q x x = (1-2 )(2-2 )2 3
q
1 1-2x1
f
cordic cps duke2 misex2 vg2
23 24 22 25 25
2 109
29 18
8
1206 654
87 29 10
In Out Cubes
Functions that cannot be processed by FFT over vectors Calculation over decision diagrams can be applied
DD-Methods over FFT
# of ri
FFT and DD-Methods
FFT – constant complexity for any function of a given length
DD – complexity O(# of nodes in DD( f ))
Compact representation that permits calculation of spectral transforms of large fucntions.
Acknowledgments
Mathematical Institute of the Serbian Academy of Sciences and Arts