View
1
Download
0
Category
Preview:
Citation preview
CS 401
Multiplication / FFTXiaorui Sun
1
StuffSolution of homework 2 will be posted on course website by the end of today
Additional office hour: Monday 2pm–4pm at SEO 1241
Midterm review: Oct 16
Midterm exam: October 18 in class1pm-1:50pm, SES 138You may use a sheet with notes on both sides, but not textbook and any other paper materialsYou may use a calculator, but not any device with transmitting functions, especially ones that can access the wireless or the Internet.Be concise with your solution, always give your high level idea.
Fast Fourier Transform
Polynomials: Coefficient RepresentationPolynomial. [coefficient representation]
Add: O(n) arithmetic operations.
Evaluate: O(n) using Horner's method.
Multiply (convolve): O(n2) using brute force.
!!
A(x) = a0 + a1x + a2x2 +"+ an-1x
n-1
!!
B(x) = b0 +b1x+b2x2 +"+ bn-1x
n-1
!!
A(x)+ B(x) = (a0 +b0 )+ (a1+b1)x +"+ (an-1 +bn-1)xn-1
!!
A(x)= a0 + (x (a1+ x (a2 +"+ x (an-2 + x (an-1))"))
A(x)´ B(x) = ci xi
i =0
2n-2å , where ci = a j bi- j
j =0
i
å4
Polynomials: Point-Value RepresentationPolynomial. [point-value representation]
Add: O(n) arithmetic operations.
Multiply: O(n), but need 2n-1 points.
Evaluate: O(n2) using Lagrange's formula.
!!
A(x) : (x0, y0 ),", (xn-1, yn-1) B(x) : (x0, z0 ),", (xn-1, zn-1)
!!
A(x)+ B(x) : (x0, y0 + z0 ),", (xn-1, yn-1 + zn-1)
A(x) = yk
(x - x j )j¹kÕ
(xk - x j )j¹kÕk=0
n-1å
0 0 0 2n-2 2 2 2 2( ) ( ) : (x , ) , , (x , )n nA x B x y z y z- -´ ´ ´!
5
Converting Between Two
RepresentationsTradeoff. Fast evaluation or fast multiplication. We want
both!
Goal. Make all ops fast by efficiently converting between
two representations.
Coefficient
Representation
O(n2)
Multiply
O(n)
Evaluate
Point-value O(n) O(n2)
!!
a0, a1,", an-1 !!
(x0, y0 ),", (xn-1, yn-1)coefficient
representation
point-valuerepresentation
FFT: O(n log n)
Inverse FFT: O(n log n)
6
Coefficient to Point-Value Representation: Intuition
Coefficient to point-value. Given a polynomial a0 + a1 x + ... + an-1 xn-1, evaluate it at n distinct points x0, ... , xn-1.Divide. Break polynomial up into even and odd powers.
A(x) = a0 + a1x + a2x2 + a3x3 + a4x4 + a5x5 + a6x6 + a7x7.Aeven(x) = a0 + a2x + a4x2 + a6x3.Aodd (x) = a1 + a3x + a5x2 + a7x3.A(-x) = Aeven(x2) + x Aodd(x2).A(-x) = Aeven(x2) - x Aodd(x2).
Intuition. Choose two points to be ±1.A(-1) = Aeven(1) + 1 Aodd(1). A(-1) = Aeven(1) - 1 Aodd(1).
Can evaluate polynomial of degree £ nat 2 points by evaluating two polynomials of
degree £ ½n at 1 point.
7
Coefficient to Point-Value Representation: Intuition
Coefficient to point-value. Given a polynomial a0 + a1 x + ... + an-1 xn-1, evaluate it at n distinct points x0, ... , xn-1.Divide. Break polynomial up into even and odd powers.
A(x) = a0 + a1x + a2x2 + a3x3 + a4x4 + a5x5 + a6x6 + a7x7.Aeven(x) = a0 + a2x + a4x2 + a6x3.Aodd (x) = a1 + a3x + a5x2 + a7x3.A(-x) = Aeven(x2) + x Aodd(x2).A(-x) = Aeven(x2) - x Aodd(x2).
Intuition. Choose four points to be ±1, ±i.A(-1) = Aeven(-1) + 1 Aodd( 1). A(-1) = Aeven(-1) - 1 Aodd(-1).A(-i) = Aeven(-1) + i Aodd(-1). A(-i) = Aeven(-1) - i Aodd(-1).
Can evaluate polynomial of degree £ nat 4 points by evaluating two polynomials of
degree £ ½n at 2 points.8
Fast Fourier TransformCoefficient to point-value. Given a polynomial a0 + a1 x + ... + an-1 xn-1, evaluate it at n distinct points x0, ... , xn-1.
Key idea: choose xk = wk where w is principal nth root of unity.
!!
y0
y1y2
y3
"yn-1
é
ë
ê ê ê ê ê ê ê
ù
û
ú ú ú ú ú ú ú
=
1 1 1 1 # 11 w1 w2 w3 # wn-1
1 w2 w4 w6 # w2(n-1)
1 w3 w6 w9 # w3(n-1)
" " " " $ "1 wn-1 w2(n-1) w3(n-1) # w(n-1)(n-1)
é
ë
ê ê ê ê ê ê ê
ù
û
ú ú ú ú ú ú ú
a0
a1
a2
a3
"an-1
é
ë
ê ê ê ê ê ê ê
ù
û
ú ú ú ú ú ú ú
Fourier matrix Fn 9
Roots of UnityDef. An nth root of unity is a complex number x such that xn
= 1.Fact. The nth roots of unity are: w0, w1, …, wn-1 where w = e 2p i / n.Pf. (wk)n = (e 2p i k / n) n = (e p i ) 2k = (-1) 2k = 1.Fact. The ½nth roots of unity are: v0, v1, …, vn/2-1 where v = e 4p i / n.Fact. w2 = v and (w2)k = vk.
w0 = v0 = 1
w1
w2 = v1 = i
w3
w4 = v2 = -1
w5
w6 = v3 = -i
w7
n = 8
10
Fast Fourier TransformGoal. Evaluate a degree n-1 polynomial A(x) = a0 + .... + an-1 xn-1 at its nth roots of unity: w0, w1, …, wn-1.
Divide. Break polynomial up into even and odd powers.Aeven(x) = a0 + a2x + a4x2 + … + an/2-2 x(n-1)/2.
Aodd (x) = a1 + a3x + a5x2 + … + an/2-1 x(n-1)/2.
A(x) = Aeven(x2) + x Aodd(x2).
Conquer. Evaluate degree Aeven(x) and Aodd(x) at the ½nth
roots of unity: v0, v1, …, vn/2-1.
Combine. A(wk+n) = Aeven(vk) + wk Aodd(vk), 0 £ k < n/2
A(wk+n/2) = Aeven(vk) - wk Aodd(vk), 0 £ k < n/2
wk+n/2 = -wk
vk = (wk)2 = (wk+n/2)2
11
fft(n, a0,a1,…,an-1) {if (n == 1) return a0
(e0,e1,…,en/2-1) ¬ FFT(n/2, a0,a2,a4,…,an-2)(d0,d1,…,dn/2-1) ¬ FFT(n/2, a1,a3,a5,…,an-1)
for k = 0 to n/2 - 1 {wk ¬ e2pik/n
yk+n/2 ¬ ek + wk dkyk+n/2 ¬ ek - wk dk
}
return (y0,y1,…,yn-1)}
FFT Algorithm
12
FFT SummaryTheorem. FFT algorithm evaluates a degree n-1 polynomial at each of the nth roots of unity in O(n log n) steps.
Running time. T(2n) = 2T(n) + O(n) Þ T(n) = O(n log n).
assumes n is a power of 2
!!
a0, a1,", an-1 !!
(w0, y0 ),", (wn-1, yn-1)
O(n log n)
coefficientrepresentation
point-valuerepresentation
13
Recursion Treea0, a1, a2, a3, a4, a5, a6, a7
a1, a3, a5, a7a0, a2, a4, a6
a3, a7a1, a5a0, a4 a2, a6
a0 a4 a2 a6 a1 a5 a3 a7
"bit-reversed" order
000 100 010 110 001 101 011 111
perfect shuffle
14
Point-Value to Coefficient Representation: Inverse DFT
Goal. Given the values y0, ... , yn-1 of a degree n-1 polynomial at the n points w0, w1, …, wn-1, find unique polynomial a0 + a1 x + ... + an-1 xn-1 that has given values at given points.
Inverse DFT
!!
a0
a1
a2
a3
"an-1
é
ë
ê ê ê ê ê ê ê
ù
û
ú ú ú ú ú ú ú
=
1 1 1 1 # 11 w1 w2 w3 # wn-1
1 w2 w4 w6 # w2(n-1)
1 w3 w6 w9 # w3(n-1)
" " " " $ "1 wn-1 w2(n-1) w3(n-1) # w(n-1)(n-1)
é
ë
ê ê ê ê ê ê ê
ù
û
ú ú ú ú ú ú ú
-1
y0
y1y2
y3
"yn-1
é
ë
ê ê ê ê ê ê ê
ù
û
ú ú ú ú ú ú ú
Fourier matrix inverse (Fn)-1 15
Claim. Inverse of Fourier matrix is given by following formula.
Consequence. To compute inverse FFT, apply same algorithm but use w-1 = e -2p i / n as principal nth root of unity (and divide by n).
!!
Gn = 1n
1 1 1 1 " 11 w-1 w-2 w-3 " w-(n-1)
1 w-2 w-4 w-6 " w-2(n-1)
1 w-3 w-6 w-9 " w-3(n-1)
# # # # $ #1 w-(n-1) w-2(n-1) w-3(n-1) " w-(n-1)(n-1)
é
ë
ê ê ê ê ê ê ê
ù
û
ú ú ú ú ú ú ú
Inverse FFT
16
Inverse FFT: Algorithm
ifft(n, a0,a1,…,an-1) {if (n == 1) return a0
(e0,e1,…,en/2-1) ¬ FFT(n/2, a0,a2,a4,…,an-2)(d0,d1,…,dn/2-1) ¬ FFT(n/2, a1,a3,a5,…,an-1)
for k = 0 to n/2 - 1 {wk ¬ e-2pik/n
yk+n/2 ¬ (ek + wk dk)yk+n/2 ¬ (ek - wk dk)
}
return (y0,y1,…,yn-1)}
17Note. Need to divide the final result by n
Inverse FFT SummaryTheorem. Inverse FFT algorithm interpolates a degree n-1 polynomial given values at each of the nth roots of unity in O(n log n) steps.
assumes n is a power of 2
!!
a0, a1,", an-1 !!
(w0, y0 ),", (wn-1, yn-1)
O(n log n)
coefficientrepresentation
O(n log n) point-valuerepresentation
18
Polynomial MultiplicationTheorem. Can multiply two degree n-1 polynomials in O(n log n) steps.
!!
a0, a1,", an-1b0, b1,", bn-1 !!
c0, c1,", c2n-2
!!
A(x0),", A(x2n-1)B(x0 ),", B(x2n-1) !!
C(x0),C(x1),",C(x2n-1)O(n)
point-value multiplication
O(n log n)FFT inverse FFT O(n log n)
coefficientrepresentation coefficient
representation
Problem 5Given two sorted arrays of n elements, find the n-thsmallest elementBy query, e.g. what is the 3-rd smallest element of first
array?
1 3 4 7 8 10 11 405 9 13 17 18 20 31 33
Problem 5 hintBinary search: find an element in a sorted array in O(log n) time
SolutionSearch(", $%&, ℎ"(ℎ): find n-th largest smallest element (in the merged array) if it is in array " with value ≥ +[", $%&]and ≤ +[/, ℎ"(ℎ]
Search(", $%&, ℎ"(ℎ)If ℎ"(ℎ < $%& return nullset 1"2 = ⌊($%& + ℎ"(ℎ) / 2⌋if +[",1"2] is n-th smallest element, return +[",1"2]If +[",1"2] is smaller than n-th smallest element
return Search(", $%&,1"2 − 1)else
return Search(",1"2 + 1, ℎ"(ℎ)
SolutionHow to determine if ![#,%#&] is n-th smallest element?
Query:• %#&-th element of ![#]• () − %#&)-th and () − %#& + 1)-th element of ![3 − #]
1. If ! #,%#& ≥ ![3 − #, ) − %#&] and ![#,%#&] <= ![3 −# ) −%#& + 1] then ![#,%#&] is the solution
2. If ![#,%#&] < ![3 − #, ) − %#&], then ![#,%#&] is greater than solution
3. If ![2,%#&] > ![3 − #, ) − %#& + 1], then ![#,%#&] is smaller than solution
Summary of Divide and Conquer Divide: We reduce a problem to several subproblems.
each sub-problem is at most a constant fraction of the size of the original problem
Conquer: Recursively solve each subproblem
Combine: Merge the solutions
Examples:• Mergesort, Binary Search, Closest Point Pair, Integer
Multiplication, Matrix Multiplication, FFTLo
g n
leve
ls
n
n/2n/2
n/4
How to solve problems by D&CDefine subproblems• Sometimes, subproblems are same to the original
problem (just input size is smaller), e.g. Mergesort• Sometimes, need to elaborate the subproblems to allow
solving subproblem recursively, e.g. Counting Inversions
Algorithm design: how to obtain solution of a subproblem from solutions of smaller subproblems?
Correctness proof: show the combine algorithm is correct• Given solutions of smaller subproblems, combine
algorithm give the solution of the right answer of original subproblem
25
Running TimeWrite down recurrence relation
! " = $ ! "% + Θ("))
Master theorem: Suppose ! " = $ ! +, + Θ(")) for all
" > %. Then,
• If $ < %) then ! " = Θ ")
• If $ = %) then ! " = Θ ")log "
• If $ > %) then ! " = Θ "34567
Recommended