Math Primer. Outline Calculus: derivatives, chain rule, gradient descent, taylor expansions Bayes Rule Fourier Transform Dynamical linear systems

  • View
    220

  • Download
    1

Embed Size (px)

Text of Math Primer. Outline Calculus: derivatives, chain rule, gradient descent, taylor expansions Bayes...

  • Slide 1

Math Primer Slide 2 Outline Calculus: derivatives, chain rule, gradient descent, taylor expansions Bayes Rule Fourier Transform Dynamical linear systems Slide 3 Calculus Derivatives Derivative=slope Slide 4 Calculus Derivative: a few common functions (x n )=nx n-1 (x -1 )=-1/x 2 = x -2 exp(x)=exp(x) log(x)=1/x cos(x)=-sin(x) Slide 5 Calculus Derivative: Chain rule Ex: gaussian z=h(x)=exp(-x 2 ),f(y)=exp(y), y=g(x)=-x 2 f(y)=exp(y)= exp(-x 2 ), g(x)=-2x z= f(y)g(x) =exp(-x 2 )(-2x) Slide 6 Calculus Finding minima: gradient descent f(x) f(x 0 ) < 0 x0x0 x* x0+xx0+x x > 0 f(x*)=0 x = - f(x 0 ) Slide 7 Calculus Example: minimizing an error function Slide 8 Calculus Taylor expansion Slide 9 Calculus Slide 10 Bayes rule Example: drawing from 2 boxes 2 boxes (B1,B2) P(B1)=0.2,P(B2)=0.8 (Prior) Balls with two colors (R,G) B1=(16R,8G), B2=(8R,16G) P(R|B1)=2/3, P(G|B1)=1/3 (Conditional) P(R|B2)=1/3, P(G|B2)=2/3 (Conditional) Slide 11 Bayes rule Joint distributions P(G,B1)=P(B1)P(G|B1)=0.2*0.33=0.066 P(G,B2)=P(B2)P(G|B2)=0.8*0.66=0.528 P(X,Y)=P(X|Y)P(Y) P(Y,X)=P(Y|X)P(X) P(Y|X)P(X)=P(X|Y)P(Y) Slide 12 Bayes rule P(Y|X)P(X)=P(X|Y)P(Y) P(Y|X)=P(X|Y)P(Y)/P(X) If you draw G, what is the probability that it came from box1? P(B1|G)=P(G|B1)P(B1)/P(G) How do you get this? Marginalize Slide 13 Bayes rule Marginalization P(G)=P(G,B1)+P(G,B2)=0.066+0.528=0.6 P(G)=P(G|B1)P(B1)+P(G|B2)P(B2) P(Y)= x P(Y,X) P(Y)= x P(Y|X)P(X) P(Y|X)=P(X|Y)P(Y)/ Y P(X|Y)P(Y) Slide 14 Bayes rule If you draw G, what is the probability that it came from Box1 or Box2? P(B1|G)=P(G|B1)P(B1)/P(G) =(0.33*0.2)/0.6=0.11 P(B2|G)=P(G|B2)P(B2)/P(G) =(0.66*0.8)/0.6=0.89 Sum to one Slide 15 Bayes rule P(A,B|C)=P(A|B,C)P(B|C) P(B|A,C)=P(A|B,C)P(B|C)/P(A|C) Slide 16 Fourier transform Basis in linear algebra Basis function: dirac Basis function: sin Slide 17 Fourier Transform Decomposition in sum of sin and cosine Power: first term is the DC Phase Fourier transform for Dirac Sin Gaussian (inverse relationship) Slide 18 Fourier Transform Convolution and products Slide 19 Fourier transform Fourier transform of a Gabor Slide 20 Fourier transform Eigenspace for liner dynamical system Slide 21 Dynamical systems Stable if Nonlinear Networks Unstable Fixed point: | 1 |>1, | 2 |>1 Slide 37 Nonlinear Networks Line Attractor: 1 =1, | 2 |