Algebra Jeff Edmonds York University COSC 6111 Fields GCD Powers mod p Fermat, Roots of Unity, & Generators Z mod p vs Complex Numbers Cryptography Other

AlgebraJeff Edmonds

York University

COSC 6111

• Fields• GCD• Powers mod p• Fermat, Roots of Unity, & Generators• Z mod p vs Complex Numbers• Cryptography• Other Finite Fields

• Vector Spaces• Colour• Error Correcting Codes

• Linear Transformations• Integrating

• Changing Basis• Fourier Transformation (sine)• Fourier Transformation (JPEG)• Fourier Transformation (Polynomials)

• Other Algebra• Taylor Expansions• Generating Functions• Primes Numbers

Fields• A Field has:• A universe U of values• Two operations: + and ×• + Identity: $0 a+0 = a• × Identity: $1 a×1 = a• Associative: a+(b+c) = (a+b)+c & a×(b×c) = (a×b)×c • Commutative: a+b = b+a & a×b = b×a • Distributive: a×(b+c) = a×b + a×c• + Inverse: "a $b a+b=0, i.e. b=-a

• (These give you a group.)

(& a×0 = 0)

Differentiates between + and ×

Fields• A Field has:• A universe U of values• Two operations: + and ×• + Identity: $0 a+0 = a• × Identity: $1 a×1 = a• Associative: a+(b+c) = (a+b)+c & a×(b×c) = (a×b)×c • Commutative: a+b = b+a & a×b = b×a • Distributive: a×(b+c) = a×b + a×c• + Inverse: "a $b a+b=0, i.e. b=-a• × Inverse: "a≠0 $b a×b=1, i.e. b=a-1

• Examples:• Reals & Rationals• Complex Numbers• Integers• Invertible Matrices

(& a×0 = 0)

Fields

• Problems for computers:• Reals

• Too much space• Lack of precision

• Integers• Lack of inverses• Grow too big

• Better field?• Finite field, eg integers mod a prime

Finite

Finite Fields

123404

241303

314202

432101

000000

43210×

321044

210433

104322

043211

432100

43210+• Integers mod 5 (Z/5)• Universe U = {0,1,2,3,4}• Two operations + and ×

• 3+4 = 7 =mod 5 2• 3×4 = 12 =mod 5 2

• Don’t think of mod 5 as a function mod5(7) = 2.

• Think of it as equivalence classes … -8 =mod 5 -3 =mod 5 2 =mod 5 7 …

• Must prove + & × are well defined [a]modp × [b]modp = [a+ip]×[b+jp] = a×b + (aj+bi+ijp)p = [a×b]modp

Finite Fields

123404

241303

314202

432101

000000

43210×

321044

210433

104322

043211

432100

43210+• Special value 0

• a+0 = a• a×0 = 0

• Special value 1 • a×1 = a

Finite Fields

123404

241303

314202

432101

000000

43210×

321044

210433

104322

043211

432100

43210+

• Associative: • a+(b+c) = (a+b)+c • a×(b×c) = (a×b)×c

• Commutative: • a+b = b+a • a×b = b×a

• Distributive: • a×(b+c) = a×b + a×c

Finite Fields

123404

241303

314202

432101

000000

43210×

321044

210433

104322

043211

432100

43210+• Inverses:

• "a $b a+b=0, i.e. b=-a• 0 = 2+(-2) =mod 5 2 + 3

• "a≠0 $b a×b=1, i.e. b=a-1

• 1 = 2×(½) =mod 5 2×(?)• 2×3 = 6 =mod 5 1

12345606

24613505

36251404

41526303

53164202

65432101

00000000

6543210×

Finite Fields

• Integers mod 7

• Multiplicative Inverse:• "a≠0 $b a×b=1, i.e. b=a-1

• Given a, find a-1

• If b = a-1, then a = b-1

• 1 = 1-1

• It is possible that a = a-1

Finite Fields

• Integers mod 6• Multiplicative Inverse:

• "a≠0 $b a×b=1, i.e. b=a-1


• No inverse for 2• Zero Divisors:

• 2×3 = 6 =mod 6 0 • No inverses for ints mod n

if n is 1234505

2402404

3030303

4204202

5432101

0000000

543210×

not prime

Finite Fields

• Integers mod 6• Multiplicative Inverse:

• "a≠0 $b a×b=1, i.e. b=a-1


• No inverse for 2• Zero Divisors:

• 2×3 = 6 =mod 6 0 • No inverses for ints mod n

if n is • Inverses for ints mod p

if p is prime• Prove by construction

using GCD alg

1234505

2402404

3030303

4204202

5432101

0000000

543210×

not prime

GCD(a,b)Input: <a,b>

= 3Output: GCD(a,b)= <21,9>

Maintain values <x,y> such that GCD(a,b) = GCD(x,y)

GCD(a,b) = GCD(x,y) = GCD(y,x mod y) = GCD(x’,y’)

Replace <x,y> with <y,x mod y>

GCD(a,b)

Extended GCD(a,b)

Input: <a,b>= <5,2,-3>5 = GCD(25,15)(2)×25 + (-3)×15 = 5

Output: <g,u,v>• g = GCD(a,b)• u×a + v×b = g

= <25,15>

Extended GCD(a,b)My instance: <a,b>

My friend’s instance: <a’,b’> = <b,a mod b>a’ = b, b’ = a mod b = a - r×b

My friend’s solution: <g’,u’,v’>• g’ = GCD(a’,b’)• u’×a’ + v’×b’ = g’• u’×b + v’×(a-r×b) = g• v’×a + (u’-v’×r)×b = g• u×a + v×b = g

My solution: <g,u,v>• g =• u =• v =

g’v’u’-v’×r

= g

Extended GCD(a,b)

12345606

24613505

36251404

41526303

53164202

65432101

00000000

6543210×

Finding Inverses• Integers mod p (Z/p)• Multiplicative Inverse:

• Given a≠0 and prime p, find b such that a×b =mod p 1

• Use Extended GCD( , ) a pOutput: <g,u,v>

• g = GCD(a,p)• a×u + p×v = g

= 1= 1

b = u

a×b =

• Multiplicative Inverse:• Given a≠0 and prime p,

find b such that a×b =mod p 1

a×b =mod p 11 – p×v =mod p 1

"a≠0 $b a×b=1, i.e. b=a-1

• Integers mod p (Z/p)

Finding Inverses

Chinese Remainder TheoremSuppose you want to compute some integer x.But instead of doing the long computation over the integers,you compute it over the integers mod p1.Then you compute it over the integers mod p2.

Input: <a1,p1,a2,p2,…,ar,pr>Output: x• i x = ai mod pi

• Unique answer ≤ p1p2…pr

Sorry. We don’t cover the algorithm.

Powers mod p

1

Start with 1 and continually multiply by b mod p.What do you get?

b×b

b2

×bb3

×bb4

×bb5

×bb6

×bbN

×b…

Input: b and NOutput: y = bN mod p

Time(N) = (N)n = Size = log(b) + log(N)

= 2(n)

N=7

N=2

N=1N=1

N=2

N=1N=1

N=2

N=1N=1

N=1

N=4N=3b4b3

b7 = b3 × b4

T(N) = 2T(N/2) + 1 = (N)Size = log(b) + log(N)

= 2(n)

Powers mod p

N=7

N=1

N=3b3

b7 = (b3)2 × b

T(N) = 1T(N/2) + 1 = (log(N))Size = log(b) + log(N)

= (n)

Powers mod p

Powers mod pInput: b and NOutput: y = bN mod p

Time(N) = (log N) = (n)

Time(N) = (N) = 2(n)

Input: b and yOutput: N such that y = bN mod p

N = logb(y) mod p

n = Size = log(b) + log(N)

1 b×b

b2

×bb3

×bb4

×bb5

×bb6

×by

×b…

A one way hard functionUseful in cryptography.

Discrete Log

Similarly:• Multiplying: p×q = N

Time = (n)• Factoring: N = p×q

Time = 2(n)

Can this go on for ever?No. There are only p elements.

b×x = b×yb-1×b×x = b-1×b×y

x = yEach node has in-degree one

and out-degree one.

1 b×b

b2

×bb3

×bb4

×bb5

×bb6

×b

Is this possible?

x

×by

×b$b-1 b×b-1=1

Fermat, Roots of Unity, & Generators

What does a graph with in and out-degree one look like?

1

b b2

b3

b4b5


Lets first focus on only these elements.

1

b b2

b3

b4b5

$r br = 1

a

ab ab2

ab3

ab4ab5

There might be another element a. abr = a

c

cb cb2

cb3

cb4cb5

There might be another element c. cbr = c Do this some q number of times.

qAre there more elements?

0The total # of elements

= rq+1 = p


1

b b2

b3

b4b5

$r br = 1

Do this some q number of times.

qAre there more elements?

0The total # of elements

= rq+1 = p

1 1 1 1 1 1 1 1 1 1 11 2 4 8 5 10 9 7 3 6 11 3 9 5 4 1 3 9 5 4 11 4 5 9 3 1 4 5 9 3 11 5 3 4 9 1 5 3 4 9 11 6 3 7 9 10 5 8 4 2 11 7 5 2 3 10 4 6 9 8 11 8 9 6 4 10 3 2 5 7 11 9 4 3 5 1 9 4 3 5 11 10 1 10 1 10 1 10 1 10 1

b2 b3 b4 b5 b6 b7 b8 b9 b10b0 b1

Eg. p=11, n=p-1=rq=10

r=5,q=2

r=2,q=5

r=1,q=10r=10,q=1


1

b b2

b3

b4b5

$r br = 1

Values of b like 2,6,7, & 8 are said to• be a generator of the field

The total # of elements = rq+1 = p

1 1 1 1 1 1 1 1 1 1 11 2 4 8 5 10 9 7 3 6 11 3 9 5 4 1 3 9 5 4 11 4 5 9 3 1 4 5 9 3 11 5 3 4 9 1 5 3 4 9 11 6 3 7 9 10 5 8 4 2 11 7 5 2 3 10 4 6 9 8 11 8 9 6 4 10 3 2 5 7 11 9 4 3 5 1 9 4 3 5 11 10 1 10 1 10 1 10 1 10 1

b2 b3 b4 b5 b6 b7 b8 b9 b10b0 b1

Eg. p=11, n=p-1=rq=10

r=10,q=1


Fermat’s Little Theorem: b≠0 bp-1 =mod p 1

1

b b2

b3

b4b5

$r br = 1

The total # of elements = rq+1 = p

Proof: bp-1 = brq = (br)q =mod p (1)q = 1


Fermat’s Little Theorem: b≠0 bp-1 =mod p 1

Euler’s Version: b≠0 bφ =mod n 1where • n = pq with p and q are prime• φ = (p-1)(q-1)• and where b is co-prime with n.

Example: b=2, p=3, q=5, n=15, r=(p-1)(q-1)=8

=mod 15 11 2×b

4×b

8×b

16×b


Fermat, Roots of Unity, & GeneratorsFermat’s Little Theorem: b≠0 bp-1 =mod p 1

Example: b=2, p=3, q=5, n=15, r=(p-1)(q-1)=8

1 2 4 8

b4 = 1

b8 = (b4)2 = 1

Euler’s Theorem: b≠0 bφ =mod n 1where • n = pq with p and q are prime• φ = (p-1)(q-1)• and where b is co-prime with n.

161

160

162

16316

416

5

166

167

168

169

1610

1611

1612

1613

1614

1615

= 1616 = 1

Z mod 17 vs Complex Numbers16th roots of unity

-1 =

i

-i

(n/2)2 = 1

(n/4)2 = n/2 = -1

(3n/4)2 = n/2 = -1

These could be Z mod 17or complex numbers

×

rr

rθ

reθi = rcosθ + irsinθ

f(θ) = reθi g(θ) = rcosθ + irsinθ

Goal: Proof f(θ) = g(θ)

f(0) = re0i = r g(0) = rcos0 + irsin0 = r

f’(θ) = ireθi g’(θ) = -rsinθ + ircosθ

f(0) = g(0)

f’(0) = ire0i = ir g’(0) = -rsin0 + ircos0 = ir

f’(0) = g’(0)

f’’(θ) = -reθi g’’(θ) = -rcosθ - rsinθ

= -f(θ) = -g(θ)

Z mod 17 vs Complex Numbers

Goal: Proof f(θ) = g(θ)f(0) = g(0)f’(0) = g’(0)

f’’(θ) = -f(θ) g’’(θ) = -g(θ)Proof by induction (over the reals) that f(θ) = g(θ)

f(θ) g(θ)

• For this θ, f(θ) = g(θ) and f’(θ) = g’(θ)• For next θ+, f(θ+) = g(θ+)• f’’(θ) = -f(θ) = -g(θ) =g’’(θ)• For next θ+, f’(θ+) = g’(θ+)

Z mod 17 vs Complex Numbers

161

160

162

16316

416

5

166

167

168

169

1610

1611

1612

1613

1614

1615

= 1616 = 1

Z mod 17 vs Complex Numbers16th roots of unity

-1 =

i

-i

These could be Z mod 17or complex numbers

rr

rθ

reθi = rcosθ + irsinθreθi × seαi = (rs)e(θ+α)i

CryptographyI publish a public key E

and hide a private key D.

I have a message m to send to him. I use E to encode it.

code = Encode(m,E)

Knowing E but not D, I cannot decode the message.

Knowing D, I decode the message. m = Decode(code,D)

Identifying Oneself I am the guy who knows D

Prove it. I will encode a message for you.

code = Encode(m,E)

Knowing D, I can decode the message.

m = Decode(code,D)

Knowing E but not D, I cannot pretend to be him.

Cryptography• I chose two primes p and q.• n = pq.• φ = (p-1)(q-1) • Euler’s Theorem:

b≠0 bφ = 1 mod n• Let e be some value co-prime with φ • Let d = e-1 mod φ• Note φ is not prime,

but gcd(φ,e) = 1 is good enough.• Note ed = 1+ φr

• I publish E = <n,e> to the world.• I keep D = <d> and <p,q,φ> private.

Cryptography• In summary:• b≠0 bφ = 1 mod n• ed =1+ φr

• c = Encode(m,E) = me mod n

Time? = (# bits in e, m, & n) using repeated squaring

Cryptography• In summary:• b≠0 bφ = 1 mod n• ed =1+ φr

• c = Encode(m,E) = me mod n

• m’ = Decode(c,D) = cd mod n = (me)d = med

= m1+ φr

= m × (mφ)r

= m × (1)r mod n = m

We have seen the finite field Z/p • with elements being the integers {0,1,2,…,p-1}• with normal + and × mod a prime integer p.

Similarly, we consider the field of (Z/p)[x]/P • with elements the polynomials

ad-1 xd-1 + … + a3 x3 + a2 x2 + a1 x + a0

• with coefficients ai in Z/p.• and degree at most d-1.

• with normal + and × mod an unfactorable polynomial P. xd - 2xd-1 - … - 3x2 - x – 4 = 0• Note this field has pd elements.

Finite Fields mod Polynomial

For example (Z/2)[x]/(x3+x+1)


Binary coefficients.

Polynomials over x.

Mod x3+x+1

All x3 removed, so elements have degree 2. There are 23=8 elements.

For example (Z/2)[x]/(x3+x+1)• x2+1 and x2+x+1 are elements• (x2+1)×(x2+x+1)

= x4+x3+x2 + x2+x+1 = x4+x3+x+1 = (x3+x+1)(x+1) + (x2 +x)= x2 +x


xx4+x3 +x+1 x3+x+1x4 +x2+x

x3+x2 +1

+1

x3 +x+1x2 +x remainder

Types of Finite Fields• Lemma: Every finite field has pd elements

for some prime p and int d.• Any two finite fields with the same number of elements

are isomorphic ie same with under some renaming of the elements.

• Eg There is not a field with 6 elements!

Is there a field with 81 elements?Yes, because 81 = 34

Is there a field with 82 elements?No, because 82 = 2∙41

Can this go on for ever?No. There are a

finite # of elements.

x+1 = y+1x+1+(-1) = y+1+(-1)

x = yEach node has in-degree one

and in out-degree one.

0+1 +1 +1 +1 +1 +1

Is this possible?

x

+1y

+1$(-1) 1+(-1)=0

Partial Proof: Consider some finite field.Every field has a zero 0 and a +1

Types of Finite Fields Every finite field has pd elements And effectively is determined.

Skip

What does a graph with in and out-degree one look like?

0

1

Lets first focus on only these elements.


0

$n 1+1+1+ … + 1 = 0Give these elements names.We don’t know how × works.Proof that for these n elements

× works act like Z/n.• r’×s’ = (1+1+…+1) × (1+1+…+1)

= (1×1 + 1×1 + … + 1×1)

= ( 1 + 1 + … + 1)

= (r×s)’

r s • A Field is distributive: a×(b+c) = a×b + a×c

r×s

r×s

1 2’

3’

4’(n-1)’

’

’

• 1×1 = 1

• By definition

• By definition

Proves × works correctly.


0Similarly, for these n elements

+ works act like Z/n.Hence, we can rename the elements.

1 2’

3’

4’(n-1)’

’

’


Similarly, for these n elements + works act like Z/n.

Hence, we can rename the elements.0

1 2

3

4(n-1)


• Proof n is prime. • Suppose n=rs• No zero divisors allowed• Hence, n is prime.


0

1 2

3

4(n-1)

• But there may be other elements in the Field.• Consider one u.• What about a×u for a in Z/p ?


0

1 2

3

4(p-1)

u

4u

0

2u

3u

(p-1) u

• What about u×u? Call it v.• What about b×v for b in Z/p ?


0

1 2

3

4(p-1) 4u

0

2u

3u

(p-1) u 4v

0

2v

3v

(p-1) v

u v= u×u


0

1 2

3

4(p-1) 4u

0

2u

3u

(p-1) u 4v

0

2v

3v

(p-1) v

• What about au + bv for a and b in Z/p ?

u v= u×u

2v

00+

3v

4v

5v

6v

v

u 2u 3u 4u 5u 6u

au+bv


0

1 2

3

4(p-1) 4u

0

2u

3u

(p-1) u 4v

0

2v

3v

(p-1) v

• What about au + bv for a and b in Z/p ?

u v= u×u

Or think of u and v as vectors in a vector space with underlying finite field Z/p.

3u + 2v = =

=

u

v

• We now have considered • Z/p = 0,1,2,…,p-1• u• v= u×u• Linear combinations au+bv

• Now consider x3, x4, x5, ..., xd-1 • And Linear combinations

a0+ a1 x + a2 x2 + a3 x3 + … + ad-1 xd-1

• Until d is such that xd has been seen before. Perhaps xd = 2xd-1 + … + 3x2 + x +4


= x

• The elements of our field then consist of• the set of polynomials

a0+ a1 x + a2 x2 + a3 x3 + … + ad-1 xd-1

• with coefficients ai in Z/p.• Degree at most d-1.• Mod xd - 2xd-1 - … - 3x2 - x – 4 = 0• (This is a polynomial that is like a prime

in that it has no factors.)• Note this field has pd elements.


Vector Spaces• A vector space has:• A universe V of objects. Eg:

• An arrow with a direction and a length• A knapsack of toys• A function

1inchNorth East

2x2exsin x

Vector Spaces• A vector space has:• A universe V of objects.• An underlying field F.• Closed under linear combinations

• If u,v V, then au + bv V

u = v =

3u + 2v =

Vector Spaces• A vector space has:• A universe V of objects.• An underlying finite field F.• Closed under linear combinations

• If u,v V, then au + bv V3u + 2v =

u = v =



3u + 2v =

u =

v =

2 x2exsin x + xexcos x

2 xexcos x+ 3 exsin x

6 x2exsin x + 7 xexcos x + 6 exsin x


• If u,v V, then au + bv V• Cannot multiply two objects producing an object.• Zero Object

0 v = 00 v =0 v =


• If u,v V, then au + bv V• Cannot multiply two objects producing an object.• Zero Object• Usual Field rules

• Associative: u+(v+w) = (u+v)+w• Commutative: u+v = v+u • Distributive: a×(u+v) = a×u + a×v• + Inverse: "v $u u+v=0, i.e. v=-u

Vector Spaces• A basis of a vector space:• A tuple <w1,w2,…,wd> of objects• Linearly independent

• wd ≠ a1w1+a2w2 +… + ad-1wd-1

• 0 ≠ a1w1+a2w2 +… + adwd

Vector Spaces• A basis of a vector space:• A tuple <w1,w2,…,wd> of objects• Linearly independent• Spans the space uniquely

"v $[a1,a2,…,ad] v = a1w1+a2w2 +… + adwd

[3,-1]

Basis = [ , ]

v = [a1,a2,…,ad] =


"v $[a1,a2,…,ad] v = a1w1+a2w2 +… + adwd

Basis = [ , ]

v = [a1,a2,…,ad] = [3,4]

Standard


"v $[a1,a2,…,ad] v = a1w1+a2w2 +… + adwd

[3,2,-4][a1,a2,…,ad] =

Basis =

v =

[ , , ]


"v $[a1,a2,…,ad] v = a1w1+a2w2 +… + adwd

[a1,a2,…,ad] =

Basis =

v =

[ , , ][2,3,6]

Standard


"v $[a1,a2,…,ad] v = a1w1+a2w2 +… + adwd

[3,7,6]

Basis =Standard

[ x2exsin x, xexcos x, exsin x ]

[a1,a2,…,ad] =

v = 3 x2exsin x + 7 xexcos x + 6 exsin x


"v $[a1,a2,…,ad] v = a1w1+a2w2 +… + adwd

• The object V is represented by the vector [a1,a2,…,ad]• The dimension of the vector space V is d.

Vector Spaces

FindBasis( V )Let w1 be any nonzero object in VLet B = {w1} and d = 1loop

<loop inv>: B linearly independent 0 ≠ a1w1+a2w2 +… + adwd

Exit if B spans VLet wd+1 be any object in V not spanned by BLet B = B {wd+1} and d = d+1

end loopreturn(B)

Note the dimension d could be infinite.

Colour• Colour:• Each frequency f of light is a “primary colour”.• Each colour contains a mix of these

• ie a linear combination a1f1+a2f2 +… + adfd

• What is the dimension d of this vector space?• Infinite, because there are an infinite

number of frequencies• Do we see all of these colours?

• Colour:• No, we have three sensors that detect frequency

so our brain only returns three different real values.• What is the dimension d of the vector space

of colours that humans see?• d = 3. Each colour is specified by a vector [255,153,0]

Colour

• Colour:• The basis colours?

• Bases = <red,green,blue>• Or = <red,blue,yellow>

Colour

We have a [n,k,r]-linear code[|code|, |message|, hamming dist]

I have a k digit messageI encode it into an n digit code

and send it to you.

I corrupt some of the digits.

I can detect up to r-1 errors.I can correct up to (r-1)/2 errors

and recover the message.

Error Correcting Codes

We have a [n,k,r]-linear code[|code|, |message|, hamming dist]

I have a k digit messageI encode it into an n digit code

and send it to you.


Think of the code words as vectors in a sub-vector space

These code words are spread evenly through the set of all n digit tuples

so that the minimum hamming distance between any two is r.


These code words are spread evenly through the set of all n digit tuples

so that the minimum hamming distance between any two is r.

1010001

1110010

1010000

1101001

0010001

0110001 0110101

1110001

1100001

1110000

• A vector space has:• A universe V of objects. Eg:

• Code words v = 1110010• An underlying field F.

The digits of the message and in the codeare from your favorite finite field F.

Eg bits: v = 1110010Bytes: v = 13 A2 7C 41 04 F3 A2


• A vector space has:• A universe V of objects. Eg:

• Code words v = 1110010• An underlying field F. • Closed under linear combinations


10100011110010

+ 0100011

Each digit is a separate binary sum.No carries.


• A basis of a vector space:• A tuple <w1,w2,…,wd> of objects• Linearly independent• Spans the space uniquely

"v $[a1,a2,…,ad] v = a1w1+a2w2 +… + adwd

10100011110010

0100011

[0,0,1,1][a1,a2,…,ad] =

w2 = 1110010w1 = 1110010

Basis = <w1,w2,…,wd>

w4 = w3 =

v = w3+w4 =

What k digit message is associated with this code word?[3,-1]

Basis = [ , ]

v =

[a1,a2,…,ad] =

/k

/k

/k

/k/k


1010001

Given my message is “Yes, I will marry you”,I send this code.

1110010

Had my message been “No, way bozo”,I would have sent this other code.

My goal is to corrupt the message

to confuse the receiver.

I do hope I get a yes.


1010001

1110010

I must change r=3 digitsto completely corrupt

the code in an undetectable way.

We say that the hamming distance between these codes is r=3

because they differ in this many digits.

Oh. This is the code for no.


Considered the n-dimensional cube of possible codes with an edge between

those that differ in one digit.

1010001

1110010

When I corrupt a code, one digit at a time,

I travel along these edges.

1010011

1010000

1110011

11100011110000

1010010


1010001

1110010

1010000

1110000

We say that the hamming distance between these codes is r=3

because this is the shortest path between them in this cube.


1010001

1110010

1010000

This is a [n,k,r]-linear codeBecause the hamming distance

between any two codes is at least r.

1101001

0010001

0110001 0110101

1110001

1100001


1110000r

1010001

1110010

1010000

1101001

0010001

0110001 0110101

1110001

1100001


1110000

This is my code

I can detect up to r-1 errors.

I must change r=3 digitsto completely corrupt

the code in an undetectable way. r

1010001

1110010

1010000

1101001

0010001

0110001 0110101

1110001

1100001


1110000

If I receive a code that is not legal, I decode to the closest legal code.

(r-1)/2

I can correct up to (r-1)/2 errors and recover the message.

r

Linear Transformations• Linear Transformations T(v) = u• Useful for:

• Transforming objects wrt the same basis.• Change the basis used to describe an object.

• Linear means: T(au+bv) = aT(u) + bT(v) • Recall"v $[a1,a2,…,ad] v = a1w1+a2w2 +… + adwd

T(v) = T(a1w1+a2w2 +… + adwd ) = a1T(w1) +a2T(w2) +… + adT(wd )• Hence we only need to know where the basis

vectors get mapped.

• Linear Transformations• We only need to know where the basis vectors

get mapped• T(v) = a1T(w1) +a2T(w2) +… + adT(wd )

Basis = [ , ]

T( ) =

T( ) =

T( ) =

Linear Transformations



T( ) =

2 ?

0 ?[ ][ ] = [ ]1

0

2

0

2 0

0 1[ ][ ] = [ ]0

1

0

1

Basis = [ , ]

T( ) = = 2 + 0

T( ) = = 0 + 1




T( ) =

2 0

0 1[ ][ ] = [ ]a

b

2a

b

Basis = [ , ]

T( ) =




cos ?

sin ?[ ][ ] = [ ]1

0

cos

sin

Basis = [ , ]

T( ) = cos

sincos

-sin

T( ) = = cos + sin

T( ) = = -sin + cos cos -sin

sin cos[ ][ ] = [ ]0

1

-sin

cos


Integrating• f(x) = x2exsin x• Can you differentiate it? • Can you integrate it?

Sure!f’(x) = 2xexsinx + x2exsinx + x2excosx

Ahh? No

I can!Think of differentiation as a Linear Transformation and then invert it.

Integrating

x2exsinxx2excosxxexsinxxexcosxexsinxexcosx

Basis =

/x( x2exsinx )

We will explain where this basis comes from later.

Integrating


Basis =

1

0

0

0

0

0

=

/x( x2exsinx )

1

1

2

0

0

0

= 2xexsinx + x2exsinx + x2excosx

1

1

2

0

0

0

Integrating


Basis =

0

1

0

0

0

0

-1

1

0

2

0

0

=

/x( x2excosx )= 2xexcosx + x2excosx - x2exsinx

1 -1

1 1

2 0

0 2

0 0

0 0

Integrating


Basis =

0

0

1

0

0

0

0

0

1

1

1

0

=

/x( xexsinx )= exsinx + xexsinx + xexcosx

1 -1 0

1 1 0

2 0 1

0 2 1

0 0 1

0 0 0

Integrating


Basis =

0

0

0

1

0

0

0

0

-1

1

0

1

=

/x( xexcosx )= excosx + xexcosx - xexsinx

1 -1 0 0

1 1 0 0

2 0 1 -1

0 2 1 1

0 0 1 0

0 0 0 1

Integrating


Basis =

0

0

0

0

1

0

0

0

0

0

1

1

=

/x( exsinx )= exsinx + excosx

1 -1 0 0 0

1 1 0 0 0

2 0 1 -1 0

0 2 1 1 0

0 0 1 0 1

0 0 0 1 1

Integrating


Basis =

0

0

0

0

0

1

0

0

0

0

-1

1

=

/x( excosx )= excosx - exsinx

1 -1 0 0 0 0

1 1 0 0 0 0

2 0 1 -1 0 0

0 2 1 1 0 0

0 0 1 0 1 -1

0 0 0 1 1 1

Integrating


Basis =

1

1

1

1

1

1

0

2

2

4

1

3

=

/x(x2exsinx + x2excosx + xexsinx + xexcosx + exsinx + excosx)

1 -1 0 0 0 0

1 1 0 0 0 0

2 0 1 -1 0 0

0 2 1 1 0 0

0 0 1 0 1 -1

0 0 0 1 1 1

= 2x2excosx + 2xexsinx + 4xexcosx + 1exsinx + 3excosx

Integrating• f(x) = x2exsin x• Can you differentiate it? • Can you integrate it?

Sure!f’(x) = 2xexsinx + x2exsinx + x2excosx

Ahh? No

I can!Think of differentiation as a

Linear Transformations and then invert it.

Integrating

D = D-1 =

½ ½ 0 0 0 0

-½ ½ 0 0 0 0

0 -1 ½ ½ 0 0

1 0 -½ ½ 0 0

-½ ½ 0 -½ ½ ½

-½ -½ ½ 0 -½ ½

1 -1 0 0 0 0

1 1 0 0 0 0

2 0 1 -1 0 0

0 2 1 1 0 0

0 0 1 0 1 -1

0 0 0 1 1 1

Integrating


Basis =

1

0

0

0

0

0

½

-½

0

1

-½

-½

=

½ ½ 0 0 0 0

-½ ½ 0 0 0 0

0 -1 ½ ½ 0 0

1 0 -½ ½ 0 0

-½ ½ 0 -½ ½ ½

-½ -½ ½ 0 -½ ½

x2exsin x x= ½ x2exsinx - ½ x2excosx + xexcosx - ½ exsinx - ½ excosx

Integrating


Basis =

/x( x2exsinx )

We will now explain where this basis comes from.The Basis must be “Closed under Differentiation”.Wedding Party: You must invite • the bride and groom.• any friend of anyone invited.

Integrating

x2exsinxx2excosxxexsinxBasis =

/x( x2exsinx ) = 2xexsinx + x2exsinx + x2excosx


Integrating

x2exsinxx2excosxxexsinxxexcosx

Basis =

/x( x2excosx )= 2xexcosx + x2excosx - x2exsinx


Integrating

x2exsinxx2excosxxexsinxxexcosxexsinx

Basis =

/x( xexsinx )= exsinx + xexsinx + xexcosx


Integrating


Basis =

/x( xexcosx )= excosx + xexcosx - xexsinx


And so on

Integrating• f(x) = x-1exsin x• Can you differentiate it? • Can you integrate it?

Sure!f’(x) = -x-2exsinx + x-1exsinx + x-1excosx

Ahh? No

I can?

Integrating

x-1exsinxx-2exsinxx-3exsinxx-4exsinxInfiniteBasis!

Basis =

/x( x-1exsinx )= -x-2exsinx + x-1exsinx + x-1excosx x-1exsinx

/x( x-2exsinx )= -2x-3exsinx + x-2exsinx + x-2excosx /x( x-3exsinx )= -3x-4exsinx + x-3exsinx + x-3excosx

Integrating• f(x) = x-1exsin x• Can you differentiate it? • Can you integrate it?

Sure!f’(x) = -x-2exsinx + x-1exsinx + x-1excosx

Ahh? No

Oops this method does not work.

Integrating

Change of Basis: T([a1,a2,…,ad]) = [A1,A2,…,Ad]Changes the basis used to describe an object.

• The standard basis of a vector space:• A tuple <w1,w2,…,wd> of basis objects• Linearly independent• Spans the space uniquely

"v $[a1,a2,…,ad], v = a1w1+a2w2 +… + adwd• The new basis of a vector space:• A tuple <W1,W2,…,Wd> of basis objects• Linearly independent• Spans the space uniquely

"v $[A1,A2,…,Ad], v = A1W1+A2W2 +… + AdWd

• Use small letters aj for the coefficients in the standard basis and capital letters Ak for the coefficients in the new basis

Changing Basis

[3,2]

v =

[a1,a2] = [11/5,32/5][A1,A2] =

T-1([A1,A2,…,Ad]) = [a1,a2,…,ad]


"v $[a1,a2,…,ad], v = a1w1 +a2w2 +… + adwd

"v $[A1,A2,…,Ad], v = A1W1+A2W2 +… + AdWd

=[w1,w2]=[ , ]

Standard Basis

NewBasis =[W1,W2]

=[ , ]

? ?

? ?[ ][ ] =[ ]a1a2

A1A2

v =

Changing Basis

[4/5, -3/5][a1,a2] = [1,0][A1,A2] =

T-1([A1,A2,…,Ad]) = [a1,a2,…,ad]


"v $[a1,a2,…,ad], v = a1w1 +a2w2 +… + adwd

"v $[A1,A2,…,Ad], v = A1W1+A2W2 +… + AdWd

=[w1,w2]=[ , ]

Standard Basis

NewBasis =[W1,W2]

=[ , ]

?

?

?

?[ ][ ] =[ ]a1a2

A1A2

-3/54/5

v =

10

4/5 -3/5

4/5 -3/5

W1[1]W1[2]

?

?[ ][ ] =[ ]10

W1[1]W1[2]

W1[1]W1[2]

Changing Basis

[3/5,4/5][a1,a2] = [0,1][A1,A2] =

T-1([A1,A2,…,Ad]) = [a1,a2,…,ad]


"v $[a1,a2,…,ad], v = a1w1 +a2w2 +… + adwd

"v $[A1,A2,…,Ad], v = A1W1+A2W2 +… + AdWd

=[w1,w2]=[ , ]

Standard Basis

NewBasis =[W1,W2]

=[ , ]

?

?[ ][ ] =[ ]a1a2

A1A2

3/5

4/5

01

v =

3/5 4/5

4/5 -3/5

3/5 4/5

W2[1]

W2[2]

[ ][ ] =[ ]01

W1[1]W1[2]

W1[1]W1[2]

W2[1]W2[2]

?

?

Changing Basis

T-1([A1,A2,…,Ad]) = [a1,a2,…,ad]


"v $[a1,a2,…,ad], v = a1w1 +a2w2 +… + adwd

"v $[A1,A2,…,Ad], v = A1W1+A2W2 +… + AdWd

=[w1,w2]=[ , ]

Standard Basis

NewBasis =[W1,W2]

=[ , ]

[3,2]

v =

[a1,a2] = [11/5,32/5][A1,A2] =

v =

[ ][ ] =[ ]a1a2

A1A2

11/532/5

32

4/5 -3/5

3/5 4/5 [ ][ ] =[ ]W1[1]

W1[2]W2[1]W2[2]

a1a2

A1A2

Changing Basis


"v $[a1,a2,…,ad], v = a1w1 +a2w2 +… + adwd

"v $[A1,A2,…,Ad], v = A1W1+A2W2 +… + AdWd

=[w1,w2]=[ , ]

Standard Basis

NewBasis =[W1,W2]

=[ , ]

[ ][ ] =[ ]W1[1]W1[2]

W2[1]W2[2]

a1a2

A1A2

[ ] [ ] =[ ]W1[1]W1[2]

W2[1]W2[2]

a1a2

A1A2

-1

Changing Basis

W1[1]W1[2]

W2[1]

W2[2]


"v $[a1,a2,…,ad], v = a1w1 +a2w2 +… + adwd

"v $[A1,A2,…,Ad], v = A1W1+A2W2 +… + AdWd

=[w1,w2]=[ , ]

Standard Basis

NewBasis =[W1,W2]

=[ , ]If the new basis vectors are orthogonal and of uniform length:• |W1|2=n, then W1∙W1 = jW1[j]W1[j] = n

• W1W2, then W1∙W2 = jW1[j]W2[j] = 0

[ ][ ]=[ ]W1[1]W1[2]

W2[1]W2[2]

n0

0n

W1[1]W2[1]

W1[2]W2[2]

[ ] = [ ]W1[1]W1[2]

W2[1]W2[2]

-1W1[1]W2[1]

W1[2]W2[2]

1/n

Changing Basis

W1[1]W1[2]

W2[1]

W2[2]


"v $[a1,a2,…,ad], v = a1w1 +a2w2 +… + adwd

"v $[A1,A2,…,Ad], v = A1W1+A2W2 +… + AdWd

=[w1,w2]=[ , ]

Standard Basis

NewBasis =[W1,W2]

=[ , ]

[ ][ ] =[ ]W1[1]W1[2]

W2[1]W2[2]

a1a2

A1A2

[ ] [ ] =[ ]W1[1]W1[2]

W2[1]W2[2]

a1a2

A1A2

-1

W1[1]W2[1]

W1[2]W2[2][ ][ ] =[ ]a1

a2

A1A2

1/n

Changing Basis

W1[1]W1[2]

W2[1]

W2[2]


"v $[a1,a2,…,ad], v = a1w1 +a2w2 +… + adwd

"v $[A1,A2,…,Ad], v = A1W1+A2W2 +… + AdWd

=[w1,w2]=[ , ]

Standard Basis

NewBasis =[W1,W2]

=[ , ]

W1[1]W2[1]

W1[2]W2[2][ ][ ] =[ ]a1

a2

A1A2

v

a1

a2v

W1 A1

v

A1 = |v|cos() = v∙W1 = j ajW1[j]

cos() = v∙W1|v||W1|

Viewed a different way:

This is the correlation between v and W1

Changing Basis

W1[1]W1[2]

W2[1]

W2[2]

|W1|

Fourier Transformation Fourier Transform• are a change of basis from the time basis to• sine/cosine basis• JPG• or polynomial basis

• Applications• Signal Processing• Compressing data (eg images with .jpg)• Multiplying integers in n logn loglogn time.• ….

Purposes: • Some operations on the data are cheaper in new format• Some concepts are easier to read from the data in new format• Some of the bits of the data in the new format are less significant

and hence can be dropped.

http://www.dspguide.com/ch8.htm

The Scientist and Engineer's Guide toDigital Signal ProcessingBy Steven W. Smith, Ph.D.

Amazingly once you include complex numbers,the FFT codefor sine/cosines and for polynomials are the SAME.

http://www.dspguide.com/ch8.htm

Fourier Transformation

Swings, capacitors, and inductorsall resonate at a given frequency,which is how the circuit picks outthe contribution of a given frequency.

A continuous periodic function

ttime

y(t)

Find the contributionof each frequency

Sine &CosineBasis

If this is the dominate musical note of frequency = 2/T,then all the other basis functions are its harmonics frequencies:

Frequency:Note on the Piano:

, 2, 3, 4, 5, 6, ...

C C G C E G ...

Surely this can’t be expressed as sum of sines and cosines.

Fourier Transformation y(x) = x

y(x) 2 sin(x) - sin(2x) + 2/3 sin(3x)

Sine &CosineBasis


y(x) -4 sin(x) + sin(2x) - 4/9 sin(3x)

y(x) = x2

Sine &CosineBasis


• The Time basis of a vector space:• A tuple <w1,w2,…,wd> of basis objects• Linearly independent• Spans the space uniquely

"v $[a1,a2,…,ad], v = a1w1+a2w2 +… + adwd• The Fourier basis of a vector space:• A tuple <W1,W2,…,Wd> of basis objects• Linearly independent• Spans the space uniquely

"v $[A1,A2,…,Ad], v = A1W1+A2W2 +… + AdWd

Change of BasisFourier Transformation

Fourier Transformation Time Domain y Frequency Domain Y

The value y[j] of the signal at each point in time j.

The amount Y[f] of frequency f in the signal for each frequency f.

Change of Basis: T(y[0],y[1],…,y[n-1]) = [YRe[0],…,YIm[n/2]]Changes the basis used to describe an object.

Time Basis =[ , ]

y =

A discrete periodic function

j

y[j]

y[0]=3

y[1]=2

=[I1,I2,…]The time basis

j’

Ij[j’]

zeroone

j

"y $[y[0],y[1],…,y[n-1]], y = y[0]I1 +y[1]I2 +… + y[n-1]In

Change of BasisFourier Transformation

The value y[j] of the signal at each point in time j.

Time Basis =[ , ]

y =


A discrete periodic function

j

y[j]

=[I1,I2,…] y = YRe[0]∙c1+YIm[0]∙s1+ ,…,YRe[n/2]∙sn/2+YIm[n/2]∙sn/2

=[c1,s1,..]

YRe[0] =11/5

YIm[0] =32/5

y =

FourierBasis

=[ , ]=[?,?]

c1

sn/2cn/2

s1

y[0]=3

y[1]=2

Change of Basis


"y $[y[0],y[1],…,y[n-1]], y = y[0]I1 +y[1]I2 +… + y[n-1]In

The amount Y[f] of frequency f in the signal for each frequency f.

Time Basis =[ , ]

y =


=[I1,I2,…] y = YRe[0]∙c1+YIm[0]∙s1+ ,…,YRe[n/2]∙sn/2+YIm[n/2]∙sn/2

=[c1,s1,..]

YRe[0] =11/5

YIm[0] =32/5

y =

FourierBasis

=[ , ] c1

sn/2cn/2

s1

y[0]=3

y[1]=2

Change of Basis


"y $[y[0],y[1],…,y[n-1]], y = y[0]I1 +y[1]I2 +… + y[n-1]In

22

0Im

0Re ][][

nn

ff

ff jsfYjcfY jy

s1[1]s2[1]

s1[2]s2[2]

Y[1]Y[2][ ] [ ] =[ ]y[1]

y[2]

-1

s1[1]s1[2]

s2[1]s2[2]

Y[1]Y[2]

y[1]y[2][ ] [ ] =[ ]

?

Correlation (DFT)Ex. 1 Signal 1 Ex. 2 Signal 2

Searching for s3 sine base

Correlation (point wise mult) of 2 above signals

Ʃ ≠ 0 signal present Ʃ = 0 signal not presentwhy? orthogonal basis

Time Basis =[ , ]

Fourier Transformation =[I1,I2,…] Fourier

Basis =[ , ]

Sine and Cosines of different frequencies are orthogonal and of (almost) uniform length:

=[c1,s1,..]

2

1

0

2 ][][ nf

n

jffff jsjsss||s

0][][ hence

, and For 1

0

jsjsssss

ngfgf

g

n

jfgfgf

[ ][ ]=[ ]s1[1]s1[2]

s2[1]s2[2]

n/2

00n/2

s1[1]s2[1]

s1[2]s2[2]

[ ] = [ ]s1[1]s1[2]

s2[1]s2[2]

-1s1[1]s2[1]

s1[2]s2[2]

2/n

OrthogonalBasis

Time Basis =[ , ]

Fourier Transformation =[I1,I2,…] Fourier

Basis =[ , ]=[c1,s1,..]

22

0Im

0Re ][][

nn

ff

ff jsfYjcfY jy

1

0Im

1

0Re ][2 ,][2

n

jf

n

jf jsjyn fYjcjyn fY

[ ][ ] =[ ]s1[1]s1[2]

s2[1]s2[2]

Y[1]Y[2]

s1[1]s2[1]

s1[2]s2[2]

2/nY[1]Y[2][ ][ ] =[ ]

y[1]y[2]

y[1]y[2]

OrthogonalBasis

Duality of FT: If Y=FT(y), then y=FT(Y)


Delta function

Impulse at y[4]

Delta function

Impulse at Yre[4]

Cosine wave

Cosine with f=4

Cosine wave

Cosine with f=4


Dualityof FT

How do you get these corner?

?


Sinc function-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1

-0.5

0

0.5

1

1.5

-100 -50 0 50 1000

1

2

3

4

5

6 Square wave

-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1-0.5

0

0.5

1

1.5

-100 -50 0 50 1000

1

2

3

4

5

6

Square wave

-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1-0.5

0

0.5

1

1.5

-100 -50 0 50 1000

1

2

3

4

5

6

Sinc function-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1

-0.5

0

0.5

1

1.5

-100 -50 0 50 1000

1

2

3

4

5

6


Dualityof FT



Gaussian

0 5 10 15 20 25 30 35 40 45 500

0.1

0.2

0.3

0.4

0.5

0 50 100 150 200 2500

1

2

3

4

5

6

Gaussian

0 5 10 15 20 25 30 35 40 45 500

0.1

0.2

0.3

0.4

0.5

0 50 100 150 200 2500

1

2

3

4

5

6

Dualityof FT

Fourier Transformation Continuous Functions nn b fY a fY ImRe and

O(log(n)) levels

Fourier Transformation FFTButterfly

Fast Fourier Transform takes O(nlogn) time! (See Recursive Slides)


Sound Signal ie how far out is the speaker drum at each point in time.

Sound is low frequency

High frequencies filtered out.

RadioSignals


Radio Carrier Signal ie A wave of magnetic field that can travel far.

One high frequency signal

RadioSignals


y(i) = y1(i) y2(i) Modulation: Their product

Carrier signalAudio Signal (shifted)Audio Signal (shifted &flipped)

RadioSignals

x[] y[]

Fourier Transformation This system takes in a signal and outputs transformed signal.

LinearFilter


In order understand this transformation, we put in a single pulse.

[] = h[] = h[]

This response h[]identifies the system.

h[] =

Linear Filter


h[]

Feed in any signal

x[] =

h[] =

Sum of contributions from each separate pulse.

Linear Filter

operator n convolutio theis where

0

hxy

jkhjxkyN

j

Computationally trying to figure out what this electronic system does to a signal takes O(nm) time.

How can we do it faster?


Y = X H Product

y = x*h Convolution

x[] =

Input

h[] =

Impulse Response

X[]

H[]

x[]*h[] =

Output

X[]H[] Multiplication takes O(n) time.

OopsFourier Transformtakes O(n2) time.

FastFourier Transform takes O(nlogn) time!

*Convolution


Y = X H Product

y = x*h Convolution

x[] =

Input

Impulse Response

X[]

x[]*h[] =

Output

*Convolution

h[] = H[] =

Impulse Response

X[]H[]

Multiplyingzeros low and high frequencies in input.

Filters out low and high frequencies in input.

Not clear what system does to input

Fourier Transformation JPEG (Image Compression)

JPGImage Compression

JPEG is two dimensional Fourier Transformexactly as done before.

Fourier Transformation JPGImage Compression

Each 88 block of valuesfrom the imageis encoded separately.



It is decomposed as a linearcombination of basis functions.

7

0

7

0, ],[,,

u vvu yxBvuF yxf

Each basis function has a coefficient,giving the contribution of this basis functionto the image.


16

12cos

16

12cos,

]7,0[,]7,0[, :functions Basis

,

yπv

xπuyxB

yx vu

vu

7

0

7

0, ],[,,

u vvu yxBvuF yxf


It is decomposed as a linearcombination of basis functions.

Each of the 64 basis functions is a two dimensional cosine.


The first basis is constant.

16

12cos

16

12cos,


,

yπv

xπuyxB

yx vu

vu

1,0,0 yxB

Its coefficient gives the average value in within block.

7

0

7

0, ],[,,

u vvu yxBvuF yxf

Because many images havelarge blocks of the same colour,this one coefficient gives muchof the key information!


Its (pos or neg) coefficient gives whether left to right the value tends to increase or decrease.

The second basis “slopes” left to right

116

12cos,0,1

x

πyxB

16

12cos

16

12cos,


,

yπv

xπuyxB

yx vu

vu

16

12cos

16

12cos,


,

yπv

xπuyxB

yx vu

vu


The second basis “slopes” left to right

Because many images havehave a gradual change in colour,this one coefficient gives morekey information!


A similar basis for top to bottom.

16

12cos

16

12cos,


,

yπv

xπuyxB

yx vu

vu

16

12cos

16

12cos,


,

yπv

xπuyxB

yx vu

vu


16

122cos1,2,0

xπyxB

The <0,2> basis is

whether the value tends to be smaller in the middle.

Its coefficient gives

This helps display the horizontal lines in images


As seen, the low frequency components of a signal are more important.Removing 90% of the bits from the high frequency componentsmight remove, only 5% of the encoded information.


Instead of using sine and cosinesas the basis,

PolynomialBasis


Instead of using sine and cosinesas the basis,

We will now use polynomials.

PolynomialBasis

Change of Basis: T([y[0],y[1],…, y[n-1]]) = [a1,a2,…,an-1]Changes the basis used to describe an object.

Time Basis =[ , ]


y[0]=3

y[1]=2

=[I0,I1,…]The time basis

x

Ij[x]

zeroone

xj

PolynomialBasis

f =

A discrete function

x

f(x)

the value f(xj) of the function at xj.

y[j] =

x0 x1 x2 x3 x4 … xn-1

These xj are fixed values.

For FFT, we set xj = e2i j/n

"f $[y[0],y[1],…,y[n-1]], f = y0 I0 +y1 I1 +… + yn-1 In-1

Change of Basis: T([y[0],y[1],…, y[n-1]]) = [a1,a2,…,an-1]Changes the basis used to describe an object.

Time Basis =[ , ]


y[0]=3

y[1]=2

=[I0,I1,…]

"f $[y[0],y[1],…,y[n-1]], f = y0 I0 +y1 I1 +… + yn-1 In-1

PolynomialBasis

f =

A discrete function

x

f(x)

x0 x1 x2 x3 x4 … xn-1

y =

FourierBasis

=[ , ]=[1,x,x2,x3..]

"f $[a0,a1,a2 ,…,an-1], f = a0+a1x +a2x2 + … + an-1xn-1

The aj are the cooeficients of the polynomial.

a1

a2

• A Fourier Transform is a change in basis.• It changes the representation of a function

• from the coefficients of the polynomial f(x) = a0+a1x +a2x2 + … + an-1xn-1

• This amounts to evaluating f at these points.

Evaluating &Interpolating

x

• to the value f(xi) at key values xi.

x0 x1 x2 x3 x4 … xn-1

y0 y1 y2 y3 y4 … yn-1


yi = f(xi)

• A Fourier Transform is a change in basis.• It changes the representation of a function


• from the coefficients of the polynomial f(x) = a0+a1x +a2x2 + … + an-1xn-1

• This amounts to evaluating f at these points.

(x0)0 (x0)1 (x0)2 (x0)3 … (x0)n-1 a0

a1

a2

a3

…

an-1

y0

y1

y2

y3

…

yn-1

=

(x1)0 (x1)1 (x1)2 (x1)3 … (x1)n-1

(xn-1)0(xn-1)1(xn-1)2 (xn-1)3…(xn-1)n-1

(x2)0 (x2)1 (x2)2 (x2)3 … (x2)n-1 (x3)0 (x3)1 (x3)2 (x3)3 … (x3)n-1

Vandermonde matrixInvertible if xi distinct.


yi = f(xi)


• to the coefficients of the polynomial f(x) = a0+a1x +a2x2 + … + an-1xn-1

• This amounts to interpolating these points.

• An Inverse Fourier Transform is the reverse. • It changes the representation of a function


x

• from the value f(xi) at key values xi.

x0 x1 x2 x3 x4 … xn-1

y0 y1 y2 y3 y4 … yn-1

yi = f(xi)


• to the coefficients of the polynomial f(x) = a0+a1x +a2x2 + … + an-1xn-1

• This amounts to interpolating these points.

Given a set of n points in the plane with distinct x-coordinates, there is exactly one (n-1)-degree polynomial going through all these points.

• An Inverse Fourier Transform is the reverse. • It changes the representation of a function


Polynomial Multiplicationf(x) = a0+a1x +a2x2 + … + an-1xn-1

g(x) = b0+b1x +b2x2 + … + bn-1xn-1

[f×g](x) = c0+c1x +c2x2 + … +c2n-2x2n-2

x5 coefficient: c5= a0×b5+a1×b4 + a2×b3 + … + a5×b0

Time = O(n2)

Too much

Convolution

Polynomial Multiplicationf(x) = a0+a1x +a2x2 + … + an-1xn-1

g(x) = b0+b1x +b2x2 + … + bn-1xn-1

[f×g](x) = c0+c1x +c2x2 + … +c2n-2x2n-2

Coefficient Domain aj Evaluation Domain yi

[a0,a1,a2 ,…,an-1]

[b0,b1,b2 ,…,bn-1]

Fast Fourier Transform takes O(nlogn) time!

yi = f(xi)zi = g(xi)

yi×zi = [g×f](xi)

Multipling values pointwisetakes O(n) time!

[c0,c1,c2 ,…,cn-1]

Multiplying Big IntegersX = 11…10100011101100010010 (N bits)Y = 10…01001100011001001111

X×Y = 10…1110110101001001010100010100110010011110

The high school algorithm takes O(N2) bit operations.Can we do it faster?

With FFT we can do it in O(N log(N) loglog(N)) time.

See Recursive Slides.

In many problems we face functions which are far more complicated than the standard functions from classical analysis. If we can represent them as series of polynomials then some properties of the functions would be easier to study.

Taylor Expansions

Taylor ExpansionsTaylor Expansions of a Function:

Functions f(x) can be expressed byF(x) = a0+a1x +a2x2 +a3x3 + …Clearly only converges if x<1 and/or ai 0.But gives the perfect answer within some range of x.

Eg: f(x) = 1/(1-x) F(x) = 1+x +x2 +x3 + …

Proof: xF(x) = x +x2 +x3 +x4 + …F(x)-xF(x) = 1F(x) = 1/(1-x)


Eg: f(x) = 1/(1-x) F(x) = 1+x +x2 +x3 +

Functions f(x) can be approximated by F(x) = a0+a1x +a2x2 +a4x3 + … + an-1xn-1

x4 + (x5)


Eg: f(x) = 1/(1-x) F(x) = 1+x +x2 +x3 + …

Eg: f(x) = 1/(1- x) F(x) = 1+ x +2 x2 +3 x3 + …

ai =i

Converges if | x| < 1

if |x| < 1/

Taylor Expansions

(Some functions?)Analytic

Eg: f(x) = 1/x F(x) = ??

Taylor Expansions of a Function:

The problem is a0 = f(0)=.


Functions f(x) can be expressed byF(x) = a0+a1x +a2x2 +a3x3 + …

F(x) = a0+a1x + a2x2+ a3x3+ a4x4 + …F’(x) =F’’(x) =F’’’(x) =

a1 + 2 a2x + 3 a3x2 + 4 a4x3+ …

a0 f(0)a1 f ’(0)

F(0) = F’(0) =F’’(0) =F’’’(0) =Fi(0) =

Proof:

2 a2 + 2∙3 a3x + 3∙4 a4x2 + …

i! ai

2∙3 a3 + 2∙3∙4 a4x + …

2 a21/2 f ’’(0)

2∙3 a3

a0 = a1 =

ai =

a2 = a3 =

1/i! f i(0)

1/1∙2∙3 f ’’’(0)



f(0) = e0 = 1f’(0) = e0 = 1f’’(0) = e0 = 1f’’’(0) = e0 = 1

Example: f(x) = ex

ai = 1/i! f i(0)

Converges for all x.



f(0) = sin(0) = 0f’(0) = cos(0) = 1f’’(0) = -sin(0) = 0f’’’(0) = -cos(0) = -1

Example: f(x) = sin(x)

ai = 1/i! f i(0)

Converges for all x.

0

)(

32

)(!

)(

)(!3

)(''')(

!2

)(''))((')()(

i

ii

axi

af

axaf

axaf

axafafxF

Taylor Series for f(x) centered at a:

Clearly requires f(x) to be infinitely differentiable at x = a:

Taylor Expansions

The nth order approximation:

The Lagrange Remainder:

Taylor Expansions

n

i

ii

nn

n

axi

af

axn

afax

afaxafafxF

0

)(

)(2

)(!

)(

)(!

)()(

!2

)(''))((')()(

)()()( xFxfxR nn

ion.approximat good a is )()!1(

)( next term The 1

)1(

nn

axn

af

)()!1(

)( ],,[ 1*

)1(*

n

n

n xn

afRxax

Taylor Expansions

36110.00000000)47832.0(!9!9

)cos(

!9

)( )(

360400000000.0)47832.0(31090.46028837)47832.0sin(95050.46028836

!7

47832.0

!5

47832.0

!3

47832.047832.0)47832.0sin(

!7!5!3 )sin(

? 2)sin(0.4783

8

99

*9

*)9(

8

8

753

753

R

xx

xx

xfxR

R

xxxxx

Application:

1

0

2 ?)sin( dtt

!7!5!3

)sin(753 xxx

xx

!7!5!3

)sin(14106

22 ttttt

70.31026815!715

1

!511

1

!37

1

3

1

!715!511!373

1

)!7!5!3

()sin(

1

0

151173

1

0

1

0

1410622

tttt

dtttt

tdtt

Application:

Taylor Expansions

1))(!3

11(lim))(

!3

11(lim

1sinlim 5353

xRx

xRxx

xx

xxxx

?1

sinlim x

xx

Application:

Taylor Expansions

Application:

Taylor Expansions

• Find solutions to differential equations.• Newton’s method to find the root of a function.• Can be extended to functions in several variables.

Generating functions:• Hiding interesting values within the coefficients

of a Taylor expansion of a function. • It is so powerful that it can solve:

• Most recurrences• Most sums• Lots of the neat math facts.

Generating-Functionologyby Herbert S. Wilf(Academic Press)

Is HIGHLY recommended!

Generating Functions


Which function has the Taylor Expansion withthe Fibonacci sequence for coefficients?

G = i=0.. Fi xi

where F0 = 0, F1=1, Fn=Fn-1 + Fn-2

G = F0 + F1 x + F2 x2 + F3 x3 + F4 x4 + F5 x5 + … -x G = -F0 x - F1 x2 - F2 x3 - F3 x4 - F4 x5 - …

-x2 G = -F0 x2 - F1 x3 - F2 x4 - F3 x5 - …

0 0 0x 0(1-x-x2) G =xG =

(1-x-x2)

The fact that the manipulation of polynomial equations can encode the same theorems that are proved by combinatorial reasoning is very significant!

Never underestimate the insights encoded into the coefficients of a polynomial!


Generating FunctionsLets count things.Eg: Let p(n) denote the # of binary trees there are with n nodes.

p(1)

p(2)

= 1

= 2

p(3) = 5

p(4)

?

= 14

?

(#L,#R) = (3,0),(2,1),(1,2),(0,3)? ?

(3,0)

(2,1)

(0,3)

(1,2)

p(0) = 1

Generating FunctionsLets count things.Eg: Let p(n) denote the # of binary trees are there with n nodes.

p(1)

p(2)

= 1

= 2

p(3) = 5

p(4) = 14

(#L,#R) = (n-1,0),(n-2,1),(n-3,2),…,(0,n-1)? ?(n-i-1,i)

p(0) = 1

p(n)

p(n-i-1) p(i)

= i=0..n-1 p(n-i-1)∙p(i)

Generating Functions(in a real cool way).

Let T denote the (infinite) set of all binary trees

Tt

tnT xP )(

.... 44444333332210 xxxxxxxxxxxxxx....145211 432 xxxx

....)4()3()2()1()0( 432 xpxpxpxpp

n

nxnp )(

For each tree, t, let n(t) denote the number of nodes in t.

The values of p(n) can be read off the coefficients of the polynomial.

Lets count things.Eg: Let p(n) denote the # of binary trees are there with n nodes.

Generation function for set T and powers n(t).

…}T={

Generating FunctionsLet T denote the (infinite) set of all binary trees

Note that a tree t is either:• the empty tree t=• or consists of:• a root node• a left tree t1,

• a right tree t2.

t=

…}T={

, , = ,t1,t2=


TTtt

ttnTT xP

21

21

,

),(

TT={t1,t2 | t1,t2 T}

trees.ofpair in nodes ofnumber total, 2121 tntnttn

Tt Tt

tntnx1 2

21 )()(

Tt

tn

Tt

tn xx2

2

1

1 )()(

TT PP

…}T={

={ , , , , , , …}

…}T={

(a+b+c)

(u+v+x) =(au+av+ax+ bu+bv+bx+ cu+cv+cx)


TTtt

ttnTT xP

21

21

,

),(

TT={t1,t2 | t1,t2 T}

trees.ofpair in nodes ofnumber total, 2121 tntnttn

Tt Tt

tntnx1 2

21 )()(

Tt

tn

Tt

tn xx2

2

1

1 )()(

TT PP

…}T={

={ , , , , , , …}

…}T={

Theorem: The generating function of the cross products of two sets is the product of the generating functions of the two sets provided the power n(t) are additive.


…}T={

Theorem: The generating function of the disjoint union of two sets is the of the generating functions of the two sets.

sum

STa

anST xP )(

ST PP

Ss

sn

Tt

tn xx )()(


TTtt

ttnTT xP

21

21

,

),,(

2121 1,, tntnttn

TT PPx

…}T={

TTtt

ttnx21

21

,

),(1

TTtt

ttnxx21

21

,

),(

={ , , , , , , , , , …} TT ={ ,t1,t2 | t1,t2 T}


…}T={

={ , , , , , , , , , …} TT ={ ,t1,t2 | t1,t2 T}

Note that a tree t is either:• the empty tree t=• or t= ,t1,t2

t= , , = ,t1,t2=

Hence, the set TT can be thought of as a set of binary trees.But does it contain all of T?No. It is missing empty tree .


…}T={

T = {} TT

Note that a tree t is either:• the empty tree t=• or t= ,t1,t2

0n

10)(}{ xxP n

TTTTTTT PPxPPPP 1}{}{

21 xPP a

acbbP

cbPaP

2

4

02

2

+-

x

xP

2

411

Taylor Expansion?

(Recall) Taylor Expansions of a Function:Functions f(x) can be expressed byF(x) = a0+a1x +a2x2 +a3x3 + …

f(0)f ’(0)1/2 f ’’(0)

a0 = a1 =

ai =

a2 =

1/i! f i(0)

x

xP

2

411

?0

0

)0(2

)0(411)0(

P

12

414

21

x)('

)('

0

0

)(

)(lim 0

xg

xfxg

xfx

L'Hôpital's Rule


(Recall) Taylor Expansions of a Function:Functions f(x) can be expressed byF(x) = a0+a1x +a2x2 +a3x3 + …

f(0)f ’(0)1/2 f ’’(0)

a0 = a1 =

ai =

a2 =

1/i! f i(0)

x

xP

2

411


...145211 432 xxxxP....)4()3()2()1()0( 432 xpxpxpxpp


p(n) Proof Sketch

Catalan Numbers

http://en.wikipedia.org/wiki/Catalan_number


Binomial Coefficients

What if n is not an integer?


Binomial Coefficients

(½)(½-1)(½-2)(½-3) …(½-n+1) (½)n = n terms, non-int

n-1 neg

(1) (1) (3) (5) … (2n-3) - (-2)n (½)n =

n-1 terms, odd

(2) (4) (6) … (2n-2) 2n-1 (n-1)! = n-1 terms, even

(2n-2)!-½ (-4)n(½)n(n-1)! = 2n-2 terms

x>y

!1)4(

2)!-(2n221

nnn

n2

1

1

22

)4(

2

n

n

nn !1!1

2)!-(2n

)4(

2

nnnn

!

21

nn !

1

!1)4(

2)!-(2n2

nnn

n

n

yn

y

0

2

1

1


Set y=-4x.

nnr

n

r yxn

ryx

0

x411

11 y Remove constant coefficient & negate.

n-1 n

n2

1

n

n

yn

1

2

1

1 n

nn

yn

n

n

1 1

22

)4(

21

nn

nx

n

n

n4

1

22

)4(

2

1

n

nn

yn

n

n

1 1

22

)4(

2

n

n

xn

n

n

1 1

222

1

1 1

221

n

n

xn

n

nx

xP

2

411

1

22

)4(

2

n

n

nn

n

n

xn

n

n

0

2

1

1

)(np

n

n

xnp

0

)(


n

n

n

2

1

1

Generating FunctionsLets count things.Eg: Let p(n) denote the # of binary trees are there with n nodes.

(in a real cool way).

We will explain this approximationwhen doing prime numbers.

)(np

n

n

n

2

1

1

Prime Number Theorem:Every integer can be uniquely decomposedinto a unique product of primes.

321212

Why is 1 not considered a prime?Because then this factorization would not be unique.

32112127

Primesp is prime if it is a positive integer and nothing but 1 and p divided into it. Eg 2,3,5,7,11,13,17,19,23,29,31,....

Primes

21

22

23

317

168

21

22

23

317 32

5168 90

Greatest Common Divisor (GCD)

21

31 32

590

Each integer can be though to asthe set of its prime multiples.

Subscripts are used to differentiate between copies.

The intersection 2∙3 = 6 isThe union 23∙32∙5∙7 = 2520 is Greatest Common Multiple

Number of Primes

Proof: By contradiction, suppose there are only a finite number.Hence, there is a maximum prime.Let it be p.Let n=p!+1.Note every prime p’ ≤ p does not divide into n, becausethe remainder is 1.Consider the decomposition of n into prime factors.It contains no primes ≤pHence there is a prime bigger than p that divides into n.Contradicting the fact that p is the biggest prime.

Theorem: There are an infinite number of primes.

iii

N

NN

th ln prime ln(10)n

10 primesdigit n of#

ln ]...1[primes#

:Theorem

n

Number of Primes

Proof: Count the # of prime factors of

NN2

Number of Primes

ba

= “a choose b”= Given a set of a objects, it is the number of ways of choosing a subset consisting of b of them.

)!(!

!

bab

a

objects) remaining theunarrange to waysobjects)(# b theunarrange to ways(#

objects thearrange to ways#

a-b

a

)1( )...2)(1)((

)1)...(2)(1)((

bbb

baaaa

Number of Primes

ba

a

ab

aba

area 2 objects of subsets of # ..0

a

= “a choose b”= Given a set of a objects, it is the number of ways of choosing a subset consisting of b of them.

ba

02

a ab

Lm 1:nn

n n

2

22 2

2/aa

a

a2 a2

a

a2

Number of Primes

ba

= “a choose b”= Given a set of a objects, it is the number of ways of choosing a subset consisting of b of them.= an integer

Prime Number Theorem:Every integer can be uniquely decomposedinto a unique product of primes, eg 3212

12

pppp kdddd

nn k

3212 321

Let

Number of Primespppp k

ddddnn k

3212 321

]...1[ np ]2...1[ nnp ...]12[ np

Lm 2: nnpnnp 2/],2...1[prime

i.e. All of these primes appear here at least once each.

Proof: )1(.)..2)(1)((

)1...()...22)(12)(2(2

nnn

npnnnnn

This p appears on the top,but can’t be cancelled by the bottom

because it is a prime.

?

?


ddddnn k

3212 321

]...1[ np ]2...1[ nnp ...]12[ np

Lm 2: nnpnnp 2/],2...1[prime

i.e. All of these primes appear here at least once each.

These only make the product bigger

And each of these is at least n.

]2...1[primes# nnn


ddddnn k

3212 321

]...1[ np ]2...1[ nnp ...]12[ np

Proof: )1(.)..2)(1)((

)1)...(22)(12)(2(2

nnn

nnnnnn

This p does not appear on the top,and can’t be put together from parts

because it is a prime.

?

i.e. None of these primes appear. Lm 3: n

npnp 2/...],12[prime

]2...1[primes# nnn


ddddnn k

3212 321

]...1[ np ]2...1[ nnp ...]12[ np

Proof: Later

?

i.e. Each prime contributes at most 2n to the product.

Lm 4: nppd dnn 2 then , divides timesofnumber theis If 2

]2...1[primes#)2( nn ]2...1[primes# nnn

Number of Primes? ]2...1[primes#)2( nn ]2...1[primes# nnn

Lm 1nn

n n

2

22 2

)(log]2...1[primes# 2 nnn )2(log2 22

1 nn )2(log]2...1[primes# 2 nn

)(log

2]2...1[primes#

2 n

nnn

?]...1[primes# N

N2N4N8N

Ni

ii

2log..1

1 ]2...12[primes#

Ni

i

i2log..1 1

2

I

I 12

N

N

N

N

ln1.39

ln

)2ln(2

Number of Primes? ]2...1[primes#)2( nn ]2...1[primes# nnn

Lm 1nn

n n

2

22 2

)(log]2...1[primes# 2 nnn )2(log2 22

1 nn )2(log]2...1[primes# 2 nn

littleln

1.39]...1[primes# N

NN

little)2(log

2]2...1[primes#

2

n

nnlittle

)(log]...1[primes#

2

N

NN little

)ln(

69.0little

)ln(

)2ln(

N

N

N

N

N

NN

ln1 ]...1[primes#

:Truth

Back to the proof of lemma 4.

Number of Primes

Proof:

Lm 5: times.most at divides 1

..iip

nn! p

n! = 1∙2∙3∙... ∙n

There is one place where p divides n!.And another.This gives of them.

∙p∙... ∙2p∙... ∙3p∙... ∙4p∙...

p

n

Number of Primes

Proof:

Lm 5: times.p

nmost at divides

1..ii

n! p

n! = 1∙2∙3∙... ∙n

There is one place where p divides n! two times.But one of these we counted in the last slideso this adds only one more to our count.And another.This gives more of them.

∙p2∙... ∙2p2∙... ∙3p2∙... ∙4p2∙...

2p

n

Number of Primes

Proof:

Lm 5: times.p

nexactly divides

1..ii

n! p

n! = 1∙2∙3∙... ∙n

There is one place where p divides n! i times.But all but one of these we counted already.So this adds only one more to our count.And another.This gives more of them.

∙pi∙... ∙2pi∙... ∙3pi∙... ∙4pi∙...

ip

n

Total: times.p

nexactly divides

1..ii

n! p

Number of Primes

Proof:

Lm 6: 122

b

a

b

a

brrqba for

b

rqb

b

rqb

b

a

b

a2

)(22

2

qb

rq 2

22

1 2

b

r


ddddnn k 321

2 321

Lm 4: nppd dnn 2 then , divides timesofnumber theis If 2

2 2

1..i1..i

ii p

n

p

n Lm 5: times.most at divides 1

..iip

nn! p

..12

2i ii p

n

p

n 0

2 and 2 then ,2log If

ii

p p

nnpni

p

n

p

n

n

i ii

p

log

12

2

Proof: n

npd 2 divides timesofnumber the

!!

!2 divides timesofnumber the

nn

np

) divides timesofnumber 2(the-)2 divides timesofnumber (the n!pn!p

Lm 6: 122

b

a

b

a

n

i

p

log

11

n p 2log

n pd 2

Number of Primes

• Primes are more or less randomly distributed.• If you want an n-bit prime,

• Generate a random n-bit number p and• Pr[p is prime] ~ 1/n

• Repeat 10n times and you likely have found a prime.

iii

N

NN

th ln prime n ln(2)

2 primesbit n of#

ln ]...1[primes#

:Theorem

n

Number of Primes

• Primes are more or less randomly distributed..• If you want p and p+2 to both be primes

• Generate a random n-bit number p and• Pr[both p and p+2 are prime] ~ 1/n

2

• Repeat 10n2 times and you likely have found twin primes.

iii

N

NN

th ln prime n ln(2)

2 primesbit n of#

ln ]...1[primes#

:Theorem

n

Conjectured for 100 yearsbut not proven

Number of Primes

• Primes are more or less randomly distributed..• If you want p and p+1 to both be primes

• Generate a random n-bit number p and• Pr[both p and p+1 are prime] ~ 0, because one must be even• Oops.

iii

N

NN

th ln prime n ln(2)

2 primesbit n of#

ln ]...1[primes#

:Theorem

n

Number of Primes

• Primes are more or less randomly distributed..• If you want p to be prime and p-1 to be a power of 2

• Generate a random n-bit number p and• Pr[p-1 is a power of 2] ~ 1 / 2n Note 1000000002 is only n-bit power of 2.• Oops

iii

N

NN

th ln prime n ln(2)

2 primesbit n of#

ln ]...1[primes#

:Theorem

n

Number of Primes

• Primes are more or less randomly distributed..• If you want p to be prime and p-1 to be a power of 2

• Generate a random ~n-bit number N that is a power of 2• Pr[N+1 is prime] ~ 1 / n• Repeat 10n times and you likely have found such

a p and p-1.• You will try N=2n,2n+1,2n+2,… 211n

getting an 11n-bit number

iii

N

NN

th ln prime n ln(2)

2 primesbit n of#

ln ]...1[primes#

:Theorem

n

Number of Primes

• Primes are more or less randomly distributed..• If you want p to be prime and p-1 divisible by 2n

• Generate a random small r,• Let p = r2n+1• Pr[p is prime] ~ 1 / n• Repeat 10n times and you likely have found good r.• Try r = 1,…,10n. • p will need log p = n + logr = n + log 10n bits.• Note this is much better than 11n

iii

N

NN

th ln prime n ln(2)

2 primesbit n of#

ln ]...1[primes#

:Theorem

n

Number of Primes

• Primes are more or less randomly distributed.• Homework questions:• # of N of the form N=pq

• # of N of the form N=pq for primes p&q• # of prime factors of N

iii

N

NN

th ln prime n ln(2)

2 primesbit n of#

ln ]...1[primes#

:Theorem

n

End

Documents

Algebra Jeff Edmonds York University COSC 6111 Fields GCD Powers mod p Fermat, Roots of Unity, & Generators Z mod p vs Complex Numbers Cryptography Other