Graph Signal Processing - Basics on FourierAnalysis

Prof. Luis Gustavo Nonato

August 23, 2017

Inner Product

Given two vectors x = (x1, . . . , xd) and y = (y1, . . . , yd) in Rd,the dot product (or inner product) of x and y is

< x, y >=d

∑i=1

xiyi

If x and y are in Cd (complex vectors) then

< x, y >=d

∑i=1

xiyi

where yi is the complex conjugate of yi.The purpose of the conjugate is to ensure that the length ofz ∈ Cd is real and nonnegative (z = x + iy, zz = x2 + y2).

Inner Product

Given two vectors x = (x1, . . . , xd) and y = (y1, . . . , yd) in Rd,the dot product (or inner product) of x and y is

< x, y >=d

∑i=1

xiyi

If x and y are in Cd (complex vectors) then

< x, y >=d

∑i=1

xiyi

where yi is the complex conjugate of yi.The purpose of the conjugate is to ensure that the length ofz ∈ Cd is real and nonnegative (z = x + iy, zz = x2 + y2).

Inner Product

Given two vectors x = (x1, . . . , xd) and y = (y1, . . . , yd) in Rd,the dot product (or inner product) of x and y is

< x, y >=d

∑i=1

xiyi

If x and y are in Cd (complex vectors) then

< x, y >=d

∑i=1

xiyi

where yi is the complex conjugate of yi.

The purpose of the conjugate is to ensure that the length ofz ∈ Cd is real and nonnegative (z = x + iy, zz = x2 + y2).

Inner Product

Given two vectors x = (x1, . . . , xd) and y = (y1, . . . , yd) in Rd,the dot product (or inner product) of x and y is

< x, y >=d

∑i=1

xiyi

If x and y are in Cd (complex vectors) then

< x, y >=d

∑i=1

xiyi

where yi is the complex conjugate of yi.The purpose of the conjugate is to ensure that the length ofz ∈ Cd is real and nonnegative (z = x + iy, zz = x2 + y2).

Inner Product

Let f and g be functions defined on the square integrable spaceL2[a, b] (f : [a, b]→ C,

∫ ba |f (t)|

2dt < ∞).

The dot product of f and g is

< f , g >=∫ b

af (t)g(t)dt

The functions f and g are said orthogonal if < f , g >= 0A collection of functions ei are orthonormal if < ei, ej >= δij

Suppose E = {e1, . . . , en} a set of orthonormal functions basis.Then E generates a subspace of L2[a, b] and any function h inthis subspace can be written as (h lies in the span of E )

h =n

∑i=1

< h, ei > ei

Inner Product

Let f and g be functions defined on the square integrable spaceL2[a, b] (f : [a, b]→ C,

∫ ba |f (t)|

2dt < ∞).The dot product of f and g is

< f , g >=∫ b

af (t)g(t)dt

The functions f and g are said orthogonal if < f , g >= 0A collection of functions ei are orthonormal if < ei, ej >= δij

Suppose E = {e1, . . . , en} a set of orthonormal functions basis.Then E generates a subspace of L2[a, b] and any function h inthis subspace can be written as (h lies in the span of E )

h =n

∑i=1

< h, ei > ei

Inner Product

Let f and g be functions defined on the square integrable spaceL2[a, b] (f : [a, b]→ C,

∫ ba |f (t)|

2dt < ∞).The dot product of f and g is

< f , g >=∫ b

af (t)g(t)dt

The functions f and g are said orthogonal if < f , g >= 0

A collection of functions ei are orthonormal if < ei, ej >= δij

Suppose E = {e1, . . . , en} a set of orthonormal functions basis.Then E generates a subspace of L2[a, b] and any function h inthis subspace can be written as (h lies in the span of E )

h =n

∑i=1

< h, ei > ei

Inner Product

Let f and g be functions defined on the square integrable spaceL2[a, b] (f : [a, b]→ C,

∫ ba |f (t)|

2dt < ∞).The dot product of f and g is

< f , g >=∫ b

af (t)g(t)dt

The functions f and g are said orthogonal if < f , g >= 0A collection of functions ei are orthonormal if < ei, ej >= δij

Suppose E = {e1, . . . , en} a set of orthonormal functions basis.Then E generates a subspace of L2[a, b] and any function h inthis subspace can be written as (h lies in the span of E )

h =n

∑i=1

< h, ei > ei

Inner Product

Let f and g be functions defined on the square integrable spaceL2[a, b] (f : [a, b]→ C,

∫ ba |f (t)|

2dt < ∞).The dot product of f and g is

< f , g >=∫ b

af (t)g(t)dt

The functions f and g are said orthogonal if < f , g >= 0A collection of functions ei are orthonormal if < ei, ej >= δij

Suppose E = {e1, . . . , en} a set of orthonormal functions basis.Then E generates a subspace of L2[a, b] and any function h inthis subspace can be written as (h lies in the span of E )

h =n

∑i=1

< h, ei > ei

Fourier Basis

Consider the Taylor expansion of the exponential function

ex = 1 + x +x2

2!+

x3

3!+

x4

4!+ · · ·

making x = it (complex number)

eit = 1 + (it) +(it)2

2!+

(it)3

3!+

(it)4

4!+ · · ·

eit =(

1− t2

2!+

t4

4!+ · · ·

)+ i

(t− t3

3!+

t5

5!− · · ·

)eit = cos(t) + i sin(t)

Fourier Basis

Consider the Taylor expansion of the exponential function

ex = 1 + x +x2

2!+

x3

3!+

x4

4!+ · · ·

making x = it (complex number)

eit = 1 + (it) +(it)2

2!+

(it)3

3!+

(it)4

4!+ · · ·

eit =(

1− t2

2!+

t4

4!+ · · ·

)+ i

(t− t3

3!+

t5

5!− · · ·

)eit = cos(t) + i sin(t)

Fourier Basis

Consider the Taylor expansion of the exponential function

ex = 1 + x +x2

2!+

x3

3!+

x4

4!+ · · ·

making x = it (complex number)

eit = 1 + (it) +(it)2

2!+

(it)3

3!+

(it)4

4!+ · · ·

eit =(

1− t2

2!+

t4

4!+ · · ·

)+ i

(t− t3

3!+

t5

5!− · · ·

)

eit = cos(t) + i sin(t)

Fourier Basis

Consider the Taylor expansion of the exponential function

ex = 1 + x +x2

2!+

x3

3!+

x4

4!+ · · ·

making x = it (complex number)

eit = 1 + (it) +(it)2

2!+

(it)3

3!+

(it)4

4!+ · · ·

eit =(

1− t2

2!+

t4

4!+ · · ·

)+ i

(t− t3

3!+

t5

5!− · · ·

)eit = cos(t) + i sin(t)

Fourier Basis

real(e−ix) real(e−2(ix)) real(e−3(ix))

One-parameter family of (Fourier) basis:

e−iλx

Fourier Basis

real(e−ix) real(e−2(ix)) real(e−3(ix))

One-parameter family of (Fourier) basis:

e−iλx

Fourier Transform

F(λ) =∫ ∞

−∞f (t) e−(iλt)dt

Fourier Transform

Fourier Transform

F(λ) =∫ ∞

−∞f (t) e−(iλt)dt

Fourier Transform

f (x) =1√2π

∫ ∞

−∞F(λ)e(iλx)dλ

Inverse Fourier Transform

Fourier Transform

F(λ) =∫ ∞

−∞f (t) e−(iλt)dt

Fourier Transform

f (x) =1√2π

∫ ∞

−∞F(λ)e(iλx)dλ

Inverse Fourier Transform

We will adopt the notation

F [f ] = F F−1[F] = f

Fourier Transform

F(λ) =∫ ∞

−∞f (t) e−(iλt)dt

Fourier Transform

f (x) =1√2π

∫ ∞

−∞F(λ)e(iλx)dλ

Inverse Fourier Transform

We will adopt the notation

F [f ] = F F−1[F] = f

This notation makes clear that F : L2 → L2.

Fourier Transform

cos(2π(3x))e−πx2

Fourier Transform

f (x) = cos(2π(3x))e−πx2 F [f ]

Fourier Transform: Properties

F and its inverse are linear operators

F [bf + cg] = bF [f ] + cF [g]

Fourier transform of a translation

F [f (x− c)](λ) = e−iλcF [f ](λ)

Fourier transform of a rescaling

F [f (cx)](λ) =1c

e−iλcF [f ](λ

c)

Fourier Transform: Properties

F and its inverse are linear operators

F [bf + cg] = bF [f ] + cF [g]

Fourier transform of a translation

F [f (x− c)](λ) = e−iλcF [f ](λ)

Fourier transform of a rescaling

F [f (cx)](λ) =1c

e−iλcF [f ](λ

c)

Fourier Transform: Properties

F and its inverse are linear operators

F [bf + cg] = bF [f ] + cF [g]

Fourier transform of a translation

F [f (x− c)](λ) = e−iλcF [f ](λ)

Fourier transform of a rescaling

F [f (cx)](λ) =1c

e−iλcF [f ](λ

c)

Fourier Transform: Properties

F and its inverse are linear operators

F [bf + cg] = bF [f ] + cF [g]

Fourier transform of a translation

F [f (x− c)](λ) = e−iλcF [f ](λ)

Fourier transform of a rescaling

F [f (cx)](λ) =1c

e−iλcF [f ](λ

c)

Fourier Transform: Properties

The convolution of two functions f and g is defined as

(f ∗ g)(x) =∫ ∞

−∞f (x− t)g(t)dt

Fourier transform of a convolution

F [f ∗ g] =√

2πF [f ]F [g]

Fourier transform of a product

F [f · g] =√

2πF [f ] ∗ F [g]

Perseval’s equation

‖F [f ]‖ = ‖f‖

Fourier Transform: Properties

The convolution of two functions f and g is defined as

(f ∗ g)(x) =∫ ∞

−∞f (x− t)g(t)dt

Fourier transform of a convolution

F [f ∗ g] =√

2πF [f ]F [g]

Fourier transform of a product

F [f · g] =√

2πF [f ] ∗ F [g]

Perseval’s equation

‖F [f ]‖ = ‖f‖

Fourier Transform: Properties

The convolution of two functions f and g is defined as

(f ∗ g)(x) =∫ ∞

−∞f (x− t)g(t)dt

Fourier transform of a convolution

F [f ∗ g] =√

2πF [f ]F [g]

Fourier transform of a product

F [f · g] =√

2πF [f ] ∗ F [g]

Perseval’s equation

‖F [f ]‖ = ‖f‖

Fourier Transform: Properties

The convolution of two functions f and g is defined as

(f ∗ g)(x) =∫ ∞

−∞f (x− t)g(t)dt

Fourier transform of a convolution

F [f ∗ g] =√

2πF [f ]F [g]

Fourier transform of a product

F [f · g] =√

2πF [f ] ∗ F [g]

Perseval’s equation

‖F [f ]‖ = ‖f‖

Fourier Transform: Properties

Impossibility of Perfect Band-Limiting

A consequence of the Paley–Wiener theorem is that a signal f (t)has perfectly band-limited Fourier transform F [f ], that is,

|F [f ](λ)| = 0 for λ > B

if only if for any T ∈ (−∞, ∞), there exists a t0 < T sucht thatf (t0) 6= 0.

In other words, a signal has perfectly bandlimited spectrum ifand only if the signal persists for all time.

Fourier Transform: Properties

Impossibility of Perfect Band-Limiting

A consequence of the Paley–Wiener theorem is that a signal f (t)has perfectly band-limited Fourier transform F [f ], that is,

|F [f ](λ)| = 0 for λ > B

if only if for any T ∈ (−∞, ∞), there exists a t0 < T sucht thatf (t0) 6= 0.

In other words, a signal has perfectly bandlimited spectrum ifand only if the signal persists for all time.

Linear Filters

A filter Lmaps a signal f into another signal L[f ].

A filter is called linear if L[bf + cg[= bL[f ] + cL[g]

A filter is time invariant if L[f (x− c)](t) = L[f (x)](t− c)

TheoremLet L be a linear and time invariant filter.There exists a function h such that

L[f ] = f ∗ h

PS. h is called the impulse response function of the filter

F [L[f ]] = F [f ]F [h]

Linear Filters

A filter Lmaps a signal f into another signal L[f ].

A filter is called linear if L[bf + cg[= bL[f ] + cL[g]

A filter is time invariant if L[f (x− c)](t) = L[f (x)](t− c)

TheoremLet L be a linear and time invariant filter.There exists a function h such that

L[f ] = f ∗ h

PS. h is called the impulse response function of the filter

F [L[f ]] = F [f ]F [h]

Linear Filters

A filter Lmaps a signal f into another signal L[f ].

A filter is called linear if L[bf + cg[= bL[f ] + cL[g]

A filter is time invariant if L[f (x− c)](t) = L[f (x)](t− c)

TheoremLet L be a linear and time invariant filter.There exists a function h such that

L[f ] = f ∗ h

PS. h is called the impulse response function of the filter

F [L[f ]] = F [f ]F [h]

Linear Filters

A filter Lmaps a signal f into another signal L[f ].

A filter is called linear if L[bf + cg[= bL[f ] + cL[g]

A filter is time invariant if L[f (x− c)](t) = L[f (x)](t− c)

TheoremLet L be a linear and time invariant filter.There exists a function h such that

L[f ] = f ∗ h

PS. h is called the impulse response function of the filter

F [L[f ]] = F [f ]F [h]

Linear Filters

A filter Lmaps a signal f into another signal L[f ].

A filter is called linear if L[bf + cg[= bL[f ] + cL[g]

A filter is time invariant if L[f (x− c)](t) = L[f (x)](t− c)

TheoremLet L be a linear and time invariant filter.There exists a function h such that

L[f ] = f ∗ h

PS. h is called the impulse response function of the filter

F [L[f ]] = F [f ]F [h]

Linear Filters

L[f ] = f ∗ h → F [L[f ]] = F [f ] · F [h]

A filter can easily be understood in the frequency domain.High-, low-, and band-pass filters are easily designed.

Linear Filters

L[f ] = f ∗ h → F [L[f ]] = F [f ] · F [h]

A filter can easily be understood in the frequency domain.High-, low-, and band-pass filters are easily designed.