Upload
others
View
11
Download
0
Embed Size (px)
Citation preview
GCT535: Sound Technology for Multimedia
Fundamentals of DSP - Part 1
Juhan Nam
Goals
โ Introducing digital audio effects
โ The fundamentals of digital signal processingโ Linear time-invariant (LTI) systemโ FIR / IIR Filtersโ Convolutionโ Impulse responseโ Frequency response
Introduction
โ Once we record or synthesize sound, we modify the acoustic characteristics using various digital audio effects โ Loudness: volume, compressor, limiter, noise gateโ Timbre: filters, equalizer, distortion, modulation, chorus, flanger, vocoderโ Pitch: pitch correction, harmonizer, vibratoโ Duration: time scale modification, resampling โ Spatial effect: panning, delay, reverberation, binaural (HRTF)
Input OutputDigital Audio Effects
Introduction
โ Filters and delaysโ Bi-quads filters: lowpass, bandpass, highpass, equalizersโ Comb filter (delayline): delay/echo, chorus, flanger, reverbโ Convolution: resampling, HRTF, reverberation (measured impulse responses)
โ Non-linear processingโ Modulation: amplitude, frequencyโ Distortion and dynamic range control: compressor, limiter, noise gate
โ Block-based processingโ Pitch shifting and Time-scale modification
Filters and Delays
โ Take the input signal ๐ฅ ๐ as a sequence of numbers and returns the output signal ๐ฆ ๐ as another sequence of numbers
โ Perform a combination of three mathematic operations upon the inputโ Multiplication: ๐ฆ ๐ = ๐ % ๐ฅ ๐โ Delay: ๐ฆ ๐ = ๐ฅ ๐ โ๐โ Summation: ๐ฆ ๐ = ๐ฅ ๐ + ๐ % ๐ฆ ๐ โ ๐
Input OutputFiltersor Delays
๐ฅ ๐ ๐ฆ ๐
Filters and Delays
โ Bi-quad filterโ Two delay elements for input or outputโ The delayed output (โfeedbackโ) causes resonanceโ Rooted from analog circuits (R-L-C)
๐ง!"
๐ฅ ๐ +
๐ง!" ๐"
๐#
๐$
๐ฅ ๐ โ 1
๐ฅ ๐ โ 2
๐ง!"
๐ฆ ๐
๐ง!"-๐"
-๐#
๐ฆ ๐ โ 1
๐ฆ ๐ โ 2
๐ฆ ๐ = ๐$ + ๐ฅ ๐ + ๐" + ๐ฅ ๐ โ 1 + ๐# + ๐ฅ ๐ โ 2 โ ๐" + ๐ฆ ๐ โ 1 โ ๐# + ๐ฆ ๐ โ 2
Filters and Delays
โ Comb filterโ Use a delayline โ Feedforward is used for chorus/flanger and feedback is for echo/reverbโ Rooted from magnetic tape recording
๐ฆ ๐ = ๐ฅ ๐ + ๐ + ๐ฅ ๐ โ ๐
๐ง!%
๐ฅ ๐ ๐ฆ ๐+
Feedforward Comb Filter
๐ฆ ๐ = ๐ฅ ๐ + ๐ + ๐ฆ(๐ โ ๐)
๐ง!&
๐ฅ ๐ ๐ฆ ๐+
๐
Feedback Comb Filter
๐
Filters and Delays
โ Convolution filterโ Conduct the convolution operation with an impulse response of a systemโ The impulse response is often measured from natural objects
โ HRTF: human ears โ Reverberation: rooms
๐ฆ ๐ = ๐$ + ๐ฅ ๐ + ๐" + ๐ฅ ๐ โ 1 + ๐# + ๐ฅ ๐ โ 2 +โฏ+ ๐% + ๐ฅ ๐ โ ๐
โ(๐)= [๐$, ๐", ๐#โฆ , ๐%]
Linear Time-Invariant Filters
โ They are called Linear Time-Invariant (LTI) filters in the context of digital signal processing
โ Linearityโ Scaling: if ๐ฅ ๐ โ ๐ฆ ๐ , then a % ๐ฅ ๐ โ a % ๐ฆ ๐โ Superposition: if ๐ฅ! ๐ โ ๐ฆ! ๐ and ๐ฅ" ๐ โ ๐ฆ" (n), then ๐ฅ! ๐ + ๐ฅ" ๐ โ
๐ฆ! ๐ + ๐ฆ" ๐
โ Time-Invarianceโ If ๐ฅ ๐ โ ๐ฆ ๐ , then ๐ฅ ๐ โ ๐ โ ๐ฆ ๐ โ ๐ for any ๐โ This means that the system does not change its behavior over time
Linear Time-Invariant Filters
โ Non-linear filtersโ ๐ฆ ๐ = ๐ฅ ๐ "
โ Time-variant filtersโ ๐ฆ ๐ = ๐ % ๐ฅ ๐ + ๐ฅ(๐ โ 1)
โ Modulation โ Amplitude or frequency modulation generates new sinusoidal components
โ Dynamic range compressionโ Input gain is a function of the input: ๐ฆ ๐ = ๐(๐ฅ ๐ ) % ๐ฅ ๐
Linear Time-Invariant System
โ Remember that sinusoids are eigenfunctions of linear systemโ The input sinusoids changes in amplitude and phase while preserving the
same frequencyโ No new sinusoidal components are created
LinearSystem
๐#$%๐ด(๐)๐#&($)๐#$%
Amplitude Response
Phase Response
The Simplest Lowpass Filter
โ Difference equation: ๐ฆ ๐ = ๐ฅ ๐ + ๐ฅ(๐ โ 1)
โ Signal flow graph
๐ง!"
๐ฅ ๐ ๐ฆ ๐+
โDelay Operatorโ
The Simplest Lowpass Filter: Sine-Wave Analysis
โ Measure the amplitude and phase changes given a sinusoidal signal input
The Simplest Lowpass Filter: Frequency Response
โ Plot the amplitude and phase change over different frequencyโ The frequency sweeps from 0 to the Nyquist rate
The Simplest Lowpass Filter: Frequency Response
โ Mathematical approachโ Use complex sinusoid as input: ๐ฅ ๐ = ๐#$)
โ Then, the output is: ๐ฆ ๐ = ๐ฅ ๐ + ๐ฅ ๐ โ 1 = ๐#$) + ๐#$()*!) = 1 + ๐*#$ % ๐#$)
= 1 + ๐*#$ % ๐ฅ(๐)
โ Frequency response: ๐ป ๐ = 1 + ๐*#$ = ๐#!" + ๐*#
!" ๐*#
!" = 2cos($
")๐*#
!"
โ Amplitude response: ๐ป(๐) = 2 cos $"
โ Phase response: โ ๐ป ๐ = โ$"
The Simplest Highpass Filter
โ Difference equation: ๐ฆ ๐ = ๐ฅ ๐ โ ๐ฅ(๐ โ 1)
โ Signal flow graph
๐ง!"
๐ฅ ๐ +
โ1
๐ฆ ๐
The Simplest Highpass Filter: Frequency Response
โ Plot the amplitude and phase change over different frequencyโ The frequency sweeps from 0 to the Nyquist rate
Finite Impulse Response (FIR) System
โ Difference equation๐ฆ ๐ = ๐+ % ๐ฅ ๐ + ๐! % ๐ฅ ๐ โ 1 + ๐" % ๐ฅ ๐ โ 2 +โฏ+ ๐, % ๐ฅ ๐ โ ๐
โ Signal flow graph
๐ง!"
๐ฅ ๐ +
๐ง!"
๐ง!"
...
๐"
๐#
๐$
๐%
๐ฅ ๐ โ 1
๐ฅ ๐ โ 2
๐ฅ ๐ โ๐
๐ฆ ๐
Impulse Response
โ The system output when the input is a unit impulseโ ๐ฅ ๐ = ๐ฟ ๐ = 1, 0, 0, 0, โฆ โ ๐ฆ ๐ = โ(๐)= [๐+, ๐!, ๐"โฆ , ๐,] (for FIR
system)
โ Characterizes the digital system as a sequence of numbersโ A system is represented just like audio samples!
๐ฅ ๐ = ๐ฟ ๐ ๐ฆ ๐ = โ(๐)โ ๐
Input OutputLTISystem
Examples: Impulse Response
โ The simplest lowpass filterโ h ๐ = 1, 1
โ The simplest highpass filterโ h ๐ = 1,โ1
โ Moving-average filter (order=5)
โ h ๐ = !-, !-, !-, !-, !-
Convolution
โ The output of LTI digital filters is represented by the convolution operation between ๐ฅ ๐ and โ ๐
โ Examplesโ The simplest lowpass filter โ ๐ฆ ๐ = 1, 1 โ ๐ฅ ๐ = 1 % ๐ฅ(๐) + 1 % ๐ฅ(๐ โ 1) = ๐ฅ ๐ + ๐ฅ(๐ โ 1)
๐ฆ ๐ = ๐ฅ ๐ โ โ ๐ = :'(!)
)
๐ฅ ๐ + โ ๐ โ ๐ = :'(!)
)
โ(๐) + ๐ฅ(๐ โ ๐)
This is more practical expression when the input is an audio streaming
Proof: Convolution
โ Method 1โ The input is represented as the sum of weighted and delayed impulses units
โ ๐ฅ ๐ = ๐ฅ$, ๐ฅ", ๐ฅ#โฆ , ๐ฅ% = ๐ฅ$ + ๐ฟ ๐ + ๐ฅ" + ๐ฟ ๐ โ 1 + ๐ฅ# + ๐ฟ ๐ โ 2 +โฏ+ ๐ฅ% +๐ฟ ๐ โ ๐
โ By the linearity and time-invariance โ ๐ฆ ๐ = ๐ฅ$ + โ ๐ + ๐ฅ" + โ ๐ โ 1 + ๐ฅ# + โ ๐ โ 2 +โฏ+ ๐ฅ% + โ ๐ โ ๐ =
โ'($% ๐ฅ(๐) + โ(๐ โ ๐)
Proof: Convolution
โ Method 2โ The impulse response can be represented as a set of weighted impulses
โ โ ๐ = ๐$, ๐", ๐#โฆ , ๐% = ๐$ + ๐ฟ ๐ + ๐" + ๐ฟ ๐ โ 1 + ๐# + ๐ฟ ๐ โ 2 +โฏ+ ๐% +๐ฟ ๐ โ ๐
โ By the linearity, the distributive property and ๐ฅ ๐ โ ๐ฟ ๐ โ ๐ = ๐ฅ(๐ โ ๐)โ ๐ฆ ๐ = ๐$ + ๐ฅ ๐ + ๐" + ๐ฅ ๐ โ 1 + ๐# + ๐ฅ ๐ โ 2 +โฏ+ ๐% + ๐ฅ ๐ โ ๐ =
โ'($% โ(๐) + ๐ฅ(๐ โ ๐)
Properties of Convolution
โ Commutative: ๐ฅ ๐ โ โ" ๐ โ โ# ๐ = ๐ฅ ๐ โ โ# ๐ โ โ" ๐
โ Associative: (๐ฅ ๐ โ โ" ๐ ) โ โ# ๐ = ๐ฅ ๐ โ (โ" ๐ โ โ# ๐ )
โ Distributive: ๐ฅ ๐ โ โ" ๐ + โ# ๐ = ๐ฅ ๐ โ โ" ๐ + ๐ฅ ๐ โ โ# ๐
Example: Convolution
โ Given ๐ฅ ๐ = ๐ฅ$, ๐ฅ", ๐ฅ#, โฆ , ๐ฅ% and โ ๐ = โ$, โ", โ#โ ๐ฆ ๐ = โ./+, โ(๐) % ๐ฅ(๐ โ ๐)โ ๐ฆ 0 = โ(0) 0 ๐ฅ(0)โ ๐ฆ 1 = โ 0 0 ๐ฅ 1 + โ(1) 0 ๐ฅ(0)โ ๐ฆ 2 = โ 0 0 ๐ฅ 2 + โ 1 0 ๐ฅ 1 + โ 2 0 ๐ฅ 0โ ๐ฆ 3 = โ 0 0 ๐ฅ 3 + โ 1 0 ๐ฅ 2 + โ 2 0 ๐ฅ 1โ ๐ฆ 4 = โ 0 0 ๐ฅ 4 + โ 1 0 ๐ฅ 3 + โ 2 0 ๐ฅ 2โ ๐ฆ 5 = โ 0 0 ๐ฅ 5 + โ 1 0 ๐ฅ 4 + โ 2 0 ๐ฅ 3โ ๐ฆ 6 = โ 1 0 ๐ฅ 5 + โ 2 0 ๐ฅ 4โ ๐ฆ 7 = โ 2 0 ๐ฅ 5
โ The size of transient region is equal to the number of delay operators
โ If the length of ๐ฅ ๐ is ๐ and the length of โ ๐ is ๐, then the length of ๐ฆ ๐ is ๐ +๐ โ 1.
Transient state
Steady state
Transient state
Demo: Convolution
Example: Convolution Reverb
โ Convolution Reverb
Room Impulse Response
Lobby
Church
A Simple Feedback Lowpass Filter
โ Difference equation: ๐ฆ ๐ = ๐ฅ ๐ + ๐ 3 ๐ฆ(๐ โ 1)
โ Signal flow graphโ When ๐ is slightly less than 1, it is called โLeaky Integratorโ
๐ง!"
๐ฅ ๐ ๐ฆ ๐+
๐
A Simple Feedback Lowpass Filter: Impulse Response
โ Impulse response: exponential decaysโ ๐ฆ 0 = ๐ฅ 0 = 1โ ๐ฆ 1 = ๐ฅ 1 + ๐ % ๐ฆ 0 = ๐โ ๐ฆ 2 = ๐ฅ 2 + ๐ % ๐ฆ 1 = ๐"
โ ๐ฆ 3 = ๐ฅ 3 + ๐ % ๐ฆ 2 = ๐"
โ โฆ
โ Stability Issue! โ If ๐ < 1, the filter output converges (stable)โ If ๐ = 1, the filter output oscillates (critical)โ If ๐ > 1, the filter output diverges (unstable)
โ(๐)=[1, ๐, ๐#, ๐*, โฆ ]
A Simple Feedback Lowpass Filter: Sine-Wave Analysis
โ Measure the amplitude and phase changes given a sinusoidal signal input
๐ = 0.9
Transient state Steady state Transient state
A Simple Feedback Lowpass Filter: Frequency Response
โ More dramatic change than the simplest lowpass filter (FIR)โ Phase response is not linear
๐ฆ ๐ = ๐ฅ ๐ + 0.9 0 ๐ฆ(๐ โ 1)
A Simple Feedback Lowpass Filter: Frequency Response
โ Mathematical approachโ Use complex sinusoid as input: ๐ฅ ๐ = ๐#$)
โ ๐ฆ ๐ = ๐บ ๐ ๐#($)01 $ ) ร ๐ฆ ๐ โ ๐ = ๐*#$2๐ฆ ๐ for any ๐โ The output is: ๐ฆ ๐ = ๐ฅ ๐ + ๐ % ๐ฆ(๐ โ 1)
๐ฆ ๐ = ๐ฅ ๐ + ๐ % ๐#$๐ฆ ๐
โ Frequency response: ๐ป ๐ = !!*345#$!
= !!*34678 $ 034#489: $
โ Amplitude response: ๐ป(๐) = !(!*34678 $ )"0(3489: $ )"
โ Phase response: โ ๐ป ๐ = โatan( 3489: $!*3 4678 $
)
โ Note that the this approach is getting complicated
Reson Filter
โ Difference equationโ ๐ฆ ๐ = ๐ฅ ๐ + 2๐ % cos(๐) % ๐ฆ ๐ โ 1 โ ๐" % ๐ฆ ๐ โ 2
โ Signal flow graph
๐ง!"
๐ฅ ๐ ๐ฆ ๐+
2๐ 0 cos๐+
๐ง!"โ๐#
+
Reson Filter
โ Mathematical approachโ Use complex sinusoid as input: ๐ฅ ๐ = ๐#$)
โ ๐ฆ ๐ = ๐บ ๐ ๐#($)01 $ ) ร ๐ฆ ๐ โ ๐ = ๐*#2$๐ฆ ๐ for any ๐โ The output is: ๐ฆ ๐ = ๐ฅ ๐ + 2๐ % cos(๐) % ๐ฆ ๐ โ 1 โ ๐" % ๐ฆ ๐ โ 2
๐ฆ ๐ = ๐ฅ ๐ + 2๐ % cos(๐) % ๐#$๐ฆ ๐ โ ๐" % ๐#"$๐ฆ ๐ โ 2โ Frequency response
โ ๐ป ๐ = !!*";4678(1)45$!0;"45$"!
= !(!*;(678 1 0#489: 1 )5$!)(!*;(678 1 *#489: 1 )5$!)
โ Now you see that the this approach is getting even more complicated โ We will introduce more intuitive method to obtain the frequency response
Reson Filter: Frequency Response
โ Generate resonance at a particular frequency โ Control the peak height by ๐ and the peak frequency by ๐
For stability: ๐ < 1
๐ < 1 is getting close to 1
๐ controls the location of the peak
Infinite Impulse Response (IIR) Filters
โ Difference equationโ ๐ฆ ๐ = ๐+ % ๐ฅ ๐ โ ๐! % ๐ฆ ๐ โ 1 โ ๐" % ๐ฆ ๐ โ 2 โโฏโ ๐< % ๐ฆ ๐ โ ๐
โ Signal flow graph๐ฅ ๐ +
๐%๐ง!"
๐ฆ ๐
๐ง!"
๐ง!"
...
-๐"
-๐#
-๐&
๐ฆ ๐ โ 1
๐ฆ ๐ โ 2
๐ฆ ๐ โ ๐
General Form of LTI Filters
โ Difference equationโ ๐ฆ ๐ = ๐+ % ๐ฅ ๐ + ๐! % ๐ฅ ๐ โ 1 + ๐" % ๐ฅ ๐ โ 2 + โฆ+ ๐, % ๐ฅ ๐ โ ๐
โ๐! % ๐ฆ ๐ โ 1 โ ๐" % ๐ฆ ๐ โ 2 โโฏโ ๐< % ๐ฆ ๐ โ ๐
โ Signal flow graph
๐ง!"
๐ฅ ๐ +
๐ง!"
๐ง!"
...
๐"
๐#
๐$
๐%
๐ฅ ๐ โ 1
๐ฅ ๐ โ 2
๐ฅ ๐ โ๐
๐ง!"
๐ฆ ๐
๐ง!"
๐ง!"
...
-๐"
-๐#
-๐&
๐ฆ ๐ โ 1
๐ฆ ๐ โ 2
๐ฆ ๐ โ ๐