View
30
Download
0
Category
Preview:
DESCRIPTION
Method of Packet Errors Cancellation Suitable for any Speech and Sound Compression Scheme STQ Workshop, Sophia-Antipolis, February 11 th 2003. Balazs KÖVESI, Dominique MASSALOUX. Introduction. Context Application like VoIP or audio streaming Possible high packet loss rate (up to 10 %) - PowerPoint PPT Presentation
Citation preview
Present document contains informations proprietary to France Telecom. Accepting this document means for its recipient he or she recognizes the confidential nature of its content and his or her engagement not to reproduce it, not to transmit it to a third party, not to reveal its content and not to use it for commercial purposes without previous FTR&D written consent.
Method of Packet Errors Cancellation Suitable for any Speech and Sound Compression Scheme
STQ Workshop, Sophia-Antipolis, February 11th 2003
Balazs KÖVESI, Dominique MASSALOUX
France Telecom R&D
Diffusion of this document is subject to France Telecom authorization D2
Introduction
s ContextQApplication like VoIP or audio streaming
–Possible high packet loss rate (up to 10 %)
s Proposition of a frame error concealment (FEC) methodQCopes with high packet loss rateQRelies on CELP synthesis schemeQIndependent from the codec typeQSpeech oriented but also suitable for musicQIncludes adaptive gain controlQAvoids "robot" voiceQEnsures the decoder memory updateQSmoothing after an erased period
France Telecom R&D
Diffusion of this document is subject to France Telecom authorization D3
Plan
s Basic principle of the new FEC method
s Implementation in a MDCT codec
s Generalization to other codec types
s Conclusion
France Telecom R&D
Diffusion of this document is subject to France Telecom authorization D4
Basic principle of the new FEC
Decoded signaldecoder
Valid data
Indication of erased data
Storage of decoded samples
Synthesis of missing samples
Decoder update
reconstructed signal
Smoothing with
decoded signal
France Telecom R&D
Diffusion of this document is subject to France Telecom authorization D5
s The MDCT transformQAnalysis with 50 % overlap
Q Synthesis with overlap-add
Implementation in a MDCT codec
memory for the next frame overlap-add
-windowing-TT FF transform
-FF TT transform-windowing-overlap-add
Analysis window n-1
20 ms
Analysis window n
t
Synthesis window n-1
decoded frame n
20 mst
Synthesis window n
France Telecom R&D
Diffusion of this document is subject to France Telecom authorization D6
s Effect of frame erasure
Qthe loose of x bitstream frames affects x+1 output frames Qthese frames have to be synthesized in the decoder
Implementation in a MDCT codec
Disturbed: frames n-1 & n 20 ms
Erased: frame n
t
France Telecom R&D
Diffusion of this document is subject to France Telecom authorization D7
s Memorizing part
QAfter decoding a valid frame
–The 40 ms output memory is updated–The energy of the frame is calculated–The energy memory buffer is updated ( 5 s )
Implementation in a MDCT codec
Valid dataDecoded signaldecoder
Erased data indication
Storage of decoded samples
Synthesis of missing samples
Decoder update
synthesized signal
Smoothing with
decoded signal
France Telecom R&D
Diffusion of this document is subject to France Telecom authorization D8
Implementation in a MDCT codec
Valid dataDecoded signaldecoder
Erased data indication
Storage of decoded samples
Synthesis of missing samples
Decoder update
synthesized signal
Smoothing with
decoded signal
Synthe-sized signal
LTP parameters (B(z))
LPC parameters (A(z))
LPCanalysis
calc. past excitation
signal
LTP filtering1/B(z)
LPC synthesi
s1/A(z)
Adaptive gain control
Memory of past decoded signal
LTP analysis & V/UV detection
France Telecom R&D
Diffusion of this document is subject to France Telecom authorization D9
Implementation in a MDCT codec
Valid dataDecoded signaldecoder
Erased data indication
Storage of decoded samples
Synthesis of missing samples
Decoder update
synthesized signal
Smoothing with
decoded signal
Synthe-sized signal
LTP parameters (B(z))
LPC parameters (A(z))
LPCanalysi
s
LTP analysis & V/UV detection
calc. past excitation
signal
LTP filtering1/B(z)
LPC synthesi
s1/A(z)
Adaptive gain control
Memory of past decoded signal
QLPC filter modelizes the spectral envelope
QCoefficients not transmitted –LPC analysis order can be higher than in a usual CELP (32 @
16kHZ) better performance on music
past decoded signalLPC coefficientsfilter A(z)
classical method
20 ms
France Telecom R&D
Diffusion of this document is subject to France Telecom authorization D10
Implementation in a MDCT codec
Valid dataDecoded signaldecoder
Erased data indication
Storage of decoded samples
Synthesis of missing samples
Decoder update
synthesized signal
Smoothing with
decoded signal
Synthe-sized signal
LTP parameters (B(z))
LPC parameters (A(z))
LPCanalysis
LTP analysis & V/UV
detection
calc. past excitation
signal
LTP filtering1/B(z)
LPC synthesi
s1/A(z)
Adaptive gain control
Memory of past decoded signal
QPrecise pitch estimation is crucial for the good performanceQOnly integer pitch (P) values are examined [50 Hz, 600 Hz]QNormalized correlations on the last 2P samplesQPitch criteria: maximum correlation +
multiple & fractional verificationsQV/UV criteria: selected correlation value +
energy value 5 s energy memory energy evolution of the last two pitch
periods
past decoded signal
p1
p2
p3
p4
correlationcalculations
France Telecom R&D
Diffusion of this document is subject to France Telecom authorization D11
Implementation in a MDCT codec
Valid dataDecoded signaldecoder
Erased data indication
Storage of decoded samples
Synthesis of missing samples
Decoder update
synthesized signal
Smoothing with
decoded signal
Synthe-sized signal
LTP parameters (B(z))
LPC parameters (A(z))
LPCanalysis
LTP analysis & V/UV detection
calc. past
excitation signal
LTP filtering1/B(z)
LPC synthesi
s1/A(z)
Adaptive gain control
Memory of past decoded signal
QLPC analysis filtering
past decoded signalA(z)
past excitation signal
France Telecom R&D
Diffusion of this document is subject to France Telecom authorization D12
Implementation in a MDCT codec
Valid dataDecoded signaldecoder
Erased data indication
Storage of decoded samples
Synthesis of missing samples
Decoder update
synthesized signal
Smoothing with
decoded signal
Synthe-sized signal
LTP parameters (B(z))
LPC parameters (A(z))
LPCanalysis
LTP analysis & V/UV detection
calc. past excitation
signal
LTP filterin
g1/B(z)
LPC synthesi
s1/A(z)
Adaptive gain control
Memory of past decoded signal
QExcitation signal generation for the LPC synthesis filteringQvoiced excitation:
–2 components–harmonic, lower frequency bands
LTP filter combined with a low pass filter–less harmonic, higher frequency bands
LTP filter combined with a high pass filter + randomly evolving pitch
France Telecom R&D
Diffusion of this document is subject to France Telecom authorization D13
QExcitation signal generation for the LPC synthesis filteringQvoiced excitation:
–2 components–harmonic, lower frequency bands
LTP filter combined with a low pass filter–less harmonic, higher frequency bands
LTP filter combined with a high pass filter + randomly evolving pitch
Implementation in a MDCT codec
Valid dataDecoded signaldecoder
Erased data indication
Storage of decoded samples
Synthesis of missing samples
Decoder update
synthesized signal
Smoothing with
decoded signal
Synthe-sized signal
LTP parameters (B(z))
LPC parameters (A(z))
LPCanalysis
LTP analysis & V/UV detection
calc. past excitation
signal
LTP filterin
g1/B(z)
LPC synthesi
s1/A(z)
Adaptive gain control
Memory of past decoded signal
Qunvoiced excitation–non harmonic, lower frequency bands–“randomized” LTP filtering + low pass filtering
+ sudden energy variations are suppressed
France Telecom R&D
Diffusion of this document is subject to France Telecom authorization D14
Implementation in a MDCT codec
Valid dataDecoded signaldecoder
Erased data indication
Storage of decoded samples
Synthesis of missing samples
Decoder update
synthesized signal
Smoothing with
decoded signal
Synthe-sized signal
LTP parameters (B(z))
LPC parameters (A(z))
LPCanalysis
LTP analysis & V/UV detection
calc. past excitation
signal
LTP filtering1/B(z)
LPC synthes
is 1/A(z)
Adaptive gain control
Memory of past decoded signal
QLPC synthesis filtering
excitation signal synthesized signal
A(z)1
France Telecom R&D
Diffusion of this document is subject to France Telecom authorization D15
Implementation in a MDCT codec
Valid dataDecoded signaldecoder
Erased data indication
Storage of decoded samples
Synthesis of missing samples
Decoder update
synthesized signal
Smoothing with
decoded signal
Synthe-sized signal
LTP parameters (B(z))
LPC parameters (A(z))
LPCanalysis
LTP analysis & V/UV detection
calc. past excitation
signal
LTP filtering1/B(z)
LPC synthesi
s1/A(z)
Adaptive gain control
Memory of past decoded signal
QImportant in case of long erased periods (> 20 ms)Q2 adaptation laws:
–stationary
–non-stationary
–The adaptations also depend on the pitch value Qdecision available from the LTP analysis
background noise level
40 ms t
t
France Telecom R&D
Diffusion of this document is subject to France Telecom authorization D16
s Recoverable information
Implementation in a MDCT codec
Synthesized frames n-1, n, n+1 20 ms
Erased: frame n-1 & n
t
decoded frame n+2
France Telecom R&D
Diffusion of this document is subject to France Telecom authorization D17
s Recoverable informationQfor the first erased frame
Implementation in a MDCT codec
MDCT transform on the first 2 synthesized frames
t
Partly recovered frame n-1IMDCT transform
France Telecom R&D
Diffusion of this document is subject to France Telecom authorization D18
s Decoder memory update
Implementation in a MDCT codec
t
Valid dataDecoded signaldecoder
Erased data indication
Storage of decoded samples
Synthesis of missing samples
Decoder update
synthesized signal
Smoothing with
decoded signal
MDCT transform on the last 2 synthesized frames
IMDCT memory to update
IMDCT transform(FT + windowing)
updated IMDCT memory
France Telecom R&D
Diffusion of this document is subject to France Telecom authorization D19
s Recoverable informationQfor the last erased frame
Implementation in a MDCT codec
t
Partly recovered frame n+1IMDCT transform
France Telecom R&D
Diffusion of this document is subject to France Telecom authorization D20
s Smoothing part Qwithout smoothing
Implementation in a MDCT codec
t
Valid dataDecoded signaldecoder
Erased data indication
Storage of decoded samples
Synthesis of missing samples
Decoder update
synthesized signal
Smoothing with
decoded signal
Synthesized frames n-1, n, n+1 decoded frame n+2
discontinuity
synthesized domain
error-free domain
France Telecom R&D
Diffusion of this document is subject to France Telecom authorization D21
s Smoothing part QA codec independent solution:
Implementation in a MDCT codec
t
Valid dataDecoded signaldecoder
Erased data indication
Storage of decoded samples
Synthesis of missing samples
Decoder update
synthesized signal
Smoothing with
decoded signal
Synthesized frames n-1, n, n+1 decoded frame n+2
synthesized domain
error-free domain
Extra synthesized samples
crossfading 1
0
France Telecom R&D
Diffusion of this document is subject to France Telecom authorization D22
s Smoothing part Qwith MDCT smoothing
Implementation in a MDCT codec
t
smooth transition at frame n+1overlap-addlike crossfading
Valid dataDecoded signaldecoder
Erased data indication
Storage of decoded samples
Synthesis of missing samples
Decoder update
synthesized signal
Smoothing with
decoded signal
synthesized domain
error-free domain
France Telecom R&D
Diffusion of this document is subject to France Telecom authorization D23
s can be adapted to any coding schemeQwas successfully implemented in
–temporal codecs (G.711, G.721, G722)–in a CELP codec (G.723.1) –in a hierarchical codec composed of a CELP and a transform codec
QMemorizing and synthesis part are codec independentQDecoder memory update
–very important for recursive codecs (CELP)–general solution: coding – decoding on the synthesized frames
–too complex for CELP–less complex solution: backtracking
QSmoothing –a general solution: crossfading–more efficient smoothing can be found for some coding schemes
(ex.: MDCT)–the decoder memory update ensure the smoothing in CELP codecs
Generalization to other codec types
Valid dataDecoded signaldecoder
Erased data indication
Storage of decoded samples
Synthesis of missing samples
Decoder update
synthesized signal
Smoothing with decoded
signal
France Telecom R&D
Diffusion of this document is subject to France Telecom authorization D24
s A general FEC method for any coding schemeQoptimal for speech voice, good performances on music Qavoids too synthetic sound for voiced framesQkeeps the nature of the unvoiced frames Qenhanced energy managementQcareful update of the decoder memoryQsmoothing after an erased period
s Informal subjective tests have shown its good behavior
s Successfully implemented in group communication applications
s Perspectives:Qspeech / music decision + enhanced music modeQ…
Conclusion
Recommended