View
75
Download
0
Category
Preview:
DESCRIPTION
HUMAN AND SYSTEMS ENGINEERING:. Gentle Introduction to Particle Filtering. Sanjay Patil 1 and Ryan Irwin 2 Graduate research assistant 1 , REU undergrad 2 Human and Systems Engineering URL: www.isip.msstate.edu/publications/seminars/msstate/2005/particle /. Abstract. Particle Filtering: - PowerPoint PPT Presentation
Citation preview
Sanjay Patil1 and Ryan Irwin2
Graduate research assistant1,REU undergrad2
Human and Systems Engineering
URL: www.isip.msstate.edu/publications/seminars/msstate/2005/particle/
HUMAN AND SYSTEMS ENGINEERING:Gentle Introduction to Particle Filtering
Page 2 of 20Particle Filtering – Gentle Introduction and Implementation Demo
Abstract
Particle Filtering:
• Most conventional techniques for speech analysis are based on modeling signals as Gaussian Mixture Models in Hidden Markov Model based systems.
• To overcome the mismatched channel conditions, and/or significantly reduce
the complexity of the models, Nonlinear approaches are expected to perform better than the conventional techniques.
• Particle filters, based on sequential Monte Carlo methods, is one such nonlinear methods.
• Particle filtering allows complete presentation of the posterior distribution of the states. Statistical estimates can be computed easily even in the presence of nonlinearities.
Page 3 of 20Particle Filtering – Gentle Introduction and Implementation Demo
Outline of Presentation
• Nonlinear Methods – necessity
• Drawing Samples from a Probability distribution. (introduce ‘Particle’)
• Sequential Monte Carlo Methods – necessity, different names – bootstrap, condensation algorithm, survival of the fittest.
• Steps in particle filtering (explaining the algorithm – block schematic)
• Actual example – (along with all the steps)
• Brief review and applications for tracking
• As can be applied to Speaker Verification
• Demo
Page 4 of 20Particle Filtering – Gentle Introduction and Implementation Demo
0 2 4 6 8 10 12 14 16 18 200
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0 2 4 6 8 10 12 14 16 18 200
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0 5 10 15 20 250
0.05
0.1
0.15
0.2
0.25
5000 samples500 samples
200 samplesTake p(x)=Gamma(4,1)
Generate some random samples
Plot basic approximation to pdf
Each sample is called as ‘Particle’
Drawing samples from a probability distribution function
• Concept of samples and its weights
Page 5 of 20Particle Filtering – Gentle Introduction and Implementation Demo
Particle filtering -
Different Names –
• Sequential Monte Carlo filters
• Bootstrap filters
• Condensation Algorithm
• Survival of the fittest
General Problem Statement – Filtering – estimation of the states
• Tracking the state (parameters or hidden variables) as it evolves over time
• Sequentially arriving (noisy and non-Gaussian) observations
• Idea is to have best possible estimate of hidden variables
Page 6 of 20Particle Filtering – Gentle Introduction and Implementation Demo
Assume that pdf p(xk-1 | y1:k-1) is available at time k -1.
• Prediction stage:
This is the prior of the state at time k ( without the information on measurement). Thus, it is the probability of the the state given only the previous measurements
• Update stage:
This is posterior pdf from predicted prior pdf and newly available measurement.
Particle filtering algorithm continue…
)|(
)|()|()|(
1:1
1:1:1
kk
kkkkkk yyp
yxpyxpyxp
General two-stage Framework
(Prediction-Update stages)
11:111:1 )|()|()|( kkkkkkk dxyxpxxpyxp
Page 7 of 20Particle Filtering – Gentle Introduction and Implementation Demo
Particle filtering algorithm step-by-step (1)
Page 8 of 20Particle Filtering – Gentle Introduction and Implementation Demo
Particle filtering step-by-step (2)
Page 9 of 20Particle Filtering – Gentle Introduction and Implementation Demo
Particle filtering step-by-step (3)
Page 10 of 20Particle Filtering – Gentle Introduction and Implementation Demo
Particle filtering step-by-step (4)
Page 11 of 20Particle Filtering – Gentle Introduction and Implementation Demo
Particle filtering step-by-step (5)
Page 12 of 20Particle Filtering – Gentle Introduction and Implementation Demo
Particle filtering step-by-step (6)
Page 13 of 20Particle Filtering – Gentle Introduction and Implementation Demo
Particle filtering - visualization
Drawing samples
Predicting next state
Updating this state
What is THIS STEP???
Resampling….
Page 14 of 20Particle Filtering – Gentle Introduction and Implementation Demo
Sampling Importance Resample algorithm (necessity)
Page 15 of 20Particle Filtering – Gentle Introduction and Implementation Demo
Applications:
All the applications are mostly tracking applications in different forms….
Visual Tracking – e.g. human motion (body parts)
Prediction of (financial) time series – e.g. mapping gold price, stocks
Quality control in semiconductor industry
Military Applications
Target recognition from single or multiple images
Guidance of missiles
What is the application for IES NSF funded project –
Time series estimation for speech signal (Java demo)
Speaker Verification (details on next slide)
Page 16 of 20Particle Filtering – Gentle Introduction and Implementation Demo
Pattern Recognition Applet
• Java applet that gives a visual of algorithms implemented at IES
• Classification of Signals: • PCA - Principle Component Analysis
• LDA - Linear Discrimination Analysis
• SVM - Support Vector Machines
• RVM - Relevance Vector Machines
• Tracking of Signals • LP - Linear Prediction
• KF - Kalman Filtering
• PF – Particle Filtering
Page 17 of 20Particle Filtering – Gentle Introduction and Implementation Demo
Pattern Classification
• Different data sets need to be differentiated without looking at all the data samples
• Classifications distinguishes between sets of data without the samples
• Algorithms separate data sets with a line of discrimination
• To have zero error the line of discrimination should completely separate the classes
• These patterns are easy to classify
Page 18 of 20Particle Filtering – Gentle Introduction and Implementation Demo
Pattern Classification
• Toroidals are not classified very successfully with a straight line
• Error should be around 50% because half of each class is separated
• A proper line of discrimination of a toroidal would be a circle enclosing only the inside set
Page 19 of 20Particle Filtering – Gentle Introduction and Implementation Demo
Signal Tracking
• The input signals are now time based with the x-axis representing time
• All the signal tracking algorithms are implemented with interpolated data
• The interpolation ensures that the input samples are at regular intervals
• Sampling is always done on regular intervals
• The linear prediction algorithm is a linear way to predict signals with no noise
Page 20 of 20Particle Filtering – Gentle Introduction and Implementation Demo
Signal Tracking
• The Kalman filter and particle filter are based on prediction of the states of the signal
• States are related to the observations through the state equation
• The particle filtering algorithm introduces process and measurement noise
• At each iteration possible states are given by the black points
• The average of the black points is where the overall state is predicted to be
Page 21 of 20Particle Filtering – Gentle Introduction and Implementation Demo
References:
• S. Haykin and E. Moulines, "From Kalman to Particle Filters," IEEE International Conference on Acoustics, Speech, and Signal Processing, Philadelphia, Pennsylvania, USA, March 2005.
• M.W. Andrews, "Learning And Inference In Nonlinear State-Space Models," Gatsby Unit for Computational Neuroscience, University College, London, U.K., December 2004.
• P.M. Djuric, J.H. Kotecha, J. Zhang, Y. Huang, T. Ghirmai, M. Bugallo, and J. Miguez, "Particle Filtering," IEEE Magazine on Signal Processing, vol 20, no 5, pp. 19-38, September 2003.
• N. Arulampalam, S. Maskell, N. Gordan, and T. Clapp, "Tutorial On Particle Filters For Online Nonlinear/ Non-Gaussian Bayesian Tracking," IEEE Transactions on Signal Processing, vol. 50, no. 2, pp. 174-188, February 2002.
• R. van der Merve, N. de Freitas, A. Doucet, and E. Wan, "The Unscented Particle Filter," Technical Report CUED/F-INFENG/TR 380, Cambridge University Engineering Department, Cambridge University, U.K., August 2000.
• S. Gannot, and M. Moonen, "On The Application Of The Unscented Kalman Filter To Speech Processing," International Workshop on Acoustic Echo and Noise, Kyoto, Japan, pp 27-30, September 2003.
• J.P. Norton, and G.V. Veres, "Improvement Of The Particle Filter By Better Choice Of The Predicted Sample Set," 15th IFAC Triennial World Congress, Barcelona, Spain, July 2002.
• J. Vermaak, C. Andrieu, A. Doucet, and S.J. Godsill, "Particle Methods For Bayesian Modeling And Enhancement Of Speech Signals," IEEE Transaction on Speech and Audio Processing, vol 10, no. 3, pp 173-185, March 2002.
Recommended