54
A Signal Processing Approach to the Detection of Pulmonary Edema EEE24B Nicole Lim Sze Ting Dr Ser Wee

A Signal Processing Approach to the Detection of Pulmonary ......Classification 4. Evaluation metrics Model Evaluation 5 TPR True positive rate TNR True negative rate ACC Detection

  • Upload
    others

  • View
    1

  • Download
    0

Embed Size (px)

Citation preview

Page 1: A Signal Processing Approach to the Detection of Pulmonary ......Classification 4. Evaluation metrics Model Evaluation 5 TPR True positive rate TNR True negative rate ACC Detection

A Signal Processing Approach to the Detection of Pulmonary Edema

EEE24B

Nicole Lim Sze Ting

Dr Ser Wee

Page 2: A Signal Processing Approach to the Detection of Pulmonary ......Classification 4. Evaluation metrics Model Evaluation 5 TPR True positive rate TNR True negative rate ACC Detection

Pulmonary Edemathe accumulation of excess fluid in the lungs

Page 3: A Signal Processing Approach to the Detection of Pulmonary ......Classification 4. Evaluation metrics Model Evaluation 5 TPR True positive rate TNR True negative rate ACC Detection

Limitations of current methods

Accuracy dependent on

clinician's experience

Costly, require bulky machines

Time-consuming

Exposure to radiation

Page 4: A Signal Processing Approach to the Detection of Pulmonary ......Classification 4. Evaluation metrics Model Evaluation 5 TPR True positive rate TNR True negative rate ACC Detection

Proposed method:

ML algorithm

Faster

More consistent and reliable

Portable;

Suitable for ambulatory use

Page 5: A Signal Processing Approach to the Detection of Pulmonary ......Classification 4. Evaluation metrics Model Evaluation 5 TPR True positive rate TNR True negative rate ACC Detection

ACC 95.7% with 5 features

via kNN (k = 3)

Previous studies

Yang, F., Ser, W., Yu, J., Foo, D., Yeo, D., Chia, P., & Wong, J. (n.d.). Lung Water Detection using Acoustic Techniques (Rep.).

Page 6: A Signal Processing Approach to the Detection of Pulmonary ......Classification 4. Evaluation metrics Model Evaluation 5 TPR True positive rate TNR True negative rate ACC Detection

AIM: Improve existing algorithm

Page 7: A Signal Processing Approach to the Detection of Pulmonary ......Classification 4. Evaluation metrics Model Evaluation 5 TPR True positive rate TNR True negative rate ACC Detection

Methodology

Page 8: A Signal Processing Approach to the Detection of Pulmonary ......Classification 4. Evaluation metrics Model Evaluation 5 TPR True positive rate TNR True negative rate ACC Detection

1

Data Collection & Pre-Processing

40s

Audio recordings of lung sounds

are labelled by a doctor.

Page 9: A Signal Processing Approach to the Detection of Pulmonary ......Classification 4. Evaluation metrics Model Evaluation 5 TPR True positive rate TNR True negative rate ACC Detection

1

Data Collection & Pre-Processing

Recordings are divided into 3

second samples.

40s

40s

40s

3s

Page 10: A Signal Processing Approach to the Detection of Pulmonary ......Classification 4. Evaluation metrics Model Evaluation 5 TPR True positive rate TNR True negative rate ACC Detection

1

Data Collection & Pre-Processing

40s

40s

3s

1024

x x x xx

x xx

x x xx

1024

x x x xx

x xx

x x xx

1024

x x x xx

x xx

x x xx

1024

x x x xx

x xx

x x xx

Samples are divided into windows

of 1024 points, with 50% overlap.

Page 11: A Signal Processing Approach to the Detection of Pulmonary ......Classification 4. Evaluation metrics Model Evaluation 5 TPR True positive rate TNR True negative rate ACC Detection

1

Data Collection & Pre-Processing

100

y

x

t

1

(t

1

, y

1

)

(t

1

, ✕ 100)

y

1

y

max

Windows are normalised such that

values range from 0 to 100.

Page 12: A Signal Processing Approach to the Detection of Pulmonary ......Classification 4. Evaluation metrics Model Evaluation 5 TPR True positive rate TNR True negative rate ACC Detection

3s

fnNo. of windows

=

Average feature value of windows

gives final feature value of sample.

1024

x x x xx

x xx

x x xxfn1024

x x x xx

x xx

x x xxfn1024

x x x xx

x xx

x x xxfn1024

x x x xx

x xx

x x xxfn

Feature Extraction

2

Page 13: A Signal Processing Approach to the Detection of Pulmonary ......Classification 4. Evaluation metrics Model Evaluation 5 TPR True positive rate TNR True negative rate ACC Detection

3

Feature Selection

(mean

A

- mean

B

)

2

variance

A

+ variance

B

Fisher’s ratio =

Fisher's Ratio

Higher FR Lower FR

Page 14: A Signal Processing Approach to the Detection of Pulmonary ......Classification 4. Evaluation metrics Model Evaluation 5 TPR True positive rate TNR True negative rate ACC Detection

Feature Extraction

2

Features used in the detection of wheezing

Features previously used in the detection of PE

Page 15: A Signal Processing Approach to the Detection of Pulmonary ......Classification 4. Evaluation metrics Model Evaluation 5 TPR True positive rate TNR True negative rate ACC Detection

Aydore, S., Sen, I., Kahya, Y., &

Mihcak, M. (n.d.). Classification of

Respiratory Signals by Linear

Analysis (Rep.).

Feature Extraction

2

Previous algorithms for the detection of wheezing

Page 16: A Signal Processing Approach to the Detection of Pulmonary ......Classification 4. Evaluation metrics Model Evaluation 5 TPR True positive rate TNR True negative rate ACC Detection

Feature Extraction

2

Features used in the detection of wheezingKurtosisDegree of peakedness of distribution

Renyi EntropyRandomness of system

Mean Crossing Irregularity & FrequencyMean-crossing behaviour

Page 17: A Signal Processing Approach to the Detection of Pulmonary ......Classification 4. Evaluation metrics Model Evaluation 5 TPR True positive rate TNR True negative rate ACC Detection

Feature Extraction

2

Fisher's Ratio of features used in the detection of wheezingKurtosisRenyi EntropyMCI

MCF

0.0010

0.0053

0.0086

0.0349

Page 18: A Signal Processing Approach to the Detection of Pulmonary ......Classification 4. Evaluation metrics Model Evaluation 5 TPR True positive rate TNR True negative rate ACC Detection

Feature Extraction

2

Features previously used in the detection of PE13 Mel-Frequency Cepstral Coefficients (MFCCs) Mimic doctor’s logarithmic

perception of lung sounds during

auscultation

Page 19: A Signal Processing Approach to the Detection of Pulmonary ......Classification 4. Evaluation metrics Model Evaluation 5 TPR True positive rate TNR True negative rate ACC Detection

Feature Extraction

2

Features previously used in the detection of PE13 Mel-Frequency Cepstral Coefficients (MFCCs) Mel scale:

Approximated

frequency resolution

of the human auditory

system

Mels

kHz

Page 20: A Signal Processing Approach to the Detection of Pulmonary ......Classification 4. Evaluation metrics Model Evaluation 5 TPR True positive rate TNR True negative rate ACC Detection

Feature Extraction

2

Features previously used in the detection of PE

Page 21: A Signal Processing Approach to the Detection of Pulmonary ......Classification 4. Evaluation metrics Model Evaluation 5 TPR True positive rate TNR True negative rate ACC Detection

Feature Extraction

2

Features previously used in the detection of PERatio & Difference between MFCCs● Hypothesised to have higher

discriminating power

● Derived from top 6 MFCC

Page 22: A Signal Processing Approach to the Detection of Pulmonary ......Classification 4. Evaluation metrics Model Evaluation 5 TPR True positive rate TNR True negative rate ACC Detection

Feature Extraction

2

Features previously used in the detection of PE

Page 23: A Signal Processing Approach to the Detection of Pulmonary ......Classification 4. Evaluation metrics Model Evaluation 5 TPR True positive rate TNR True negative rate ACC Detection

Using Fisher's Ratio in Feature Selection

3

Feature Selection

● Features with higher FR values

are added first

○ Reduce number of features

○ Higher classification accuracy

○ Shorten training time

Page 24: A Signal Processing Approach to the Detection of Pulmonary ......Classification 4. Evaluation metrics Model Evaluation 5 TPR True positive rate TNR True negative rate ACC Detection

3

Feature Selection

Final feature ranking by FR

Page 25: A Signal Processing Approach to the Detection of Pulmonary ......Classification 4. Evaluation metrics Model Evaluation 5 TPR True positive rate TNR True negative rate ACC Detection

Signal Classifiersk-Nearest Neighbours (kNN)Most common label amongst k

nearest neighbours

Support Vector Machines (SVM)Decision boundary/hyperplane

determined by support vectors

Signal Classification

4

Page 26: A Signal Processing Approach to the Detection of Pulmonary ......Classification 4. Evaluation metrics Model Evaluation 5 TPR True positive rate TNR True negative rate ACC Detection

Evaluation metrics

Model Evaluation

5

TPRTrue positive rate

TNRTrue negative rate

ACCDetection accuracy

10-fold cross validation

Page 27: A Signal Processing Approach to the Detection of Pulmonary ......Classification 4. Evaluation metrics Model Evaluation 5 TPR True positive rate TNR True negative rate ACC Detection

Evaluation metrics

Model Evaluation

5ACC

ACC

Healthy Unhealthy

Algorithm

Health

yU

nh

ealth

y

Do

cto

r

Page 28: A Signal Processing Approach to the Detection of Pulmonary ......Classification 4. Evaluation metrics Model Evaluation 5 TPR True positive rate TNR True negative rate ACC Detection

Evaluation metrics

Model Evaluation

5Missed

diagnosis TPR

Healthy Unhealthy

Algorithm

Health

yU

nh

ealth

y

Do

cto

r

Page 29: A Signal Processing Approach to the Detection of Pulmonary ......Classification 4. Evaluation metrics Model Evaluation 5 TPR True positive rate TNR True negative rate ACC Detection

Evaluation metrics

Model Evaluation

5

False alarmsTNR

Healthy Unhealthy

Algorithm

Health

yU

nh

ealth

y

Do

cto

r

Page 30: A Signal Processing Approach to the Detection of Pulmonary ......Classification 4. Evaluation metrics Model Evaluation 5 TPR True positive rate TNR True negative rate ACC Detection

Evaluation metrics

Model Evaluation

5

TPRTrue positive rate

TNRTrue negative rate

ACCDetection accuracy

10-fold cross validation

Page 31: A Signal Processing Approach to the Detection of Pulmonary ......Classification 4. Evaluation metrics Model Evaluation 5 TPR True positive rate TNR True negative rate ACC Detection

10-fold cross validation

Model Evaluation

5 training training training

training training training

training training training

training

testing

Page 32: A Signal Processing Approach to the Detection of Pulmonary ......Classification 4. Evaluation metrics Model Evaluation 5 TPR True positive rate TNR True negative rate ACC Detection

10-fold cross validation

Model Evaluation

5 training training training

training training training

training training training

training

testing testing testing

testing testing testing

testing testing testing

testing

Page 33: A Signal Processing Approach to the Detection of Pulmonary ......Classification 4. Evaluation metrics Model Evaluation 5 TPR True positive rate TNR True negative rate ACC Detection

Removing features that cause decrease in ACC

Performance Improvement

6

Page 34: A Signal Processing Approach to the Detection of Pulmonary ......Classification 4. Evaluation metrics Model Evaluation 5 TPR True positive rate TNR True negative rate ACC Detection

Removing features that cause decrease in ACC

Performance Improvement

6 1. Rank features by decrease in

ACC caused

2. Remove features from the

model in order of descending

decrease caused

Page 35: A Signal Processing Approach to the Detection of Pulmonary ......Classification 4. Evaluation metrics Model Evaluation 5 TPR True positive rate TNR True negative rate ACC Detection

Results & Discussion

Page 36: A Signal Processing Approach to the Detection of Pulmonary ......Classification 4. Evaluation metrics Model Evaluation 5 TPR True positive rate TNR True negative rate ACC Detection

Best Model:kNN

ACC

86

TPR

89

TNR

82

kNN (k = 1)on 24 features

Page 37: A Signal Processing Approach to the Detection of Pulmonary ......Classification 4. Evaluation metrics Model Evaluation 5 TPR True positive rate TNR True negative rate ACC Detection

Best Model:kNN

Effect of removing 6 features

TPR

89 88

ACC

86 85

TNR

82 80

24 18 24 18 24 18

Page 38: A Signal Processing Approach to the Detection of Pulmonary ......Classification 4. Evaluation metrics Model Evaluation 5 TPR True positive rate TNR True negative rate ACC Detection

Best Model:SVM

SVM (C = 1)on 26 features

ACC

86

TPR

88

TNR

82

Page 39: A Signal Processing Approach to the Detection of Pulmonary ......Classification 4. Evaluation metrics Model Evaluation 5 TPR True positive rate TNR True negative rate ACC Detection

Best Model:SVM

Effect of removing 9 features

TPR

88 88

ACC

86 86

TNR

82 83

26 17 26 17 26 17

Page 40: A Signal Processing Approach to the Detection of Pulmonary ......Classification 4. Evaluation metrics Model Evaluation 5 TPR True positive rate TNR True negative rate ACC Detection

Comparison to results of previous

works

kNN

ACC

85 85

13 18TPR

85 88

13 18TNR

85 80

13 18

YF ME

Page 41: A Signal Processing Approach to the Detection of Pulmonary ......Classification 4. Evaluation metrics Model Evaluation 5 TPR True positive rate TNR True negative rate ACC Detection

Comparison to results of previous

works

SVM

TPR

88 88

ACC

87 86

TNR

84 83

12 17 12 17 12 17

YF ME

Page 42: A Signal Processing Approach to the Detection of Pulmonary ......Classification 4. Evaluation metrics Model Evaluation 5 TPR True positive rate TNR True negative rate ACC Detection

Comparison to results of previous

works

● New features with higher FR

values via ratio/difference of

MFCC

● Did not improve ACC

Evaluation of my approach

Page 43: A Signal Processing Approach to the Detection of Pulmonary ......Classification 4. Evaluation metrics Model Evaluation 5 TPR True positive rate TNR True negative rate ACC Detection

Strategies hypothesised to improve ACC

Strategy Evaluation

Features used for the detection of other breathing anomalies

Derivation of features using ratio and difference

Removal of features that decrease ACC to improve performance

Page 44: A Signal Processing Approach to the Detection of Pulmonary ......Classification 4. Evaluation metrics Model Evaluation 5 TPR True positive rate TNR True negative rate ACC Detection

Strategies hypothesised to improve ACCFeatures used for the detection of other breathing anomalies ● Not useful in distinguishing

healthy and unhealthy signals

○ PE: crackle sounds

Strategy Evaluation

Page 45: A Signal Processing Approach to the Detection of Pulmonary ......Classification 4. Evaluation metrics Model Evaluation 5 TPR True positive rate TNR True negative rate ACC Detection

Strategies hypothesised to improve ACCDerivation of features using ratio and difference● Can derive features with higher

FR ⇒ ↑ discriminating power

● Time-consuming

○ Comparison of boxplots

Strategy Evaluation

Page 46: A Signal Processing Approach to the Detection of Pulmonary ......Classification 4. Evaluation metrics Model Evaluation 5 TPR True positive rate TNR True negative rate ACC Detection

Strategies hypothesised to improve ACC

Renyi entropy Randomness of system

Mean Crossing Irregularity (MCI) & Mean Crossing Frequency (MCF) Mean-crossing behaviour

Removal of features that decrease ACC to improve performance● ↓ training time

● ↓ algorithm complexity

● Can potentially improve ACC

○ SVM algorithm

Strategy Evaluation

Page 47: A Signal Processing Approach to the Detection of Pulmonary ......Classification 4. Evaluation metrics Model Evaluation 5 TPR True positive rate TNR True negative rate ACC Detection

Limitations

Uncertainty in performance evaluation due to validation method

Quality of data & reliability of doctor

Page 48: A Signal Processing Approach to the Detection of Pulmonary ......Classification 4. Evaluation metrics Model Evaluation 5 TPR True positive rate TNR True negative rate ACC Detection

Future Work

Automate derivation of ratio/ difference-based features

Vary number of MFCCs

Page 49: A Signal Processing Approach to the Detection of Pulmonary ......Classification 4. Evaluation metrics Model Evaluation 5 TPR True positive rate TNR True negative rate ACC Detection

Conclusion

Page 50: A Signal Processing Approach to the Detection of Pulmonary ......Classification 4. Evaluation metrics Model Evaluation 5 TPR True positive rate TNR True negative rate ACC Detection

Yang Feng's algorithm performed better

Proposed algorithm● SVM (C = 1) ● 17 features ● ACC 85.8, TPR 88, TNR 83

Key points

Page 51: A Signal Processing Approach to the Detection of Pulmonary ......Classification 4. Evaluation metrics Model Evaluation 5 TPR True positive rate TNR True negative rate ACC Detection

SVM > kNN

TPR

88

ACC

85

TNR

80

24 18 24 18 24 18

Key points

SVM kNN

8886 83

17 17 17

Page 52: A Signal Processing Approach to the Detection of Pulmonary ......Classification 4. Evaluation metrics Model Evaluation 5 TPR True positive rate TNR True negative rate ACC Detection

Wheeze detection features for pulmonary edema detection

Key points

Difference/ratio-based feature derivation for features with higher Fisher's ratio values

Removing features that decrease accuracy to improve performance

Page 53: A Signal Processing Approach to the Detection of Pulmonary ......Classification 4. Evaluation metrics Model Evaluation 5 TPR True positive rate TNR True negative rate ACC Detection

Dr Ser WeeResearch mentor

AcknowledgementsDr Shi Wen

Advice on MATLAB and data handling

Mr Low Kay SiangTeacher-mentor

Page 54: A Signal Processing Approach to the Detection of Pulmonary ......Classification 4. Evaluation metrics Model Evaluation 5 TPR True positive rate TNR True negative rate ACC Detection

Q&A