Classifiers Optimization Using Swarm Algorithms

Faculty of Computers and Information, Minia University and SRGE member

Moataz Kilany

http://www.egyptscience.net

Bio-inspiring and evolutionary computation: Trends, applications and open issues workshop, 7 Nov. 2015 Faculty of Computers and Information, Cairo University

Overview

Introduction Classification Algorithms. Classifiers Optimization.

History. SVM Parameters optimization using WWO.

SVM Application on human activity accelerometer-based data.

SVM Optimization results.SRGE Workshop, Cairo University (07-November-2015)

2

Introduction3

Classification Algorithms.Text Classification.Multimedia.Time series and sequence data

classification. Classification Optimization.

SRGE Workshop, Cairo University (07-November-2015)

Classification Algorithms

4

Well-known algorithms. K-Nearest Neighbor. Decision Trees. SVM.

Target High accuracy with high generalization. Over fitting / under fitting (Generalization). Cross Validation.

Classifier Parameters)Penalty and gamma parameters SVM(.

SRGE Workshop, Cairo University (07-November-2015)

Classifiers Optimization

5

Optimization algorithms has been employed to optimize classifiers results.Meta-heuristic swarm-based algorithms.Genetic algorithms.Particle Swarm Optimization.Other Swarms algorithms.

SRGE Workshop, Cairo University (07-November-2015)

Classifiers Optimization

Efforts.6

GA applied for tuning classifiers parameters (Accuracy and Distortion), by J. J. Merelo et.al. Tested on K-means, Genetic VQ, Genetic K-means, LVQ,

G-LVQ. GA Applied by Ha-Nam Nguyen et. al. to optimize

Kernel function for SVM classifier. Michael R. Peterson et. al. applied GA algorithm to

optimize KNN classifier. PSO applied by many researchers for optimization.

P.J. Garcia Nieto et. al. employed PSO to optimize kernel parameter setting of SVM.

SRGE Workshop, Cairo University (07-November-2015)

7

Introductiono Water Wave Optimization Algorithm (WWO).

o Meta-heuristic algorithm.o Water waves metaphor.o Simplicity and speed.

o Support Vector Machine (SVM).oKernel function parameters.

o Penalty (C) parameter.o Gamma parameter.

Optimizing accelerometer-based data classification using WWO

SRGE Workshop, Cairo University (07-November-2015)

8

Optimizing accelerometer-based data classification using WWO

SRGE Workshop, Cairo University (07-November-2015)

oPenalty Parameter (C)

9

Optimizing accelerometer-based data classification using WWO

SRGE Workshop, Cairo University (07-November-2015)

oGamma Parameter.

10

Optimizing accelerometer-based data classification using WWO

SRGE Workshop, Cairo University (07-November-2015)

oGamma Parameter.

Optimizing accelerometer-based data classification using WWO

11

Human Activity Data Set.Accelerometer Setting.

SRGE Workshop, Cairo University (07-November-2015)

Optimizing accelerometer-based data classification using WWO

12

Human Activity Data Set.Data Set Specifications.

Data set components.Data set size.Sampling frequency.

0 1502 2215 2153 11 1667 2072 2047 12 1611 1957 1906 13 1601 1939 1831 14 1643 1965 1879 15 1604 1959 1921 16 1640 1829 1940 17 1607 1910 1910 18 1546 2045 1910 19 1529 2049 1972 1

ID X Y Z Lable

SRGE Workshop, Cairo University (07-November-2015)

Optimizing accelerometer-based data classification using WWO

13

Human Activity Data Set.Basic 7 Human Activities.

1) Working at Computer2) Standing Up, Walking and Going up\down stairs3) Standing4) Walking5) Going Up\Down Stairs6) Walking and Talking with Someone7) Talking while Standing

SRGE Workshop, Cairo University (07-November-2015)

Optimizing accelerometer-based data classification using WWO

14

Optimization Setting.Tuning penalty (c) and gamma

parameters of SVM Kernel function.Resampling strategy

Averaging windows of accelerometer readings of length (4 / 8 / 16 / 32 seconds).

SRGE Workshop, Cairo University (07-November-2015)

Optimizing accelerometer-based data classification using WWO

15

Primary results of classification process. Running SVM with gamma parameter

randomly initialized to small numbers resulted in accuracies in 70 – 80% for 15 folds of validation.

Windows of 4-sec average samples.

SRGE Workshop, Cairo University (07-November-2015)

Optimizing accelerometer-based data classification using WWO

16

Primary results of classification process. Sampling window of 4-seconds.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 150%

20%

40%

60%

80% 72.41%

Fold ID

Accu

racy

SRGE Workshop, Cairo University (07-November-2015)

Optimizing accelerometer-based data classification using WWO

17

1 2 3 4 5 6 7 8 9 10 11 12 13 14 150%20%40%60%80%

100%120%

4 -sec4-sec Trend8-sec12-sec32-sec32-sec Trend

Fold ID

Accu

racy

Primary results of classification process. Different Sampling Windows.

SRGE Workshop, Cairo University (07-November-2015)

Optimizing accelerometer-based data classification using WWO

18

Results upon optimizing Gamma and Penalty (C) Parameters.Gamma optimized in range [1,100].Penalty (C) optimized in range

Resulted in accuracies in 99 – 100% for 15 folds of validation.

SRGE Workshop, Cairo University (07-November-2015)

Optimizing accelerometer-based data classification using WWO

19

1 2 3 4 5 6 7 8 9 10 11 12 13 14 1597.5%98.0%98.5%99.0%99.5%

100.0%100.5%

4 -sec4-sec Trend16-sec32-secMoving average (32-sec)

Fold ID

Accu

racy

Primary results of classification process. Optimizing Gamma , Penalty, Different Sampling Windows.

SRGE Workshop, Cairo University (07-November-2015)

Results and conclusion

20

Optimizing SVM for classification of human-activity dataset resulted in a proper selection for SVM parameters and accuracy reached to 100%.

Classifier parameters tuning will result in high accuracy and low overfitting.

Resampling can affect classification accuracy.

SRGE Workshop, Cairo University (07-November-2015)

Thanks and Acknowledgement21

SRGE Workshop, Cairo University (07-November-2015)