View
214
Download
1
Category
Tags:
Preview:
Citation preview
Machine Learning in Engineering Problems
Jzau-Shenlg Lin ( 林灶生 )
Dept. of Computer Science and Information Engineering,
Nat’l Chin-Yi Institute of Technology
Outline
Introduction Artificial Neural Networks (ANN) Fuzzy-, Possibilistic-, and Rough- Systems Cerebellar Model Arithmetic Computer (CMAC) Genetic Algorithm (GA) Artificial Immune System (AIS) Ant Colony System (ACS) Support Vector Machine (SVM) Conclusions
Introduction(1/2) Machine learning is a research strategy, in which comput
ers can modeling or implement the humans’ learning behaviors.
It also reconstructs the intelligent architecture in its intelligIt also reconstructs the intelligent architecture in its intelligent base to reinforce the performance for itself.ent base to reinforce the performance for itself.
H.A. Simon indicated that learning is an adaptive activity for a system to causes the system doing the same or similar task more effectively.
R.S. Michalski thought that learning is the representation to configure or revise the experimental tasks.
The experts who design expert systems presented that learning is extracting intelligence.
Introduction(2/2)
Learning is a very important feature for the intelligent behavior.
The applications for the machine learning include: Robots Computer game Signal processing – Compressing, Recognition, watermarking, … Network topology – The shortest path, Channel assignment, … Several optimization problems in engineering
1.Artificial Neural Networks (ANN) (1/6)
An ANN is a Massively parallel distributed Processor that has a natural propensity for storing experiential knowledge and making it available for use. It resembles the brain in two respects:
(a) Knowledge is acquired by the network through a learning process.
(b) Interneuron connection strengths known as synaptic weights are used to
store the knowledge. Neural Networks are referred to in the literature as neurocomputers, c
onnectionist networks, parallel distributed processors, etc.
Neurobiological model
Axon( 神經軸 )— 輸出路徑Soma( 神經核 )— 細胞本體
Synapse( 神經連接線 )-- 依電位變化傳遞
Dendrites( 神經樹 )-- 輸入路徑
1. Artificial Neural Networks (ANN) (2/6)
Models of a Neuron
1kw
2kw
knw
::
)(k
kyOutput
kThreshold
Summingjunction
Activationfunction
nx
2x
1x
Synaptic Weights
Bias input
n
jjjkk xw
1,
)( kkk uy
1. Artificial Neural Networks (ANN) (3/6)
Type of Activation Function
0
1
0
1)(
v
v
if
ifv
2/1
2/12/1
2/1
0
,
,1
)(
v
v
v
vv )exp(1
1)(
avv
-2 -1.5 -1 -0.5 0 0.5 1 1.5 20
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
-2 -1.5 -1 -0.5 0 0.5 1 1.5 20
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
-10 -5 0 5 100
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
)(v )(v)(v
(a) Threshold Function (b) Piecewise-linear Function (c) Sigmoid Function
Slope=a
1. Artificial Neural Networks (ANN) (4/6)
Feedforward Architecture
Input layerof source nodes
Output layerof neurons
Input layerof source nodes
Layer ofhidden neurons
Output layerof neurons
1. Artificial Neural Networks (ANN) (5/6) Recurrent and 2-D Lattice Networks
Input layer of source nodes
Z-1 Z-1 Z-1 Z-1
1. Artificial Neural Networks (ANN) (6/6) Classifications
Neural Networks
Optimal Nets
● Hopfield-Tank net● Annealing net● Bolzmann machine
Fixed Nets
● Hamming net● Hopfield net● Bi-direction associative memory
Unsupervised Nets
● Self- organization map ● ART● Neocognitron● Competitive learning● Principle component analysis (PCA)● Independent component analysis (ICA)
Supervised Nets
● Perceptron● Back-Propagation delay
● Probabilistic net● Multilayer Perceptron● ADALINE● LVQ● Counter propagation net
2.1 Fuzzy-Systems
Fuzzy C-Means (FCM)
Hard-C-Means (HCM) Fuzzy-C-Means (FCM)
n
x
c
iixHCM wzJ
1 1
2
2
1
k
zw
ky y
i 1
n
x
c
iix
mixFCM wzuJ
1 1
2,2
1
1
1)1/(12
)1/(12
,)(
)(
c
mx
mix
ixz
z
n
x
c
i
mix
n
x
c
ix
mix
i
z
w
1 1,
1 1,
2.1 Fuzzy-Systems
Penalized FCM and Compensated FCM
Penalized FCM (PFCM) Compensated FCM (CFCM)
n
x
c
i
mix
n
x
c
ix
mix
i
z
w
1 1,
1 1,
i
mn
x
c
iix
n
x
c
iix
mixPFCM uvwzuJ ln
2
1
2
1
1 1,
1 1
2,
n
x
c
i
mix
n
x
mix
i
1 1,
1,
1
1)1/(12
)1/(12
,)ln(
)ln(
c
mx
miix
ixz
z
n
x
c
i
mix
n
x
c
ix
mix
i
z
w
1 1,
1 1,
i
mn
x
c
iix
n
x
c
iix
mixCFCM uvwzuJ tanh
2
1
2
1
1 1,
1 1
2,
n
x
c
i
mix
n
x
mix
i
1 1,
1,
1
1)1/(12
)1/(12
,)tanh(
)tanh(
c
mx
miix
ixz
z
2.1 Fuzzy-Systems
The curves of ln(i) and tanh (-i) within 0 i 1
0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1-2.5
-2
-1.5
-1
-0.5
0
2.1 Fuzzy-Systems
Fuzzification in the training example --Butterfly
x1
x2
x3
x4
x5
x6
x7
x8x9
x10
x11
x12
x13
x14
x15
1 1.2 1.4 1.6 1.8 20
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
fuzzyfica tion pa ramete r *:FCM +:P FCM o:CFHNN
me
mb
ers
hip
gra
de
s o
f z8
2.2 Possibilistic-System
Possibilistic C-Means (PCM) – Proposed by Krishnapuram and Keller
c
iix
n
xi
n
x
c
iixixPCM twztJ
1,
11 1
2, )1(
2
1
i : Scale parameter at the i-th cluster.
ixt , : Possibilistic typicality value of training sample belonging to the i-th cluster.
xz
Fuzzy Possibilistic C-Means (FPCM) -- Proposed by Pal, Pal, and Bezdek
n
x
c
iixix
mixFPCM wztJ
1 1
2,,2
1
1
1)1/(12
)1/(12
,)(
)(
c
mx
mix
ixz
z
1
1 )1/(12
)1/(12
,)(
)(
n
y miy
mix
ixz
zt
2.3 Fuzzy-, Possibilistic-Systems(1/7)
FPCM-- Membership and Typicality
z1
z2
zn
Training sample
1,1 2,1 c,1
1,2 2,2 c,2
1,n 2,n cn,
Cluster
1 2 ….. c
c
iix
1, 1
Membership function
1,1t 2,1t ct ,1
1,2t 2,2t ct ,2
1,nt 2,nt cnt ,
Cluster
1 2 ….. c
n
xixt
1, 1
Typicality function
2.3 Fuzzy-, Possibilistic-Systems(2/7)
Data Set FCM m=3
FPCM m=3, =3
x p1 p2 1,x 2,x 1,x 2,x 1,xt 2,xt 1 -3.34 0.00 0.9524 0.0476 0.9538 0.0462 0.0227 0.0012 2 -3.34 1.67 0.9599 0.0401 0.9572 0.0438 0.0368 0.015 3 -3.34 0.00 0.9972 0.0028 0.9976 0.0024 0.8664 0.0016 4 -1.67 -1.67 0.9218 0.0782 0.9249 0.0751 0.0178 0.0014 5 -1.67 0.00 9.9075 0.0925 0.9060 0.0940 0.0287 0.0031 6 0.00 0.00 0.5000 0.5000 0.5001 0.4999 0.0067 0.0067 7 1.67 0.00 0.0925 0.9075 0.0927 0.9073 0.0028 0.0301 8 3.34 1.67 0.0401 0.9599 0.0415 0.9585 0.0015 0.0385 9 3.34 0.00 0.0028 0.9972 0.0017 0.9983 0.0016 0.8654 10 3.34 -1.67 0.0782 0.9218 0.0745 0.9255 0.0014 0.0193 11 5.00 0.00 0.0476 0.9524 0.0456 0.9546 0.0010 0.0210 12 0.00 10.00 0.5000 0.5000 0.4997 0.5003 0.0005 0.0005
Class center (-3.1947, 0.3138) (3.1946, 0.3134)
(-3.2045, 0.2702) (3.2048, 0.2657)
Simulated data set
x1
x2
x3
x4
x5
x6
x7
x8
x9
x10
x11
x12
Typicality order
1,xt 2,xt
3 9 2 8 5 7 1 11 4 10 6 6 7 5 9 3 8 2
10 4 11 1 12 12
2.3 Fuzzy-, Possibilistic-Systems(3/7)
Penalized FPCM
n
x
c
ixixi
mix
n
x
c
iixix
mixPFPCM tuvwztuJ
1 1,,
1 1
2,, lnln
2
1
2
1
1
1)1/(12
)1/(12
,)ln(
)ln(
c
mx
miix
ixz
z
1
1 )1/(12
)1/(12
,)ln(
)ln(
n
y myiy
mxix
ixz
zt
n
x
c
i
mix
n
x
mix
i
1 1,
1,
n
x
c
iix
c
iix
x
t
t
1 1,
1,
=JFPCM - scale factors based on clusters and training samples
2.3 Fuzzy-, Possibilistic-Systems(4/7)
2.3 Fuzzy-, Possibilistic-Systems(5/7)
Compensated FPCM
n
x
c
ixixi
mix
n
x
c
iixix
mixCFPCM tuwztuJ
1 1,,
1 1
2,, tantan
2
1
2
1
1
1)1/(12
)1/(12
,)tan(
)tan(
c
mx
miix
ixz
z
1
1 )1/(12
)1/(12
,)tan(
)tan(
n
y myiy
mxix
ixz
zt
n
x
c
i
mix
n
x
mix
i
1 1,
1,
n
x
c
iix
c
iix
x
t
t
1 1,
1,
=JFPCM + scale factors based on clusters and training samples
2.3 Fuzzy-, Possibilistic-Systems(6/7)
Original Image
DCTDCTAC coefficients
DC coefficient
PFPCM/CFPCM
PFPCM/CFPCM CodebookCodebook
DC + Index
Transmission
Encoder
CodebookCodebook
DC + Index
DC coefficient
AC coefficientsIDCTIDCT
Reconstructed Image Decoder
2.3 Fuzzy-, Possibilistic-Systems(7/7)
OriginalImage LBG
DCT +LBG(VQ)
DCT +CFPCM(VQ)
2.4 Rough- System (1/5)
Rough set Let R be a binary equivalence relation defined on a universal set Z is a subset of the Cartesian product, . An equivalence relation is a binary relation, R, that satisfies
R is reflexive : R is symmetric : R is transitive :
can be defined as the union of all equivalence classes in Z/R that are contained in A such that
can be also defined as the union of all equivalence classes in Z/R that overlap with A like the following equation
R Z Z
R Z Z
1 1 1( , )z Z z z R 1 2 1 2 2 1( , ( , ) ) ( , )z z Z z z R z z R
1 2 3 1 2 2 3 1 3( , , ( , ) ( , ) ) ( , )z z z Z z z R z z R z z R
)(AR
},][|]{[)( ZzAzzAR RR
)(AR
},][|]{[)( ZzAzzAR RR
2.4 Rough- System (2/5)
)(AR
Rough set A rough set can be represented by and with the given set A as
And the rough boundary of A by the equivalence classes Z/R is distinct as
Interconnection models in the architecture of rough neurons
R Z Z
)(AR
)(),()( ARARAR
)()()( ARARARB
s
s
soutput
soutput
r
r
s
s
soutput
soutput
r
r
s
s
soutput
soutput
r
r
(a) Fully connected (b) Exciting model (c) Inhibiting model
2.4 Rough- System (3/5)
Rough Neurons (Proposed by Lingras in 1998) Definition for the Exciting model in the rough neurons
R Z Z
s
s
soutput
soutput
r
r
m
rrrss outputwinput
1,
m
rrrss outputwinput
1,
))(),(max( sss inputtinputtoutput
min( ( ), ( ))s s soutput t input t input
xext
1
1)(
2.4 Rough- System (4/5)
Rough Fuzzy Hopfield Neural Network (RFHNN)
R Z Z
Netx,i
:
Netx,i
:
)( y
n
1yx,i;y,i zW
)( y
n
1y
x,i;y,i zW
ixI , xz
ixI , xz
ix,
ix,
ix
2
y
n
yx,i;y,ixx,i IzWzNet ,
1
ix
2
y
n
y
x,i;y,ixx,i IzWzNet ,
1
2 2
1 1 1 1 1 1
1 1( ) ( ) ( ) ( )
2 2
n c n n c nmmx yx,i;y,ix yx,i;y,i x,ix,i
x i y x i y
E z W z z W z
n
x
m
ixixm
x,i
c
ix,i II
1,,
1
)(2
1
n
h
m
ih
m
iyiyixW
1 ,
,,;,
)(
)(
n
h
m
ih
m
iyiyixW
1 ,
,,;,
)(
)(
),min( ,,,ixixix
),max( ,,, ixixix 1
1
1/1
,
,,
c
j
m
jx
ixix
Net
Net
1
1
1/1
,
,,
c
j
m
jx
ixix
Net
Net
Processing to the Multi-Spectral Image using RFHNN
R Z Z
2.4 Rough- System (5/5)
MorphologyProcessing
Result
Fuzzy Competitive Learning Network (FCLN) Penalized Fuzzy Competitive Learning Network (PFCLN) Compensated Fuzzy Competitive Learning Network (CFCLN) Rough Fuzzy Competitive Learning Network (RFCLN) Fuzzy Hopfield Neural Network (FHNN) Penalized Fuzzy Hopfield Neural Network (PFHNN) Compensated Fuzzy Hopfield Neural Network (CFHNN) Fuzzy-Possibilistic Hopfield Neural Network (FPHNN) Rough Fuzzy Hopfield Neural Network (RFHNN)
R Z Z
2.5 Fuzzy-, Possibilistic-, Rough- Systems + Artificial Neural networks
3. Cerebellar Model Arithmetic Computer (CMAC) (1/4)
CMAC, named Cerebella Model Articulation Controller, was proposed by J.S. Albus in1975.
CMAC is a model of associate memory network. In the training phase, the CMAC updates the weights in
memory by using a transformation from input samples. It can easily obtain the outputs by looking up the weights
in memory in accordance with the input vectors in the recognition phase.
Due to a simple manner with memory architecture, the CMAC can be easily implemented into hardware circuit.
3. Cerebellar Model Arithmetic Computer (CMAC) (2/4)
**)()( ,...,2,1, Ai
A
yydWW ai
oldiainewi
Traditional CMAC Architecture
a1
a2
a4
a5
a6
a7
aN
w1
w2
w4
w5
w6
w7
wN
Input Vector
y
a w
AXS YA
P
d
i
yiE1
2)1(
3. Cerebellar Model Arithmetic Computer (CMAC) (3/4)Modified CMAC Architecture with Clustering Memory
m94 m2m1
Input Pattern Memory Output Weights
Class 1
Class 2
Class 16
a0 ~ a15
a0 ~ a15
a0 ~ a15
Sum 94
Sum 2
Sum 1
m94
m94
m2
m2
m1
m1
1
2
3
4
5
6
7
8
coding
coding
.
.
.
.
.
....
.
.
.
.
.
.
.
.
.
61626364
.
.
.
.
.
.
.
.
.
coding
1
2
N
Input signalswith quantizingbinary code
3. Cerebellar Model Arithmetic Computer (CMAC) (4/4)Applied CMAC to Character Recognition
8 error pixels in characters
14 error pixels in characters
18 error pixels in characters
4. Genetic Algorithm (GA)(1/4)
Evolutionary computing was introduced in the 1960s by I. Rechenberg in his work "Evolution strategies" (Evolutions strategie in original). His idea was then developed by other researchers. Genetic Algorithms (GAs) were invented by John Holland and developed by him and his students and colleagues at University of Michigan, 1970’s . Directed search algorithms based on the mechanics of biological evolution and Survival with a fitness function. Functions of GA:
• Chromosome– string of DNA– consists of genes– a solution of the problem
• Fitness– measure the chromosome– survival or not
• Reproduction– crossover
▪ two chromosomes ▪ combine the genes from parents▪ form new chromosomes
– mutation▪ occurs on single chromosome▪ elements of DNA are a bit changed
4. Genetic Algorithm (GA)(2/4)
Simple Genetic Algorithm(){
Randomly initialize population;evaluate population;while(termination criterion not reached){
select solutions for next population with a fitness function (reproduction);
perform crossover and mutation;evaluate population (to produce new offspring);
}}
4. Genetic Algorithm (GA)(3/4)
Population
Evaluate
Fitness Function
Evolution Circumstance
Reproduction
crossover
mutationNew Offspring
Evolutionary Procedure
Roulette-WheelSelection
Combing GA with other systems GA + Fuzzy Algorithm
GA + Possibilistic Algorithm
GA + Rough algorithm
GA + Artificial Neural Network
4. Genetic Algorithm (GA)(4/4)
The AIS transfers the characteristics of natural immune system with mathematic model into computing system in algorithm manner to solve the engineering problems.
The AIS is based on Jerne’s idiotypic network theory (Jerne, 1973), which suggests that the immune system maintains a network of interconnected B-cells.
5. Artificial Immune System (AIS)(1/9)
Natural immune system in human body
自然殺手細胞
細胞及分泌物
淋巴細胞吞噬細胞 補體
T 細胞及淋巴球B 細胞及抗體
5. Artificial Immune System (AIS)(2/9)
The structure of multi-protection and -defense system in immune system
5. Artificial Immune System (AIS)(3/9)
Models in AIS 1. Antibody Network
2. Evolutionary Algorithm Immune Genetic Algorithm (IGA) Immune Evolutionary Programming (IEP) Immune Evolutionary Strategy (IES)
3. Colonel Selection Principle (CSP)
5. Artificial Immune System (AIS)(4/9)
Antibody Network (1)
抗體群Ab population
抗原激發
選擇抗體Ab Selection
相似成熟度Affinity Maturation Selection
重選抗體Ab Re-selection
繁殖 Clone 衰亡 Death
非激發細胞激發細胞
5. Artificial Immune System (AIS)(5/9)
Antibody Network (2)
Inhibit immune reaction
Start Immune System
Input external Ags
B-Cell’s surface
Character of Ags displayed by B-cell
Activate lymphoid cells
Th cells secrete to start immune reaction
Activate Ts to secrete IL¯
Generate Abs
All Ags are removed?
Generate Abs Continuously?
Stop immune reaction
N
Y
YN
IL
5. Artificial Immune System (AIS)(6/9)
Immune Genetic Algorithm with Constraint (IGAC)
抗原與目的函數、制約條件的對應
生成起始抗體群
抗體與抗原親和度 (R1) 計算
親和度 (R1)=1?
抗原排除 ( 結束 )
記憶親和度 (R1)高的抗體
生成新抗體
抗體與記憶親和度 (R2)的計算
排除適量親和度 (R2)高的抗體
促進和抑制
Y
N
5. Artificial Immune System (AIS)(7/9)
Immune Genetic Algorithm with Vaccination (IGAV)
啟動
最佳族群?
選擇、交配、及突變
族群更新及適應因子計算
停止 疫苗注射
免疫選擇
Y
N
5. Artificial Immune System (AIS)(8/9)
Antibody Network (2) + Fuzzy Algorithm to Image segmentation
5. Artificial Immune System (AIS)(9/9)
Ant system algorithm, based on behavior of real ants, is a natural approach to establish from their nest to food source.
An ant moves randomly and detects a previously laid pheromone on a path in order to find the shortest way between their nest and the food source.
Ant system algorithm is an important methodology to apply on non-linear optimal problems recently.
It is a parallel architecture to force ants move simultaneously, independently, and without supervisor.
6. Ant Colony System (ACS)(1/4)
6. Ant Colony System (ACS)(2/4)
Each ant chooses the next position to visit in accordance with the visibility of the position and the ph
eromone intensity. The k-th ant starting from position i decides to visit
position j with the probability defined as follows:
where is the visibility of position j from position i, and are two heuristically defined paramet
ers.
( )( )
( )
ij ijkij
i i
tp t
t
1ij
ijd
6. Ant Colony System (ACS)(3/4)
We define the pheromone intensity on path (i, j) at time t to be and to assign a random value to it when t = 0.
Along the path from i to j, a trail substance is laid on path (i, j) and defined as:
where Q is a constant and is the tour length of the
k-th ant.
0
kkij
Qif k th ant uses path in its tour
L
otherwise
( )ij t
kL
6. Ant Colony System (ACS)(4/4)
When the ant has completed a position and a cycle of n iterations is consisted, the laid trail substance is used to update the amount of substance previously laid as the following equation:
and
where is a coefficient of persistence of the tail and is the quantity of trail substance laid on path (i, j) by the k-th ant during a cycle( between time t and t + n).
( ) ( )ij ij ijt n t
1
mk
ij ijk
kij
6.1 Annealing algorithm + ACS (AACS)
In the scheme of ant system algorithm, the total cost function for the network topology from node i to k and cooling schedule can be defined as
where
The probability that the k-th ant starting from node i to visit node j undergo random thermal perturbations at a given temperature T conforms to a Boltzmann distribution
( ) [ ( )] [ ]ik iki k
E t t
1/ ( , )
0ik
ikd if path i k exists
otherwise
( ) /( ) ijE t Tk
ijp t e
( ) /
( ) /( )
ij
i
E t Tkij
E t T
ep t
e
1( ) tanh( ) ( 1)
1tT t w T t
and
6.2 The Application of AACS to Shortest Problem(1/4)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20Source
Destination
230 130
90
4530
65
50
6227
25
136 58
43
15
40
32
17
29
35
60 250
30120
32
220
61
20
88
144
161
71
54 26
61
147
60194
1108977
14
150 40
220 72
22
72
24
16
Length = 142
the conventional ant system algorithm
(a) CM model (b) DM model
0 50 100 150 200 250 30080
82
84
86
88
90
92
94
96
98
100
the number of ants
succ
essf
ul p
erce
ntag
e
o : = 0.2 + : = 0.4* : = 0.6 : = 0.8 : = 1.0
0 50 100 150 200 250 30010
20
30
40
50
60
70
80
90
100
the number of ant
the n
um
ber
of
success
o : = 0.2 + : = 0.4* : = 0.6 : = 0.8 : = 1.0
6.2 The Application of AACS to Shortest Problem(2/4)
Conventional ant algorithm with the Roulette wheel selection
(a) CM model (b) DM model
6.2 The Application of AACS to Shortest Problem(3/4)
0 50 100 150 200 250 30030
40
50
60
70
80
90
100
the number of ants
succ
essf
ul p
erce
ntag
e
0 50 100 150 200 250 30020
30
40
50
60
70
80
90
100
the number of ants
succ
essf
ul p
erce
ntag
e
o:=0.2 +:=0.4*:=0.6 :=0.8 :=1.0The value of
o:=0.2 +:=0.4*:=0.6 :=0.8 :=1.0The value of
Experimental results The annealed ant system algorithm with the Roulette wheel selection
(a) CM model (b) DM model
0 50 100 150 200 250 30060
65
70
75
80
85
90
95
100
the number of ants
succ
essf
ul p
erce
ntag
e
0 50 100 150 200 250 3000
10
20
30
40
50
60
70
80
90
100
the number of ants
succ
essf
ul p
erce
ntag
e
o:w=0.98, =0.8x:w=0.98, =0.99+:w=0.998, =0.8 :w=0.998, =0.99
o:w=0.98, =0.8x:w=0.98, =0.99+:w=0.998, =0.8 :w=0.998, =0.99
6.2 The Application of AACS to Shortest Problem(4/4)
7. Support Vector Machine (SVM)(1/6)
SVM was first introduced by Boser et al. in 1992. SVM is now regarded as an important example of “
kernel methods”, arguably the hottest area in machine learning.
Class 1
Class 2
Class 1
Class 2
Class 1
Class 2
7. SVM -- Large-margin Decision Boundary(2/6) The decision boundary should be as far away from
the data of both classes as possible We should maximize the margin, m
Class 1
Class 2
m
7. SVM -- Nonlinear Decision Boundary(3/6)
(.)
Input space
( )( )
( )
( )
( )
( )
( )
( )
( )( )
( )
Feature space
( )( )
( )( )
( )
( ) ( )
Nonlinear Transform
7. SVM – Application to the Face Detection(4/6)
Support Vectors :
SVM Training
1 2 3 4 5 i 1i 2i 3i
…
Training
From Sami Romdhani et al.
7. SVM – Application to the Face Detection(5/6)
From Sami Romdhani et al.
1 2 3 4 5 i 1i 2i 3i
…
D D D D D D D D D
Output
2. ClassificationIs this path a face ?
> T Face<= T Non-Face
7. SVM – Application to the Face Detection(6/6)
Fro
m S
ami R
omdh
ani
et a
l.
8. Conclusions Machine-learning research has been making great progress
in many directions such as (1)The improvement of classification accuracy by learning ensembles of classifiers (2) Methods for scaling up supervised learning algorithms (3) reinforcement learning (4) the learning of complex
Machine-learning techniques are being applied to several problems including knowledge discovery in databases, language processing, robot control, and combinatorial optimization, as well as to more traditional problems such as speech recognition, face recognition, handwriting recognition, medical data analysis, and game playing.
Thanks !
Recommended