Support Vector Machine (Part 2)aykut/classes/spring2016/bbm406/... · The Solution of Quiz-2 •The...

Support Vector Machine(Part 2)

OUTLINE

• Multi-class classification

• Nonlinear mapping

• Kernel

The Solution of Quiz-2• The maximum margin weight vector will be parallel to the shortest line connecting points of the

two classes, that is, the line between (1,1) and (2,3), giving a weight vector of (1,2)

• Working algebraically, with the standard constraint that:

Minimize ||w|| subject to

• For some a;

a+2a + b = -1

2a + 6a + b = 1

a= 2/5, b=- 11/5, so the optimal hyperplane is given by w = (2/5 , 4/5) and b = -11/5.

The large margin M is 2/ ||w|| 2/4

25+

16

25= 5

Multi-class Classification

Multi-class Classification

Multi-class Classification

Source code example: http://scikit-learn.org/stable/modules/multiclass.html

Source: [http://www.cs.stevens.edu/~mordohai/classes/cs559_s10/Week10.pdf]

Source: [http://www.cs.stevens.edu/~mordohai/classes/cs559_s10/Week10.pdf]

Source: [http://www.cs.stevens.edu/~mordohai/classes/cs559_s10/Week10.pdf]

Source: [http://www.cs.stevens.edu/~mordohai/classes/cs559_s10/Week10.pdf]

Source: [http://www.cs.stevens.edu/~mordohai/classes/cs559_s10/Week10.pdf]

Source: [http://www.cs.stevens.edu/~mordohai/classes/cs559_s10/Week10.pdf]

Source: [http://www.cs.stevens.edu/~mordohai/classes/cs559_s10/Week10.pdf]

Source: [http://www.cs.stevens.edu/~mordohai/classes/cs559_s10/Week10.pdf]

Source: [http://sli.ics.uci.edu/Classes/2016W-178?action=download&upname=07_svm.pdf]

Source: [https://class.coursera.org/ml-005/lecture/74]

Source: [https://class.coursera.org/ml-005/lecture/74]

Source: [https://class.coursera.org/ml-005/lecture/74]

Source: [https://class.coursera.org/ml-005/lecture/74]

Source: [https://class.coursera.org/ml-005/lecture/74]

Gaussian Kernel Example (1)

Source: [https://class.coursera.org/ml-005/lecture/74]

Gaussian Kernel Example (2)

Source: [https://class.coursera.org/ml-005/lecture/74]

Gaussian Kernel Example (3)

Source: [https://class.coursera.org/ml-005/lecture/74]

Source: [https://class.coursera.org/ml-005/lecture/74]

Source: [https://class.coursera.org/ml-005/lecture/74]

Source: [https://class.coursera.org/ml-005/lecture/74]

Source: [https://class.coursera.org/ml-005/lecture/74]

Demo

• http://scikit-learn.org/stable/modules/svm.html

• http://scikit-learn.org/stable/auto_examples/plot_multilabel.html#example-plot-multilabel-py

• http://scikit-learn.org/stable/auto_examples/svm/plot_svm_nonlinear.html

• http://scikit-learn.org/stable/auto_examples/svm/plot_svm_kernels.html

• http://scikit-learn.org/stable/auto_examples/svm/plot_custom_kernel.html

References

• http://cs229.stanford.edu/notes/cs229-notes3.pdf

• https://class.coursera.org/ml-005/lecture/74

• http://www.cs.stevens.edu/~mordohai/classes/cs559_s10/Week10.pdf

• http://nlp.stanford.edu/IR-book/html/htmledition/support-vector-machines-the-linearly-separable-case-1.html

• http://www.holehouse.org/mlclass/12_Support_Vector_Machines.html