Departamento de Electr onica, Telecomunica˘c~oes e Inform ...Departamento de Electr onica,...

Departamento de Electrónica, Telecomunicações e Informática

EXPLORAÇÃO de DADOS & DATA MINING

Exercises: Classification Linear Classifiers and Non-Linear (SVM and NN) )

1. The following table describes a simple toy example (see lecture slides) with two classeswhere each object (instance) is described by two features.

attribute 1 attribute 2 class−1 1 1

1 −1 1−1 −1 1

1 1 −1

(a) Represent the data set in the space.

(b) The output y of a binary linear classifier is computed by

y = sign(wTx + w0)

where y is either 1 or −1 according to the argument of sign.• The problem is linearly separable? If so find one possible solution, e.g, find w

and w0 ≡ b which allows to assign the correct label to x.(c) Apply the perceptron learning rules (starting values w2 = w1 = w0 = 0 and learning

rate 0.5).

2. Three operators of RapidMiner are available to design linear classifiers described by thedecision surface: perceptron, linear regression and SVM.

• Use linear regression. Write the hyperplane equation• Use perceptron. What is the meaning of the parameters of the operator? Write the

hyperplane equation.

• Use SVM with linear configuration (kernel type: linear or dot). Write the hyperplaneequation. Give different values to parameter C.

3. The following table describes a simple toy example with two classes where each object(instance) is described by two features.

attribute 1 attribute 2 class−1 1 1

1 −1 1−1 −1 −1

1 1 −1

is it a linear problem? Try to solve this problem in RapidMiner

1

(a) SVM can be a linear and non-linear classifier. How is this achieved?

(b) Neural Networks are

• Linear classifiers with input layer and output layer with linear activation functions• Non-linear Classifiers with hidden layers with sigmoid activation functions and

output layer might have linear activation function.

4. Create a complete project to classify the data set Ripley-Set. Your project should consider

• First Phase: The evaluation of the classification by the use of cross-validation (X-validation operators).

• Second Phase: Comparison of two different classifiers (t-test).

(a) Use first only linear classifiers

(b) Use SVM in a non-linear configuration, e.g. ( kernel type: RBF). Note that twoparameters (C and γ) can be changed.

(c) Use a neural network classifier.

5. Classification into multi-classes: create a project in RapidMiner to classify the Iris dataset.

• Try the linear regression classification. How is the decision made?• How are SVM used in multiclass problems? What strategy is used in your RapidMiner

project.

• Use a multi-class perceptron.

6. With the operator generate data is possible to create artificial data which is

• Linearly separable data• Non-linearly separable data

Create data with two attributes (features) and apply the different classifiers. Is it possibleto illustrate the decision surfaces in RapidMiner?

Further Remarks

• In a SVM classifier what is a support vector?

• With Linear SVM it is possible ro compute the vector w while with non-linear SVM it isnot. What is the reason?

• What is the meaning of kernel tricky

• If a Neural Network (NN) has hidden layers only makes sense that its neurons have non-linear activation functions. Try to find a justification.

2

~8l-no.-d~ f~e.üc..\~--

L Q..\assJhccc\C'neL'~iJ::€ __a.oeLL~ .aenne.s CSvt\,o ~

\

(\~ ---

(,_1,-1

_ O',A" ~ óe ('" ''',\.H'", ( i - -I \ y T

"'\\ \ _L; ir:: L, '1 ,~s II _I,) I ~ :: O .l-() -1 ,,), :.

I \ \ I Ilo \ , -, 1 () I \ OI - , \- ) .\ \

-\

\ {\ \\0 \ \~_O j -'\ \ \ '\ lU I-( '\ - ~') ,- f- -

I.. =

~

;

I (-'I-I) I --I = .,,'.,..~ (r- -'{',')J -.)

I =-'J

,\

oi.

Wi:l -i.