Modeling using WEKA
Index• WEKA Introduction• WEKA file formats• Loading data• Univariate analysis• Data Manipulation• Feature Selection• Creating Training, Validation and Test Sets• Model Execution - Logistic Regression• Model Analysis - ROC Curve• Model Analysis – Cost/Benefit Analysis• Re-apply model on new data• Weka Plus and Negatives
Introduction• Weka is a collection of machine learning algorithms for data mining tasks• The algorithms can either be applied directly to a dataset or called from your
own Java code.• ARFF – Attribute relation file format
Dataset File formats
Load Data Set
Univariate Analysis
Univariate Analysis• Current Relation – Dataset name, number of records, number of attributes in
dataset.
• Attribute details- All attributes to select for univariate analysis
Univariate Analysis• Selected attribute –
• Provides information about attribute type, Missing values, Distinct values, etc.
Univariate Analysis• Selected attribute – histogram
• Dispersion of attribute.
Univariate Analysis• All attribute visualization/plots
Data Manipulation• Changing data type of field• Missing values update• Creating BINS from data• Standardize data• Outlier Treatment• Creating new calculated fields
1. Convert NA to 0• Flow > Preprocess > Edit > Right Click on Attribute > Replace Values
1. Convert NA to 0
2. Changing data types of attribute• Flow > Preprocess > Filter > Choose > Filters > Unsupervised > Attribute >
2. Changing data types
3. Creating BINS• Flow > Preprocess > Filter > Choose > Filters > Unsupervised > Attribute >
Discretize
3. Creating BINS• Provide attribute number, Number of BINs to be created and click on ‘Apply’
3. Creating BINS• Click on attribute to see the bins and distribution•
3. Creating Custom BINS• Flow > Preprocess > Filter > Choose > Filters > Unsupervised > Attribute >
AddExpression• ifelse(a2 > 0, ifelse(a2 > 10,ifelse(a2 > 20,4, 3), 2), 1)
4. Standardize data • To convert all numeric attributes in data to zero mean and unit variance.• Flow > Preprocess > Filter > Choose > Filters > Unsupervised > Attribute >
Standardize
•
4. Standardize data
4. Standardize data- Log values • To convert specific numeric attributes to log.• Flow > Preprocess > Filter > Choose > Filters > Unsupervised > Attribute >
Numeric Transform
•
4. Standardize data- Log values • Provide value for attribute number which is to be converted to log value.• Also provide method name – log. Here we can provide any other methods such
as abs,round,floor•
5. Identify Outliers• Flow > Preprocess > Filter > Choose > Filters > Unsupervised > Attribute >
Interquartile Range• Outliers can be identified for separate attribute or for all together
5. Identify Outliers
5. Remove Outliers• Flow > Preprocess > Filter > Choose > Filters > Unsupervised > Instance >
RemoveWithValues
5. Remove Outliers• Params : attributeIndices - Attribute number, NominalIndices=Nominal value
of outlier in Outlier attribute
5. Transform Outliers• Flow > Preprocess > Filter > Choose > Filters > Unsupervised > Attribute >
AddExpression – this option will create new field e.g : ifelse(a2 > 1000,200, 1)•
6. New Calculated fields• This is helpful In case any new field is to be derived from existing fields• Flow > Preprocess > Filter > Choose > Filters > Unsupervised > Attribute >
Add Expression
6. New Calculated fields• This is helpful In case any new field is to be derived from existing fields• Provide expression/equation and new field name
Feature Selection/Attribute selection
1. Info Gain2. Correlation
Feature Selection – Info Gain• Flow > Select Attribute > Attribute evaluator > Choose >
Feature Selection – Correlation• Flow > Select Attribute > Attribute evaluator > Choose >
Features for Model• Features selected for model
Creating Training, Validation and Test Sets
Creating data sets1. Dividing data into 60-20-20 % (Train-Test-Evaluate)2. Weka inbuilt methods
Creating data sets• For 60%-20%-20%• Step 1-
• Flow > Preprocess > Filter > Choose > Unsupervised > Instance > Resample
Creating data sets• Step 2- Parameters for resample
• Flow > Preprocess > Filter > Choose > Unsupervised > Instance > Resample > • Check noReplacement = True, sample size percent – 60 > ok > Apply
Creating data sets• Step 3 – Check• After apply we can check the current relation for number of records selected
•
• Step 4 – Save the result as filename_train.arff• Step 5 – Click on ‘Undo’ to get to original data set• Step 6 – Change the Resample parameters again
• Parameters - > Invert selection = True, noReplacement = True, sampleSizePercent = 60
Creating data sets• Step 7 – Apply and check results as below
Creating data sets• Step 8 – Don’t save the results• Step 9 – Open Resample parameters set below parameters
• Invert selection = False, noReplacement = True, sampleSizePercent = 50 • OK > Apply Check the results
Creating data sets• Step 10 – Check the results• Step 11 – Save the results as Test Data.
• Step 13 – Click on Undo to get earlier 40% of dataset• Step 13 – Parameters invertSelection = True, noReplacement=True ,
sampleSizePercent = 50• Step 14 – Ok > Apply • Step 15 – Save as Evaluation Data
Creating data sets
• .
Creating data sets2. Weka inbuilt – Flow > Classify > Test Options > Use Training Set – With this option selected data set will be used as training set to create model
Creating data sets2. Weka inbuilt – Flow > Classify > Test Options > Use Supplied Test Set – With this option selected data set will be used as test set to create model
Creating data sets2. Weka inbuilt – Flow > Classify > Test Options > Use Supplied Test Set – With this option selected data set will be used as test set to test model
Creating data sets2. Weka inbuilt – Flow > Classify > Test Options > Cross Validation – With this option selected data set will be divided into 10 folds create model internally and weka will take average of all these models to show final model on UI
Creating data sets2. Weka inbuilt – Flow > Classify > Test Options > Percentage Split – With this option selected data set will be divided into Training and Test set for model creation
Logistic Regression
Logistic Regression• Flow > Classify > Functions > Logistic
Logistic Regression• Parameter selection
Logistic Regression
• Model Results:
Model Analysis• Flow > Right Click on Model > Visualize threshold curve > ROC Curve
Model Analysis- ROC Curve
Model Analysis- Cost Benefit Analysis• Flow > Classify > Right click model > Cost/Benefit Analysis
Model Analysis- Cost/Benefit Analysis• Flow > Classify > Right click model > Cost/Benefit Analysis > Threshold Bar• Sliding the bar under Threshold label will change the accuracy and threshold
curve
Save prediction output to file• Flow > Classify > Test Options > More Options > Output Predictions > Text Bar
to provide file name• Parameters: Choose: to provide file type, Attributes : First-last to get all fields,
outPutFile : File name to save data
Re-apply model on new data
WEKA Pluses:• Platform independent and portable, java library can be invoked from any
program in any language• User friendly GUI, with built in visualization, Simpler to use than R, large
collection of different data mining algorithms• Better results for classification and cluster modeling• Ease of designing solutions.• Provides 3 ways to use the software: the GUI, a Java API, and a command line
interface (CLI)• Can work with Spark, BigData using other packages on Experimenter or in
batch mode.
WEKA Limitation:• Visualizations can be managed better in R with different packages like ggplot• Not really flexible for data manipulation• Accepts only limited file format lile CSV,ARFF• Limited documentation available on Explorer.
THANK YOU !
Decision Tree
Decision Tree: Algorithm selection
Decision Tree: Setting params for algo
Decision Tree: Execution
Tree Visualization