Embed Size (px)
Citation preview
Department of Computer Science, University of Waikato, New Zealand
Bernhard Pfahringer(based on material by Eibe Frank, Mark
Hall, and Peter Reutemann)
WEKA: A Machine Learning ToolkitThe Explorer
• Classification and Regression
• Clustering• Association Rules• Attribute Selection• Data Visualization
The ExperimenterThe Knowledge Flow GUIOther UtilitiesConclusions
Machine Learning with WEKA
5/14/2007 University of Waikato 2
WEKA: the bird
Copyright: Martin Kramer ([email protected])
The Weka or woodhen (Gallirallus australis) is an endemic bird of New Zealand. (Source: WikiPedia)
5/14/2007 University of Waikato 3
WEKA: the softwareMachine learning/data mining software written in Java (distributed under the GNU Public License)Used for research, education, and applicationsComplements “Data Mining” by Witten & FrankMain features:
Comprehensive set of data pre-processing tools, learning algorithms and evaluation methodsGraphical user interfaces (incl. data visualization)Environment for comparing learning algorithms
5/14/2007 University of Waikato 4
HistoryProject funded by the NZ government since 1993
Develop state-of-the art workbench of data mining toolsExplore fielded applicationsDevelop new fundamental methods
5/14/2007 University of Waikato 5
History (2)Late 1992 - funding was applied for by Ian Witten1993 - development of the interface and infrastructure
WEKA acronym coined by Geoff HolmesWEKA’s file format “ARFF” was created by Andrew DonkinARFF was rumored to stand for AAndrew’s RRidiculous FFile FFormat
Sometime in 1994 - first internal release of WEKATCL/TK user interface + learning algorithms written mostly in CVery much beta softwareChanges for the b1 release included (among others):“Ambiguous and Unsupported menu commands removed.”
“Crashing processes handled (in most cases :-)”
October 1996 - first public release: WEKA 2.1
5/14/2007 University of Waikato 6
History (3)July 1997 - WEKA 2.2
Schemes: 1R, T2, K*, M5, M5Class, IB1-4, FOIL, PEBLS, support for C5Included a facility (based on Unix makefiles) for configuring and running large scale experiments
Early 1997 - decision was made to rewrite WEKA in JavaOriginated from code written by Eibe Frank for his PhDOriginally codenamed JAWS (JAJAva WWeka SSystem)
May 1998 - WEKA 2.3Last release of the TCL/TK-based system
Mid 1999 - WEKA 3 (100% Java) releasedVersion to complement the Data Mining bookDevelopment version (including GUI)
5/14/2007 University of Waikato 7
The GUI back then…
TCL/
TK in
terfa
ce o
f Wek
a 2.
1
5/14/2007 University of Waikato 8
WEKA: versionsThere are several versions of WEKA:
WEKA 3.4: “book version” compatible with description in data mining bookWEKA 3.5.5: “development version” with lots of improvements
This talk is based on a nightly snapshot of WEKA 3.5.5 (12-Feb-2007)
5/14/2007 University of Waikato 9
@relation heart-disease-simplified
@attribute age numeric@attribute sex { female, male}@attribute chest_pain_type { typ_angina, asympt, non_anginal, atyp_angina}@attribute cholesterol numeric@attribute exercise_induced_angina { no, yes}@attribute class { present, not_present}
@data63,male,typ_angina,233,no,not_present67,male,asympt,286,yes,present67,male,asympt,229,yes,present38,female,non_anginal,?,no,not_present...
WEKA only deals with “flat” files
5/14/2007 University of Waikato 10
@relation heart-disease-simplified
@attribute age numeric@attribute sex { female, male}@attribute chest_pain_type { typ_angina, asympt, non_anginal, atyp_angina}@attribute cholesterol numeric@attribute exercise_induced_angina { no, yes}@attribute class { present, not_present}
@data63,male,typ_angina,233,no,not_present67,male,asympt,286,yes,present67,male,asympt,229,yes,present38,female,non_anginal,?,no,not_present...
WEKA only deals with “flat” files
5/14/2007 University of Waikato 11
java weka.gui.GUIChooser
5/14/2007 University of Waikato 12
5/14/2007 University of Waikato 13
5/14/2007 University of Waikato 14
java -jar weka.jar
5/14/2007 University of Waikato 15
Explorer: pre-processing the dataData can be imported from a file in various formats: ARFF, CSV, C4.5, binaryData can also be read from a URL or from an SQL database (using JDBC)Pre-processing tools in WEKA are called “filters”WEKA contains filters for:
Discretization, normalization, resampling, attribute selection, transforming and combining attributes, …
5/14/2007 University of Waikato 16
5/14/2007 University of Waikato 17
5/14/2007 University of Waikato 18
5/14/2007 University of Waikato 19
5/14/2007 University of Waikato 20
5/14/2007 University of Waikato 21
5/14/2007 University of Waikato 22
5/14/2007 University of Waikato 23
5/14/2007 University of Waikato 24
5/14/2007 University of Waikato 25
5/14/2007 University of Waikato 26
5/14/2007 University of Waikato 27
5/14/2007 University of Waikato 28
5/14/2007 University of Waikato 29
5/14/2007 University of Waikato 30
5/14/2007 University of Waikato 31
5/14/2007 University of Waikato 32
5/14/2007 University of Waikato 33
5/14/2007 University of Waikato 34
5/14/2007 University of Waikato 35
5/14/2007 University of Waikato 36
5/14/2007 University of Waikato 37
Explorer: building “classifiers”Classifiers in WEKA are models for predicting nominal or numeric quantitiesImplemented learning schemes include:
Decision trees and lists, instance-based classifiers, support vector machines, multi-layer perceptrons, logistic regression, Bayes’ nets, …
“Meta”-classifiers include:Bagging, boosting, stacking, error-correcting output codes, locally weighted learning, …
5/14/2007 University of Waikato 38
5/14/2007 University of Waikato 39
5/14/2007 University of Waikato 40
5/14/2007 University of Waikato 41
5/14/2007 University of Waikato 42
5/14/2007 University of Waikato 43
5/14/2007 University of Waikato 44
5/14/2007 University of Waikato 45
5/14/2007 University of Waikato 46
5/14/2007 University of Waikato 47
5/14/2007 University of Waikato 48
5/14/2007 University of Waikato 49
5/14/2007 University of Waikato 50
5/14/2007 University of Waikato 51
5/14/2007 University of Waikato 52
5/14/2007 University of Waikato 53
5/14/2007 University of Waikato 54
5/14/2007 University of Waikato 55
5/14/2007 University of Waikato 56
5/14/2007 University of Waikato 57
5/14/2007 University of Waikato 58
5/14/2007 University of Waikato 59
5/14/2007 University of Waikato 60
5/14/2007 University of Waikato 61
5/14/2007 University of Waikato 62
5/14/2007 University of Waikato 63
5/14/2007 University of Waikato 64
5/14/2007 University of Waikato 65
5/14/2007 University of Waikato 66
5/14/2007 University of Waikato 67
5/14/2007 University of Waikato 68
5/14/2007 University of Waikato 69
5/14/2007 University of Waikato 70
5/14/2007 University of Waikato 71
5/14/2007 University of Waikato 72QuickTime?and a TIFF (LZW) decompressor are needed to see this picture.
5/14/2007 University of Waikato 73QuickTime?and a TIFF (LZW) decompressor are needed to see this picture.
5/14/2007 University of Waikato 74
5/14/2007 University of Waikato 75
5/14/2007 University of Waikato 76
5/14/2007 University of Waikato 77
5/14/2007 University of Waikato 78
QuickTime?and a TIFF (LZW) decompressor are needed to see this pictu
5/14/2007 University of Waikato 79
5/14/2007 University of Waikato 80
5/14/2007 University of Waikato 81
5/14/2007 University of Waikato 82
5/14/2007 University of Waikato 83
5/14/2007 University of Waikato 84
5/14/2007 University of Waikato 85
5/14/2007 University of Waikato 86
5/14/2007 University of Waikato 87
5/14/2007 University of Waikato 88
5/14/2007 University of Waikato 89
5/14/2007 University of Waikato 90
5/14/2007 University of Waikato 91
5/14/2007 University of Waikato 92
5/14/2007 University of Waikato 93
Explorer: clustering dataWEKA contains “clusterers” for finding groups of similar instances in a datasetSome implemented schemes are:
k-Means, EM, Cobweb, X-means, FarthestFirstClusters can be visualized and compared to “true”clusters (if given)Evaluation based on loglikelihood if clustering scheme produces a probability distribution
5/14/2007 University of Waikato 94
5/14/2007 University of Waikato 95
5/14/2007 University of Waikato 96
5/14/2007 University of Waikato 97
5/14/2007 University of Waikato 98
5/14/2007 University of Waikato 99
5/14/2007 University of Waikato 100
5/14/2007 University of Waikato 101
5/14/2007 University of Waikato 102
5/14/2007 University of Waikato 103
Explorer: finding associationsWEKA contains the Apriori algorithm (among others) for learning association rules
Works only with discrete dataCan identify statistical dependencies between groups of attributes:
milk, butter ⇒ bread, eggs (with confidence 0.9 and support 2000)
Apriori can compute all rules that have a given minimum support and exceed a given confidence
5/14/2007 University of Waikato 104
5/14/2007 University of Waikato 105
5/14/2007 University of Waikato 106
5/14/2007 University of Waikato 107
Explorer: attribute selectionPanel that can be used to investigate which (subsets of) attributes are the most predictive onesAttribute selection methods contain two parts:
A search method: best-first, forward selection, random, exhaustive, genetic algorithm, rankingAn evaluation method: correlation-based, wrapper, information gain, chi-squared, …
Very flexible: WEKA allows (almost) arbitrary combinations of these two
5/14/2007 University of Waikato 108
5/14/2007 University of Waikato 109
5/14/2007 University of Waikato 110
5/14/2007 University of Waikato 111
5/14/2007 University of Waikato 112
5/14/2007 University of Waikato 113
5/14/2007 University of Waikato 114
Explorer: data visualizationVisualization very useful in practice: e.g. helps to determine difficulty of the learning problemWEKA can visualize single attributes (1-d) and pairs of attributes (2-d)
To do: rotating 3-d visualizations (Xgobi-style)Color-coded class values“Jitter” option to deal with nominal attributes (and to detect “hidden” data points)“Zoom-in” function
5/14/2007 University of Waikato 115
5/14/2007 University of Waikato 116
5/14/2007 University of Waikato 117
5/14/2007 University of Waikato 118
5/14/2007 University of Waikato 119
5/14/2007 University of Waikato 120
5/14/2007 University of Waikato 121
5/14/2007 University of Waikato 122
5/14/2007 University of Waikato 123
5/14/2007 University of Waikato 124
5/14/2007 University of Waikato 125
Performing experimentsExperimenter makes it easy to compare the performance of different learning schemesFor classification and regression problemsResults can be written into file or databaseEvaluation options: cross-validation, learning curve, hold-outCan also iterate over different parameter settingsSignificance-testing built in!
5/14/2007 University of Waikato 126
5/14/2007 University of Waikato 127
5/14/2007 University of Waikato 128
5/14/2007 University of Waikato 129
5/14/2007 University of Waikato 130
5/14/2007 University of Waikato 131
5/14/2007 University of Waikato 132
5/14/2007 University of Waikato 133
5/14/2007 University of Waikato 134
5/14/2007 University of Waikato 135
5/14/2007 University of Waikato 136
The Knowledge Flow GUI
Java-Beans-based interface for setting up and running machine learning experimentsData sources, classifiers, etc. are beans and can be connected graphicallyData “flows” through components: e.g.,“data source” -> “filter” -> “classifier” -> “evaluator”Layouts can be saved and loaded again latercf. Clementine ™
5/14/2007 University of Waikato 137
5/14/2007 University of Waikato 138
5/14/2007 University of Waikato 139
5/14/2007 University of Waikato 140
5/14/2007 University of Waikato 141
5/14/2007 University of Waikato 142
5/14/2007 University of Waikato 143
5/14/2007 University of Waikato 144
5/14/2007 University of Waikato 145
5/14/2007 University of Waikato 146
5/14/2007 University of Waikato 147
5/14/2007 University of Waikato 148
5/14/2007 University of Waikato 149
5/14/2007 University of Waikato 150
5/14/2007 University of Waikato 151
5/14/2007 University of Waikato 152
Sourceforge.net – Downloads
5/14/2007 University of Waikato 153
Sourceforge.net – Web Traffic
WekaWiki launched – 05/2005
WekaDoc Wiki introduced – 12/2005
5/14/2007 University of Waikato 154
Projects based on WEKA45 projects currently (30/01/07) listed on the WekaWikiIncorporate/wrap WEKA
GRB Tool Shed - a tool to aid gamma ray burst researchYALE - facility for large scale ML experimentsGATE - NLP workbench with a WEKA interfaceJudge - document clustering and classificationRWeka - an R interface to Weka
Extend/modify WEKABioWeka - extension library for knowledge discovery in biologyWekaMetal - meta learning extension to WEKAWeka-Parallel - parallel processing for WEKAGrid Weka - grid computing using WEKAWeka-CG - computational genetics tool library
5/14/2007 University of Waikato 155
WEKA and PENTAHOPentaho – The leader in Open Source Business Intelligence (BI)September 2006 – Pentaho acquires the Weka project (exclusive license and SF.net page)Weka will be used/integrated as data mining component in their BI suiteWeka will be still available as GPL open source softwareMost likely to evolve 2 editions:
Community editionBI oriented edition
5/14/2007 University of Waikato 156
Limitations of WEKATraditional algorithms need to have all data in main memory==> big datasets are an issueSolution:
Incremental schemesStream algorithmsMOA “MMassive OOnline AAnalysis”(not only a flightless bird, but also extinct!)
5/14/2007 University of Waikato 157
Conclusion: try it yourself!WEKA is available at
http://www.cs.waikato.ac.nz/ml/wekaAlso has a list of projects based on WEKA(probably incomplete list of) WEKA contributors:Abdelaziz Mahoui, Alexander K. Seewald, Ashraf M. Kibriya, Bernhard Pfahringer, Brent Martin, Peter Flach, Eibe Frank, Gabi Schmidberger, Ian H. Witten, J. Lindgren, Janice Boughton, Jason Wells, Len Trigg, Lucio de Souza Coelho, Malcolm Ware, Mark Hall, Remco Bouckaert, Richard Kirkby, Shane Butler, Shane Legg, Stuart Inglis, Sylvain Roy, Tony Voyle, Xin Xu, Yong Wang, Zhihai Wang
