Upload
ngothien
View
215
Download
1
Embed Size (px)
Citation preview
Prof. Dr. Bernhard Humm, Darmstadt University of Applied Sciences. www.fbi.h-da.de/~b.humm. 30.05.20161
8. Machine LearningApplied Artificial Intelligence
Prof. Dr. Bernhard HummFaculty of Computer ScienceHochschule Darmstadt – University of Applied Sciences
Prof. Dr. Bernhard Humm, Darmstadt University of Applied Sciences. www.fbi.h-da.de/~b.humm. 30.05.2016
RetrospectiveNatural Language Processing
• Name and explain different areas of NLP
• What are the “7 levels of language understanding“?
• What is tokenizing, sentence splitting, POS tagging, and parsing?
• What do language resources offer to NLP? Give examples
• What do NLP frameworks offer? Give examples
• What do NLP web services offer? Give examples
2
Agenda
• Overview
• ML Applications
• ML Tasks
• ML Approaches
• Methodology
• ML Tools
• Services / Product Map
Prof. Dr. Bernhard Humm, Darmstadt University of Applied Sciences. www.fbi.h-da.de/~b.humm. 30.05.20163
Prof. Dr. Bernhard Humm, Darmstadt University of Applied Sciences. www.fbi.h-da.de/~b.humm. 30.05.2016
What is Machine Learning (ML)?
4
Generating a model based on inputs and using it for making decisions or predictions
( rather than programming instructions explicitly )
Agenda
• Overview
• ML Applications
• ML Tasks
• ML Approaches
• Methodology
• ML Tools
• Services / Product Map
Prof. Dr. Bernhard Humm, Darmstadt University of Applied Sciences. www.fbi.h-da.de/~b.humm. 30.05.20165
Prof. Dr. Bernhard Humm, Darmstadt University of Applied Sciences. www.fbi.h-da.de/~b.humm. 30.05.2016
Applications of ML:Spam filtering
• Task: classify new e-mails as spam or not spam
6
Spam filter
New e-mails
Automaticallyclassified
Manuallyclassified
Corrections
ML input
Prof. Dr. Bernhard Humm, Darmstadt University of Applied Sciences. www.fbi.h-da.de/~b.humm. 30.05.2016
Stock market analysis
• Task: make recommendations on buying and selling stocks
7
Prediction
Current stock values
History ofstock values
ML input
Recommendation
Decision
Image source: Wikimedia
Prof. Dr. Bernhard Humm, Darmstadt University of Applied Sciences. www.fbi.h-da.de/~b.humm. 30.05.2016
Detecting credit card fraud
• Task: Detect fraud in credit card payments
8
Fraud detection
CC payments
Automaticallyclassified
Manuallyclassified
Corrections
ML input
Prof. Dr. Bernhard Humm, Darmstadt University of Applied Sciences. www.fbi.h-da.de/~b.humm. 30.05.2016
Recommender systems
• Task: Recommending customers suitable products
9
Recommender system
Order
Recommendationof related products
ML input
Purchasing behaviourof other customersor customer groups
Agenda
• Overview
• ML Applications
• ML Tasks
• ML Approaches
• Methodology
• ML Tools
• Services / Product Map
Prof. Dr. Bernhard Humm, Darmstadt University of Applied Sciences. www.fbi.h-da.de/~b.humm. 30.05.201610
Prof. Dr. Bernhard Humm, Darmstadt University of Applied Sciences. www.fbi.h-da.de/~b.humm. 30.05.2016
Categories of ML tasks
• P.S. Other categorizations / groupings are possible
11
Machine Learning Task
SupervisedLearning
UnsupervisedLearning
ReinforcementLearning
Classifi-cation
Regression ClusteringFeature
selection / extraction
Topic modeling
Prof. Dr. Bernhard Humm, Darmstadt University of Applied Sciences. www.fbi.h-da.de/~b.humm. 30.05.2016
Categories of ML tasks
• Given: Example inputs and desired outputs
• Goal: Learn a general rule that maps inputs to outputs
Supervisedlearning
• Given: Data inputs (e.g., documents)
• Goal: Find structure in the inputs
Unsupervisedlearning
• Setting: An agent interacts with a dynamic environment in which it must perform a goal
• Goal: Improving the agent‘s behaviour
Reinforcement learning
12
Prof. Dr. Bernhard Humm, Darmstadt University of Applied Sciences. www.fbi.h-da.de/~b.humm. 30.05.2016
Supervised learning subcategories
• Given: Training inputs (records) which aredivided into two or more classes
• Goal: Produce model to classify new inputs
• Examples: spam filter, fraud detection, …
Classification
• Given: Training data (records) withcontinuous (not discrete) output values
• Goal: Produce model to predict outputvalues for new inputs
• Example: stock value prediction
Regression
13
Prof. Dr. Bernhard Humm, Darmstadt University of Applied Sciences. www.fbi.h-da.de/~b.humm. 30.05.2016
Unsupervised learning subcategories
•Given: Set of input records
•Goal: Identifying clusters (groups of similar records)
•Example: Customer groupingClustering
•Given: Set of input records with attributes („features“)
•Goal: Find a subset of the original attributes that areequally well suited for classification / clustering tasks
Feature selection / extraction
•Given: Set of text documents
•Goal: Find abstract topics that occur in severaldocuments and classify documents accordingly
Topic modeling
14
Agenda
• Overview
• ML Applications
• ML Tasks
• ML Approaches
• Methodology
• ML Tools
• Services / Product Map
Prof. Dr. Bernhard Humm, Darmstadt University of Applied Sciences. www.fbi.h-da.de/~b.humm. 30.05.201615
Prof. Dr. Bernhard Humm, Darmstadt University of Applied Sciences. www.fbi.h-da.de/~b.humm. 30.05.2016
Decision Tree Learning
• Used for supervised learning
(classification, regression)
• Training input: Training data
(records) with output values
(discrete or continuous)
• Learning result: decision tree that
allows classifying / predicting output
values of new data records
• Example (figure): Decision tree for
classfying passengers on the Titanic
in survived / died
16 Image source: Wikipedia
Prof. Dr. Bernhard Humm, Darmstadt University of Applied Sciences. www.fbi.h-da.de/~b.humm. 30.05.2016
Artificial Neural Networks (ANN)
• Inspired by brain / nervous system:
- Neurons connected via dentrites
- Reduce resistance if fired repeatedly
• Artificial Neuron:
- Weighted inputs
- Function, e.g., weighted sum
- Filter, e.g, threshold output
• Artificial Neural Network (ANN):
- Input layer, output layer, and possibly
intermediate layers of neurons
- Training phase: weights are adjusted via
known cases
- Regognition phase: output is produced for
new cases
17 Source: Ivan Galkin, U. MASS Lowell ( http://ulcar.uml.edu/~iag/CS/Intro-to-ANN.html )
Prof. Dr. Bernhard Humm, Darmstadt University of Applied Sciences. www.fbi.h-da.de/~b.humm. 30.05.2016
Deep learning
18
• Cascade of many layers for feature extraction and transformation
• Levels form a hierarchy of concepts.
• Each successive layer uses the output from the previous layer as
input
• Applications include feature selection / extraction (unsupervised)
and classification (supervised).
• ANNs are often used, but
other approaches are possible,
too
Prof. Dr. Bernhard Humm, Darmstadt University of Applied Sciences. www.fbi.h-da.de/~b.humm. 30.05.2016
BayesianNetworks
• Directed acyclic graph (DAG) with:
- Nodes: random variables
+ probability function
- Edges: conditional
dependencies
• Example:
- Causes, diseases, symptoms
• Bayes Network inference allows answering questions like:
- What is the probability of a lung disease in case of a cough?
19
Source: Goodman & Tenenbaum https://probmods.org/
Prof. Dr. Bernhard Humm, Darmstadt University of Applied Sciences. www.fbi.h-da.de/~b.humm. 30.05.2016
Inductive Logic Programming
• Given:
- Set of logic facts (background knowledge), e.g.
male(Tom), female(Eve), parent (Tom, Eve)
- Positive and / or negative examples, e.g.,
daughter (Eve, Tom)
• Learning goal:
- General rules that are consistent with the examples and the
background knowledge, e.g.,
parent(p1, p2) and female(p2) daughter(p2, p1)
20
George
TomMary
Helen
Nancy
Eve
parent
male female
Agenda
• Overview
• ML Applications
• ML Tasks
• ML Approaches
• Methodology
• ML Tools
• Services / Product Map
Prof. Dr. Bernhard Humm, Darmstadt University of Applied Sciences. www.fbi.h-da.de/~b.humm. 30.05.201621
Evaluation,Planning exams
Prof. Dr. Bernhard Humm, Darmstadt University of Applied Sciences. www.fbi.h-da.de/~b.humm. 30.05.2016
Training / test data setsin supervised machine learning
1. Obtain a set of data with input and output values,
e.g., manually classifying paintings as portraits, still lifes,
landscapes, etc.
2. Separate the data set into two
disjoint subsets:
a. Training set
b. Test set
3. Apply machine learning, e.g., train an Artificial Neural Network
(ANN) with the training set
4. Feed the ANN with the test set (omitting the ouput values) and
collect the output values computed by the ANN
5. Compare the computed output values with the expected ones and
compute measures like precision, recall, and F-Measure22
Source: Wikipedia
Prof. Dr. Bernhard Humm, Darmstadt University of Applied Sciences. www.fbi.h-da.de/~b.humm. 30.05.2016
The problem of overfitting
• It is easy to optimize precision, recall, and F-measure for the
training set
• A model that simply memorizes all data points will leads to an
F-Measure of 100% on the
training set
• However, the F-Measure
for the test set will be worse
• Memorizing is an example of
overfitting:
the model has too many
parameters (is too complex)
relative to the number of training data points
• Find the appropriate level of abstraction for the problem domain
23
Image Source: Wikipedia
Prof. Dr. Bernhard Humm, Darmstadt University of Applied Sciences. www.fbi.h-da.de/~b.humm. 30.05.2016
Process Overview
24
Agenda
• Overview
• ML Applications
• ML Tasks
• ML Approaches
• Methodology
• ML Tools
• Services / Product Map
Prof. Dr. Bernhard Humm, Darmstadt University of Applied Sciences. www.fbi.h-da.de/~b.humm. 30.05.201625
Prof. Dr. Bernhard Humm, Darmstadt University of Applied Sciences. www.fbi.h-da.de/~b.humm. 30.05.2016
RapidMiner example:Customer segregation
26
https://rapidminer.com/
Prof. Dr. Bernhard Humm, Darmstadt University of Applied Sciences. www.fbi.h-da.de/~b.humm. 30.05.2016
Analyzing the data
27
Prof. Dr. Bernhard Humm, Darmstadt University of Applied Sciences. www.fbi.h-da.de/~b.humm. 30.05.2016
Process for generating a decision tree
28
Prof. Dr. Bernhard Humm, Darmstadt University of Applied Sciences. www.fbi.h-da.de/~b.humm. 30.05.2016
Decisiontree
29
Prof. Dr. Bernhard Humm, Darmstadt University of Applied Sciences. www.fbi.h-da.de/~b.humm. 30.05.2016
Using the decision tree forcustomer segregation
30
Prof. Dr. Bernhard Humm, Darmstadt University of Applied Sciences. www.fbi.h-da.de/~b.humm. 30.05.2016
Validation
31
Prof. Dr. Bernhard Humm, Darmstadt University of Applied Sciences. www.fbi.h-da.de/~b.humm. 30.05.2016
Precision, recall, accuracy
32
Agenda
• Overview
• ML Applications
• ML Tasks
• ML Approaches
• Methodology
• ML Tools
• Services / Product Map
Prof. Dr. Bernhard Humm, Darmstadt University of Applied Sciences. www.fbi.h-da.de/~b.humm. 30.05.201637
Prof. Dr. Bernhard Humm, Darmstadt University of Applied Sciences. www.fbi.h-da.de/~b.humm. 30.05.2016
ML Services Map
38
Machine learning libraries
Machine learning web services
Machine learning development environments / frameworks
IDEs and frameworksfor experimenting with
different ML approaches and
configuring solutions
Web services for forexperimenting with
different ML approaches and
configuring solutions
Algorithms for classification, regression, clustering, feature selection / extraction, topic modeling, etc. using different approaches, e.g., decisiontree learning, Artificial Neural Networks, Bayes networks, inductive logic
programming, Support Vector machines, Hidden Markov Chains, etc.
Prof. Dr. Bernhard Humm, Darmstadt University of Applied Sciences. www.fbi.h-da.de/~b.humm. 30.05.2016
ML Product Map
39
Machine learning libraries
Machine learning web services
Machine learning development environments / frameworks
Google Prediction API, Microsoft Azure Machine Learning, bigml,
wise.io, procog, ersatz, …
TensorFlow, DL4J, Torch, Caffeee, Theano, Eblearn, OpenNN, aisolver,
CURRENNT, …
SPSS Modeler, RapidMiner, WEKA, Orange, Shogun, scikt-learn, …
Prof. Dr. Bernhard Humm, Darmstadt University of Applied Sciences. www.fbi.h-da.de/~b.humm. 30.05.2016
ML product table
40
Product Library IDE / Framework Web service
aisolver *
bigml *
Caffee *
CURRENNT *
DL4J *
eblearn *
Encog *
ersatz *
Fast Artificial Neural Network Library *
Google Prediction API *
Jaden * *
Java Neural Network Framework Neuroph * *
Joone * *
Microsoft Azure Machine Learning *
OpenNN - Open Neural Networks Library *
Prof. Dr. Bernhard Humm, Darmstadt University of Applied Sciences. www.fbi.h-da.de/~b.humm. 30.05.2016
ML product table (cont‘d)
41
Product Library IDE / Framework Web service
Orange * *
procog *
RapidMiner * *
scikit-learn * *
Shogun * *
SPSS Modeler * *
TensorFlow *
Theano *
Torch *
WEKA * *
wise.io *