Embed Size (px)
DESCRIPTION
How do you go from writing rules for classifying data to using powerful machine learning algorithms to do the same in easy comprehensible steps.
Citation preview
From Linguistic Rules to Machine Learning
Cohan Sujay CarlosAiaioo Labs
Bangalore, [email protected]
What is a Classifier?
A machine learning tool used to apply a label to data.
Classification used in Text Categorization
The UN Security Council adopts its first clear condemnation of Syria for its continuing crackdown on protests, as the army continues its advance into Hama.
Warwickshire's Clarke equalled the first-class record of seven catches for an outfielder in an innings but Lancashire took control on day three.
Politics Sports
How to Build a Classifier for Text Categorization
The UN Security Council adopts its first clear condemnation of Syria for its continuing crackdown on protests, as the army continues its advance into Hama.
Warwickshire's Clarke equalled the first-class record of seven catches for an outfielder in an innings but Lancashire took control on day three.
How do you tell which label (Politics/Sports) is suitable?
In this case it’s only wordsthat you will need!
Classification used in Text Categorization
The UN Security Council adopts its first clear condemnation of Syria for its continuing crackdown on protests, as the army continues its advance into Hama.
Warwickshire's Clarke equalled the first-class record of seven catches for an outfielder in an innings but Lancashire took control on day three.
See the words?
Rule-Based Text Categorization
UN Security CouncilAdoptsCondemnationSyriaCrackdownProtestsArmyHama
WarwickshireClarkeFirst-classRecordCatchesOutfielderInningsLancashire
Gazetteers (word lists)
So you can just use word lists for classification?Yeah, but they won’t work very well.Can you see why word lists alone won’t work very well?
How can you go:
from the starting point (word lists)
to a really cool classification algorithm
All you need is weights!
Rule-Based to Naïve Bayesian
Rule-Based with Weights
UNAdoptsCondemnationSyriaCrackdownProtestsArmyHama
WarwickshireClarkeFirst-classRecordCatchesOutfielderInningsLancashire
Let’s improve the gazetteers with weights
Politics Sports
1.00.10.20.30.81.00.81.0
0.30.10.60.30.61.00.90.5
These weights are nothing but P(Category|Word).
Rule-Based with Weights
UNAdoptsCondemnationSyriaCrackdownProtestsArmyHama
WarwickshireClarkeFirst-classRecordCatchesOutfielderInningsLancashire
Let’s improve the gazetteers with weights
Politics Sports
1.00.10.20.30.81.00.81.0
0.30.10.60.30.61.00.90.5
P(Politics|Word) P(Sports|Word)
Rule-Based with Weights
UNAdoptsCondemnationSyriaCrackdownProtestsArmyHama
P(Politics|UN)P(Politics|Adopts)P(Politics|Condemnation)P(Politics|Syria)P(Politics|Crackdown)P(Politics|Protests)P(Politics|Army)P(Politics|Hama)
Politics
1.00.10.20.30.81.00.81.0
Rule-Based with Weights
UNAdopts
P(Politics | “UN”)P(Politics | “Adopts”)
Politics
1.00.1
How can you learn these probabilities automatically?
Rule-Based with Weights
UNAdopts
P(Politics | “UN”)P(Politics | “Adopts”)
Politics
1.00.1
How can you learn these probabilities automatically?
Estimation
P(Politics | “UN”) = 20/20
Statistically not a very accurate estimator - denominator is small.
Rule-Based with Weights
UNAdopts
P(Politics | “UN”)P(Politics | “Adopts”)
Politics
1.00.1
How can you learn these probabilities automatically?
Instead you Estimate
P(“UN”|Politics) = 20/40000{ C(“UN” in politics) / C(all words in category politics) }
Statistically this is a better estimator
Rule-Based with Weights
UNAdopts
P(Politics | “UN”)P(Politics | “Adopts”)
Politics
1.00.1
How can you learn these probabilities automatically?
That’s so cool! Time to learn how to do that!
A Naïve Bayesian classifier uses a P(Politics | “UN”) estimate calculated from P(“UN”|Politics).
In other words, we are looking to turnP(F|E) into P(E|F).
There is an equation to do this :P(E|F) = P(F|E) * P(E) / P(F) [Bayesian Inversion]
Use Bayesian Inversion!
So finally … you have …
UNAdopts
P(“UN”|Politics)* P(Politics)/P(“UN”)
Politics
1.00.1
That was easy wasn’t it?!
P(“Adopts”|Politics)*P(Politics)/P(“Adopts”)
These don’t have to be only words. They can be ANY sort of feature (word pairs, syntax).
You have just LearntHow to Build
A Naïve Bayesian ClassifierStarting from Linguistic Rules (Word Lists)
Cohan Sujay CarlosAiaioo Labs
Bangalore, [email protected]
![TuX : Distributed Graph Computation for Machine Learning · for Machine Learning ... 1 Introduction Distributed graph engines, such as Pregel [30], ... Figure 1: Examples of machine](https://img.pdfslide.us/doc/110x75/5b1feb1f7f8b9ad0458b49c8/tux-distributed-graph-computation-for-machine-learning-for-machine-learning.jpg)
