CS460/626 : Natural Language Processing/Speech NLP and the WebProcessing/Speech, NLP and the Web

(Lecture 4 Wordnet and Word Sense Disambiguation cntd)Disambiguation, cntd)

Pushpak BhattacharyyaPushpak BhattacharyyaCSE Dept., IIT Bombay

11th J 201111th Jan, 2011

Foundation of Wordnet: Lexical MatrixMatrix

Meaning-form relationshipMeaning form relationshipMeanings container Word forms container

M1

M2W1

W2M2

M3

.

W2

W3

.

Mk-1

.

.

Wk 1

.

.

Mk 1Mk

Wk-1Wk

Many to many relationship

Meaning-formMeaning-formg

Homonymy (accidental identity or word borrowing)

Polysemy(shades of ( y g) (meaning)Eg: Fall

1. The kingdom fellfell

2. The fruit fellHomography(same picture)

Eg: river bank and f

Homophony(same sound)Eg: write and

financial bank right

Distributional Similarity

Words which are semantically similar tend to appear in syntactically similartend to appear in syntactically similar contexts.The neighbors of the words tend to beThe neighbors of the words tend to be the same.Technically known as DistributionalTechnically known as Distributional Similarity.

Synset+Gloss+ExampleCrucially needed for concept explication, wordnet building using

another wordnet and wordnet linking.

English Synset: {earthquake, quake, temblor, seism} -- (shaking and vibration at the surface of the earth resulting from underground movement along a fault plane of from volcanic activity)y)

Hindi Synset: , , , , , -, - , , , - - " "

(shaking of the surface of earth; many were killed in the earthquake in Gujarat)

Marathi Synset: , - " "

Semantic Relations

Hypernymy and HyponymyRelation between word senses (synsets)Relation between word senses (synsets)X is a hyponym of Y if X is a kind of YHyponymy is transitive and asymmetricalHyponymy is transitive and asymmetricalHypernymy is inverse of Hyponymy

(li > i l > i t tit > tit )(lion->animal->animate entity->entity)

Semantic Relations (continued)

Meronymy and HolonymyPart-whole relation branch is a part of treePart whole relation, branch is a part of treeX is a meronymy of Y if X is a part of YHolonymy is the inverse relation ofHolonymy is the inverse relation of Meronymy

{kitchen} {house}{kitchen} . {house}

Lexical Relation

AntonymyOppositeness in meaningOppositeness in meaning Relation between word formsOften determined by phonetics wordOften determined by phonetics, word length etc. ({rise, ascend} vs. {fall, descend})})

WordNet Sub-Graph

Hyponymy

WordNet Sub Graph

Dwelling,abode

kitchenMeronymy

Hypernymy

Hyponymy

bedroombckyard

Me

Gloss

Hyponymy

house,homeA place that serves as the living quarters of one or mor efamilies

veranda

ronymy

study

y

guestroom hermitage cottage

Troponym and Entailment

Entailment{snoring sleeping}{snoring sleeping}

TroponymTroponym{limp, strut walk}{whisper talk}{whisper talk}

Entailment

Snoring entails sleeping.Buying entails paying.

Proper Temporal Inclusion. p pInclusion can be in any way.

Sleeping temporally includes snoring.p g p y gBuying temporally includes paying.

Co-extensiveness (Troponymy)Co extensiveness. (Troponymy)Limping is a manner of walking.

Opposition among verbs.

{Rise,ascend} {fall,descend}Tie-untie (do-undo)Tie-untie (do-undo)

Walk-run (slow,fast)Teach-learn (same activity different perspective)Rise fall (motion upward or downward)Rise-fall (motion upward or downward)

Opposition and Entailment.Hit or miss (entail aim) . Backward presupposition.Succeed or fail (entail try.)

The causal relationship.Show- see.Give- have.

Causation and Entailment. Giving entails having. Feeding entails eating.

Kinds of AntonymyKinds of AntonymySizeSize SmallSmall -- BigBigS eS e Small Small BigBigQualityQuality Good Good BadBadStateState WarmWarm CoolCoolStateState Warm Warm CoolCoolPersonalityPersonality Dr. JekylDr. Jekyl-- Mr. HydeMr. HydeDirectionDirection EastEast WestWestDirectionDirection EastEast-- WestWestActionAction Buy Buy SellSellAmo ntAmo nt LittlLittl A l tA l tAmountAmount Little Little A lotA lotPlacePlace Far Far NearNearTimeTime Day Day -- NightNightGenderGender Boy Boy -- GirlGirl

Kinds of MeronymyKinds of MeronymyComponentComponent--objectobject Head Head -- BodyBodyStaffStaff objectobject WoodWood TableTableStaffStaff--objectobject Wood Wood -- TableTable

MemberMember--collectioncollection Tree Tree -- ForestForest

FeatureFeature--ActivityActivity Speech Speech -- ConferenceConference

PlacePlace--AreaArea Palo AltoPalo Alto -- CaliforniaCaliforniaPlacePlace AreaArea Palo Alto Palo Alto CaliforniaCalifornia

PhasePhase--StateState Youth Youth -- LifeLife

ResourceResource--processprocess Pen Pen -- WritingWriting

ActorActor--ActAct Physician Physician --yyTreatmentTreatment

GradationStateState Childhood, Youth, Old Childhood, Youth, Old

ageage

TemperatureTemperature Hot, Warm, ColdHot, Warm, Cold

ActionAction Sleep Doze WakeSleep Doze WakeActionAction Sleep, Doze, WakeSleep, Doze, Wake

Metonymy

Associated with Metaphors which are epitomes of semanticsepitomes of semanticsOxford Advanced Learners Dictionary definition: The use of a word or phrasedefinition: The use of a word or phrase to mean something different from the literal meaningliteral meaningDoes it mean Careless Usage?!

Insight from Sanskritic Tradition

Power of a wordAbhidha Lakshana VyanjanaAbhidha, Lakshana, Vyanjana

Meaning of Hall:The hall is packed (avidha)The hall is packed (avidha)The hall burst into laughing (lakshana)Th H ll i f ll ( id d tThe Hall is full (unsaid: and so we cannot enter) (vyanjana)

Metaphors in Indian Tradition

upamana and upameya Former: object being comparedFormer: object being comparedLatter: object being compared withPuru was like a lion in the battle withPuru was like a lion in the battle with Alexander (Puru: upameya; Lion: upamana)p )

Upamana, rupak, atishayoktiupamana: Explicit comparison

Puru was like a lion in the battle with Alexander

rupak: Implicit comparisonPuru was a lion in the battle with Alexanderh k ( )Atishayokti (exaggeration): upamana

and upameya droppedP fl d B t th li f htPurus army fled. But the lion fought on.

Modern study (1956 onwards, y ( ,Richards et. al.)

Three constituents of metaphorVehicle (items used metaphorically)Tenor (the metaphorical meaning of the former)Tenor (the metaphorical meaning of the former)Ground (the basis for metaphorical extension)

The foot of the mountainVehicle: :footTenor: lower portionGround: spatial parallel between the relationshipGround: spatial parallel between the relationship between the foot to the human body and the lower portion of the mountain with the rest of the mountain

Interaction of semantic fieldsInteraction of semantic fields(Haas)

Core vs. peripheral semantic fieldsInteraction of two words in metonymicInteraction of two words in metonymic relation brings in new semantic fields with selective inclusion of featureswith selective inclusion of featuresLeg of a table

D t t t hDoes not stretch or moveDoes stand and support

Lakoffs (1987) contribution

Source DomainTarget DomainTarget DomainMapping Relations

Mapping Relations: ontological pp g gcorrespondences

Anger is heat of fluid in

HeatHeat(i) Container(i) Container(ii) Agitation of (ii) Agitation of

AngerAngerBodyBodyAgitation of Agitation of

container( ) g tat o o( ) g tat o ofluidfluid(iii) Limit of (iii) Limit of resistenceresistence

g tat o og tat o omindmindLimit of ability Limit of ability to suppressto suppress

(iv) Explosion(iv) Explosionpppp

Loss of controlLoss of control

Image SchemasCategories: Container ContainedQuantity

M i l i d O t tMore is up, less is down: Outputs rose dramatically; accidents rates were lowerLinear scales and paths: Ram is by far the best pe fo meperformer

TimeStationary event: we are coming to exam timey gStationary observer: weeks rush by

Causation: desperation drove her to extreme stepssteps

Patterns of Metonymy

Container for containedThe kettle boiled (water)( )

Possessor for possessed/attributeWhere are you parked? (car)y p ( )

Represented entity for representativeThe government will announce new targetsThe government will announce new targets

Whole for partI am going to fill up the car with petrolI am going to fill up the car with petrol

Patterns of Metonymy (contd)

Part for wholeI noticed several new faces in the classI noticed several new faces in the class

Place for institutionLalbaug witnessed the largest GanapatiLalbaug witnessed the largest Ganapati

Q ti C h t t tQuestion: Can you have part-part metonymy

Purpose of MetonymyMore idiomatic/natural way of expression

More natural to say the kettle is boiling as opposed to the water in the kettle is boilingopposed to the water in the kettle is boiling

EconomyRoom 23 is answering (but not *is asleep)

Ease of access to referentHe is in the phone book (but not *on the back of my hand)y )

Highlighting of associated relationThe car in the front decided to turn right (but not *to smoke a cigarette)to smoke a cigarette)

Feature sharing not necessary

In a restaurant:Jalebii ko abhi dudh chaiye (no featureJalebii ko abhi dudh chaiye (no feature sharing)The elephant now wants some coffeeThe elephant now wants some coffee (feature sharing)

Proverbs

Describes a specific event or state of affairs which is applicableaffairs which is applicable metaphorically to a range of events or states of affairs provided they have thestates of affairs provided they have the same or sufficiently similar image-schematic structureschematic structure

WSD APPROACHESWSD APPROACHES

WSD Problem Definition

Obtain the sense ofA set of target words or ofA set of target words, or ofAll words (all word WSD, more difficult)

Against aAgainst aSense repository (like the wordnet), orA th ( t d t dA thesaurus (not same as wordnet, does not have semantic relations)

U i thUsing theContext in which the word appears.

Example word: operationoperation ((computer science) data processing in which the result is completely specified by a rule (especially the processing that results from a single instruction)) "it can perform millions of operations per second"operation military operation (activity by a military or naval force (as aoperation, military operation (activity by a military or naval force (as a maneuver or campaign)) "it was a joint operation of the navy and air force"operation, surgery, surgical operation, surgical procedure, surgical process (a medical procedure involving an incision with instruments; performed to repair d d l b d ) " h ll h d l hdamage or arrest disease in a living body) "they will schedule the operation as soon as an operating room is available"; "he died while undergoing surgery"mathematical process, mathematical operation, operation ((mathematics) calculation by mathematical methods) "the problems at the end of the chapter demonstrated the mathematical processes involved in the derivation"; "they were learning the basic operations of arithmetic"

KNOWLEDEGE BASED v/s MACHINE LEARNINGBASED v/s HYBRID APPROACHESBASED v/s HYBRID APPROACHES

Knowledge Based ApproachesRely on knowledge resources like WordNet, Thesaurus etc.May use grammar rules for disambiguation.May use hand coded rules for disambiguation.

Machine Learning Based ApproachesRely on corpus evidence.Train a model using tagged or untagged corpus.Probabilistic/Statistical models.Probabilistic/Statistical models.

Hybrid ApproachesUse corpus evidence as well as semantic relations form

36WordNet.

SELECTIONAL PREFERENCES(INDIAN TRADITION)

Desire of some words in the sentence (aakaangksha).I saw the boy with long hair.The verb saw and the noun boy desire an object here.

Appropriateness of some other words in the sentence toAppropriateness of some other words in the sentence to fulfil that desire (yogyataa).

I saw the boy with long hair.The PP with long hair can be appropriately connected only to boy and not sa not saw.

In case, the ambiguity is still present, proximity (sannidhi) can determine the meaning.can determine the meaning.

E.g. I saw the boy with a telescope.The PP with a telescope can be attached to both boy and saw, so ambiguity still present. It is then attached to boy using the proximity check.

3737

SELECTIONAL PREFERENCES(RECENT LINGUISTIC THEORY)

There are words which demand arguments, like, verbs, prepositions, adjectives and sometimes nouns. These arguments are typically nouns.Arguments must have the property to fulfil the demand. They must satisfy selectional preferences.

ExampleGive (verb)

agent animateobj direct obj indirectj

I gave him the bookI gave him the book (yesterday in the school) -> adjunct

How does this help in WSD?38

How does this help in WSD?One type of contextual information is the information about the type of arguments that a word takes.

38

WSD USING SELECTIONAL PREFERENCESAND ARGUMENTS

Sense 1 Sense 2This airlines serves dinner in the evening flight.serve (Verb)

This airlines serves the sector between Agra & Delhi.serve (Verb)

agentobject edible

agentobject sector

Requires exhaustive enumeration of:Argument-structure of verbs.

Selectional preferences of arguments.p g

Description of properties of words such that meeting the selectional preference criteria can be decided.

E.g. This flight serves the region between Mumbai and Delhi

How do you decide if region is compatible with sector3939

How do you decide if region is compatible with sector

OVERLAP BASED APPROACHESRequire a Machine Readable Dictionary (MRD).

Find the overlap between the features of different senses of anambiguous word (sense bag) and the features of the words in itscontext (context bag).

These features could be sense definitions, example sentences, hypernyms etc.

The features could also be given weights.

The sense which has the maximum overlap is selected as the contextually appropriate sense.

4040

LESKS ALGORITHMSense Bag: contains the words in the definition of a candidate sense of the ambiguous word.gContext Bag: contains the words in the definition of each sense of each context word.E.g. On burning coal we get ash.

Sense 1Trees of the olive family with pinnate leaves

Sense 1A piece of glowing carbon or burnt wood.

Ash Coal

Trees of the olive family with pinnate leaves, thin furrowed bark and gray branches.

Sense 2The solid residue left when combustiblematerial is thoroughly burned or oxidized.

p g g

Sense 2charcoal.

Sense 3A black solid combustible substance formed

Sense 3To convert into ash

A black solid combustible substance formed by the partial decomposition of vegetable matter without free access to air and under the influence of moisture and often increased pressure and temperature that is widely used as a fuel for burning

4141g

In this case Sense 2 of ash would be the winner sense.

WALKERS ALGORITHMWALKERS ALGORITHMA Thesaurus Based approach.Step 1: For each sense of the target word find the thesaurus category to p g g ywhich that sense belongs.Step 2: Calculate the score for each sense by using the context words. A context words will add 1 to the score of the sense if the thesaurus category of the word matches that of the sense.

E.g. The money in this bank fetches an interest of 8% per annumTarget word: bankClue words from the context: money, interest, annum, fetch

Sense1: Finance Sense2: Location

Money +1 0Context wordsadd 1 to theMoney +1 0

Interest +1 0

Fetch 0 0

Annum +1 0

sense when the topic of theword matches thatof the sense

42Annum +1 0

Total 3 0