IntroductiontoSemantics

Announcements•  PartofnextTuesday’sclasswillfocusonques6onsaboutthemidterm.Pleasesendtopicsthatyouwouldliketohavere-explained.Onlythosetopicsthatstudentsrequestwillbecovered.

•  Reading:C14J&M,V2:probabilis6cparsingotherthandependencyparsing.

•  ReadC.3ofcourseworkspostfordependencyparsing.•  C.14J&M,V3:anotherreadingondependencyparsing

•  HstudentswhohaveaconflictwithanotherclassforthemidtermreplytoPiazzapostoremailElsbethatect2150@columbia.edu•  Op6onswillbelaterThursdayorMondayevening

SIRI: One shot Question and answer What’s required?

WatsonandjeopardyBangBang"his"Silverhammercamedownuponherhead"

Question-Answering/Dialog• “Match”

Wheredoestheinformationfortheanswerscomefrom?

• Databases• hWp://newyork.citysearch.com

• Theweb

• Dic6onaries

• Inference

OneViewofNLArchitecture

Syntax Semantics Morphology

Knowledge Representation/

Meaning Representation

Application output

OneViewofNLArchitecture

Syntax

Semantics

Morphology

Knowledge Representation/

Meaning Representation

Application output

AnotherView:UnderstandinginWatson• Massiveparallelism

•  Exploitmassiveparallelismintheconsidera6onofmul6pleinterpreta6onsandhypotheses.

• Manyexperts•  Facilitatetheintegra6on,applica6on,andcontextualevalua6onofawiderangeoflooselycoupledprobabilis6cques6onandcontentanaly6cs.

•  Pervasiveconfidenceintegra6on•  Nocomponentcommitstoananswer;allcomponentsproducefeaturesandassociatedconfidences,scoringdifferentques6onandcontentinterpreta6ons.

•  Integrateshallowanddeepknowledge•  Balancetheuseofstrictseman6csandshallowseman6cs,leveragingmanylooselyformedontologies.

WhatisSemantics?• MeaningRepresenta6on

• Seman6cParsingorSeman6cInterpreta6on

• Wordmeaningdisambigua6on

• Entailmentandinference

• Informa6onextrac6on

MeaningRepresentation• Torepresentques6ons

• Torepresentknowledgedrawnfromtext

WhatCanServeasaMeaningRepresentation?

• Anythingthatallowsusto• Answerques6ons(WhatisthebestFrenchrestaurantintheEastVillage?)

• Determinetruth(IsColumbiaUnivinNYC?)• Drawinferences(IfOneDineRestaurantisonthetopflooroftheFreedomTowerandtheFreedomToweristhetallestbuildinginNewYorkCity,thenOneDineisthetopmostrestaurantinNewYorkCity.)

Whatkindsofmeaningdowewanttocapture?

•  Categories/en66es•  Tau,Jane,Asiancuisine,vegetarian

•  Events•  takingataxi,destruc6onofpowergridinPuertoRico

•  Time•  Oct30,nextweek,in2months

•  Aspect•  Kathyknowshowtorun.Kathyisrunning.Kathyrantotherestaurantin5min.

•  Beliefs,DesiresandInten6ons(BDI)

MeaningRepresentations

• Allrepresent‘linguis6cmeaning’of Ihaveacar

andstateofaffairsinsomeworld• Allconsistofstructures,composedofsymbolsrepresen6ngobjectsandrela6onsamongthem•  FOPC:

)}(),(),()({, yCarxyHadThingxSHaverxHavingyx ∧∧∧∃

AStandardRepresentation:Predicate-ArgumentStructure

•  Representsconceptsandrela6onshipsamongthem•  Nounsasconceptsorarguments(red(ball))•  Adjec6ves,adverbs,verbsaspredicates(red(ball))

•  Subcategoriza6on(or,argument)framesspecifynumber,posi6on,andsyntac6ccategoryofarguments•  NPlikesNP•  NPlikesInf-VP•  NPlikesNPInf-VP

Semantic(Thematic)Roles• Subcatframeslinkargumentsinsurfacestructurewiththeirseman6croles• Agent:GeorgehitBill.BillwashitbyGeorge.• Pa6ent:GeorgehitBill.BillwashitbyGeorge.

• Theclaimofatheoryofseman6crolesisthattheseargumentsofpredicatescanbeusefullyclassifiedintoasmallsetofseman6callycontenlulclasses• Andthattheseclassesareusefulforexplaininglotsofthings

Commonsemanticroles•  Agent:ini6atorordoerintheevent•  Pa(ent:affecteden6tyintheevent;undergoestheac6on•  Suekilledtherat.

•  Theme:objectintheeventundergoingachangeofstateorloca6on,orofwhichloca6onispredicated•  Theicemelted

•  Experiencer:feelsorperceivetheevent• Billlikespizza.

•  S(mulus:thethingthatisfeltorperceived

Commonsemanticroles•  Goal:

•  BillrantoCopleySquare.•  Recipient(mayormaynotbedis6nguishedfromGoal):

•  BillgavethebooktoMary.

•  Benefac(ve(maybegroupedwithRecipient):•  BillcookeddinnerforMary.

•  Source:•  Billtookapencilfromthepile.

•  Instrument:•  Billatetheburritowithaplas6cspork.

•  Loca(on:•  BillsitsunderthetreeonWednesdays

CommonsemanticrolesTryforyourself!1.  Thesubmarinesankatroopship.2.  Dorishidthemoneyintheflowerpot.3.  Emmano6cedthestain.4.  Wecrossedthestreet.5.  Theboysclimbedthewall.6.  Thechefcookedagreatmeal.7.  Thecomputerpinpointedtheerror.8.  AmadbulldamagedthefenceonJack’sfarm.9.  ThecompanywrotemealeWer.10. Jackopenedthelockwithapaperclip.

Linkingofthematicrolestosyntacticpositions• Johnopenedthedoor• AGENTTHEME• ThedoorwasopenedbyJohn• THEMEAGENT• Thedooropened• THEME• Johnopenedthedoorwiththekey• AGENTTHEMEINSTRUMENT

DeeperSemantics• FromtheWSJ…

• Hemeltedherreservewithahusky-voicedpaeantohereyes.

•  Ifwelabelthecons6tuentsHeandherreserveastheMelterandMelted,thenthoselabelsloseanymeaningtheymighthavehad.

•  IfwemakethemAgentandThemethenwecandomoreinference.

SelectionalRestrictions• Selec6onalRestric6ons:constraintsonthetypesofargumentsverbstakeGeorgeassassinatedthesenator.*Thespiderassassinatedthefly.assassinate:inten5onal(poli5cal?)killing

• Theastronautmarriedthestar.

Problems• Whatexactlyisarole?• What’stherightsetofroles?• Aresuchrolesuniversals?• Aretheserolesatomic?

•  I.e.Agents•  Animate,Voli6onal,Directcausers,etc

• Canweautoma6callylabelsyntac6ccons6tuentswiththema6croles?

FirstOrderPredicateCalculus• Notidealasameaningrepresenta6onanddoesn'tdoeverythingwewant--butbeWerthanmany…•  Supportsthedetermina6onoftruth•  Supportscomposi6onalityofmeaning•  Supportsques6on-answering(viavariables)•  Supportsinference

AbstractMeaningRepresentation(AMR)• Aimingtocaptureadeeplevelofseman6cs

• Originalgoal:asinterlinguaformachinetransla6on

• Alsonowsee:AMRparsers,genera6onfromAMR,summariza6onusingAMR

Knight, AMR Release 1.0, 2014

TemporalRepresentations• Howdowerepresent6meandtemporalrela6onshipsbetweenevents?ItseemsonlyyesterdaythatMarthaStewartwasinprisonbutnowshehasapopularTVshow.Thereisnojus6ce.

• Wheredowegettemporalinforma6on?• Verbtense•  Temporalexpressions•  Sequenceofpresenta6on

• Linearrepresenta6ons:Reichenbach‘47

RepresentingTime

Example from Russell and Norvig

RepresentingTime

Example from Jurafsky and Martin

Utterance time (U): when the utterance occurs Reference time (R): the temporal point-of-view of the utterance Event time (E): when events described in the utterance occur

VerbsandEventTypes:Aspect•  Sta6ves:statesorproper6esofobjectsatapar6cularpointin6me

Iamhungry.• Ac6vi6es:eventswithnoclearendpoint

Iamea-ng.• Accomplishments:eventswithdura6onsandendpointsthatresultinsomechangeofstate

Iatedinner.• Achievements:eventsthatchangestatebuthavenopar6culardura6on–theyoccurinaninstant

Igotthebill.

Beliefs,DesiresandIntentions• Veryhardtorepresentinternalspeakerstateslikebelieving,knowing,wan6ng,assuming,imagining•  NotwellmodeledbyasimpleDBlookupapproachso..

•  Truthintheworldvs.truthinsomepossibleworldGeorgeimaginedthathecoulddance.Georgebelievedthathecoulddance.

• AugmentFOPCwithspecialmodaloperatorsthattakelogicalformulaeasarguments,e.g.believe,know

Believes(George,dance(George))Knows(Bill,Believes(George,dance(George)))

• Mutualbelief:IbelieveyoubelieveIbelieve….• Prac6calimportance:modelingbeliefindialogue

• Clark’sgrounding

TakeAway• Seman6csmeansmanythings

• Meaningrepresenta6ons•  Seman6cparsingorseman6crolelabeling•  Determiningwordmeaning•  Useofseman6cinforma6onforanendapplca6on•  Inference•  Temporalrela6ons:tense,aspect•  Informa6onextrac6on•  Belief,desireandintent

• Workinseman6cscanfallintoanyoftheseareas

LexicalSemantics• Groundingforrepresenta6onofwordmeaning• Usedforworddisambigua6on•  Seman6cinterpreta6on

• Pre-cursortorepresenta6onsusedindistribu6onalseman6cs,vectorrepresenta6onsandneuralnets

• Topics• Homonymy,Polysemy,Synonymy• Onlineresources:WordNet

Preliminaries• What’saword?

• Defini6onswe’veusedovertheclass:Types,tokens,stems,roots,inflectedforms,etc...

•  Lexeme:Anentryinalexiconconsis6ngofapairingofaformwithasinglemeaningrepresenta6on

•  Lexicon:Acollec6onoflexemes

Relationshipsbetweenwordmeanings• Homonymy• Polysemy• Synonymy• Antonymy• Hypernomy• Hyponomy• Meronomy

Homonymy• Homonymy:

•  Lexemesthatshareaform•  Phonological,orthographicorboth

• Buthaveunrelated,dis6nctmeanings• Clearexamples:

•  Bat(woodens6ck-likething)vs•  Bat(flyingscarymammalthing)•  Orbank(financialins6tu6on)versusbank(riverside)

• Canbehomophones,homographs,orboth:•  Homophones:

•  Writeandright•  Pieceandpeace

HomonymycausesproblemsforNLPapplications• Text-to-Speech

•  Sameorthographicformbutdifferentphonologicalform•  bassvsbass

• Informa6onretrieval• Differentmeaningssameorthographicform

•  QUERY:batcare• MachineTransla6on• Speechrecogni6on

• Why?

Polysemy• ThebankisconstructedfromredbrickIwithdrewthemoneyfromthebank

• Arethosethesamesense?• OrconsiderthefollowingWSJexample

• Whilesomebanksfurnishspermonlytomarriedwomen,othersarelessrestric6ve

• Whichsenseofbankisthis?•  Isitdis6nctfrom(homonymouswith)theriverbanksense?

•  Howaboutthesavingsbanksense?

Polysemy• Asinglelexemewithmul6plerelatedmeanings(bankthebuilding,bankthefinancialins6tu6on)

• Mostnon-rarewordshavemul6plemeanings•  Thenumberofmeaningsisrelatedtoitsfrequency

• Verbstendmoretopolysemy• Dis6nguishingpolysemyfromhomonymyisn’talwayseasy(ornecessary)

MetaphorandMetonymy• Specifictypesofpolysemy• Metaphor:

• GermanywillpullSloveniaoutofitseconomicslump.

•  Ispent2hoursonthathomework.• Metonymy

•  TheWhiteHouseannouncedyesterday.•  Thischaptertalksaboutpart-of-speechtagging

• Bank(building)andbank(financialins6tu6on)

Howdoweknowwhenawordhasmorethanonesense?• ATISexamples

• Whichflightsservebreakfast?• DoesAmericaWestservePhiladelphia?

• The“zeugma”test:

•  ?DoesUnitedservebreakfastandSanJose?

Synonyms• Wordthathavethesamemeaninginsomeorallcontexts.•  filbert/hazelnut•  couch/sofa•  big/large•  automobile/car•  vomit/throwup•  Water/H20

• Twolexemesaresynonymsiftheycanbesuccessfullysubs6tutedforeachotherinallsitua6ons•  Ifsotheyhavethesameproposi(onalmeaning

Synonyms• Buttherearefew(orno)examplesofperfectsynonymy.• Whyshouldthatbe?•  Evenifmanyaspectsofmeaningareiden6cal•  S6llmaynotpreservetheacceptabilitybasedonno6onsofpoliteness,slang,register,genre,etc.

• Example:• WaterandH20

Somemoreterminology•  Lemmasandwordforms

•  Alexemeisanabstractpairingofmeaningandform•  Alemmaorcita(onformisthegramma6calformthatisusedtorepresentalexeme.

•  Carpetisthelemmaforcarpets•  Dormiristhelemmaforduermes.

•  Specificsurfaceformscarpets,sung,duermesarecalledwordforms

•  Thelemmabankhastwosenses:•  Instead,abankcanholdtheinvestmentsinacustodialaccountintheclient’sname

•  Butasagricultureburgeonsontheeastbank,theriverwillshrinkevenmore.

• Asenseisadiscreterepresenta6onofoneaspectofthemeaningofaword

Synonymyisarelationbetweensensesratherthanwords

• Considerthewordsbigandlarge• Aretheysynonyms?

•  Howbigisthatplane?• WouldIbeflyingonalargeorsmallplane?

• Howabouthere:•  MissNelson,forinstance,becameakindofbigsistertoBenjamin.

•  ?MissNelson,forinstance,becameakindoflargesistertoBenjamin.

• Why?•  bighasasensethatmeansbeingolder,orgrownup•  largelacksthissense

Antonyms• Sensesthatareoppositeswithrespecttoonefeatureoftheirmeaning

• Otherwise,theyareverysimilar!•  dark/light•  short/long•  hot/cold•  up/down•  in/out

• Moreformally:antonymscan•  defineabinaryopposi6onoratoppositeendsofascale(long/short,fast/slow)

•  Bereversives:rise/fall,up/down

Hyponymy• Onesenseisahyponymofanotherifthefirstsenseismorespecific,deno6ngasubclassoftheother•  carisahyponymofvehicle•  dogisahyponymofanimal•  mangoisahyponymoffruit

•  Conversely•  vehicleisahypernym/superordinateofcar•  animalisahypernymofdog•  fruitisahypernymofmango

superordinate vehicle fruit furniture mammal

hyponym car mango chair dog

Hypernymymoreformally• Extensional:

•  Theclassdenotedbythesuperordinate•  extensionallyincludestheclassdenotedbythehyponym

• Entailment:• AsenseAisahyponymofsenseBifbeinganAentailsbeingaB

• Hyponymyisusuallytransi6ve•  (AhypoBandBhypoCentailsAhypoC)

• Whywouldhypernyms/hyponymsbeimportanttoconstruc6ngameaningrepresenta6on?

II.WordNet• Ahierarchicallyorganizedlexicaldatabase• On-linethesaurus+aspectsofadic6onary

•  Versionsforotherlanguagesareunderdevelopment

Category Unique Forms

Noun 117,097

Verb 11,488

Adjective 22,141

Adverb 4,601

WordNet

•  Whereitis:•  hWps://wordnet.princeton.edu/

FormatofWordnetEntries

WordNetNounRelations

WordNetVerbRelations

WordNetHierarchies

Howis“sense”deYinedinWordNet?• Thesetofnear-synonymsforaWordNetsenseiscalledasynset(synonymset);it’stheirversionofasenseoraconcept

• Example:chumpasanountomean•  ‘apersonwhoisgullibleandeasytotakeadvantageof’

•  Eachofthesesensessharethissamegloss•  ThusforWordNet,themeaningofthissenseofchumpisthislist.

NextThursday:Wordsensedisambigua6onTuesday:Seman6cparsingandmidterm