• Home

  • Documents

  • Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task...
97
Lecture 16: Dialogue Alan Ri4er (many slides from Greg Durrett)

Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

  • Upload
    others

  • View
    0

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

Lecture16:Dialogue

AlanRi4er(many slides from Greg Durrett)

Page 2: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

ThisLecture

‣ Chatbotdialoguesystems

‣ Task-orienteddialogue

‣ OtherdialogueapplicaAons

Page 3: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

Chatbots

Page 4: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

TuringTest(1950)‣ ImitaAongame:AandBarelockedinroomsandanswerC’squesAonsviatypewriter.BotharetryingtoactlikeB

Page 5: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

TuringTest(1950)‣ ImitaAongame:AandBarelockedinroomsandanswerC’squesAonsviatypewriter.BotharetryingtoactlikeB

OriginalInterpretaAon:

Page 6: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

TuringTest(1950)‣ ImitaAongame:AandBarelockedinroomsandanswerC’squesAonsviatypewriter.BotharetryingtoactlikeB

A B

C

B B

trainedjudge

OriginalInterpretaAon:

Page 7: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

TuringTest(1950)‣ ImitaAongame:AandBarelockedinroomsandanswerC’squesAonsviatypewriter.BotharetryingtoactlikeB

A B

C

B B

trainedjudge

OriginalInterpretaAon:

Page 8: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

TuringTest(1950)‣ ImitaAongame:AandBarelockedinroomsandanswerC’squesAonsviatypewriter.BotharetryingtoactlikeB

A B

C

B B

trainedjudgeC trainedjudge

OriginalInterpretaAon: StandardInterpretaAon:

Page 9: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

TuringTest(1950)‣ ImitaAongame:AandBarelockedinroomsandanswerC’squesAonsviatypewriter.BotharetryingtoactlikeB

A B

C

B B

trainedjudgeC trainedjudge

OriginalInterpretaAon: StandardInterpretaAon:

‣ Thetestisnot“doesthiscomputerseemhuman-liketorandompeoplewithawebbrowser?”

Page 10: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

ELIZA‣ Created1964-1966atMIT,heavilyscripted

‣ DOCTORscriptwasmostsuccessful:repeatsuser’sinput,asksinanequesAons

Weizenbaum(1966)

Page 11: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

ELIZA

‣ IdenAfykeyword,idenAfycontext,applytransformaAonrule

Page 12: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

ELIZA

(.*)you(.*)me

WhydoyouthinkI$2you?

‣ IdenAfykeyword,idenAfycontext,applytransformaAonrule

Page 13: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

ELIZA

(.*)you(.*)me

WhydoyouthinkI$2you?

‣ IdenAfykeyword,idenAfycontext,applytransformaAonrule

My(.)(.*)

Whatelsecomestomindwhenyou thinkofyour$1?

Page 14: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

ELIZA

(.*)you(.*)me

WhydoyouthinkI$2you?

‣ IdenAfykeyword,idenAfycontext,applytransformaAonrule

‣ Veryli4leneedtogeneratenewcontent,butcanonlyhaveonetypeofconversaAon

My(.)(.*)

Whatelsecomestomindwhenyou thinkofyour$1?

Page 15: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

Cleverbot‣ Carpenter(1986),onlinesystembuiltin2006

User:Hitherecleverbot.Cleverbot:Howareyou?User:Doingokay.whatplansdoyouhavetoday?Cleverbot:Idorevision.User:Whatareyourevisingfor?Cleverbot:Mathshistory.User:Howoldareyou?Cleverbot:That'snoneofyourbusiness.User:Whatismathshistory?Cleverbot:Idon’tknow,farmingmaybe

Page 16: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

Cleverbot‣ Carpenter(1986),onlinesystembuiltin2006

‣ “Nearestneighbors”:humansaysstatementA,findahumanresponseinhuman-humanorhuman-computerchatstostatementA,repeatthat

User:Hitherecleverbot.Cleverbot:Howareyou?User:Doingokay.whatplansdoyouhavetoday?Cleverbot:Idorevision.User:Whatareyourevisingfor?Cleverbot:Mathshistory.User:Howoldareyou?Cleverbot:That'snoneofyourbusiness.User:Whatismathshistory?Cleverbot:Idon’tknow,farmingmaybe

Page 17: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

Cleverbot‣ Carpenter(1986),onlinesystembuiltin2006

‣ “Nearestneighbors”:humansaysstatementA,findahumanresponseinhuman-humanorhuman-computerchatstostatementA,repeatthat

User:Hitherecleverbot.Cleverbot:Howareyou?User:Doingokay.whatplansdoyouhavetoday?Cleverbot:Idorevision.User:Whatareyourevisingfor?Cleverbot:Mathshistory.User:Howoldareyou?Cleverbot:That'snoneofyourbusiness.User:Whatismathshistory?Cleverbot:Idon’tknow,farmingmaybe

‣ Canofengivesensibleanswers,butthebotdoesn’treallyimposehigh-level discoursestructure

Page 18: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

Data-DrivenApproaches‣ CantreatasamachinetranslaAonproblem:“translate”fromcurrentu4erancetonextone

Ri4eretal.(2011)

Page 19: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

Data-DrivenApproaches‣ CantreatasamachinetranslaAonproblem:“translate”fromcurrentu4erancetonextone

Ri4eretal.(2011)

Page 20: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

Data-DrivenApproaches‣ CantreatasamachinetranslaAonproblem:“translate”fromcurrentu4erancetonextone

‣ Filterthedata,usestaAsAcalmeasurestopruneextractedphrasestogetbe4erperformance

Ri4eretal.(2011)

Page 21: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

Data-DrivenApproaches

Ri4eretal.(2011)

Page 22: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

Seq2seqmodels

Whatareyoudoing

I

<s>

am going home [STOP]

‣ JustlikeconvenAonalMT,cantrainseq2seqmodelsforthistask

Page 23: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

Seq2seqmodels

Whatareyoudoing

I

<s>

am going home [STOP]

‣ JustlikeconvenAonalMT,cantrainseq2seqmodelsforthistask

‣Whymightthismodelperformpoorly?Whatmightitbebadat?

Page 24: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

Seq2seqmodels

Whatareyoudoing

I

<s>

am going home [STOP]

‣ JustlikeconvenAonalMT,cantrainseq2seqmodelsforthistask

‣Whymightthismodelperformpoorly?Whatmightitbebadat?

‣ Hardtoevaluate:

Page 25: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

LackofDiversity

Lietal.(2016)

‣ Trainingtomaximizelikelihoodgivesasystemthatpreferscommonresponses:

Page 26: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

LackofDiversity

Lietal.(2016)

‣ SoluAon:mutualinformaAoncriterion;responseRshouldbepredicAveofuseru4eranceUaswell

Page 27: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

LackofDiversity

Lietal.(2016)

‣ SoluAon:mutualinformaAoncriterion;responseRshouldbepredicAveofuseru4eranceUaswell

‣ StandardcondiAonallikelihood: logP (R|U)

Page 28: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

LackofDiversity

Lietal.(2016)

‣ SoluAon:mutualinformaAoncriterion;responseRshouldbepredicAveofuseru4eranceUaswell

‣MutualinformaAon:

‣ StandardcondiAonallikelihood: logP (R|U)

logP (R,U)

P (R)P (U)= logP (R|U)� logP (R)

‣ logP(R)canreflectprobabiliAesunderalanguagemodel

Page 29: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

LackofDiversity

Lietal.(2016)

‣ OpenSubAtlesdata

Page 30: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

Futureofchatbots‣ HowdeepcanaconversaAonbewithoutmoresemanAcgrounding?Basicfactsaren’tevenconsistent…

Lietal.(2016)Persona…

Page 31: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

Futureofchatbots‣ HowdeepcanaconversaAonbewithoutmoresemanAcgrounding?Basicfactsaren’tevenconsistent…

‣ Canforcechatbotstogiveconsistentanswers,butsAllprobablynotveryinteresAng

Lietal.(2016)Persona…

Page 32: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

Futureofchatbots

‣ XiaoIce:MicrosofchatbotinChinese,20Musers,averageuserinteracts60Ames/month

‣ HowdeepcanaconversaAonbewithoutmoresemanAcgrounding?Basicfactsaren’tevenconsistent…

‣ Canforcechatbotstogiveconsistentanswers,butsAllprobablynotveryinteresAng

Lietal.(2016)Persona…

Page 33: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

Futureofchatbots

‣ XiaoIce:MicrosofchatbotinChinese,20Musers,averageuserinteracts60Ames/month

‣ Peopledoseemtoliketalkingtothem…?

‣ HowdeepcanaconversaAonbewithoutmoresemanAcgrounding?Basicfactsaren’tevenconsistent…

‣ Canforcechatbotstogiveconsistentanswers,butsAllprobablynotveryinteresAng

Lietal.(2016)Persona…

Page 34: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

Task-OrientedDialogue

Page 35: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

Task-OrientedDialogue‣ QuesAonanswering/search:

Page 36: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

Task-OrientedDialogue‣ QuesAonanswering/search:

Page 37: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

Task-OrientedDialogue

Google,what’sthemostvaluable

Americancompany?

‣ QuesAonanswering/search:

Page 38: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

Task-OrientedDialogue

Google,what’sthemostvaluable

Americancompany?

Apple

‣ QuesAonanswering/search:

Page 39: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

Task-OrientedDialogue

Google,what’sthemostvaluable

Americancompany?

Apple

WhoisitsCEO?

‣ QuesAonanswering/search:

Page 40: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

Task-OrientedDialogue

Google,what’sthemostvaluable

Americancompany?

Apple

WhoisitsCEO?

TimCook

‣ QuesAonanswering/search:

Page 41: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

Task-OrientedDialogue‣ Personalassistants/APIfront-ends:

Page 42: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

Task-OrientedDialogue

Siri,findmeagoodsushi restaurantinChelsea

‣ Personalassistants/APIfront-ends:

Page 43: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

Task-OrientedDialogue

Siri,findmeagoodsushi restaurantinChelsea

SushiSekiChelseaisasushirestaurantinChelseawith4.4stars

onGoogle

‣ Personalassistants/APIfront-ends:

Page 44: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

Task-OrientedDialogue

Siri,findmeagoodsushi restaurantinChelsea

SushiSekiChelseaisasushirestaurantinChelseawith4.4stars

onGoogle

‣ Personalassistants/APIfront-ends:

Howexpensiveisit?

Page 45: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

Task-OrientedDialogue

Siri,findmeagoodsushi restaurantinChelsea

SushiSekiChelseaisasushirestaurantinChelseawith4.4stars

onGoogle

‣ Personalassistants/APIfront-ends:

Howexpensiveisit?

Entreesarearound$30each

Page 46: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

Task-OrientedDialogue

Siri,findmeagoodsushi restaurantinChelsea

SushiSekiChelseaisasushirestaurantinChelseawith4.4stars

onGoogle

‣ Personalassistants/APIfront-ends:

Howexpensiveisit?

Entreesarearound$30each

Findmesomethingcheaper

Page 47: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

Task-OrientedDialogue‣ Personalassistants/APIfront-ends:

Page 48: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

Task-OrientedDialogue

HeyAlexa,whyisn’tmyAmazon orderhere?

‣ Personalassistants/APIfront-ends:

Page 49: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

Task-OrientedDialogue

HeyAlexa,whyisn’tmyAmazon orderhere?

Letmeretrieveyourorder. Yourorderwasscheduledtoarrive

at4pmtoday.

‣ Personalassistants/APIfront-ends:

Page 50: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

Task-OrientedDialogue

HeyAlexa,whyisn’tmyAmazon orderhere?

Letmeretrieveyourorder. Yourorderwasscheduledtoarrive

at4pmtoday.

‣ Personalassistants/APIfront-ends:

Itnevercame

Page 51: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

Task-OrientedDialogue

HeyAlexa,whyisn’tmyAmazon orderhere?

Letmeretrieveyourorder. Yourorderwasscheduledtoarrive

at4pmtoday.

‣ Personalassistants/APIfront-ends:

Itnevercame

Okay,Icanputyouthroughtocustomerservice.

Page 52: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

AirTravelInformaAonService(ATIS)‣ Givenanu4erance,predictadomain-specificsemanAcinterpretaAon

DARPA(early1990s),FigurefromTuretal.(2010)

‣ CanformulateassemanAcparsing,butsimpleslot-fillingsoluAons(classifiers)workwelltoo

Page 53: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

FullDialogueTask‣ Parsing/languageunderstandingisjustonepieceofasystem

Youngetal.(2013)

Page 54: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

FullDialogueTask‣ Parsing/languageunderstandingisjustonepieceofasystem

Youngetal.(2013)

‣ Dialoguestate:reflectsanyinformaAonabouttheconversaAon(e.g.,searchhistory)

Page 55: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

FullDialogueTask‣ Parsing/languageunderstandingisjustonepieceofasystem

Youngetal.(2013)

‣ Dialoguestate:reflectsanyinformaAonabouttheconversaAon(e.g.,searchhistory)

‣ Useru4erance->updatedialoguestate->takeacAon(e.g.,querytherestaurantdatabase)->saysomething

Page 56: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

FullDialogueTask

Page 57: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

FullDialogueTask

FindmeagoodsushirestaurantinChelsea

Page 58: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

FullDialogueTask

FindmeagoodsushirestaurantinChelsea

restaurant_type <- sushi

Page 59: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

FullDialogueTask

FindmeagoodsushirestaurantinChelsea

restaurant_type <- sushi

location <- Chelsea

Page 60: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

FullDialogueTask

FindmeagoodsushirestaurantinChelsea

restaurant_type <- sushi

location <- Chelsea

curr_result <- execute_search()

Page 61: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

FullDialogueTask

FindmeagoodsushirestaurantinChelsea

restaurant_type <- sushi

location <- Chelsea

SushiSekiChelseaisasushirestaurantinChelseawith4.4starsonGoogle

curr_result <- execute_search()

Page 62: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

FullDialogueTask

FindmeagoodsushirestaurantinChelsea

restaurant_type <- sushi

location <- Chelsea

SushiSekiChelseaisasushirestaurantinChelseawith4.4starsonGoogle

curr_result <- execute_search()

Howexpensiveisit?

Page 63: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

FullDialogueTask

FindmeagoodsushirestaurantinChelsea

restaurant_type <- sushi

location <- Chelsea

SushiSekiChelseaisasushirestaurantinChelseawith4.4starsonGoogle

curr_result <- execute_search()

Howexpensiveisit?get_value(cost, curr_result)

Page 64: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

FullDialogueTask

FindmeagoodsushirestaurantinChelsea

restaurant_type <- sushi

location <- Chelsea

SushiSekiChelseaisasushirestaurantinChelseawith4.4starsonGoogle

curr_result <- execute_search()

Howexpensiveisit?get_value(cost, curr_result)

Entreesarearound$30each

Page 65: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

POMDP-basedDialogueSystems

Youngetal.(2013)

‣ POMDP:useristhe“environment,”anu4eranceisanoisysignalofstate

Page 66: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

POMDP-basedDialogueSystems

Youngetal.(2013)

‣ Dialoguemodel:canlooklikeaparseroranykindofencodermodel

‣ POMDP:useristhe“environment,”anu4eranceisanoisysignalofstate

Page 67: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

POMDP-basedDialogueSystems

Youngetal.(2013)

‣ Dialoguemodel:canlooklikeaparseroranykindofencodermodel

‣ POMDP:useristhe“environment,”anu4eranceisanoisysignalofstate

‣ Generator:usetemplatesorseq2seqmodel

Page 68: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

POMDP-basedDialogueSystems

Youngetal.(2013)

‣ Dialoguemodel:canlooklikeaparseroranykindofencodermodel

‣ POMDP:useristhe“environment,”anu4eranceisanoisysignalofstate

‣ Generator:usetemplatesorseq2seqmodel

‣Wheredorewardscomefrom?

Page 69: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

RewardforcompleAngtask?

FindmeagoodsushirestaurantinChelsea

restaurant_type <- sushi

location <- Chelsea

SushiSekiChelseaisasushirestaurantinChelseawith4.4starsonGoogle

make_reservation(curr_result)

Howexpensiveisit?

+1

…

OkaymakemeareservaAon!

curr_result <- execute_search()

Page 70: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

RewardforcompleAngtask?

FindmeagoodsushirestaurantinChelsea

restaurant_type <- sushi

location <- Chelsea

SushiSekiChelseaisasushirestaurantinChelseawith4.4starsonGoogle

make_reservation(curr_result)

Howexpensiveisit?

+1

…

OkaymakemeareservaAon!

curr_result <- execute_search()

Veryindirectsignal ofwhatshould happenuphere

Page 71: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

Usergivesreward?

FindmeagoodsushirestaurantinChelsea

restaurant_type <- sushi

location <- Chelsea

SushiSekiChelseaisasushirestaurantinChelseawith4.4starsonGoogle

curr_result <- execute_search()

Howexpensiveisit?get_value(cost, curr_result)

Entreesarearound$30each

+1

+1

Page 72: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

Usergivesreward?

FindmeagoodsushirestaurantinChelsea

restaurant_type <- sushi

location <- Chelsea

SushiSekiChelseaisasushirestaurantinChelseawith4.4starsonGoogle

curr_result <- execute_search()

Howexpensiveisit?get_value(cost, curr_result)

Entreesarearound$30each

+1

+1

Howdoestheuserknowtherightsearchhappened?

Page 73: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

Wizard-of-Oz

Kelley(early1980s),FordandSmith(1982)

‣ LearningfromdemonstraAons:“wizard”pullstheleversandmakesthedialoguesystemupdateitsstateandtakeacAons

Page 74: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

FullDialogueTask

FindmeagoodsushirestaurantinChelsea

Page 75: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

FullDialogueTask

FindmeagoodsushirestaurantinChelsea

restaurant_type <- sushi

location <- Chelsea

curr_result <- execute_search(){wizardenters

these

Page 76: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

FullDialogueTask

FindmeagoodsushirestaurantinChelsea

restaurant_type <- sushi

location <- Chelsea

curr_result <- execute_search(){wizardenters

these

SushiSekiChelseaisasushirestaurantinChelseawith4.4starsonGoogle{wizardtypesthis

outorinvokes templates

Page 77: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

FullDialogueTask

FindmeagoodsushirestaurantinChelsea

restaurant_type <- sushi

location <- Chelsea

curr_result <- execute_search(){wizardenters

these

SushiSekiChelseaisasushirestaurantinChelseawith4.4starsonGoogle{wizardtypesthis

outorinvokes templates

‣Wizardcanbeatrainedexpertandknowexactlywhatthedialoguesystemsissupposedtodo

Page 78: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

LearningfromStaAcTraces

Bordesetal.(2017)

‣ Usingeitherwizard-of-OzorotherannotaAons,cancollectstaActracesandtrainfromthese

Page 79: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

FullDialogueTask

FindmeagoodsushirestaurantinChelsea

restaurant_type <- sushi

location <- Chelsea

curr_result <- execute_search()

Page 80: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

FullDialogueTask

FindmeagoodsushirestaurantinChelsea

restaurant_type <- sushi

location <- Chelsea

curr_result <- execute_search()stars <- 4+

Page 81: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

FullDialogueTask

FindmeagoodsushirestaurantinChelsea

restaurant_type <- sushi

location <- Chelsea

curr_result <- execute_search()

‣ Useraskedfora“good”restaurant—doesthatmeanweshouldfilterbystarraAng?Whatdoes“good”mean?

stars <- 4+

Page 82: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

FullDialogueTask

FindmeagoodsushirestaurantinChelsea

restaurant_type <- sushi

location <- Chelsea

curr_result <- execute_search()

‣ Useraskedfora“good”restaurant—doesthatmeanweshouldfilterbystarraAng?Whatdoes“good”mean?

‣ HardtochangesystembehavioriftrainingfromstaActraces,especiallyifsystemcapabiliAesordesiredbehaviorchange

stars <- 4+

Page 83: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

Goal-orientedDialogue

‣ BigCompanies:AppleSiri(VocalIQ),GoogleAllo,AmazonAlexa,MicrosofCortana,FacebookM,SamsungBixby,TencentWeChat

‣ Startups:

‣ Lotsofcoolworkthat’snotpublicyet

‣ Tonsofindustryinterest!

Page 84: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

OtherDialogueApplicaAons

Page 85: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

Search/QAasDialogue

‣ “HasChrisPra4wonanOscar?”/“HashewonanOscar”

Page 86: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

QAasDialogue‣ DialogueisaverynaturalwaytofindinformaAonfromasearchengineoraQAsystem

Iyyeretal.(2017)

Page 87: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

QAasDialogue‣ DialogueisaverynaturalwaytofindinformaAonfromasearchengineoraQAsystem

Iyyeretal.(2017)

‣ QAishardenoughonitsown

‣ Challenges:

Page 88: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

QAasDialogue‣ DialogueisaverynaturalwaytofindinformaAonfromasearchengineoraQAsystem

Iyyeretal.(2017)

‣ QAishardenoughonitsown

‣ Usersmovethegoalposts

‣ Challenges:

Page 89: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

QAasDialogue‣ UWQuACdataset:QuesAonAnsweringinContext

Choietal.(2018)

Page 90: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

SearchasDialogue

‣ Googlecandealwithmisspellings,somoremisspellingshappen—Googlehastodomore!

Page 91: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

DialogueMissionCreep

System

Erroranalysis

Be4ermodel

Data

MostNLPtasks

Page 92: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

DialogueMissionCreep

System

Erroranalysis

Be4ermodel

‣ FixeddistribuAon(e.g.,naturallanguagesentences),errorrate->0

Data

MostNLPtasks

Page 93: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

DialogueMissionCreep

System

Erroranalysis

Be4ermodel

‣ FixeddistribuAon(e.g.,naturallanguagesentences),errorrate->0

Data

MostNLPtasks

System

Erroranalysis

Be4ermodel

Data

Dialogue/Search/QA

Page 94: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

DialogueMissionCreep

System

Erroranalysis

Be4ermodel

‣ FixeddistribuAon(e.g.,naturallanguagesentences),errorrate->0

DataHarderData

MostNLPtasks

System

Erroranalysis

Be4ermodel

Data

Dialogue/Search/QA

???

Page 95: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

DialogueMissionCreep

System

Erroranalysis

Be4ermodel

‣ FixeddistribuAon(e.g.,naturallanguagesentences),errorrate->0

Data

‣ Errorrate->???;“missioncreep”fromHCIelement

HarderData

MostNLPtasks

System

Erroranalysis

Be4ermodel

Data

Dialogue/Search/QA

???

Page 96: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

DialogueMissionCreep

‣ Highvisibility—yourproducthastoworkreallywell!

Page 97: Lecture 16: Dialoguearitter.github.io/courses/5525_slides_v2/lec16-dialogue.pdfFull Dialogue Task ‣ Parsing / language understanding is just one piece of a system Young et al. (2013)

Takeaways

‣ Somedecentchatbots,applicaAons:predicAvetextinput,…

‣ Task-orienteddialoguesystemsaregrowinginscopeandcomplexity

‣Moreandmoreproblemsarebeingformulatedasdialogue—interesAngapplicaAonsbutchallengingtogetworkingwell


Parsing III (Top-down parsing: recursive descent & LL(1) )

Parsing III (Top-down parsing: recursive descent & LL(1) )

Documents
Lec16 MOS Es

Lec16 MOS Es

Documents
Lec16, 3D Technologies, v1.01.ppt - Sharifce.sharif.edu/courses/94-95/1/ce342-1/resources/root/Lectures/Lec16... · 3D Technologies Types of 3D Displays Stereoscopic Provides a different

Lec16, 3D Technologies, v1.01.ppt - Sharifce.sharif.edu/courses/94-95/1/ce342-1/resources/root/Lectures/Lec16... · 3D Technologies Types of 3D Displays Stereoscopic Provides a different

Documents
Parsing Expression Grammar and Packrat Parsing (Survey)

Parsing Expression Grammar and Packrat Parsing (Survey)

Documents
Baina Lec16 ModuleIV Sorting Algorithms 3 Advanced Sorting Algorithmes

Baina Lec16 ModuleIV Sorting Algorithms 3 Advanced Sorting Algorithmes

Documents
LEC16 Hackney Challenge

LEC16 Hackney Challenge

Business
Learning for semantic parsing using statistical syntactic parsing techniques

Learning for semantic parsing using statistical syntactic parsing techniques

Technology
Statistical Dependency Parsing - Uppsala Universitynivre/docs/StatState.pdf · I Methods for statistical dependency parsing I Chart parsing techniques I Parsing as constraint satisfaction

Statistical Dependency Parsing - Uppsala Universitynivre/docs/StatState.pdf · I Methods for statistical dependency parsing I Chart parsing techniques I Parsing as constraint satisfaction

Documents
Transport Phenomena LEC16 21b

Transport Phenomena LEC16 21b

Documents
Lec16 Synthesis

Lec16 Synthesis

Documents
MAE 640 Lec16

MAE 640 Lec16

Documents
October 2008csa3180: Setence Parsing Algorithms 1 1 CSA350: NLP Algorithms Sentence Parsing I The Parsing Problem Parsing as Search Top Down/Bottom Up

October 2008csa3180: Setence Parsing Algorithms 1 1 CSA350: NLP Algorithms Sentence Parsing I The Parsing Problem Parsing as Search Top Down/Bottom Up

Documents
Top-Down Parsing - recursive descent - predictive parsing

Top-Down Parsing - recursive descent - predictive parsing

Documents
LEC16 Dist Para File Systems - ranger.uta.edu

LEC16 Dist Para File Systems - ranger.uta.edu

Documents
– 1 – CSCE 531 Spring 2006 Lecture 8 Bottom Up Parsing Topics Overview Bottom-Up Parsing Handles Shift-reduce parsing Operator precedence parsing Readings:

– 1 – CSCE 531 Spring 2006 Lecture 8 Bottom Up Parsing Topics Overview Bottom-Up Parsing Handles Shift-reduce parsing Operator precedence parsing Readings:

Documents
u.cs.biu.ac.ilu.cs.biu.ac.il/~89-680/parsing-algorithms.pdf · Title: parsing-algorithms

u.cs.biu.ac.ilu.cs.biu.ac.il/~89-680/parsing-algorithms.pdf · Title: parsing-algorithms

Documents
Chart Parsing and Probabilistic Parsing

Chart Parsing and Probabilistic Parsing

Documents
Lec16: Medical Image Registration (Advanced): Deformable Registration

Lec16: Medical Image Registration (Advanced): Deformable Registration

Science
lec16 11 02 - Oregon State University

lec16 11 02 - Oregon State University

Documents
Lec16 Sept10 Maxwell Eqns

Lec16 Sept10 Maxwell Eqns

Documents
Parsing Constituency: Nodes in a Parse Treehollingk/classes/lec16.CL1-f11.pdf · 2 Top-Down Search • Observation: trees must be rooted with an S node • Parsing strategy: • Start

Parsing Constituency: Nodes in a Parse Treehollingk/classes/lec16.CL1-f11.pdf · 2 Top-Down Search • Observation: trees must be rooted with an S node • Parsing strategy: • Start

Documents
Lec16 Ethernet

Lec16 Ethernet

Documents
Bare-Bones Dependency Parsing - Uppsala Universitynivre/docs/BareBones.pdf · I Parsing methods for bare-bones dependency parsing I Chart parsing techniques ... [Kuhlmann and Satta

Bare-Bones Dependency Parsing - Uppsala Universitynivre/docs/BareBones.pdf · I Parsing methods for bare-bones dependency parsing I Chart parsing techniques ... [Kuhlmann and Satta

Documents
Weighted Parsing, Probabilistic Parsing

Weighted Parsing, Probabilistic Parsing

Documents
Lec16 Modules

Lec16 Modules

Documents
Lec16 Drainage Design Hoogout Method

Lec16 Drainage Design Hoogout Method

Documents
Lec16 Partially Prestressed Concrete Beams

Lec16 Partially Prestressed Concrete Beams

Documents
Parsing III (Top-down parsing: recursive descent & LL(1) )

Parsing III (Top-down parsing: recursive descent & LL(1) )

Documents
LEC16 13 Politics

LEC16 13 Politics

Documents
Lecture 16 Interval, Scales, Tuning and Temperament - Ifaculty.tamuc.edu/cbertulani/music/lectures/Lec16/Lec16.pdf · 3 SCALE – A sucession of notes in ascending order (e.g., Pythagorean,

Lecture 16 Interval, Scales, Tuning and Temperament - Ifaculty.tamuc.edu/cbertulani/music/lectures/Lec16/Lec16.pdf · 3 SCALE – A sucession of notes in ascending order (e.g., Pythagorean,

Documents