Upload
others
View
1
Download
0
Embed Size (px)
Citation preview
ELMo
ELMoTasks:Sen.ment
themoviewasgrossandoverwrought,butIlikedit
thismoviewasgreat!wouldwatchagain
thismoviewasnotreallyveryenjoyable
++—
‣Whyarecontext-dependentembeddingsusefulhere?
Petersetal.(2018)
ELMoTasks:NER
BarackObamawilltraveltoHangzhoutodayfortheG20meeBng.
PERSON LOC ORG
B-PER I-PER O O O B-LOC B-ORGO O O O O
‣Whyarecontext-sensi.veembeddingsusefulhere?
Petersetal.(2018)
ELMoTasks:SRL
‣ Iden.fyargumentsoftheverb
‣Whyarecontext-dependentembeddingsusefulhere?
Petersetal.(2018)
ELMo
‣ SST5=sen.ment,SQuAD=QA(cominglater)
Petersetal.(2018)
‣ Training:1BwordsoftexttakenfromMTdata(news,etc.)
‣ Largegainsacrossmanytasks
ELMo
Petersetal.(2018)
‣ Nearestneighborsofwordsincontext
ELMo
‣ Highlyflexible:canuseasanembeddinglayerfornearlyanytask
‣ Canbebigandslow:bigLSTMscansuckupGPUmemory
‣ Strictlybe]erthanword2vecifyoucanaffordit
‣ AllenNLP:callingELMoisjustoneline,plentyofmodelsfordifferenttasksbuiltontopofit
Petersetal.(2018)