7
ELMo

ELMo - University of Texas at Austingdurrett/courses/sp2019/... · 2019. 4. 18. · ELMo Tasks: Sen.ment the movie was gross and overwrought, but I liked it this movie was great!

  • Upload
    others

  • View
    1

  • Download
    0

Embed Size (px)

Citation preview

  • ELMo

  • ELMoTasks:Sen.ment

    themoviewasgrossandoverwrought,butIlikedit

    thismoviewasgreat!wouldwatchagain

    thismoviewasnotreallyveryenjoyable

    ++—

    ‣Whyarecontext-dependentembeddingsusefulhere?

    Petersetal.(2018)

  • ELMoTasks:NER

    BarackObamawilltraveltoHangzhoutodayfortheG20meeBng.

    PERSON LOC ORG

    B-PER I-PER O O O B-LOC B-ORGO O O O O

    ‣Whyarecontext-sensi.veembeddingsusefulhere?

    Petersetal.(2018)

  • ELMoTasks:SRL

    ‣ Iden.fyargumentsoftheverb

    ‣Whyarecontext-dependentembeddingsusefulhere?

    Petersetal.(2018)

  • ELMo

    ‣ SST5=sen.ment,SQuAD=QA(cominglater)

    Petersetal.(2018)

    ‣ Training:1BwordsoftexttakenfromMTdata(news,etc.)

    ‣ Largegainsacrossmanytasks

  • ELMo

    Petersetal.(2018)

    ‣ Nearestneighborsofwordsincontext

  • ELMo

    ‣ Highlyflexible:canuseasanembeddinglayerfornearlyanytask

    ‣ Canbebigandslow:bigLSTMscansuckupGPUmemory

    ‣ Strictlybe]erthanword2vecifyoucanaffordit

    ‣ AllenNLP:callingELMoisjustoneline,plentyofmodelsfordifferenttasksbuiltontopofit

    Petersetal.(2018)