A Skeleton-Based Model for Promoting Coherence Among ... · A Skeleton-Based Model for Promoting...

NarrativeStoryGenerationASkeleton-BasedModelforPromotingCoherence 4-11-2018

JingjingXu,XuanchengRen,Yi Zhang,QiZeng, Xiaoyan Cai,XuSun

MOEKeyLabofComputationalLinguistics,SchoolofEECS,PekingUniversitySchoolofAutomation,NorthwesternPolytechnical University

jingjingxu@pku.edu.cn

ASkeleton-BasedModelforPromotingCoherenceAmongSentencesinNarrativeStoryGeneration

NarrativeStoryGenerationASkeleton-BasedModelforPromotingCoherence 4-11-2018 2 of28

p Introduction

Ø Task

Ø Challenge

p Skeleton-BasedModel

Ø Overview

Ø Skeleton-BasedGenerativeModule

Ø SkeletonExtractionModule

Outline

p Experiment

Ø Dataset

Ø Baselines

Ø Results

p Analysis

Ø IncrementalAnalysis

Ø ErrorAnalysis

p Conclusion

Introduction

NarrativeStoryGeneration

TaskDefinitionInput:Ashort description ofasceneoranevent.Output:Arelevant narrativestoryfollowingtheinput.

Input:Fans cametogethertocelebrate theopening ofanewstudio foranartist.

Output:Theartist providedchampagne influtesforeveryone.Friendstoasted andcheered theartistassheopened hernewstudio.

Input: LastweekIattendedawedding forthefirsttime.

Output: Therewerealotoffamilies there.Theywerealltakingpictures together.Everyonewasveryhappy.Thebride andgroom gottorideinalimothattheyrented.

q Highrequirementsontightsemanticconnectionamongsentences

q Ifthegiveninputisabout“artistandstudioopening”

Theartistprovidedchampagneforeveryone

Fanstoastedandcheeredtheartist

Thefanswereveryhappytoseethisevent

Thefootballgameissoexciting

Theweatherissocoldandalltreesarecoveredwithheavysnow

Difficulty

Motivation

q Theconnectionamongsentencesismainlyreflectedthroughkeyphrases (orskeleton)

q Weexplicitlymodeltherelationamongskeletonsinthiswork.

q Motivatedbythisfact:

1.Wedonotgenerateasentencefromlefttoright

2.Instead,wefirstgenerateaskeleton

3.Expandtheskeletontoacompleteandfluentsentence

Skeleton-basedModel

Approach

q Ourmodelcontainstwoparts

Ø Skeleton-basedgenerativemodule

1. Input-to-skeletoncomponent

2. Skeleton-to-targetcomponent

Ø Skeletonextractionmodule

Overview

q GenerationProcess

Skeleton-BasedGenerativeModule

Fanscametogethertocelebratetheopeningofanewstudioforanartist.

The artist provided champagne in flutes for everyone.

artist champagne

Friends toasted cheered artist.

Friends toasted and cheered the artist as she opened her new studio.

q Input-to-SkeletonComponent

Ø Learnthedependencybetweensourceinputsandtargetskeletons

Ø Model:ASeq2Seq

Ø TrainingLoss:Cross-entropyloss

q Skeleton-to-TargetComponent

Ø Expandaskeletontoacompletesentence

Ø Model:Seq2Seq

Ø Encoder:LSTM

Ø Decoder:LSTM

Ø TrainingLoss:Cross-EntropyLoss

TheStructureofSkeleton-BasedGenerativeModule

q SkeletonsassupervisorysignalØ Inreal-worlddatasets,thehuman-annotatedskeletonisusuallyunavailable

q Rule-basedapproaches

Ø Advantage:Simple

Ø Drawback:Difficulttodefinetheunifiedrulesofextractingskeletons

q Toaddressthisproblem,webuildaskeletonextractionmoduletoautomaticallyexploresentenceskeletons

Skeleton-ExtractiveModule

q Reformulateskeletonextractionasasentencecompressionproblem

Ø Advantage:sentencecompressionrequiressystemstokeepcore

information,whichisconsistentwithourtarget.

Ø Drawback:sentencecompressionresultsusuallyaregrammatical,which

limitsthequalityofskeletons.

Skeleton-ExtractionModule

Thelady wearingthepinkshirtdecidedtostopplayingthevideoandchattedwithotherguests.

Lady decidedtostopplayingthevideo

compress

q Sequencetosequencemodel

Ø Encoder:LSTM

Ø Decoder:LSTM

q Input:originaltext

q Output:theskeletonoftheoriginaltext

q Lossfunction:cross-entropyloss

ThestructureofSkeletonExtractionModule

qMotivation

q Usethefeedbackofgenerativemoduletoimproveit

q Challenge

q Lossisnolongerdifferentiable

q Approach

q ReinforcementLearningMethod

q Treatthefeedbackofgenerativemoduleasreward

q Optimizetheextractionmodelby

Howtoimprovethequalityofskeleton-extractionmodule

qMultiplicationofthecross-entropyloss

• K:Upperboundofthereward,

• R1:Cross-entropylossesintheinput-to-skeletoncomponent

• R2:Cross-entropylossesintheskeleton-to-targetcomponent,respectively.

Reward

Experiment

q Source:

ü VisualStorytelling

q Dataset:

ü Input:thefirstsentence

ü Output:thefollowingsentences

Dataset

Having a good time bonding and talking.

[M] got exhausted by the heat.

Sky illuminated with a brilliance of gold and orange hues.

q Entity-EnhancedSeq2SeqModel(EESeq2Seq)

ü Combineentitycontextandtextcontexttogether

q Dependency-TreeEnhancedSeq2SeqModel(DE-Seq2Seq)

ü Basedondependencyparsinglabels

q Generalized-TemplateEnhancedSeq2SeqModel(GE-Seq2Seq)

ü Useexistingknowledgebases(WordNet,VerbNet)togetageneralized

sentencerepresentation

Baselines

q AutomaticEvaluation

Results

q HumanEvaluation

ü 100items

ü Fluencyandcoherence

Results

q Input1:[Female]andherfriendhadanightoutonthetown.

Ø EE-Seq2Seq:Theywereveryhappytoseeus.[Male]and[female]were

gettingmarried today.

Ø DE-Seq2Seq:Wewalkedthroughthestreetsofthecityandfoundaplaceto

sit.Thewholefamilycameouttosupporttheirfamilies.

Ø GE-Seq2Seq:MyfriendsandIwenttoabarlastnight.[Female]wasso

happytobethere.

Ø ProposedModel:Theywenttothebar.Theyhadagreattime.

Generatedexamples

Analysis

q 4typesoferrors

ü Irrelevantscene

ü Chaoticsyntax

ü Chaotictimeline

ü Repeatedscene

ErrorAnalysis

A. Weproposeanewskeleton-basedmodelforgeneratingcoherentnarrativestories

B. Experimentalresultsshowthatourmodelsignificantlyimprovesthequalityofgeneratedstories

C. Erroranalysisshowsthattherearestillmanychallengesinnarrativestorygeneration,whichwe

wouldliketoexploreinthefuture

Conclusion

Thank You!

Ifyouhaveanyquestion,pleasesendane-mailtojingjingxu@pku.edu.cn

A Skeleton-Based Model for Promoting Coherence Among ... · A Skeleton-Based Model for Promoting...

Documents

Human skeleton learning the vocabulary for a skeleton

Matrix Sketching Over Sliding Windowslifeifei/papers/slidingmatrix.pdf · 2016. 4. 18. · Matrix Sketching Over Sliding Windows Zhewei Wei1, Xuancheng Liu2, Feifei Li3, Shuo Shang2,

The Skeleton Part A - Axial Skeleton

Electronic Coherence, Vibrational Coherence, and Solvent ...chemgroups.northwestern.edu/hupp/Publications/85.pdf · Electronic Coherence, Vibrational Coherence, and Solvent Degrees

Satyan L. Devadoss, Rahul Shah, Xuancheng Shao and Ezra Winston- Visibility Graphs and Deformations of Associahedra

The Skeletal System. Axial Skeleton Appendicular Skeleton

Shaun Smith Introduction to Coherence - Oracledownload.oracle.com/otndocs/...oracle_coherence_for... · Agenda • Motivation for Coherence • What is Coherence? … and what Coherence

Technology Background Development environment, Skeleton ... · Technology Background Development environment, Skeleton and Libraries ... Development environment, Skeleton and Libraries

The Skeleton: Appendicular Skeleton

Chapter 7: The Axial Skeleton 1. Human Skeleton Human Skeleton = 206 Bones 1.Axial Skeleton: -longitudinal axis -80 bones 2.Appendicular Skeleton: -limbs

Coherence Managed Coherence Servers - Part 3 - Maven Integration

IT Science: Skeleton - Chapter Review Questions The … · IT Science: Skeleton - Chapter Review Questions The Human Skeleton 1. What does a skeleton symbolize? _____

The Human Skeleton. Divisions of the Skeleton Axial skeleton – skull, vertebrae, and bony thorax Appendicular skeleton – bones of the arms and legs, including

Topic # 13.1: Human Skeleton(Axial Skeleton and Appendicular Skeleton) Slo # 13.1.1: Define Skeleton and differentiate between cartilage and bones; Knowledge

Necromancy - onemonk.comonemonk.com/ewExternalFiles/necromancy.pdf · Necromancer Skeleton 1 Skeleton 2 Skeleton 3 Skeleton 4 Minion 1 Skeleton 5 Minion 2 Minion 3 Minion 4 Minion

The Axial Skeleton & Appendicular Skeleton

The Skeleton is divided into 2 parts: The Axial Skeleton & The Appendicular Skeleton

SKELETAL SYSTEM Bone & Tissue Axial Skeleton Appendicular Skeleton Articulations

Oracle Coherence Product Overview Raanan Dagan / Coherence Team

The Axial Skeleton - Napa Valley College · The Axial Skeleton The axial skeleton ... Appendicular skeleton . Copyright © 2009 Pearson Education, Inc., publishing as Pearson Benjamin