Visual Storytelling via Predicting Anchor Word Embeddings in the Stories

Bowen Zhang1 Hexiang Hu1 Fei Sha21University of Southern California, 2 Google AI

zhan734@usc.edu, hexiangh@usc.edu, fsha@google.com∗

Abstract

We propose a learning model for the task of visual story-telling. The main idea is to predict anchor word embeddingsfrom the images and use the embeddings and the image fea-tures jointly to generate narrative sentences. We use theembeddings of randomly sampled nouns from the ground-truth stories as the target anchor word embeddings to learnthe predictor. To narrate a sequence of images, we use thepredicted anchor word embeddings and the image featuresas the joint input to a seq2seq model. As opposed to state-of-the-art methods, the proposed model is simple in design,easy to optimize, and attains the best results in most auto-matic evaluation metrics. In human evaluation, the methodalso outperforms competing methods.

1. Introduction

Visual storytelling, ie, narrating a sequence of images,is a challenging task [8, 6]. It demands an understandingof the underlying storyline of the images. The process isnaturally subjective. It often focuses more on conveying thenarrator’s own interpretation than describing the images infactual terms.

For example, as pointed out by the creators of the pop-ular dataset VIST [6], concatenating the descriptions of theimages does not give rise to a desirable narrative story. Ta-ble 1 illustrates the difference in the corpus statistics on theaforementioned dataset. Despite similar in length, story andcaption use very different sets of words. At least 40% ofwords that appear in stories do not appear in captions.

While this discrepancy has been well-documented, it isunclear how this insight could be used to devising effec-tive models for visual storytelling. The task seems natu-rally gravitating to the method of SEQ2SEQ where we learna mapping to encode a sequence of image features then todecode by outputting a sequence of words [3]. This methodmet some successes and is followed by others [5, 11, 12].

In this paper, we take a step toward identifying whatmight be needed for generating a narrative story. We hy-

∗On leave from U. of Southern California (feisha@usc.edu)

Storys Captions

Vocabulary Size 29,614 24,534Avg. Sent. Length 11.4 11.9

# of Nouns 6,831 7,772# of Verbs 5,217 3,202# of Adjectives 2,089 2,018# of Adverbs 1,505 286

Table 1. Statistics of stories and captions on the VIST dataset [6]pothesize that each narrative story needs to have a sequenceof anchor words. For simplicity, we assume one anchorword per image. The anchor words form a prior on whatcan be “said” about the images. To narrate a sequence ofimages, our learning model just needs to predict the anchorword embedding for each image in turn and then supply theembeddings to a SEQ2SEQ model to generate the story.

But then, what are the anchor words? They are not ex-plicitly given in the annotated dataset. As a first step, wehave shown that we can use the words in the ground-truthstories as anchor words and learn a predictive model (fromthe image features) to predict the anchor word embeddingswhen the ground-truth stories are not available.

As opposed to several best-performing models for thesame task, our model is simple in design and does not needto use reinforcement learning to optimize [5, 11, 12]. Yet itattains the best performance in several evaluation metrics.

We describe the idea of using anchor words in section 3,supported by the evidence that such words, when added toa vanilla SEQ2SEQ model for story generation, significantlyimprove its performance. We then describe how to train apredictive model to predict its embedding. In section 4, wereport our evaluation results and conclude in section 5.

2. Related WorkThere is a large body of work in the intersection of vision

and language, cf. [7, 10].Image captioning is closely related to visual storytelling.

SEQ2SEQ and its variants are among the most popular learn-ing approaches for the task [13, 10].

From the very beginning, the creators of the dataset forvisual storytelling highlighted the difference of captioningfrom narratives [6]. In essence, narrative stories go beyond

h2<latexit sha1_base64="mmlmkiSlrgXNmMzl4Qiy3+KOIIE=">AAAB8HicbVBNS8NAEJ34WetX1aOXxSJ4KkkR9Fj04rGC/ZA2lM120yzdbMLuRCihv8KLB0W8+nO8+W/ctjlo64OBx3szzMwLUikMuu63s7a+sbm1Xdop7+7tHxxWjo7bJsk04y2WyER3A2q4FIq3UKDk3VRzGgeSd4Lx7czvPHFtRKIecJJyP6YjJULBKFrpsR9RzKNBfTqoVN2aOwdZJV5BqlCgOah89YcJy2KukElqTM9zU/RzqlEwyaflfmZ4StmYjnjPUkVjbvx8fvCUnFtlSMJE21JI5urviZzGxkziwHbGFCOz7M3E/7xehuG1nwuVZsgVWywKM0kwIbPvyVBozlBOLKFMC3srYRHVlKHNqGxD8JZfXiXtes1za979ZbVxU8RRglM4gwvw4AoacAdNaAGDGJ7hFd4c7bw4787HonXNKWZO4A+czx/DiZBe</latexit><latexit sha1_base64="mmlmkiSlrgXNmMzl4Qiy3+KOIIE=">AAAB8HicbVBNS8NAEJ34WetX1aOXxSJ4KkkR9Fj04rGC/ZA2lM120yzdbMLuRCihv8KLB0W8+nO8+W/ctjlo64OBx3szzMwLUikMuu63s7a+sbm1Xdop7+7tHxxWjo7bJsk04y2WyER3A2q4FIq3UKDk3VRzGgeSd4Lx7czvPHFtRKIecJJyP6YjJULBKFrpsR9RzKNBfTqoVN2aOwdZJV5BqlCgOah89YcJy2KukElqTM9zU/RzqlEwyaflfmZ4StmYjnjPUkVjbvx8fvCUnFtlSMJE21JI5urviZzGxkziwHbGFCOz7M3E/7xehuG1nwuVZsgVWywKM0kwIbPvyVBozlBOLKFMC3srYRHVlKHNqGxD8JZfXiXtes1za979ZbVxU8RRglM4gwvw4AoacAdNaAGDGJ7hFd4c7bw4787HonXNKWZO4A+czx/DiZBe</latexit><latexit sha1_base64="mmlmkiSlrgXNmMzl4Qiy3+KOIIE=">AAAB8HicbVBNS8NAEJ34WetX1aOXxSJ4KkkR9Fj04rGC/ZA2lM120yzdbMLuRCihv8KLB0W8+nO8+W/ctjlo64OBx3szzMwLUikMuu63s7a+sbm1Xdop7+7tHxxWjo7bJsk04y2WyER3A2q4FIq3UKDk3VRzGgeSd4Lx7czvPHFtRKIecJJyP6YjJULBKFrpsR9RzKNBfTqoVN2aOwdZJV5BqlCgOah89YcJy2KukElqTM9zU/RzqlEwyaflfmZ4StmYjnjPUkVjbvx8fvCUnFtlSMJE21JI5urviZzGxkziwHbGFCOz7M3E/7xehuG1nwuVZsgVWywKM0kwIbPvyVBozlBOLKFMC3srYRHVlKHNqGxD8JZfXiXtes1za979ZbVxU8RRglM4gwvw4AoacAdNaAGDGJ7hFd4c7bw4787HonXNKWZO4A+czx/DiZBe</latexit><latexit sha1_base64="hP+6LrUf2d3tZaldqaQQvEKMXyw=">AAAB2XicbZDNSgMxFIXv1L86Vq1rN8EiuCozbnQpuHFZwbZCO5RM5k4bmskMyR2hDH0BF25EfC93vo3pz0JbDwQ+zknIvSculLQUBN9ebWd3b/+gfugfNfzjk9Nmo2fz0gjsilzl5jnmFpXU2CVJCp8LgzyLFfbj6f0i77+gsTLXTzQrMMr4WMtUCk7O6oyaraAdLMW2IVxDC9YaNb+GSS7KDDUJxa0dhEFBUcUNSaFw7g9LiwUXUz7GgUPNM7RRtRxzzi6dk7A0N+5oYkv394uKZ9bOstjdzDhN7Ga2MP/LBiWlt1EldVESarH6KC0Vo5wtdmaJNChIzRxwYaSblYkJN1yQa8Z3HYSbG29D77odBu3wMYA6nMMFXEEIN3AHD9CBLghI4BXevYn35n2suqp569LO4I+8zx84xIo4</latexit><latexit sha1_base64="+ujoKiOktwAK1ELQ0zE3/Cq50Dw=">AAAB5XicbZBLSwMxFIXv1FetVatbN8EiuCoz3eiy4MZlBfuQdiiZNNMJTTJDckcoQ3+FGxeK+Jfc+W9MHwttPRD4OCch954ok8Ki7397pZ3dvf2D8mHlqHp8clo7q3ZtmhvGOyyVqelH1HIpNO+gQMn7meFURZL3oundIu89c2NFqh9xlvFQ0YkWsWAUnfU0TCgWyag5H9XqfsNfimxDsIY6rNUe1b6G45Tlimtkklo7CPwMw4IaFEzyeWWYW55RNqUTPnCoqeI2LJYDz8mVc8YkTo07GsnS/f2ioMramYrcTUUxsZvZwvwvG+QY34aF0FmOXLPVR3EuCaZksT0ZC8MZypkDyoxwsxKWUEMZuo4qroRgc+Vt6DYbgd8IHnwowwVcwjUEcAMtuIc2dICBghd4g3fPeK/ex6qukrfu7Rz+yPv8AYsdjwM=</latexit><latexit sha1_base64="+ujoKiOktwAK1ELQ0zE3/Cq50Dw=">AAAB5XicbZBLSwMxFIXv1FetVatbN8EiuCoz3eiy4MZlBfuQdiiZNNMJTTJDckcoQ3+FGxeK+Jfc+W9MHwttPRD4OCch954ok8Ki7397pZ3dvf2D8mHlqHp8clo7q3ZtmhvGOyyVqelH1HIpNO+gQMn7meFURZL3oundIu89c2NFqh9xlvFQ0YkWsWAUnfU0TCgWyag5H9XqfsNfimxDsIY6rNUe1b6G45Tlimtkklo7CPwMw4IaFEzyeWWYW55RNqUTPnCoqeI2LJYDz8mVc8YkTo07GsnS/f2ioMramYrcTUUxsZvZwvwvG+QY34aF0FmOXLPVR3EuCaZksT0ZC8MZypkDyoxwsxKWUEMZuo4qroRgc+Vt6DYbgd8IHnwowwVcwjUEcAMtuIc2dICBghd4g3fPeK/ex6qukrfu7Rz+yPv8AYsdjwM=</latexit><latexit sha1_base64="jfsZDp6qidS9lGUUzLHNyLmF2eU=">AAAB8HicbVBNS8NAEJ3Ur1q/qh69BIvgqSS96LHoxWMF+yFtKJvtplm6uwm7E6GE/govHhTx6s/x5r9x2+agrQ8GHu/NMDMvTAU36HnfTmljc2t7p7xb2ds/ODyqHp90TJJpyto0EYnuhcQwwRVrI0fBeqlmRIaCdcPJ7dzvPjFteKIecJqyQJKx4hGnBK30OIgJ5vGwMRtWa17dW8BdJ35BalCgNax+DUYJzSRTSAUxpu97KQY50cipYLPKIDMsJXRCxqxvqSKSmSBfHDxzL6wycqNE21LoLtTfEzmRxkxlaDslwdisenPxP6+fYXQd5FylGTJFl4uiTLiYuPPv3RHXjKKYWkKo5vZWl8ZEE4o2o4oNwV99eZ10GnXfq/v3Xq15U8RRhjM4h0vw4QqacActaAMFCc/wCm+Odl6cd+dj2VpyiplT+APn8wfCSZBa</latexit><latexit sha1_base64="mmlmkiSlrgXNmMzl4Qiy3+KOIIE=">AAAB8HicbVBNS8NAEJ34WetX1aOXxSJ4KkkR9Fj04rGC/ZA2lM120yzdbMLuRCihv8KLB0W8+nO8+W/ctjlo64OBx3szzMwLUikMuu63s7a+sbm1Xdop7+7tHxxWjo7bJsk04y2WyER3A2q4FIq3UKDk3VRzGgeSd4Lx7czvPHFtRKIecJJyP6YjJULBKFrpsR9RzKNBfTqoVN2aOwdZJV5BqlCgOah89YcJy2KukElqTM9zU/RzqlEwyaflfmZ4StmYjnjPUkVjbvx8fvCUnFtlSMJE21JI5urviZzGxkziwHbGFCOz7M3E/7xehuG1nwuVZsgVWywKM0kwIbPvyVBozlBOLKFMC3srYRHVlKHNqGxD8JZfXiXtes1za979ZbVxU8RRglM4gwvw4AoacAdNaAGDGJ7hFd4c7bw4787HonXNKWZO4A+czx/DiZBe</latexit><latexit sha1_base64="mmlmkiSlrgXNmMzl4Qiy3+KOIIE=">AAAB8HicbVBNS8NAEJ34WetX1aOXxSJ4KkkR9Fj04rGC/ZA2lM120yzdbMLuRCihv8KLB0W8+nO8+W/ctjlo64OBx3szzMwLUikMuu63s7a+sbm1Xdop7+7tHxxWjo7bJsk04y2WyER3A2q4FIq3UKDk3VRzGgeSd4Lx7czvPHFtRKIecJJyP6YjJULBKFrpsR9RzKNBfTqoVN2aOwdZJV5BqlCgOah89YcJy2KukElqTM9zU/RzqlEwyaflfmZ4StmYjnjPUkVjbvx8fvCUnFtlSMJE21JI5urviZzGxkziwHbGFCOz7M3E/7xehuG1nwuVZsgVWywKM0kwIbPvyVBozlBOLKFMC3srYRHVlKHNqGxD8JZfXiXtes1za979ZbVxU8RRglM4gwvw4AoacAdNaAGDGJ7hFd4c7bw4787HonXNKWZO4A+czx/DiZBe</latexit><latexit sha1_base64="mmlmkiSlrgXNmMzl4Qiy3+KOIIE=">AAAB8HicbVBNS8NAEJ34WetX1aOXxSJ4KkkR9Fj04rGC/ZA2lM120yzdbMLuRCihv8KLB0W8+nO8+W/ctjlo64OBx3szzMwLUikMuu63s7a+sbm1Xdop7+7tHxxWjo7bJsk04y2WyER3A2q4FIq3UKDk3VRzGgeSd4Lx7czvPHFtRKIecJJyP6YjJULBKFrpsR9RzKNBfTqoVN2aOwdZJV5BqlCgOah89YcJy2KukElqTM9zU/RzqlEwyaflfmZ4StmYjnjPUkVjbvx8fvCUnFtlSMJE21JI5urviZzGxkziwHbGFCOz7M3E/7xehuG1nwuVZsgVWywKM0kwIbPvyVBozlBOLKFMC3srYRHVlKHNqGxD8JZfXiXtes1za979ZbVxU8RRglM4gwvw4AoacAdNaAGDGJ7hFd4c7bw4787HonXNKWZO4A+czx/DiZBe</latexit><latexit sha1_base64="mmlmkiSlrgXNmMzl4Qiy3+KOIIE=">AAAB8HicbVBNS8NAEJ34WetX1aOXxSJ4KkkR9Fj04rGC/ZA2lM120yzdbMLuRCihv8KLB0W8+nO8+W/ctjlo64OBx3szzMwLUikMuu63s7a+sbm1Xdop7+7tHxxWjo7bJsk04y2WyER3A2q4FIq3UKDk3VRzGgeSd4Lx7czvPHFtRKIecJJyP6YjJULBKFrpsR9RzKNBfTqoVN2aOwdZJV5BqlCgOah89YcJy2KukElqTM9zU/RzqlEwyaflfmZ4StmYjnjPUkVjbvx8fvCUnFtlSMJE21JI5urviZzGxkziwHbGFCOz7M3E/7xehuG1nwuVZsgVWywKM0kwIbPvyVBozlBOLKFMC3srYRHVlKHNqGxD8JZfXiXtes1za979ZbVxU8RRglM4gwvw4AoacAdNaAGDGJ7hFd4c7bw4787HonXNKWZO4A+czx/DiZBe</latexit><latexit sha1_base64="mmlmkiSlrgXNmMzl4Qiy3+KOIIE=">AAAB8HicbVBNS8NAEJ34WetX1aOXxSJ4KkkR9Fj04rGC/ZA2lM120yzdbMLuRCihv8KLB0W8+nO8+W/ctjlo64OBx3szzMwLUikMuu63s7a+sbm1Xdop7+7tHxxWjo7bJsk04y2WyER3A2q4FIq3UKDk3VRzGgeSd4Lx7czvPHFtRKIecJJyP6YjJULBKFrpsR9RzKNBfTqoVN2aOwdZJV5BqlCgOah89YcJy2KukElqTM9zU/RzqlEwyaflfmZ4StmYjnjPUkVjbvx8fvCUnFtlSMJE21JI5urviZzGxkziwHbGFCOz7M3E/7xehuG1nwuVZsgVWywKM0kwIbPvyVBozlBOLKFMC3srYRHVlKHNqGxD8JZfXiXtes1za979ZbVxU8RRglM4gwvw4AoacAdNaAGDGJ7hFd4c7bw4787HonXNKWZO4A+czx/DiZBe</latexit><latexit sha1_base64="mmlmkiSlrgXNmMzl4Qiy3+KOIIE=">AAAB8HicbVBNS8NAEJ34WetX1aOXxSJ4KkkR9Fj04rGC/ZA2lM120yzdbMLuRCihv8KLB0W8+nO8+W/ctjlo64OBx3szzMwLUikMuu63s7a+sbm1Xdop7+7tHxxWjo7bJsk04y2WyER3A2q4FIq3UKDk3VRzGgeSd4Lx7czvPHFtRKIecJJyP6YjJULBKFrpsR9RzKNBfTqoVN2aOwdZJV5BqlCgOah89YcJy2KukElqTM9zU/RzqlEwyaflfmZ4StmYjnjPUkVjbvx8fvCUnFtlSMJE21JI5urviZzGxkziwHbGFCOz7M3E/7xehuG1nwuVZsgVWywKM0kwIbPvyVBozlBOLKFMC3srYRHVlKHNqGxD8JZfXiXtes1za979ZbVxU8RRglM4gwvw4AoacAdNaAGDGJ7hFd4c7bw4787HonXNKWZO4A+czx/DiZBe</latexit>

h3<latexit sha1_base64="kJr2proJj0Oy6K1Gs9lHpH7M7aA=">AAAB8HicbVBNS8NAEJ3Ur1q/qh69BIvgqSQq6LHoxWMFWyttKJvtplm6uwm7E6GE/govHhTx6s/x5r9x2+agrQ8GHu/NMDMvTAU36HnfTmlldW19o7xZ2dre2d2r7h+0TZJpylo0EYnuhMQwwRVrIUfBOqlmRIaCPYSjm6n/8MS04Ym6x3HKAkmGikecErTSYy8mmMf980m/WvPq3gzuMvELUoMCzX71qzdIaCaZQiqIMV3fSzHIiUZOBZtUeplhKaEjMmRdSxWRzAT57OCJe2KVgRsl2pZCd6b+nsiJNGYsQ9spCcZm0ZuK/3ndDKOrIOcqzZApOl8UZcLFxJ1+7w64ZhTF2BJCNbe3ujQmmlC0GVVsCP7iy8ukfVb3vbp/d1FrXBdxlOEIjuEUfLiEBtxCE1pAQcIzvMKbo50X5935mLeWnGLmEP7A+fwBxQ6QXw==</latexit><latexit sha1_base64="kJr2proJj0Oy6K1Gs9lHpH7M7aA=">AAAB8HicbVBNS8NAEJ3Ur1q/qh69BIvgqSQq6LHoxWMFWyttKJvtplm6uwm7E6GE/govHhTx6s/x5r9x2+agrQ8GHu/NMDMvTAU36HnfTmlldW19o7xZ2dre2d2r7h+0TZJpylo0EYnuhMQwwRVrIUfBOqlmRIaCPYSjm6n/8MS04Ym6x3HKAkmGikecErTSYy8mmMf980m/WvPq3gzuMvELUoMCzX71qzdIaCaZQiqIMV3fSzHIiUZOBZtUeplhKaEjMmRdSxWRzAT57OCJe2KVgRsl2pZCd6b+nsiJNGYsQ9spCcZm0ZuK/3ndDKOrIOcqzZApOl8UZcLFxJ1+7w64ZhTF2BJCNbe3ujQmmlC0GVVsCP7iy8ukfVb3vbp/d1FrXBdxlOEIjuEUfLiEBtxCE1pAQcIzvMKbo50X5935mLeWnGLmEP7A+fwBxQ6QXw==</latexit><latexit sha1_base64="kJr2proJj0Oy6K1Gs9lHpH7M7aA=">AAAB8HicbVBNS8NAEJ3Ur1q/qh69BIvgqSQq6LHoxWMFWyttKJvtplm6uwm7E6GE/govHhTx6s/x5r9x2+agrQ8GHu/NMDMvTAU36HnfTmlldW19o7xZ2dre2d2r7h+0TZJpylo0EYnuhMQwwRVrIUfBOqlmRIaCPYSjm6n/8MS04Ym6x3HKAkmGikecErTSYy8mmMf980m/WvPq3gzuMvELUoMCzX71qzdIaCaZQiqIMV3fSzHIiUZOBZtUeplhKaEjMmRdSxWRzAT57OCJe2KVgRsl2pZCd6b+nsiJNGYsQ9spCcZm0ZuK/3ndDKOrIOcqzZApOl8UZcLFxJ1+7w64ZhTF2BJCNbe3ujQmmlC0GVVsCP7iy8ukfVb3vbp/d1FrXBdxlOEIjuEUfLiEBtxCE1pAQcIzvMKbo50X5935mLeWnGLmEP7A+fwBxQ6QXw==</latexit><latexit sha1_base64="kJr2proJj0Oy6K1Gs9lHpH7M7aA=">AAAB8HicbVBNS8NAEJ3Ur1q/qh69BIvgqSQq6LHoxWMFWyttKJvtplm6uwm7E6GE/govHhTx6s/x5r9x2+agrQ8GHu/NMDMvTAU36HnfTmlldW19o7xZ2dre2d2r7h+0TZJpylo0EYnuhMQwwRVrIUfBOqlmRIaCPYSjm6n/8MS04Ym6x3HKAkmGikecErTSYy8mmMf980m/WvPq3gzuMvELUoMCzX71qzdIaCaZQiqIMV3fSzHIiUZOBZtUeplhKaEjMmRdSxWRzAT57OCJe2KVgRsl2pZCd6b+nsiJNGYsQ9spCcZm0ZuK/3ndDKOrIOcqzZApOl8UZcLFxJ1+7w64ZhTF2BJCNbe3ujQmmlC0GVVsCP7iy8ukfVb3vbp/d1FrXBdxlOEIjuEUfLiEBtxCE1pAQcIzvMKbo50X5935mLeWnGLmEP7A+fwBxQ6QXw==</latexit>

h4<latexit sha1_base64="na7C4enMR0C6wbXkJ8Qlg2nlvQc=">AAAB8HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cK9kPaUDbbTbN0swm7E6GE/govHhTx6s/x5r9x2+agrQ8GHu/NMDMvSKUw6LrfTmltfWNzq7xd2dnd2z+oHh61TZJpxlsskYnuBtRwKRRvoUDJu6nmNA4k7wTj25nfeeLaiEQ94CTlfkxHSoSCUbTSYz+imEeDy+mgWnPr7hxklXgFqUGB5qD61R8mLIu5QiapMT3PTdHPqUbBJJ9W+pnhKWVjOuI9SxWNufHz+cFTcmaVIQkTbUshmau/J3IaGzOJA9sZU4zMsjcT//N6GYbXfi5UmiFXbLEozCTBhMy+J0OhOUM5sYQyLeythEVUU4Y2o4oNwVt+eZW0L+qeW/fuL2uNmyKOMpzAKZyDB1fQgDtoQgsYxPAMr/DmaOfFeXc+Fq0lp5g5hj9wPn8AxpOQYA==</latexit><latexit sha1_base64="na7C4enMR0C6wbXkJ8Qlg2nlvQc=">AAAB8HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cK9kPaUDbbTbN0swm7E6GE/govHhTx6s/x5r9x2+agrQ8GHu/NMDMvSKUw6LrfTmltfWNzq7xd2dnd2z+oHh61TZJpxlsskYnuBtRwKRRvoUDJu6nmNA4k7wTj25nfeeLaiEQ94CTlfkxHSoSCUbTSYz+imEeDy+mgWnPr7hxklXgFqUGB5qD61R8mLIu5QiapMT3PTdHPqUbBJJ9W+pnhKWVjOuI9SxWNufHz+cFTcmaVIQkTbUshmau/J3IaGzOJA9sZU4zMsjcT//N6GYbXfi5UmiFXbLEozCTBhMy+J0OhOUM5sYQyLeythEVUU4Y2o4oNwVt+eZW0L+qeW/fuL2uNmyKOMpzAKZyDB1fQgDtoQgsYxPAMr/DmaOfFeXc+Fq0lp5g5hj9wPn8AxpOQYA==</latexit><latexit sha1_base64="na7C4enMR0C6wbXkJ8Qlg2nlvQc=">AAAB8HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cK9kPaUDbbTbN0swm7E6GE/govHhTx6s/x5r9x2+agrQ8GHu/NMDMvSKUw6LrfTmltfWNzq7xd2dnd2z+oHh61TZJpxlsskYnuBtRwKRRvoUDJu6nmNA4k7wTj25nfeeLaiEQ94CTlfkxHSoSCUbTSYz+imEeDy+mgWnPr7hxklXgFqUGB5qD61R8mLIu5QiapMT3PTdHPqUbBJJ9W+pnhKWVjOuI9SxWNufHz+cFTcmaVIQkTbUshmau/J3IaGzOJA9sZU4zMsjcT//N6GYbXfi5UmiFXbLEozCTBhMy+J0OhOUM5sYQyLeythEVUU4Y2o4oNwVt+eZW0L+qeW/fuL2uNmyKOMpzAKZyDB1fQgDtoQgsYxPAMr/DmaOfFeXc+Fq0lp5g5hj9wPn8AxpOQYA==</latexit><latexit sha1_base64="hP+6LrUf2d3tZaldqaQQvEKMXyw=">AAAB2XicbZDNSgMxFIXv1L86Vq1rN8EiuCozbnQpuHFZwbZCO5RM5k4bmskMyR2hDH0BF25EfC93vo3pz0JbDwQ+zknIvSculLQUBN9ebWd3b/+gfugfNfzjk9Nmo2fz0gjsilzl5jnmFpXU2CVJCp8LgzyLFfbj6f0i77+gsTLXTzQrMMr4WMtUCk7O6oyaraAdLMW2IVxDC9YaNb+GSS7KDDUJxa0dhEFBUcUNSaFw7g9LiwUXUz7GgUPNM7RRtRxzzi6dk7A0N+5oYkv394uKZ9bOstjdzDhN7Ga2MP/LBiWlt1EldVESarH6KC0Vo5wtdmaJNChIzRxwYaSblYkJN1yQa8Z3HYSbG29D77odBu3wMYA6nMMFXEEIN3AHD9CBLghI4BXevYn35n2suqp569LO4I+8zx84xIo4</latexit><latexit sha1_base64="F9OWCat2IFkLcCw/zQvtuiXkmik=">AAAB5XicbZBLSwMxFIXv1FetVatbN8EiuCozItil4MZlBfuQdiiZNNMJTTJDckcoQ3+FGxeK+Jfc+W9MHwttPRD4OCch954ok8Ki7397pa3tnd298n7loHp4dFw7qXZsmhvG2yyVqelF1HIpNG+jQMl7meFURZJ3o8ndPO8+c2NFqh9xmvFQ0bEWsWAUnfU0SCgWyfB6NqzV/Ya/ENmEYAV1WKk1rH0NRinLFdfIJLW2H/gZhgU1KJjks8ogtzyjbELHvO9QU8VtWCwGnpEL54xInBp3NJKF+/tFQZW1UxW5m4piYtezuflf1s8xboaF0FmOXLPlR3EuCaZkvj0ZCcMZyqkDyoxwsxKWUEMZuo4qroRgfeVN6Fw1Ar8RPPhQhjM4h0sI4AZu4R5a0AYGCl7gDd494716H8u6St6qt1P4I+/zB44RjwU=</latexit><latexit sha1_base64="F9OWCat2IFkLcCw/zQvtuiXkmik=">AAAB5XicbZBLSwMxFIXv1FetVatbN8EiuCozItil4MZlBfuQdiiZNNMJTTJDckcoQ3+FGxeK+Jfc+W9MHwttPRD4OCch954ok8Ki7397pa3tnd298n7loHp4dFw7qXZsmhvG2yyVqelF1HIpNG+jQMl7meFURZJ3o8ndPO8+c2NFqh9xmvFQ0bEWsWAUnfU0SCgWyfB6NqzV/Ya/ENmEYAV1WKk1rH0NRinLFdfIJLW2H/gZhgU1KJjks8ogtzyjbELHvO9QU8VtWCwGnpEL54xInBp3NJKF+/tFQZW1UxW5m4piYtezuflf1s8xboaF0FmOXLPlR3EuCaZkvj0ZCcMZyqkDyoxwsxKWUEMZuo4qroRgfeVN6Fw1Ar8RPPhQhjM4h0sI4AZu4R5a0AYGCl7gDd494716H8u6St6qt1P4I+/zB44RjwU=</latexit><latexit sha1_base64="2nDsOCu0yi+rwGkOBLXxKnjeg20=">AAAB8HicbVBNS8NAEJ3Ur1q/qh69BIvgqSQi6LHoxWMF+yFtKJvtplm6uwm7E6GE/govHhTx6s/x5r9x2+agrQ8GHu/NMDMvTAU36HnfTmltfWNzq7xd2dnd2z+oHh61TZJpylo0EYnuhsQwwRVrIUfBuqlmRIaCdcLx7czvPDFteKIecJKyQJKR4hGnBK302I8J5vHgcjqo1ry6N4e7SvyC1KBAc1D96g8TmkmmkApiTM/3UgxyopFTwaaVfmZYSuiYjFjPUkUkM0E+P3jqnlll6EaJtqXQnau/J3IijZnI0HZKgrFZ9mbif14vw+g6yLlKM2SKLhZFmXAxcWffu0OuGUUxsYRQze2tLo2JJhRtRhUbgr/88ippX9R9r+7fe7XGTRFHGU7gFM7BhytowB00oQUUJDzDK7w52nlx3p2PRWvJKWaO4Q+czx/FU5Bc</latexit><latexit sha1_base64="na7C4enMR0C6wbXkJ8Qlg2nlvQc=">AAAB8HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cK9kPaUDbbTbN0swm7E6GE/govHhTx6s/x5r9x2+agrQ8GHu/NMDMvSKUw6LrfTmltfWNzq7xd2dnd2z+oHh61TZJpxlsskYnuBtRwKRRvoUDJu6nmNA4k7wTj25nfeeLaiEQ94CTlfkxHSoSCUbTSYz+imEeDy+mgWnPr7hxklXgFqUGB5qD61R8mLIu5QiapMT3PTdHPqUbBJJ9W+pnhKWVjOuI9SxWNufHz+cFTcmaVIQkTbUshmau/J3IaGzOJA9sZU4zMsjcT//N6GYbXfi5UmiFXbLEozCTBhMy+J0OhOUM5sYQyLeythEVUU4Y2o4oNwVt+eZW0L+qeW/fuL2uNmyKOMpzAKZyDB1fQgDtoQgsYxPAMr/DmaOfFeXc+Fq0lp5g5hj9wPn8AxpOQYA==</latexit><latexit sha1_base64="na7C4enMR0C6wbXkJ8Qlg2nlvQc=">AAAB8HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cK9kPaUDbbTbN0swm7E6GE/govHhTx6s/x5r9x2+agrQ8GHu/NMDMvSKUw6LrfTmltfWNzq7xd2dnd2z+oHh61TZJpxlsskYnuBtRwKRRvoUDJu6nmNA4k7wTj25nfeeLaiEQ94CTlfkxHSoSCUbTSYz+imEeDy+mgWnPr7hxklXgFqUGB5qD61R8mLIu5QiapMT3PTdHPqUbBJJ9W+pnhKWVjOuI9SxWNufHz+cFTcmaVIQkTbUshmau/J3IaGzOJA9sZU4zMsjcT//N6GYbXfi5UmiFXbLEozCTBhMy+J0OhOUM5sYQyLeythEVUU4Y2o4oNwVt+eZW0L+qeW/fuL2uNmyKOMpzAKZyDB1fQgDtoQgsYxPAMr/DmaOfFeXc+Fq0lp5g5hj9wPn8AxpOQYA==</latexit><latexit sha1_base64="na7C4enMR0C6wbXkJ8Qlg2nlvQc=">AAAB8HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cK9kPaUDbbTbN0swm7E6GE/govHhTx6s/x5r9x2+agrQ8GHu/NMDMvSKUw6LrfTmltfWNzq7xd2dnd2z+oHh61TZJpxlsskYnuBtRwKRRvoUDJu6nmNA4k7wTj25nfeeLaiEQ94CTlfkxHSoSCUbTSYz+imEeDy+mgWnPr7hxklXgFqUGB5qD61R8mLIu5QiapMT3PTdHPqUbBJJ9W+pnhKWVjOuI9SxWNufHz+cFTcmaVIQkTbUshmau/J3IaGzOJA9sZU4zMsjcT//N6GYbXfi5UmiFXbLEozCTBhMy+J0OhOUM5sYQyLeythEVUU4Y2o4oNwVt+eZW0L+qeW/fuL2uNmyKOMpzAKZyDB1fQgDtoQgsYxPAMr/DmaOfFeXc+Fq0lp5g5hj9wPn8AxpOQYA==</latexit><latexit sha1_base64="na7C4enMR0C6wbXkJ8Qlg2nlvQc=">AAAB8HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cK9kPaUDbbTbN0swm7E6GE/govHhTx6s/x5r9x2+agrQ8GHu/NMDMvSKUw6LrfTmltfWNzq7xd2dnd2z+oHh61TZJpxlsskYnuBtRwKRRvoUDJu6nmNA4k7wTj25nfeeLaiEQ94CTlfkxHSoSCUbTSYz+imEeDy+mgWnPr7hxklXgFqUGB5qD61R8mLIu5QiapMT3PTdHPqUbBJJ9W+pnhKWVjOuI9SxWNufHz+cFTcmaVIQkTbUshmau/J3IaGzOJA9sZU4zMsjcT//N6GYbXfi5UmiFXbLEozCTBhMy+J0OhOUM5sYQyLeythEVUU4Y2o4oNwVt+eZW0L+qeW/fuL2uNmyKOMpzAKZyDB1fQgDtoQgsYxPAMr/DmaOfFeXc+Fq0lp5g5hj9wPn8AxpOQYA==</latexit><latexit sha1_base64="na7C4enMR0C6wbXkJ8Qlg2nlvQc=">AAAB8HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cK9kPaUDbbTbN0swm7E6GE/govHhTx6s/x5r9x2+agrQ8GHu/NMDMvSKUw6LrfTmltfWNzq7xd2dnd2z+oHh61TZJpxlsskYnuBtRwKRRvoUDJu6nmNA4k7wTj25nfeeLaiEQ94CTlfkxHSoSCUbTSYz+imEeDy+mgWnPr7hxklXgFqUGB5qD61R8mLIu5QiapMT3PTdHPqUbBJJ9W+pnhKWVjOuI9SxWNufHz+cFTcmaVIQkTbUshmau/J3IaGzOJA9sZU4zMsjcT//N6GYbXfi5UmiFXbLEozCTBhMy+J0OhOUM5sYQyLeythEVUU4Y2o4oNwVt+eZW0L+qeW/fuL2uNmyKOMpzAKZyDB1fQgDtoQgsYxPAMr/DmaOfFeXc+Fq0lp5g5hj9wPn8AxpOQYA==</latexit><latexit sha1_base64="na7C4enMR0C6wbXkJ8Qlg2nlvQc=">AAAB8HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cK9kPaUDbbTbN0swm7E6GE/govHhTx6s/x5r9x2+agrQ8GHu/NMDMvSKUw6LrfTmltfWNzq7xd2dnd2z+oHh61TZJpxlsskYnuBtRwKRRvoUDJu6nmNA4k7wTj25nfeeLaiEQ94CTlfkxHSoSCUbTSYz+imEeDy+mgWnPr7hxklXgFqUGB5qD61R8mLIu5QiapMT3PTdHPqUbBJJ9W+pnhKWVjOuI9SxWNufHz+cFTcmaVIQkTbUshmau/J3IaGzOJA9sZU4zMsjcT//N6GYbXfi5UmiFXbLEozCTBhMy+J0OhOUM5sYQyLeythEVUU4Y2o4oNwVt+eZW0L+qeW/fuL2uNmyKOMpzAKZyDB1fQgDtoQgsYxPAMr/DmaOfFeXc+Fq0lp5g5hj9wPn8AxpOQYA==</latexit>

h1<latexit sha1_base64="dochqQ3t9vPo6NBOxL3kYGN6Txw=">AAAB8HicbVBNS8NAEJ3Ur1q/qh69BIvgqSQi6LHoxWMF+yFtKJvtplm6uwm7E6GE/govHhTx6s/x5r9x2+agrQ8GHu/NMDMvTAU36HnfTmltfWNzq7xd2dnd2z+oHh61TZJpylo0EYnuhsQwwRVrIUfBuqlmRIaCdcLx7czvPDFteKIecJKyQJKR4hGnBK302I8J5vHAnw6qNa/uzeGuEr8gNSjQHFS/+sOEZpIppIIY0/O9FIOcaORUsGmlnxmWEjomI9azVBHJTJDPD566Z1YZulGibSl05+rviZxIYyYytJ2SYGyWvZn4n9fLMLoOcq7SDJmii0VRJlxM3Nn37pBrRlFMLCFUc3urS2OiCUWbUcWG4C+/vEraF3Xfq/v3l7XGTRFHGU7gFM7BhytowB00oQUUJDzDK7w52nlx3p2PRWvJKWaO4Q+czx/CBJBd</latexit><latexit sha1_base64="dochqQ3t9vPo6NBOxL3kYGN6Txw=">AAAB8HicbVBNS8NAEJ3Ur1q/qh69BIvgqSQi6LHoxWMF+yFtKJvtplm6uwm7E6GE/govHhTx6s/x5r9x2+agrQ8GHu/NMDMvTAU36HnfTmltfWNzq7xd2dnd2z+oHh61TZJpylo0EYnuhsQwwRVrIUfBuqlmRIaCdcLx7czvPDFteKIecJKyQJKR4hGnBK302I8J5vHAnw6qNa/uzeGuEr8gNSjQHFS/+sOEZpIppIIY0/O9FIOcaORUsGmlnxmWEjomI9azVBHJTJDPD566Z1YZulGibSl05+rviZxIYyYytJ2SYGyWvZn4n9fLMLoOcq7SDJmii0VRJlxM3Nn37pBrRlFMLCFUc3urS2OiCUWbUcWG4C+/vEraF3Xfq/v3l7XGTRFHGU7gFM7BhytowB00oQUUJDzDK7w52nlx3p2PRWvJKWaO4Q+czx/CBJBd</latexit><latexit sha1_base64="dochqQ3t9vPo6NBOxL3kYGN6Txw=">AAAB8HicbVBNS8NAEJ3Ur1q/qh69BIvgqSQi6LHoxWMF+yFtKJvtplm6uwm7E6GE/govHhTx6s/x5r9x2+agrQ8GHu/NMDMvTAU36HnfTmltfWNzq7xd2dnd2z+oHh61TZJpylo0EYnuhsQwwRVrIUfBuqlmRIaCdcLx7czvPDFteKIecJKyQJKR4hGnBK302I8J5vHAnw6qNa/uzeGuEr8gNSjQHFS/+sOEZpIppIIY0/O9FIOcaORUsGmlnxmWEjomI9azVBHJTJDPD566Z1YZulGibSl05+rviZxIYyYytJ2SYGyWvZn4n9fLMLoOcq7SDJmii0VRJlxM3Nn37pBrRlFMLCFUc3urS2OiCUWbUcWG4C+/vEraF3Xfq/v3l7XGTRFHGU7gFM7BhytowB00oQUUJDzDK7w52nlx3p2PRWvJKWaO4Q+czx/CBJBd</latexit><latexit sha1_base64="dochqQ3t9vPo6NBOxL3kYGN6Txw=">AAAB8HicbVBNS8NAEJ3Ur1q/qh69BIvgqSQi6LHoxWMF+yFtKJvtplm6uwm7E6GE/govHhTx6s/x5r9x2+agrQ8GHu/NMDMvTAU36HnfTmltfWNzq7xd2dnd2z+oHh61TZJpylo0EYnuhsQwwRVrIUfBuqlmRIaCdcLx7czvPDFteKIecJKyQJKR4hGnBK302I8J5vHAnw6qNa/uzeGuEr8gNSjQHFS/+sOEZpIppIIY0/O9FIOcaORUsGmlnxmWEjomI9azVBHJTJDPD566Z1YZulGibSl05+rviZxIYyYytJ2SYGyWvZn4n9fLMLoOcq7SDJmii0VRJlxM3Nn37pBrRlFMLCFUc3urS2OiCUWbUcWG4C+/vEraF3Xfq/v3l7XGTRFHGU7gFM7BhytowB00oQUUJDzDK7w52nlx3p2PRWvJKWaO4Q+czx/CBJBd</latexit>

h5<latexit sha1_base64="nIdznJ6Kjoay6WratTCU5MxFLeM=">AAAB8HicbVBNS8NAEJ3Ur1q/qh69BIvgqSSi6LHoxWMFWyttKJvtplm6uwm7E6GE/govHhTx6s/x5r9x2+agrQ8GHu/NMDMvTAU36HnfTmlldW19o7xZ2dre2d2r7h+0TZJpylo0EYnuhMQwwRVrIUfBOqlmRIaCPYSjm6n/8MS04Ym6x3HKAkmGikecErTSYy8mmMf9i0m/WvPq3gzuMvELUoMCzX71qzdIaCaZQiqIMV3fSzHIiUZOBZtUeplhKaEjMmRdSxWRzAT57OCJe2KVgRsl2pZCd6b+nsiJNGYsQ9spCcZm0ZuK/3ndDKOrIOcqzZApOl8UZcLFxJ1+7w64ZhTF2BJCNbe3ujQmmlC0GVVsCP7iy8ukfVb3vbp/d15rXBdxlOEIjuEUfLiEBtxCE1pAQcIzvMKbo50X5935mLeWnGLmEP7A+fwByBiQYQ==</latexit><latexit sha1_base64="nIdznJ6Kjoay6WratTCU5MxFLeM=">AAAB8HicbVBNS8NAEJ3Ur1q/qh69BIvgqSSi6LHoxWMFWyttKJvtplm6uwm7E6GE/govHhTx6s/x5r9x2+agrQ8GHu/NMDMvTAU36HnfTmlldW19o7xZ2dre2d2r7h+0TZJpylo0EYnuhMQwwRVrIUfBOqlmRIaCPYSjm6n/8MS04Ym6x3HKAkmGikecErTSYy8mmMf9i0m/WvPq3gzuMvELUoMCzX71qzdIaCaZQiqIMV3fSzHIiUZOBZtUeplhKaEjMmRdSxWRzAT57OCJe2KVgRsl2pZCd6b+nsiJNGYsQ9spCcZm0ZuK/3ndDKOrIOcqzZApOl8UZcLFxJ1+7w64ZhTF2BJCNbe3ujQmmlC0GVVsCP7iy8ukfVb3vbp/d15rXBdxlOEIjuEUfLiEBtxCE1pAQcIzvMKbo50X5935mLeWnGLmEP7A+fwByBiQYQ==</latexit><latexit sha1_base64="nIdznJ6Kjoay6WratTCU5MxFLeM=">AAAB8HicbVBNS8NAEJ3Ur1q/qh69BIvgqSSi6LHoxWMFWyttKJvtplm6uwm7E6GE/govHhTx6s/x5r9x2+agrQ8GHu/NMDMvTAU36HnfTmlldW19o7xZ2dre2d2r7h+0TZJpylo0EYnuhMQwwRVrIUfBOqlmRIaCPYSjm6n/8MS04Ym6x3HKAkmGikecErTSYy8mmMf9i0m/WvPq3gzuMvELUoMCzX71qzdIaCaZQiqIMV3fSzHIiUZOBZtUeplhKaEjMmRdSxWRzAT57OCJe2KVgRsl2pZCd6b+nsiJNGYsQ9spCcZm0ZuK/3ndDKOrIOcqzZApOl8UZcLFxJ1+7w64ZhTF2BJCNbe3ujQmmlC0GVVsCP7iy8ukfVb3vbp/d15rXBdxlOEIjuEUfLiEBtxCE1pAQcIzvMKbo50X5935mLeWnGLmEP7A+fwByBiQYQ==</latexit><latexit sha1_base64="nIdznJ6Kjoay6WratTCU5MxFLeM=">AAAB8HicbVBNS8NAEJ3Ur1q/qh69BIvgqSSi6LHoxWMFWyttKJvtplm6uwm7E6GE/govHhTx6s/x5r9x2+agrQ8GHu/NMDMvTAU36HnfTmlldW19o7xZ2dre2d2r7h+0TZJpylo0EYnuhMQwwRVrIUfBOqlmRIaCPYSjm6n/8MS04Ym6x3HKAkmGikecErTSYy8mmMf9i0m/WvPq3gzuMvELUoMCzX71qzdIaCaZQiqIMV3fSzHIiUZOBZtUeplhKaEjMmRdSxWRzAT57OCJe2KVgRsl2pZCd6b+nsiJNGYsQ9spCcZm0ZuK/3ndDKOrIOcqzZApOl8UZcLFxJ1+7w64ZhTF2BJCNbe3ujQmmlC0GVVsCP7iy8ukfVb3vbp/d15rXBdxlOEIjuEUfLiEBtxCE1pAQcIzvMKbo50X5935mLeWnGLmEP7A+fwByBiQYQ==</latexit>

Bi-direct. GRU

Sent. Decoder

Sent. Decoder

Sent. Decoder

Sent. Decoder

Sent. Decoder

S1<latexit sha1_base64="a7Fo4ps7OQruarNUuL2BknYMm3E=">AAAB8HicbVA9SwNBEJ2LXzF+RS1tFoNgFe5E0DJoYxnRxEgSwt5mL1myu3fszgnhyK+wsVDE1p9j579xk1yhiQ8GHu/NMDMvTKSw6PvfXmFldW19o7hZ2tre2d0r7x80bZwaxhsslrFphdRyKTRvoEDJW4nhVIWSP4Sj66n/8MSNFbG+x3HCu4oOtIgEo+ikx86QYnbXCya9csWv+jOQZRLkpAI56r3yV6cfs1RxjUxSa9uBn2A3owYFk3xS6qSWJ5SN6IC3HdVUcdvNZgdPyIlT+iSKjSuNZKb+nsiosnasQtepKA7tojcV//PaKUaX3UzoJEWu2XxRlEqCMZl+T/rCcIZy7AhlRrhbCRtSQxm6jEouhGDx5WXSPKsGfjW4Pa/UrvI4inAEx3AKAVxADW6gDg1goOAZXuHNM96L9+59zFsLXj5zCH/gff4AofGQSA==</latexit><latexit sha1_base64="a7Fo4ps7OQruarNUuL2BknYMm3E=">AAAB8HicbVA9SwNBEJ2LXzF+RS1tFoNgFe5E0DJoYxnRxEgSwt5mL1myu3fszgnhyK+wsVDE1p9j579xk1yhiQ8GHu/NMDMvTKSw6PvfXmFldW19o7hZ2tre2d0r7x80bZwaxhsslrFphdRyKTRvoEDJW4nhVIWSP4Sj66n/8MSNFbG+x3HCu4oOtIgEo+ikx86QYnbXCya9csWv+jOQZRLkpAI56r3yV6cfs1RxjUxSa9uBn2A3owYFk3xS6qSWJ5SN6IC3HdVUcdvNZgdPyIlT+iSKjSuNZKb+nsiosnasQtepKA7tojcV//PaKUaX3UzoJEWu2XxRlEqCMZl+T/rCcIZy7AhlRrhbCRtSQxm6jEouhGDx5WXSPKsGfjW4Pa/UrvI4inAEx3AKAVxADW6gDg1goOAZXuHNM96L9+59zFsLXj5zCH/gff4AofGQSA==</latexit><latexit sha1_base64="a7Fo4ps7OQruarNUuL2BknYMm3E=">AAAB8HicbVA9SwNBEJ2LXzF+RS1tFoNgFe5E0DJoYxnRxEgSwt5mL1myu3fszgnhyK+wsVDE1p9j579xk1yhiQ8GHu/NMDMvTKSw6PvfXmFldW19o7hZ2tre2d0r7x80bZwaxhsslrFphdRyKTRvoEDJW4nhVIWSP4Sj66n/8MSNFbG+x3HCu4oOtIgEo+ikx86QYnbXCya9csWv+jOQZRLkpAI56r3yV6cfs1RxjUxSa9uBn2A3owYFk3xS6qSWJ5SN6IC3HdVUcdvNZgdPyIlT+iSKjSuNZKb+nsiosnasQtepKA7tojcV//PaKUaX3UzoJEWu2XxRlEqCMZl+T/rCcIZy7AhlRrhbCRtSQxm6jEouhGDx5WXSPKsGfjW4Pa/UrvI4inAEx3AKAVxADW6gDg1goOAZXuHNM96L9+59zFsLXj5zCH/gff4AofGQSA==</latexit><latexit sha1_base64="a7Fo4ps7OQruarNUuL2BknYMm3E=">AAAB8HicbVA9SwNBEJ2LXzF+RS1tFoNgFe5E0DJoYxnRxEgSwt5mL1myu3fszgnhyK+wsVDE1p9j579xk1yhiQ8GHu/NMDMvTKSw6PvfXmFldW19o7hZ2tre2d0r7x80bZwaxhsslrFphdRyKTRvoEDJW4nhVIWSP4Sj66n/8MSNFbG+x3HCu4oOtIgEo+ikx86QYnbXCya9csWv+jOQZRLkpAI56r3yV6cfs1RxjUxSa9uBn2A3owYFk3xS6qSWJ5SN6IC3HdVUcdvNZgdPyIlT+iSKjSuNZKb+nsiosnasQtepKA7tojcV//PaKUaX3UzoJEWu2XxRlEqCMZl+T/rCcIZy7AhlRrhbCRtSQxm6jEouhGDx5WXSPKsGfjW4Pa/UrvI4inAEx3AKAVxADW6gDg1goOAZXuHNM96L9+59zFsLXj5zCH/gff4AofGQSA==</latexit>

S2<latexit sha1_base64="EvRZmqv8k0ZBkGCPnfECbBJz4q4=">AAAB8HicbVBNS8NAEJ3Ur1q/qh69LBbBU0mKoMeiF48V7Ye0oWy2m3bpbhJ2J0IJ/RVePCji1Z/jzX/jts1BWx8MPN6bYWZekEhh0HW/ncLa+sbmVnG7tLO7t39QPjxqmTjVjDdZLGPdCajhUkS8iQIl7ySaUxVI3g7GNzO//cS1EXH0gJOE+4oOIxEKRtFKj70Rxey+X5v2yxW36s5BVomXkwrkaPTLX71BzFLFI2SSGtP13AT9jGoUTPJpqZcanlA2pkPetTSiihs/mx88JWdWGZAw1rYiJHP190RGlTETFdhORXFklr2Z+J/XTTG88jMRJSnyiC0WhakkGJPZ92QgNGcoJ5ZQpoW9lbAR1ZShzahkQ/CWX14lrVrVc6ve3UWlfp3HUYQTOIVz8OAS6nALDWgCAwXP8ApvjnZenHfnY9FacPKZY/gD5/MHo3aQSQ==</latexit><latexit sha1_base64="EvRZmqv8k0ZBkGCPnfECbBJz4q4=">AAAB8HicbVBNS8NAEJ3Ur1q/qh69LBbBU0mKoMeiF48V7Ye0oWy2m3bpbhJ2J0IJ/RVePCji1Z/jzX/jts1BWx8MPN6bYWZekEhh0HW/ncLa+sbmVnG7tLO7t39QPjxqmTjVjDdZLGPdCajhUkS8iQIl7ySaUxVI3g7GNzO//cS1EXH0gJOE+4oOIxEKRtFKj70Rxey+X5v2yxW36s5BVomXkwrkaPTLX71BzFLFI2SSGtP13AT9jGoUTPJpqZcanlA2pkPetTSiihs/mx88JWdWGZAw1rYiJHP190RGlTETFdhORXFklr2Z+J/XTTG88jMRJSnyiC0WhakkGJPZ92QgNGcoJ5ZQpoW9lbAR1ZShzahkQ/CWX14lrVrVc6ve3UWlfp3HUYQTOIVz8OAS6nALDWgCAwXP8ApvjnZenHfnY9FacPKZY/gD5/MHo3aQSQ==</latexit><latexit sha1_base64="EvRZmqv8k0ZBkGCPnfECbBJz4q4=">AAAB8HicbVBNS8NAEJ3Ur1q/qh69LBbBU0mKoMeiF48V7Ye0oWy2m3bpbhJ2J0IJ/RVePCji1Z/jzX/jts1BWx8MPN6bYWZekEhh0HW/ncLa+sbmVnG7tLO7t39QPjxqmTjVjDdZLGPdCajhUkS8iQIl7ySaUxVI3g7GNzO//cS1EXH0gJOE+4oOIxEKRtFKj70Rxey+X5v2yxW36s5BVomXkwrkaPTLX71BzFLFI2SSGtP13AT9jGoUTPJpqZcanlA2pkPetTSiihs/mx88JWdWGZAw1rYiJHP190RGlTETFdhORXFklr2Z+J/XTTG88jMRJSnyiC0WhakkGJPZ92QgNGcoJ5ZQpoW9lbAR1ZShzahkQ/CWX14lrVrVc6ve3UWlfp3HUYQTOIVz8OAS6nALDWgCAwXP8ApvjnZenHfnY9FacPKZY/gD5/MHo3aQSQ==</latexit><latexit sha1_base64="hP+6LrUf2d3tZaldqaQQvEKMXyw=">AAAB2XicbZDNSgMxFIXv1L86Vq1rN8EiuCozbnQpuHFZwbZCO5RM5k4bmskMyR2hDH0BF25EfC93vo3pz0JbDwQ+zknIvSculLQUBN9ebWd3b/+gfugfNfzjk9Nmo2fz0gjsilzl5jnmFpXU2CVJCp8LgzyLFfbj6f0i77+gsTLXTzQrMMr4WMtUCk7O6oyaraAdLMW2IVxDC9YaNb+GSS7KDDUJxa0dhEFBUcUNSaFw7g9LiwUXUz7GgUPNM7RRtRxzzi6dk7A0N+5oYkv394uKZ9bOstjdzDhN7Ga2MP/LBiWlt1EldVESarH6KC0Vo5wtdmaJNChIzRxwYaSblYkJN1yQa8Z3HYSbG29D77odBu3wMYA6nMMFXEEIN3AHD9CBLghI4BXevYn35n2suqp569LO4I+8zx84xIo4</latexit><latexit sha1_base64="tMiWwctrEsyLC+905KWbD0d427k=">AAAB5XicbZDNSgMxFIXv1L9aq1a3boJFcFVmutGl4MZlRaettEPJpJk2NMkMyR2hDH0KNy4U8ZXc+TamPwttPRD4OCch9544k8Ki7397pa3tnd298n7loHp4dFw7qbZtmhvGQ5bK1HRjarkUmocoUPJuZjhVseSdeHI7zzvP3FiR6kecZjxSdKRFIhhFZz31xxSLh0FzNqjV/Ya/ENmEYAV1WKk1qH31hynLFdfIJLW2F/gZRgU1KJjks0o/tzyjbEJHvOdQU8VtVCwGnpEL5wxJkhp3NJKF+/tFQZW1UxW7m4ri2K5nc/O/rJdjch0VQmc5cs2WHyW5JJiS+fZkKAxnKKcOKDPCzUrYmBrK0HVUcSUE6ytvQrvZCPxGcO9DGc7gHC4hgCu4gTtoQQgMFLzAG7x7xnv1PpZ1lbxVb6fwR97nD2vxju4=</latexit><latexit sha1_base64="tMiWwctrEsyLC+905KWbD0d427k=">AAAB5XicbZDNSgMxFIXv1L9aq1a3boJFcFVmutGl4MZlRaettEPJpJk2NMkMyR2hDH0KNy4U8ZXc+TamPwttPRD4OCch9544k8Ki7397pa3tnd298n7loHp4dFw7qbZtmhvGQ5bK1HRjarkUmocoUPJuZjhVseSdeHI7zzvP3FiR6kecZjxSdKRFIhhFZz31xxSLh0FzNqjV/Ya/ENmEYAV1WKk1qH31hynLFdfIJLW2F/gZRgU1KJjks0o/tzyjbEJHvOdQU8VtVCwGnpEL5wxJkhp3NJKF+/tFQZW1UxW7m4ri2K5nc/O/rJdjch0VQmc5cs2WHyW5JJiS+fZkKAxnKKcOKDPCzUrYmBrK0HVUcSUE6ytvQrvZCPxGcO9DGc7gHC4hgCu4gTtoQQgMFLzAG7x7xnv1PpZ1lbxVb6fwR97nD2vxju4=</latexit><latexit sha1_base64="pVbJjCDRYadWzC2rfjdYJQK77VI=">AAAB8HicbVA9SwNBEJ3zM8avqKXNYhCswl0aLYM2lhHNhyRH2NtskiW7e8funBCO/AobC0Vs/Tl2/hs3yRWa+GDg8d4MM/OiRAqLvv/tra1vbG5tF3aKu3v7B4elo+OmjVPDeIPFMjbtiFouheYNFCh5OzGcqkjyVjS+mfmtJ26siPUDThIeKjrUYiAYRSc9dkcUs/teddorlf2KPwdZJUFOypCj3it9dfsxSxXXyCS1thP4CYYZNSiY5NNiN7U8oWxMh7zjqKaK2zCbHzwl507pk0FsXGkkc/X3REaVtRMVuU5FcWSXvZn4n9dJcXAVZkInKXLNFosGqSQYk9n3pC8MZygnjlBmhLuVsBE1lKHLqOhCCJZfXiXNaiXwK8GdX65d53EU4BTO4AICuIQa3EIdGsBAwTO8wptnvBfv3ftYtK55+cwJ/IH3+QOiNpBF</latexit><latexit sha1_base64="EvRZmqv8k0ZBkGCPnfECbBJz4q4=">AAAB8HicbVBNS8NAEJ3Ur1q/qh69LBbBU0mKoMeiF48V7Ye0oWy2m3bpbhJ2J0IJ/RVePCji1Z/jzX/jts1BWx8MPN6bYWZekEhh0HW/ncLa+sbmVnG7tLO7t39QPjxqmTjVjDdZLGPdCajhUkS8iQIl7ySaUxVI3g7GNzO//cS1EXH0gJOE+4oOIxEKRtFKj70Rxey+X5v2yxW36s5BVomXkwrkaPTLX71BzFLFI2SSGtP13AT9jGoUTPJpqZcanlA2pkPetTSiihs/mx88JWdWGZAw1rYiJHP190RGlTETFdhORXFklr2Z+J/XTTG88jMRJSnyiC0WhakkGJPZ92QgNGcoJ5ZQpoW9lbAR1ZShzahkQ/CWX14lrVrVc6ve3UWlfp3HUYQTOIVz8OAS6nALDWgCAwXP8ApvjnZenHfnY9FacPKZY/gD5/MHo3aQSQ==</latexit><latexit sha1_base64="EvRZmqv8k0ZBkGCPnfECbBJz4q4=">AAAB8HicbVBNS8NAEJ3Ur1q/qh69LBbBU0mKoMeiF48V7Ye0oWy2m3bpbhJ2J0IJ/RVePCji1Z/jzX/jts1BWx8MPN6bYWZekEhh0HW/ncLa+sbmVnG7tLO7t39QPjxqmTjVjDdZLGPdCajhUkS8iQIl7ySaUxVI3g7GNzO//cS1EXH0gJOE+4oOIxEKRtFKj70Rxey+X5v2yxW36s5BVomXkwrkaPTLX71BzFLFI2SSGtP13AT9jGoUTPJpqZcanlA2pkPetTSiihs/mx88JWdWGZAw1rYiJHP190RGlTETFdhORXFklr2Z+J/XTTG88jMRJSnyiC0WhakkGJPZ92QgNGcoJ5ZQpoW9lbAR1ZShzahkQ/CWX14lrVrVc6ve3UWlfp3HUYQTOIVz8OAS6nALDWgCAwXP8ApvjnZenHfnY9FacPKZY/gD5/MHo3aQSQ==</latexit><latexit sha1_base64="EvRZmqv8k0ZBkGCPnfECbBJz4q4=">AAAB8HicbVBNS8NAEJ3Ur1q/qh69LBbBU0mKoMeiF48V7Ye0oWy2m3bpbhJ2J0IJ/RVePCji1Z/jzX/jts1BWx8MPN6bYWZekEhh0HW/ncLa+sbmVnG7tLO7t39QPjxqmTjVjDdZLGPdCajhUkS8iQIl7ySaUxVI3g7GNzO//cS1EXH0gJOE+4oOIxEKRtFKj70Rxey+X5v2yxW36s5BVomXkwrkaPTLX71BzFLFI2SSGtP13AT9jGoUTPJpqZcanlA2pkPetTSiihs/mx88JWdWGZAw1rYiJHP190RGlTETFdhORXFklr2Z+J/XTTG88jMRJSnyiC0WhakkGJPZ92QgNGcoJ5ZQpoW9lbAR1ZShzahkQ/CWX14lrVrVc6ve3UWlfp3HUYQTOIVz8OAS6nALDWgCAwXP8ApvjnZenHfnY9FacPKZY/gD5/MHo3aQSQ==</latexit><latexit sha1_base64="EvRZmqv8k0ZBkGCPnfECbBJz4q4=">AAAB8HicbVBNS8NAEJ3Ur1q/qh69LBbBU0mKoMeiF48V7Ye0oWy2m3bpbhJ2J0IJ/RVePCji1Z/jzX/jts1BWx8MPN6bYWZekEhh0HW/ncLa+sbmVnG7tLO7t39QPjxqmTjVjDdZLGPdCajhUkS8iQIl7ySaUxVI3g7GNzO//cS1EXH0gJOE+4oOIxEKRtFKj70Rxey+X5v2yxW36s5BVomXkwrkaPTLX71BzFLFI2SSGtP13AT9jGoUTPJpqZcanlA2pkPetTSiihs/mx88JWdWGZAw1rYiJHP190RGlTETFdhORXFklr2Z+J/XTTG88jMRJSnyiC0WhakkGJPZ92QgNGcoJ5ZQpoW9lbAR1ZShzahkQ/CWX14lrVrVc6ve3UWlfp3HUYQTOIVz8OAS6nALDWgCAwXP8ApvjnZenHfnY9FacPKZY/gD5/MHo3aQSQ==</latexit><latexit sha1_base64="EvRZmqv8k0ZBkGCPnfECbBJz4q4=">AAAB8HicbVBNS8NAEJ3Ur1q/qh69LBbBU0mKoMeiF48V7Ye0oWy2m3bpbhJ2J0IJ/RVePCji1Z/jzX/jts1BWx8MPN6bYWZekEhh0HW/ncLa+sbmVnG7tLO7t39QPjxqmTjVjDdZLGPdCajhUkS8iQIl7ySaUxVI3g7GNzO//cS1EXH0gJOE+4oOIxEKRtFKj70Rxey+X5v2yxW36s5BVomXkwrkaPTLX71BzFLFI2SSGtP13AT9jGoUTPJpqZcanlA2pkPetTSiihs/mx88JWdWGZAw1rYiJHP190RGlTETFdhORXFklr2Z+J/XTTG88jMRJSnyiC0WhakkGJPZ92QgNGcoJ5ZQpoW9lbAR1ZShzahkQ/CWX14lrVrVc6ve3UWlfp3HUYQTOIVz8OAS6nALDWgCAwXP8ApvjnZenHfnY9FacPKZY/gD5/MHo3aQSQ==</latexit><latexit sha1_base64="EvRZmqv8k0ZBkGCPnfECbBJz4q4=">AAAB8HicbVBNS8NAEJ3Ur1q/qh69LBbBU0mKoMeiF48V7Ye0oWy2m3bpbhJ2J0IJ/RVePCji1Z/jzX/jts1BWx8MPN6bYWZekEhh0HW/ncLa+sbmVnG7tLO7t39QPjxqmTjVjDdZLGPdCajhUkS8iQIl7ySaUxVI3g7GNzO//cS1EXH0gJOE+4oOIxEKRtFKj70Rxey+X5v2yxW36s5BVomXkwrkaPTLX71BzFLFI2SSGtP13AT9jGoUTPJpqZcanlA2pkPetTSiihs/mx88JWdWGZAw1rYiJHP190RGlTETFdhORXFklr2Z+J/XTTG88jMRJSnyiC0WhakkGJPZ92QgNGcoJ5ZQpoW9lbAR1ZShzahkQ/CWX14lrVrVc6ve3UWlfp3HUYQTOIVz8OAS6nALDWgCAwXP8ApvjnZenHfnY9FacPKZY/gD5/MHo3aQSQ==</latexit>

S3<latexit sha1_base64="lgrr4rXuSWAn2I2EwBNCB/EqU+k=">AAAB8HicbVBNS8NAEJ3Ur1q/qh69BIvgqSQq6LHoxWNF+yFtKJvtpl26uwm7E6GE/govHhTx6s/x5r9x2+agrQ8GHu/NMDMvTAQ36HnfTmFldW19o7hZ2tre2d0r7x80TZxqyho0FrFuh8QwwRVrIEfB2olmRIaCtcLRzdRvPTFteKwecJywQJKB4hGnBK302B0SzO5755NeueJVvRncZeLnpAI56r3yV7cf01QyhVQQYzq+l2CQEY2cCjYpdVPDEkJHZMA6lioimQmy2cET98QqfTeKtS2F7kz9PZERacxYhrZTEhyaRW8q/ud1UoyugoyrJEWm6HxRlAoXY3f6vdvnmlEUY0sI1dze6tIh0YSizahkQ/AXX14mzbOq71X9u4tK7TqPowhHcAyn4MMl1OAW6tAAChKe4RXeHO28OO/Ox7y14OQzh/AHzucPpPuQSg==</latexit><latexit sha1_base64="lgrr4rXuSWAn2I2EwBNCB/EqU+k=">AAAB8HicbVBNS8NAEJ3Ur1q/qh69BIvgqSQq6LHoxWNF+yFtKJvtpl26uwm7E6GE/govHhTx6s/x5r9x2+agrQ8GHu/NMDMvTAQ36HnfTmFldW19o7hZ2tre2d0r7x80TZxqyho0FrFuh8QwwRVrIEfB2olmRIaCtcLRzdRvPTFteKwecJywQJKB4hGnBK302B0SzO5755NeueJVvRncZeLnpAI56r3yV7cf01QyhVQQYzq+l2CQEY2cCjYpdVPDEkJHZMA6lioimQmy2cET98QqfTeKtS2F7kz9PZERacxYhrZTEhyaRW8q/ud1UoyugoyrJEWm6HxRlAoXY3f6vdvnmlEUY0sI1dze6tIh0YSizahkQ/AXX14mzbOq71X9u4tK7TqPowhHcAyn4MMl1OAW6tAAChKe4RXeHO28OO/Ox7y14OQzh/AHzucPpPuQSg==</latexit><latexit sha1_base64="lgrr4rXuSWAn2I2EwBNCB/EqU+k=">AAAB8HicbVBNS8NAEJ3Ur1q/qh69BIvgqSQq6LHoxWNF+yFtKJvtpl26uwm7E6GE/govHhTx6s/x5r9x2+agrQ8GHu/NMDMvTAQ36HnfTmFldW19o7hZ2tre2d0r7x80TZxqyho0FrFuh8QwwRVrIEfB2olmRIaCtcLRzdRvPTFteKwecJywQJKB4hGnBK302B0SzO5755NeueJVvRncZeLnpAI56r3yV7cf01QyhVQQYzq+l2CQEY2cCjYpdVPDEkJHZMA6lioimQmy2cET98QqfTeKtS2F7kz9PZERacxYhrZTEhyaRW8q/ud1UoyugoyrJEWm6HxRlAoXY3f6vdvnmlEUY0sI1dze6tIh0YSizahkQ/AXX14mzbOq71X9u4tK7TqPowhHcAyn4MMl1OAW6tAAChKe4RXeHO28OO/Ox7y14OQzh/AHzucPpPuQSg==</latexit><latexit sha1_base64="lgrr4rXuSWAn2I2EwBNCB/EqU+k=">AAAB8HicbVBNS8NAEJ3Ur1q/qh69BIvgqSQq6LHoxWNF+yFtKJvtpl26uwm7E6GE/govHhTx6s/x5r9x2+agrQ8GHu/NMDMvTAQ36HnfTmFldW19o7hZ2tre2d0r7x80TZxqyho0FrFuh8QwwRVrIEfB2olmRIaCtcLRzdRvPTFteKwecJywQJKB4hGnBK302B0SzO5755NeueJVvRncZeLnpAI56r3yV7cf01QyhVQQYzq+l2CQEY2cCjYpdVPDEkJHZMA6lioimQmy2cET98QqfTeKtS2F7kz9PZERacxYhrZTEhyaRW8q/ud1UoyugoyrJEWm6HxRlAoXY3f6vdvnmlEUY0sI1dze6tIh0YSizahkQ/AXX14mzbOq71X9u4tK7TqPowhHcAyn4MMl1OAW6tAAChKe4RXeHO28OO/Ox7y14OQzh/AHzucPpPuQSg==</latexit>

S4<latexit sha1_base64="hWfIhyLO9Eu4xlCF9ryscNsrcWM=">AAAB8HicbVBNS8NAEJ3Ur1q/qh69LBbBU0mkoMeiF48V7Ye0oWy2m3bpbhJ2J0IJ/RVePCji1Z/jzX/jts1BWx8MPN6bYWZekEhh0HW/ncLa+sbmVnG7tLO7t39QPjxqmTjVjDdZLGPdCajhUkS8iQIl7ySaUxVI3g7GNzO//cS1EXH0gJOE+4oOIxEKRtFKj70Rxey+X5v2yxW36s5BVomXkwrkaPTLX71BzFLFI2SSGtP13AT9jGoUTPJpqZcanlA2pkPetTSiihs/mx88JWdWGZAw1rYiJHP190RGlTETFdhORXFklr2Z+J/XTTG88jMRJSnyiC0WhakkGJPZ92QgNGcoJ5ZQpoW9lbAR1ZShzahkQ/CWX14lrYuq51a9u1qlfp3HUYQTOIVz8OAS6nALDWgCAwXP8ApvjnZenHfnY9FacPKZY/gD5/MHpoCQSw==</latexit><latexit sha1_base64="hWfIhyLO9Eu4xlCF9ryscNsrcWM=">AAAB8HicbVBNS8NAEJ3Ur1q/qh69LBbBU0mkoMeiF48V7Ye0oWy2m3bpbhJ2J0IJ/RVePCji1Z/jzX/jts1BWx8MPN6bYWZekEhh0HW/ncLa+sbmVnG7tLO7t39QPjxqmTjVjDdZLGPdCajhUkS8iQIl7ySaUxVI3g7GNzO//cS1EXH0gJOE+4oOIxEKRtFKj70Rxey+X5v2yxW36s5BVomXkwrkaPTLX71BzFLFI2SSGtP13AT9jGoUTPJpqZcanlA2pkPetTSiihs/mx88JWdWGZAw1rYiJHP190RGlTETFdhORXFklr2Z+J/XTTG88jMRJSnyiC0WhakkGJPZ92QgNGcoJ5ZQpoW9lbAR1ZShzahkQ/CWX14lrYuq51a9u1qlfp3HUYQTOIVz8OAS6nALDWgCAwXP8ApvjnZenHfnY9FacPKZY/gD5/MHpoCQSw==</latexit><latexit sha1_base64="hWfIhyLO9Eu4xlCF9ryscNsrcWM=">AAAB8HicbVBNS8NAEJ3Ur1q/qh69LBbBU0mkoMeiF48V7Ye0oWy2m3bpbhJ2J0IJ/RVePCji1Z/jzX/jts1BWx8MPN6bYWZekEhh0HW/ncLa+sbmVnG7tLO7t39QPjxqmTjVjDdZLGPdCajhUkS8iQIl7ySaUxVI3g7GNzO//cS1EXH0gJOE+4oOIxEKRtFKj70Rxey+X5v2yxW36s5BVomXkwrkaPTLX71BzFLFI2SSGtP13AT9jGoUTPJpqZcanlA2pkPetTSiihs/mx88JWdWGZAw1rYiJHP190RGlTETFdhORXFklr2Z+J/XTTG88jMRJSnyiC0WhakkGJPZ92QgNGcoJ5ZQpoW9lbAR1ZShzahkQ/CWX14lrYuq51a9u1qlfp3HUYQTOIVz8OAS6nALDWgCAwXP8ApvjnZenHfnY9FacPKZY/gD5/MHpoCQSw==</latexit><latexit sha1_base64="hP+6LrUf2d3tZaldqaQQvEKMXyw=">AAAB2XicbZDNSgMxFIXv1L86Vq1rN8EiuCozbnQpuHFZwbZCO5RM5k4bmskMyR2hDH0BF25EfC93vo3pz0JbDwQ+zknIvSculLQUBN9ebWd3b/+gfugfNfzjk9Nmo2fz0gjsilzl5jnmFpXU2CVJCp8LgzyLFfbj6f0i77+gsTLXTzQrMMr4WMtUCk7O6oyaraAdLMW2IVxDC9YaNb+GSS7KDDUJxa0dhEFBUcUNSaFw7g9LiwUXUz7GgUPNM7RRtRxzzi6dk7A0N+5oYkv394uKZ9bOstjdzDhN7Ga2MP/LBiWlt1EldVESarH6KC0Vo5wtdmaJNChIzRxwYaSblYkJN1yQa8Z3HYSbG29D77odBu3wMYA6nMMFXEEIN3AHD9CBLghI4BXevYn35n2suqp569LO4I+8zx84xIo4</latexit><latexit sha1_base64="H2dMCEIt4GQSxCf+Q3grO/+RwlY=">AAAB5XicbZDNSgMxFIXv1L9aq1a3boJFcFVmRNCl4MZlRaettEPJpJk2NMkMyR2hDH0KNy4U8ZXc+TamPwttPRD4OCch9544k8Ki7397pY3Nre2d8m5lr7p/cFg7qrZsmhvGQ5bK1HRiarkUmocoUPJOZjhVseTteHw7y9vP3FiR6kecZDxSdKhFIhhFZz31RhSLh/7ltF+r+w1/LrIOwRLqsFSzX/vqDVKWK66RSWptN/AzjApqUDDJp5VebnlG2ZgOedehporbqJgPPCVnzhmQJDXuaCRz9/eLgiprJyp2NxXFkV3NZuZ/WTfH5DoqhM5y5JotPkpySTAls+3JQBjOUE4cUGaEm5WwETWUoeuo4koIVldeh9ZFI/Abwb0PZTiBUziHAK7gBu6gCSEwUPACb/DuGe/V+1jUVfKWvR3DH3mfP27ljvA=</latexit><latexit sha1_base64="H2dMCEIt4GQSxCf+Q3grO/+RwlY=">AAAB5XicbZDNSgMxFIXv1L9aq1a3boJFcFVmRNCl4MZlRaettEPJpJk2NMkMyR2hDH0KNy4U8ZXc+TamPwttPRD4OCch9544k8Ki7397pY3Nre2d8m5lr7p/cFg7qrZsmhvGQ5bK1HRiarkUmocoUPJOZjhVseTteHw7y9vP3FiR6kecZDxSdKhFIhhFZz31RhSLh/7ltF+r+w1/LrIOwRLqsFSzX/vqDVKWK66RSWptN/AzjApqUDDJp5VebnlG2ZgOedehporbqJgPPCVnzhmQJDXuaCRz9/eLgiprJyp2NxXFkV3NZuZ/WTfH5DoqhM5y5JotPkpySTAls+3JQBjOUE4cUGaEm5WwETWUoeuo4koIVldeh9ZFI/Abwb0PZTiBUziHAK7gBu6gCSEwUPACb/DuGe/V+1jUVfKWvR3DH3mfP27ljvA=</latexit><latexit sha1_base64="aGU2DKT5YLYCqqzN3zE3gHqhRCk=">AAAB8HicbVA9SwNBEJ2LXzF+RS1tFoNgFe5E0DJoYxnRxEgSwt5mL1myu3fszgnhyK+wsVDE1p9j579xk1yhiQ8GHu/NMDMvTKSw6PvfXmFldW19o7hZ2tre2d0r7x80bZwaxhsslrFphdRyKTRvoEDJW4nhVIWSP4Sj66n/8MSNFbG+x3HCu4oOtIgEo+ikx86QYnbXO5/0yhW/6s9AlkmQkwrkqPfKX51+zFLFNTJJrW0HfoLdjBoUTPJJqZNanlA2ogPedlRTxW03mx08ISdO6ZMoNq40kpn6eyKjytqxCl2noji0i95U/M9rpxhddjOhkxS5ZvNFUSoJxmT6PekLwxnKsSOUGeFuJWxIDWXoMiq5EILFl5dJ86wa+NXg1q/UrvI4inAEx3AKAVxADW6gDg1goOAZXuHNM96L9+59zFsLXj5zCH/gff4ApUCQRw==</latexit><latexit sha1_base64="hWfIhyLO9Eu4xlCF9ryscNsrcWM=">AAAB8HicbVBNS8NAEJ3Ur1q/qh69LBbBU0mkoMeiF48V7Ye0oWy2m3bpbhJ2J0IJ/RVePCji1Z/jzX/jts1BWx8MPN6bYWZekEhh0HW/ncLa+sbmVnG7tLO7t39QPjxqmTjVjDdZLGPdCajhUkS8iQIl7ySaUxVI3g7GNzO//cS1EXH0gJOE+4oOIxEKRtFKj70Rxey+X5v2yxW36s5BVomXkwrkaPTLX71BzFLFI2SSGtP13AT9jGoUTPJpqZcanlA2pkPetTSiihs/mx88JWdWGZAw1rYiJHP190RGlTETFdhORXFklr2Z+J/XTTG88jMRJSnyiC0WhakkGJPZ92QgNGcoJ5ZQpoW9lbAR1ZShzahkQ/CWX14lrYuq51a9u1qlfp3HUYQTOIVz8OAS6nALDWgCAwXP8ApvjnZenHfnY9FacPKZY/gD5/MHpoCQSw==</latexit><latexit sha1_base64="hWfIhyLO9Eu4xlCF9ryscNsrcWM=">AAAB8HicbVBNS8NAEJ3Ur1q/qh69LBbBU0mkoMeiF48V7Ye0oWy2m3bpbhJ2J0IJ/RVePCji1Z/jzX/jts1BWx8MPN6bYWZekEhh0HW/ncLa+sbmVnG7tLO7t39QPjxqmTjVjDdZLGPdCajhUkS8iQIl7ySaUxVI3g7GNzO//cS1EXH0gJOE+4oOIxEKRtFKj70Rxey+X5v2yxW36s5BVomXkwrkaPTLX71BzFLFI2SSGtP13AT9jGoUTPJpqZcanlA2pkPetTSiihs/mx88JWdWGZAw1rYiJHP190RGlTETFdhORXFklr2Z+J/XTTG88jMRJSnyiC0WhakkGJPZ92QgNGcoJ5ZQpoW9lbAR1ZShzahkQ/CWX14lrYuq51a9u1qlfp3HUYQTOIVz8OAS6nALDWgCAwXP8ApvjnZenHfnY9FacPKZY/gD5/MHpoCQSw==</latexit><latexit sha1_base64="hWfIhyLO9Eu4xlCF9ryscNsrcWM=">AAAB8HicbVBNS8NAEJ3Ur1q/qh69LBbBU0mkoMeiF48V7Ye0oWy2m3bpbhJ2J0IJ/RVePCji1Z/jzX/jts1BWx8MPN6bYWZekEhh0HW/ncLa+sbmVnG7tLO7t39QPjxqmTjVjDdZLGPdCajhUkS8iQIl7ySaUxVI3g7GNzO//cS1EXH0gJOE+4oOIxEKRtFKj70Rxey+X5v2yxW36s5BVomXkwrkaPTLX71BzFLFI2SSGtP13AT9jGoUTPJpqZcanlA2pkPetTSiihs/mx88JWdWGZAw1rYiJHP190RGlTETFdhORXFklr2Z+J/XTTG88jMRJSnyiC0WhakkGJPZ92QgNGcoJ5ZQpoW9lbAR1ZShzahkQ/CWX14lrYuq51a9u1qlfp3HUYQTOIVz8OAS6nALDWgCAwXP8ApvjnZenHfnY9FacPKZY/gD5/MHpoCQSw==</latexit><latexit sha1_base64="hWfIhyLO9Eu4xlCF9ryscNsrcWM=">AAAB8HicbVBNS8NAEJ3Ur1q/qh69LBbBU0mkoMeiF48V7Ye0oWy2m3bpbhJ2J0IJ/RVePCji1Z/jzX/jts1BWx8MPN6bYWZekEhh0HW/ncLa+sbmVnG7tLO7t39QPjxqmTjVjDdZLGPdCajhUkS8iQIl7ySaUxVI3g7GNzO//cS1EXH0gJOE+4oOIxEKRtFKj70Rxey+X5v2yxW36s5BVomXkwrkaPTLX71BzFLFI2SSGtP13AT9jGoUTPJpqZcanlA2pkPetTSiihs/mx88JWdWGZAw1rYiJHP190RGlTETFdhORXFklr2Z+J/XTTG88jMRJSnyiC0WhakkGJPZ92QgNGcoJ5ZQpoW9lbAR1ZShzahkQ/CWX14lrYuq51a9u1qlfp3HUYQTOIVz8OAS6nALDWgCAwXP8ApvjnZenHfnY9FacPKZY/gD5/MHpoCQSw==</latexit><latexit sha1_base64="hWfIhyLO9Eu4xlCF9ryscNsrcWM=">AAAB8HicbVBNS8NAEJ3Ur1q/qh69LBbBU0mkoMeiF48V7Ye0oWy2m3bpbhJ2J0IJ/RVePCji1Z/jzX/jts1BWx8MPN6bYWZekEhh0HW/ncLa+sbmVnG7tLO7t39QPjxqmTjVjDdZLGPdCajhUkS8iQIl7ySaUxVI3g7GNzO//cS1EXH0gJOE+4oOIxEKRtFKj70Rxey+X5v2yxW36s5BVomXkwrkaPTLX71BzFLFI2SSGtP13AT9jGoUTPJpqZcanlA2pkPetTSiihs/mx88JWdWGZAw1rYiJHP190RGlTETFdhORXFklr2Z+J/XTTG88jMRJSnyiC0WhakkGJPZ92QgNGcoJ5ZQpoW9lbAR1ZShzahkQ/CWX14lrYuq51a9u1qlfp3HUYQTOIVz8OAS6nALDWgCAwXP8ApvjnZenHfnY9FacPKZY/gD5/MHpoCQSw==</latexit><latexit sha1_base64="hWfIhyLO9Eu4xlCF9ryscNsrcWM=">AAAB8HicbVBNS8NAEJ3Ur1q/qh69LBbBU0mkoMeiF48V7Ye0oWy2m3bpbhJ2J0IJ/RVePCji1Z/jzX/jts1BWx8MPN6bYWZekEhh0HW/ncLa+sbmVnG7tLO7t39QPjxqmTjVjDdZLGPdCajhUkS8iQIl7ySaUxVI3g7GNzO//cS1EXH0gJOE+4oOIxEKRtFKj70Rxey+X5v2yxW36s5BVomXkwrkaPTLX71BzFLFI2SSGtP13AT9jGoUTPJpqZcanlA2pkPetTSiihs/mx88JWdWGZAw1rYiJHP190RGlTETFdhORXFklr2Z+J/XTTG88jMRJSnyiC0WhakkGJPZ92QgNGcoJ5ZQpoW9lbAR1ZShzahkQ/CWX14lrYuq51a9u1qlfp3HUYQTOIVz8OAS6nALDWgCAwXP8ApvjnZenHfnY9FacPKZY/gD5/MHpoCQSw==</latexit>

S5<latexit sha1_base64="hldavJDFT6zTc0kRuU+sSkWJWGs=">AAAB8HicbVBNS8NAEJ3Ur1q/qh69BIvgqSSi6LHoxWNF+yFtKJvtpl26uwm7E6GE/govHhTx6s/x5r9x2+agrQ8GHu/NMDMvTAQ36HnfTmFldW19o7hZ2tre2d0r7x80TZxqyho0FrFuh8QwwRVrIEfB2olmRIaCtcLRzdRvPTFteKwecJywQJKB4hGnBK302B0SzO57F5NeueJVvRncZeLnpAI56r3yV7cf01QyhVQQYzq+l2CQEY2cCjYpdVPDEkJHZMA6lioimQmy2cET98QqfTeKtS2F7kz9PZERacxYhrZTEhyaRW8q/ud1UoyugoyrJEWm6HxRlAoXY3f6vdvnmlEUY0sI1dze6tIh0YSizahkQ/AXX14mzbOq71X9u/NK7TqPowhHcAyn4MMl1OAW6tAAChKe4RXeHO28OO/Ox7y14OQzh/AHzucPqAWQTA==</latexit><latexit sha1_base64="hldavJDFT6zTc0kRuU+sSkWJWGs=">AAAB8HicbVBNS8NAEJ3Ur1q/qh69BIvgqSSi6LHoxWNF+yFtKJvtpl26uwm7E6GE/govHhTx6s/x5r9x2+agrQ8GHu/NMDMvTAQ36HnfTmFldW19o7hZ2tre2d0r7x80TZxqyho0FrFuh8QwwRVrIEfB2olmRIaCtcLRzdRvPTFteKwecJywQJKB4hGnBK302B0SzO57F5NeueJVvRncZeLnpAI56r3yV7cf01QyhVQQYzq+l2CQEY2cCjYpdVPDEkJHZMA6lioimQmy2cET98QqfTeKtS2F7kz9PZERacxYhrZTEhyaRW8q/ud1UoyugoyrJEWm6HxRlAoXY3f6vdvnmlEUY0sI1dze6tIh0YSizahkQ/AXX14mzbOq71X9u/NK7TqPowhHcAyn4MMl1OAW6tAAChKe4RXeHO28OO/Ox7y14OQzh/AHzucPqAWQTA==</latexit><latexit sha1_base64="hldavJDFT6zTc0kRuU+sSkWJWGs=">AAAB8HicbVBNS8NAEJ3Ur1q/qh69BIvgqSSi6LHoxWNF+yFtKJvtpl26uwm7E6GE/govHhTx6s/x5r9x2+agrQ8GHu/NMDMvTAQ36HnfTmFldW19o7hZ2tre2d0r7x80TZxqyho0FrFuh8QwwRVrIEfB2olmRIaCtcLRzdRvPTFteKwecJywQJKB4hGnBK302B0SzO57F5NeueJVvRncZeLnpAI56r3yV7cf01QyhVQQYzq+l2CQEY2cCjYpdVPDEkJHZMA6lioimQmy2cET98QqfTeKtS2F7kz9PZERacxYhrZTEhyaRW8q/ud1UoyugoyrJEWm6HxRlAoXY3f6vdvnmlEUY0sI1dze6tIh0YSizahkQ/AXX14mzbOq71X9u/NK7TqPowhHcAyn4MMl1OAW6tAAChKe4RXeHO28OO/Ox7y14OQzh/AHzucPqAWQTA==</latexit><latexit sha1_base64="hP+6LrUf2d3tZaldqaQQvEKMXyw=">AAAB2XicbZDNSgMxFIXv1L86Vq1rN8EiuCozbnQpuHFZwbZCO5RM5k4bmskMyR2hDH0BF25EfC93vo3pz0JbDwQ+zknIvSculLQUBN9ebWd3b/+gfugfNfzjk9Nmo2fz0gjsilzl5jnmFpXU2CVJCp8LgzyLFfbj6f0i77+gsTLXTzQrMMr4WMtUCk7O6oyaraAdLMW2IVxDC9YaNb+GSS7KDDUJxa0dhEFBUcUNSaFw7g9LiwUXUz7GgUPNM7RRtRxzzi6dk7A0N+5oYkv394uKZ9bOstjdzDhN7Ga2MP/LBiWlt1EldVESarH6KC0Vo5wtdmaJNChIzRxwYaSblYkJN1yQa8Z3HYSbG29D77odBu3wMYA6nMMFXEEIN3AHD9CBLghI4BXevYn35n2suqp569LO4I+8zx84xIo4</latexit><latexit sha1_base64="Q6Uw3Rc0bhCIF0QVu23ma+Wse+Q=">AAAB5XicbZDNSgMxFIXv1L9aq1a3boJFcFVmBNGl4MZlRaettEPJpJk2NMkMyR2hDH0KNy4U8ZXc+TamPwttPRD4OCch9544k8Ki7397pY3Nre2d8m5lr7p/cFg7qrZsmhvGQ5bK1HRiarkUmocoUPJOZjhVseTteHw7y9vP3FiR6kecZDxSdKhFIhhFZz31RhSLh/7ltF+r+w1/LrIOwRLqsFSzX/vqDVKWK66RSWptN/AzjApqUDDJp5VebnlG2ZgOedehporbqJgPPCVnzhmQJDXuaCRz9/eLgiprJyp2NxXFkV3NZuZ/WTfH5DoqhM5y5JotPkpySTAls+3JQBjOUE4cUGaEm5WwETWUoeuo4koIVldeh9ZFI/Abwb0PZTiBUziHAK7gBu6gCSEwUPACb/DuGe/V+1jUVfKWvR3DH3mfP3BfjvE=</latexit><latexit sha1_base64="Q6Uw3Rc0bhCIF0QVu23ma+Wse+Q=">AAAB5XicbZDNSgMxFIXv1L9aq1a3boJFcFVmBNGl4MZlRaettEPJpJk2NMkMyR2hDH0KNy4U8ZXc+TamPwttPRD4OCch9544k8Ki7397pY3Nre2d8m5lr7p/cFg7qrZsmhvGQ5bK1HRiarkUmocoUPJOZjhVseTteHw7y9vP3FiR6kecZDxSdKhFIhhFZz31RhSLh/7ltF+r+w1/LrIOwRLqsFSzX/vqDVKWK66RSWptN/AzjApqUDDJp5VebnlG2ZgOedehporbqJgPPCVnzhmQJDXuaCRz9/eLgiprJyp2NxXFkV3NZuZ/WTfH5DoqhM5y5JotPkpySTAls+3JQBjOUE4cUGaEm5WwETWUoeuo4koIVldeh9ZFI/Abwb0PZTiBUziHAK7gBu6gCSEwUPACb/DuGe/V+1jUVfKWvR3DH3mfP3BfjvE=</latexit><latexit sha1_base64="QkiIT3bHHOi9JCXmJ281Kf+c9JQ=">AAAB8HicbVA9SwNBEJ2LXzF+RS1tFoNgFe4E0TJoYxnRxEgSwt5mL1myu3fszgnhyK+wsVDE1p9j579xk1yhiQ8GHu/NMDMvTKSw6PvfXmFldW19o7hZ2tre2d0r7x80bZwaxhsslrFphdRyKTRvoEDJW4nhVIWSP4Sj66n/8MSNFbG+x3HCu4oOtIgEo+ikx86QYnbXO5/0yhW/6s9AlkmQkwrkqPfKX51+zFLFNTJJrW0HfoLdjBoUTPJJqZNanlA2ogPedlRTxW03mx08ISdO6ZMoNq40kpn6eyKjytqxCl2noji0i95U/M9rpxhddjOhkxS5ZvNFUSoJxmT6PekLwxnKsSOUGeFuJWxIDWXoMiq5EILFl5dJ86wa+NXg1q/UrvI4inAEx3AKAVxADW6gDg1goOAZXuHNM96L9+59zFsLXj5zCH/gff4ApsWQSA==</latexit><latexit sha1_base64="hldavJDFT6zTc0kRuU+sSkWJWGs=">AAAB8HicbVBNS8NAEJ3Ur1q/qh69BIvgqSSi6LHoxWNF+yFtKJvtpl26uwm7E6GE/govHhTx6s/x5r9x2+agrQ8GHu/NMDMvTAQ36HnfTmFldW19o7hZ2tre2d0r7x80TZxqyho0FrFuh8QwwRVrIEfB2olmRIaCtcLRzdRvPTFteKwecJywQJKB4hGnBK302B0SzO57F5NeueJVvRncZeLnpAI56r3yV7cf01QyhVQQYzq+l2CQEY2cCjYpdVPDEkJHZMA6lioimQmy2cET98QqfTeKtS2F7kz9PZERacxYhrZTEhyaRW8q/ud1UoyugoyrJEWm6HxRlAoXY3f6vdvnmlEUY0sI1dze6tIh0YSizahkQ/AXX14mzbOq71X9u/NK7TqPowhHcAyn4MMl1OAW6tAAChKe4RXeHO28OO/Ox7y14OQzh/AHzucPqAWQTA==</latexit><latexit sha1_base64="hldavJDFT6zTc0kRuU+sSkWJWGs=">AAAB8HicbVBNS8NAEJ3Ur1q/qh69BIvgqSSi6LHoxWNF+yFtKJvtpl26uwm7E6GE/govHhTx6s/x5r9x2+agrQ8GHu/NMDMvTAQ36HnfTmFldW19o7hZ2tre2d0r7x80TZxqyho0FrFuh8QwwRVrIEfB2olmRIaCtcLRzdRvPTFteKwecJywQJKB4hGnBK302B0SzO57F5NeueJVvRncZeLnpAI56r3yV7cf01QyhVQQYzq+l2CQEY2cCjYpdVPDEkJHZMA6lioimQmy2cET98QqfTeKtS2F7kz9PZERacxYhrZTEhyaRW8q/ud1UoyugoyrJEWm6HxRlAoXY3f6vdvnmlEUY0sI1dze6tIh0YSizahkQ/AXX14mzbOq71X9u/NK7TqPowhHcAyn4MMl1OAW6tAAChKe4RXeHO28OO/Ox7y14OQzh/AHzucPqAWQTA==</latexit><latexit sha1_base64="hldavJDFT6zTc0kRuU+sSkWJWGs=">AAAB8HicbVBNS8NAEJ3Ur1q/qh69BIvgqSSi6LHoxWNF+yFtKJvtpl26uwm7E6GE/govHhTx6s/x5r9x2+agrQ8GHu/NMDMvTAQ36HnfTmFldW19o7hZ2tre2d0r7x80TZxqyho0FrFuh8QwwRVrIEfB2olmRIaCtcLRzdRvPTFteKwecJywQJKB4hGnBK302B0SzO57F5NeueJVvRncZeLnpAI56r3yV7cf01QyhVQQYzq+l2CQEY2cCjYpdVPDEkJHZMA6lioimQmy2cET98QqfTeKtS2F7kz9PZERacxYhrZTEhyaRW8q/ud1UoyugoyrJEWm6HxRlAoXY3f6vdvnmlEUY0sI1dze6tIh0YSizahkQ/AXX14mzbOq71X9u/NK7TqPowhHcAyn4MMl1OAW6tAAChKe4RXeHO28OO/Ox7y14OQzh/AHzucPqAWQTA==</latexit><latexit sha1_base64="hldavJDFT6zTc0kRuU+sSkWJWGs=">AAAB8HicbVBNS8NAEJ3Ur1q/qh69BIvgqSSi6LHoxWNF+yFtKJvtpl26uwm7E6GE/govHhTx6s/x5r9x2+agrQ8GHu/NMDMvTAQ36HnfTmFldW19o7hZ2tre2d0r7x80TZxqyho0FrFuh8QwwRVrIEfB2olmRIaCtcLRzdRvPTFteKwecJywQJKB4hGnBK302B0SzO57F5NeueJVvRncZeLnpAI56r3yV7cf01QyhVQQYzq+l2CQEY2cCjYpdVPDEkJHZMA6lioimQmy2cET98QqfTeKtS2F7kz9PZERacxYhrZTEhyaRW8q/ud1UoyugoyrJEWm6HxRlAoXY3f6vdvnmlEUY0sI1dze6tIh0YSizahkQ/AXX14mzbOq71X9u/NK7TqPowhHcAyn4MMl1OAW6tAAChKe4RXeHO28OO/Ox7y14OQzh/AHzucPqAWQTA==</latexit><latexit sha1_base64="hldavJDFT6zTc0kRuU+sSkWJWGs=">AAAB8HicbVBNS8NAEJ3Ur1q/qh69BIvgqSSi6LHoxWNF+yFtKJvtpl26uwm7E6GE/govHhTx6s/x5r9x2+agrQ8GHu/NMDMvTAQ36HnfTmFldW19o7hZ2tre2d0r7x80TZxqyho0FrFuh8QwwRVrIEfB2olmRIaCtcLRzdRvPTFteKwecJywQJKB4hGnBK302B0SzO57F5NeueJVvRncZeLnpAI56r3yV7cf01QyhVQQYzq+l2CQEY2cCjYpdVPDEkJHZMA6lioimQmy2cET98QqfTeKtS2F7kz9PZERacxYhrZTEhyaRW8q/ud1UoyugoyrJEWm6HxRlAoXY3f6vdvnmlEUY0sI1dze6tIh0YSizahkQ/AXX14mzbOq71X9u/NK7TqPowhHcAyn4MMl1OAW6tAAChKe4RXeHO28OO/Ox7y14OQzh/AHzucPqAWQTA==</latexit><latexit sha1_base64="hldavJDFT6zTc0kRuU+sSkWJWGs=">AAAB8HicbVBNS8NAEJ3Ur1q/qh69BIvgqSSi6LHoxWNF+yFtKJvtpl26uwm7E6GE/govHhTx6s/x5r9x2+agrQ8GHu/NMDMvTAQ36HnfTmFldW19o7hZ2tre2d0r7x80TZxqyho0FrFuh8QwwRVrIEfB2olmRIaCtcLRzdRvPTFteKwecJywQJKB4hGnBK302B0SzO57F5NeueJVvRncZeLnpAI56r3yV7cf01QyhVQQYzq+l2CQEY2cCjYpdVPDEkJHZMA6lioimQmy2cET98QqfTeKtS2F7kz9PZERacxYhrZTEhyaRW8q/ud1UoyugoyrJEWm6HxRlAoXY3f6vdvnmlEUY0sI1dze6tIh0YSizahkQ/AXX14mzbOq71X9u/NK7TqPowhHcAyn4MMl1OAW6tAAChKe4RXeHO28OO/Ox7y14OQzh/AHzucPqAWQTA==</latexit>

F (·)<latexit sha1_base64="KXhygCD6B/BoWAqz+uh0ByXP7Tk=">AAAB73icbVBNS8NAEJ34WetX1aOXYBHqpSQi6LEoiMcK9gPaUDabTbt0sxt3J0Ip/RNePCji1b/jzX/jts1BWx8MPN6bYWZemApu0PO+nZXVtfWNzcJWcXtnd2+/dHDYNCrTlDWoEkq3Q2KY4JI1kKNg7VQzkoSCtcLhzdRvPTFtuJIPOEpZkJC+5DGnBK3Uvq10aaTwrFcqe1VvBneZ+DkpQ456r/TVjRTNEiaRCmJMx/dSDMZEI6eCTYrdzLCU0CHps46lkiTMBOPZvRP31CqRGyttS6I7U39PjElizCgJbWdCcGAWvan4n9fJML4KxlymGTJJ54viTLio3OnzbsQ1oyhGlhCqub3VpQOiCUUbUdGG4C++vEya51Xfq/r3F+XadR5HAY7hBCrgwyXU4A7q0AAKAp7hFd6cR+fFeXc+5q0rTj5zBH/gfP4AMa6PZw==</latexit><latexit sha1_base64="KXhygCD6B/BoWAqz+uh0ByXP7Tk=">AAAB73icbVBNS8NAEJ34WetX1aOXYBHqpSQi6LEoiMcK9gPaUDabTbt0sxt3J0Ip/RNePCji1b/jzX/jts1BWx8MPN6bYWZemApu0PO+nZXVtfWNzcJWcXtnd2+/dHDYNCrTlDWoEkq3Q2KY4JI1kKNg7VQzkoSCtcLhzdRvPTFtuJIPOEpZkJC+5DGnBK3Uvq10aaTwrFcqe1VvBneZ+DkpQ456r/TVjRTNEiaRCmJMx/dSDMZEI6eCTYrdzLCU0CHps46lkiTMBOPZvRP31CqRGyttS6I7U39PjElizCgJbWdCcGAWvan4n9fJML4KxlymGTJJ54viTLio3OnzbsQ1oyhGlhCqub3VpQOiCUUbUdGG4C++vEya51Xfq/r3F+XadR5HAY7hBCrgwyXU4A7q0AAKAp7hFd6cR+fFeXc+5q0rTj5zBH/gfP4AMa6PZw==</latexit><latexit sha1_base64="KXhygCD6B/BoWAqz+uh0ByXP7Tk=">AAAB73icbVBNS8NAEJ34WetX1aOXYBHqpSQi6LEoiMcK9gPaUDabTbt0sxt3J0Ip/RNePCji1b/jzX/jts1BWx8MPN6bYWZemApu0PO+nZXVtfWNzcJWcXtnd2+/dHDYNCrTlDWoEkq3Q2KY4JI1kKNg7VQzkoSCtcLhzdRvPTFtuJIPOEpZkJC+5DGnBK3Uvq10aaTwrFcqe1VvBneZ+DkpQ456r/TVjRTNEiaRCmJMx/dSDMZEI6eCTYrdzLCU0CHps46lkiTMBOPZvRP31CqRGyttS6I7U39PjElizCgJbWdCcGAWvan4n9fJML4KxlymGTJJ54viTLio3OnzbsQ1oyhGlhCqub3VpQOiCUUbUdGG4C++vEya51Xfq/r3F+XadR5HAY7hBCrgwyXU4A7q0AAKAp7hFd6cR+fFeXc+5q0rTj5zBH/gfP4AMa6PZw==</latexit><latexit sha1_base64="KXhygCD6B/BoWAqz+uh0ByXP7Tk=">AAAB73icbVBNS8NAEJ34WetX1aOXYBHqpSQi6LEoiMcK9gPaUDabTbt0sxt3J0Ip/RNePCji1b/jzX/jts1BWx8MPN6bYWZemApu0PO+nZXVtfWNzcJWcXtnd2+/dHDYNCrTlDWoEkq3Q2KY4JI1kKNg7VQzkoSCtcLhzdRvPTFtuJIPOEpZkJC+5DGnBK3Uvq10aaTwrFcqe1VvBneZ+DkpQ456r/TVjRTNEiaRCmJMx/dSDMZEI6eCTYrdzLCU0CHps46lkiTMBOPZvRP31CqRGyttS6I7U39PjElizCgJbWdCcGAWvan4n9fJML4KxlymGTJJ54viTLio3OnzbsQ1oyhGlhCqub3VpQOiCUUbUdGG4C++vEya51Xfq/r3F+XadR5HAY7hBCrgwyXU4A7q0AAKAp7hFd6cR+fFeXc+5q0rTj5zBH/gfP4AMa6PZw==</latexit>

F (·)<latexit sha1_base64="KXhygCD6B/BoWAqz+uh0ByXP7Tk=">AAAB73icbVBNS8NAEJ34WetX1aOXYBHqpSQi6LEoiMcK9gPaUDabTbt0sxt3J0Ip/RNePCji1b/jzX/jts1BWx8MPN6bYWZemApu0PO+nZXVtfWNzcJWcXtnd2+/dHDYNCrTlDWoEkq3Q2KY4JI1kKNg7VQzkoSCtcLhzdRvPTFtuJIPOEpZkJC+5DGnBK3Uvq10aaTwrFcqe1VvBneZ+DkpQ456r/TVjRTNEiaRCmJMx/dSDMZEI6eCTYrdzLCU0CHps46lkiTMBOPZvRP31CqRGyttS6I7U39PjElizCgJbWdCcGAWvan4n9fJML4KxlymGTJJ54viTLio3OnzbsQ1oyhGlhCqub3VpQOiCUUbUdGG4C++vEya51Xfq/r3F+XadR5HAY7hBCrgwyXU4A7q0AAKAp7hFd6cR+fFeXc+5q0rTj5zBH/gfP4AMa6PZw==</latexit><latexit sha1_base64="KXhygCD6B/BoWAqz+uh0ByXP7Tk=">AAAB73icbVBNS8NAEJ34WetX1aOXYBHqpSQi6LEoiMcK9gPaUDabTbt0sxt3J0Ip/RNePCji1b/jzX/jts1BWx8MPN6bYWZemApu0PO+nZXVtfWNzcJWcXtnd2+/dHDYNCrTlDWoEkq3Q2KY4JI1kKNg7VQzkoSCtcLhzdRvPTFtuJIPOEpZkJC+5DGnBK3Uvq10aaTwrFcqe1VvBneZ+DkpQ456r/TVjRTNEiaRCmJMx/dSDMZEI6eCTYrdzLCU0CHps46lkiTMBOPZvRP31CqRGyttS6I7U39PjElizCgJbWdCcGAWvan4n9fJML4KxlymGTJJ54viTLio3OnzbsQ1oyhGlhCqub3VpQOiCUUbUdGG4C++vEya51Xfq/r3F+XadR5HAY7hBCrgwyXU4A7q0AAKAp7hFd6cR+fFeXc+5q0rTj5zBH/gfP4AMa6PZw==</latexit><latexit sha1_base64="KXhygCD6B/BoWAqz+uh0ByXP7Tk=">AAAB73icbVBNS8NAEJ34WetX1aOXYBHqpSQi6LEoiMcK9gPaUDabTbt0sxt3J0Ip/RNePCji1b/jzX/jts1BWx8MPN6bYWZemApu0PO+nZXVtfWNzcJWcXtnd2+/dHDYNCrTlDWoEkq3Q2KY4JI1kKNg7VQzkoSCtcLhzdRvPTFtuJIPOEpZkJC+5DGnBK3Uvq10aaTwrFcqe1VvBneZ+DkpQ456r/TVjRTNEiaRCmJMx/dSDMZEI6eCTYrdzLCU0CHps46lkiTMBOPZvRP31CqRGyttS6I7U39PjElizCgJbWdCcGAWvan4n9fJML4KxlymGTJJ54viTLio3OnzbsQ1oyhGlhCqub3VpQOiCUUbUdGG4C++vEya51Xfq/r3F+XadR5HAY7hBCrgwyXU4A7q0AAKAp7hFd6cR+fFeXc+5q0rTj5zBH/gfP4AMa6PZw==</latexit><latexit sha1_base64="KXhygCD6B/BoWAqz+uh0ByXP7Tk=">AAAB73icbVBNS8NAEJ34WetX1aOXYBHqpSQi6LEoiMcK9gPaUDabTbt0sxt3J0Ip/RNePCji1b/jzX/jts1BWx8MPN6bYWZemApu0PO+nZXVtfWNzcJWcXtnd2+/dHDYNCrTlDWoEkq3Q2KY4JI1kKNg7VQzkoSCtcLhzdRvPTFtuJIPOEpZkJC+5DGnBK3Uvq10aaTwrFcqe1VvBneZ+DkpQ456r/TVjRTNEiaRCmJMx/dSDMZEI6eCTYrdzLCU0CHps46lkiTMBOPZvRP31CqRGyttS6I7U39PjElizCgJbWdCcGAWvan4n9fJML4KxlymGTJJ54viTLio3OnzbsQ1oyhGlhCqub3VpQOiCUUbUdGG4C++vEya51Xfq/r3F+XadR5HAY7hBCrgwyXU4A7q0AAKAp7hFd6cR+fFeXc+5q0rTj5zBH/gfP4AMa6PZw==</latexit>

F (·)<latexit sha1_base64="KXhygCD6B/BoWAqz+uh0ByXP7Tk=">AAAB73icbVBNS8NAEJ34WetX1aOXYBHqpSQi6LEoiMcK9gPaUDabTbt0sxt3J0Ip/RNePCji1b/jzX/jts1BWx8MPN6bYWZemApu0PO+nZXVtfWNzcJWcXtnd2+/dHDYNCrTlDWoEkq3Q2KY4JI1kKNg7VQzkoSCtcLhzdRvPTFtuJIPOEpZkJC+5DGnBK3Uvq10aaTwrFcqe1VvBneZ+DkpQ456r/TVjRTNEiaRCmJMx/dSDMZEI6eCTYrdzLCU0CHps46lkiTMBOPZvRP31CqRGyttS6I7U39PjElizCgJbWdCcGAWvan4n9fJML4KxlymGTJJ54viTLio3OnzbsQ1oyhGlhCqub3VpQOiCUUbUdGG4C++vEya51Xfq/r3F+XadR5HAY7hBCrgwyXU4A7q0AAKAp7hFd6cR+fFeXc+5q0rTj5zBH/gfP4AMa6PZw==</latexit><latexit sha1_base64="KXhygCD6B/BoWAqz+uh0ByXP7Tk=">AAAB73icbVBNS8NAEJ34WetX1aOXYBHqpSQi6LEoiMcK9gPaUDabTbt0sxt3J0Ip/RNePCji1b/jzX/jts1BWx8MPN6bYWZemApu0PO+nZXVtfWNzcJWcXtnd2+/dHDYNCrTlDWoEkq3Q2KY4JI1kKNg7VQzkoSCtcLhzdRvPTFtuJIPOEpZkJC+5DGnBK3Uvq10aaTwrFcqe1VvBneZ+DkpQ456r/TVjRTNEiaRCmJMx/dSDMZEI6eCTYrdzLCU0CHps46lkiTMBOPZvRP31CqRGyttS6I7U39PjElizCgJbWdCcGAWvan4n9fJML4KxlymGTJJ54viTLio3OnzbsQ1oyhGlhCqub3VpQOiCUUbUdGG4C++vEya51Xfq/r3F+XadR5HAY7hBCrgwyXU4A7q0AAKAp7hFd6cR+fFeXc+5q0rTj5zBH/gfP4AMa6PZw==</latexit><latexit sha1_base64="KXhygCD6B/BoWAqz+uh0ByXP7Tk=">AAAB73icbVBNS8NAEJ34WetX1aOXYBHqpSQi6LEoiMcK9gPaUDabTbt0sxt3J0Ip/RNePCji1b/jzX/jts1BWx8MPN6bYWZemApu0PO+nZXVtfWNzcJWcXtnd2+/dHDYNCrTlDWoEkq3Q2KY4JI1kKNg7VQzkoSCtcLhzdRvPTFtuJIPOEpZkJC+5DGnBK3Uvq10aaTwrFcqe1VvBneZ+DkpQ456r/TVjRTNEiaRCmJMx/dSDMZEI6eCTYrdzLCU0CHps46lkiTMBOPZvRP31CqRGyttS6I7U39PjElizCgJbWdCcGAWvan4n9fJML4KxlymGTJJ54viTLio3OnzbsQ1oyhGlhCqub3VpQOiCUUbUdGG4C++vEya51Xfq/r3F+XadR5HAY7hBCrgwyXU4A7q0AAKAp7hFd6cR+fFeXc+5q0rTj5zBH/gfP4AMa6PZw==</latexit><latexit sha1_base64="KXhygCD6B/BoWAqz+uh0ByXP7Tk=">AAAB73icbVBNS8NAEJ34WetX1aOXYBHqpSQi6LEoiMcK9gPaUDabTbt0sxt3J0Ip/RNePCji1b/jzX/jts1BWx8MPN6bYWZemApu0PO+nZXVtfWNzcJWcXtnd2+/dHDYNCrTlDWoEkq3Q2KY4JI1kKNg7VQzkoSCtcLhzdRvPTFtuJIPOEpZkJC+5DGnBK3Uvq10aaTwrFcqe1VvBneZ+DkpQ456r/TVjRTNEiaRCmJMx/dSDMZEI6eCTYrdzLCU0CHps46lkiTMBOPZvRP31CqRGyttS6I7U39PjElizCgJbWdCcGAWvan4n9fJML4KxlymGTJJ54viTLio3OnzbsQ1oyhGlhCqub3VpQOiCUUbUdGG4C++vEya51Xfq/r3F+XadR5HAY7hBCrgwyXU4A7q0AAKAp7hFd6cR+fFeXc+5q0rTj5zBH/gfP4AMa6PZw==</latexit>

F (·)<latexit sha1_base64="KXhygCD6B/BoWAqz+uh0ByXP7Tk=">AAAB73icbVBNS8NAEJ34WetX1aOXYBHqpSQi6LEoiMcK9gPaUDabTbt0sxt3J0Ip/RNePCji1b/jzX/jts1BWx8MPN6bYWZemApu0PO+nZXVtfWNzcJWcXtnd2+/dHDYNCrTlDWoEkq3Q2KY4JI1kKNg7VQzkoSCtcLhzdRvPTFtuJIPOEpZkJC+5DGnBK3Uvq10aaTwrFcqe1VvBneZ+DkpQ456r/TVjRTNEiaRCmJMx/dSDMZEI6eCTYrdzLCU0CHps46lkiTMBOPZvRP31CqRGyttS6I7U39PjElizCgJbWdCcGAWvan4n9fJML4KxlymGTJJ54viTLio3OnzbsQ1oyhGlhCqub3VpQOiCUUbUdGG4C++vEya51Xfq/r3F+XadR5HAY7hBCrgwyXU4A7q0AAKAp7hFd6cR+fFeXc+5q0rTj5zBH/gfP4AMa6PZw==</latexit><latexit sha1_base64="KXhygCD6B/BoWAqz+uh0ByXP7Tk=">AAAB73icbVBNS8NAEJ34WetX1aOXYBHqpSQi6LEoiMcK9gPaUDabTbt0sxt3J0Ip/RNePCji1b/jzX/jts1BWx8MPN6bYWZemApu0PO+nZXVtfWNzcJWcXtnd2+/dHDYNCrTlDWoEkq3Q2KY4JI1kKNg7VQzkoSCtcLhzdRvPTFtuJIPOEpZkJC+5DGnBK3Uvq10aaTwrFcqe1VvBneZ+DkpQ456r/TVjRTNEiaRCmJMx/dSDMZEI6eCTYrdzLCU0CHps46lkiTMBOPZvRP31CqRGyttS6I7U39PjElizCgJbWdCcGAWvan4n9fJML4KxlymGTJJ54viTLio3OnzbsQ1oyhGlhCqub3VpQOiCUUbUdGG4C++vEya51Xfq/r3F+XadR5HAY7hBCrgwyXU4A7q0AAKAp7hFd6cR+fFeXc+5q0rTj5zBH/gfP4AMa6PZw==</latexit><latexit sha1_base64="KXhygCD6B/BoWAqz+uh0ByXP7Tk=">AAAB73icbVBNS8NAEJ34WetX1aOXYBHqpSQi6LEoiMcK9gPaUDabTbt0sxt3J0Ip/RNePCji1b/jzX/jts1BWx8MPN6bYWZemApu0PO+nZXVtfWNzcJWcXtnd2+/dHDYNCrTlDWoEkq3Q2KY4JI1kKNg7VQzkoSCtcLhzdRvPTFtuJIPOEpZkJC+5DGnBK3Uvq10aaTwrFcqe1VvBneZ+DkpQ456r/TVjRTNEiaRCmJMx/dSDMZEI6eCTYrdzLCU0CHps46lkiTMBOPZvRP31CqRGyttS6I7U39PjElizCgJbWdCcGAWvan4n9fJML4KxlymGTJJ54viTLio3OnzbsQ1oyhGlhCqub3VpQOiCUUbUdGG4C++vEya51Xfq/r3F+XadR5HAY7hBCrgwyXU4A7q0AAKAp7hFd6cR+fFeXc+5q0rTj5zBH/gfP4AMa6PZw==</latexit><latexit sha1_base64="KXhygCD6B/BoWAqz+uh0ByXP7Tk=">AAAB73icbVBNS8NAEJ34WetX1aOXYBHqpSQi6LEoiMcK9gPaUDabTbt0sxt3J0Ip/RNePCji1b/jzX/jts1BWx8MPN6bYWZemApu0PO+nZXVtfWNzcJWcXtnd2+/dHDYNCrTlDWoEkq3Q2KY4JI1kKNg7VQzkoSCtcLhzdRvPTFtuJIPOEpZkJC+5DGnBK3Uvq10aaTwrFcqe1VvBneZ+DkpQ456r/TVjRTNEiaRCmJMx/dSDMZEI6eCTYrdzLCU0CHps46lkiTMBOPZvRP31CqRGyttS6I7U39PjElizCgJbWdCcGAWvan4n9fJML4KxlymGTJJ54viTLio3OnzbsQ1oyhGlhCqub3VpQOiCUUbUdGG4C++vEya51Xfq/r3F+XadR5HAY7hBCrgwyXU4A7q0AAKAp7hFd6cR+fFeXc+5q0rTj5zBH/gfP4AMa6PZw==</latexit>

F (·)<latexit sha1_base64="KXhygCD6B/BoWAqz+uh0ByXP7Tk=">AAAB73icbVBNS8NAEJ34WetX1aOXYBHqpSQi6LEoiMcK9gPaUDabTbt0sxt3J0Ip/RNePCji1b/jzX/jts1BWx8MPN6bYWZemApu0PO+nZXVtfWNzcJWcXtnd2+/dHDYNCrTlDWoEkq3Q2KY4JI1kKNg7VQzkoSCtcLhzdRvPTFtuJIPOEpZkJC+5DGnBK3Uvq10aaTwrFcqe1VvBneZ+DkpQ456r/TVjRTNEiaRCmJMx/dSDMZEI6eCTYrdzLCU0CHps46lkiTMBOPZvRP31CqRGyttS6I7U39PjElizCgJbWdCcGAWvan4n9fJML4KxlymGTJJ54viTLio3OnzbsQ1oyhGlhCqub3VpQOiCUUbUdGG4C++vEya51Xfq/r3F+XadR5HAY7hBCrgwyXU4A7q0AAKAp7hFd6cR+fFeXc+5q0rTj5zBH/gfP4AMa6PZw==</latexit><latexit sha1_base64="KXhygCD6B/BoWAqz+uh0ByXP7Tk=">AAAB73icbVBNS8NAEJ34WetX1aOXYBHqpSQi6LEoiMcK9gPaUDabTbt0sxt3J0Ip/RNePCji1b/jzX/jts1BWx8MPN6bYWZemApu0PO+nZXVtfWNzcJWcXtnd2+/dHDYNCrTlDWoEkq3Q2KY4JI1kKNg7VQzkoSCtcLhzdRvPTFtuJIPOEpZkJC+5DGnBK3Uvq10aaTwrFcqe1VvBneZ+DkpQ456r/TVjRTNEiaRCmJMx/dSDMZEI6eCTYrdzLCU0CHps46lkiTMBOPZvRP31CqRGyttS6I7U39PjElizCgJbWdCcGAWvan4n9fJML4KxlymGTJJ54viTLio3OnzbsQ1oyhGlhCqub3VpQOiCUUbUdGG4C++vEya51Xfq/r3F+XadR5HAY7hBCrgwyXU4A7q0AAKAp7hFd6cR+fFeXc+5q0rTj5zBH/gfP4AMa6PZw==</latexit><latexit sha1_base64="KXhygCD6B/BoWAqz+uh0ByXP7Tk=">AAAB73icbVBNS8NAEJ34WetX1aOXYBHqpSQi6LEoiMcK9gPaUDabTbt0sxt3J0Ip/RNePCji1b/jzX/jts1BWx8MPN6bYWZemApu0PO+nZXVtfWNzcJWcXtnd2+/dHDYNCrTlDWoEkq3Q2KY4JI1kKNg7VQzkoSCtcLhzdRvPTFtuJIPOEpZkJC+5DGnBK3Uvq10aaTwrFcqe1VvBneZ+DkpQ456r/TVjRTNEiaRCmJMx/dSDMZEI6eCTYrdzLCU0CHps46lkiTMBOPZvRP31CqRGyttS6I7U39PjElizCgJbWdCcGAWvan4n9fJML4KxlymGTJJ54viTLio3OnzbsQ1oyhGlhCqub3VpQOiCUUbUdGG4C++vEya51Xfq/r3F+XadR5HAY7hBCrgwyXU4A7q0AAKAp7hFd6cR+fFeXc+5q0rTj5zBH/gfP4AMa6PZw==</latexit><latexit sha1_base64="KXhygCD6B/BoWAqz+uh0ByXP7Tk=">AAAB73icbVBNS8NAEJ34WetX1aOXYBHqpSQi6LEoiMcK9gPaUDabTbt0sxt3J0Ip/RNePCji1b/jzX/jts1BWx8MPN6bYWZemApu0PO+nZXVtfWNzcJWcXtnd2+/dHDYNCrTlDWoEkq3Q2KY4JI1kKNg7VQzkoSCtcLhzdRvPTFtuJIPOEpZkJC+5DGnBK3Uvq10aaTwrFcqe1VvBneZ+DkpQ456r/TVjRTNEiaRCmJMx/dSDMZEI6eCTYrdzLCU0CHps46lkiTMBOPZvRP31CqRGyttS6I7U39PjElizCgJbWdCcGAWvan4n9fJML4KxlymGTJJ54viTLio3OnzbsQ1oyhGlhCqub3VpQOiCUUbUdGG4C++vEya51Xfq/r3F+XadR5HAY7hBCrgwyXU4A7q0AAKAp7hFd6cR+fFeXc+5q0rTj5zBH/gfP4AMa6PZw==</latexit>

Figure 1. Conceptual diagram of our approach for visual story-telling. The key difference from a typical SEQ2SEQ model is thecomponent of predicting anchor word embeddings from the im-ages. The predictions are then fused with the image features as theinputs for generating desired narrative sentences.

B@4 M R C

Image Only 13.9 ±0.4 35.2 ±0.1 29.5 ±0.3 8.4 ±0.9

Anchoringwith Noun 17.2±0.2 39.0±0.2 33.8±0.1 15.7±0.4

with Verb 16.5±0.1 37.9±0.2 34.7±0.2 13.0±0.2

with Adj. 15.2±0.3 36.6±0.1 31.9±0.2 11.3±0.2

with Adv. 14.9±0.3 35.9±0.1 31.0±0.2 10.4±0.3

Table 2. Adding ground-truth words as anchor words to aSEQ2SEQ model significantly improves its performance whereonly image features are used. The higher numerical value indi-cates better performance.

the factual enumeration of objects and activities depicted inthe images, which is often adequate for image captioning.

Recent approaches for visual storytelling have been us-ing reinforcement learning (RL) to optimize complicatedmodels [5, 12]. The approach proposed in this paper has theadvantage of a simplified design and learning procedure, yetattains the best performance on several evaluation metrics.

3. Approach

The task of visual storytelling is to generate a sequenceof narrative sentences {Si, i = 1, 2, . . . , N}, one for eachof the N images { Ii, i = 1, 2, . . . , N}. The order of theimages is important and is fixed. Each of the generated sen-tences Si could contain a variable length of words.

The main idea behind our approach is straightforward.For each image, we learn and apply a model to predict itsanchor word embedding. The predicted embedding is thenconcatenated with the image feature. The combined featureis fed into a SEQ2SEQ [9] where the narrative sentence isgenerated as output. Fig. 1 illustrates the model design.

The key challenge is to learn the anchor word predic-tion model when the dataset does not provide anchor wordsexplicitly. We begin by describing how we overcome thischallenge. Then we introduce our model in detail.

3.1. What is an anchor word?

We are inspired by the comparison between narrative sto-ries and captions on the same sequence of images, shown inTable 1. In particular, a large number of words used in nar-ration do not appear in captions. Intuitively speaking, theyare less likely visually grounded.

Thus, we conjecture that possible candidates for anchorwords are the words in the narrative sentences. The analysisin Table 2 confirms the usefulness of this hypothesis.

Specifically, we train a model as in Fig. 1 with two vari-ants. In the first variant, we supply only the image features.The results are reported in the row labeled as “Image Only”.In the second variant (“Anchoring”), we select all the noun(alternatively, verb, adjective, or adverb) words as anchorwords – one word per sentence in the story. We then trainthe SEQ2SEQ model by combining the image feature andthe word embedding end to end. The results are reportedin rows labeled with the part-of-speech (POS) tags of theselected words. For simplicity, all anchor words have thesame POS tags. If there are multiple words with the samePOS tags in each sentence, we randomly select one.

There are two points worth making. First, adding anchorwords, irrespective of their types, significantly improves theperformance of the SEQ2SEQ model with image featuresonly. Note that the results in “Image Only” is on par withstate-of-the-art results [12]. Secondly, among all POS tags,nouns as anchor words seem to be the most beneficial oneson all metrics except R(OUGE) where verbs improve more.

In the rest of this paper, we use nouns in the stories asthe anchor words.

3.2. Model and Learning

The data for our learning task is augmented with a list ofanchor words {wi | i = 1, 2, . . . , N} corresponding to theimages. Next, we explain how to learn each component.

Anchor word embedding predictor We learn a modelF (Ii) to predict wi. F (·) is parameterized by a one-hidden-layer multi-layer perception (MLP) with ReLU non-linearity. The input could be the features for the ith imageor all the images in the same sequence. In practice, there isno significant difference.

To be able to generalize to new anchor words, we predictits embedding and cast learning F (·) as a regression prob-lem. To obtain the target (ie, the “ground-truth” embedding)for the word wi, we take the embeddings from the “Anchor-ing” model in Table 2. F (·) is then optimized to reduce themean squared error between the predictions and the targetanchor words embeddings.

Story generation model Similar to state-of-the-art visualstorytelling methods [12, 6], we use a SEQ2SEQ model [9]as story generator. Concretely, a bidirectional gated recur-rent neural network [2](GRU) is used to encode the concate-

Method B@1 B@2 B@3 B@4 M R C

AREL [12] 63.8 39.1 23.2 14.1 35.0 29.5 9.4Show, Reward and Tell [11] 43.4 21.4 10.4 5.2 - - 11.4HSRL w/ Joint Training [5] - - - 12.3 35.2 30.8 10.7SEQ2SEQ+Heuristics [6] - - - - 31.4 - -H-Attn-Rank[14] - - 20.8 - 33.9 29.8 7.4StoryAnchor: Image Only 62.2 ±2.5 38.3 ±1.7 22.7 ±0.8 13.9 ±0.4 35.2 ±0.1 29.5 ±0.3 8.4 ±0.9

StoryAnchor: w/ Predicted Nouns 65.1 ±0.3 40.0 ±0.2 23.4 ±0 14.0 ±0.1 35.5 ±0.0 30.0 ±0.1 9.9 ±0.1

Table 3. Comparison of state-of-the-art method for the visual storytelling task on the VIST dataset. Our “Image Only” model is a reimple-mentation of XE+SS [12] with the authors’ public available codes.

nated feature of the image and the predicted anchor wordembedding and to produce a sequence of hidden states

vi = BiGRU(Ii, F (Ii)) (1)The sequence of the hidden states is then decoded by a one-layer GRU. Both the encoder and the decoder are trained tomaximize the likelihood of ground-truth stories.

3.3. Other Implementation Details

Visual and textual representation We extracted the2048 dimension feature from the penultimate layer ofResNet-152 [4] as visual representations. The 512-dimensional word embedding is randomly initialized,which are fine-tuned in the training. Note that the anchorwords are sharing the word embeddings with the words inthe vocabulary.

Model details The concatenated features of the image andthe anchor word embedding are projected into a 2048 di-mensional feature with a one-hidden-layer MLP. Then, aone-layer BiGRU with 256-dimensional hidden states gen-erates contextual embedding vi of 512 dimensions, to serveas hidden states representation. A standard SEQ2SEQ de-coder with one-layer GRU with 512 hidden dimensions isused on top of these hidden states to generate a story.

Optimization As mentioned, the model is trained in twostages. In the first stage, ground-truth anchor words (nounsin the stories) are used to train the encoder-decoder as wellas the embeddings end to end. The model is trained withmini-batches and ADAM for 100 epochs. Each mini-batchcontains 64 sampled stories. The learning rate is initializedas 4e-4 and schedule sampling [1] has been used. The prob-ability of schedule sampling is first set to be 0.05, increasedby 0.05 every 5 epochs till 25 epochs. In the second stage,the predictor F (·) is trained. Specifically, we use the modelthat achieves the highest Meteor score on the validation setin the first stage training as a pre-trained model. We usethe same optimization hyper-parameter to train the predic-tor with encoder-decoder model in an end-to-end way. Theencoder-decoder and the word embeddings are kept fixed.

Inference At the inference time (ie, narrating a sequenceof images), we perform beam search for sentence decodingwith a beam size of 3.

4. Experiments4.1. Experimental Setups

Dataset We use the VIST dataset [6] for evaluation. Itcontains 10,032 visual albums with 50,136 stories. Eachstory contains five narrative sentences, corresponding to fivegrounded images respectively.

Evaluation We follow the evaluation setup used in [6,12, 14, 5]. For each testing album, we sample one image se-quence and generate a story based on that image sequence.The story is then scored against all 5 reference stories of thatalbum. We use the evaluation code provided by the [14]1.We report results with average BLEU, METEOR, ROUGE,and CIDER over the test split. We evaluate over 3 randomruns and compute the means and variances of the metrics.

Identifying anchor words We use NLTK POS tagger toget the tags. Each sentence contains on average 2.63 nouns,2.0 verbs, 0.8 adjectives, and 0.5 adverbs. We use ’UNK’ asthe anchor word when there is no corresponding POS tag.

4.2. Main Results

We compare our method (StoryAnchor) to several state-of-the-art methods [6, 14, 12, 11, 5]. Figure 3 shows that ourmodel performs significantly better than others in almost allevaluation metrics. In ROUGE and CIDER, approaches ofusing reinforcement learning seem to perform well.

We also conduct human evaluations to compare the out-puts of our model and AREL [12]. We follow [12] and de-sign three questions to evaluate the relevance, concreteness,and coherence of generated stories and image sequences.150 generated stories from the test splits are evaluated. Foreach story, 5 AMT workers are assigned. The reports arereported in Table 5. Our approach performs better.

4.3. Analysis

Is visual storytelling fundamentally out of reach of ma-chines? Are the metrics being used now to guide the de-sign of our systems the right ones?

1https://github.com/lichengunc/vist_eval. This isthe most commonly used evaluation script nowadays.

B@1 B@2 B@3 B@4 M R C

Human 51.2 ±0.2 25.0 ±0.2 11.7 ±0.2 5.6 ±0.2 28.4 ±0.1 24.5 ±0.1 7.8 ±0.1

StoryAnchor: Image Only 58.6 ±0.2 34.7 ±0.2 20.0 ±0.1 11.2 ±0.1 34.0 ±0.1 28.3 ±0.1 8.8 ±0.1

StoryAnchor: w/ Predicted Nouns 60.7 ±0.2 35.8 ±0.2 20.3 ±0.1 11.9 ±0.1 34.5 ±0.0 28.9 ±0.0 10.1 ±0.1

StoryAnchor: w/ Ground-truth Nouns 65.1 ±0.2 40.3 ±0.1 23.9 ±0.1 14.7 ±0.0 37.7 ±0.1 32.3 ±0.1 16.2 ±0.1

Table 4. Evaluating human performance by automatic evaluation procedures. Machine outperforms human in all metrics.

StoryAnchor: AREL Tiew/ Predicted Nouns

Relevance 53.2% 40.4% 6.4%Concreteness 45.1% 38.1% 16.8%Coherence 48.9% 42.3% 8.8%

Table 5. Human evaluation on the generated stories

StoryAnchor: AREL Human Unsurew/ Predicted Nouns

19.9% 18.0% 57.8% 4.2%

Table 6. Which stories are preferred by human readers

The results in Table 4 highlight the issues. There, we as-sess how well human storyteller would do. For each album,we randomly select one human-written ground-truth storyas “generated” story and the other 4 as “reference” stories.We then evaluate human performance by scoring the gen-erated story. For a fair comparison, we re-evaluated all ofthe learning models with 4 sampled reference stories. Meanevaluation performances over five random runs are reported.

Clearly, the learning models outperform human story-teller significantly in every metric! Yet, our “Turing test”suggests the opposite. In Table 6, over 450 stories (3 foreach of the 150 sequences of images), we report the per-centages of 150 AMT workers’ preference of stories by twolearning models and one human annotator. Human story-telling is much more preferred. The misalignment betweenhuman evaluation and automatic evaluation metrics is likelya bottleneck for developing new methods for this task.

5. Conclusion

The proposed StoryAnchor model is simpler in design.Yet, it attains the best results on most automatic evalua-tion metrics. The key insight is to use “anchor words” tomodel the evolvement of the underlying storyline. Crudely,those words are the “topics” or “states” of the narrators.While those notions are not explicitly annotated in the cur-rent dataset, we have selected the nouns in the ground-truthstories as targets for learning an anchor word predictor.

References

[1] S. Bengio, O. Vinyals, N. Jaitly, and N. Shazeer. Scheduledsampling for sequence prediction with recurrent neural net-works. In Advances in Neural Information Processing Sys-tems, pages 1171–1179, 2015.

[2] J. Chung, C. Gulcehre, K. Cho, and Y. Bengio. Empiricalevaluation of gated recurrent neural networks on sequencemodeling. arXiv preprint arXiv:1412.3555, 2014.

[3] D. Gonzalez-Rico and G. Fuentes-Pineda. Contextualize,show and tell: a neural visual storyteller. arXiv preprintarXiv:1806.00738, 2018.

[4] K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learn-ing for image recognition. In Proceedings of the IEEE con-ference on computer vision and pattern recognition, pages770–778, 2016.

[5] Q. Huang, Z. Gan, A. Celikyilmaz, D. Wu, J. Wang, andX. He. Hierarchically structured reinforcement learning fortopically coherent visual story generation. arXiv preprintarXiv:1805.08191, 2018.

[6] T.-H. K. Huang, F. Ferraro, N. Mostafazadeh, I. Misra,A. Agrawal, J. Devlin, R. Girshick, X. He, P. Kohli, D. Batra,et al. Visual storytelling. In Proceedings of the 2016 Confer-ence of the North American Chapter of the Association forComputational Linguistics: Human Language Technologies,pages 1233–1239, 2016.

[7] T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ra-manan, P. Dollar, and C. L. Zitnick. Microsoft coco: Com-mon objects in context. In European conference on computervision, pages 740–755. Springer, 2014.

[8] C. C. Park and G. Kim. Expressing an image stream with asequence of natural sentences. In Advances in neural infor-mation processing systems, pages 73–81, 2015.

[9] I. Sutskever, O. Vinyals, and Q. V. Le. Sequence to sequencelearning with neural networks. In Advances in neural infor-mation processing systems, pages 3104–3112, 2014.

[10] O. Vinyals, A. Toshev, S. Bengio, and D. Erhan. Show andtell: A neural image caption generator. In Proceedings ofthe IEEE conference on computer vision and pattern recog-nition, pages 3156–3164, 2015.

[11] J. Wang, J. Fu, J. Tang, Z. Li, and T. Mei. Show, reward andtell: Automatic generation of narrative paragraph from photostream by adversarial training. AAAI, 2018.

[12] X. Wang, W. Chen, Y.-F. Wang, and W. Y. Wang. No met-rics are perfect: Adversarial reward learning for visual story-telling. arXiv preprint arXiv:1804.09160, 2018.

[13] K. Xu, J. Ba, R. Kiros, K. Cho, A. Courville, R. Salakhudi-nov, R. Zemel, and Y. Bengio. Show, attend and tell: Neuralimage caption generation with visual attention. In Interna-tional conference on machine learning, pages 2048–2057,2015.

[14] L. Yu, M. Bansal, and T. L. Berg. Hierarchically-attentivernn for album summarization and storytelling. arXiv preprintarXiv:1708.02977, 2017.