Not All Images are Worth 16x16 Words: DynamicTransformers for Efficient Image Recognition

Yulin Wang1∗ Rui Huang1∗ Shiji Song1 Zeyi Huang2 Gao Huang1,3†1Department of Automation, BNRist, Tsinghua University, Beijing, China

2Huawei Technologies Ltd., China3Beijing Academy of Artificial Intelligence, Beijing, China

{wang-yl19, hr20}@mails.tsinghua.edu.cn, huangzeyi2@huawei.com.{shijis, gaohuang}@tsinghua.edu.cn

Abstract

Vision Transformers (ViT) have achieved remarkable success in large-scale imagerecognition. They split every 2D image into a fixed number of patches, eachof which is treated as a token. Generally, representing an image with more to-kens would lead to higher prediction accuracy, while it also results in drasticallyincreased computational cost. To achieve a decent trade-off between accuracyand speed, the number of tokens is empirically set to 16x16 or 14x14. In thispaper, we argue that every image has its own characteristics, and ideally the to-ken number should be conditioned on each individual input. In fact, we haveobserved that there exist a considerable number of “easy” images which can beaccurately predicted with a mere number of 4x4 tokens, while only a small frac-tion of “hard” ones need a finer representation. Inspired by this phenomenon,we propose a Dynamic Transformer to automatically configure a proper num-ber of tokens for each input image. This is achieved by cascading multipleTransformers with increasing numbers of tokens, which are sequentially acti-vated in an adaptive fashion at test time, i.e., the inference is terminated oncea sufficiently confident prediction is produced. We further design efficient fea-ture reuse and relationship reuse mechanisms across different components of theDynamic Transformer to reduce redundant computations. Extensive empiricalresults on ImageNet, CIFAR-10, and CIFAR-100 demonstrate that our methodsignificantly outperforms the competitive baselines in terms of both theoreticalcomputational efficiency and practical inference speed. Code and pre-trained mod-els (based on PyTorch and MindSpore) are available at https://github.com/blackfeather-wang/Dynamic-Vision-Transformer and https://github.com/blackfeather-wang/Dynamic-Vision-Transformer-MindSpore.

1 IntroductionTransformers, the dominant self-attention-based models in natural language processing (NLP) [10,40, 3], have been successfully adapted to image recognition problems [11, 55, 38, 17] recently. Inparticular, vision Transformers achieve state-of-the-art performance on the large scale ImageNetbenchmark [9], while exhibit excellent scalability with the further growing dataset size (e.g., on JFT-300M [11]). These models split each image into a fixed number of patches and embed them into 1Dtokens as inputs. Typically, representing the data using more tokens contributes to higher predictionaccuracy, but leads to intensive computational cost, which grows quadratically with respect to the

∗Equal contribution.†Corresponding author.

35th Conference on Neural Information Processing Systems (NeurIPS 2021), Sydney, Australia.

arX

iv:2

105.

1507

5v2

[cs

.CV

] 2

6 O

ct 2

021

token number in self-attention blocks. For a proper trade-off between efficiency and effectiveness,existing works empirically adopt 14x14 or 16x16 tokens [11, 55].

Table 1: Accuracy and computa-tional cost of T2T-ViT-12 with dif-ferent token numbers on ImageNet.# of Tokens 14x14 7x7 4x4

Accuracy 76.7% 70.3% 60.8%FLOPs 1.78G 0.47G 0.21G

In this paper, we argue that it may not be optimal to treat allsamples with the same number of tokens. In fact, there existconsiderable variations among different images (e.g., contents,scales of objects, backgrounds, etc.). Therefore, the number ofrepresentative tokens should ideally be configured specificallyfor each input. This issue is critical for the computationalefficiency of the models. For example, we train a T2T-ViT-12[55] with varying token numbers, and report the corresponding accuracy and FLOPs in Table 1.One can observe that adopting the officially recommended 14x14 tokens only correctly recognizes∼15.9% (76.7% v.s. 60.8%) more test samples compared to that of using 4x4 tokens, while increasesthe computational cost by 8.5x (1.78G v.s. 0.21G). In other words, computational resources arewasted on applying the unnecessary 14x14 tokens to many “easy” images for which 4x4 tokens aresufficient.

Easy

Hard

97.3%Apple

21.4%Egret

86.9%Egret

15.2%Palace

43.7%Palace

91.4%Palace

… … …

Terminate & Output

Continue

ReuseComputation

Figure 1: Examples for DVT.

Motivated by this observation, we propose a novel Dy-namic Vision Transformer (DVT) framework, aiming toautomatically configure a decent token number conditionedon each image for high computational efficiency. In spe-cific, a cascade of Transformers are trained using increas-ing number of tokens. At test time, these models are se-quentially activated starting with less tokens. Once a pre-diction with sufficient confidence has been produced, theinference procedure will be terminated immediately. As aconsequence, the computation is unevenly allocated among“easy” and “hard” samples by adjusting the token number,yielding a considerable improvement in efficiency. Impor-tantly, we further develop feature-wise and relationship-wise reuse mechanisms to reduce redundant computations.The former allows the downstream models to be trained onthe basis of previously extracted deep features, while thelater enables leveraging existing upstream self-attentionrelationships to learn more accurate attention maps. Illus-trative examples of our method are given in Figure 1.

Notably, DVT is designed as a general framework. Most of the state-of-the-art image recognitionTransformers, such as ViT [11], DeiT [38], and T2T-ViT [55], can be straightforwardly deployed asits backbones for higher efficiency. Our method is also appealing in its flexibility. The computationalcost of DVT is able to be adjusted online by simply adapting the early-termination criterion. Thischaracteristic makes DVT suitable for the cases where the available computational resources fluctuatedynamically or a minimal power consumption is required to achieve a given performance. Bothsituations are ubiquitous in real-world applications (e.g., searching engines and mobile apps).

The performance of DVT is evaluated on ImageNet [9] and CIFAR [26] with T2T-ViT [55] and DeiT[38]. Experimental results show that DVT significantly improves the efficiency of the backbones. Forexamples, DVT reduces the computational cost of T2T-ViT by 1.6-3.6x without sacrificing accuracy.The real inference speed on a NVIDIA 2080Ti GPU is consistent with our theoretical results.

2 Related WorkVision Transformers. Inspired by the success of Transformers on NLP tasks [10, 40, 3, 43], visionTransformers (ViT) have recently been developed for image recognition [11]. Although ViT by itselfis not comparable with state-of-the-art convolutional networks (CNN) on the standard ImageNetbenchmark, it attains excellent results when pre-trained on the larger JFT-300M dataset. DeiT [38]studies the training strategy of ViT and proposes a knowledge distilling-based approach, surpassingthe performance of ResNet [19]. Some following works such as T2T-ViT [55], TNT [17], CaiT [39],DeepViT [62], CPVT [6], LocalViT [28] and CrossViT [5] focus on improving the architecture designof ViT. Another line of research proposes to integrate the inductive bias of CNN into Transformers[49, 8, 54, 16]. There are also attempts to adapt ViT for other vision tasks (e.g., object detection,semantic segmentation, etc.) [30, 44, 12, 20, 57, 60, 14]. The most majority of these concurrent

2

works represent each image with a fix number of tokens. To the best of our knowledge, we are thefirst to consider configuring token numbers conditioned on the inputs.

Efficient deep networks. Computational efficiency plays a critical role in real-world scenarios, wherethe executed computation translates into power consumption, carbon emission or latency. A numberof works have been done on reducing the computational cost of CNNs [22, 33, 21, 53, 59, 31, 35].However, designing efficient vision Transformers is still an under-explored topic. T2T-ViT [55]proposes a light-weighted tokens-to-token module and obtains a competitive accuracy-parametertrade-off compared to MobileNetV2 [33]. LeViT [16] accelerates the inference of Transformermodels by involving convolutional layers. Swin Transformer [30] introduces an efficient shiftedwindow-based approach in multi-stage vision Transformers. Compared to these models with fixedcomputational graphs, the proposed DVT framework improves the efficiency by adaptively changingthe architecture of the network on a per-sample basis.

Dynamic models. Designing dynamic architectures is an effective approach for efficient deeplearning [18]. In the context of recognition tasks, MSDNet and its variants [23, 52, 27] develop amulti-classifier CNN architecture to perform early exiting for easy samples. Another type of dynamicCNNs skips redundant layers [41, 45, 51] or channels [29] conditioned on the inputs. Besides,the spatial adaptive paradigm [15, 4, 47, 42, 46] has been proposed for efficient image and videorecognition. Although these works are related to DVT on the spirit of adaptive computation, they aredeveloped based on CNN, while DVT is tailored for vision Transformers.

Notably, our work may seem similar to the multi-exit networks [23, 52, 27, 47] from the lens ofearly-exiting. However, DVT differentiates itself from these existing works in several importantaspects. For example, a major efficiency gain in both MSDNet [23] and RANet [52] comes fromthe adaptive depth, namely processing “easier” samples using fewer layers within a single network.In comparison, DVT cascades multiple Transformers to infer several entire networks at test time.Besides, DVT does not leverage the multi-scale architecture or dense connection in MSDNet [23].Compared with RANet [52], DVT introduces the adaptive token number and the relationship reusemechanism tailored for ViT. In addition, IMTA [27] studies the training techniques of multi-exitmodels, which are actually complementary to DVT. Another difference is that all these networks[23, 52, 27, 47] adopt pure convolutional models.

3 Dynamic Vision TransformerVision Transformers [11, 17, 38, 55] split each 2D image into 1D tokens, while model their longrange interaction with the self-attention mechanism [40]. As aforementioned, to correctly recognizesome “hard” images and achieve high accuracy, the number of tokens usually needs to be large,leading to the quadratically grown computational cost. However, “easier” images that make up thebulk of the datasets typically require far fewer tokens and much less costs (as shown in Table 1).Inspired by this observation, we propose a Dynamic Vision Transformers (DVT), aiming to improvethe computational efficiency of Transformers via adaptively reducing the number of representativetokens for each input.

In specific, we propose to deploy multiple Transformers trained with increasing number of tokens,such that one can sequentially activate them for each test image until obtaining a convincing prediction(e.g., with sufficient confidence). The computation is allocated unevenly across different samples forimproving the overall efficiency. It is worth noting that, if all the Transformers are learned separately,the computation performed by upstream models will simply be abandoned once a downstreamTransformer is activated, resulting in considerable inefficiency. To alleviate this problem, we introducethe efficient feature and relationship reuse mechanisms.

3.1 OverviewInference. We start by describing the inference procedure of DVT, which is shown in Figure 2. Foreach test sample, we first coarsely represent it using a small number of 1D token embeddings. Thiscan be achieved by either straightforwardly flattening the split image patches [11, 17] or leveragingtechniques like the tokens-to-token module [55]. We infer a vision Transformer with these few tokensto obtain a quick prediction. This process enjoys high efficiency since the computational cost ofTransformers grows quadratically with respect to token number. Then the prediction will be evaluatedwith certain criterion to determine whether it is reliable enough to be retrieved immediately. In thispaper, early-termination is performed when the model is sufficiently confident (details in Section 3.3).

3

No

No

No

Yes

Yes

Yes

Relationship Reuse Feature Reuse

Vision Transformer

✱

✱

…

✱

Relationship Reuse Feature Reuse

Vision Transformer

Vision Transformer

… … … …

Exit? Output

Exit? Output

Exit? Output

…

Classification Token

✱

Image Tokens

Position Embedding

TerminationCriterion

Exit?

Forward

Figure 2: An overview of Dynamic Vision Transformers (DVT). Under the objective of configuringproper token numbers conditioned on the inputs, we cascade multiple Transformers with increasingnumber of tokens. At test time, they are sequentially activated until a convincing prediction (e.g.sufficiently confident) has been obtained or the final model has been inferred. The feature andrelationship reuse mechanisms allow reusing computation across different Transformers.

Once the prediction fails to meet the termination criterion, the original input image will be splitinto more tokens for more accurate but computationally more expensive inference. Note that, herethe dimension of each token embedding remains unchanged, while the number of tokens increases,enabling more fine-grained representation. An additional Transformer with the same architectureas the previous one but different parameters will be activated. By design, this stage trades offcomputation for higher accuracy on some “difficult” test samples. To improve the efficiency, thenew model can reuse the previously learned features and relationships, which will be introduced inSection 3.2. Similarly, after obtaining a new prediction, the termination criterion will be applied, andthe above procedure will proceed until the sample exits or the final Transformer has been inferred.

Training. For training, we simply train DVT to produce correct predictions at all exits (i.e., eachwith the corresponding number of tokens). Formally, the optimization objective is

minimize1

|Dtrain|∑

(x,y)∈Dtrain

[∑iLCE(pi, y)

], (1)

where (x, y) denote a sample in the training set Dtrain and its corresponding label. We adopt thestandard cross-entropy loss function LCE(·), while pi denotes the softmax prediction probabilityoutput by the ith exit. We find that such a simple training objective works well in practice.

Transformer backbone. DVT is proposed as a general and flexible framework. It can be built ontop of most existing vision Transformers like ViT [11], DeiT [38] and T2T-ViT [55] to improve theirefficiency. The architecture of Transformers simply follows the implementation of these backbones.

3.2 Feature and Relationship Reuse

An important challenge to develop our DVT approach is how to facilitate the reuse of computation.That is, once a downstream Transformer with more tokens is inferred, it is obviously inefficient ifthe computation performed in previous models is abandoned. The upstream models, although beingbased on smaller number of input tokens, are trained with the same objective, and have extractedvaluable information for fulfilling the task. Therefore, we propose two mechanisms to reuse thelearned deep features and self-attention relationships. Both of them are able to improve the testaccuracy significantly by involving minimal extra computational cost.

Background. For the ease of introduction, we first revisit the basic formulation of vision Trans-formers. The Transformer encoders consist of alternatively stacked multi-head self-attention (MSA)and multi-layer perceptron (MLP) blocks [40, 11]. The layer normalization (LN) [2] and residual

4

LN

Multi-head

Self-attention

… C+

LN

ML

P …+

Extract Context Embedding fl

<latexit sha1_base64="NW9/G341DqyFZwS1ROiqNShZDUY=">AAAIBnicnVXLbtNQEJ22gEt4tIUlG4uoEgsa4gqJLivYsGBRUJNWKlVlO9etiV/4Qami7FmyhY9gh9jyG/wB/AVn5jrN26HYajp37pkzd17XThL4Wd5s/lpaXrl2/YaxerN26/adu2vrG/faWVykrmq5cRCnh46dqcCPVCv380AdJqmyQydQB073Be8ffFBp5sfRfn6RqOPQPo18z3ftHKq2d9IL+rWT9Xqz0ZTHnBasUqhT+ezFGysFvaUOxeRSQSEpiiiHHJBNGd4jsqhJCXTH1IMuheTLvqI+1WBbAKWAsKHt4vcUq6NSG2HNnJlYu/AS4C+FpUmbsImBSyGzN1P2C2Fm7TzunnDy2S7w3ym5QmhzOoN2kd0A+a92HEtOHu1IDD5iSkTD0bklSyFZ4ZObI1HlYEigY7mD/RSyK5aDPJtik0nsnFtb9n8LkrW8dktsQX/klPr9/7MyayB1OIes8VuwMEfw8/n30Q/M7cE+GmPW76ZwZpKJHfA6YOU42c68tKqq0hnqqrPGWfBGuIcehphATt2tZCwEN8kzzpiBJ4SlrmGL3tArqaD2w6y51DgC53xPDuKLsebKO0DMOnuC08SIS0kP+5IRrsgWvQerLRGxX7Nk0R2yqKuHmZ3l05EJYys+IXdaB9pB75uXs8TzWeUrktnnquheneWLpzgsLfMpbt0DPBEWPaFtegxJyY3TqPAbil/O26Ai/Sm/k5gYtqlMIHdLVc3Cipts+taoQgaQODpfsPpEV7OoQodyO+Zyk/D0VmGHsfMZPlZiM+gd1MXHupCpq0Kfl8wavzhzV4lLfyEWcw7v/vnIBD3uifZUJghYfB2tyW/htNDebljNhvX6aX33efmdXKUH9JAeoWef0S69pD3cEC69o8/0hb4an4xvxnfjh4YuL5U292nsMX7+BQSFh84=</latexit><latexit sha1_base64="NW9/G341DqyFZwS1ROiqNShZDUY=">AAAIBnicnVXLbtNQEJ22gEt4tIUlG4uoEgsa4gqJLivYsGBRUJNWKlVlO9etiV/4Qami7FmyhY9gh9jyG/wB/AVn5jrN26HYajp37pkzd17XThL4Wd5s/lpaXrl2/YaxerN26/adu2vrG/faWVykrmq5cRCnh46dqcCPVCv380AdJqmyQydQB073Be8ffFBp5sfRfn6RqOPQPo18z3ftHKq2d9IL+rWT9Xqz0ZTHnBasUqhT+ezFGysFvaUOxeRSQSEpiiiHHJBNGd4jsqhJCXTH1IMuheTLvqI+1WBbAKWAsKHt4vcUq6NSG2HNnJlYu/AS4C+FpUmbsImBSyGzN1P2C2Fm7TzunnDy2S7w3ym5QmhzOoN2kd0A+a92HEtOHu1IDD5iSkTD0bklSyFZ4ZObI1HlYEigY7mD/RSyK5aDPJtik0nsnFtb9n8LkrW8dktsQX/klPr9/7MyayB1OIes8VuwMEfw8/n30Q/M7cE+GmPW76ZwZpKJHfA6YOU42c68tKqq0hnqqrPGWfBGuIcehphATt2tZCwEN8kzzpiBJ4SlrmGL3tArqaD2w6y51DgC53xPDuKLsebKO0DMOnuC08SIS0kP+5IRrsgWvQerLRGxX7Nk0R2yqKuHmZ3l05EJYys+IXdaB9pB75uXs8TzWeUrktnnquheneWLpzgsLfMpbt0DPBEWPaFtegxJyY3TqPAbil/O26Ai/Sm/k5gYtqlMIHdLVc3Cipts+taoQgaQODpfsPpEV7OoQodyO+Zyk/D0VmGHsfMZPlZiM+gd1MXHupCpq0Kfl8wavzhzV4lLfyEWcw7v/vnIBD3uifZUJghYfB2tyW/htNDebljNhvX6aX33efmdXKUH9JAeoWef0S69pD3cEC69o8/0hb4an4xvxnfjh4YuL5U292nsMX7+BQSFh84=</latexit><latexit sha1_base64="NW9/G341DqyFZwS1ROiqNShZDUY=">AAAIBnicnVXLbtNQEJ22gEt4tIUlG4uoEgsa4gqJLivYsGBRUJNWKlVlO9etiV/4Qami7FmyhY9gh9jyG/wB/AVn5jrN26HYajp37pkzd17XThL4Wd5s/lpaXrl2/YaxerN26/adu2vrG/faWVykrmq5cRCnh46dqcCPVCv380AdJqmyQydQB073Be8ffFBp5sfRfn6RqOPQPo18z3ftHKq2d9IL+rWT9Xqz0ZTHnBasUqhT+ezFGysFvaUOxeRSQSEpiiiHHJBNGd4jsqhJCXTH1IMuheTLvqI+1WBbAKWAsKHt4vcUq6NSG2HNnJlYu/AS4C+FpUmbsImBSyGzN1P2C2Fm7TzunnDy2S7w3ym5QmhzOoN2kd0A+a92HEtOHu1IDD5iSkTD0bklSyFZ4ZObI1HlYEigY7mD/RSyK5aDPJtik0nsnFtb9n8LkrW8dktsQX/klPr9/7MyayB1OIes8VuwMEfw8/n30Q/M7cE+GmPW76ZwZpKJHfA6YOU42c68tKqq0hnqqrPGWfBGuIcehphATt2tZCwEN8kzzpiBJ4SlrmGL3tArqaD2w6y51DgC53xPDuKLsebKO0DMOnuC08SIS0kP+5IRrsgWvQerLRGxX7Nk0R2yqKuHmZ3l05EJYys+IXdaB9pB75uXs8TzWeUrktnnquheneWLpzgsLfMpbt0DPBEWPaFtegxJyY3TqPAbil/O26Ai/Sm/k5gYtqlMIHdLVc3Cipts+taoQgaQODpfsPpEV7OoQodyO+Zyk/D0VmGHsfMZPlZiM+gd1MXHupCpq0Kfl8wavzhzV4lLfyEWcw7v/vnIBD3uifZUJghYfB2tyW/htNDebljNhvX6aX33efmdXKUH9JAeoWef0S69pD3cEC69o8/0hb4an4xvxnfjh4YuL5U292nsMX7+BQSFh84=</latexit><latexit sha1_base64="NW9/G341DqyFZwS1ROiqNShZDUY=">AAAIBnicnVXLbtNQEJ22gEt4tIUlG4uoEgsa4gqJLivYsGBRUJNWKlVlO9etiV/4Qami7FmyhY9gh9jyG/wB/AVn5jrN26HYajp37pkzd17XThL4Wd5s/lpaXrl2/YaxerN26/adu2vrG/faWVykrmq5cRCnh46dqcCPVCv380AdJqmyQydQB073Be8ffFBp5sfRfn6RqOPQPo18z3ftHKq2d9IL+rWT9Xqz0ZTHnBasUqhT+ezFGysFvaUOxeRSQSEpiiiHHJBNGd4jsqhJCXTH1IMuheTLvqI+1WBbAKWAsKHt4vcUq6NSG2HNnJlYu/AS4C+FpUmbsImBSyGzN1P2C2Fm7TzunnDy2S7w3ym5QmhzOoN2kd0A+a92HEtOHu1IDD5iSkTD0bklSyFZ4ZObI1HlYEigY7mD/RSyK5aDPJtik0nsnFtb9n8LkrW8dktsQX/klPr9/7MyayB1OIes8VuwMEfw8/n30Q/M7cE+GmPW76ZwZpKJHfA6YOU42c68tKqq0hnqqrPGWfBGuIcehphATt2tZCwEN8kzzpiBJ4SlrmGL3tArqaD2w6y51DgC53xPDuKLsebKO0DMOnuC08SIS0kP+5IRrsgWvQerLRGxX7Nk0R2yqKuHmZ3l05EJYys+IXdaB9pB75uXs8TzWeUrktnnquheneWLpzgsLfMpbt0DPBEWPaFtegxJyY3TqPAbil/O26Ai/Sm/k5gYtqlMIHdLVc3Cipts+taoQgaQODpfsPpEV7OoQodyO+Zyk/D0VmGHsfMZPlZiM+gd1MXHupCpq0Kfl8wavzhzV4lLfyEWcw7v/vnIBD3uifZUJghYfB2tyW/htNDebljNhvX6aX33efmdXKUH9JAeoWef0S69pD3cEC69o8/0hb4an4xvxnfjh4YuL5U292nsMX7+BQSFh84=</latexit>

fl�1<latexit sha1_base64="KOsfPxQ53nLO3S/6xxhxOV6mNGo=">AAAIJXicpVXLbtNQEJ22gEt4tbBkYxFVYkGCXSHRZQUbFiwKappKpaps5zq14hd+UKooO36ALXwAX8MOIbFiC3/BmbHTvJ1W2Go6nnvmzJ3XvXbse2lmGL9WVteuXb+hrd+s3bp95+69jc37B2mUJ45qOZEfJYe2lSrfC1Ur8zJfHcaJsgLbV22795LX2x9UknpRuJ+dx+o4sLqh53qOlUHVdk/6fsMcnGzUjaYhjz4rmKVQp/LZizbXPtE76lBEDuUUkKKQMsg+WZTiPSKTDIqhO6Y+dAkkT9YVDagG2xwoBYQFbQ+/XXwdldoQ38yZirUDLz7+EljqtAWbCLgEMnvTZT0XZtYu4u4LJ+/tHP/tkiuANqNTaJfZDZGXteNYMnJpR2LwEFMsGo7OKVlyyQrvXB+LKgNDDB3LHawnkB2xHOZZF5tUYufcWrL+R5Cs5W+nxOb0tzI6G6xckdp/RcI+fanSGeQC34CFPoZfzL+PbmFuF/bhBHPxbglnKnnaAa8NVs4C2+kXVlVRnqLqRU45R+4Y98jDCOPLrnuVjLngpnkmGVPwBLAsKtyit/Ra6lv4YdZMOiAEZ1WFIrw96QsbiHl7j7GbCHEp6XBPMsIVadB7sFoSEfvVS5aif5b1/Ciz83zaMn9sxTvkPuxAO5wM/WLSeHqrfIVyMnBVik6e54tnPCgtsxnuogd4Xkx6Stv0BJKS86hZ4TcQv5y3YUUGM36nMRFsE5lP7pblU3XZM6UK6UPi6DzBFju6mkUVOpCzM5Nzhqe3CjuKnffwsRKbQm+jLh6+c5m6KvRZyVzgl2fuKnEV98dyztHNsBgZo8dd0XZlgoDF3WlO35SzwsF20zSa5ptn9d0X5S26Tg/pET1Gzz6nXXpFezgh2Ndn+kJftW/ad+2H9rOArq6UNg9o4tF+/wNh0Y8N</latexit><latexit sha1_base64="KOsfPxQ53nLO3S/6xxhxOV6mNGo=">AAAIJXicpVXLbtNQEJ22gEt4tbBkYxFVYkGCXSHRZQUbFiwKappKpaps5zq14hd+UKooO36ALXwAX8MOIbFiC3/BmbHTvJ1W2Go6nnvmzJ3XvXbse2lmGL9WVteuXb+hrd+s3bp95+69jc37B2mUJ45qOZEfJYe2lSrfC1Ur8zJfHcaJsgLbV22795LX2x9UknpRuJ+dx+o4sLqh53qOlUHVdk/6fsMcnGzUjaYhjz4rmKVQp/LZizbXPtE76lBEDuUUkKKQMsg+WZTiPSKTDIqhO6Y+dAkkT9YVDagG2xwoBYQFbQ+/XXwdldoQ38yZirUDLz7+EljqtAWbCLgEMnvTZT0XZtYu4u4LJ+/tHP/tkiuANqNTaJfZDZGXteNYMnJpR2LwEFMsGo7OKVlyyQrvXB+LKgNDDB3LHawnkB2xHOZZF5tUYufcWrL+R5Cs5W+nxOb0tzI6G6xckdp/RcI+fanSGeQC34CFPoZfzL+PbmFuF/bhBHPxbglnKnnaAa8NVs4C2+kXVlVRnqLqRU45R+4Y98jDCOPLrnuVjLngpnkmGVPwBLAsKtyit/Ra6lv4YdZMOiAEZ1WFIrw96QsbiHl7j7GbCHEp6XBPMsIVadB7sFoSEfvVS5aif5b1/Ciz83zaMn9sxTvkPuxAO5wM/WLSeHqrfIVyMnBVik6e54tnPCgtsxnuogd4Xkx6Stv0BJKS86hZ4TcQv5y3YUUGM36nMRFsE5lP7pblU3XZM6UK6UPi6DzBFju6mkUVOpCzM5Nzhqe3CjuKnffwsRKbQm+jLh6+c5m6KvRZyVzgl2fuKnEV98dyztHNsBgZo8dd0XZlgoDF3WlO35SzwsF20zSa5ptn9d0X5S26Tg/pET1Gzz6nXXpFezgh2Ndn+kJftW/ad+2H9rOArq6UNg9o4tF+/wNh0Y8N</latexit><latexit sha1_base64="KOsfPxQ53nLO3S/6xxhxOV6mNGo=">AAAIJXicpVXLbtNQEJ22gEt4tbBkYxFVYkGCXSHRZQUbFiwKappKpaps5zq14hd+UKooO36ALXwAX8MOIbFiC3/BmbHTvJ1W2Go6nnvmzJ3XvXbse2lmGL9WVteuXb+hrd+s3bp95+69jc37B2mUJ45qOZEfJYe2lSrfC1Ur8zJfHcaJsgLbV22795LX2x9UknpRuJ+dx+o4sLqh53qOlUHVdk/6fsMcnGzUjaYhjz4rmKVQp/LZizbXPtE76lBEDuUUkKKQMsg+WZTiPSKTDIqhO6Y+dAkkT9YVDagG2xwoBYQFbQ+/XXwdldoQ38yZirUDLz7+EljqtAWbCLgEMnvTZT0XZtYu4u4LJ+/tHP/tkiuANqNTaJfZDZGXteNYMnJpR2LwEFMsGo7OKVlyyQrvXB+LKgNDDB3LHawnkB2xHOZZF5tUYufcWrL+R5Cs5W+nxOb0tzI6G6xckdp/RcI+fanSGeQC34CFPoZfzL+PbmFuF/bhBHPxbglnKnnaAa8NVs4C2+kXVlVRnqLqRU45R+4Y98jDCOPLrnuVjLngpnkmGVPwBLAsKtyit/Ra6lv4YdZMOiAEZ1WFIrw96QsbiHl7j7GbCHEp6XBPMsIVadB7sFoSEfvVS5aif5b1/Ciz83zaMn9sxTvkPuxAO5wM/WLSeHqrfIVyMnBVik6e54tnPCgtsxnuogd4Xkx6Stv0BJKS86hZ4TcQv5y3YUUGM36nMRFsE5lP7pblU3XZM6UK6UPi6DzBFju6mkUVOpCzM5Nzhqe3CjuKnffwsRKbQm+jLh6+c5m6KvRZyVzgl2fuKnEV98dyztHNsBgZo8dd0XZlgoDF3WlO35SzwsF20zSa5ptn9d0X5S26Tg/pET1Gzz6nXXpFezgh2Ndn+kJftW/ad+2H9rOArq6UNg9o4tF+/wNh0Y8N</latexit><latexit sha1_base64="KOsfPxQ53nLO3S/6xxhxOV6mNGo=">AAAIJXicpVXLbtNQEJ22gEt4tbBkYxFVYkGCXSHRZQUbFiwKappKpaps5zq14hd+UKooO36ALXwAX8MOIbFiC3/BmbHTvJ1W2Go6nnvmzJ3XvXbse2lmGL9WVteuXb+hrd+s3bp95+69jc37B2mUJ45qOZEfJYe2lSrfC1Ur8zJfHcaJsgLbV22795LX2x9UknpRuJ+dx+o4sLqh53qOlUHVdk/6fsMcnGzUjaYhjz4rmKVQp/LZizbXPtE76lBEDuUUkKKQMsg+WZTiPSKTDIqhO6Y+dAkkT9YVDagG2xwoBYQFbQ+/XXwdldoQ38yZirUDLz7+EljqtAWbCLgEMnvTZT0XZtYu4u4LJ+/tHP/tkiuANqNTaJfZDZGXteNYMnJpR2LwEFMsGo7OKVlyyQrvXB+LKgNDDB3LHawnkB2xHOZZF5tUYufcWrL+R5Cs5W+nxOb0tzI6G6xckdp/RcI+fanSGeQC34CFPoZfzL+PbmFuF/bhBHPxbglnKnnaAa8NVs4C2+kXVlVRnqLqRU45R+4Y98jDCOPLrnuVjLngpnkmGVPwBLAsKtyit/Ra6lv4YdZMOiAEZ1WFIrw96QsbiHl7j7GbCHEp6XBPMsIVadB7sFoSEfvVS5aif5b1/Ciz83zaMn9sxTvkPuxAO5wM/WLSeHqrfIVyMnBVik6e54tnPCgtsxnuogd4Xkx6Stv0BJKS86hZ4TcQv5y3YUUGM36nMRFsE5lP7pblU3XZM6UK6UPi6DzBFju6mkUVOpCzM5Nzhqe3CjuKnffwsRKbQm+jLh6+c5m6KvRZyVzgl2fuKnEV98dyztHNsBgZo8dd0XZlgoDF3WlO35SzwsF20zSa5ptn9d0X5S26Tg/pET1Gzz6nXXpFezgh2Ndn+kJftW/ad+2H9rOArq6UNg9o4tF+/wNh0Y8N</latexit>

fl+1<latexit sha1_base64="CoB6tAWh6vEp3Z7SYV0BRz7vyyk=">AAAIJXicpVXLbtNQEJ22gEt4tIUlG4uoEhI02BUSXVawYcGioKapVKrKdq5TK37hB6WKsuMH2MIH8DXsEBIrtvAXnBk7zdtpha2m47lnztx53WvHvpdmhvFraXnl2vUb2urN2q3bd+6urW/cO0ijPHFU04n8KDm0rVT5XqiamZf56jBOlBXYvmrZ3Ze83vqgktSLwv3sPFbHgdUJPddzrAyqlnvS8x+b/ZP1utEw5NGnBbMU6lQ+e9HGyid6R22KyKGcAlIUUgbZJ4tSvEdkkkExdMfUgy6B5Mm6oj7VYJsDpYCwoO3it4Ovo1Ib4ps5U7F24MXHXwJLnTZhEwGXQGZvuqznwszaedw94eS9neO/XXIF0GZ0Cu0iuwHysnYcS0Yu7UgMHmKKRcPROSVLLlnhnesjUWVgiKFjuY31BLIjloM862KTSuycW0vW/wiStfztlNic/lZGZ4OVK1L7r0jYpy9VOoNc4LdgoY/g5/Pvo1uY24V9OMZcvJvCmUqedsBrg5WzwHb6hVVVlKeoepFTzpE7wj30MMT4sutuJWMuuEmeccYUPAEsiwo36S29lvoWfpg1kw4IwVlVoQhvV/rCBmLW3mPsJkJcSjrck4xwRbboPVgtiYj96iVL0T+Len6Y2Vk+bZk/tuIdch+2oR1Mhn4xaTy9Vb5CORm4KkUnz/LFMx6UltkUd9EDPC8mPaVtegJJyXnUqPAbiF/O26Ai/Sm/k5gItonMJ3fL4qm67JlShfQhcXSeYIsdXc2iCh3I2ZnJOcPTW4Udxs57+FiJTaG3URcP37lMXRX6rGQu8Iszd5W4ivtjMefwZpiPjNHjrmg7MkHA4u40J2/KaeFgu2EaDfPNs/rui/IWXaUH9JAeoWef0y69oj2cEOzrM32hr9o37bv2Q/tZQJeXSpv7NPZov/8BUlePCw==</latexit><latexit sha1_base64="CoB6tAWh6vEp3Z7SYV0BRz7vyyk=">AAAIJXicpVXLbtNQEJ22gEt4tIUlG4uoEhI02BUSXVawYcGioKapVKrKdq5TK37hB6WKsuMH2MIH8DXsEBIrtvAXnBk7zdtpha2m47lnztx53WvHvpdmhvFraXnl2vUb2urN2q3bd+6urW/cO0ijPHFU04n8KDm0rVT5XqiamZf56jBOlBXYvmrZ3Ze83vqgktSLwv3sPFbHgdUJPddzrAyqlnvS8x+b/ZP1utEw5NGnBbMU6lQ+e9HGyid6R22KyKGcAlIUUgbZJ4tSvEdkkkExdMfUgy6B5Mm6oj7VYJsDpYCwoO3it4Ovo1Ib4ps5U7F24MXHXwJLnTZhEwGXQGZvuqznwszaedw94eS9neO/XXIF0GZ0Cu0iuwHysnYcS0Yu7UgMHmKKRcPROSVLLlnhnesjUWVgiKFjuY31BLIjloM862KTSuycW0vW/wiStfztlNic/lZGZ4OVK1L7r0jYpy9VOoNc4LdgoY/g5/Pvo1uY24V9OMZcvJvCmUqedsBrg5WzwHb6hVVVlKeoepFTzpE7wj30MMT4sutuJWMuuEmeccYUPAEsiwo36S29lvoWfpg1kw4IwVlVoQhvV/rCBmLW3mPsJkJcSjrck4xwRbboPVgtiYj96iVL0T+Len6Y2Vk+bZk/tuIdch+2oR1Mhn4xaTy9Vb5CORm4KkUnz/LFMx6UltkUd9EDPC8mPaVtegJJyXnUqPAbiF/O26Ai/Sm/k5gItonMJ3fL4qm67JlShfQhcXSeYIsdXc2iCh3I2ZnJOcPTW4Udxs57+FiJTaG3URcP37lMXRX6rGQu8Iszd5W4ivtjMefwZpiPjNHjrmg7MkHA4u40J2/KaeFgu2EaDfPNs/rui/IWXaUH9JAeoWef0y69oj2cEOzrM32hr9o37bv2Q/tZQJeXSpv7NPZov/8BUlePCw==</latexit><latexit sha1_base64="CoB6tAWh6vEp3Z7SYV0BRz7vyyk=">AAAIJXicpVXLbtNQEJ22gEt4tIUlG4uoEhI02BUSXVawYcGioKapVKrKdq5TK37hB6WKsuMH2MIH8DXsEBIrtvAXnBk7zdtpha2m47lnztx53WvHvpdmhvFraXnl2vUb2urN2q3bd+6urW/cO0ijPHFU04n8KDm0rVT5XqiamZf56jBOlBXYvmrZ3Ze83vqgktSLwv3sPFbHgdUJPddzrAyqlnvS8x+b/ZP1utEw5NGnBbMU6lQ+e9HGyid6R22KyKGcAlIUUgbZJ4tSvEdkkkExdMfUgy6B5Mm6oj7VYJsDpYCwoO3it4Ovo1Ib4ps5U7F24MXHXwJLnTZhEwGXQGZvuqznwszaedw94eS9neO/XXIF0GZ0Cu0iuwHysnYcS0Yu7UgMHmKKRcPROSVLLlnhnesjUWVgiKFjuY31BLIjloM862KTSuycW0vW/wiStfztlNic/lZGZ4OVK1L7r0jYpy9VOoNc4LdgoY/g5/Pvo1uY24V9OMZcvJvCmUqedsBrg5WzwHb6hVVVlKeoepFTzpE7wj30MMT4sutuJWMuuEmeccYUPAEsiwo36S29lvoWfpg1kw4IwVlVoQhvV/rCBmLW3mPsJkJcSjrck4xwRbboPVgtiYj96iVL0T+Len6Y2Vk+bZk/tuIdch+2oR1Mhn4xaTy9Vb5CORm4KkUnz/LFMx6UltkUd9EDPC8mPaVtegJJyXnUqPAbiF/O26Ai/Sm/k5gItonMJ3fL4qm67JlShfQhcXSeYIsdXc2iCh3I2ZnJOcPTW4Udxs57+FiJTaG3URcP37lMXRX6rGQu8Iszd5W4ivtjMefwZpiPjNHjrmg7MkHA4u40J2/KaeFgu2EaDfPNs/rui/IWXaUH9JAeoWef0y69oj2cEOzrM32hr9o37bv2Q/tZQJeXSpv7NPZov/8BUlePCw==</latexit><latexit sha1_base64="CoB6tAWh6vEp3Z7SYV0BRz7vyyk=">AAAIJXicpVXLbtNQEJ22gEt4tIUlG4uoEhI02BUSXVawYcGioKapVKrKdq5TK37hB6WKsuMH2MIH8DXsEBIrtvAXnBk7zdtpha2m47lnztx53WvHvpdmhvFraXnl2vUb2urN2q3bd+6urW/cO0ijPHFU04n8KDm0rVT5XqiamZf56jBOlBXYvmrZ3Ze83vqgktSLwv3sPFbHgdUJPddzrAyqlnvS8x+b/ZP1utEw5NGnBbMU6lQ+e9HGyid6R22KyKGcAlIUUgbZJ4tSvEdkkkExdMfUgy6B5Mm6oj7VYJsDpYCwoO3it4Ovo1Ib4ps5U7F24MXHXwJLnTZhEwGXQGZvuqznwszaedw94eS9neO/XXIF0GZ0Cu0iuwHysnYcS0Yu7UgMHmKKRcPROSVLLlnhnesjUWVgiKFjuY31BLIjloM862KTSuycW0vW/wiStfztlNic/lZGZ4OVK1L7r0jYpy9VOoNc4LdgoY/g5/Pvo1uY24V9OMZcvJvCmUqedsBrg5WzwHb6hVVVlKeoepFTzpE7wj30MMT4sutuJWMuuEmeccYUPAEsiwo36S29lvoWfpg1kw4IwVlVoQhvV/rCBmLW3mPsJkJcSjrck4xwRbboPVgtiYj96iVL0T+Len6Y2Vk+bZk/tuIdch+2oR1Mhn4xaTy9Vb5CORm4KkUnz/LFMx6UltkUd9EDPC8mPaVtegJJyXnUqPAbiF/O26Ai/Sm/k5gItonMJ3fL4qm67JlShfQhcXSeYIsdXc2iCh3I2ZnJOcPTW4Udxs57+FiJTaG3URcP37lMXRX6rGQu8Iszd5W4ivtjMefwZpiPjNHjrmg7MkHA4u40J2/KaeFgu2EaDfPNs/rui/IWXaUH9JAeoWef0y69oj2cEOzrM32hr9o37bv2Q/tZQJeXSpv7NPZov/8BUlePCw==</latexit>

lth<latexit sha1_base64="1J8xf6kBytgiBeG1tIn98QXnqsA=">AAAIM3icpVXLbtNQEJ22gEt4pXTJxiKqxIKGpEKiCxaV2LBgUVDTVmpLZTs3iRW/sK9pqyg7/oMtfAAfg9ghFrDgHzgzdpq30wpbTcdzz5y587rXjjw30bXa96XllRs3bxmrt0t37t67/6C89nA/CdPYUQ0n9ML40LYS5bmBamhXe+owipXl2546sLuveP3go4oTNwz29EWkTnyrHbgt17E0VKflde9971ircx2EsW95Pd3p90/LlVq1Jo85LdRzoUL5sxuurXyiY2pSSA6l5JOigDRkjyxK8B5RnWoUQXdCPehiSK6sK+pTCbYpUAoIC9ouftv4Osq1Ab6ZMxFrB148/MWwNGkDNiFwMWT2Zsp6KsysncfdE07e2wX+2zmXD62mDrSL7AbIq9pxLJpatC0xuIgpEg1H5+QsqWSFd26ORKXBEEHHchPrMWRHLAd5NsUmkdg5t5as/xYka/nbybEp/SmMzgYrV6T0X5GwT0+qdAY5w2/CwhzBz+ffQ7cwdwv2wRhz9m4IZyJ52gavDVbOAtuZl1ZFUXZQ9SynnKPWCPfQwxDjya67hYyp4CZ5xhkT8PiwzCrcoHf0Ruqb+WFWLR0QgLOoQiHervSFDcSsvUfYTYi4lHS4KxnhimzSB7BaEhH7NXOWrH8W9fwws7N82jJ/bMU75D5sQjuYDPNy0nh6i3wFcjJwVbJOnuWLZ9zPLfUUd9YDPC91ekZb9BSSkvOoWuDXF7+ct0FF+lN+JzEhbGOZT+6WxVN11TOlCOlB4uhcwWY7up5FEdqXs1PLOcPTW4Qdxs57OC/EJtDbqIuL71Smrgh9ljNn+MWZu05c2f2xmHN4M8xHRujxlmjbMkHA4u6sT96U08L+VrVeq9bfPq/svMxv0VV6RI/pCXr2Be3Qa9rFCeFgD5/pC301vhk/jJ/Grwy6vJTbrNPYY/z9Bx/ilWk=</latexit><latexit sha1_base64="1J8xf6kBytgiBeG1tIn98QXnqsA=">AAAIM3icpVXLbtNQEJ22gEt4pXTJxiKqxIKGpEKiCxaV2LBgUVDTVmpLZTs3iRW/sK9pqyg7/oMtfAAfg9ghFrDgHzgzdpq30wpbTcdzz5y587rXjjw30bXa96XllRs3bxmrt0t37t67/6C89nA/CdPYUQ0n9ML40LYS5bmBamhXe+owipXl2546sLuveP3go4oTNwz29EWkTnyrHbgt17E0VKflde9971ircx2EsW95Pd3p90/LlVq1Jo85LdRzoUL5sxuurXyiY2pSSA6l5JOigDRkjyxK8B5RnWoUQXdCPehiSK6sK+pTCbYpUAoIC9ouftv4Osq1Ab6ZMxFrB148/MWwNGkDNiFwMWT2Zsp6KsysncfdE07e2wX+2zmXD62mDrSL7AbIq9pxLJpatC0xuIgpEg1H5+QsqWSFd26ORKXBEEHHchPrMWRHLAd5NsUmkdg5t5as/xYka/nbybEp/SmMzgYrV6T0X5GwT0+qdAY5w2/CwhzBz+ffQ7cwdwv2wRhz9m4IZyJ52gavDVbOAtuZl1ZFUXZQ9SynnKPWCPfQwxDjya67hYyp4CZ5xhkT8PiwzCrcoHf0Ruqb+WFWLR0QgLOoQiHervSFDcSsvUfYTYi4lHS4KxnhimzSB7BaEhH7NXOWrH8W9fwws7N82jJ/bMU75D5sQjuYDPNy0nh6i3wFcjJwVbJOnuWLZ9zPLfUUd9YDPC91ekZb9BSSkvOoWuDXF7+ct0FF+lN+JzEhbGOZT+6WxVN11TOlCOlB4uhcwWY7up5FEdqXs1PLOcPTW4Qdxs57OC/EJtDbqIuL71Smrgh9ljNn+MWZu05c2f2xmHN4M8xHRujxlmjbMkHA4u6sT96U08L+VrVeq9bfPq/svMxv0VV6RI/pCXr2Be3Qa9rFCeFgD5/pC301vhk/jJ/Grwy6vJTbrNPYY/z9Bx/ilWk=</latexit><latexit sha1_base64="1J8xf6kBytgiBeG1tIn98QXnqsA=">AAAIM3icpVXLbtNQEJ22gEt4pXTJxiKqxIKGpEKiCxaV2LBgUVDTVmpLZTs3iRW/sK9pqyg7/oMtfAAfg9ghFrDgHzgzdpq30wpbTcdzz5y587rXjjw30bXa96XllRs3bxmrt0t37t67/6C89nA/CdPYUQ0n9ML40LYS5bmBamhXe+owipXl2546sLuveP3go4oTNwz29EWkTnyrHbgt17E0VKflde9971ircx2EsW95Pd3p90/LlVq1Jo85LdRzoUL5sxuurXyiY2pSSA6l5JOigDRkjyxK8B5RnWoUQXdCPehiSK6sK+pTCbYpUAoIC9ouftv4Osq1Ab6ZMxFrB148/MWwNGkDNiFwMWT2Zsp6KsysncfdE07e2wX+2zmXD62mDrSL7AbIq9pxLJpatC0xuIgpEg1H5+QsqWSFd26ORKXBEEHHchPrMWRHLAd5NsUmkdg5t5as/xYka/nbybEp/SmMzgYrV6T0X5GwT0+qdAY5w2/CwhzBz+ffQ7cwdwv2wRhz9m4IZyJ52gavDVbOAtuZl1ZFUXZQ9SynnKPWCPfQwxDjya67hYyp4CZ5xhkT8PiwzCrcoHf0Ruqb+WFWLR0QgLOoQiHervSFDcSsvUfYTYi4lHS4KxnhimzSB7BaEhH7NXOWrH8W9fwws7N82jJ/bMU75D5sQjuYDPNy0nh6i3wFcjJwVbJOnuWLZ9zPLfUUd9YDPC91ekZb9BSSkvOoWuDXF7+ct0FF+lN+JzEhbGOZT+6WxVN11TOlCOlB4uhcwWY7up5FEdqXs1PLOcPTW4Qdxs57OC/EJtDbqIuL71Smrgh9ljNn+MWZu05c2f2xmHN4M8xHRujxlmjbMkHA4u6sT96U08L+VrVeq9bfPq/svMxv0VV6RI/pCXr2Be3Qa9rFCeFgD5/pC301vhk/jJ/Grwy6vJTbrNPYY/z9Bx/ilWk=</latexit><latexit sha1_base64="1J8xf6kBytgiBeG1tIn98QXnqsA=">AAAIM3icpVXLbtNQEJ22gEt4pXTJxiKqxIKGpEKiCxaV2LBgUVDTVmpLZTs3iRW/sK9pqyg7/oMtfAAfg9ghFrDgHzgzdpq30wpbTcdzz5y587rXjjw30bXa96XllRs3bxmrt0t37t67/6C89nA/CdPYUQ0n9ML40LYS5bmBamhXe+owipXl2546sLuveP3go4oTNwz29EWkTnyrHbgt17E0VKflde9971ircx2EsW95Pd3p90/LlVq1Jo85LdRzoUL5sxuurXyiY2pSSA6l5JOigDRkjyxK8B5RnWoUQXdCPehiSK6sK+pTCbYpUAoIC9ouftv4Osq1Ab6ZMxFrB148/MWwNGkDNiFwMWT2Zsp6KsysncfdE07e2wX+2zmXD62mDrSL7AbIq9pxLJpatC0xuIgpEg1H5+QsqWSFd26ORKXBEEHHchPrMWRHLAd5NsUmkdg5t5as/xYka/nbybEp/SmMzgYrV6T0X5GwT0+qdAY5w2/CwhzBz+ffQ7cwdwv2wRhz9m4IZyJ52gavDVbOAtuZl1ZFUXZQ9SynnKPWCPfQwxDjya67hYyp4CZ5xhkT8PiwzCrcoHf0Ruqb+WFWLR0QgLOoQiHervSFDcSsvUfYTYi4lHS4KxnhimzSB7BaEhH7NXOWrH8W9fwws7N82jJ/bMU75D5sQjuYDPNy0nh6i3wFcjJwVbJOnuWLZ9zPLfUUd9YDPC91ekZb9BSSkvOoWuDXF7+ct0FF+lN+JzEhbGOZT+6WxVN11TOlCOlB4uhcwWY7up5FEdqXs1PLOcPTW4Qdxs57OC/EJtDbqIuL71Smrgh9ljNn+MWZu05c2f2xmHN4M8xHRujxlmjbMkHA4u6sT96U08L+VrVeq9bfPq/svMxv0VV6RI/pCXr2Be3Qa9rFCeFgD5/pC301vhk/jJ/Grwy6vJTbrNPYY/z9Bx/ilWk=</latexit>

Transformer Layer

Vision Transformer

✱ …✱

✱

MLP

Reshape

Upsample

Flatten

LN

Classification Token

✱Image Tokens

Position Embedding CForward Concatenation ( )

… … …

zupL<latexit sha1_base64="qJscFy2HYZVHPyOSQ3ZG/FDhv54=">AAAIQXicpVXLbtNQEJ22QEp4pbBk4xJVYkFDUiHRZQUbFl0U1LSVmhLZzk1qxS/sa9oSZYfE17CFD+Ar+AO6Q2zZcGbsNM86rbDVdDz3zJk7r3ut0HViXa3+XFhcunHzVmH5dvHO3Xv3H5RWHu7FQRLZqm4HbhAdWGasXMdXde1oVx2EkTI9y1X7Vvc1r+9/VFHsBP6uPgvVkWd2fKft2KaGqllabVhe71P/fa+x2tDqVPtB5JluLwn7/SZ02/1is1SuVqryGNNCLRPKlD07wcrSZ2pQiwKyKSGPFPmkIbtkUoz3kGpUpRC6I+pBF0FyZF1Rn4qwTYBSQJjQdvHbwddhpvXxzZyxWNvw4uIvgqVBa7AJgIsgszdD1hNhZu1l3D3h5L2d4b+VcXnQajqGdp7dAHlVO45FU5s2JQYHMYWi4ejsjCWRrPDOjZGoNBhC6FhuYT2CbIvlIM+G2MQSO+fWlPVfgmQtf9sZNqHz3OgssHJFiv8VCft0pUonkFP8OiyMEfzl/LvoFuZuw94fY07fNeGMJU+b4LXAyllgO+PCKi/KY1Q9zSnnqD3CPfQwxLiy624uYyK4SZ5xxhg8HizTCtfpHW1LfVM/zKqlA3xw5lUowNuVvrCAmLX3ELsJEJeSDnckI1yRdfoAVlMiYr9GxpL2z7yeH2Z2lk9L5o+teIfchy1oB5NhXEwaT2+eL19OBq5K2smzfPGMe5mlnuJOe4DnpUbPaYOeQVJyHlVy/Hril/M2qEh/yu8kJoBtJPPJ3TJ/qq56puQhXUgcnSPYdEfXs8hDe3J2ajlneHrzsMPYeQ+nudgYegt1cfCdyNTloU8y5hQ/P3PXiSu9P+ZzDm+Gy5Eherwt2o5MELC4O2uTN+W0sLdRqVUrtbcvyluvslt0mR7TE3qKnn1JW/SGdnBC2PSFvtI3+l74UTgv/C78SaGLC5nNIxp7Cn//Acwvmtw=</latexit><latexit sha1_base64="qJscFy2HYZVHPyOSQ3ZG/FDhv54=">AAAIQXicpVXLbtNQEJ22QEp4pbBk4xJVYkFDUiHRZQUbFl0U1LSVmhLZzk1qxS/sa9oSZYfE17CFD+Ar+AO6Q2zZcGbsNM86rbDVdDz3zJk7r3ut0HViXa3+XFhcunHzVmH5dvHO3Xv3H5RWHu7FQRLZqm4HbhAdWGasXMdXde1oVx2EkTI9y1X7Vvc1r+9/VFHsBP6uPgvVkWd2fKft2KaGqllabVhe71P/fa+x2tDqVPtB5JluLwn7/SZ02/1is1SuVqryGNNCLRPKlD07wcrSZ2pQiwKyKSGPFPmkIbtkUoz3kGpUpRC6I+pBF0FyZF1Rn4qwTYBSQJjQdvHbwddhpvXxzZyxWNvw4uIvgqVBa7AJgIsgszdD1hNhZu1l3D3h5L2d4b+VcXnQajqGdp7dAHlVO45FU5s2JQYHMYWi4ejsjCWRrPDOjZGoNBhC6FhuYT2CbIvlIM+G2MQSO+fWlPVfgmQtf9sZNqHz3OgssHJFiv8VCft0pUonkFP8OiyMEfzl/LvoFuZuw94fY07fNeGMJU+b4LXAyllgO+PCKi/KY1Q9zSnnqD3CPfQwxLiy624uYyK4SZ5xxhg8HizTCtfpHW1LfVM/zKqlA3xw5lUowNuVvrCAmLX3ELsJEJeSDnckI1yRdfoAVlMiYr9GxpL2z7yeH2Z2lk9L5o+teIfchy1oB5NhXEwaT2+eL19OBq5K2smzfPGMe5mlnuJOe4DnpUbPaYOeQVJyHlVy/Hril/M2qEh/yu8kJoBtJPPJ3TJ/qq56puQhXUgcnSPYdEfXs8hDe3J2ajlneHrzsMPYeQ+nudgYegt1cfCdyNTloU8y5hQ/P3PXiSu9P+ZzDm+Gy5Eherwt2o5MELC4O2uTN+W0sLdRqVUrtbcvyluvslt0mR7TE3qKnn1JW/SGdnBC2PSFvtI3+l74UTgv/C78SaGLC5nNIxp7Cn//Acwvmtw=</latexit><latexit sha1_base64="qJscFy2HYZVHPyOSQ3ZG/FDhv54=">AAAIQXicpVXLbtNQEJ22QEp4pbBk4xJVYkFDUiHRZQUbFl0U1LSVmhLZzk1qxS/sa9oSZYfE17CFD+Ar+AO6Q2zZcGbsNM86rbDVdDz3zJk7r3ut0HViXa3+XFhcunHzVmH5dvHO3Xv3H5RWHu7FQRLZqm4HbhAdWGasXMdXde1oVx2EkTI9y1X7Vvc1r+9/VFHsBP6uPgvVkWd2fKft2KaGqllabVhe71P/fa+x2tDqVPtB5JluLwn7/SZ02/1is1SuVqryGNNCLRPKlD07wcrSZ2pQiwKyKSGPFPmkIbtkUoz3kGpUpRC6I+pBF0FyZF1Rn4qwTYBSQJjQdvHbwddhpvXxzZyxWNvw4uIvgqVBa7AJgIsgszdD1hNhZu1l3D3h5L2d4b+VcXnQajqGdp7dAHlVO45FU5s2JQYHMYWi4ejsjCWRrPDOjZGoNBhC6FhuYT2CbIvlIM+G2MQSO+fWlPVfgmQtf9sZNqHz3OgssHJFiv8VCft0pUonkFP8OiyMEfzl/LvoFuZuw94fY07fNeGMJU+b4LXAyllgO+PCKi/KY1Q9zSnnqD3CPfQwxLiy624uYyK4SZ5xxhg8HizTCtfpHW1LfVM/zKqlA3xw5lUowNuVvrCAmLX3ELsJEJeSDnckI1yRdfoAVlMiYr9GxpL2z7yeH2Z2lk9L5o+teIfchy1oB5NhXEwaT2+eL19OBq5K2smzfPGMe5mlnuJOe4DnpUbPaYOeQVJyHlVy/Hril/M2qEh/yu8kJoBtJPPJ3TJ/qq56puQhXUgcnSPYdEfXs8hDe3J2ajlneHrzsMPYeQ+nudgYegt1cfCdyNTloU8y5hQ/P3PXiSu9P+ZzDm+Gy5Eherwt2o5MELC4O2uTN+W0sLdRqVUrtbcvyluvslt0mR7TE3qKnn1JW/SGdnBC2PSFvtI3+l74UTgv/C78SaGLC5nNIxp7Cn//Acwvmtw=</latexit><latexit sha1_base64="qJscFy2HYZVHPyOSQ3ZG/FDhv54=">AAAIQXicpVXLbtNQEJ22QEp4pbBk4xJVYkFDUiHRZQUbFl0U1LSVmhLZzk1qxS/sa9oSZYfE17CFD+Ar+AO6Q2zZcGbsNM86rbDVdDz3zJk7r3ut0HViXa3+XFhcunHzVmH5dvHO3Xv3H5RWHu7FQRLZqm4HbhAdWGasXMdXde1oVx2EkTI9y1X7Vvc1r+9/VFHsBP6uPgvVkWd2fKft2KaGqllabVhe71P/fa+x2tDqVPtB5JluLwn7/SZ02/1is1SuVqryGNNCLRPKlD07wcrSZ2pQiwKyKSGPFPmkIbtkUoz3kGpUpRC6I+pBF0FyZF1Rn4qwTYBSQJjQdvHbwddhpvXxzZyxWNvw4uIvgqVBa7AJgIsgszdD1hNhZu1l3D3h5L2d4b+VcXnQajqGdp7dAHlVO45FU5s2JQYHMYWi4ejsjCWRrPDOjZGoNBhC6FhuYT2CbIvlIM+G2MQSO+fWlPVfgmQtf9sZNqHz3OgssHJFiv8VCft0pUonkFP8OiyMEfzl/LvoFuZuw94fY07fNeGMJU+b4LXAyllgO+PCKi/KY1Q9zSnnqD3CPfQwxLiy624uYyK4SZ5xxhg8HizTCtfpHW1LfVM/zKqlA3xw5lUowNuVvrCAmLX3ELsJEJeSDnckI1yRdfoAVlMiYr9GxpL2z7yeH2Z2lk9L5o+teIfchy1oB5NhXEwaT2+eL19OBq5K2smzfPGMe5mlnuJOe4DnpUbPaYOeQVJyHlVy/Hril/M2qEh/yu8kJoBtJPPJ3TJ/qq56puQhXUgcnSPYdEfXs8hDe3J2ajlneHrzsMPYeQ+nudgYegt1cfCdyNTloU8y5hQ/P3PXiSu9P+ZzDm+Gy5Eherwt2o5MELC4O2uTN+W0sLdRqVUrtbcvyluvslt0mR7TE3qKnn1JW/SGdnBC2PSFvtI3+l74UTgv/C78SaGLC5nNIxp7Cn//Acwvmtw=</latexit>

Figure 3: Illustration of the feature reuse mechanism. A layer-wise context embedding is learnedbased on the final representations output by the upstream model, i.e., zup

L , and integrated into theMLP block of each downstream Transformer layer.

connection [19] are applied before and after each block, respectively. Let zl ∈ RN×D denote theoutput of the lth Transformer layer, where N is the number of tokens for each sample, and D is thedimension of each token. Note that N = HW + 1, which corresponds to H×W patches of theoriginal image and a single learnable classification token. Formally, we have

z′l = MSA(LN(zl−1)) + zl−1, l ∈ {1, . . . , L}, (2)

zl = MLP(LN(z′l)) + z′l, l ∈ {1, . . . , L}, (3)where L is the total number of layers in the Transformer. The classification token in zL will be fedinto a LN layer followed by a fully-connected layer for the final prediction. For simplicity, here weomit the details on the position embedding, which is unrelated to our main idea. No modification isperformed on it in addition to the configurations of backbones.

Feature reuse. All the Transformers in DVT share the same goal of extracting discriminativerepresentations for accurate recognition. Therefore, it is straightforward that downstream modelsshould be learned on the basis of previously obtained deep features, rather than extracting featuresfrom scratch. The former is more efficient since the computation performed in an upstream modelcontributes to both itself and the successive models. To implement this idea, we propose a featurereuse mechanism (see: Figure 3). In specific, we leverage the image tokens output by the final layerof the upstream Transformer, i.e., zup

L , to learn a layer-wise embedding El for the downstream model:

El = fl(zupL ) ∈ RN×D

′. (4)

Herein, fl: RN×D→RN×D′consists of a sequence of operations starting with a LN-MLP (RD→RD′

),which introduces nonlinearity and allows more flexible transformations. Then the image tokens arereshaped to the corresponding locations in the original image, upsampled and flattened to match thetoken number of the downstream model. Typically, we use a small D′ for an efficient fl.

Consequently, the embedding El is injected into the downstream model, providing prior knowledgeon recognizing the input image. Formally, we replace Eq. (3) by:

zl = MLP(LN(Concat(z′l,El))) + z′l, l ∈ {1, . . . , L}, (5)where El is concatenated with the intermediate tokens z′l. We simply increase the dimension of LNand the first layer of MLP from D to D+D′. Since El is based on the upstream outputs zup

L that haveless tokens than z′l, it actually concludes the context information of the input image for each token inz′l. Therefore, we name El as the context embedding. Besides, we do not reuse the classification tokenand pad zero for it in Eq. (5), which we empirically find beneficial for the performance. Intuitively,Eqs. (4) and (5) allow training the downstream model to flexibly exploit the information within zup

Lon a per-layer basis, under the objective of minimizing the final recognition loss (Eq. (1)). Thisfeature reuse formulation can also be interpreted as implicitly enlarging the depth of the model.

5

Upstream Attention Logits

Nup<latexit sha1_base64="yhU64bt9VoWUvaXwaZuoU7F/ELs=">AAAIM3icpVXLbtNQEJ22QEp4NKVLNhZRJRY0xBUSXUZiwwKhgpq2UltFtnPTWvEL+5q2irLjP9jCB/AxiB1iAQv+gTNjp3k7rbDVdDz3zJk7r3vtyHMTXa9/X1peuXX7Tmn1bvne/QcP1yrrj/aTMI0d1XRCL4wPbStRnhuopna1pw6jWFm+7akDu/uK1w8+qjhxw2BPX0bqxLdOA7fjOpaGqlXZeNvqHWt1oYMw9i2vl0b9fqtSrdfq8hjTgpkLVcqf3XB95RMdU5tCciglnxQFpCF7ZFGC94hMqlME3Qn1oIshubKuqE9l2KZAKSAsaLv4PcXXUa4N8M2ciVg78OLhL4alQZuwCYGLIbM3Q9ZTYWbtPO6ecPLeLvHfzrl8aDWdQbvIboC8rh3HoqlDOxKDi5gi0XB0Ts6SSlZ458ZIVBoMEXQst7EeQ3bEcpBnQ2wSiZ1za8n6b0Gylr+dHJvSn8LobLByRcr/FQn79KRK55Az/BYsjBH8fP49dAtzd2AfjDFn76ZwJpKnHfDaYOUssJ1xZVUU5RmqnuWUc9QZ4R56GGI82XW3kDEV3CTPOGMCHh+WWYWb9J7eSH0zP8yqpQMCcBZVKMTblb6wgZi19wi7CRGXkg53JSNckS36AFZLImK/Rs6S9c+inh9mdpZPW+aPrXiH3IdtaAeTYVxNGk9vka9ATgauStbJs3zxjPu5pZ7iznqA58Wk57RNzyApOY9qBX598ct5G1SkP+V3EhPCNpb55G5ZPFXXPVOKkB4kjs4VbLajm1kUoX05O7WcMzy9Rdhh7LyHi0JsAr2Nurj4TmXqitDnOXOGX5y5m8SV3R+LOYc3w3xkhB7viPZUJghY3J3m5E05Lexv18x6zXz3otpo5LfoKj2mJ/QUPfuSGvSadnFCONjDZ/pCX0vfSj9KP0u/MujyUm6zQWNP6e8/hHOVWQ==</latexit><latexit sha1_base64="yhU64bt9VoWUvaXwaZuoU7F/ELs=">AAAIM3icpVXLbtNQEJ22QEp4NKVLNhZRJRY0xBUSXUZiwwKhgpq2UltFtnPTWvEL+5q2irLjP9jCB/AxiB1iAQv+gTNjp3k7rbDVdDz3zJk7r3vtyHMTXa9/X1peuXX7Tmn1bvne/QcP1yrrj/aTMI0d1XRCL4wPbStRnhuopna1pw6jWFm+7akDu/uK1w8+qjhxw2BPX0bqxLdOA7fjOpaGqlXZeNvqHWt1oYMw9i2vl0b9fqtSrdfq8hjTgpkLVcqf3XB95RMdU5tCciglnxQFpCF7ZFGC94hMqlME3Qn1oIshubKuqE9l2KZAKSAsaLv4PcXXUa4N8M2ciVg78OLhL4alQZuwCYGLIbM3Q9ZTYWbtPO6ecPLeLvHfzrl8aDWdQbvIboC8rh3HoqlDOxKDi5gi0XB0Ts6SSlZ458ZIVBoMEXQst7EeQ3bEcpBnQ2wSiZ1za8n6b0Gylr+dHJvSn8LobLByRcr/FQn79KRK55Az/BYsjBH8fP49dAtzd2AfjDFn76ZwJpKnHfDaYOUssJ1xZVUU5RmqnuWUc9QZ4R56GGI82XW3kDEV3CTPOGMCHh+WWYWb9J7eSH0zP8yqpQMCcBZVKMTblb6wgZi19wi7CRGXkg53JSNckS36AFZLImK/Rs6S9c+inh9mdpZPW+aPrXiH3IdtaAeTYVxNGk9vka9ATgauStbJs3zxjPu5pZ7iznqA58Wk57RNzyApOY9qBX598ct5G1SkP+V3EhPCNpb55G5ZPFXXPVOKkB4kjs4VbLajm1kUoX05O7WcMzy9Rdhh7LyHi0JsAr2Nurj4TmXqitDnOXOGX5y5m8SV3R+LOYc3w3xkhB7viPZUJghY3J3m5E05Lexv18x6zXz3otpo5LfoKj2mJ/QUPfuSGvSadnFCONjDZ/pCX0vfSj9KP0u/MujyUm6zQWNP6e8/hHOVWQ==</latexit><latexit sha1_base64="yhU64bt9VoWUvaXwaZuoU7F/ELs=">AAAIM3icpVXLbtNQEJ22QEp4NKVLNhZRJRY0xBUSXUZiwwKhgpq2UltFtnPTWvEL+5q2irLjP9jCB/AxiB1iAQv+gTNjp3k7rbDVdDz3zJk7r3vtyHMTXa9/X1peuXX7Tmn1bvne/QcP1yrrj/aTMI0d1XRCL4wPbStRnhuopna1pw6jWFm+7akDu/uK1w8+qjhxw2BPX0bqxLdOA7fjOpaGqlXZeNvqHWt1oYMw9i2vl0b9fqtSrdfq8hjTgpkLVcqf3XB95RMdU5tCciglnxQFpCF7ZFGC94hMqlME3Qn1oIshubKuqE9l2KZAKSAsaLv4PcXXUa4N8M2ciVg78OLhL4alQZuwCYGLIbM3Q9ZTYWbtPO6ecPLeLvHfzrl8aDWdQbvIboC8rh3HoqlDOxKDi5gi0XB0Ts6SSlZ458ZIVBoMEXQst7EeQ3bEcpBnQ2wSiZ1za8n6b0Gylr+dHJvSn8LobLByRcr/FQn79KRK55Az/BYsjBH8fP49dAtzd2AfjDFn76ZwJpKnHfDaYOUssJ1xZVUU5RmqnuWUc9QZ4R56GGI82XW3kDEV3CTPOGMCHh+WWYWb9J7eSH0zP8yqpQMCcBZVKMTblb6wgZi19wi7CRGXkg53JSNckS36AFZLImK/Rs6S9c+inh9mdpZPW+aPrXiH3IdtaAeTYVxNGk9vka9ATgauStbJs3zxjPu5pZ7iznqA58Wk57RNzyApOY9qBX598ct5G1SkP+V3EhPCNpb55G5ZPFXXPVOKkB4kjs4VbLajm1kUoX05O7WcMzy9Rdhh7LyHi0JsAr2Nurj4TmXqitDnOXOGX5y5m8SV3R+LOYc3w3xkhB7viPZUJghY3J3m5E05Lexv18x6zXz3otpo5LfoKj2mJ/QUPfuSGvSadnFCONjDZ/pCX0vfSj9KP0u/MujyUm6zQWNP6e8/hHOVWQ==</latexit><latexit sha1_base64="yhU64bt9VoWUvaXwaZuoU7F/ELs=">AAAIM3icpVXLbtNQEJ22QEp4NKVLNhZRJRY0xBUSXUZiwwKhgpq2UltFtnPTWvEL+5q2irLjP9jCB/AxiB1iAQv+gTNjp3k7rbDVdDz3zJk7r3vtyHMTXa9/X1peuXX7Tmn1bvne/QcP1yrrj/aTMI0d1XRCL4wPbStRnhuopna1pw6jWFm+7akDu/uK1w8+qjhxw2BPX0bqxLdOA7fjOpaGqlXZeNvqHWt1oYMw9i2vl0b9fqtSrdfq8hjTgpkLVcqf3XB95RMdU5tCciglnxQFpCF7ZFGC94hMqlME3Qn1oIshubKuqE9l2KZAKSAsaLv4PcXXUa4N8M2ciVg78OLhL4alQZuwCYGLIbM3Q9ZTYWbtPO6ecPLeLvHfzrl8aDWdQbvIboC8rh3HoqlDOxKDi5gi0XB0Ts6SSlZ458ZIVBoMEXQst7EeQ3bEcpBnQ2wSiZ1za8n6b0Gylr+dHJvSn8LobLByRcr/FQn79KRK55Az/BYsjBH8fP49dAtzd2AfjDFn76ZwJpKnHfDaYOUssJ1xZVUU5RmqnuWUc9QZ4R56GGI82XW3kDEV3CTPOGMCHh+WWYWb9J7eSH0zP8yqpQMCcBZVKMTblb6wgZi19wi7CRGXkg53JSNckS36AFZLImK/Rs6S9c+inh9mdpZPW+aPrXiH3IdtaAeTYVxNGk9vka9ATgauStbJs3zxjPu5pZ7iznqA58Wk57RNzyApOY9qBX598ct5G1SkP+V3EhPCNpb55G5ZPFXXPVOKkB4kjs4VbLajm1kUoX05O7WcMzy9Rdhh7LyHi0JsAr2Nurj4TmXqitDnOXOGX5y5m8SV3R+LOYc3w3xkhB7viPZUJghY3J3m5E05Lexv18x6zXz3otpo5LfoKj2mJ/QUPfuSGvSadnFCONjDZ/pCX0vfSj9KP0u/MujyUm6zQWNP6e8/hHOVWQ==</latexit>

Nup<latexit sha1_base64="yhU64bt9VoWUvaXwaZuoU7F/ELs=">AAAIM3icpVXLbtNQEJ22QEp4NKVLNhZRJRY0xBUSXUZiwwKhgpq2UltFtnPTWvEL+5q2irLjP9jCB/AxiB1iAQv+gTNjp3k7rbDVdDz3zJk7r3vtyHMTXa9/X1peuXX7Tmn1bvne/QcP1yrrj/aTMI0d1XRCL4wPbStRnhuopna1pw6jWFm+7akDu/uK1w8+qjhxw2BPX0bqxLdOA7fjOpaGqlXZeNvqHWt1oYMw9i2vl0b9fqtSrdfq8hjTgpkLVcqf3XB95RMdU5tCciglnxQFpCF7ZFGC94hMqlME3Qn1oIshubKuqE9l2KZAKSAsaLv4PcXXUa4N8M2ciVg78OLhL4alQZuwCYGLIbM3Q9ZTYWbtPO6ecPLeLvHfzrl8aDWdQbvIboC8rh3HoqlDOxKDi5gi0XB0Ts6SSlZ458ZIVBoMEXQst7EeQ3bEcpBnQ2wSiZ1za8n6b0Gylr+dHJvSn8LobLByRcr/FQn79KRK55Az/BYsjBH8fP49dAtzd2AfjDFn76ZwJpKnHfDaYOUssJ1xZVUU5RmqnuWUc9QZ4R56GGI82XW3kDEV3CTPOGMCHh+WWYWb9J7eSH0zP8yqpQMCcBZVKMTblb6wgZi19wi7CRGXkg53JSNckS36AFZLImK/Rs6S9c+inh9mdpZPW+aPrXiH3IdtaAeTYVxNGk9vka9ATgauStbJs3zxjPu5pZ7iznqA58Wk57RNzyApOY9qBX598ct5G1SkP+V3EhPCNpb55G5ZPFXXPVOKkB4kjs4VbLajm1kUoX05O7WcMzy9Rdhh7LyHi0JsAr2Nurj4TmXqitDnOXOGX5y5m8SV3R+LOYc3w3xkhB7viPZUJghY3J3m5E05Lexv18x6zXz3otpo5LfoKj2mJ/QUPfuSGvSadnFCONjDZ/pCX0vfSj9KP0u/MujyUm6zQWNP6e8/hHOVWQ==</latexit><latexit sha1_base64="yhU64bt9VoWUvaXwaZuoU7F/ELs=">AAAIM3icpVXLbtNQEJ22QEp4NKVLNhZRJRY0xBUSXUZiwwKhgpq2UltFtnPTWvEL+5q2irLjP9jCB/AxiB1iAQv+gTNjp3k7rbDVdDz3zJk7r3vtyHMTXa9/X1peuXX7Tmn1bvne/QcP1yrrj/aTMI0d1XRCL4wPbStRnhuopna1pw6jWFm+7akDu/uK1w8+qjhxw2BPX0bqxLdOA7fjOpaGqlXZeNvqHWt1oYMw9i2vl0b9fqtSrdfq8hjTgpkLVcqf3XB95RMdU5tCciglnxQFpCF7ZFGC94hMqlME3Qn1oIshubKuqE9l2KZAKSAsaLv4PcXXUa4N8M2ciVg78OLhL4alQZuwCYGLIbM3Q9ZTYWbtPO6ecPLeLvHfzrl8aDWdQbvIboC8rh3HoqlDOxKDi5gi0XB0Ts6SSlZ458ZIVBoMEXQst7EeQ3bEcpBnQ2wSiZ1za8n6b0Gylr+dHJvSn8LobLByRcr/FQn79KRK55Az/BYsjBH8fP49dAtzd2AfjDFn76ZwJpKnHfDaYOUssJ1xZVUU5RmqnuWUc9QZ4R56GGI82XW3kDEV3CTPOGMCHh+WWYWb9J7eSH0zP8yqpQMCcBZVKMTblb6wgZi19wi7CRGXkg53JSNckS36AFZLImK/Rs6S9c+inh9mdpZPW+aPrXiH3IdtaAeTYVxNGk9vka9ATgauStbJs3zxjPu5pZ7iznqA58Wk57RNzyApOY9qBX598ct5G1SkP+V3EhPCNpb55G5ZPFXXPVOKkB4kjs4VbLajm1kUoX05O7WcMzy9Rdhh7LyHi0JsAr2Nurj4TmXqitDnOXOGX5y5m8SV3R+LOYc3w3xkhB7viPZUJghY3J3m5E05Lexv18x6zXz3otpo5LfoKj2mJ/QUPfuSGvSadnFCONjDZ/pCX0vfSj9KP0u/MujyUm6zQWNP6e8/hHOVWQ==</latexit><latexit sha1_base64="yhU64bt9VoWUvaXwaZuoU7F/ELs=">AAAIM3icpVXLbtNQEJ22QEp4NKVLNhZRJRY0xBUSXUZiwwKhgpq2UltFtnPTWvEL+5q2irLjP9jCB/AxiB1iAQv+gTNjp3k7rbDVdDz3zJk7r3vtyHMTXa9/X1peuXX7Tmn1bvne/QcP1yrrj/aTMI0d1XRCL4wPbStRnhuopna1pw6jWFm+7akDu/uK1w8+qjhxw2BPX0bqxLdOA7fjOpaGqlXZeNvqHWt1oYMw9i2vl0b9fqtSrdfq8hjTgpkLVcqf3XB95RMdU5tCciglnxQFpCF7ZFGC94hMqlME3Qn1oIshubKuqE9l2KZAKSAsaLv4PcXXUa4N8M2ciVg78OLhL4alQZuwCYGLIbM3Q9ZTYWbtPO6ecPLeLvHfzrl8aDWdQbvIboC8rh3HoqlDOxKDi5gi0XB0Ts6SSlZ458ZIVBoMEXQst7EeQ3bEcpBnQ2wSiZ1za8n6b0Gylr+dHJvSn8LobLByRcr/FQn79KRK55Az/BYsjBH8fP49dAtzd2AfjDFn76ZwJpKnHfDaYOUssJ1xZVUU5RmqnuWUc9QZ4R56GGI82XW3kDEV3CTPOGMCHh+WWYWb9J7eSH0zP8yqpQMCcBZVKMTblb6wgZi19wi7CRGXkg53JSNckS36AFZLImK/Rs6S9c+inh9mdpZPW+aPrXiH3IdtaAeTYVxNGk9vka9ATgauStbJs3zxjPu5pZ7iznqA58Wk57RNzyApOY9qBX598ct5G1SkP+V3EhPCNpb55G5ZPFXXPVOKkB4kjs4VbLajm1kUoX05O7WcMzy9Rdhh7LyHi0JsAr2Nurj4TmXqitDnOXOGX5y5m8SV3R+LOYc3w3xkhB7viPZUJghY3J3m5E05Lexv18x6zXz3otpo5LfoKj2mJ/QUPfuSGvSadnFCONjDZ/pCX0vfSj9KP0u/MujyUm6zQWNP6e8/hHOVWQ==</latexit><latexit sha1_base64="yhU64bt9VoWUvaXwaZuoU7F/ELs=">AAAIM3icpVXLbtNQEJ22QEp4NKVLNhZRJRY0xBUSXUZiwwKhgpq2UltFtnPTWvEL+5q2irLjP9jCB/AxiB1iAQv+gTNjp3k7rbDVdDz3zJk7r3vtyHMTXa9/X1peuXX7Tmn1bvne/QcP1yrrj/aTMI0d1XRCL4wPbStRnhuopna1pw6jWFm+7akDu/uK1w8+qjhxw2BPX0bqxLdOA7fjOpaGqlXZeNvqHWt1oYMw9i2vl0b9fqtSrdfq8hjTgpkLVcqf3XB95RMdU5tCciglnxQFpCF7ZFGC94hMqlME3Qn1oIshubKuqE9l2KZAKSAsaLv4PcXXUa4N8M2ciVg78OLhL4alQZuwCYGLIbM3Q9ZTYWbtPO6ecPLeLvHfzrl8aDWdQbvIboC8rh3HoqlDOxKDi5gi0XB0Ts6SSlZ458ZIVBoMEXQst7EeQ3bEcpBnQ2wSiZ1za8n6b0Gylr+dHJvSn8LobLByRcr/FQn79KRK55Az/BYsjBH8fP49dAtzd2AfjDFn76ZwJpKnHfDaYOUssJ1xZVUU5RmqnuWUc9QZ4R56GGI82XW3kDEV3CTPOGMCHh+WWYWb9J7eSH0zP8yqpQMCcBZVKMTblb6wgZi19wi7CRGXkg53JSNckS36AFZLImK/Rs6S9c+inh9mdpZPW+aPrXiH3IdtaAeTYVxNGk9vka9ATgauStbJs3zxjPu5pZ7iznqA58Wk57RNzyApOY9qBX598ct5G1SkP+V3EhPCNpb55G5ZPFXXPVOKkB4kjs4VbLajm1kUoX05O7WcMzy9Rdhh7LyHi0JsAr2Nurj4TmXqitDnOXOGX5y5m8SV3R+LOYc3w3xkhB7viPZUJghY3J3m5E05Lexv18x6zXz3otpo5LfoKj2mJ/QUPfuSGvSadnFCONjDZ/pCX0vfSj9KP0u/MujyUm6zQWNP6e8/hHOVWQ==</latexit>

NAttup

<latexit sha1_base64="ILMj6mzJ5SLDORmvDY9TR/7Bw8A=">AAAIRnicpVXLbtNQEJ22QEJ4NIUlG4uoiAUNSYVEl0FsWCBUUNNWaktlOzepFb+wr2mrKDuWfA1b+AB+gT+ADTvEljNjp3k7rbDVdDz3zJk7r3ut0HViXav9WFpeuXb9RqF4s3Tr9p27q+W1e7txkES2atqBG0T7lhkr1/FVUzvaVfthpEzPctWe1X3J63sfVRQ7gb+jz0N15Jkd32k7tqmhOi4/evO+d6jVmfaDyDPd3gut+/3jMVUSQlOu1Ko1eYxpoZ4JFcqe7WBt5RMdUosCsikhjxT5pCG7ZFKM94DqVKMQuiPqQRdBcmRdUZ9KsE2AUkCY0Hbx28HXQab18c2csVjb8OLiL4KlQeuwCYCLILM3Q9YTYWbtPO6ecPLezvHfyrg8aDWdQLvIboC8rB3HoqlNWxKDg5hC0XB0dsaSSFZ458ZIVBoMIXQst7AeQbbFcpBnQ2xiiZ1za8r6T0Gylr/tDJvQr9zoLLByRUr/FQn7dKVKp5BT/AYsjBH8fP4ddAtzt2HvjzGn77pwxpKnLfBaYOUssJ1xYZUX5QmqnuaUc9Qe4R56GGJc2XU3lzER3CTPOGMMHg+WaYWb9I5eS31TP8yqpQN8cOZVKMDblb6wgJi19xC7CRCXkg53JCNckQ36AFZTImK/RsaS9s+inh9mdpZPS+aPrXiH3IctaAeTYVxMGk9vni9fTgauStrJs3zxjHuZpZ7iTnuA56VOT2mTnkBSch5Vc/x64pfzNqhIf8rvJCaAbSTzyd2yeKoue6bkIV1IHJ0j2HRHV7PIQ3tydmo5Z3h687DD2HkPZ7nYGHoLdXHwncjU5aFPM+YUvzhzV4krvT8Wcw5vhvnIED3eFm1HJghY3J31yZtyWtjdrNZr1frbZ5VGI7tFi/SAHtJj9OxzatAr2sYJYdNn+kJf6Vvhe+F34U/hbwpdXsps7tPYU6R/eB+dOQ==</latexit><latexit sha1_base64="ILMj6mzJ5SLDORmvDY9TR/7Bw8A=">AAAIRnicpVXLbtNQEJ22QEJ4NIUlG4uoiAUNSYVEl0FsWCBUUNNWaktlOzepFb+wr2mrKDuWfA1b+AB+gT+ADTvEljNjp3k7rbDVdDz3zJk7r3ut0HViXav9WFpeuXb9RqF4s3Tr9p27q+W1e7txkES2atqBG0T7lhkr1/FVUzvaVfthpEzPctWe1X3J63sfVRQ7gb+jz0N15Jkd32k7tqmhOi4/evO+d6jVmfaDyDPd3gut+/3jMVUSQlOu1Ko1eYxpoZ4JFcqe7WBt5RMdUosCsikhjxT5pCG7ZFKM94DqVKMQuiPqQRdBcmRdUZ9KsE2AUkCY0Hbx28HXQab18c2csVjb8OLiL4KlQeuwCYCLILM3Q9YTYWbtPO6ecPLezvHfyrg8aDWdQLvIboC8rB3HoqlNWxKDg5hC0XB0dsaSSFZ458ZIVBoMIXQst7AeQbbFcpBnQ2xiiZ1za8r6T0Gylr/tDJvQr9zoLLByRUr/FQn7dKVKp5BT/AYsjBH8fP4ddAtzt2HvjzGn77pwxpKnLfBaYOUssJ1xYZUX5QmqnuaUc9Qe4R56GGJc2XU3lzER3CTPOGMMHg+WaYWb9I5eS31TP8yqpQN8cOZVKMDblb6wgJi19xC7CRCXkg53JCNckQ36AFZTImK/RsaS9s+inh9mdpZPS+aPrXiH3IctaAeTYVxMGk9vni9fTgauStrJs3zxjHuZpZ7iTnuA56VOT2mTnkBSch5Vc/x64pfzNqhIf8rvJCaAbSTzyd2yeKoue6bkIV1IHJ0j2HRHV7PIQ3tydmo5Z3h687DD2HkPZ7nYGHoLdXHwncjU5aFPM+YUvzhzV4krvT8Wcw5vhvnIED3eFm1HJghY3J31yZtyWtjdrNZr1frbZ5VGI7tFi/SAHtJj9OxzatAr2sYJYdNn+kJf6Vvhe+F34U/hbwpdXsps7tPYU6R/eB+dOQ==</latexit><latexit sha1_base64="ILMj6mzJ5SLDORmvDY9TR/7Bw8A=">AAAIRnicpVXLbtNQEJ22QEJ4NIUlG4uoiAUNSYVEl0FsWCBUUNNWaktlOzepFb+wr2mrKDuWfA1b+AB+gT+ADTvEljNjp3k7rbDVdDz3zJk7r3ut0HViXav9WFpeuXb9RqF4s3Tr9p27q+W1e7txkES2atqBG0T7lhkr1/FVUzvaVfthpEzPctWe1X3J63sfVRQ7gb+jz0N15Jkd32k7tqmhOi4/evO+d6jVmfaDyDPd3gut+/3jMVUSQlOu1Ko1eYxpoZ4JFcqe7WBt5RMdUosCsikhjxT5pCG7ZFKM94DqVKMQuiPqQRdBcmRdUZ9KsE2AUkCY0Hbx28HXQab18c2csVjb8OLiL4KlQeuwCYCLILM3Q9YTYWbtPO6ecPLezvHfyrg8aDWdQLvIboC8rB3HoqlNWxKDg5hC0XB0dsaSSFZ458ZIVBoMIXQst7AeQbbFcpBnQ2xiiZ1za8r6T0Gylr/tDJvQr9zoLLByRUr/FQn7dKVKp5BT/AYsjBH8fP4ddAtzt2HvjzGn77pwxpKnLfBaYOUssJ1xYZUX5QmqnuaUc9Qe4R56GGJc2XU3lzER3CTPOGMMHg+WaYWb9I5eS31TP8yqpQN8cOZVKMDblb6wgJi19xC7CRCXkg53JCNckQ36AFZTImK/RsaS9s+inh9mdpZPS+aPrXiH3IctaAeTYVxMGk9vni9fTgauStrJs3zxjHuZpZ7iTnuA56VOT2mTnkBSch5Vc/x64pfzNqhIf8rvJCaAbSTzyd2yeKoue6bkIV1IHJ0j2HRHV7PIQ3tydmo5Z3h687DD2HkPZ7nYGHoLdXHwncjU5aFPM+YUvzhzV4krvT8Wcw5vhvnIED3eFm1HJghY3J31yZtyWtjdrNZr1frbZ5VGI7tFi/SAHtJj9OxzatAr2sYJYdNn+kJf6Vvhe+F34U/hbwpdXsps7tPYU6R/eB+dOQ==</latexit><latexit sha1_base64="ILMj6mzJ5SLDORmvDY9TR/7Bw8A=">AAAIRnicpVXLbtNQEJ22QEJ4NIUlG4uoiAUNSYVEl0FsWCBUUNNWaktlOzepFb+wr2mrKDuWfA1b+AB+gT+ADTvEljNjp3k7rbDVdDz3zJk7r3ut0HViXav9WFpeuXb9RqF4s3Tr9p27q+W1e7txkES2atqBG0T7lhkr1/FVUzvaVfthpEzPctWe1X3J63sfVRQ7gb+jz0N15Jkd32k7tqmhOi4/evO+d6jVmfaDyDPd3gut+/3jMVUSQlOu1Ko1eYxpoZ4JFcqe7WBt5RMdUosCsikhjxT5pCG7ZFKM94DqVKMQuiPqQRdBcmRdUZ9KsE2AUkCY0Hbx28HXQab18c2csVjb8OLiL4KlQeuwCYCLILM3Q9YTYWbtPO6ecPLezvHfyrg8aDWdQLvIboC8rB3HoqlNWxKDg5hC0XB0dsaSSFZ458ZIVBoMIXQst7AeQbbFcpBnQ2xiiZ1za8r6T0Gylr/tDJvQr9zoLLByRUr/FQn7dKVKp5BT/AYsjBH8fP4ddAtzt2HvjzGn77pwxpKnLfBaYOUssJ1xYZUX5QmqnuaUc9Qe4R56GGJc2XU3lzER3CTPOGMMHg+WaYWb9I5eS31TP8yqpQN8cOZVKMDblb6wgJi19xC7CRCXkg53JCNckQ36AFZTImK/RsaS9s+inh9mdpZPS+aPrXiH3IctaAeTYVxMGk9vni9fTgauStrJs3zxjHuZpZ7iTnuA56VOT2mTnkBSch5Vc/x64pfzNqhIf8rvJCaAbSTzyd2yeKoue6bkIV1IHJ0j2HRHV7PIQ3tydmo5Z3h687DD2HkPZ7nYGHoLdXHwncjU5aFPM+YUvzhzV4krvT8Wcw5vhvnIED3eFm1HJghY3J31yZtyWtjdrNZr1frbZ5VGI7tFi/SAHtJj9OxzatAr2sYJYdNn+kJf6Vvhe+F34U/hbwpdXsps7tPYU6R/eB+dOQ==</latexit>

Flatten

NAttup

<latexit sha1_base64="ILMj6mzJ5SLDORmvDY9TR/7Bw8A=">AAAIRnicpVXLbtNQEJ22QEJ4NIUlG4uoiAUNSYVEl0FsWCBUUNNWaktlOzepFb+wr2mrKDuWfA1b+AB+gT+ADTvEljNjp3k7rbDVdDz3zJk7r3ut0HViXav9WFpeuXb9RqF4s3Tr9p27q+W1e7txkES2atqBG0T7lhkr1/FVUzvaVfthpEzPctWe1X3J63sfVRQ7gb+jz0N15Jkd32k7tqmhOi4/evO+d6jVmfaDyDPd3gut+/3jMVUSQlOu1Ko1eYxpoZ4JFcqe7WBt5RMdUosCsikhjxT5pCG7ZFKM94DqVKMQuiPqQRdBcmRdUZ9KsE2AUkCY0Hbx28HXQab18c2csVjb8OLiL4KlQeuwCYCLILM3Q9YTYWbtPO6ecPLezvHfyrg8aDWdQLvIboC8rB3HoqlNWxKDg5hC0XB0dsaSSFZ458ZIVBoMIXQst7AeQbbFcpBnQ2xiiZ1za8r6T0Gylr/tDJvQr9zoLLByRUr/FQn7dKVKp5BT/AYsjBH8fP4ddAtzt2HvjzGn77pwxpKnLfBaYOUssJ1xYZUX5QmqnuaUc9Qe4R56GGJc2XU3lzER3CTPOGMMHg+WaYWb9I5eS31TP8yqpQN8cOZVKMDblb6wgJi19xC7CRCXkg53JCNckQ36AFZTImK/RsaS9s+inh9mdpZPS+aPrXiH3IctaAeTYVxMGk9vni9fTgauStrJs3zxjHuZpZ7iTnuA56VOT2mTnkBSch5Vc/x64pfzNqhIf8rvJCaAbSTzyd2yeKoue6bkIV1IHJ0j2HRHV7PIQ3tydmo5Z3h687DD2HkPZ7nYGHoLdXHwncjU5aFPM+YUvzhzV4krvT8Wcw5vhvnIED3eFm1HJghY3J31yZtyWtjdrNZr1frbZ5VGI7tFi/SAHtJj9OxzatAr2sYJYdNn+kJf6Vvhe+F34U/hbwpdXsps7tPYU6R/eB+dOQ==</latexit><latexit sha1_base64="ILMj6mzJ5SLDORmvDY9TR/7Bw8A=">AAAIRnicpVXLbtNQEJ22QEJ4NIUlG4uoiAUNSYVEl0FsWCBUUNNWaktlOzepFb+wr2mrKDuWfA1b+AB+gT+ADTvEljNjp3k7rbDVdDz3zJk7r3ut0HViXav9WFpeuXb9RqF4s3Tr9p27q+W1e7txkES2atqBG0T7lhkr1/FVUzvaVfthpEzPctWe1X3J63sfVRQ7gb+jz0N15Jkd32k7tqmhOi4/evO+d6jVmfaDyDPd3gut+/3jMVUSQlOu1Ko1eYxpoZ4JFcqe7WBt5RMdUosCsikhjxT5pCG7ZFKM94DqVKMQuiPqQRdBcmRdUZ9KsE2AUkCY0Hbx28HXQab18c2csVjb8OLiL4KlQeuwCYCLILM3Q9YTYWbtPO6ecPLezvHfyrg8aDWdQLvIboC8rB3HoqlNWxKDg5hC0XB0dsaSSFZ458ZIVBoMIXQst7AeQbbFcpBnQ2xiiZ1za8r6T0Gylr/tDJvQr9zoLLByRUr/FQn7dKVKp5BT/AYsjBH8fP4ddAtzt2HvjzGn77pwxpKnLfBaYOUssJ1xYZUX5QmqnuaUc9Qe4R56GGJc2XU3lzER3CTPOGMMHg+WaYWb9I5eS31TP8yqpQN8cOZVKMDblb6wgJi19xC7CRCXkg53JCNckQ36AFZTImK/RsaS9s+inh9mdpZPS+aPrXiH3IctaAeTYVxMGk9vni9fTgauStrJs3zxjHuZpZ7iTnuA56VOT2mTnkBSch5Vc/x64pfzNqhIf8rvJCaAbSTzyd2yeKoue6bkIV1IHJ0j2HRHV7PIQ3tydmo5Z3h687DD2HkPZ7nYGHoLdXHwncjU5aFPM+YUvzhzV4krvT8Wcw5vhvnIED3eFm1HJghY3J31yZtyWtjdrNZr1frbZ5VGI7tFi/SAHtJj9OxzatAr2sYJYdNn+kJf6Vvhe+F34U/hbwpdXsps7tPYU6R/eB+dOQ==</latexit><latexit sha1_base64="ILMj6mzJ5SLDORmvDY9TR/7Bw8A=">AAAIRnicpVXLbtNQEJ22QEJ4NIUlG4uoiAUNSYVEl0FsWCBUUNNWaktlOzepFb+wr2mrKDuWfA1b+AB+gT+ADTvEljNjp3k7rbDVdDz3zJk7r3ut0HViXav9WFpeuXb9RqF4s3Tr9p27q+W1e7txkES2atqBG0T7lhkr1/FVUzvaVfthpEzPctWe1X3J63sfVRQ7gb+jz0N15Jkd32k7tqmhOi4/evO+d6jVmfaDyDPd3gut+/3jMVUSQlOu1Ko1eYxpoZ4JFcqe7WBt5RMdUosCsikhjxT5pCG7ZFKM94DqVKMQuiPqQRdBcmRdUZ9KsE2AUkCY0Hbx28HXQab18c2csVjb8OLiL4KlQeuwCYCLILM3Q9YTYWbtPO6ecPLezvHfyrg8aDWdQLvIboC8rB3HoqlNWxKDg5hC0XB0dsaSSFZ458ZIVBoMIXQst7AeQbbFcpBnQ2xiiZ1za8r6T0Gylr/tDJvQr9zoLLByRUr/FQn7dKVKp5BT/AYsjBH8fP4ddAtzt2HvjzGn77pwxpKnLfBaYOUssJ1xYZUX5QmqnuaUc9Qe4R56GGJc2XU3lzER3CTPOGMMHg+WaYWb9I5eS31TP8yqpQN8cOZVKMDblb6wgJi19xC7CRCXkg53JCNckQ36AFZTImK/RsaS9s+inh9mdpZPS+aPrXiH3IctaAeTYVxMGk9vni9fTgauStrJs3zxjHuZpZ7iTnuA56VOT2mTnkBSch5Vc/x64pfzNqhIf8rvJCaAbSTzyd2yeKoue6bkIV1IHJ0j2HRHV7PIQ3tydmo5Z3h687DD2HkPZ7nYGHoLdXHwncjU5aFPM+YUvzhzV4krvT8Wcw5vhvnIED3eFm1HJghY3J31yZtyWtjdrNZr1frbZ5VGI7tFi/SAHtJj9OxzatAr2sYJYdNn+kJf6Vvhe+F34U/hbwpdXsps7tPYU6R/eB+dOQ==</latexit><latexit sha1_base64="ILMj6mzJ5SLDORmvDY9TR/7Bw8A=">AAAIRnicpVXLbtNQEJ22QEJ4NIUlG4uoiAUNSYVEl0FsWCBUUNNWaktlOzepFb+wr2mrKDuWfA1b+AB+gT+ADTvEljNjp3k7rbDVdDz3zJk7r3ut0HViXav9WFpeuXb9RqF4s3Tr9p27q+W1e7txkES2atqBG0T7lhkr1/FVUzvaVfthpEzPctWe1X3J63sfVRQ7gb+jz0N15Jkd32k7tqmhOi4/evO+d6jVmfaDyDPd3gut+/3jMVUSQlOu1Ko1eYxpoZ4JFcqe7WBt5RMdUosCsikhjxT5pCG7ZFKM94DqVKMQuiPqQRdBcmRdUZ9KsE2AUkCY0Hbx28HXQab18c2csVjb8OLiL4KlQeuwCYCLILM3Q9YTYWbtPO6ecPLezvHfyrg8aDWdQLvIboC8rB3HoqlNWxKDg5hC0XB0dsaSSFZ458ZIVBoMIXQst7AeQbbFcpBnQ2xiiZ1za8r6T0Gylr/tDJvQr9zoLLByRUr/FQn7dKVKp5BT/AYsjBH8fP4ddAtzt2HvjzGn77pwxpKnLfBaYOUssJ1xYZUX5QmqnuaUc9Qe4R56GGJc2XU3lzER3CTPOGMMHg+WaYWb9I5eS31TP8yqpQN8cOZVKMDblb6wgJi19xC7CRCXkg53JCNckQ36AFZTImK/RsaS9s+inh9mdpZPS+aPrXiH3IctaAeTYVxMGk9vni9fTgauStrJs3zxjHuZpZ7iTnuA56VOT2mTnkBSch5Vc/x64pfzNqhIf8rvJCaAbSTzyd2yeKoue6bkIV1IHJ0j2HRHV7PIQ3tydmo5Z3h687DD2HkPZ7nYGHoLdXHwncjU5aFPM+YUvzhzV4krvT8Wcw5vhvnIED3eFm1HJghY3J31yZtyWtjdrNZr1frbZ5VGI7tFi/SAHtJj9OxzatAr2sYJYdNn+kJf6Vvhe+F34U/hbwpdXsps7tPYU6R/eB+dOQ==</latexit> ……

…

8 > < > : <latexit sha1_base64="O/VfR3tx0Oo9X3tsrLCNOjnJhnU=">AAAIV3icpVXNThNRFD6AQq1/oEs3EwnGhdSWmMCS6MaFCzT8JQwhM9PbctP5c+aOSJrufA2fxq0+AG+gT6HfOXdKoYUpxJkUzpz7ne/c83evn4Y6N83m2czs3J278wu1e/X7Dx4+ery49GQ3T4osUDtBEibZvu/lKtSx2jHahGo/zZQX+aHa83vveH3vi8pyncTb5jRVh5HXjXVHB56B6mhx3Q1Vx7j9uuurro77Xqi7sWoP6i9c1/7qrorbI72b6e6xaRwtLjcbTXmcSaFVCstUPlvJ0tw3cqlNCQVUUESKYjKQQ/Iox3tALWpSCt0h9aHLIGlZVzSgOmwLoBQQHrQ9/O3i66DUxvhmzlysA3gJ8ctg6dAKbBLgMsjszZH1QphZex13Xzh5b6f475dcEbSGjqGdZjdE3tSOYzHUoQ2JQSOmVDQcXVCyFJIV3rlzISoDhhQ6lttYzyAHYjnMsyM2ucTOufVk/bcgWcvfQYkt6E9ldD5YuSL1/4qEfYZSpRPIFr8KC+cC/nr+bXQLc3dgH19itu+KcOaSpw3w+mDlLLCdc25VFeUxqm5zyjnqXOAeeRhhQtl1r5KxENw4z2XGHDwRLG2Fd+gTfZD6Wj/MaqQDYnBWVSjB25O+8IG4au8pdpMgLiUdriUjXJFV+gxWTyJiv07JYvtnWs+PMnuVT1/mj614h9yHbWiHk+GcTxpPb5WvWE4Grort5Kt88YxHpaWZ4LY9wPPSote0Rq8gKTmPGhV+I/HLeRtWZDDhdxyTwDaT+eRumT5VNz1TqpAhJI5OC9bu6HYWVehIzk4j5wxPbxV2FDvv4WslNofeR100vguZuir0Scls8dMzd5u47P0xnXN0M1yPTNHjHdF2ZYKAxd3ZGr8pJ4XdtUar2Wh9fLO8+ba8RWv0jJ7TS/TsOm3Se9rCCRHQd/pBP+nXwtnC39p8rWahszOlzVO69NSW/gFRz57K</latexit><latexit sha1_base64="O/VfR3tx0Oo9X3tsrLCNOjnJhnU=">AAAIV3icpVXNThNRFD6AQq1/oEs3EwnGhdSWmMCS6MaFCzT8JQwhM9PbctP5c+aOSJrufA2fxq0+AG+gT6HfOXdKoYUpxJkUzpz7ne/c83evn4Y6N83m2czs3J278wu1e/X7Dx4+ery49GQ3T4osUDtBEibZvu/lKtSx2jHahGo/zZQX+aHa83vveH3vi8pyncTb5jRVh5HXjXVHB56B6mhx3Q1Vx7j9uuurro77Xqi7sWoP6i9c1/7qrorbI72b6e6xaRwtLjcbTXmcSaFVCstUPlvJ0tw3cqlNCQVUUESKYjKQQ/Iox3tALWpSCt0h9aHLIGlZVzSgOmwLoBQQHrQ9/O3i66DUxvhmzlysA3gJ8ctg6dAKbBLgMsjszZH1QphZex13Xzh5b6f475dcEbSGjqGdZjdE3tSOYzHUoQ2JQSOmVDQcXVCyFJIV3rlzISoDhhQ6lttYzyAHYjnMsyM2ucTOufVk/bcgWcvfQYkt6E9ldD5YuSL1/4qEfYZSpRPIFr8KC+cC/nr+bXQLc3dgH19itu+KcOaSpw3w+mDlLLCdc25VFeUxqm5zyjnqXOAeeRhhQtl1r5KxENw4z2XGHDwRLG2Fd+gTfZD6Wj/MaqQDYnBWVSjB25O+8IG4au8pdpMgLiUdriUjXJFV+gxWTyJiv07JYvtnWs+PMnuVT1/mj614h9yHbWiHk+GcTxpPb5WvWE4Grort5Kt88YxHpaWZ4LY9wPPSote0Rq8gKTmPGhV+I/HLeRtWZDDhdxyTwDaT+eRumT5VNz1TqpAhJI5OC9bu6HYWVehIzk4j5wxPbxV2FDvv4WslNofeR100vguZuir0Scls8dMzd5u47P0xnXN0M1yPTNHjHdF2ZYKAxd3ZGr8pJ4XdtUar2Wh9fLO8+ba8RWv0jJ7TS/TsOm3Se9rCCRHQd/pBP+nXwtnC39p8rWahszOlzVO69NSW/gFRz57K</latexit><latexit sha1_base64="O/VfR3tx0Oo9X3tsrLCNOjnJhnU=">AAAIV3icpVXNThNRFD6AQq1/oEs3EwnGhdSWmMCS6MaFCzT8JQwhM9PbctP5c+aOSJrufA2fxq0+AG+gT6HfOXdKoYUpxJkUzpz7ne/c83evn4Y6N83m2czs3J278wu1e/X7Dx4+ery49GQ3T4osUDtBEibZvu/lKtSx2jHahGo/zZQX+aHa83vveH3vi8pyncTb5jRVh5HXjXVHB56B6mhx3Q1Vx7j9uuurro77Xqi7sWoP6i9c1/7qrorbI72b6e6xaRwtLjcbTXmcSaFVCstUPlvJ0tw3cqlNCQVUUESKYjKQQ/Iox3tALWpSCt0h9aHLIGlZVzSgOmwLoBQQHrQ9/O3i66DUxvhmzlysA3gJ8ctg6dAKbBLgMsjszZH1QphZex13Xzh5b6f475dcEbSGjqGdZjdE3tSOYzHUoQ2JQSOmVDQcXVCyFJIV3rlzISoDhhQ6lttYzyAHYjnMsyM2ucTOufVk/bcgWcvfQYkt6E9ldD5YuSL1/4qEfYZSpRPIFr8KC+cC/nr+bXQLc3dgH19itu+KcOaSpw3w+mDlLLCdc25VFeUxqm5zyjnqXOAeeRhhQtl1r5KxENw4z2XGHDwRLG2Fd+gTfZD6Wj/MaqQDYnBWVSjB25O+8IG4au8pdpMgLiUdriUjXJFV+gxWTyJiv07JYvtnWs+PMnuVT1/mj614h9yHbWiHk+GcTxpPb5WvWE4Grort5Kt88YxHpaWZ4LY9wPPSote0Rq8gKTmPGhV+I/HLeRtWZDDhdxyTwDaT+eRumT5VNz1TqpAhJI5OC9bu6HYWVehIzk4j5wxPbxV2FDvv4WslNofeR100vguZuir0Scls8dMzd5u47P0xnXN0M1yPTNHjHdF2ZYKAxd3ZGr8pJ4XdtUar2Wh9fLO8+ba8RWv0jJ7TS/TsOm3Se9rCCRHQd/pBP+nXwtnC39p8rWahszOlzVO69NSW/gFRz57K</latexit><latexit sha1_base64="O/VfR3tx0Oo9X3tsrLCNOjnJhnU=">AAAIV3icpVXNThNRFD6AQq1/oEs3EwnGhdSWmMCS6MaFCzT8JQwhM9PbctP5c+aOSJrufA2fxq0+AG+gT6HfOXdKoYUpxJkUzpz7ne/c83evn4Y6N83m2czs3J278wu1e/X7Dx4+ery49GQ3T4osUDtBEibZvu/lKtSx2jHahGo/zZQX+aHa83vveH3vi8pyncTb5jRVh5HXjXVHB56B6mhx3Q1Vx7j9uuurro77Xqi7sWoP6i9c1/7qrorbI72b6e6xaRwtLjcbTXmcSaFVCstUPlvJ0tw3cqlNCQVUUESKYjKQQ/Iox3tALWpSCt0h9aHLIGlZVzSgOmwLoBQQHrQ9/O3i66DUxvhmzlysA3gJ8ctg6dAKbBLgMsjszZH1QphZex13Xzh5b6f475dcEbSGjqGdZjdE3tSOYzHUoQ2JQSOmVDQcXVCyFJIV3rlzISoDhhQ6lttYzyAHYjnMsyM2ucTOufVk/bcgWcvfQYkt6E9ldD5YuSL1/4qEfYZSpRPIFr8KC+cC/nr+bXQLc3dgH19itu+KcOaSpw3w+mDlLLCdc25VFeUxqm5zyjnqXOAeeRhhQtl1r5KxENw4z2XGHDwRLG2Fd+gTfZD6Wj/MaqQDYnBWVSjB25O+8IG4au8pdpMgLiUdriUjXJFV+gxWTyJiv07JYvtnWs+PMnuVT1/mj614h9yHbWiHk+GcTxpPb5WvWE4Grort5Kt88YxHpaWZ4LY9wPPSote0Rq8gKTmPGhV+I/HLeRtWZDDhdxyTwDaT+eRumT5VNz1TqpAhJI5OC9bu6HYWVehIzk4j5wxPbxV2FDvv4WslNofeR100vguZuir0Scls8dMzd5u47P0xnXN0M1yPTNHjHdF2ZYKAxd3ZGr8pJ4XdtUar2Wh9fLO8+ba8RWv0jJ7TS/TsOm3Se9rCCRHQd/pBP+nXwtnC39p8rWahszOlzVO69NSW/gFRz57K</latexit>

N2up

<latexit sha1_base64="q3TnFz6LS3HyxJx4AaVkfZpP4O8=">AAAINXicpVXLbtNQEJ22QEp4pSBWbCyiSixoSCIkuozEhgVCBTVtpbZEtnOTWvEL+5q2irLjR9jCB/AtLNghBCt+gTNjp3k7rbDVdDz3zJk7r3ut0HViXa1+X1ldu3b9RmH9ZvHW7Tt375U27u/FQRLZqmkHbhAdWGasXMdXTe1oVx2EkTI9y1X7Vu8lr+9/VFHsBP6uPg/VsWd2fafj2KaGqlV6+OZ9vdU/0upM+0HkmW4/CQeDVqlcrVTlMWaFWiaUKXt2go21T3REbQrIpoQ8UuSThuySSTHeQ6pRlULojqkPXQTJkXVFAyrCNgFKAWFC28NvF1+HmdbHN3PGYm3Di4u/CJYGbcImAC6CzN4MWU+EmbWLuPvCyXs7x38r4/Kg1XQC7TK7IfKydhyLpg5tSwwOYgpFw9HZGUsiWeGdG2NRaTCE0LHcxnoE2RbLYZ4NsYklds6tKeu/Bcla/rYzbEJ/cqOzwMoVKf5XJOzTlSqdQk7xW7AwxvCL+XfRLczdgb0/wZy+m8IZS562wWuBlbPAdsaFVV6UJ6h6mlPOUWeMe+RhhHFl171cxkRw0zyTjDF4PFimFW7SO3ot9U39MKuWDvDBmVehAG9P+sICYt7eQ+wmQFxKOtyRjHBFtugDWE2JiP0aGUvaP8t6fpTZeT4tmT+24h1yH7ahHU6GcTFpPL15vnw5GbgqaSfP88Uz7mWWeoY77QGelxo9ozo9haTkPKrk+PXEL+dtWJHBjN9pTADbSOaTu2X5VF32TMlDupA4Okew6Y6uZpGH9uTs1HLO8PTmYUex8x7OcrEx9Bbq4uA7kanLQ59mzCl+eeauEld6fyznHN0Mi5Eherwj2q5MELC4O2vTN+WssFev1KqV2tvn5UYju0XX6RE9pifo2RfUoFe0gxPCho/P9IW+Fr4VfhR+Fn6l0NWVzOYBTTyFv/8AP8aV/Q==</latexit><latexit sha1_base64="q3TnFz6LS3HyxJx4AaVkfZpP4O8=">AAAINXicpVXLbtNQEJ22QEp4pSBWbCyiSixoSCIkuozEhgVCBTVtpbZEtnOTWvEL+5q2irLjR9jCB/AtLNghBCt+gTNjp3k7rbDVdDz3zJk7r3ut0HViXa1+X1ldu3b9RmH9ZvHW7Tt375U27u/FQRLZqmkHbhAdWGasXMdXTe1oVx2EkTI9y1X7Vu8lr+9/VFHsBP6uPg/VsWd2fafj2KaGqlV6+OZ9vdU/0upM+0HkmW4/CQeDVqlcrVTlMWaFWiaUKXt2go21T3REbQrIpoQ8UuSThuySSTHeQ6pRlULojqkPXQTJkXVFAyrCNgFKAWFC28NvF1+HmdbHN3PGYm3Di4u/CJYGbcImAC6CzN4MWU+EmbWLuPvCyXs7x38r4/Kg1XQC7TK7IfKydhyLpg5tSwwOYgpFw9HZGUsiWeGdG2NRaTCE0LHcxnoE2RbLYZ4NsYklds6tKeu/Bcla/rYzbEJ/cqOzwMoVKf5XJOzTlSqdQk7xW7AwxvCL+XfRLczdgb0/wZy+m8IZS562wWuBlbPAdsaFVV6UJ6h6mlPOUWeMe+RhhHFl171cxkRw0zyTjDF4PFimFW7SO3ot9U39MKuWDvDBmVehAG9P+sICYt7eQ+wmQFxKOtyRjHBFtugDWE2JiP0aGUvaP8t6fpTZeT4tmT+24h1yH7ahHU6GcTFpPL15vnw5GbgqaSfP88Uz7mWWeoY77QGelxo9ozo9haTkPKrk+PXEL+dtWJHBjN9pTADbSOaTu2X5VF32TMlDupA4Okew6Y6uZpGH9uTs1HLO8PTmYUex8x7OcrEx9Bbq4uA7kanLQ59mzCl+eeauEld6fyznHN0Mi5Eherwj2q5MELC4O2vTN+WssFev1KqV2tvn5UYju0XX6RE9pifo2RfUoFe0gxPCho/P9IW+Fr4VfhR+Fn6l0NWVzOYBTTyFv/8AP8aV/Q==</latexit><latexit sha1_base64="q3TnFz6LS3HyxJx4AaVkfZpP4O8=">AAAINXicpVXLbtNQEJ22QEp4pSBWbCyiSixoSCIkuozEhgVCBTVtpbZEtnOTWvEL+5q2irLjR9jCB/AtLNghBCt+gTNjp3k7rbDVdDz3zJk7r3ut0HViXa1+X1ldu3b9RmH9ZvHW7Tt375U27u/FQRLZqmkHbhAdWGasXMdXTe1oVx2EkTI9y1X7Vu8lr+9/VFHsBP6uPg/VsWd2fafj2KaGqlV6+OZ9vdU/0upM+0HkmW4/CQeDVqlcrVTlMWaFWiaUKXt2go21T3REbQrIpoQ8UuSThuySSTHeQ6pRlULojqkPXQTJkXVFAyrCNgFKAWFC28NvF1+HmdbHN3PGYm3Di4u/CJYGbcImAC6CzN4MWU+EmbWLuPvCyXs7x38r4/Kg1XQC7TK7IfKydhyLpg5tSwwOYgpFw9HZGUsiWeGdG2NRaTCE0LHcxnoE2RbLYZ4NsYklds6tKeu/Bcla/rYzbEJ/cqOzwMoVKf5XJOzTlSqdQk7xW7AwxvCL+XfRLczdgb0/wZy+m8IZS562wWuBlbPAdsaFVV6UJ6h6mlPOUWeMe+RhhHFl171cxkRw0zyTjDF4PFimFW7SO3ot9U39MKuWDvDBmVehAG9P+sICYt7eQ+wmQFxKOtyRjHBFtugDWE2JiP0aGUvaP8t6fpTZeT4tmT+24h1yH7ahHU6GcTFpPL15vnw5GbgqaSfP88Uz7mWWeoY77QGelxo9ozo9haTkPKrk+PXEL+dtWJHBjN9pTADbSOaTu2X5VF32TMlDupA4Okew6Y6uZpGH9uTs1HLO8PTmYUex8x7OcrEx9Bbq4uA7kanLQ59mzCl+eeauEld6fyznHN0Mi5Eherwj2q5MELC4O2vTN+WssFev1KqV2tvn5UYju0XX6RE9pifo2RfUoFe0gxPCho/P9IW+Fr4VfhR+Fn6l0NWVzOYBTTyFv/8AP8aV/Q==</latexit><latexit sha1_base64="q3TnFz6LS3HyxJx4AaVkfZpP4O8=">AAAINXicpVXLbtNQEJ22QEp4pSBWbCyiSixoSCIkuozEhgVCBTVtpbZEtnOTWvEL+5q2irLjR9jCB/AtLNghBCt+gTNjp3k7rbDVdDz3zJk7r3ut0HViXa1+X1ldu3b9RmH9ZvHW7Tt375U27u/FQRLZqmkHbhAdWGasXMdXTe1oVx2EkTI9y1X7Vu8lr+9/VFHsBP6uPg/VsWd2fafj2KaGqlV6+OZ9vdU/0upM+0HkmW4/CQeDVqlcrVTlMWaFWiaUKXt2go21T3REbQrIpoQ8UuSThuySSTHeQ6pRlULojqkPXQTJkXVFAyrCNgFKAWFC28NvF1+HmdbHN3PGYm3Di4u/CJYGbcImAC6CzN4MWU+EmbWLuPvCyXs7x38r4/Kg1XQC7TK7IfKydhyLpg5tSwwOYgpFw9HZGUsiWeGdG2NRaTCE0LHcxnoE2RbLYZ4NsYklds6tKeu/Bcla/rYzbEJ/cqOzwMoVKf5XJOzTlSqdQk7xW7AwxvCL+XfRLczdgb0/wZy+m8IZS562wWuBlbPAdsaFVV6UJ6h6mlPOUWeMe+RhhHFl171cxkRw0zyTjDF4PFimFW7SO3ot9U39MKuWDvDBmVehAG9P+sICYt7eQ+wmQFxKOtyRjHBFtugDWE2JiP0aGUvaP8t6fpTZeT4tmT+24h1yH7ahHU6GcTFpPL15vnw5GbgqaSfP88Uz7mWWeoY77QGelxo9ozo9haTkPKrk+PXEL+dtWJHBjN9pTADbSOaTu2X5VF32TMlDupA4Okew6Y6uZpGH9uTs1HLO8PTmYUex8x7OcrEx9Bbq4uA7kanLQ59mzCl+eeauEld6fyznHN0Mi5Eherwj2q5MELC4O2vTN+WssFev1KqV2tvn5UYju0XX6RE9pifo2RfUoFe0gxPCho/P9IW+Fr4VfhR+Fn6l0NWVzOYBTTyFv/8AP8aV/Q==</latexit>

MLP

Nup<latexit sha1_base64="yhU64bt9VoWUvaXwaZuoU7F/ELs=">AAAIM3icpVXLbtNQEJ22QEp4NKVLNhZRJRY0xBUSXUZiwwKhgpq2UltFtnPTWvEL+5q2irLjP9jCB/AxiB1iAQv+gTNjp3k7rbDVdDz3zJk7r3vtyHMTXa9/X1peuXX7Tmn1bvne/QcP1yrrj/aTMI0d1XRCL4wPbStRnhuopna1pw6jWFm+7akDu/uK1w8+qjhxw2BPX0bqxLdOA7fjOpaGqlXZeNvqHWt1oYMw9i2vl0b9fqtSrdfq8hjTgpkLVcqf3XB95RMdU5tCciglnxQFpCF7ZFGC94hMqlME3Qn1oIshubKuqE9l2KZAKSAsaLv4PcXXUa4N8M2ciVg78OLhL4alQZuwCYGLIbM3Q9ZTYWbtPO6ecPLeLvHfzrl8aDWdQbvIboC8rh3HoqlDOxKDi5gi0XB0Ts6SSlZ458ZIVBoMEXQst7EeQ3bEcpBnQ2wSiZ1za8n6b0Gylr+dHJvSn8LobLByRcr/FQn79KRK55Az/BYsjBH8fP49dAtzd2AfjDFn76ZwJpKnHfDaYOUssJ1xZVUU5RmqnuWUc9QZ4R56GGI82XW3kDEV3CTPOGMCHh+WWYWb9J7eSH0zP8yqpQMCcBZVKMTblb6wgZi19wi7CRGXkg53JSNckS36AFZLImK/Rs6S9c+inh9mdpZPW+aPrXiH3IdtaAeTYVxNGk9vka9ATgauStbJs3zxjPu5pZ7iznqA58Wk57RNzyApOY9qBX598ct5G1SkP+V3EhPCNpb55G5ZPFXXPVOKkB4kjs4VbLajm1kUoX05O7WcMzy9Rdhh7LyHi0JsAr2Nurj4TmXqitDnOXOGX5y5m8SV3R+LOYc3w3xkhB7viPZUJghY3J3m5E05Lexv18x6zXz3otpo5LfoKj2mJ/QUPfuSGvSadnFCONjDZ/pCX0vfSj9KP0u/MujyUm6zQWNP6e8/hHOVWQ==</latexit><latexit sha1_base64="yhU64bt9VoWUvaXwaZuoU7F/ELs=">AAAIM3icpVXLbtNQEJ22QEp4NKVLNhZRJRY0xBUSXUZiwwKhgpq2UltFtnPTWvEL+5q2irLjP9jCB/AxiB1iAQv+gTNjp3k7rbDVdDz3zJk7r3vtyHMTXa9/X1peuXX7Tmn1bvne/QcP1yrrj/aTMI0d1XRCL4wPbStRnhuopna1pw6jWFm+7akDu/uK1w8+qjhxw2BPX0bqxLdOA7fjOpaGqlXZeNvqHWt1oYMw9i2vl0b9fqtSrdfq8hjTgpkLVcqf3XB95RMdU5tCciglnxQFpCF7ZFGC94hMqlME3Qn1oIshubKuqE9l2KZAKSAsaLv4PcXXUa4N8M2ciVg78OLhL4alQZuwCYGLIbM3Q9ZTYWbtPO6ecPLeLvHfzrl8aDWdQbvIboC8rh3HoqlDOxKDi5gi0XB0Ts6SSlZ458ZIVBoMEXQst7EeQ3bEcpBnQ2wSiZ1za8n6b0Gylr+dHJvSn8LobLByRcr/FQn79KRK55Az/BYsjBH8fP49dAtzd2AfjDFn76ZwJpKnHfDaYOUssJ1xZVUU5RmqnuWUc9QZ4R56GGI82XW3kDEV3CTPOGMCHh+WWYWb9J7eSH0zP8yqpQMCcBZVKMTblb6wgZi19wi7CRGXkg53JSNckS36AFZLImK/Rs6S9c+inh9mdpZPW+aPrXiH3IdtaAeTYVxNGk9vka9ATgauStbJs3zxjPu5pZ7iznqA58Wk57RNzyApOY9qBX598ct5G1SkP+V3EhPCNpb55G5ZPFXXPVOKkB4kjs4VbLajm1kUoX05O7WcMzy9Rdhh7LyHi0JsAr2Nurj4TmXqitDnOXOGX5y5m8SV3R+LOYc3w3xkhB7viPZUJghY3J3m5E05Lexv18x6zXz3otpo5LfoKj2mJ/QUPfuSGvSadnFCONjDZ/pCX0vfSj9KP0u/MujyUm6zQWNP6e8/hHOVWQ==</latexit><latexit sha1_base64="yhU64bt9VoWUvaXwaZuoU7F/ELs=">AAAIM3icpVXLbtNQEJ22QEp4NKVLNhZRJRY0xBUSXUZiwwKhgpq2UltFtnPTWvEL+5q2irLjP9jCB/AxiB1iAQv+gTNjp3k7rbDVdDz3zJk7r3vtyHMTXa9/X1peuXX7Tmn1bvne/QcP1yrrj/aTMI0d1XRCL4wPbStRnhuopna1pw6jWFm+7akDu/uK1w8+qjhxw2BPX0bqxLdOA7fjOpaGqlXZeNvqHWt1oYMw9i2vl0b9fqtSrdfq8hjTgpkLVcqf3XB95RMdU5tCciglnxQFpCF7ZFGC94hMqlME3Qn1oIshubKuqE9l2KZAKSAsaLv4PcXXUa4N8M2ciVg78OLhL4alQZuwCYGLIbM3Q9ZTYWbtPO6ecPLeLvHfzrl8aDWdQbvIboC8rh3HoqlDOxKDi5gi0XB0Ts6SSlZ458ZIVBoMEXQst7EeQ3bEcpBnQ2wSiZ1za8n6b0Gylr+dHJvSn8LobLByRcr/FQn79KRK55Az/BYsjBH8fP49dAtzd2AfjDFn76ZwJpKnHfDaYOUssJ1xZVUU5RmqnuWUc9QZ4R56GGI82XW3kDEV3CTPOGMCHh+WWYWb9J7eSH0zP8yqpQMCcBZVKMTblb6wgZi19wi7CRGXkg53JSNckS36AFZLImK/Rs6S9c+inh9mdpZPW+aPrXiH3IdtaAeTYVxNGk9vka9ATgauStbJs3zxjPu5pZ7iznqA58Wk57RNzyApOY9qBX598ct5G1SkP+V3EhPCNpb55G5ZPFXXPVOKkB4kjs4VbLajm1kUoX05O7WcMzy9Rdhh7LyHi0JsAr2Nurj4TmXqitDnOXOGX5y5m8SV3R+LOYc3w3xkhB7viPZUJghY3J3m5E05Lexv18x6zXz3otpo5LfoKj2mJ/QUPfuSGvSadnFCONjDZ/pCX0vfSj9KP0u/MujyUm6zQWNP6e8/hHOVWQ==</latexit><latexit sha1_base64="yhU64bt9VoWUvaXwaZuoU7F/ELs=">AAAIM3icpVXLbtNQEJ22QEp4NKVLNhZRJRY0xBUSXUZiwwKhgpq2UltFtnPTWvEL+5q2irLjP9jCB/AxiB1iAQv+gTNjp3k7rbDVdDz3zJk7r3vtyHMTXa9/X1peuXX7Tmn1bvne/QcP1yrrj/aTMI0d1XRCL4wPbStRnhuopna1pw6jWFm+7akDu/uK1w8+qjhxw2BPX0bqxLdOA7fjOpaGqlXZeNvqHWt1oYMw9i2vl0b9fqtSrdfq8hjTgpkLVcqf3XB95RMdU5tCciglnxQFpCF7ZFGC94hMqlME3Qn1oIshubKuqE9l2KZAKSAsaLv4PcXXUa4N8M2ciVg78OLhL4alQZuwCYGLIbM3Q9ZTYWbtPO6ecPLeLvHfzrl8aDWdQbvIboC8rh3HoqlDOxKDi5gi0XB0Ts6SSlZ458ZIVBoMEXQst7EeQ3bEcpBnQ2wSiZ1za8n6b0Gylr+dHJvSn8LobLByRcr/FQn79KRK55Az/BYsjBH8fP49dAtzd2AfjDFn76ZwJpKnHfDaYOUssJ1xZVUU5RmqnuWUc9QZ4R56GGI82XW3kDEV3CTPOGMCHh+WWYWb9J7eSH0zP8yqpQMCcBZVKMTblb6wgZi19wi7CRGXkg53JSNckS36AFZLImK/Rs6S9c+inh9mdpZPW+aPrXiH3IdtaAeTYVxNGk9vka9ATgauStbJs3zxjPu5pZ7iznqA58Wk57RNzyApOY9qBX598ct5G1SkP+V3EhPCNpb55G5ZPFXXPVOKkB4kjs4VbLajm1kUoX05O7WcMzy9Rdhh7LyHi0JsAr2Nurj4TmXqitDnOXOGX5y5m8SV3R+LOYc3w3xkhB7viPZUJghY3J3m5E05Lexv18x6zXz3otpo5LfoKj2mJ/QUPfuSGvSadnFCONjDZ/pCX0vfSj9KP0u/MujyUm6zQWNP6e8/hHOVWQ==</latexit>

Aup<latexit sha1_base64="O6gIUyIY3maqhTj7QdUHO0uPpd4=">AAAIP3icpVXLbtNQEJ22QEp4pbBkgSGqxIKGpEKiyyA2LFgU1LSVmlLZzk1ixS/sa9oqyg6+hi18AJ/BFwAbxJYdZ8ZOk+bhtMJW0/HcM2fuvO61QteJdbX6fWl55crVa4XV68UbN2/dvlNau7sbB0lkq4YduEG0b5mxch1fNbSjXbUfRsr0LFftWb2XvL73QUWxE/g7+jRUh57Z8Z22Y5saqqPSg6Zn6q7V7r8YvOs3Hza1OtF+EHmm20/CwaB4VCpXK1V5jGmhlgllyp7tYG3lIzWpRQHZlJBHinzSkF0yKcZ7QDWqUgjdIfWhiyA5sq5oQEXYJkApIExoe/jt4Osg0/r4Zs5YrG14cfEXwdKgddgEwEWQ2Zsh64kws3Yed184eW+n+G9lXB60mrrQLrIbIi9qx7FoatOWxOAgplA0HJ2dsSSSFd65MRaVBkMIHcstrEeQbbEc5tkQm1hi59yasv5TkKzlbzvDJvQrNzoLrFyR4n9Fwj5dqdIx5BS/AQtjDD+ffwfdwtxt2PvnmNN3XThjydMWeC2wchbYzjizyouyi6qnOeUctce4Rx5GGFd23ctlTAQ3yXOeMQaPB8u0wg16S6+lvqkfZtXSAT448yoU4O1JX1hAzNp7iN0EiEtJhzuSEa7IBr0HqykRsV8jY0n7Z1HPjzI7y6cl88dWvEPuwxa0w8kwziaNpzfPly8nA1cl7eRZvnjGvcxST3GnPcDzUqOntElPICk5jyo5fj3xy3kbVmQw5XcSE8A2kvnkblk8VRc9U/KQLiSOzhFsuqPLWeShPTk7tZwzPL152FHsvIeTXGwMvYW6OPhOZOry0McZc4pfnLnLxJXeH4s5RzfDfGSIHm+LtiMTBCzuztrkTTkt7G5WatVK7c2zcr2e3aKrdJ8e0WP07HOq0yvaxglh0yf6TF/oa+Fb4Ufhd+FPCl1eymzu0bmn8Pcf1auaEA==</latexit><latexit sha1_base64="O6gIUyIY3maqhTj7QdUHO0uPpd4=">AAAIP3icpVXLbtNQEJ22QEp4pbBkgSGqxIKGpEKiyyA2LFgU1LSVmlLZzk1ixS/sa9oqyg6+hi18AJ/BFwAbxJYdZ8ZOk+bhtMJW0/HcM2fuvO61QteJdbX6fWl55crVa4XV68UbN2/dvlNau7sbB0lkq4YduEG0b5mxch1fNbSjXbUfRsr0LFftWb2XvL73QUWxE/g7+jRUh57Z8Z22Y5saqqPSg6Zn6q7V7r8YvOs3Hza1OtF+EHmm20/CwaB4VCpXK1V5jGmhlgllyp7tYG3lIzWpRQHZlJBHinzSkF0yKcZ7QDWqUgjdIfWhiyA5sq5oQEXYJkApIExoe/jt4Osg0/r4Zs5YrG14cfEXwdKgddgEwEWQ2Zsh64kws3Yed184eW+n+G9lXB60mrrQLrIbIi9qx7FoatOWxOAgplA0HJ2dsSSSFd65MRaVBkMIHcstrEeQbbEc5tkQm1hi59yasv5TkKzlbzvDJvQrNzoLrFyR4n9Fwj5dqdIx5BS/AQtjDD+ffwfdwtxt2PvnmNN3XThjydMWeC2wchbYzjizyouyi6qnOeUctce4Rx5GGFd23ctlTAQ3yXOeMQaPB8u0wg16S6+lvqkfZtXSAT448yoU4O1JX1hAzNp7iN0EiEtJhzuSEa7IBr0HqykRsV8jY0n7Z1HPjzI7y6cl88dWvEPuwxa0w8kwziaNpzfPly8nA1cl7eRZvnjGvcxST3GnPcDzUqOntElPICk5jyo5fj3xy3kbVmQw5XcSE8A2kvnkblk8VRc9U/KQLiSOzhFsuqPLWeShPTk7tZwzPL152FHsvIeTXGwMvYW6OPhOZOry0McZc4pfnLnLxJXeH4s5RzfDfGSIHm+LtiMTBCzuztrkTTkt7G5WatVK7c2zcr2e3aKrdJ8e0WP07HOq0yvaxglh0yf6TF/oa+Fb4Ufhd+FPCl1eymzu0bmn8Pcf1auaEA==</latexit><latexit sha1_base64="O6gIUyIY3maqhTj7QdUHO0uPpd4=">AAAIP3icpVXLbtNQEJ22QEp4pbBkgSGqxIKGpEKiyyA2LFgU1LSVmlLZzk1ixS/sa9oqyg6+hi18AJ/BFwAbxJYdZ8ZOk+bhtMJW0/HcM2fuvO61QteJdbX6fWl55crVa4XV68UbN2/dvlNau7sbB0lkq4YduEG0b5mxch1fNbSjXbUfRsr0LFftWb2XvL73QUWxE/g7+jRUh57Z8Z22Y5saqqPSg6Zn6q7V7r8YvOs3Hza1OtF+EHmm20/CwaB4VCpXK1V5jGmhlgllyp7tYG3lIzWpRQHZlJBHinzSkF0yKcZ7QDWqUgjdIfWhiyA5sq5oQEXYJkApIExoe/jt4Osg0/r4Zs5YrG14cfEXwdKgddgEwEWQ2Zsh64kws3Yed184eW+n+G9lXB60mrrQLrIbIi9qx7FoatOWxOAgplA0HJ2dsSSSFd65MRaVBkMIHcstrEeQbbEc5tkQm1hi59yasv5TkKzlbzvDJvQrNzoLrFyR4n9Fwj5dqdIx5BS/AQtjDD+ffwfdwtxt2PvnmNN3XThjydMWeC2wchbYzjizyouyi6qnOeUctce4Rx5GGFd23ctlTAQ3yXOeMQaPB8u0wg16S6+lvqkfZtXSAT448yoU4O1JX1hAzNp7iN0EiEtJhzuSEa7IBr0HqykRsV8jY0n7Z1HPjzI7y6cl88dWvEPuwxa0w8kwziaNpzfPly8nA1cl7eRZvnjGvcxST3GnPcDzUqOntElPICk5jyo5fj3xy3kbVmQw5XcSE8A2kvnkblk8VRc9U/KQLiSOzhFsuqPLWeShPTk7tZwzPL152FHsvIeTXGwMvYW6OPhOZOry0McZc4pfnLnLxJXeH4s5RzfDfGSIHm+LtiMTBCzuztrkTTkt7G5WatVK7c2zcr2e3aKrdJ8e0WP07HOq0yvaxglh0yf6TF/oa+Fb4Ufhd+FPCl1eymzu0bmn8Pcf1auaEA==</latexit><latexit sha1_base64="O6gIUyIY3maqhTj7QdUHO0uPpd4=">AAAIP3icpVXLbtNQEJ22QEp4pbBkgSGqxIKGpEKiyyA2LFgU1LSVmlLZzk1ixS/sa9oqyg6+hi18AJ/BFwAbxJYdZ8ZOk+bhtMJW0/HcM2fuvO61QteJdbX6fWl55crVa4XV68UbN2/dvlNau7sbB0lkq4YduEG0b5mxch1fNbSjXbUfRsr0LFftWb2XvL73QUWxE/g7+jRUh57Z8Z22Y5saqqPSg6Zn6q7V7r8YvOs3Hza1OtF+EHmm20/CwaB4VCpXK1V5jGmhlgllyp7tYG3lIzWpRQHZlJBHinzSkF0yKcZ7QDWqUgjdIfWhiyA5sq5oQEXYJkApIExoe/jt4Osg0/r4Zs5YrG14cfEXwdKgddgEwEWQ2Zsh64kws3Yed184eW+n+G9lXB60mrrQLrIbIi9qx7FoatOWxOAgplA0HJ2dsSSSFd65MRaVBkMIHcstrEeQbbEc5tkQm1hi59yasv5TkKzlbzvDJvQrNzoLrFyR4n9Fwj5dqdIx5BS/AQtjDD+ffwfdwtxt2PvnmNN3XThjydMWeC2wchbYzjizyouyi6qnOeUctce4Rx5GGFd23ctlTAQ3yXOeMQaPB8u0wg16S6+lvqkfZtXSAT448yoU4O1JX1hAzNp7iN0EiEtJhzuSEa7IBr0HqykRsV8jY0n7Z1HPjzI7y6cl88dWvEPuwxa0w8kwziaNpzfPly8nA1cl7eRZvnjGvcxST3GnPcDzUqOntElPICk5jyo5fj3xy3kbVmQw5XcSE8A2kvnkblk8VRc9U/KQLiSOzhFsuqPLWeShPTk7tZwzPL152FHsvIeTXGwMvYW6OPhOZOry0McZc4pfnLnLxJXeH4s5RzfDfGSIHm+LtiMTBCzuztrkTTkt7G5WatVK7c2zcr2e3aKrdJ8e0WP07HOq0yvaxglh0yf6TF/oa+Fb4Ufhd+FPCl1eymzu0bmn8Pcf1auaEA==</latexit>

Multi-head

Self-attention

… …

… Al<latexit sha1_base64="RaWDMvpaZMu1FDrCmowPXEM+m5M=">AAAIL3icpVXLbtNQEJ22QEp4NIUlG4uoEgsa4gqJLoPYsGBRUNNWaqvIdm5SK35hX1OqKDv+gi18AF+D2CCW9C84M3aat9MKW03Hc8+cufO61448N9H1+q+V1bVbt++U1u+W791/8HCjsvnoIAnT2FFNJ/TC+Mi2EuW5gWpqV3vqKIqV5dueOrR7b3j98JOKEzcM9vVFpE59qxu4HdexNFStysaJb+kzu9N/PWj1vUG5VanWa3V5jFnBzIUq5c9euLn2hU6oTSE5lJJPigLSkD2yKMF7TCbVKYLulPrQxZBcWVc0oDJsU6AUEBa0Pfx28XWcawN8M2ci1g68ePiLYWnQFmxC4GLI7M2Q9VSYWbuIuy+cvLcL/LdzLh9aTWfQLrMbIq9rx7Fo6tCuxOAipkg0HJ2Ts6SSFd65MRaVBkMEHcttrMeQHbEc5tkQm0Ri59xasv5XkKzlbyfHpnRZGJ0NVq5I+b8iYZ+eVOkccobfhoUxhl/Mv49uYe4O7IMJ5uzdEs5E8rQLXhusnAW2M66siqI8Q9WznHKOOmPcIw8jjCe77hUypoKb5plkTMDjwzKrcJM+0Dupb+aHWbV0QADOogqFeHvSFzYQ8/YeYTch4lLS4a5khCuyTR/BaklE7NfIWbL+Wdbzo8zO82nL/LEV75D7sA3tcDKMq0nj6S3yFcjJwFXJOnmeL55xP7fUM9xZD/C8mPSCdug5JCXnUa3Ary9+OW/Digxm/E5jQtjGMp/cLcun6rpnShHSg8TRuYLNdnQziyK0L2enlnOGp7cIO4qd9/C5EJtAb6MuLr5Tmboi9HnOnOGXZ+4mcWX3x3LO0c2wGBmhxzui7coEAYu705y+KWeFg52aWa+Z719WG438Fl2nJ/SUnqFnX1GD3tIeTgi+Tb/SN/pe+lH6Wfpd+pNBV1dym8c08ZQu/wGbT5LZ</latexit><latexit sha1_base64="RaWDMvpaZMu1FDrCmowPXEM+m5M=">AAAIL3icpVXLbtNQEJ22QEp4NIUlG4uoEgsa4gqJLoPYsGBRUNNWaqvIdm5SK35hX1OqKDv+gi18AF+D2CCW9C84M3aat9MKW03Hc8+cufO61448N9H1+q+V1bVbt++U1u+W791/8HCjsvnoIAnT2FFNJ/TC+Mi2EuW5gWpqV3vqKIqV5dueOrR7b3j98JOKEzcM9vVFpE59qxu4HdexNFStysaJb+kzu9N/PWj1vUG5VanWa3V5jFnBzIUq5c9euLn2hU6oTSE5lJJPigLSkD2yKMF7TCbVKYLulPrQxZBcWVc0oDJsU6AUEBa0Pfx28XWcawN8M2ci1g68ePiLYWnQFmxC4GLI7M2Q9VSYWbuIuy+cvLcL/LdzLh9aTWfQLrMbIq9rx7Fo6tCuxOAipkg0HJ2Ts6SSFd65MRaVBkMEHcttrMeQHbEc5tkQm0Ri59xasv5XkKzlbyfHpnRZGJ0NVq5I+b8iYZ+eVOkccobfhoUxhl/Mv49uYe4O7IMJ5uzdEs5E8rQLXhusnAW2M66siqI8Q9WznHKOOmPcIw8jjCe77hUypoKb5plkTMDjwzKrcJM+0Dupb+aHWbV0QADOogqFeHvSFzYQ8/YeYTch4lLS4a5khCuyTR/BaklE7NfIWbL+Wdbzo8zO82nL/LEV75D7sA3tcDKMq0nj6S3yFcjJwFXJOnmeL55xP7fUM9xZD/C8mPSCdug5JCXnUa3Ary9+OW/Digxm/E5jQtjGMp/cLcun6rpnShHSg8TRuYLNdnQziyK0L2enlnOGp7cIO4qd9/C5EJtAb6MuLr5Tmboi9HnOnOGXZ+4mcWX3x3LO0c2wGBmhxzui7coEAYu705y+KWeFg52aWa+Z719WG438Fl2nJ/SUnqFnX1GD3tIeTgi+Tb/SN/pe+lH6Wfpd+pNBV1dym8c08ZQu/wGbT5LZ</latexit><latexit sha1_base64="RaWDMvpaZMu1FDrCmowPXEM+m5M=">AAAIL3icpVXLbtNQEJ22QEp4NIUlG4uoEgsa4gqJLoPYsGBRUNNWaqvIdm5SK35hX1OqKDv+gi18AF+D2CCW9C84M3aat9MKW03Hc8+cufO61448N9H1+q+V1bVbt++U1u+W791/8HCjsvnoIAnT2FFNJ/TC+Mi2EuW5gWpqV3vqKIqV5dueOrR7b3j98JOKEzcM9vVFpE59qxu4HdexNFStysaJb+kzu9N/PWj1vUG5VanWa3V5jFnBzIUq5c9euLn2hU6oTSE5lJJPigLSkD2yKMF7TCbVKYLulPrQxZBcWVc0oDJsU6AUEBa0Pfx28XWcawN8M2ci1g68ePiLYWnQFmxC4GLI7M2Q9VSYWbuIuy+cvLcL/LdzLh9aTWfQLrMbIq9rx7Fo6tCuxOAipkg0HJ2Ts6SSFd65MRaVBkMEHcttrMeQHbEc5tkQm0Ri59xasv5XkKzlbyfHpnRZGJ0NVq5I+b8iYZ+eVOkccobfhoUxhl/Mv49uYe4O7IMJ5uzdEs5E8rQLXhusnAW2M66siqI8Q9WznHKOOmPcIw8jjCe77hUypoKb5plkTMDjwzKrcJM+0Dupb+aHWbV0QADOogqFeHvSFzYQ8/YeYTch4lLS4a5khCuyTR/BaklE7NfIWbL+Wdbzo8zO82nL/LEV75D7sA3tcDKMq0nj6S3yFcjJwFXJOnmeL55xP7fUM9xZD/C8mPSCdug5JCXnUa3Ary9+OW/Digxm/E5jQtjGMp/cLcun6rpnShHSg8TRuYLNdnQziyK0L2enlnOGp7cIO4qd9/C5EJtAb6MuLr5Tmboi9HnOnOGXZ+4mcWX3x3LO0c2wGBmhxzui7coEAYu705y+KWeFg52aWa+Z719WG438Fl2nJ/SUnqFnX1GD3tIeTgi+Tb/SN/pe+lH6Wfpd+pNBV1dym8c08ZQu/wGbT5LZ</latexit><latexit sha1_base64="RaWDMvpaZMu1FDrCmowPXEM+m5M=">AAAIL3icpVXLbtNQEJ22QEp4NIUlG4uoEgsa4gqJLoPYsGBRUNNWaqvIdm5SK35hX1OqKDv+gi18AF+D2CCW9C84M3aat9MKW03Hc8+cufO61448N9H1+q+V1bVbt++U1u+W791/8HCjsvnoIAnT2FFNJ/TC+Mi2EuW5gWpqV3vqKIqV5dueOrR7b3j98JOKEzcM9vVFpE59qxu4HdexNFStysaJb+kzu9N/PWj1vUG5VanWa3V5jFnBzIUq5c9euLn2hU6oTSE5lJJPigLSkD2yKMF7TCbVKYLulPrQxZBcWVc0oDJsU6AUEBa0Pfx28XWcawN8M2ci1g68ePiLYWnQFmxC4GLI7M2Q9VSYWbuIuy+cvLcL/LdzLh9aTWfQLrMbIq9rx7Fo6tCuxOAipkg0HJ2Ts6SSFd65MRaVBkMEHcttrMeQHbEc5tkQm0Ri59xasv5XkKzlbyfHpnRZGJ0NVq5I+b8iYZ+eVOkccobfhoUxhl/Mv49uYe4O7IMJ5uzdEs5E8rQLXhusnAW2M66siqI8Q9WznHKOOmPcIw8jjCe77hUypoKb5plkTMDjwzKrcJM+0Dupb+aHWbV0QADOogqFeHvSFzYQ8/YeYTch4lLS4a5khCuyTR/BaklE7NfIWbL+Wdbzo8zO82nL/LEV75D7sA3tcDKMq0nj6S3yFcjJwFXJOnmeL55xP7fUM9xZD/C8mPSCdug5JCXnUa3Ary9+OW/Digxm/E5jQtjGMp/cLcun6rpnShHSg8TRuYLNdnQziyK0L2enlnOGp7cIO4qd9/C5EJtAb6MuLr5Tmboi9HnOnOGXZ+4mcWX3x3LO0c2wGBmhxzui7coEAYu705y+KWeFg52aWa+Z719WG438Fl2nJ/SUnqFnX1GD3tIeTgi+Tb/SN/pe+lH6Wfpd+pNBV1dym8c08ZQu/wGbT5LZ</latexit>

Softmax

…

…

8 > < > : <latexit sha1_base64="O/VfR3tx0Oo9X3tsrLCNOjnJhnU=">AAAIV3icpVXNThNRFD6AQq1/oEs3EwnGhdSWmMCS6MaFCzT8JQwhM9PbctP5c+aOSJrufA2fxq0+AG+gT6HfOXdKoYUpxJkUzpz7ne/c83evn4Y6N83m2czs3J278wu1e/X7Dx4+ery49GQ3T4osUDtBEibZvu/lKtSx2jHahGo/zZQX+aHa83vveH3vi8pyncTb5jRVh5HXjXVHB56B6mhx3Q1Vx7j9uuurro77Xqi7sWoP6i9c1/7qrorbI72b6e6xaRwtLjcbTXmcSaFVCstUPlvJ0tw3cqlNCQVUUESKYjKQQ/Iox3tALWpSCt0h9aHLIGlZVzSgOmwLoBQQHrQ9/O3i66DUxvhmzlysA3gJ8ctg6dAKbBLgMsjszZH1QphZex13Xzh5b6f475dcEbSGjqGdZjdE3tSOYzHUoQ2JQSOmVDQcXVCyFJIV3rlzISoDhhQ6lttYzyAHYjnMsyM2ucTOufVk/bcgWcvfQYkt6E9ldD5YuSL1/4qEfYZSpRPIFr8KC+cC/nr+bXQLc3dgH19itu+KcOaSpw3w+mDlLLCdc25VFeUxqm5zyjnqXOAeeRhhQtl1r5KxENw4z2XGHDwRLG2Fd+gTfZD6Wj/MaqQDYnBWVSjB25O+8IG4au8pdpMgLiUdriUjXJFV+gxWTyJiv07JYvtnWs+PMnuVT1/mj614h9yHbWiHk+GcTxpPb5WvWE4Grort5Kt88YxHpaWZ4LY9wPPSote0Rq8gKTmPGhV+I/HLeRtWZDDhdxyTwDaT+eRumT5VNz1TqpAhJI5OC9bu6HYWVehIzk4j5wxPbxV2FDvv4WslNofeR100vguZuir0Scls8dMzd5u47P0xnXN0M1yPTNHjHdF2ZYKAxd3ZGr8pJ4XdtUar2Wh9fLO8+ba8RWv0jJ7TS/TsOm3Se9rCCRHQd/pBP+nXwtnC39p8rWahszOlzVO69NSW/gFRz57K</latexit><latexit sha1_base64="O/VfR3tx0Oo9X3tsrLCNOjnJhnU=">AAAIV3icpVXNThNRFD6AQq1/oEs3EwnGhdSWmMCS6MaFCzT8JQwhM9PbctP5c+aOSJrufA2fxq0+AG+gT6HfOXdKoYUpxJkUzpz7ne/c83evn4Y6N83m2czs3J278wu1e/X7Dx4+ery49GQ3T4osUDtBEibZvu/lKtSx2jHahGo/zZQX+aHa83vveH3vi8pyncTb5jRVh5HXjXVHB56B6mhx3Q1Vx7j9uuurro77Xqi7sWoP6i9c1/7qrorbI72b6e6xaRwtLjcbTXmcSaFVCstUPlvJ0tw3cqlNCQVUUESKYjKQQ/Iox3tALWpSCt0h9aHLIGlZVzSgOmwLoBQQHrQ9/O3i66DUxvhmzlysA3gJ8ctg6dAKbBLgMsjszZH1QphZex13Xzh5b6f475dcEbSGjqGdZjdE3tSOYzHUoQ2JQSOmVDQcXVCyFJIV3rlzISoDhhQ6lttYzyAHYjnMsyM2ucTOufVk/bcgWcvfQYkt6E9ldD5YuSL1/4qEfYZSpRPIFr8KC+cC/nr+bXQLc3dgH19itu+KcOaSpw3w+mDlLLCdc25VFeUxqm5zyjnqXOAeeRhhQtl1r5KxENw4z2XGHDwRLG2Fd+gTfZD6Wj/MaqQDYnBWVSjB25O+8IG4au8pdpMgLiUdriUjXJFV+gxWTyJiv07JYvtnWs+PMnuVT1/mj614h9yHbWiHk+GcTxpPb5WvWE4Grort5Kt88YxHpaWZ4LY9wPPSote0Rq8gKTmPGhV+I/HLeRtWZDDhdxyTwDaT+eRumT5VNz1TqpAhJI5OC9bu6HYWVehIzk4j5wxPbxV2FDvv4WslNofeR100vguZuir0Scls8dMzd5u47P0xnXN0M1yPTNHjHdF2ZYKAxd3ZGr8pJ4XdtUar2Wh9fLO8+ba8RWv0jJ7TS/TsOm3Se9rCCRHQd/pBP+nXwtnC39p8rWahszOlzVO69NSW/gFRz57K</latexit><latexit sha1_base64="O/VfR3tx0Oo9X3tsrLCNOjnJhnU=">AAAIV3icpVXNThNRFD6AQq1/oEs3EwnGhdSWmMCS6MaFCzT8JQwhM9PbctP5c+aOSJrufA2fxq0+AG+gT6HfOXdKoYUpxJkUzpz7ne/c83evn4Y6N83m2czs3J278wu1e/X7Dx4+ery49GQ3T4osUDtBEibZvu/lKtSx2jHahGo/zZQX+aHa83vveH3vi8pyncTb5jRVh5HXjXVHB56B6mhx3Q1Vx7j9uuurro77Xqi7sWoP6i9c1/7qrorbI72b6e6xaRwtLjcbTXmcSaFVCstUPlvJ0tw3cqlNCQVUUESKYjKQQ/Iox3tALWpSCt0h9aHLIGlZVzSgOmwLoBQQHrQ9/O3i66DUxvhmzlysA3gJ8ctg6dAKbBLgMsjszZH1QphZex13Xzh5b6f475dcEbSGjqGdZjdE3tSOYzHUoQ2JQSOmVDQcXVCyFJIV3rlzISoDhhQ6lttYzyAHYjnMsyM2ucTOufVk/bcgWcvfQYkt6E9ldD5YuSL1/4qEfYZSpRPIFr8KC+cC/nr+bXQLc3dgH19itu+KcOaSpw3w+mDlLLCdc25VFeUxqm5zyjnqXOAeeRhhQtl1r5KxENw4z2XGHDwRLG2Fd+gTfZD6Wj/MaqQDYnBWVSjB25O+8IG4au8pdpMgLiUdriUjXJFV+gxWTyJiv07JYvtnWs+PMnuVT1/mj614h9yHbWiHk+GcTxpPb5WvWE4Grort5Kt88YxHpaWZ4LY9wPPSote0Rq8gKTmPGhV+I/HLeRtWZDDhdxyTwDaT+eRumT5VNz1TqpAhJI5OC9bu6HYWVehIzk4j5wxPbxV2FDvv4WslNofeR100vguZuir0Scls8dMzd5u47P0xnXN0M1yPTNHjHdF2ZYKAxd3ZGr8pJ4XdtUar2Wh9fLO8+ba8RWv0jJ7TS/TsOm3Se9rCCRHQd/pBP+nXwtnC39p8rWahszOlzVO69NSW/gFRz57K</latexit><latexit sha1_base64="O/VfR3tx0Oo9X3tsrLCNOjnJhnU=">AAAIV3icpVXNThNRFD6AQq1/oEs3EwnGhdSWmMCS6MaFCzT8JQwhM9PbctP5c+aOSJrufA2fxq0+AG+gT6HfOXdKoYUpxJkUzpz7ne/c83evn4Y6N83m2czs3J278wu1e/X7Dx4+ery49GQ3T4osUDtBEibZvu/lKtSx2jHahGo/zZQX+aHa83vveH3vi8pyncTb5jRVh5HXjXVHB56B6mhx3Q1Vx7j9uuurro77Xqi7sWoP6i9c1/7qrorbI72b6e6xaRwtLjcbTXmcSaFVCstUPlvJ0tw3cqlNCQVUUESKYjKQQ/Iox3tALWpSCt0h9aHLIGlZVzSgOmwLoBQQHrQ9/O3i66DUxvhmzlysA3gJ8ctg6dAKbBLgMsjszZH1QphZex13Xzh5b6f475dcEbSGjqGdZjdE3tSOYzHUoQ2JQSOmVDQcXVCyFJIV3rlzISoDhhQ6lttYzyAHYjnMsyM2ucTOufVk/bcgWcvfQYkt6E9ldD5YuSL1/4qEfYZSpRPIFr8KC+cC/nr+bXQLc3dgH19itu+KcOaSpw3w+mDlLLCdc25VFeUxqm5zyjnqXOAeeRhhQtl1r5KxENw4z2XGHDwRLG2Fd+gTfZD6Wj/MaqQDYnBWVSjB25O+8IG4au8pdpMgLiUdriUjXJFV+gxWTyJiv07JYvtnWs+PMnuVT1/mj614h9yHbWiHk+GcTxpPb5WvWE4Grort5Kt88YxHpaWZ4LY9wPPSote0Rq8gKTmPGhV+I/HLeRtWZDDhdxyTwDaT+eRumT5VNz1TqpAhJI5OC9bu6HYWVehIzk4j5wxPbxV2FDvv4WslNofeR100vguZuir0Scls8dMzd5u47P0xnXN0M1yPTNHjHdF2ZYKAxd3ZGr8pJ4XdtUar2Wh9fLO8+ba8RWv0jJ7TS/TsOm3Se9rCCRHQd/pBP+nXwtnC39p8rWahszOlzVO69NSW/gFRz57K</latexit>

N2up

<latexit sha1_base64="q3TnFz6LS3HyxJx4AaVkfZpP4O8=">AAAINXicpVXLbtNQEJ22QEp4pSBWbCyiSixoSCIkuozEhgVCBTVtpbZEtnOTWvEL+5q2irLjR9jCB/AtLNghBCt+gTNjp3k7rbDVdDz3zJk7r3ut0HViXa1+X1ldu3b9RmH9ZvHW7Tt375U27u/FQRLZqmkHbhAdWGasXMdXTe1oVx2EkTI9y1X7Vu8lr+9/VFHsBP6uPg/VsWd2fafj2KaGqlV6+OZ9vdU/0upM+0HkmW4/CQeDVqlcrVTlMWaFWiaUKXt2go21T3REbQrIpoQ8UuSThuySSTHeQ6pRlULojqkPXQTJkXVFAyrCNgFKAWFC28NvF1+HmdbHN3PGYm3Di4u/CJYGbcImAC6CzN4MWU+EmbWLuPvCyXs7x38r4/Kg1XQC7TK7IfKydhyLpg5tSwwOYgpFw9HZGUsiWeGdG2NRaTCE0LHcxnoE2RbLYZ4NsYklds6tKeu/Bcla/rYzbEJ/cqOzwMoVKf5XJOzTlSqdQk7xW7AwxvCL+XfRLczdgb0/wZy+m8IZS562wWuBlbPAdsaFVV6UJ6h6mlPOUWeMe+RhhHFl171cxkRw0zyTjDF4PFimFW7SO3ot9U39MKuWDvDBmVehAG9P+sICYt7eQ+wmQFxKOtyRjHBFtugDWE2JiP0aGUvaP8t6fpTZeT4tmT+24h1yH7ahHU6GcTFpPL15vnw5GbgqaSfP88Uz7mWWeoY77QGelxo9ozo9haTkPKrk+PXEL+dtWJHBjN9pTADbSOaTu2X5VF32TMlDupA4Okew6Y6uZpGH9uTs1HLO8PTmYUex8x7OcrEx9Bbq4uA7kanLQ59mzCl+eeauEld6fyznHN0Mi5Eherwj2q5MELC4O2vTN+WssFev1KqV2tvn5UYju0XX6RE9pifo2RfUoFe0gxPCho/P9IW+Fr4VfhR+Fn6l0NWVzOYBTTyFv/8AP8aV/Q==</latexit><latexit sha1_base64="q3TnFz6LS3HyxJx4AaVkfZpP4O8=">AAAINXicpVXLbtNQEJ22QEp4pSBWbCyiSixoSCIkuozEhgVCBTVtpbZEtnOTWvEL+5q2irLjR9jCB/AtLNghBCt+gTNjp3k7rbDVdDz3zJk7r3ut0HViXa1+X1ldu3b9RmH9ZvHW7Tt375U27u/FQRLZqmkHbhAdWGasXMdXTe1oVx2EkTI9y1X7Vu8lr+9/VFHsBP6uPg/VsWd2fafj2KaGqlV6+OZ9vdU/0upM+0HkmW4/CQeDVqlcrVTlMWaFWiaUKXt2go21T3REbQrIpoQ8UuSThuySSTHeQ6pRlULojqkPXQTJkXVFAyrCNgFKAWFC28NvF1+HmdbHN3PGYm3Di4u/CJYGbcImAC6CzN4MWU+EmbWLuPvCyXs7x38r4/Kg1XQC7TK7IfKydhyLpg5tSwwOYgpFw9HZGUsiWeGdG2NRaTCE0LHcxnoE2RbLYZ4NsYklds6tKeu/Bcla/rYzbEJ/cqOzwMoVKf5XJOzTlSqdQk7xW7AwxvCL+XfRLczdgb0/wZy+m8IZS562wWuBlbPAdsaFVV6UJ6h6mlPOUWeMe+RhhHFl171cxkRw0zyTjDF4PFimFW7SO3ot9U39MKuWDvDBmVehAG9P+sICYt7eQ+wmQFxKOtyRjHBFtugDWE2JiP0aGUvaP8t6fpTZeT4tmT+24h1yH7ahHU6GcTFpPL15vnw5GbgqaSfP88Uz7mWWeoY77QGelxo9ozo9haTkPKrk+PXEL+dtWJHBjN9pTADbSOaTu2X5VF32TMlDupA4Okew6Y6uZpGH9uTs1HLO8PTmYUex8x7OcrEx9Bbq4uA7kanLQ59mzCl+eeauEld6fyznHN0Mi5Eherwj2q5MELC4O2vTN+WssFev1KqV2tvn5UYju0XX6RE9pifo2RfUoFe0gxPCho/P9IW+Fr4VfhR+Fn6l0NWVzOYBTTyFv/8AP8aV/Q==</latexit><latexit sha1_base64="q3TnFz6LS3HyxJx4AaVkfZpP4O8=">AAAINXicpVXLbtNQEJ22QEp4pSBWbCyiSixoSCIkuozEhgVCBTVtpbZEtnOTWvEL+5q2irLjR9jCB/AtLNghBCt+gTNjp3k7rbDVdDz3zJk7r3ut0HViXa1+X1ldu3b9RmH9ZvHW7Tt375U27u/FQRLZqmkHbhAdWGasXMdXTe1oVx2EkTI9y1X7Vu8lr+9/VFHsBP6uPg/VsWd2fafj2KaGqlV6+OZ9vdU/0upM+0HkmW4/CQeDVqlcrVTlMWaFWiaUKXt2go21T3REbQrIpoQ8UuSThuySSTHeQ6pRlULojqkPXQTJkXVFAyrCNgFKAWFC28NvF1+HmdbHN3PGYm3Di4u/CJYGbcImAC6CzN4MWU+EmbWLuPvCyXs7x38r4/Kg1XQC7TK7IfKydhyLpg5tSwwOYgpFw9HZGUsiWeGdG2NRaTCE0LHcxnoE2RbLYZ4NsYklds6tKeu/Bcla/rYzbEJ/cqOzwMoVKf5XJOzTlSqdQk7xW7AwxvCL+XfRLczdgb0/wZy+m8IZS562wWuBlbPAdsaFVV6UJ6h6mlPOUWeMe+RhhHFl171cxkRw0zyTjDF4PFimFW7SO3ot9U39MKuWDvDBmVehAG9P+sICYt7eQ+wmQFxKOtyRjHBFtugDWE2JiP0aGUvaP8t6fpTZeT4tmT+24h1yH7ahHU6GcTFpPL15vnw5GbgqaSfP88Uz7mWWeoY77QGelxo9ozo9haTkPKrk+PXEL+dtWJHBjN9pTADbSOaTu2X5VF32TMlDupA4Okew6Y6uZpGH9uTs1HLO8PTmYUex8x7OcrEx9Bbq4uA7kanLQ59mzCl+eeauEld6fyznHN0Mi5Eherwj2q5MELC4O2vTN+WssFev1KqV2tvn5UYju0XX6RE9pifo2RfUoFe0gxPCho/P9IW+Fr4VfhR+Fn6l0NWVzOYBTTyFv/8AP8aV/Q==</latexit><latexit sha1_base64="q3TnFz6LS3HyxJx4AaVkfZpP4O8=">AAAINXicpVXLbtNQEJ22QEp4pSBWbCyiSixoSCIkuozEhgVCBTVtpbZEtnOTWvEL+5q2irLjR9jCB/AtLNghBCt+gTNjp3k7rbDVdDz3zJk7r3ut0HViXa1+X1ldu3b9RmH9ZvHW7Tt375U27u/FQRLZqmkHbhAdWGasXMdXTe1oVx2EkTI9y1X7Vu8lr+9/VFHsBP6uPg/VsWd2fafj2KaGqlV6+OZ9vdU/0upM+0HkmW4/CQeDVqlcrVTlMWaFWiaUKXt2go21T3REbQrIpoQ8UuSThuySSTHeQ6pRlULojqkPXQTJkXVFAyrCNgFKAWFC28NvF1+HmdbHN3PGYm3Di4u/CJYGbcImAC6CzN4MWU+EmbWLuPvCyXs7x38r4/Kg1XQC7TK7IfKydhyLpg5tSwwOYgpFw9HZGUsiWeGdG2NRaTCE0LHcxnoE2RbLYZ4NsYklds6tKeu/Bcla/rYzbEJ/cqOzwMoVKf5XJOzTlSqdQk7xW7AwxvCL+XfRLczdgb0/wZy+m8IZS562wWuBlbPAdsaFVV6UJ6h6mlPOUWeMe+RhhHFl171cxkRw0zyTjDF4PFimFW7SO3ot9U39MKuWDvDBmVehAG9P+sICYt7eQ+wmQFxKOtyRjHBFtugDWE2JiP0aGUvaP8t6fpTZeT4tmT+24h1yH7ahHU6GcTFpPL15vnw5GbgqaSfP88Uz7mWWeoY77QGelxo9ozo9haTkPKrk+PXEL+dtWJHBjN9pTADbSOaTu2X5VF32TMlDupA4Okew6Y6uZpGH9uTs1HLO8PTmYUex8x7OcrEx9Bbq4uA7kanLQ59mzCl+eeauEld6fyznHN0Mi5Eherwj2q5MELC4O2vTN+WssFev1KqV2tvn5UYju0XX6RE9pifo2RfUoFe0gxPCho/P9IW+Fr4VfhR+Fn6l0NWVzOYBTTyFv/8AP8aV/Q==</latexit>

Nup<latexit sha1_base64="yhU64bt9VoWUvaXwaZuoU7F/ELs=">AAAIM3icpVXLbtNQEJ22QEp4NKVLNhZRJRY0xBUSXUZiwwKhgpq2UltFtnPTWvEL+5q2irLjP9jCB/AxiB1iAQv+gTNjp3k7rbDVdDz3zJk7r3vtyHMTXa9/X1peuXX7Tmn1bvne/QcP1yrrj/aTMI0d1XRCL4wPbStRnhuopna1pw6jWFm+7akDu/uK1w8+qjhxw2BPX0bqxLdOA7fjOpaGqlXZeNvqHWt1oYMw9i2vl0b9fqtSrdfq8hjTgpkLVcqf3XB95RMdU5tCciglnxQFpCF7ZFGC94hMqlME3Qn1oIshubKuqE9l2KZAKSAsaLv4PcXXUa4N8M2ciVg78OLhL4alQZuwCYGLIbM3Q9ZTYWbtPO6ecPLeLvHfzrl8aDWdQbvIboC8rh3HoqlDOxKDi5gi0XB0Ts6SSlZ458ZIVBoMEXQst7EeQ3bEcpBnQ2wSiZ1za8n6b0Gylr+dHJvSn8LobLByRcr/FQn79KRK55Az/BYsjBH8fP49dAtzd2AfjDFn76ZwJpKnHfDaYOUssJ1xZVUU5RmqnuWUc9QZ4R56GGI82XW3kDEV3CTPOGMCHh+WWYWb9J7eSH0zP8yqpQMCcBZVKMTblb6wgZi19wi7CRGXkg53JSNckS36AFZLImK/Rs6S9c+inh9mdpZPW+aPrXiH3IdtaAeTYVxNGk9vka9ATgauStbJs3zxjPu5pZ7iznqA58Wk57RNzyApOY9qBX598ct5G1SkP+V3EhPCNpb55G5ZPFXXPVOKkB4kjs4VbLajm1kUoX05O7WcMzy9Rdhh7LyHi0JsAr2Nurj4TmXqitDnOXOGX5y5m8SV3R+LOYc3w3xkhB7viPZUJghY3J3m5E05Lexv18x6zXz3otpo5LfoKj2mJ/QUPfuSGvSadnFCONjDZ/pCX0vfSj9KP0u/MujyUm6zQWNP6e8/hHOVWQ==</latexit><latexit sha1_base64="yhU64bt9VoWUvaXwaZuoU7F/ELs=">AAAIM3icpVXLbtNQEJ22QEp4NKVLNhZRJRY0xBUSXUZiwwKhgpq2UltFtnPTWvEL+5q2irLjP9jCB/AxiB1iAQv+gTNjp3k7rbDVdDz3zJk7r3vtyHMTXa9/X1peuXX7Tmn1bvne/QcP1yrrj/aTMI0d1XRCL4wPbStRnhuopna1pw6jWFm+7akDu/uK1w8+qjhxw2BPX0bqxLdOA7fjOpaGqlXZeNvqHWt1oYMw9i2vl0b9fqtSrdfq8hjTgpkLVcqf3XB95RMdU5tCciglnxQFpCF7ZFGC94hMqlME3Qn1oIshubKuqE9l2KZAKSAsaLv4PcXXUa4N8M2ciVg78OLhL4alQZuwCYGLIbM3Q9ZTYWbtPO6ecPLeLvHfzrl8aDWdQbvIboC8rh3HoqlDOxKDi5gi0XB0Ts6SSlZ458ZIVBoMEXQst7EeQ3bEcpBnQ2wSiZ1za8n6b0Gylr+dHJvSn8LobLByRcr/FQn79KRK55Az/BYsjBH8fP49dAtzd2AfjDFn76ZwJpKnHfDaYOUssJ1xZVUU5RmqnuWUc9QZ4R56GGI82XW3kDEV3CTPOGMCHh+WWYWb9J7eSH0zP8yqpQMCcBZVKMTblb6wgZi19wi7CRGXkg53JSNckS36AFZLImK/Rs6S9c+inh9mdpZPW+aPrXiH3IdtaAeTYVxNGk9vka9ATgauStbJs3zxjPu5pZ7iznqA58Wk57RNzyApOY9qBX598ct5G1SkP+V3EhPCNpb55G5ZPFXXPVOKkB4kjs4VbLajm1kUoX05O7WcMzy9Rdhh7LyHi0JsAr2Nurj4TmXqitDnOXOGX5y5m8SV3R+LOYc3w3xkhB7viPZUJghY3J3m5E05Lexv18x6zXz3otpo5LfoKj2mJ/QUPfuSGvSadnFCONjDZ/pCX0vfSj9KP0u/MujyUm6zQWNP6e8/hHOVWQ==</latexit><latexit sha1_base64="yhU64bt9VoWUvaXwaZuoU7F/ELs=">AAAIM3icpVXLbtNQEJ22QEp4NKVLNhZRJRY0xBUSXUZiwwKhgpq2UltFtnPTWvEL+5q2irLjP9jCB/AxiB1iAQv+gTNjp3k7rbDVdDz3zJk7r3vtyHMTXa9/X1peuXX7Tmn1bvne/QcP1yrrj/aTMI0d1XRCL4wPbStRnhuopna1pw6jWFm+7akDu/uK1w8+qjhxw2BPX0bqxLdOA7fjOpaGqlXZeNvqHWt1oYMw9i2vl0b9fqtSrdfq8hjTgpkLVcqf3XB95RMdU5tCciglnxQFpCF7ZFGC94hMqlME3Qn1oIshubKuqE9l2KZAKSAsaLv4PcXXUa4N8M2ciVg78OLhL4alQZuwCYGLIbM3Q9ZTYWbtPO6ecPLeLvHfzrl8aDWdQbvIboC8rh3HoqlDOxKDi5gi0XB0Ts6SSlZ458ZIVBoMEXQst7EeQ3bEcpBnQ2wSiZ1za8n6b0Gylr+dHJvSn8LobLByRcr/FQn79KRK55Az/BYsjBH8fP49dAtzd2AfjDFn76ZwJpKnHfDaYOUssJ1xZVUU5RmqnuWUc9QZ4R56GGI82XW3kDEV3CTPOGMCHh+WWYWb9J7eSH0zP8yqpQMCcBZVKMTblb6wgZi19wi7CRGXkg53JSNckS36AFZLImK/Rs6S9c+inh9mdpZPW+aPrXiH3IdtaAeTYVxNGk9vka9ATgauStbJs3zxjPu5pZ7iznqA58Wk57RNzyApOY9qBX598ct5G1SkP+V3EhPCNpb55G5ZPFXXPVOKkB4kjs4VbLajm1kUoX05O7WcMzy9Rdhh7LyHi0JsAr2Nurj4TmXqitDnOXOGX5y5m8SV3R+LOYc3w3xkhB7viPZUJghY3J3m5E05Lexv18x6zXz3otpo5LfoKj2mJ/QUPfuSGvSadnFCONjDZ/pCX0vfSj9KP0u/MujyUm6zQWNP6e8/hHOVWQ==</latexit><latexit sha1_base64="yhU64bt9VoWUvaXwaZuoU7F/ELs=">AAAIM3icpVXLbtNQEJ22QEp4NKVLNhZRJRY0xBUSXUZiwwKhgpq2UltFtnPTWvEL+5q2irLjP9jCB/AxiB1iAQv+gTNjp3k7rbDVdDz3zJk7r3vtyHMTXa9/X1peuXX7Tmn1bvne/QcP1yrrj/aTMI0d1XRCL4wPbStRnhuopna1pw6jWFm+7akDu/uK1w8+qjhxw2BPX0bqxLdOA7fjOpaGqlXZeNvqHWt1oYMw9i2vl0b9fqtSrdfq8hjTgpkLVcqf3XB95RMdU5tCciglnxQFpCF7ZFGC94hMqlME3Qn1oIshubKuqE9l2KZAKSAsaLv4PcXXUa4N8M2ciVg78OLhL4alQZuwCYGLIbM3Q9ZTYWbtPO6ecPLeLvHfzrl8aDWdQbvIboC8rh3HoqlDOxKDi5gi0XB0Ts6SSlZ458ZIVBoMEXQst7EeQ3bEcpBnQ2wSiZ1za8n6b0Gylr+dHJvSn8LobLByRcr/FQn79KRK55Az/BYsjBH8fP49dAtzd2AfjDFn76ZwJpKnHfDaYOUssJ1xZVUU5RmqnuWUc9QZ4R56GGI82XW3kDEV3CTPOGMCHh+WWYWb9J7eSH0zP8yqpQMCcBZVKMTblb6wgZi19wi7CRGXkg53JSNckS36AFZLImK/Rs6S9c+inh9mdpZPW+aPrXiH3IdtaAeTYVxNGk9vka9ATgauStbJs3zxjPu5pZ7iznqA58Wk57RNzyApOY9qBX598ct5G1SkP+V3EhPCNpb55G5ZPFXXPVOKkB4kjs4VbLajm1kUoX05O7WcMzy9Rdhh7LyHi0JsAr2Nurj4TmXqitDnOXOGX5y5m8SV3R+LOYc3w3xkhB7viPZUJghY3J3m5E05Lexv18x6zXz3otpo5LfoKj2mJ/QUPfuSGvSadnFCONjDZ/pCX0vfSj9KP0u/MujyUm6zQWNP6e8/hHOVWQ==</latexit>

NH<latexit sha1_base64="Y9kVeCvcL/nBr35sK+EdzmGBktw=">AAAIMnicpVXLbtNQEJ22gEt4pWXJxiKqxIKGpEKiywo2XSBUUNNWaktlOzepFb+wr2mrKDu+gy18AD8DO8QGJD6CM2OneTutsNV0PPfMmTuve+3IcxNdq31fWFy6cfOWsXy7dOfuvfsPyiure0mYxo5qOKEXxge2lSjPDVRDu9pTB1GsLN/21L7decXr+x9VnLhhsKsvInXsW+3AbbmOpaE6Ka++ed890upcB2HsW153u9c7KVdq1Zo85qRQz4UK5c9OuLL0iY6oSSE5lJJPigLSkD2yKMF7SHWqUQTdMXWhiyG5sq6oRyXYpkApICxoO/ht4+sw1wb4Zs5ErB148fAXw9KkNdiEwMWQ2Zsp66kws3YWd1c4eW8X+G/nXD60mk6hnWfXR17VjmPR1KJNicFFTJFoODonZ0klK7xzcygqDYYIOpabWI8hO2LZz7MpNonEzrm1ZP23IFnL306OTelPYXQ2WLkipf+KhH16UqUzyBl+HRbmEH42/y66hblbsA9GmLN3TTgTydMmeG2wchbYzry0KoryFFXPcso5ag1xDzwMMJ7sulPImApunGeUMQGPD8uswg16R6+lvpkfZtXSAQE4iyoU4u1IX9hATNt7hN2EiEtJh7uSEa7IOn0AqyURsV8zZ8n6Z17PDzI7zact88dWvEPuwya0/ckwLyeNp7fIVyAnA1cl6+RpvnjG/dxST3BnPcDzUqdntEFPISk5j6oFfn3xy3nrV6Q34XccE8I2lvnkbpk/VVc9U4qQHiSOzhVstqPrWRShfTk7tZwzPL1F2EHsvIfzQmwCvY26uPhOZeqK0Gc5c4afn7nrxJXdH/M5BzfDbGSEHm+Jti0TBCzuzvr4TTkp7G1U67Vq/e3zytbL/BZdpkf0mJ6gZ1/QFm3TDk4Izupn+kJfjW/GD+On8SuDLi7kNg9p5DH+/gOL8JSz</latexit><latexit sha1_base64="Y9kVeCvcL/nBr35sK+EdzmGBktw=">AAAIMnicpVXLbtNQEJ22gEt4pWXJxiKqxIKGpEKiywo2XSBUUNNWaktlOzepFb+wr2mrKDu+gy18AD8DO8QGJD6CM2OneTutsNV0PPfMmTuve+3IcxNdq31fWFy6cfOWsXy7dOfuvfsPyiure0mYxo5qOKEXxge2lSjPDVRDu9pTB1GsLN/21L7decXr+x9VnLhhsKsvInXsW+3AbbmOpaE6Ka++ed890upcB2HsW153u9c7KVdq1Zo85qRQz4UK5c9OuLL0iY6oSSE5lJJPigLSkD2yKMF7SHWqUQTdMXWhiyG5sq6oRyXYpkApICxoO/ht4+sw1wb4Zs5ErB148fAXw9KkNdiEwMWQ2Zsp66kws3YWd1c4eW8X+G/nXD60mk6hnWfXR17VjmPR1KJNicFFTJFoODonZ0klK7xzcygqDYYIOpabWI8hO2LZz7MpNonEzrm1ZP23IFnL306OTelPYXQ2WLkipf+KhH16UqUzyBl+HRbmEH42/y66hblbsA9GmLN3TTgTydMmeG2wchbYzry0KoryFFXPcso5ag1xDzwMMJ7sulPImApunGeUMQGPD8uswg16R6+lvpkfZtXSAQE4iyoU4u1IX9hATNt7hN2EiEtJh7uSEa7IOn0AqyURsV8zZ8n6Z17PDzI7zact88dWvEPuwya0/ckwLyeNp7fIVyAnA1cl6+RpvnjG/dxST3BnPcDzUqdntEFPISk5j6oFfn3xy3nrV6Q34XccE8I2lvnkbpk/VVc9U4qQHiSOzhVstqPrWRShfTk7tZwzPL1F2EHsvIfzQmwCvY26uPhOZeqK0Gc5c4afn7nrxJXdH/M5BzfDbGSEHm+Jti0TBCzuzvr4TTkp7G1U67Vq/e3zytbL/BZdpkf0mJ6gZ1/QFm3TDk4Izupn+kJfjW/GD+On8SuDLi7kNg9p5DH+/gOL8JSz</latexit><latexit sha1_base64="Y9kVeCvcL/nBr35sK+EdzmGBktw=">AAAIMnicpVXLbtNQEJ22gEt4pWXJxiKqxIKGpEKiywo2XSBUUNNWaktlOzepFb+wr2mrKDu+gy18AD8DO8QGJD6CM2OneTutsNV0PPfMmTuve+3IcxNdq31fWFy6cfOWsXy7dOfuvfsPyiure0mYxo5qOKEXxge2lSjPDVRDu9pTB1GsLN/21L7decXr+x9VnLhhsKsvInXsW+3AbbmOpaE6Ka++ed890upcB2HsW153u9c7KVdq1Zo85qRQz4UK5c9OuLL0iY6oSSE5lJJPigLSkD2yKMF7SHWqUQTdMXWhiyG5sq6oRyXYpkApICxoO/ht4+sw1wb4Zs5ErB148fAXw9KkNdiEwMWQ2Zsp66kws3YWd1c4eW8X+G/nXD60mk6hnWfXR17VjmPR1KJNicFFTJFoODonZ0klK7xzcygqDYYIOpabWI8hO2LZz7MpNonEzrm1ZP23IFnL306OTelPYXQ2WLkipf+KhH16UqUzyBl+HRbmEH42/y66hblbsA9GmLN3TTgTydMmeG2wchbYzry0KoryFFXPcso5ag1xDzwMMJ7sulPImApunGeUMQGPD8uswg16R6+lvpkfZtXSAQE4iyoU4u1IX9hATNt7hN2EiEtJh7uSEa7IOn0AqyURsV8zZ8n6Z17PDzI7zact88dWvEPuwya0/ckwLyeNp7fIVyAnA1cl6+RpvnjG/dxST3BnPcDzUqdntEFPISk5j6oFfn3xy3nrV6Q34XccE8I2lvnkbpk/VVc9U4qQHiSOzhVstqPrWRShfTk7tZwzPL1F2EHsvIfzQmwCvY26uPhOZeqK0Gc5c4afn7nrxJXdH/M5BzfDbGSEHm+Jti0TBCzuzvr4TTkp7G1U67Vq/e3zytbL/BZdpkf0mJ6gZ1/QFm3TDk4Izupn+kJfjW/GD+On8SuDLi7kNg9p5DH+/gOL8JSz</latexit><latexit sha1_base64="Y9kVeCvcL/nBr35sK+EdzmGBktw=">AAAIMnicpVXLbtNQEJ22gEt4pWXJxiKqxIKGpEKiywo2XSBUUNNWaktlOzepFb+wr2mrKDu+gy18AD8DO8QGJD6CM2OneTutsNV0PPfMmTuve+3IcxNdq31fWFy6cfOWsXy7dOfuvfsPyiure0mYxo5qOKEXxge2lSjPDVRDu9pTB1GsLN/21L7decXr+x9VnLhhsKsvInXsW+3AbbmOpaE6Ka++ed890upcB2HsW153u9c7KVdq1Zo85qRQz4UK5c9OuLL0iY6oSSE5lJJPigLSkD2yKMF7SHWqUQTdMXWhiyG5sq6oRyXYpkApICxoO/ht4+sw1wb4Zs5ErB148fAXw9KkNdiEwMWQ2Zsp66kws3YWd1c4eW8X+G/nXD60mk6hnWfXR17VjmPR1KJNicFFTJFoODonZ0klK7xzcygqDYYIOpabWI8hO2LZz7MpNonEzrm1ZP23IFnL306OTelPYXQ2WLkipf+KhH16UqUzyBl+HRbmEH42/y66hblbsA9GmLN3TTgTydMmeG2wchbYzry0KoryFFXPcso5ag1xDzwMMJ7sulPImApunGeUMQGPD8uswg16R6+lvpkfZtXSAQE4iyoU4u1IX9hATNt7hN2EiEtJh7uSEa7IOn0AqyURsV8zZ8n6Z17PDzI7zact88dWvEPuwya0/ckwLyeNp7fIVyAnA1cl6+RpvnjG/dxST3BnPcDzUqdntEFPISk5j6oFfn3xy3nrV6Q34XccE8I2lvnkbpk/VVc9U4qQHiSOzhVstqPrWRShfTk7tZwzPL1F2EHsvIfzQmwCvY26uPhOZeqK0Gc5c4afn7nrxJXdH/M5BzfDbGSEHm+Jti0TBCzuzvr4TTkp7G1U67Vq/e3zytbL/BZdpkf0mJ6gZ1/QFm3TDk4Izupn+kJfjW/GD+On8SuDLi7kNg9p5DH+/gOL8JSz</latexit>

Reshape

NH<latexit sha1_base64="Y9kVeCvcL/nBr35sK+EdzmGBktw=">AAAIMnicpVXLbtNQEJ22gEt4pWXJxiKqxIKGpEKiywo2XSBUUNNWaktlOzepFb+wr2mrKDu+gy18AD8DO8QGJD6CM2OneTutsNV0PPfMmTuve+3IcxNdq31fWFy6cfOWsXy7dOfuvfsPyiure0mYxo5qOKEXxge2lSjPDVRDu9pTB1GsLN/21L7decXr+x9VnLhhsKsvInXsW+3AbbmOpaE6Ka++ed890upcB2HsW153u9c7KVdq1Zo85qRQz4UK5c9OuLL0iY6oSSE5lJJPigLSkD2yKMF7SHWqUQTdMXWhiyG5sq6oRyXYpkApICxoO/ht4+sw1wb4Zs5ErB148fAXw9KkNdiEwMWQ2Zsp66kws3YWd1c4eW8X+G/nXD60mk6hnWfXR17VjmPR1KJNicFFTJFoODonZ0klK7xzcygqDYYIOpabWI8hO2LZz7MpNonEzrm1ZP23IFnL306OTelPYXQ2WLkipf+KhH16UqUzyBl+HRbmEH42/y66hblbsA9GmLN3TTgTydMmeG2wchbYzry0KoryFFXPcso5ag1xDzwMMJ7sulPImApunGeUMQGPD8uswg16R6+lvpkfZtXSAQE4iyoU4u1IX9hATNt7hN2EiEtJh7uSEa7IOn0AqyURsV8zZ8n6Z17PDzI7zact88dWvEPuwya0/ckwLyeNp7fIVyAnA1cl6+RpvnjG/dxST3BnPcDzUqdntEFPISk5j6oFfn3xy3nrV6Q34XccE8I2lvnkbpk/VVc9U4qQHiSOzhVstqPrWRShfTk7tZwzPL1F2EHsvIfzQmwCvY26uPhOZeqK0Gc5c4afn7nrxJXdH/M5BzfDbGSEHm+Jti0TBCzuzvr4TTkp7G1U67Vq/e3zytbL/BZdpkf0mJ6gZ1/QFm3TDk4Izupn+kJfjW/GD+On8SuDLi7kNg9p5DH+/gOL8JSz</latexit><latexit sha1_base64="Y9kVeCvcL/nBr35sK+EdzmGBktw=">AAAIMnicpVXLbtNQEJ22gEt4pWXJxiKqxIKGpEKiywo2XSBUUNNWaktlOzepFb+wr2mrKDu+gy18AD8DO8QGJD6CM2OneTutsNV0PPfMmTuve+3IcxNdq31fWFy6cfOWsXy7dOfuvfsPyiure0mYxo5qOKEXxge2lSjPDVRDu9pTB1GsLN/21L7decXr+x9VnLhhsKsvInXsW+3AbbmOpaE6Ka++ed890upcB2HsW153u9c7KVdq1Zo85qRQz4UK5c9OuLL0iY6oSSE5lJJPigLSkD2yKMF7SHWqUQTdMXWhiyG5sq6oRyXYpkApICxoO/ht4+sw1wb4Zs5ErB148fAXw9KkNdiEwMWQ2Zsp66kws3YWd1c4eW8X+G/nXD60mk6hnWfXR17VjmPR1KJNicFFTJFoODonZ0klK7xzcygqDYYIOpabWI8hO2LZz7MpNonEzrm1ZP23IFnL306OTelPYXQ2WLkipf+KhH16UqUzyBl+HRbmEH42/y66hblbsA9GmLN3TTgTydMmeG2wchbYzry0KoryFFXPcso5ag1xDzwMMJ7sulPImApunGeUMQGPD8uswg16R6+lvpkfZtXSAQE4iyoU4u1IX9hATNt7hN2EiEtJh7uSEa7IOn0AqyURsV8zZ8n6Z17PDzI7zact88dWvEPuwya0/ckwLyeNp7fIVyAnA1cl6+RpvnjG/dxST3BnPcDzUqdntEFPISk5j6oFfn3xy3nrV6Q34XccE8I2lvnkbpk/VVc9U4qQHiSOzhVstqPrWRShfTk7tZwzPL1F2EHsvIfzQmwCvY26uPhOZeqK0Gc5c4afn7nrxJXdH/M5BzfDbGSEHm+Jti0TBCzuzvr4TTkp7G1U67Vq/e3zytbL/BZdpkf0mJ6gZ1/QFm3TDk4Izupn+kJfjW/GD+On8SuDLi7kNg9p5DH+/gOL8JSz</latexit><latexit sha1_base64="Y9kVeCvcL/nBr35sK+EdzmGBktw=">AAAIMnicpVXLbtNQEJ22gEt4pWXJxiKqxIKGpEKiywo2XSBUUNNWaktlOzepFb+wr2mrKDu+gy18AD8DO8QGJD6CM2OneTutsNV0PPfMmTuve+3IcxNdq31fWFy6cfOWsXy7dOfuvfsPyiure0mYxo5qOKEXxge2lSjPDVRDu9pTB1GsLN/21L7decXr+x9VnLhhsKsvInXsW+3AbbmOpaE6Ka++ed890upcB2HsW153u9c7KVdq1Zo85qRQz4UK5c9OuLL0iY6oSSE5lJJPigLSkD2yKMF7SHWqUQTdMXWhiyG5sq6oRyXYpkApICxoO/ht4+sw1wb4Zs5ErB148fAXw9KkNdiEwMWQ2Zsp66kws3YWd1c4eW8X+G/nXD60mk6hnWfXR17VjmPR1KJNicFFTJFoODonZ0klK7xzcygqDYYIOpabWI8hO2LZz7MpNonEzrm1ZP23IFnL306OTelPYXQ2WLkipf+KhH16UqUzyBl+HRbmEH42/y66hblbsA9GmLN3TTgTydMmeG2wchbYzry0KoryFFXPcso5ag1xDzwMMJ7sulPImApunGeUMQGPD8uswg16R6+lvpkfZtXSAQE4iyoU4u1IX9hATNt7hN2EiEtJh7uSEa7IOn0AqyURsV8zZ8n6Z17PDzI7zact88dWvEPuwya0/ckwLyeNp7fIVyAnA1cl6+RpvnjG/dxST3BnPcDzUqdntEFPISk5j6oFfn3xy3nrV6Q34XccE8I2lvnkbpk/VVc9U4qQHiSOzhVstqPrWRShfTk7tZwzPL1F2EHsvIfzQmwCvY26uPhOZeqK0Gc5c4afn7nrxJXdH/M5BzfDbGSEHm+Jti0TBCzuzvr4TTkp7G1U67Vq/e3zytbL/BZdpkf0mJ6gZ1/QFm3TDk4Izupn+kJfjW/GD+On8SuDLi7kNg9p5DH+/gOL8JSz</latexit><latexit sha1_base64="Y9kVeCvcL/nBr35sK+EdzmGBktw=">AAAIMnicpVXLbtNQEJ22gEt4pWXJxiKqxIKGpEKiywo2XSBUUNNWaktlOzepFb+wr2mrKDu+gy18AD8DO8QGJD6CM2OneTutsNV0PPfMmTuve+3IcxNdq31fWFy6cfOWsXy7dOfuvfsPyiure0mYxo5qOKEXxge2lSjPDVRDu9pTB1GsLN/21L7decXr+x9VnLhhsKsvInXsW+3AbbmOpaE6Ka++ed890upcB2HsW153u9c7KVdq1Zo85qRQz4UK5c9OuLL0iY6oSSE5lJJPigLSkD2yKMF7SHWqUQTdMXWhiyG5sq6oRyXYpkApICxoO/ht4+sw1wb4Zs5ErB148fAXw9KkNdiEwMWQ2Zsp66kws3YWd1c4eW8X+G/nXD60mk6hnWfXR17VjmPR1KJNicFFTJFoODonZ0klK7xzcygqDYYIOpabWI8hO2LZz7MpNonEzrm1ZP23IFnL306OTelPYXQ2WLkipf+KhH16UqUzyBl+HRbmEH42/y66hblbsA9GmLN3TTgTydMmeG2wchbYzry0KoryFFXPcso5ag1xDzwMMJ7sulPImApunGeUMQGPD8uswg16R6+lvpkfZtXSAQE4iyoU4u1IX9hATNt7hN2EiEtJh7uSEa7IOn0AqyURsV8zZ8n6Z17PDzI7zact88dWvEPuwya0/ckwLyeNp7fIVyAnA1cl6+RpvnjG/dxST3BnPcDzUqdntEFPISk5j6oFfn3xy3nrV6Q34XccE8I2lvnkbpk/VVc9U4qQHiSOzhVstqPrWRShfTk7tZwzPL1F2EHsvIfzQmwCvY26uPhOZeqK0Gc5c4afn7nrxJXdH/M5BzfDbGSEHm+Jti0TBCzuzvr4TTkp7G1U67Vq/e3zytbL/BZdpkf0mJ6gZ1/QFm3TDk4Izupn+kJfjW/GD+On8SuDLi7kNg9p5DH+/gOL8JSz</latexit>

Upsample

NH<latexit sha1_base64="Y9kVeCvcL/nBr35sK+EdzmGBktw=">AAAIMnicpVXLbtNQEJ22gEt4pWXJxiKqxIKGpEKiywo2XSBUUNNWaktlOzepFb+wr2mrKDu+gy18AD8DO8QGJD6CM2OneTutsNV0PPfMmTuve+3IcxNdq31fWFy6cfOWsXy7dOfuvfsPyiure0mYxo5qOKEXxge2lSjPDVRDu9pTB1GsLN/21L7decXr+x9VnLhhsKsvInXsW+3AbbmOpaE6Ka++ed890upcB2HsW153u9c7KVdq1Zo85qRQz4UK5c9OuLL0iY6oSSE5lJJPigLSkD2yKMF7SHWqUQTdMXWhiyG5sq6oRyXYpkApICxoO/ht4+sw1wb4Zs5ErB148fAXw9KkNdiEwMWQ2Zsp66kws3YWd1c4eW8X+G/nXD60mk6hnWfXR17VjmPR1KJNicFFTJFoODonZ0klK7xzcygqDYYIOpabWI8hO2LZz7MpNonEzrm1ZP23IFnL306OTelPYXQ2WLkipf+KhH16UqUzyBl+HRbmEH42/y66hblbsA9GmLN3TTgTydMmeG2wchbYzry0KoryFFXPcso5ag1xDzwMMJ7sulPImApunGeUMQGPD8uswg16R6+lvpkfZtXSAQE4iyoU4u1IX9hATNt7hN2EiEtJh7uSEa7IOn0AqyURsV8zZ8n6Z17PDzI7zact88dWvEPuwya0/ckwLyeNp7fIVyAnA1cl6+RpvnjG/dxST3BnPcDzUqdntEFPISk5j6oFfn3xy3nrV6Q34XccE8I2lvnkbpk/VVc9U4qQHiSOzhVstqPrWRShfTk7tZwzPL1F2EHsvIfzQmwCvY26uPhOZeqK0Gc5c4afn7nrxJXdH/M5BzfDbGSEHm+Jti0TBCzuzvr4TTkp7G1U67Vq/e3zytbL/BZdpkf0mJ6gZ1/QFm3TDk4Izupn+kJfjW/GD+On8SuDLi7kNg9p5DH+/gOL8JSz</latexit><latexit sha1_base64="Y9kVeCvcL/nBr35sK+EdzmGBktw=">AAAIMnicpVXLbtNQEJ22gEt4pWXJxiKqxIKGpEKiywo2XSBUUNNWaktlOzepFb+wr2mrKDu+gy18AD8DO8QGJD6CM2OneTutsNV0PPfMmTuve+3IcxNdq31fWFy6cfOWsXy7dOfuvfsPyiure0mYxo5qOKEXxge2lSjPDVRDu9pTB1GsLN/21L7decXr+x9VnLhhsKsvInXsW+3AbbmOpaE6Ka++ed890upcB2HsW153u9c7KVdq1Zo85qRQz4UK5c9OuLL0iY6oSSE5lJJPigLSkD2yKMF7SHWqUQTdMXWhiyG5sq6oRyXYpkApICxoO/ht4+sw1wb4Zs5ErB148fAXw9KkNdiEwMWQ2Zsp66kws3YWd1c4eW8X+G/nXD60mk6hnWfXR17VjmPR1KJNicFFTJFoODonZ0klK7xzcygqDYYIOpabWI8hO2LZz7MpNonEzrm1ZP23IFnL306OTelPYXQ2WLkipf+KhH16UqUzyBl+HRbmEH42/y66hblbsA9GmLN3TTgTydMmeG2wchbYzry0KoryFFXPcso5ag1xDzwMMJ7sulPImApunGeUMQGPD8uswg16R6+lvpkfZtXSAQE4iyoU4u1IX9hATNt7hN2EiEtJh7uSEa7IOn0AqyURsV8zZ8n6Z17PDzI7zact88dWvEPuwya0/ckwLyeNp7fIVyAnA1cl6+RpvnjG/dxST3BnPcDzUqdntEFPISk5j6oFfn3xy3nrV6Q34XccE8I2lvnkbpk/VVc9U4qQHiSOzhVstqPrWRShfTk7tZwzPL1F2EHsvIfzQmwCvY26uPhOZeqK0Gc5c4afn7nrxJXdH/M5BzfDbGSEHm+Jti0TBCzuzvr4TTkp7G1U67Vq/e3zytbL/BZdpkf0mJ6gZ1/QFm3TDk4Izupn+kJfjW/GD+On8SuDLi7kNg9p5DH+/gOL8JSz</latexit><latexit sha1_base64="Y9kVeCvcL/nBr35sK+EdzmGBktw=">AAAIMnicpVXLbtNQEJ22gEt4pWXJxiKqxIKGpEKiywo2XSBUUNNWaktlOzepFb+wr2mrKDu+gy18AD8DO8QGJD6CM2OneTutsNV0PPfMmTuve+3IcxNdq31fWFy6cfOWsXy7dOfuvfsPyiure0mYxo5qOKEXxge2lSjPDVRDu9pTB1GsLN/21L7decXr+x9VnLhhsKsvInXsW+3AbbmOpaE6Ka++ed890upcB2HsW153u9c7KVdq1Zo85qRQz4UK5c9OuLL0iY6oSSE5lJJPigLSkD2yKMF7SHWqUQTdMXWhiyG5sq6oRyXYpkApICxoO/ht4+sw1wb4Zs5ErB148fAXw9KkNdiEwMWQ2Zsp66kws3YWd1c4eW8X+G/nXD60mk6hnWfXR17VjmPR1KJNicFFTJFoODonZ0klK7xzcygqDYYIOpabWI8hO2LZz7MpNonEzrm1ZP23IFnL306OTelPYXQ2WLkipf+KhH16UqUzyBl+HRbmEH42/y66hblbsA9GmLN3TTgTydMmeG2wchbYzry0KoryFFXPcso5ag1xDzwMMJ7sulPImApunGeUMQGPD8uswg16R6+lvpkfZtXSAQE4iyoU4u1IX9hATNt7hN2EiEtJh7uSEa7IOn0AqyURsV8zZ8n6Z17PDzI7zact88dWvEPuwya0/ckwLyeNp7fIVyAnA1cl6+RpvnjG/dxST3BnPcDzUqdntEFPISk5j6oFfn3xy3nrV6Q34XccE8I2lvnkbpk/VVc9U4qQHiSOzhVstqPrWRShfTk7tZwzPL1F2EHsvIfzQmwCvY26uPhOZeqK0Gc5c4afn7nrxJXdH/M5BzfDbGSEHm+Jti0TBCzuzvr4TTkp7G1U67Vq/e3zytbL/BZdpkf0mJ6gZ1/QFm3TDk4Izupn+kJfjW/GD+On8SuDLi7kNg9p5DH+/gOL8JSz</latexit><latexit sha1_base64="Y9kVeCvcL/nBr35sK+EdzmGBktw=">AAAIMnicpVXLbtNQEJ22gEt4pWXJxiKqxIKGpEKiywo2XSBUUNNWaktlOzepFb+wr2mrKDu+gy18AD8DO8QGJD6CM2OneTutsNV0PPfMmTuve+3IcxNdq31fWFy6cfOWsXy7dOfuvfsPyiure0mYxo5qOKEXxge2lSjPDVRDu9pTB1GsLN/21L7decXr+x9VnLhhsKsvInXsW+3AbbmOpaE6Ka++ed890upcB2HsW153u9c7KVdq1Zo85qRQz4UK5c9OuLL0iY6oSSE5lJJPigLSkD2yKMF7SHWqUQTdMXWhiyG5sq6oRyXYpkApICxoO/ht4+sw1wb4Zs5ErB148fAXw9KkNdiEwMWQ2Zsp66kws3YWd1c4eW8X+G/nXD60mk6hnWfXR17VjmPR1KJNicFFTJFoODonZ0klK7xzcygqDYYIOpabWI8hO2LZz7MpNonEzrm1ZP23IFnL306OTelPYXQ2WLkipf+KhH16UqUzyBl+HRbmEH42/y66hblbsA9GmLN3TTgTydMmeG2wchbYzry0KoryFFXPcso5ag1xDzwMMJ7sulPImApunGeUMQGPD8uswg16R6+lvpkfZtXSAQE4iyoU4u1IX9hATNt7hN2EiEtJh7uSEa7IOn0AqyURsV8zZ8n6Z17PDzI7zact88dWvEPuwya0/ckwLyeNp7fIVyAnA1cl6+RpvnjG/dxST3BnPcDzUqdntEFPISk5j6oFfn3xy3nrV6Q34XccE8I2lvnkbpk/VVc9U4qQHiSOzhVstqPrWRShfTk7tZwzPL1F2EHsvIfzQmwCvY26uPhOZeqK0Gc5c4afn7nrxJXdH/M5BzfDbGSEHm+Jti0TBCzuzvr4TTkp7G1U67Vq/e3zytbL/BZdpkf0mJ6gZ1/QFm3TDk4Izupn+kJfjW/GD+On8SuDLi7kNg9p5DH+/gOL8JSz</latexit>

N<latexit sha1_base64="nISD5kUpa/bV6As3fiK88hp8XnM=">AAAIH3icpVVLT9RQFD6AWhwfgC7dNE5IXMg4JSayJLpxYQwYBkiQmLZzB5rpy/ZWJISdO7f6A/w17oxb/oH+C79zbod5dyC2YTg99zvfued1r5eGQa6bzYu5+YUbN29Zi7drd+7eu7+0vPJgN0+KzFctPwmTbN9zcxUGsWrpQIdqP82UG3mh2vO6r3h975PK8iCJd/Rpqg4j9ygOOoHvaqi2335YrjcbTXnsccEphTqVz1aysvCF3lObEvKpoIgUxaQhh+RSjveAHGpSCt0hnUGXQQpkXdE51WBbAKWAcKHt4vcIXwelNsY3c+Zi7cNLiL8MljatwiYBLoPM3mxZL4SZtdO4z4ST93aK/17JFUGr6RjaWXY95FXtOBZNHdqQGALElIqGo/NLlkKywju3B6LSYEihY7mN9QyyL5a9PNtik0vsnFtX1v8IkrX87ZfYgv5WRueBlStS+69I2GcoVTqBbPBrsLAH8NP5d9AtzN2BfTzEbN5V4cwlTxvg9cDKWWA7+9KqKspjVN3klHPUGeDue+hjQtl1t5KxENwozzBjDp4IlqbCLXpHb6S+xg+zaumAGJxVFUrwdqUvPCAm7T3FbhLEpaTDA8kIV2SNPoLVlYjYr12ymP6Z1fP9zE7y6cn8sRXvkPuwDW1vMuzLSePprfIVy8nAVTGdPMkXz3hUWuoxbtMDPC8OPaN1egpJyXnUqPAbiV/OW68i52N+RzEJbDOZT+6W2VN11TOlChlC4ugCwZodXc+iCh3J2anlnOHprcL2Y+c9fK7E5tB7qEuA70Kmrgp9UjIb/OzMXScuc3/M5uzfDNORKXq8I9ojmSBgcXc6ozfluLC73nCaDWf7eX3zZXmLLtIjekxP0LMvaJNe0xZOCJ6Dr/SNvls/rJ/WL+u3gc7PlTYPaeixLv4BJ2aMmA==</latexit><latexit sha1_base64="nISD5kUpa/bV6As3fiK88hp8XnM=">AAAIH3icpVVLT9RQFD6AWhwfgC7dNE5IXMg4JSayJLpxYQwYBkiQmLZzB5rpy/ZWJISdO7f6A/w17oxb/oH+C79zbod5dyC2YTg99zvfued1r5eGQa6bzYu5+YUbN29Zi7drd+7eu7+0vPJgN0+KzFctPwmTbN9zcxUGsWrpQIdqP82UG3mh2vO6r3h975PK8iCJd/Rpqg4j9ygOOoHvaqi2335YrjcbTXnsccEphTqVz1aysvCF3lObEvKpoIgUxaQhh+RSjveAHGpSCt0hnUGXQQpkXdE51WBbAKWAcKHt4vcIXwelNsY3c+Zi7cNLiL8MljatwiYBLoPM3mxZL4SZtdO4z4ST93aK/17JFUGr6RjaWXY95FXtOBZNHdqQGALElIqGo/NLlkKywju3B6LSYEihY7mN9QyyL5a9PNtik0vsnFtX1v8IkrX87ZfYgv5WRueBlStS+69I2GcoVTqBbPBrsLAH8NP5d9AtzN2BfTzEbN5V4cwlTxvg9cDKWWA7+9KqKspjVN3klHPUGeDue+hjQtl1t5KxENwozzBjDp4IlqbCLXpHb6S+xg+zaumAGJxVFUrwdqUvPCAm7T3FbhLEpaTDA8kIV2SNPoLVlYjYr12ymP6Z1fP9zE7y6cn8sRXvkPuwDW1vMuzLSePprfIVy8nAVTGdPMkXz3hUWuoxbtMDPC8OPaN1egpJyXnUqPAbiV/OW68i52N+RzEJbDOZT+6W2VN11TOlChlC4ugCwZodXc+iCh3J2anlnOHprcL2Y+c9fK7E5tB7qEuA70Kmrgp9UjIb/OzMXScuc3/M5uzfDNORKXq8I9ojmSBgcXc6ozfluLC73nCaDWf7eX3zZXmLLtIjekxP0LMvaJNe0xZOCJ6Dr/SNvls/rJ/WL+u3gc7PlTYPaeixLv4BJ2aMmA==</latexit><latexit sha1_base64="nISD5kUpa/bV6As3fiK88hp8XnM=">AAAIH3icpVVLT9RQFD6AWhwfgC7dNE5IXMg4JSayJLpxYQwYBkiQmLZzB5rpy/ZWJISdO7f6A/w17oxb/oH+C79zbod5dyC2YTg99zvfued1r5eGQa6bzYu5+YUbN29Zi7drd+7eu7+0vPJgN0+KzFctPwmTbN9zcxUGsWrpQIdqP82UG3mh2vO6r3h975PK8iCJd/Rpqg4j9ygOOoHvaqi2335YrjcbTXnsccEphTqVz1aysvCF3lObEvKpoIgUxaQhh+RSjveAHGpSCt0hnUGXQQpkXdE51WBbAKWAcKHt4vcIXwelNsY3c+Zi7cNLiL8MljatwiYBLoPM3mxZL4SZtdO4z4ST93aK/17JFUGr6RjaWXY95FXtOBZNHdqQGALElIqGo/NLlkKywju3B6LSYEihY7mN9QyyL5a9PNtik0vsnFtX1v8IkrX87ZfYgv5WRueBlStS+69I2GcoVTqBbPBrsLAH8NP5d9AtzN2BfTzEbN5V4cwlTxvg9cDKWWA7+9KqKspjVN3klHPUGeDue+hjQtl1t5KxENwozzBjDp4IlqbCLXpHb6S+xg+zaumAGJxVFUrwdqUvPCAm7T3FbhLEpaTDA8kIV2SNPoLVlYjYr12ymP6Z1fP9zE7y6cn8sRXvkPuwDW1vMuzLSePprfIVy8nAVTGdPMkXz3hUWuoxbtMDPC8OPaN1egpJyXnUqPAbiV/OW68i52N+RzEJbDOZT+6W2VN11TOlChlC4ugCwZodXc+iCh3J2anlnOHprcL2Y+c9fK7E5tB7qEuA70Kmrgp9UjIb/OzMXScuc3/M5uzfDNORKXq8I9ojmSBgcXc6ozfluLC73nCaDWf7eX3zZXmLLtIjekxP0LMvaJNe0xZOCJ6Dr/SNvls/rJ/WL+u3gc7PlTYPaeixLv4BJ2aMmA==</latexit><latexit sha1_base64="nISD5kUpa/bV6As3fiK88hp8XnM=">AAAIH3icpVVLT9RQFD6AWhwfgC7dNE5IXMg4JSayJLpxYQwYBkiQmLZzB5rpy/ZWJISdO7f6A/w17oxb/oH+C79zbod5dyC2YTg99zvfued1r5eGQa6bzYu5+YUbN29Zi7drd+7eu7+0vPJgN0+KzFctPwmTbN9zcxUGsWrpQIdqP82UG3mh2vO6r3h975PK8iCJd/Rpqg4j9ygOOoHvaqi2335YrjcbTXnsccEphTqVz1aysvCF3lObEvKpoIgUxaQhh+RSjveAHGpSCt0hnUGXQQpkXdE51WBbAKWAcKHt4vcIXwelNsY3c+Zi7cNLiL8MljatwiYBLoPM3mxZL4SZtdO4z4ST93aK/17JFUGr6RjaWXY95FXtOBZNHdqQGALElIqGo/NLlkKywju3B6LSYEihY7mN9QyyL5a9PNtik0vsnFtX1v8IkrX87ZfYgv5WRueBlStS+69I2GcoVTqBbPBrsLAH8NP5d9AtzN2BfTzEbN5V4cwlTxvg9cDKWWA7+9KqKspjVN3klHPUGeDue+hjQtl1t5KxENwozzBjDp4IlqbCLXpHb6S+xg+zaumAGJxVFUrwdqUvPCAm7T3FbhLEpaTDA8kIV2SNPoLVlYjYr12ymP6Z1fP9zE7y6cn8sRXvkPuwDW1vMuzLSePprfIVy8nAVTGdPMkXz3hUWuoxbtMDPC8OPaN1egpJyXnUqPAbiV/OW68i52N+RzEJbDOZT+6W2VN11TOlChlC4ugCwZodXc+iCh3J2anlnOHprcL2Y+c9fK7E5tB7qEuA70Kmrgp9UjIb/OzMXScuc3/M5uzfDNORKXq8I9ojmSBgcXc6ozfluLC73nCaDWf7eX3zZXmLLtIjekxP0LMvaJNe0xZOCJ6Dr/SNvls/rJ/WL+u3gc7PlTYPaeixLv4BJ2aMmA==</latexit>

N<latexit sha1_base64="nISD5kUpa/bV6As3fiK88hp8XnM=">AAAIH3icpVVLT9RQFD6AWhwfgC7dNE5IXMg4JSayJLpxYQwYBkiQmLZzB5rpy/ZWJISdO7f6A/w17oxb/oH+C79zbod5dyC2YTg99zvfued1r5eGQa6bzYu5+YUbN29Zi7drd+7eu7+0vPJgN0+KzFctPwmTbN9zcxUGsWrpQIdqP82UG3mh2vO6r3h975PK8iCJd/Rpqg4j9ygOOoHvaqi2335YrjcbTXnsccEphTqVz1aysvCF3lObEvKpoIgUxaQhh+RSjveAHGpSCt0hnUGXQQpkXdE51WBbAKWAcKHt4vcIXwelNsY3c+Zi7cNLiL8MljatwiYBLoPM3mxZL4SZtdO4z4ST93aK/17JFUGr6RjaWXY95FXtOBZNHdqQGALElIqGo/NLlkKywju3B6LSYEihY7mN9QyyL5a9PNtik0vsnFtX1v8IkrX87ZfYgv5WRueBlStS+69I2GcoVTqBbPBrsLAH8NP5d9AtzN2BfTzEbN5V4cwlTxvg9cDKWWA7+9KqKspjVN3klHPUGeDue+hjQtl1t5KxENwozzBjDp4IlqbCLXpHb6S+xg+zaumAGJxVFUrwdqUvPCAm7T3FbhLEpaTDA8kIV2SNPoLVlYjYr12ymP6Z1fP9zE7y6cn8sRXvkPuwDW1vMuzLSePprfIVy8nAVTGdPMkXz3hUWuoxbtMDPC8OPaN1egpJyXnUqPAbiV/OW68i52N+RzEJbDOZT+6W2VN11TOlChlC4ugCwZodXc+iCh3J2anlnOHprcL2Y+c9fK7E5tB7qEuA70Kmrgp9UjIb/OzMXScuc3/M5uzfDNORKXq8I9ojmSBgcXc6ozfluLC73nCaDWf7eX3zZXmLLtIjekxP0LMvaJNe0xZOCJ6Dr/SNvls/rJ/WL+u3gc7PlTYPaeixLv4BJ2aMmA==</latexit><latexit sha1_base64="nISD5kUpa/bV6As3fiK88hp8XnM=">AAAIH3icpVVLT9RQFD6AWhwfgC7dNE5IXMg4JSayJLpxYQwYBkiQmLZzB5rpy/ZWJISdO7f6A/w17oxb/oH+C79zbod5dyC2YTg99zvfued1r5eGQa6bzYu5+YUbN29Zi7drd+7eu7+0vPJgN0+KzFctPwmTbN9zcxUGsWrpQIdqP82UG3mh2vO6r3h975PK8iCJd/Rpqg4j9ygOOoHvaqi2335YrjcbTXnsccEphTqVz1aysvCF3lObEvKpoIgUxaQhh+RSjveAHGpSCt0hnUGXQQpkXdE51WBbAKWAcKHt4vcIXwelNsY3c+Zi7cNLiL8MljatwiYBLoPM3mxZL4SZtdO4z4ST93aK/17JFUGr6RjaWXY95FXtOBZNHdqQGALElIqGo/NLlkKywju3B6LSYEihY7mN9QyyL5a9PNtik0vsnFtX1v8IkrX87ZfYgv5WRueBlStS+69I2GcoVTqBbPBrsLAH8NP5d9AtzN2BfTzEbN5V4cwlTxvg9cDKWWA7+9KqKspjVN3klHPUGeDue+hjQtl1t5KxENwozzBjDp4IlqbCLXpHb6S+xg+zaumAGJxVFUrwdqUvPCAm7T3FbhLEpaTDA8kIV2SNPoLVlYjYr12ymP6Z1fP9zE7y6cn8sRXvkPuwDW1vMuzLSePprfIVy8nAVTGdPMkXz3hUWuoxbtMDPC8OPaN1egpJyXnUqPAbiV/OW68i52N+RzEJbDOZT+6W2VN11TOlChlC4ugCwZodXc+iCh3J2anlnOHprcL2Y+c9fK7E5tB7qEuA70Kmrgp9UjIb/OzMXScuc3/M5uzfDNORKXq8I9ojmSBgcXc6ozfluLC73nCaDWf7eX3zZXmLLtIjekxP0LMvaJNe0xZOCJ6Dr/SNvls/rJ/WL+u3gc7PlTYPaeixLv4BJ2aMmA==</latexit><latexit sha1_base64="nISD5kUpa/bV6As3fiK88hp8XnM=">AAAIH3icpVVLT9RQFD6AWhwfgC7dNE5IXMg4JSayJLpxYQwYBkiQmLZzB5rpy/ZWJISdO7f6A/w17oxb/oH+C79zbod5dyC2YTg99zvfued1r5eGQa6bzYu5+YUbN29Zi7drd+7eu7+0vPJgN0+KzFctPwmTbN9zcxUGsWrpQIdqP82UG3mh2vO6r3h975PK8iCJd/Rpqg4j9ygOOoHvaqi2335YrjcbTXnsccEphTqVz1aysvCF3lObEvKpoIgUxaQhh+RSjveAHGpSCt0hnUGXQQpkXdE51WBbAKWAcKHt4vcIXwelNsY3c+Zi7cNLiL8MljatwiYBLoPM3mxZL4SZtdO4z4ST93aK/17JFUGr6RjaWXY95FXtOBZNHdqQGALElIqGo/NLlkKywju3B6LSYEihY7mN9QyyL5a9PNtik0vsnFtX1v8IkrX87ZfYgv5WRueBlStS+69I2GcoVTqBbPBrsLAH8NP5d9AtzN2BfTzEbN5V4cwlTxvg9cDKWWA7+9KqKspjVN3klHPUGeDue+hjQtl1t5KxENwozzBjDp4IlqbCLXpHb6S+xg+zaumAGJxVFUrwdqUvPCAm7T3FbhLEpaTDA8kIV2SNPoLVlYjYr12ymP6Z1fP9zE7y6cn8sRXvkPuwDW1vMuzLSePprfIVy8nAVTGdPMkXz3hUWuoxbtMDPC8OPaN1egpJyXnUqPAbiV/OW68i52N+RzEJbDOZT+6W2VN11TOlChlC4ugCwZodXc+iCh3J2anlnOHprcL2Y+c9fK7E5tB7qEuA70Kmrgp9UjIb/OzMXScuc3/M5uzfDNORKXq8I9ojmSBgcXc6ozfluLC73nCaDWf7eX3zZXmLLtIjekxP0LMvaJNe0xZOCJ6Dr/SNvls/rJ/WL+u3gc7PlTYPaeixLv4BJ2aMmA==</latexit><latexit sha1_base64="nISD5kUpa/bV6As3fiK88hp8XnM=">AAAIH3icpVVLT9RQFD6AWhwfgC7dNE5IXMg4JSayJLpxYQwYBkiQmLZzB5rpy/ZWJISdO7f6A/w17oxb/oH+C79zbod5dyC2YTg99zvfued1r5eGQa6bzYu5+YUbN29Zi7drd+7eu7+0vPJgN0+KzFctPwmTbN9zcxUGsWrpQIdqP82UG3mh2vO6r3h975PK8iCJd/Rpqg4j9ygOOoHvaqi2335YrjcbTXnsccEphTqVz1aysvCF3lObEvKpoIgUxaQhh+RSjveAHGpSCt0hnUGXQQpkXdE51WBbAKWAcKHt4vcIXwelNsY3c+Zi7cNLiL8MljatwiYBLoPM3mxZL4SZtdO4z4ST93aK/17JFUGr6RjaWXY95FXtOBZNHdqQGALElIqGo/NLlkKywju3B6LSYEihY7mN9QyyL5a9PNtik0vsnFtX1v8IkrX87ZfYgv5WRueBlStS+69I2GcoVTqBbPBrsLAH8NP5d9AtzN2BfTzEbN5V4cwlTxvg9cDKWWA7+9KqKspjVN3klHPUGeDue+hjQtl1t5KxENwozzBjDp4IlqbCLXpHb6S+xg+zaumAGJxVFUrwdqUvPCAm7T3FbhLEpaTDA8kIV2SNPoLVlYjYr12ymP6Z1fP9zE7y6cn8sRXvkPuwDW1vMuzLSePprfIVy8nAVTGdPMkXz3hUWuoxbtMDPC8OPaN1egpJyXnUqPAbiV/OW68i52N+RzEJbDOZT+6W2VN11TOlChlC4ugCwZodXc+iCh3J2anlnOHprcL2Y+c9fK7E5tB7qEuA70Kmrgp9UjIb/OzMXScuc3/M5uzfDNORKXq8I9ojmSBgcXc6ozfluLC73nCaDWf7eX3zZXmLLtIjekxP0LMvaJNe0xZOCJ6Dr/SNvls/rJ/WL+u3gc7PlTYPaeixLv4BJ2aMmA==</latexit>

+

Figure 4: Illustration of the relationship reuse mechanism. Weleverage the learned self-attention relationships from all upstreamlayers and attention heads, i.e., Aup, to refine the downstreamattention maps. The addition operation of logits is adopted. Notethat NH denotes the head number for multi-head self-attention.

Relationship reuse. A promi-nent advantage of vision Trans-formers is that their self-attentionblocks enable integrating infor-mation across the entire im-age, which effectively modelsthe long-range dependencies inthe data. Typically, the modelsneed to learn a group of atten-tion maps at each layer to de-scribe the relationships among to-kens. Apart from the deep fea-tures mentioned above, the down-stream models also have access tothe self-attention maps producedin previous models. We arguethat these learned relationshipsare also capable of being reusedto facilitate the learning of down-stream Transformers.

Given the input representation zl, the self-attention is performed as follows. First, the query, key andvalue matrices Ql, Kl and Vl are computed via linear projections:

Ql = zlWQl , Kl = zlW

Kl , Vl = zlW

Vl , (6)

where WQl , WK

l and WVl are weight matrices. Then the attention map is calculated by a scaled

dot-product operation with softmax to aggregate the values of all tokens, namely

Attention(zl) = Softmax(Al)Vl, Al = QlKl>/√d. (7)

Here d is the hidden dimension of Q or K, and Al ∈ RN×N denotes the logits of the attentionmap. Note that we omit the details on the multi-head attention mechanism for clarity, where Al mayinclude multiple attention maps. Such a simplification does not affect the description of our method.

For relationship reuse, we first concatenate the attention logits produced by all layers of the upstreammodel (i.e., Aup

l , l ∈ {1, . . . , L}):Aup = Concat(Aup

1 ,Aup2 , . . . ,A

upL) ∈ RNup×Nup×NAtt

up , (8)

where Nup and NAttup denote the number of tokens and all attention maps in the upstream model,

respectively. Typically, we have NAttup = NHL, where NH is the number of heads for the multi-head

attention and L is the number of layers. Then the downstream Transformer learns attention maps byleveraging both its own tokens and Aup simultaneously. Formally, we replace Eq. (7) by

Attention(zl) = Softmax(Al + rl(Aup))Vl, Al = QlKl

>/√d, (9)

where rl(·) is a transformation network that integrates the information provided by Aup to refine thedownstream attention logits Al. The architecture of rl(·) is presented in Figure 4, which includes aMLP for nonlinearity followed by an upsample operation to match the size of attention maps. Formulti-head attention, the output dimension of the MLP will be set to the number of heads.

HW<latexit sha1_base64="hQu4FWVilYJ68mOXTqA359+8ff4=">AAAIIHicpVXLThRREC1AbRwfgC7ddJyQuJCxm5DIksQNCxdIGIYEienuuTN0pl/2QySEnUu3+gF+jTvjUr9A/8JTdXuYdw/E7jBU1z116tbrXjcJ/Cy3rF8Li0u3bt8xlu/W7t1/8HBlde3RYRYXqaeaXhzE6ZHrZCrwI9XM/TxQR0mqnNANVMvtveL11geVZn4cHeTniToJnW7kd3zPyaHa3229W61bDUsec1KwS6FO5bMXry19orfUppg8KigkRRHlkANyKMN7TDZZlEB3QhfQpZB8WVd0STXYFkApIBxoe/jt4uu41Eb4Zs5MrD14CfCXwtKkddjEwKWQ2Zsp64Uws3YW94Vw8t7O8d8tuUJoczqFdp5dH3ldO44lpw5tSww+YkpEw9F5JUshWeGdm0NR5WBIoGO5jfUUsieW/TybYpNJ7JxbR9b/CJK1/O2V2IL+VkbngpUrUvuvSNhnIFU6g6zxG7Awh/Cz+Q/QLczdgX00wqzfdeHMJE/b4HXByllgO/PKqirKU1Rd55Rz1BniHngYYALZda+SsRDcOM8oYwaeEJa6wk3ap9dSX+2HWXPpgAicVRWK8fakL1wgpu09wW5ixKWkw33JCFdkg96D1ZGI2K9Zsuj+mdfzg8xO8+nK/LEV75D7sA1tfzLMq0nj6a3yFcnJwFXRnTzNF894WFrmE9y6B3hebHpBm/QckpLzqFHhNxS/nLd+RS4n/I5jYtimMp/cLfOn6rpnShUygMTR+YLVO7qZRRU6lLMzl3OGp7cKO4id9/CxEptB76IuPr4Lmboq9FnJrPHzM3eTuPT9MZ9zcDPMRibo8Y5ouzJBwOLutMdvyknhcLNhWw37zVZ9Z6u8RZfpCT2lZ+jZl7RDu7SHE8ID+2f6Ql+Nb8Z344fxU0MXF0qbxzTyGL//AcDcjOU=</latexit><latexit sha1_base64="hQu4FWVilYJ68mOXTqA359+8ff4=">AAAIIHicpVXLThRREC1AbRwfgC7ddJyQuJCxm5DIksQNCxdIGIYEienuuTN0pl/2QySEnUu3+gF+jTvjUr9A/8JTdXuYdw/E7jBU1z116tbrXjcJ/Cy3rF8Li0u3bt8xlu/W7t1/8HBlde3RYRYXqaeaXhzE6ZHrZCrwI9XM/TxQR0mqnNANVMvtveL11geVZn4cHeTniToJnW7kd3zPyaHa3229W61bDUsec1KwS6FO5bMXry19orfUppg8KigkRRHlkANyKMN7TDZZlEB3QhfQpZB8WVd0STXYFkApIBxoe/jt4uu41Eb4Zs5MrD14CfCXwtKkddjEwKWQ2Zsp64Uws3YW94Vw8t7O8d8tuUJoczqFdp5dH3ldO44lpw5tSww+YkpEw9F5JUshWeGdm0NR5WBIoGO5jfUUsieW/TybYpNJ7JxbR9b/CJK1/O2V2IL+VkbngpUrUvuvSNhnIFU6g6zxG7Awh/Cz+Q/QLczdgX00wqzfdeHMJE/b4HXByllgO/PKqirKU1Rd55Rz1BniHngYYALZda+SsRDcOM8oYwaeEJa6wk3ap9dSX+2HWXPpgAicVRWK8fakL1wgpu09wW5ixKWkw33JCFdkg96D1ZGI2K9Zsuj+mdfzg8xO8+nK/LEV75D7sA1tfzLMq0nj6a3yFcnJwFXRnTzNF894WFrmE9y6B3hebHpBm/QckpLzqFHhNxS/nLd+RS4n/I5jYtimMp/cLfOn6rpnShUygMTR+YLVO7qZRRU6lLMzl3OGp7cKO4id9/CxEptB76IuPr4Lmboq9FnJrPHzM3eTuPT9MZ9zcDPMRibo8Y5ouzJBwOLutMdvyknhcLNhWw37zVZ9Z6u8RZfpCT2lZ+jZl7RDu7SHE8ID+2f6Ql+Nb8Z344fxU0MXF0qbxzTyGL//AcDcjOU=</latexit><latexit sha1_base64="hQu4FWVilYJ68mOXTqA359+8ff4=">AAAIIHicpVXLThRREC1AbRwfgC7ddJyQuJCxm5DIksQNCxdIGIYEienuuTN0pl/2QySEnUu3+gF+jTvjUr9A/8JTdXuYdw/E7jBU1z116tbrXjcJ/Cy3rF8Li0u3bt8xlu/W7t1/8HBlde3RYRYXqaeaXhzE6ZHrZCrwI9XM/TxQR0mqnNANVMvtveL11geVZn4cHeTniToJnW7kd3zPyaHa3229W61bDUsec1KwS6FO5bMXry19orfUppg8KigkRRHlkANyKMN7TDZZlEB3QhfQpZB8WVd0STXYFkApIBxoe/jt4uu41Eb4Zs5MrD14CfCXwtKkddjEwKWQ2Zsp64Uws3YW94Vw8t7O8d8tuUJoczqFdp5dH3ldO44lpw5tSww+YkpEw9F5JUshWeGdm0NR5WBIoGO5jfUUsieW/TybYpNJ7JxbR9b/CJK1/O2V2IL+VkbngpUrUvuvSNhnIFU6g6zxG7Awh/Cz+Q/QLczdgX00wqzfdeHMJE/b4HXByllgO/PKqirKU1Rd55Rz1BniHngYYALZda+SsRDcOM8oYwaeEJa6wk3ap9dSX+2HWXPpgAicVRWK8fakL1wgpu09wW5ixKWkw33JCFdkg96D1ZGI2K9Zsuj+mdfzg8xO8+nK/LEV75D7sA1tfzLMq0nj6a3yFcnJwFXRnTzNF894WFrmE9y6B3hebHpBm/QckpLzqFHhNxS/nLd+RS4n/I5jYtimMp/cLfOn6rpnShUygMTR+YLVO7qZRRU6lLMzl3OGp7cKO4id9/CxEptB76IuPr4Lmboq9FnJrPHzM3eTuPT9MZ9zcDPMRibo8Y5ouzJBwOLutMdvyknhcLNhWw37zVZ9Z6u8RZfpCT2lZ+jZl7RDu7SHE8ID+2f6Ql+Nb8Z344fxU0MXF0qbxzTyGL//AcDcjOU=</latexit><latexit sha1_base64="hQu4FWVilYJ68mOXTqA359+8ff4=">AAAIIHicpVXLThRREC1AbRwfgC7ddJyQuJCxm5DIksQNCxdIGIYEienuuTN0pl/2QySEnUu3+gF+jTvjUr9A/8JTdXuYdw/E7jBU1z116tbrXjcJ/Cy3rF8Li0u3bt8xlu/W7t1/8HBlde3RYRYXqaeaXhzE6ZHrZCrwI9XM/TxQR0mqnNANVMvtveL11geVZn4cHeTniToJnW7kd3zPyaHa3229W61bDUsec1KwS6FO5bMXry19orfUppg8KigkRRHlkANyKMN7TDZZlEB3QhfQpZB8WVd0STXYFkApIBxoe/jt4uu41Eb4Zs5MrD14CfCXwtKkddjEwKWQ2Zsp64Uws3YW94Vw8t7O8d8tuUJoczqFdp5dH3ldO44lpw5tSww+YkpEw9F5JUshWeGdm0NR5WBIoGO5jfUUsieW/TybYpNJ7JxbR9b/CJK1/O2V2IL+VkbngpUrUvuvSNhnIFU6g6zxG7Awh/Cz+Q/QLczdgX00wqzfdeHMJE/b4HXByllgO/PKqirKU1Rd55Rz1BniHngYYALZda+SsRDcOM8oYwaeEJa6wk3ap9dSX+2HWXPpgAicVRWK8fakL1wgpu09wW5ixKWkw33JCFdkg96D1ZGI2K9Zsuj+mdfzg8xO8+nK/LEV75D7sA1tfzLMq0nj6a3yFcnJwFXRnTzNF894WFrmE9y6B3hebHpBm/QckpLzqFHhNxS/nLd+RS4n/I5jYtimMp/cLfOn6rpnShUygMTR+YLVO7qZRRU6lLMzl3OGp7cKO4id9/CxEptB76IuPr4Lmboq9FnJrPHzM3eTuPT9MZ9zcDPMRibo8Y5ouzJBwOLutMdvyknhcLNhWw37zVZ9Z6u8RZfpCT2lZ+jZl7RDu7SHE8ID+2f6Ql+Nb8Z344fxU0MXF0qbxzTyGL//AcDcjOU=</latexit>

HW<latexit sha1_base64="hQu4FWVilYJ68mOXTqA359+8ff4=">AAAIIHicpVXLThRREC1AbRwfgC7ddJyQuJCxm5DIksQNCxdIGIYEienuuTN0pl/2QySEnUu3+gF+jTvjUr9A/8JTdXuYdw/E7jBU1z116tbrXjcJ/Cy3rF8Li0u3bt8xlu/W7t1/8HBlde3RYRYXqaeaXhzE6ZHrZCrwI9XM/TxQR0mqnNANVMvtveL11geVZn4cHeTniToJnW7kd3zPyaHa3229W61bDUsec1KwS6FO5bMXry19orfUppg8KigkRRHlkANyKMN7TDZZlEB3QhfQpZB8WVd0STXYFkApIBxoe/jt4uu41Eb4Zs5MrD14CfCXwtKkddjEwKWQ2Zsp64Uws3YW94Vw8t7O8d8tuUJoczqFdp5dH3ldO44lpw5tSww+YkpEw9F5JUshWeGdm0NR5WBIoGO5jfUUsieW/TybYpNJ7JxbR9b/CJK1/O2V2IL+VkbngpUrUvuvSNhnIFU6g6zxG7Awh/Cz+Q/QLczdgX00wqzfdeHMJE/b4HXByllgO/PKqirKU1Rd55Rz1BniHngYYALZda+SsRDcOM8oYwaeEJa6wk3ap9dSX+2HWXPpgAicVRWK8fakL1wgpu09wW5ixKWkw33JCFdkg96D1ZGI2K9Zsuj+mdfzg8xO8+nK/LEV75D7sA1tfzLMq0nj6a3yFcnJwFXRnTzNF894WFrmE9y6B3hebHpBm/QckpLzqFHhNxS/nLd+RS4n/I5jYtimMp/cLfOn6rpnShUygMTR+YLVO7qZRRU6lLMzl3OGp7cKO4id9/CxEptB76IuPr4Lmboq9FnJrPHzM3eTuPT9MZ9zcDPMRibo8Y5ouzJBwOLutMdvyknhcLNhWw37zVZ9Z6u8RZfpCT2lZ+jZl7RDu7SHE8ID+2f6Ql+Nb8Z344fxU0MXF0qbxzTyGL//AcDcjOU=</latexit><latexit sha1_base64="hQu4FWVilYJ68mOXTqA359+8ff4=">AAAIIHicpVXLThRREC1AbRwfgC7ddJyQuJCxm5DIksQNCxdIGIYEienuuTN0pl/2QySEnUu3+gF+jTvjUr9A/8JTdXuYdw/E7jBU1z116tbrXjcJ/Cy3rF8Li0u3bt8xlu/W7t1/8HBlde3RYRYXqaeaXhzE6ZHrZCrwI9XM/TxQR0mqnNANVMvtveL11geVZn4cHeTniToJnW7kd3zPyaHa3229W61bDUsec1KwS6FO5bMXry19orfUppg8KigkRRHlkANyKMN7TDZZlEB3QhfQpZB8WVd0STXYFkApIBxoe/jt4uu41Eb4Zs5MrD14CfCXwtKkddjEwKWQ2Zsp64Uws3YW94Vw8t7O8d8tuUJoczqFdp5dH3ldO44lpw5tSww+YkpEw9F5JUshWeGdm0NR5WBIoGO5jfUUsieW/TybYpNJ7JxbR9b/CJK1/O2V2IL+VkbngpUrUvuvSNhnIFU6g6zxG7Awh/Cz+Q/QLczdgX00wqzfdeHMJE/b4HXByllgO/PKqirKU1Rd55Rz1BniHngYYALZda+SsRDcOM8oYwaeEJa6wk3ap9dSX+2HWXPpgAicVRWK8fakL1wgpu09wW5ixKWkw33JCFdkg96D1ZGI2K9Zsuj+mdfzg8xO8+nK/LEV75D7sA1tfzLMq0nj6a3yFcnJwFXRnTzNF894WFrmE9y6B3hebHpBm/QckpLzqFHhNxS/nLd+RS4n/I5jYtimMp/cLfOn6rpnShUygMTR+YLVO7qZRRU6lLMzl3OGp7cKO4id9/CxEptB76IuPr4Lmboq9FnJrPHzM3eTuPT9MZ9zcDPMRibo8Y5ouzJBwOLutMdvyknhcLNhWw37zVZ9Z6u8RZfpCT2lZ+jZl7RDu7SHE8ID+2f6Ql+Nb8Z344fxU0MXF0qbxzTyGL//AcDcjOU=</latexit><latexit sha1_base64="hQu4FWVilYJ68mOXTqA359+8ff4=">AAAIIHicpVXLThRREC1AbRwfgC7ddJyQuJCxm5DIksQNCxdIGIYEienuuTN0pl/2QySEnUu3+gF+jTvjUr9A/8JTdXuYdw/E7jBU1z116tbrXjcJ/Cy3rF8Li0u3bt8xlu/W7t1/8HBlde3RYRYXqaeaXhzE6ZHrZCrwI9XM/TxQR0mqnNANVMvtveL11geVZn4cHeTniToJnW7kd3zPyaHa3229W61bDUsec1KwS6FO5bMXry19orfUppg8KigkRRHlkANyKMN7TDZZlEB3QhfQpZB8WVd0STXYFkApIBxoe/jt4uu41Eb4Zs5MrD14CfCXwtKkddjEwKWQ2Zsp64Uws3YW94Vw8t7O8d8tuUJoczqFdp5dH3ldO44lpw5tSww+YkpEw9F5JUshWeGdm0NR5WBIoGO5jfUUsieW/TybYpNJ7JxbR9b/CJK1/O2V2IL+VkbngpUrUvuvSNhnIFU6g6zxG7Awh/Cz+Q/QLczdgX00wqzfdeHMJE/b4HXByllgO/PKqirKU1Rd55Rz1BniHngYYALZda+SsRDcOM8oYwaeEJa6wk3ap9dSX+2HWXPpgAicVRWK8fakL1wgpu09wW5ixKWkw33JCFdkg96D1ZGI2K9Zsuj+mdfzg8xO8+nK/LEV75D7sA1tfzLMq0nj6a3yFcnJwFXRnTzNF894WFrmE9y6B3hebHpBm/QckpLzqFHhNxS/nLd+RS4n/I5jYtimMp/cLfOn6rpnShUygMTR+YLVO7qZRRU6lLMzl3OGp7cKO4id9/CxEptB76IuPr4Lmboq9FnJrPHzM3eTuPT9MZ9zcDPMRibo8Y5ouzJBwOLutMdvyknhcLNhWw37zVZ9Z6u8RZfpCT2lZ+jZl7RDu7SHE8ID+2f6Ql+Nb8Z344fxU0MXF0qbxzTyGL//AcDcjOU=</latexit><latexit sha1_base64="hQu4FWVilYJ68mOXTqA359+8ff4=">AAAIIHicpVXLThRREC1AbRwfgC7ddJyQuJCxm5DIksQNCxdIGIYEienuuTN0pl/2QySEnUu3+gF+jTvjUr9A/8JTdXuYdw/E7jBU1z116tbrXjcJ/Cy3rF8Li0u3bt8xlu/W7t1/8HBlde3RYRYXqaeaXhzE6ZHrZCrwI9XM/TxQR0mqnNANVMvtveL11geVZn4cHeTniToJnW7kd3zPyaHa3229W61bDUsec1KwS6FO5bMXry19orfUppg8KigkRRHlkANyKMN7TDZZlEB3QhfQpZB8WVd0STXYFkApIBxoe/jt4uu41Eb4Zs5MrD14CfCXwtKkddjEwKWQ2Zsp64Uws3YW94Vw8t7O8d8tuUJoczqFdp5dH3ldO44lpw5tSww+YkpEw9F5JUshWeGdm0NR5WBIoGO5jfUUsieW/TybYpNJ7JxbR9b/CJK1/O2V2IL+VkbngpUrUvuvSNhnIFU6g6zxG7Awh/Cz+Q/QLczdgX00wqzfdeHMJE/b4HXByllgO/PKqirKU1Rd55Rz1BniHngYYALZda+SsRDcOM8oYwaeEJa6wk3ap9dSX+2HWXPpgAicVRWK8fakL1wgpu09wW5ixKWkw33JCFdkg96D1ZGI2K9Zsuj+mdfzg8xO8+nK/LEV75D7sA1tfzLMq0nj6a3yFcnJwFXRnTzNF894WFrmE9y6B3hebHpBm/QckpLzqFHhNxS/nLd+RS4n/I5jYtimMp/cLfOn6rpnShUygMTR+YLVO7qZRRU6lLMzl3OGp7cKO4id9/CxEptB76IuPr4Lmboq9FnJrPHzM3eTuPT9MZ9zcDPMRibo8Y5ouzJBwOLutMdvyknhcLNhWw37zVZ9Z6u8RZfpCT2lZ+jZl7RDu7SHE8ID+2f6Ql+Nb8Z344fxU0MXF0qbxzTyGL//AcDcjOU=</latexit>

HW<latexit sha1_base64="hQu4FWVilYJ68mOXTqA359+8ff4=">AAAIIHicpVXLThRREC1AbRwfgC7ddJyQuJCxm5DIksQNCxdIGIYEienuuTN0pl/2QySEnUu3+gF+jTvjUr9A/8JTdXuYdw/E7jBU1z116tbrXjcJ/Cy3rF8Li0u3bt8xlu/W7t1/8HBlde3RYRYXqaeaXhzE6ZHrZCrwI9XM/TxQR0mqnNANVMvtveL11geVZn4cHeTniToJnW7kd3zPyaHa3229W61bDUsec1KwS6FO5bMXry19orfUppg8KigkRRHlkANyKMN7TDZZlEB3QhfQpZB8WVd0STXYFkApIBxoe/jt4uu41Eb4Zs5MrD14CfCXwtKkddjEwKWQ2Zsp64Uws3YW94Vw8t7O8d8tuUJoczqFdp5dH3ldO44lpw5tSww+YkpEw9F5JUshWeGdm0NR5WBIoGO5jfUUsieW/TybYpNJ7JxbR9b/CJK1/O2V2IL+VkbngpUrUvuvSNhnIFU6g6zxG7Awh/Cz+Q/QLczdgX00wqzfdeHMJE/b4HXByllgO/PKqirKU1Rd55Rz1BniHngYYALZda+SsRDcOM8oYwaeEJa6wk3ap9dSX+2HWXPpgAicVRWK8fakL1wgpu09wW5ixKWkw33JCFdkg96D1ZGI2K9Zsuj+mdfzg8xO8+nK/LEV75D7sA1tfzLMq0nj6a3yFcnJwFXRnTzNF894WFrmE9y6B3hebHpBm/QckpLzqFHhNxS/nLd+RS4n/I5jYtimMp/cLfOn6rpnShUygMTR+YLVO7qZRRU6lLMzl3OGp7cKO4id9/CxEptB76IuPr4Lmboq9FnJrPHzM3eTuPT9MZ9zcDPMRibo8Y5ouzJBwOLutMdvyknhcLNhWw37zVZ9Z6u8RZfpCT2lZ+jZl7RDu7SHE8ID+2f6Ql+Nb8Z344fxU0MXF0qbxzTyGL//AcDcjOU=</latexit><latexit sha1_base64="hQu4FWVilYJ68mOXTqA359+8ff4=">AAAIIHicpVXLThRREC1AbRwfgC7ddJyQuJCxm5DIksQNCxdIGIYEienuuTN0pl/2QySEnUu3+gF+jTvjUr9A/8JTdXuYdw/E7jBU1z116tbrXjcJ/Cy3rF8Li0u3bt8xlu/W7t1/8HBlde3RYRYXqaeaXhzE6ZHrZCrwI9XM/TxQR0mqnNANVMvtveL11geVZn4cHeTniToJnW7kd3zPyaHa3229W61bDUsec1KwS6FO5bMXry19orfUppg8KigkRRHlkANyKMN7TDZZlEB3QhfQpZB8WVd0STXYFkApIBxoe/jt4uu41Eb4Zs5MrD14CfCXwtKkddjEwKWQ2Zsp64Uws3YW94Vw8t7O8d8tuUJoczqFdp5dH3ldO44lpw5tSww+YkpEw9F5JUshWeGdm0NR5WBIoGO5jfUUsieW/TybYpNJ7JxbR9b/CJK1/O2V2IL+VkbngpUrUvuvSNhnIFU6g6zxG7Awh/Cz+Q/QLczdgX00wqzfdeHMJE/b4HXByllgO/PKqirKU1Rd55Rz1BniHngYYALZda+SsRDcOM8oYwaeEJa6wk3ap9dSX+2HWXPpgAicVRWK8fakL1wgpu09wW5ixKWkw33JCFdkg96D1ZGI2K9Zsuj+mdfzg8xO8+nK/LEV75D7sA1tfzLMq0nj6a3yFcnJwFXRnTzNF894WFrmE9y6B3hebHpBm/QckpLzqFHhNxS/nLd+RS4n/I5jYtimMp/cLfOn6rpnShUygMTR+YLVO7qZRRU6lLMzl3OGp7cKO4id9/CxEptB76IuPr4Lmboq9FnJrPHzM3eTuPT9MZ9zcDPMRibo8Y5ouzJBwOLutMdvyknhcLNhWw37zVZ9Z6u8RZfpCT2lZ+jZl7RDu7SHE8ID+2f6Ql+Nb8Z344fxU0MXF0qbxzTyGL//AcDcjOU=</latexit><latexit sha1_base64="hQu4FWVilYJ68mOXTqA359+8ff4=">AAAIIHicpVXLThRREC1AbRwfgC7ddJyQuJCxm5DIksQNCxdIGIYEienuuTN0pl/2QySEnUu3+gF+jTvjUr9A/8JTdXuYdw/E7jBU1z116tbrXjcJ/Cy3rF8Li0u3bt8xlu/W7t1/8HBlde3RYRYXqaeaXhzE6ZHrZCrwI9XM/TxQR0mqnNANVMvtveL11geVZn4cHeTniToJnW7kd3zPyaHa3229W61bDUsec1KwS6FO5bMXry19orfUppg8KigkRRHlkANyKMN7TDZZlEB3QhfQpZB8WVd0STXYFkApIBxoe/jt4uu41Eb4Zs5MrD14CfCXwtKkddjEwKWQ2Zsp64Uws3YW94Vw8t7O8d8tuUJoczqFdp5dH3ldO44lpw5tSww+YkpEw9F5JUshWeGdm0NR5WBIoGO5jfUUsieW/TybYpNJ7JxbR9b/CJK1/O2V2IL+VkbngpUrUvuvSNhnIFU6g6zxG7Awh/Cz+Q/QLczdgX00wqzfdeHMJE/b4HXByllgO/PKqirKU1Rd55Rz1BniHngYYALZda+SsRDcOM8oYwaeEJa6wk3ap9dSX+2HWXPpgAicVRWK8fakL1wgpu09wW5ixKWkw33JCFdkg96D1ZGI2K9Zsuj+mdfzg8xO8+nK/LEV75D7sA1tfzLMq0nj6a3yFcnJwFXRnTzNF894WFrmE9y6B3hebHpBm/QckpLzqFHhNxS/nLd+RS4n/I5jYtimMp/cLfOn6rpnShUygMTR+YLVO7qZRRU6lLMzl3OGp7cKO4id9/CxEptB76IuPr4Lmboq9FnJrPHzM3eTuPT9MZ9zcDPMRibo8Y5ouzJBwOLutMdvyknhcLNhWw37zVZ9Z6u8RZfpCT2lZ+jZl7RDu7SHE8ID+2f6Ql+Nb8Z344fxU0MXF0qbxzTyGL//AcDcjOU=</latexit><latexit sha1_base64="hQu4FWVilYJ68mOXTqA359+8ff4=">AAAIIHicpVXLThRREC1AbRwfgC7ddJyQuJCxm5DIksQNCxdIGIYEienuuTN0pl/2QySEnUu3+gF+jTvjUr9A/8JTdXuYdw/E7jBU1z116tbrXjcJ/Cy3rF8Li0u3bt8xlu/W7t1/8HBlde3RYRYXqaeaXhzE6ZHrZCrwI9XM/TxQR0mqnNANVMvtveL11geVZn4cHeTniToJnW7kd3zPyaHa3229W61bDUsec1KwS6FO5bMXry19orfUppg8KigkRRHlkANyKMN7TDZZlEB3QhfQpZB8WVd0STXYFkApIBxoe/jt4uu41Eb4Zs5MrD14CfCXwtKkddjEwKWQ2Zsp64Uws3YW94Vw8t7O8d8tuUJoczqFdp5dH3ldO44lpw5tSww+YkpEw9F5JUshWeGdm0NR5WBIoGO5jfUUsieW/TybYpNJ7JxbR9b/CJK1/O2V2IL+VkbngpUrUvuvSNhnIFU6g6zxG7Awh/Cz+Q/QLczdgX00wqzfdeHMJE/b4HXByllgO/PKqirKU1Rd55Rz1BniHngYYALZda+SsRDcOM8oYwaeEJa6wk3ap9dSX+2HWXPpgAicVRWK8fakL1wgpu09wW5ixKWkw33JCFdkg96D1ZGI2K9Zsuj+mdfzg8xO8+nK/LEV75D7sA1tfzLMq0nj6a3yFcnJwFXRnTzNF894WFrmE9y6B3hebHpBm/QckpLzqFHhNxS/nLd+RS4n/I5jYtimMp/cLfOn6rpnShUygMTR+YLVO7qZRRU6lLMzl3OGp7cKO4id9/CxEptB76IuPr4Lmboq9FnJrPHzM3eTuPT9MZ9zcDPMRibo8Y5ouzJBwOLutMdvyknhcLNhWw37zVZ9Z6u8RZfpCT2lZ+jZl7RDu7SHE8ID+2f6Ql+Nb8Z344fxU0MXF0qbxzTyGL//AcDcjOU=</latexit>

H<latexit sha1_base64="wW+DQYgbAMe5BZV7Z5zM+P6V9LU=">AAAIH3icpVVLT9RQFD6AWhwfgC7dNE5IXMg4JSSyJHHDwgUYBkiQmLZzZ2imL9tbkRB27tzqD/DXuDNu+Qf6L/zOuR1mmEcHYhuG03O/8517Xvd6aRjkutm8nJtfuHP3nrV4v/bg4aPHS8srT/bzpMh81fKTMMkOPTdXYRCrlg50qA7TTLmRF6oDr/eG1w8+qSwPknhPn6XqOHK7cdAJfFdDtbv9YbnebDTlsccFpxTqVD47ycrCF3pPbUrIp4IiUhSThhySSzneI3KoSSl0x3QOXQYpkHVFF1SDbQGUAsKFtoffLr6OSm2Mb+bMxdqHlxB/GSxtWoVNAlwGmb3Zsl4IM2uncZ8LJ+/tDP+9kiuCVtMJtLPs+sib2nEsmjq0KTEEiCkVDUfnlyyFZIV3bg9FpcGQQsdyG+sZZF8s+3m2xSaX2Dm3rqz/ESRr+dsvsQX9rYzOAytXpPZfkbDPUKp0Ctng12BhD+Gn8++hW5i7A/v4GrN5V4UzlzxtgtcDK2eB7ewrq6ooT1B1k1POUWeIe+BhgAll171KxkJwozzXGXPwRLA0FW7RO3or9TV+mFVLB8TgrKpQgrcnfeEBMWnvKXaTIC4lHR5IRrgia/QRrK5ExH7tksX0z6yeH2R2kk9P5o+teIfch21o+5NhX00aT2+Vr1hOBq6K6eRJvnjGo9JSj3GbHuB5cegVrdNLSErOo0aF30j8ct76FbkY8zuKSWCbyXxyt8yeqpueKVXIEBJHFwjW7Oh2FlXoSM5OLecMT28VdhA77+FzJTaH3kNdAnwXMnVV6NOS2eBnZ+42cZn7Yzbn4GaYjkzR4x3RdmWCgMXd6YzelOPC/nrDaTac3Y361kZ5iy7SM3pOL9Czr2mLtmkHJwTPwVf6Rt+tH9ZP65f120Dn50qbp3TtsS7/AfS/jIQ=</latexit><latexit sha1_base64="wW+DQYgbAMe5BZV7Z5zM+P6V9LU=">AAAIH3icpVVLT9RQFD6AWhwfgC7dNE5IXMg4JSSyJHHDwgUYBkiQmLZzZ2imL9tbkRB27tzqD/DXuDNu+Qf6L/zOuR1mmEcHYhuG03O/8517Xvd6aRjkutm8nJtfuHP3nrV4v/bg4aPHS8srT/bzpMh81fKTMMkOPTdXYRCrlg50qA7TTLmRF6oDr/eG1w8+qSwPknhPn6XqOHK7cdAJfFdDtbv9YbnebDTlsccFpxTqVD47ycrCF3pPbUrIp4IiUhSThhySSzneI3KoSSl0x3QOXQYpkHVFF1SDbQGUAsKFtoffLr6OSm2Mb+bMxdqHlxB/GSxtWoVNAlwGmb3Zsl4IM2uncZ8LJ+/tDP+9kiuCVtMJtLPs+sib2nEsmjq0KTEEiCkVDUfnlyyFZIV3bg9FpcGQQsdyG+sZZF8s+3m2xSaX2Dm3rqz/ESRr+dsvsQX9rYzOAytXpPZfkbDPUKp0Ctng12BhD+Gn8++hW5i7A/v4GrN5V4UzlzxtgtcDK2eB7ewrq6ooT1B1k1POUWeIe+BhgAll171KxkJwozzXGXPwRLA0FW7RO3or9TV+mFVLB8TgrKpQgrcnfeEBMWnvKXaTIC4lHR5IRrgia/QRrK5ExH7tksX0z6yeH2R2kk9P5o+teIfch21o+5NhX00aT2+Vr1hOBq6K6eRJvnjGo9JSj3GbHuB5cegVrdNLSErOo0aF30j8ct76FbkY8zuKSWCbyXxyt8yeqpueKVXIEBJHFwjW7Oh2FlXoSM5OLecMT28VdhA77+FzJTaH3kNdAnwXMnVV6NOS2eBnZ+42cZn7Yzbn4GaYjkzR4x3RdmWCgMXd6YzelOPC/nrDaTac3Y361kZ5iy7SM3pOL9Czr2mLtmkHJwTPwVf6Rt+tH9ZP65f120Dn50qbp3TtsS7/AfS/jIQ=</latexit><latexit sha1_base64="wW+DQYgbAMe5BZV7Z5zM+P6V9LU=">AAAIH3icpVVLT9RQFD6AWhwfgC7dNE5IXMg4JSSyJHHDwgUYBkiQmLZzZ2imL9tbkRB27tzqD/DXuDNu+Qf6L/zOuR1mmEcHYhuG03O/8517Xvd6aRjkutm8nJtfuHP3nrV4v/bg4aPHS8srT/bzpMh81fKTMMkOPTdXYRCrlg50qA7TTLmRF6oDr/eG1w8+qSwPknhPn6XqOHK7cdAJfFdDtbv9YbnebDTlsccFpxTqVD47ycrCF3pPbUrIp4IiUhSThhySSzneI3KoSSl0x3QOXQYpkHVFF1SDbQGUAsKFtoffLr6OSm2Mb+bMxdqHlxB/GSxtWoVNAlwGmb3Zsl4IM2uncZ8LJ+/tDP+9kiuCVtMJtLPs+sib2nEsmjq0KTEEiCkVDUfnlyyFZIV3bg9FpcGQQsdyG+sZZF8s+3m2xSaX2Dm3rqz/ESRr+dsvsQX9rYzOAytXpPZfkbDPUKp0Ctng12BhD+Gn8++hW5i7A/v4GrN5V4UzlzxtgtcDK2eB7ewrq6ooT1B1k1POUWeIe+BhgAll171KxkJwozzXGXPwRLA0FW7RO3or9TV+mFVLB8TgrKpQgrcnfeEBMWnvKXaTIC4lHR5IRrgia/QRrK5ExH7tksX0z6yeH2R2kk9P5o+teIfch21o+5NhX00aT2+Vr1hOBq6K6eRJvnjGo9JSj3GbHuB5cegVrdNLSErOo0aF30j8ct76FbkY8zuKSWCbyXxyt8yeqpueKVXIEBJHFwjW7Oh2FlXoSM5OLecMT28VdhA77+FzJTaH3kNdAnwXMnVV6NOS2eBnZ+42cZn7Yzbn4GaYjkzR4x3RdmWCgMXd6YzelOPC/nrDaTac3Y361kZ5iy7SM3pOL9Czr2mLtmkHJwTPwVf6Rt+tH9ZP65f120Dn50qbp3TtsS7/AfS/jIQ=</latexit><latexit sha1_base64="wW+DQYgbAMe5BZV7Z5zM+P6V9LU=">AAAIH3icpVVLT9RQFD6AWhwfgC7dNE5IXMg4JSSyJHHDwgUYBkiQmLZzZ2imL9tbkRB27tzqD/DXuDNu+Qf6L/zOuR1mmEcHYhuG03O/8517Xvd6aRjkutm8nJtfuHP3nrV4v/bg4aPHS8srT/bzpMh81fKTMMkOPTdXYRCrlg50qA7TTLmRF6oDr/eG1w8+qSwPknhPn6XqOHK7cdAJfFdDtbv9YbnebDTlsccFpxTqVD47ycrCF3pPbUrIp4IiUhSThhySSzneI3KoSSl0x3QOXQYpkHVFF1SDbQGUAsKFtoffLr6OSm2Mb+bMxdqHlxB/GSxtWoVNAlwGmb3Zsl4IM2uncZ8LJ+/tDP+9kiuCVtMJtLPs+sib2nEsmjq0KTEEiCkVDUfnlyyFZIV3bg9FpcGQQsdyG+sZZF8s+3m2xSaX2Dm3rqz/ESRr+dsvsQX9rYzOAytXpPZfkbDPUKp0Ctng12BhD+Gn8++hW5i7A/v4GrN5V4UzlzxtgtcDK2eB7ewrq6ooT1B1k1POUWeIe+BhgAll171KxkJwozzXGXPwRLA0FW7RO3or9TV+mFVLB8TgrKpQgrcnfeEBMWnvKXaTIC4lHR5IRrgia/QRrK5ExH7tksX0z6yeH2R2kk9P5o+teIfch21o+5NhX00aT2+Vr1hOBq6K6eRJvnjGo9JSj3GbHuB5cegVrdNLSErOo0aF30j8ct76FbkY8zuKSWCbyXxyt8yeqpueKVXIEBJHFwjW7Oh2FlXoSM5OLecMT28VdhA77+FzJTaH3kNdAnwXMnVV6NOS2eBnZ+42cZn7Yzbn4GaYjkzR4x3RdmWCgMXd6YzelOPC/nrDaTac3Y361kZ5iy7SM3pOL9Czr2mLtmkHJwTPwVf6Rt+tH9ZP65f120Dn50qbp3TtsS7/AfS/jIQ=</latexit>

W<latexit sha1_base64="D2lzFC37F0W69/6moC4ez4oVgR4=">AAAIH3icpVVLT9RQFD6AWhxfoEs3jRMSFzJOCYksSdy4cAGGYUiQmLZzB5rpy/ZWJISdO7f6A/w17oxb/oH+C79zbod5dyC2YTg99zvfued1r5eGQa6bzcuFxaVbt+9Yy3dr9+4/ePhoZfXxfp4Uma9afhIm2YHn5ioMYtXSgQ7VQZopN/JC1fZ6r3m9/UlleZDEe/osVUeRexwH3cB3NVS77Q8r9WajKY89KTilUKfy2UlWl77Qe+pQQj4VFJGimDTkkFzK8R6SQ01KoTuic+gySIGsK7qgGmwLoBQQLrQ9/B7j67DUxvhmzlysfXgJ8ZfB0qY12CTAZZDZmy3rhTCzdhb3uXDy3s7w3yu5Img1nUA7z66PvK4dx6KpS1sSQ4CYUtFwdH7JUkhWeOf2UFQaDCl0LHewnkH2xbKfZ1tscomdc+vK+h9Bspa//RJb0N/K6DywckVq/xUJ+wylSqeQDX4dFvYQfjb/HrqFubuwj0eYzbsmnLnkaQu8Hlg5C2xnX1lVRXmCqpucco66Q9wDDwNMKLvuVTIWghvnGWXMwRPB0lS4Re/ordTX+GFWLR0Qg7OqQgnenvSFB8S0vafYTYK4lHR4IBnhiqzTR7C6EhH7tUsW0z/zen6Q2Wk+PZk/tuIdch92oO1Phn01aTy9Vb5iORm4KqaTp/niGY9KSz3BbXqA58Whl7RBLyApOY8aFX4j8ct561fkYsLvOCaBbSbzyd0yf6que6ZUIUNIHF0gWLOjm1lUoSM5O7WcMzy9VdhB7LyHz5XYHHoPdQnwXcjUVaFPS2aDn5+5m8Rl7o/5nIObYTYyRY93RXssEwQs7k5n/KacFPY3Gk6z4exu1rc3y1t0mZ7SM3qOnn1F2/SGdnBC8Bx8pW/03fph/bR+Wb8NdHGhtHlCI491+Q9ow4yT</latexit><latexit sha1_base64="D2lzFC37F0W69/6moC4ez4oVgR4=">AAAIH3icpVVLT9RQFD6AWhxfoEs3jRMSFzJOCYksSdy4cAGGYUiQmLZzB5rpy/ZWJISdO7f6A/w17oxb/oH+C79zbod5dyC2YTg99zvfued1r5eGQa6bzcuFxaVbt+9Yy3dr9+4/ePhoZfXxfp4Uma9afhIm2YHn5ioMYtXSgQ7VQZopN/JC1fZ6r3m9/UlleZDEe/osVUeRexwH3cB3NVS77Q8r9WajKY89KTilUKfy2UlWl77Qe+pQQj4VFJGimDTkkFzK8R6SQ01KoTuic+gySIGsK7qgGmwLoBQQLrQ9/B7j67DUxvhmzlysfXgJ8ZfB0qY12CTAZZDZmy3rhTCzdhb3uXDy3s7w3yu5Img1nUA7z66PvK4dx6KpS1sSQ4CYUtFwdH7JUkhWeOf2UFQaDCl0LHewnkH2xbKfZ1tscomdc+vK+h9Bspa//RJb0N/K6DywckVq/xUJ+wylSqeQDX4dFvYQfjb/HrqFubuwj0eYzbsmnLnkaQu8Hlg5C2xnX1lVRXmCqpucco66Q9wDDwNMKLvuVTIWghvnGWXMwRPB0lS4Re/ordTX+GFWLR0Qg7OqQgnenvSFB8S0vafYTYK4lHR4IBnhiqzTR7C6EhH7tUsW0z/zen6Q2Wk+PZk/tuIdch92oO1Phn01aTy9Vb5iORm4KqaTp/niGY9KSz3BbXqA58Whl7RBLyApOY8aFX4j8ct561fkYsLvOCaBbSbzyd0yf6que6ZUIUNIHF0gWLOjm1lUoSM5O7WcMzy9VdhB7LyHz5XYHHoPdQnwXcjUVaFPS2aDn5+5m8Rl7o/5nIObYTYyRY93RXssEwQs7k5n/KacFPY3Gk6z4exu1rc3y1t0mZ7SM3qOnn1F2/SGdnBC8Bx8pW/03fph/bR+Wb8NdHGhtHlCI491+Q9ow4yT</latexit><latexit sha1_base64="D2lzFC37F0W69/6moC4ez4oVgR4=">AAAIH3icpVVLT9RQFD6AWhxfoEs3jRMSFzJOCYksSdy4cAGGYUiQmLZzB5rpy/ZWJISdO7f6A/w17oxb/oH+C79zbod5dyC2YTg99zvfued1r5eGQa6bzcuFxaVbt+9Yy3dr9+4/ePhoZfXxfp4Uma9afhIm2YHn5ioMYtXSgQ7VQZopN/JC1fZ6r3m9/UlleZDEe/osVUeRexwH3cB3NVS77Q8r9WajKY89KTilUKfy2UlWl77Qe+pQQj4VFJGimDTkkFzK8R6SQ01KoTuic+gySIGsK7qgGmwLoBQQLrQ9/B7j67DUxvhmzlysfXgJ8ZfB0qY12CTAZZDZmy3rhTCzdhb3uXDy3s7w3yu5Img1nUA7z66PvK4dx6KpS1sSQ4CYUtFwdH7JUkhWeOf2UFQaDCl0LHewnkH2xbKfZ1tscomdc+vK+h9Bspa//RJb0N/K6DywckVq/xUJ+wylSqeQDX4dFvYQfjb/HrqFubuwj0eYzbsmnLnkaQu8Hlg5C2xnX1lVRXmCqpucco66Q9wDDwNMKLvuVTIWghvnGWXMwRPB0lS4Re/ordTX+GFWLR0Qg7OqQgnenvSFB8S0vafYTYK4lHR4IBnhiqzTR7C6EhH7tUsW0z/zen6Q2Wk+PZk/tuIdch92oO1Phn01aTy9Vb5iORm4KqaTp/niGY9KSz3BbXqA58Whl7RBLyApOY8aFX4j8ct561fkYsLvOCaBbSbzyd0yf6que6ZUIUNIHF0gWLOjm1lUoSM5O7WcMzy9VdhB7LyHz5XYHHoPdQnwXcjUVaFPS2aDn5+5m8Rl7o/5nIObYTYyRY93RXssEwQs7k5n/KacFPY3Gk6z4exu1rc3y1t0mZ7SM3qOnn1F2/SGdnBC8Bx8pW/03fph/bR+Wb8NdHGhtHlCI491+Q9ow4yT</latexit><latexit sha1_base64="D2lzFC37F0W69/6moC4ez4oVgR4=">AAAIH3icpVVLT9RQFD6AWhxfoEs3jRMSFzJOCYksSdy4cAGGYUiQmLZzB5rpy/ZWJISdO7f6A/w17oxb/oH+C79zbod5dyC2YTg99zvfued1r5eGQa6bzcuFxaVbt+9Yy3dr9+4/ePhoZfXxfp4Uma9afhIm2YHn5ioMYtXSgQ7VQZopN/JC1fZ6r3m9/UlleZDEe/osVUeRexwH3cB3NVS77Q8r9WajKY89KTilUKfy2UlWl77Qe+pQQj4VFJGimDTkkFzK8R6SQ01KoTuic+gySIGsK7qgGmwLoBQQLrQ9/B7j67DUxvhmzlysfXgJ8ZfB0qY12CTAZZDZmy3rhTCzdhb3uXDy3s7w3yu5Img1nUA7z66PvK4dx6KpS1sSQ4CYUtFwdH7JUkhWeOf2UFQaDCl0LHewnkH2xbKfZ1tscomdc+vK+h9Bspa//RJb0N/K6DywckVq/xUJ+wylSqeQDX4dFvYQfjb/HrqFubuwj0eYzbsmnLnkaQu8Hlg5C2xnX1lVRXmCqpucco66Q9wDDwNMKLvuVTIWghvnGWXMwRPB0lS4Re/ordTX+GFWLR0Qg7OqQgnenvSFB8S0vafYTYK4lHR4IBnhiqzTR7C6EhH7tUsW0z/zen6Q2Wk+PZk/tuIdch92oO1Phn01aTy9Vb5iORm4KqaTp/niGY9KSz3BbXqA58Whl7RBLyApOY8aFX4j8ct561fkYsLvOCaBbSbzyd0yf6que6ZUIUNIHF0gWLOjm1lUoSM5O7WcMzy9VdhB7LyHz5XYHHoPdQnwXcjUVaFPS2aDn5+5m8Rl7o/5nIObYTYyRY93RXssEwQs7k5n/KacFPY3Gk6z4exu1rc3y1t0mZ7SM3qOnn1F2/SGdnBC8Bx8pW/03fph/bR+Wb8NdHGhtHlCI491+Q9ow4yT</latexit>

H 0<latexit sha1_base64="O/Gu7TzXVuP87DPlHKhEelmcrig=">AAAIIHicpVVLT9RQFD6AOji+QJduGidEFzK2hESWJG5YuEDCAAkS03buQDN92d6KhLBz6VZ/gL/GnXGpv0D/hd85t8O8OxDbMJye+53v3PO610vDINe2/WtufuHGzVu1xdv1O3fv3X+wtPxwL0+KzFctPwmT7MBzcxUGsWrpQIfqIM2UG3mh2ve6r3h9/4PK8iCJd/VZqo4i9zgOOoHvaqh2tp6+W2rYTVsea1xwSqFB5bOdLC98orfUpoR8KigiRTFpyCG5lOM9JIdsSqE7onPoMkiBrCu6oDpsC6AUEC60Xfwe4+uw1Mb4Zs5crH14CfGXwdKiFdgkwGWQ2Zsl64Uws3Ya97lw8t7O8N8ruSJoNZ1AO8uuh7yqHceiqUMbEkOAmFLRcHR+yVJIVnjn1kBUGgwpdCy3sZ5B9sWyl2dLbHKJnXPryvofQbKWv/0SW9Dfyug8sHJF6v8VCfsMpUqnkA1+FRbWAH46/y66hbk7sI+HmM27Ipy55GkDvB5YOQtsZ11aVUV5gqqbnHKOOgPcfQ99TCi77lYyFoIb5RlmzMETwdJUuEU79Frqa/wwq5YOiMFZVaEEb1f6wgNi0t5T7CZBXEo6PJCMcEVW6T1YXYmI/Voli+mfWT3fz+wkn57MH1vxDrkP29D2JsO6nDSe3ipfsZwMXBXTyZN88YxHpaUe4zY9wPPi0Atao+eQlJxHzQq/kfjlvPUqcjHmdxSTwDaT+eRumT1VVz1TqpAhJI4uEKzZ0fUsqtCRnJ1azhme3ipsP3bew8dKbA69h7oE+C5k6qrQpyWzwc/O3HXiMvfHbM7+zTAdmaLHO6I9lgkCFnenM3pTjgt7a03Hbjpv1hub6+UtukiP6Qk9Q8++pE3aom2cED7YP9MX+lr7Vvte+1H7aaDzc6XNIxp6ar//AU29jLU=</latexit><latexit sha1_base64="O/Gu7TzXVuP87DPlHKhEelmcrig=">AAAIIHicpVVLT9RQFD6AOji+QJduGidEFzK2hESWJG5YuEDCAAkS03buQDN92d6KhLBz6VZ/gL/GnXGpv0D/hd85t8O8OxDbMJye+53v3PO610vDINe2/WtufuHGzVu1xdv1O3fv3X+wtPxwL0+KzFctPwmT7MBzcxUGsWrpQIfqIM2UG3mh2ve6r3h9/4PK8iCJd/VZqo4i9zgOOoHvaqh2tp6+W2rYTVsea1xwSqFB5bOdLC98orfUpoR8KigiRTFpyCG5lOM9JIdsSqE7onPoMkiBrCu6oDpsC6AUEC60Xfwe4+uw1Mb4Zs5crH14CfGXwdKiFdgkwGWQ2Zsl64Uws3Ya97lw8t7O8N8ruSJoNZ1AO8uuh7yqHceiqUMbEkOAmFLRcHR+yVJIVnjn1kBUGgwpdCy3sZ5B9sWyl2dLbHKJnXPryvofQbKWv/0SW9Dfyug8sHJF6v8VCfsMpUqnkA1+FRbWAH46/y66hbk7sI+HmM27Ipy55GkDvB5YOQtsZ11aVUV5gqqbnHKOOgPcfQ99TCi77lYyFoIb5RlmzMETwdJUuEU79Frqa/wwq5YOiMFZVaEEb1f6wgNi0t5T7CZBXEo6PJCMcEVW6T1YXYmI/Voli+mfWT3fz+wkn57MH1vxDrkP29D2JsO6nDSe3ipfsZwMXBXTyZN88YxHpaUe4zY9wPPi0Atao+eQlJxHzQq/kfjlvPUqcjHmdxSTwDaT+eRumT1VVz1TqpAhJI4uEKzZ0fUsqtCRnJ1azhme3ipsP3bew8dKbA69h7oE+C5k6qrQpyWzwc/O3HXiMvfHbM7+zTAdmaLHO6I9lgkCFnenM3pTjgt7a03Hbjpv1hub6+UtukiP6Qk9Q8++pE3aom2cED7YP9MX+lr7Vvte+1H7aaDzc6XNIxp6ar//AU29jLU=</latexit><latexit sha1_base64="O/Gu7TzXVuP87DPlHKhEelmcrig=">AAAIIHicpVVLT9RQFD6AOji+QJduGidEFzK2hESWJG5YuEDCAAkS03buQDN92d6KhLBz6VZ/gL/GnXGpv0D/hd85t8O8OxDbMJye+53v3PO610vDINe2/WtufuHGzVu1xdv1O3fv3X+wtPxwL0+KzFctPwmT7MBzcxUGsWrpQIfqIM2UG3mh2ve6r3h9/4PK8iCJd/VZqo4i9zgOOoHvaqh2tp6+W2rYTVsea1xwSqFB5bOdLC98orfUpoR8KigiRTFpyCG5lOM9JIdsSqE7onPoMkiBrCu6oDpsC6AUEC60Xfwe4+uw1Mb4Zs5crH14CfGXwdKiFdgkwGWQ2Zsl64Uws3Ya97lw8t7O8N8ruSJoNZ1AO8uuh7yqHceiqUMbEkOAmFLRcHR+yVJIVnjn1kBUGgwpdCy3sZ5B9sWyl2dLbHKJnXPryvofQbKWv/0SW9Dfyug8sHJF6v8VCfsMpUqnkA1+FRbWAH46/y66hbk7sI+HmM27Ipy55GkDvB5YOQtsZ11aVUV5gqqbnHKOOgPcfQ99TCi77lYyFoIb5RlmzMETwdJUuEU79Frqa/wwq5YOiMFZVaEEb1f6wgNi0t5T7CZBXEo6PJCMcEVW6T1YXYmI/Voli+mfWT3fz+wkn57MH1vxDrkP29D2JsO6nDSe3ipfsZwMXBXTyZN88YxHpaUe4zY9wPPi0Atao+eQlJxHzQq/kfjlvPUqcjHmdxSTwDaT+eRumT1VVz1TqpAhJI4uEKzZ0fUsqtCRnJ1azhme3ipsP3bew8dKbA69h7oE+C5k6qrQpyWzwc/O3HXiMvfHbM7+zTAdmaLHO6I9lgkCFnenM3pTjgt7a03Hbjpv1hub6+UtukiP6Qk9Q8++pE3aom2cED7YP9MX+lr7Vvte+1H7aaDzc6XNIxp6ar//AU29jLU=</latexit><latexit sha1_base64="O/Gu7TzXVuP87DPlHKhEelmcrig=">AAAIIHicpVVLT9RQFD6AOji+QJduGidEFzK2hESWJG5YuEDCAAkS03buQDN92d6KhLBz6VZ/gL/GnXGpv0D/hd85t8O8OxDbMJye+53v3PO610vDINe2/WtufuHGzVu1xdv1O3fv3X+wtPxwL0+KzFctPwmT7MBzcxUGsWrpQIfqIM2UG3mh2ve6r3h9/4PK8iCJd/VZqo4i9zgOOoHvaqh2tp6+W2rYTVsea1xwSqFB5bOdLC98orfUpoR8KigiRTFpyCG5lOM9JIdsSqE7onPoMkiBrCu6oDpsC6AUEC60Xfwe4+uw1Mb4Zs5crH14CfGXwdKiFdgkwGWQ2Zsl64Uws3Ya97lw8t7O8N8ruSJoNZ1AO8uuh7yqHceiqUMbEkOAmFLRcHR+yVJIVnjn1kBUGgwpdCy3sZ5B9sWyl2dLbHKJnXPryvofQbKWv/0SW9Dfyug8sHJF6v8VCfsMpUqnkA1+FRbWAH46/y66hbk7sI+HmM27Ipy55GkDvB5YOQtsZ11aVUV5gqqbnHKOOgPcfQ99TCi77lYyFoIb5RlmzMETwdJUuEU79Frqa/wwq5YOiMFZVaEEb1f6wgNi0t5T7CZBXEo6PJCMcEVW6T1YXYmI/Voli+mfWT3fz+wkn57MH1vxDrkP29D2JsO6nDSe3ipfsZwMXBXTyZN88YxHpaUe4zY9wPPi0Atao+eQlJxHzQq/kfjlvPUqcjHmdxSTwDaT+eRumT1VVz1TqpAhJI4uEKzZ0fUsqtCRnJ1azhme3ipsP3bew8dKbA69h7oE+C5k6qrQpyWzwc/O3HXiMvfHbM7+zTAdmaLHO6I9lgkCFnenM3pTjgt7a03Hbjpv1hub6+UtukiP6Qk9Q8++pE3aom2cED7YP9MX+lr7Vvte+1H7aaDzc6XNIxp6ar//AU29jLU=</latexit>

W 0<latexit sha1_base64="/ieNYCV8srZPF4G3UuL+DQckLoQ=">AAAIIHicpVXLThRREC1AbRwfgC7ddJwQXcjYTUhkSeLGhQskDEOCxHT33Bk60y/7IRLCzqVb/QC/xp1xqV+gf+Gpuj3Muwdidxiq6546det1r5sEfpZb1q+FxaUbN28Zy7drd+7eu7+yuvbgIIuL1FNNLw7i9NB1MhX4kWrmfh6owyRVTugGquX2XvJ664NKMz+O9vOzRB2HTjfyO77n5FDttZ68W61bDUsec1KwS6FO5bMbry19orfUppg8KigkRRHlkANyKMN7RDZZlEB3TOfQpZB8WVd0QTXYFkApIBxoe/jt4uuo1Eb4Zs5MrD14CfCXwtKkddjEwKWQ2Zsp64Uws3YW97lw8t7O8N8tuUJoczqBdp5dH3lVO44lpw5tSww+YkpEw9F5JUshWeGdm0NR5WBIoGO5jfUUsieW/TybYpNJ7JxbR9b/CJK1/O2V2IL+VkbngpUrUvuvSNhnIFU6hazxG7Awh/Cz+ffRLczdgX00wqzfdeHMJE/b4HXByllgO/PSqirKE1Rd55Rz1BniHngYYALZda+SsRDcOM8oYwaeEJa6wk3ao9dSX+2HWXPpgAicVRWK8fakL1wgpu09wW5ixKWkw33JCFdkg96D1ZGI2K9Zsuj+mdfzg8xO8+nK/LEV75D7sA1tfzLMy0nj6a3yFcnJwFXRnTzNF894WFrmE9y6B3hebHpOm/QMkpLzqFHhNxS/nLd+RS4m/I5jYtimMp/cLfOn6qpnShUygMTR+YLVO7qeRRU6lLMzl3OGp7cKO4id9/CxEptB76IuPr4Lmboq9GnJrPHzM3eduPT9MZ9zcDPMRibo8Y5ouzJBwOLutMdvyknhYLNhWw37zVZ9Z6u8RZfpET2mp+jZF7RDr2gXJ4QH9s/0hb4a34zvxg/jp4YuLpQ2D2nkMX7/A8HBjMQ=</latexit><latexit sha1_base64="/ieNYCV8srZPF4G3UuL+DQckLoQ=">AAAIIHicpVXLThRREC1AbRwfgC7ddJwQXcjYTUhkSeLGhQskDEOCxHT33Bk60y/7IRLCzqVb/QC/xp1xqV+gf+Gpuj3Muwdidxiq6546det1r5sEfpZb1q+FxaUbN28Zy7drd+7eu7+yuvbgIIuL1FNNLw7i9NB1MhX4kWrmfh6owyRVTugGquX2XvJ664NKMz+O9vOzRB2HTjfyO77n5FDttZ68W61bDUsec1KwS6FO5bMbry19orfUppg8KigkRRHlkANyKMN7RDZZlEB3TOfQpZB8WVd0QTXYFkApIBxoe/jt4uuo1Eb4Zs5MrD14CfCXwtKkddjEwKWQ2Zsp64Uws3YW97lw8t7O8N8tuUJoczqBdp5dH3lVO44lpw5tSww+YkpEw9F5JUshWeGdm0NR5WBIoGO5jfUUsieW/TybYpNJ7JxbR9b/CJK1/O2V2IL+VkbngpUrUvuvSNhnIFU6hazxG7Awh/Cz+ffRLczdgX00wqzfdeHMJE/b4HXByllgO/PSqirKE1Rd55Rz1BniHngYYALZda+SsRDcOM8oYwaeEJa6wk3ao9dSX+2HWXPpgAicVRWK8fakL1wgpu09wW5ixKWkw33JCFdkg96D1ZGI2K9Zsuj+mdfzg8xO8+nK/LEV75D7sA1tfzLMy0nj6a3yFcnJwFXRnTzNF894WFrmE9y6B3hebHpOm/QMkpLzqFHhNxS/nLd+RS4m/I5jYtimMp/cLfOn6qpnShUygMTR+YLVO7qeRRU6lLMzl3OGp7cKO4id9/CxEptB76IuPr4Lmboq9GnJrPHzM3eduPT9MZ9zcDPMRibo8Y5ouzJBwOLutMdvyknhYLNhWw37zVZ9Z6u8RZfpET2mp+jZF7RDr2gXJ4QH9s/0hb4a34zvxg/jp4YuLpQ2D2nkMX7/A8HBjMQ=</latexit><latexit sha1_base64="/ieNYCV8srZPF4G3UuL+DQckLoQ=">AAAIIHicpVXLThRREC1AbRwfgC7ddJwQXcjYTUhkSeLGhQskDEOCxHT33Bk60y/7IRLCzqVb/QC/xp1xqV+gf+Gpuj3Muwdidxiq6546det1r5sEfpZb1q+FxaUbN28Zy7drd+7eu7+yuvbgIIuL1FNNLw7i9NB1MhX4kWrmfh6owyRVTugGquX2XvJ664NKMz+O9vOzRB2HTjfyO77n5FDttZ68W61bDUsec1KwS6FO5bMbry19orfUppg8KigkRRHlkANyKMN7RDZZlEB3TOfQpZB8WVd0QTXYFkApIBxoe/jt4uuo1Eb4Zs5MrD14CfCXwtKkddjEwKWQ2Zsp64Uws3YW97lw8t7O8N8tuUJoczqBdp5dH3lVO44lpw5tSww+YkpEw9F5JUshWeGdm0NR5WBIoGO5jfUUsieW/TybYpNJ7JxbR9b/CJK1/O2V2IL+VkbngpUrUvuvSNhnIFU6hazxG7Awh/Cz+ffRLczdgX00wqzfdeHMJE/b4HXByllgO/PSqirKE1Rd55Rz1BniHngYYALZda+SsRDcOM8oYwaeEJa6wk3ao9dSX+2HWXPpgAicVRWK8fakL1wgpu09wW5ixKWkw33JCFdkg96D1ZGI2K9Zsuj+mdfzg8xO8+nK/LEV75D7sA1tfzLMy0nj6a3yFcnJwFXRnTzNF894WFrmE9y6B3hebHpOm/QMkpLzqFHhNxS/nLd+RS4m/I5jYtimMp/cLfOn6qpnShUygMTR+YLVO7qeRRU6lLMzl3OGp7cKO4id9/CxEptB76IuPr4Lmboq9GnJrPHzM3eduPT9MZ9zcDPMRibo8Y5ouzJBwOLutMdvyknhYLNhWw37zVZ9Z6u8RZfpET2mp+jZF7RDr2gXJ4QH9s/0hb4a34zvxg/jp4YuLpQ2D2nkMX7/A8HBjMQ=</latexit><latexit sha1_base64="/ieNYCV8srZPF4G3UuL+DQckLoQ=">AAAIIHicpVXLThRREC1AbRwfgC7ddJwQXcjYTUhkSeLGhQskDEOCxHT33Bk60y/7IRLCzqVb/QC/xp1xqV+gf+Gpuj3Muwdidxiq6546det1r5sEfpZb1q+FxaUbN28Zy7drd+7eu7+yuvbgIIuL1FNNLw7i9NB1MhX4kWrmfh6owyRVTugGquX2XvJ664NKMz+O9vOzRB2HTjfyO77n5FDttZ68W61bDUsec1KwS6FO5bMbry19orfUppg8KigkRRHlkANyKMN7RDZZlEB3TOfQpZB8WVd0QTXYFkApIBxoe/jt4uuo1Eb4Zs5MrD14CfCXwtKkddjEwKWQ2Zsp64Uws3YW97lw8t7O8N8tuUJoczqBdp5dH3lVO44lpw5tSww+YkpEw9F5JUshWeGdm0NR5WBIoGO5jfUUsieW/TybYpNJ7JxbR9b/CJK1/O2V2IL+VkbngpUrUvuvSNhnIFU6hazxG7Awh/Cz+ffRLczdgX00wqzfdeHMJE/b4HXByllgO/PSqirKE1Rd55Rz1BniHngYYALZda+SsRDcOM8oYwaeEJa6wk3ao9dSX+2HWXPpgAicVRWK8fakL1wgpu09wW5ixKWkw33JCFdkg96D1ZGI2K9Zsuj+mdfzg8xO8+nK/LEV75D7sA1tfzLMy0nj6a3yFcnJwFXRnTzNF894WFrmE9y6B3hebHpOm/QMkpLzqFHhNxS/nLd+RS4m/I5jYtimMp/cLfOn6qpnShUygMTR+YLVO7qeRRU6lLMzl3OGp7cKO4id9/CxEptB76IuPr4Lmboq9GnJrPHzM3eduPT9MZ9zcDPMRibo8Y5ouzJBwOLutMdvyknhYLNhWw37zVZ9Z6u8RZfpET2mp+jZF7RDr2gXJ4QH9s/0hb4a34zvxg/jp4YuLpQ2D2nkMX7/A8HBjMQ=</latexit>

HW<latexit sha1_base64="hQu4FWVilYJ68mOXTqA359+8ff4=">AAAIIHicpVXLThRREC1AbRwfgC7ddJyQuJCxm5DIksQNCxdIGIYEienuuTN0pl/2QySEnUu3+gF+jTvjUr9A/8JTdXuYdw/E7jBU1z116tbrXjcJ/Cy3rF8Li0u3bt8xlu/W7t1/8HBlde3RYRYXqaeaXhzE6ZHrZCrwI9XM/TxQR0mqnNANVMvtveL11geVZn4cHeTniToJnW7kd3zPyaHa3229W61bDUsec1KwS6FO5bMXry19orfUppg8KigkRRHlkANyKMN7TDZZlEB3QhfQpZB8WVd0STXYFkApIBxoe/jt4uu41Eb4Zs5MrD14CfCXwtKkddjEwKWQ2Zsp64Uws3YW94Vw8t7O8d8tuUJoczqFdp5dH3ldO44lpw5tSww+YkpEw9F5JUshWeGdm0NR5WBIoGO5jfUUsieW/TybYpNJ7JxbR9b/CJK1/O2V2IL+VkbngpUrUvuvSNhnIFU6g6zxG7Awh/Cz+Q/QLczdgX00wqzfdeHMJE/b4HXByllgO/PKqirKU1Rd55Rz1BniHngYYALZda+SsRDcOM8oYwaeEJa6wk3ap9dSX+2HWXPpgAicVRWK8fakL1wgpu09wW5ixKWkw33JCFdkg96D1ZGI2K9Zsuj+mdfzg8xO8+nK/LEV75D7sA1tfzLMq0nj6a3yFcnJwFXRnTzNF894WFrmE9y6B3hebHpBm/QckpLzqFHhNxS/nLd+RS4n/I5jYtimMp/cLfOn6rpnShUygMTR+YLVO7qZRRU6lLMzl3OGp7cKO4id9/CxEptB76IuPr4Lmboq9FnJrPHzM3eTuPT9MZ9zcDPMRibo8Y5ouzJBwOLutMdvyknhcLNhWw37zVZ9Z6u8RZfpCT2lZ+jZl7RDu7SHE8ID+2f6Ql+Nb8Z344fxU0MXF0qbxzTyGL//AcDcjOU=</latexit><latexit sha1_base64="hQu4FWVilYJ68mOXTqA359+8ff4=">AAAIIHicpVXLThRREC1AbRwfgC7ddJyQuJCxm5DIksQNCxdIGIYEienuuTN0pl/2QySEnUu3+gF+jTvjUr9A/8JTdXuYdw/E7jBU1z116tbrXjcJ/Cy3rF8Li0u3bt8xlu/W7t1/8HBlde3RYRYXqaeaXhzE6ZHrZCrwI9XM/TxQR0mqnNANVMvtveL11geVZn4cHeTniToJnW7kd3zPyaHa3229W61bDUsec1KwS6FO5bMXry19orfUppg8KigkRRHlkANyKMN7TDZZlEB3QhfQpZB8WVd0STXYFkApIBxoe/jt4uu41Eb4Zs5MrD14CfCXwtKkddjEwKWQ2Zsp64Uws3YW94Vw8t7O8d8tuUJoczqFdp5dH3ldO44lpw5tSww+YkpEw9F5JUshWeGdm0NR5WBIoGO5jfUUsieW/TybYpNJ7JxbR9b/CJK1/O2V2IL+VkbngpUrUvuvSNhnIFU6g6zxG7Awh/Cz+Q/QLczdgX00wqzfdeHMJE/b4HXByllgO/PKqirKU1Rd55Rz1BniHngYYALZda+SsRDcOM8oYwaeEJa6wk3ap9dSX+2HWXPpgAicVRWK8fakL1wgpu09wW5ixKWkw33JCFdkg96D1ZGI2K9Zsuj+mdfzg8xO8+nK/LEV75D7sA1tfzLMq0nj6a3yFcnJwFXRnTzNF894WFrmE9y6B3hebHpBm/QckpLzqFHhNxS/nLd+RS4n/I5jYtimMp/cLfOn6rpnShUygMTR+YLVO7qZRRU6lLMzl3OGp7cKO4id9/CxEptB76IuPr4Lmboq9FnJrPHzM3eTuPT9MZ9zcDPMRibo8Y5ouzJBwOLutMdvyknhcLNhWw37zVZ9Z6u8RZfpCT2lZ+jZl7RDu7SHE8ID+2f6Ql+Nb8Z344fxU0MXF0qbxzTyGL//AcDcjOU=</latexit><latexit sha1_base64="hQu4FWVilYJ68mOXTqA359+8ff4=">AAAIIHicpVXLThRREC1AbRwfgC7ddJyQuJCxm5DIksQNCxdIGIYEienuuTN0pl/2QySEnUu3+gF+jTvjUr9A/8JTdXuYdw/E7jBU1z116tbrXjcJ/Cy3rF8Li0u3bt8xlu/W7t1/8HBlde3RYRYXqaeaXhzE6ZHrZCrwI9XM/TxQR0mqnNANVMvtveL11geVZn4cHeTniToJnW7kd3zPyaHa3229W61bDUsec1KwS6FO5bMXry19orfUppg8KigkRRHlkANyKMN7TDZZlEB3QhfQpZB8WVd0STXYFkApIBxoe/jt4uu41Eb4Zs5MrD14CfCXwtKkddjEwKWQ2Zsp64Uws3YW94Vw8t7O8d8tuUJoczqFdp5dH3ldO44lpw5tSww+YkpEw9F5JUshWeGdm0NR5WBIoGO5jfUUsieW/TybYpNJ7JxbR9b/CJK1/O2V2IL+VkbngpUrUvuvSNhnIFU6g6zxG7Awh/Cz+Q/QLczdgX00wqzfdeHMJE/b4HXByllgO/PKqirKU1Rd55Rz1BniHngYYALZda+SsRDcOM8oYwaeEJa6wk3ap9dSX+2HWXPpgAicVRWK8fakL1wgpu09wW5ixKWkw33JCFdkg96D1ZGI2K9Zsuj+mdfzg8xO8+nK/LEV75D7sA1tfzLMq0nj6a3yFcnJwFXRnTzNF894WFrmE9y6B3hebHpBm/QckpLzqFHhNxS/nLd+RS4n/I5jYtimMp/cLfOn6rpnShUygMTR+YLVO7qZRRU6lLMzl3OGp7cKO4id9/CxEptB76IuPr4Lmboq9FnJrPHzM3eTuPT9MZ9zcDPMRibo8Y5ouzJBwOLutMdvyknhcLNhWw37zVZ9Z6u8RZfpCT2lZ+jZl7RDu7SHE8ID+2f6Ql+Nb8Z344fxU0MXF0qbxzTyGL//AcDcjOU=</latexit><latexit sha1_base64="hQu4FWVilYJ68mOXTqA359+8ff4=">AAAIIHicpVXLThRREC1AbRwfgC7ddJyQuJCxm5DIksQNCxdIGIYEienuuTN0pl/2QySEnUu3+gF+jTvjUr9A/8JTdXuYdw/E7jBU1z116tbrXjcJ/Cy3rF8Li0u3bt8xlu/W7t1/8HBlde3RYRYXqaeaXhzE6ZHrZCrwI9XM/TxQR0mqnNANVMvtveL11geVZn4cHeTniToJnW7kd3zPyaHa3229W61bDUsec1KwS6FO5bMXry19orfUppg8KigkRRHlkANyKMN7TDZZlEB3QhfQpZB8WVd0STXYFkApIBxoe/jt4uu41Eb4Zs5MrD14CfCXwtKkddjEwKWQ2Zsp64Uws3YW94Vw8t7O8d8tuUJoczqFdp5dH3ldO44lpw5tSww+YkpEw9F5JUshWeGdm0NR5WBIoGO5jfUUsieW/TybYpNJ7JxbR9b/CJK1/O2V2IL+VkbngpUrUvuvSNhnIFU6g6zxG7Awh/Cz+Q/QLczdgX00wqzfdeHMJE/b4HXByllgO/PKqirKU1Rd55Rz1BniHngYYALZda+SsRDcOM8oYwaeEJa6wk3ap9dSX+2HWXPpgAicVRWK8fakL1wgpu09wW5ixKWkw33JCFdkg96D1ZGI2K9Zsuj+mdfzg8xO8+nK/LEV75D7sA1tfzLMq0nj6a3yFcnJwFXRnTzNF894WFrmE9y6B3hebHpBm/QckpLzqFHhNxS/nLd+RS4n/I5jYtimMp/cLfOn6rpnShUygMTR+YLVO7qZRRU6lLMzl3OGp7cKO4id9/CxEptB76IuPr4Lmboq9FnJrPHzM3eTuPT9MZ9zcDPMRibo8Y5ouzJBwOLutMdvyknhcLNhWw37zVZ9Z6u8RZfpCT2lZ+jZl7RDu7SHE8ID+2f6Ql+Nb8Z344fxU0MXF0qbxzTyGL//AcDcjOU=</latexit>

HW<latexit sha1_base64="hQu4FWVilYJ68mOXTqA359+8ff4=">AAAIIHicpVXLThRREC1AbRwfgC7ddJyQuJCxm5DIksQNCxdIGIYEienuuTN0pl/2QySEnUu3+gF+jTvjUr9A/8JTdXuYdw/E7jBU1z116tbrXjcJ/Cy3rF8Li0u3bt8xlu/W7t1/8HBlde3RYRYXqaeaXhzE6ZHrZCrwI9XM/TxQR0mqnNANVMvtveL11geVZn4cHeTniToJnW7kd3zPyaHa3229W61bDUsec1KwS6FO5bMXry19orfUppg8KigkRRHlkANyKMN7TDZZlEB3QhfQpZB8WVd0STXYFkApIBxoe/jt4uu41Eb4Zs5MrD14CfCXwtKkddjEwKWQ2Zsp64Uws3YW94Vw8t7O8d8tuUJoczqFdp5dH3ldO44lpw5tSww+YkpEw9F5JUshWeGdm0NR5WBIoGO5jfUUsieW/TybYpNJ7JxbR9b/CJK1/O2V2IL+VkbngpUrUvuvSNhnIFU6g6zxG7Awh/Cz+Q/QLczdgX00wqzfdeHMJE/b4HXByllgO/PKqirKU1Rd55Rz1BniHngYYALZda+SsRDcOM8oYwaeEJa6wk3ap9dSX+2HWXPpgAicVRWK8fakL1wgpu09wW5ixKWkw33JCFdkg96D1ZGI2K9Zsuj+mdfzg8xO8+nK/LEV75D7sA1tfzLMq0nj6a3yFcnJwFXRnTzNF894WFrmE9y6B3hebHpBm/QckpLzqFHhNxS/nLd+RS4n/I5jYtimMp/cLfOn6rpnShUygMTR+YLVO7qZRRU6lLMzl3OGp7cKO4id9/CxEptB76IuPr4Lmboq9FnJrPHzM3eTuPT9MZ9zcDPMRibo8Y5ouzJBwOLutMdvyknhcLNhWw37zVZ9Z6u8RZfpCT2lZ+jZl7RDu7SHE8ID+2f6Ql+Nb8Z344fxU0MXF0qbxzTyGL//AcDcjOU=</latexit><latexit sha1_base64="hQu4FWVilYJ68mOXTqA359+8ff4=">AAAIIHicpVXLThRREC1AbRwfgC7ddJyQuJCxm5DIksQNCxdIGIYEienuuTN0pl/2QySEnUu3+gF+jTvjUr9A/8JTdXuYdw/E7jBU1z116tbrXjcJ/Cy3rF8Li0u3bt8xlu/W7t1/8HBlde3RYRYXqaeaXhzE6ZHrZCrwI9XM/TxQR0mqnNANVMvtveL11geVZn4cHeTniToJnW7kd3zPyaHa3229W61bDUsec1KwS6FO5bMXry19orfUppg8KigkRRHlkANyKMN7TDZZlEB3QhfQpZB8WVd0STXYFkApIBxoe/jt4uu41Eb4Zs5MrD14CfCXwtKkddjEwKWQ2Zsp64Uws3YW94Vw8t7O8d8tuUJoczqFdp5dH3ldO44lpw5tSww+YkpEw9F5JUshWeGdm0NR5WBIoGO5jfUUsieW/TybYpNJ7JxbR9b/CJK1/O2V2IL+VkbngpUrUvuvSNhnIFU6g6zxG7Awh/Cz+Q/QLczdgX00wqzfdeHMJE/b4HXByllgO/PKqirKU1Rd55Rz1BniHngYYALZda+SsRDcOM8oYwaeEJa6wk3ap9dSX+2HWXPpgAicVRWK8fakL1wgpu09wW5ixKWkw33JCFdkg96D1ZGI2K9Zsuj+mdfzg8xO8+nK/LEV75D7sA1tfzLMq0nj6a3yFcnJwFXRnTzNF894WFrmE9y6B3hebHpBm/QckpLzqFHhNxS/nLd+RS4n/I5jYtimMp/cLfOn6rpnShUygMTR+YLVO7qZRRU6lLMzl3OGp7cKO4id9/CxEptB76IuPr4Lmboq9FnJrPHzM3eTuPT9MZ9zcDPMRibo8Y5ouzJBwOLutMdvyknhcLNhWw37zVZ9Z6u8RZfpCT2lZ+jZl7RDu7SHE8ID+2f6Ql+Nb8Z344fxU0MXF0qbxzTyGL//AcDcjOU=</latexit><latexit sha1_base64="hQu4FWVilYJ68mOXTqA359+8ff4=">AAAIIHicpVXLThRREC1AbRwfgC7ddJyQuJCxm5DIksQNCxdIGIYEienuuTN0pl/2QySEnUu3+gF+jTvjUr9A/8JTdXuYdw/E7jBU1z116tbrXjcJ/Cy3rF8Li0u3bt8xlu/W7t1/8HBlde3RYRYXqaeaXhzE6ZHrZCrwI9XM/TxQR0mqnNANVMvtveL11geVZn4cHeTniToJnW7kd3zPyaHa3229W61bDUsec1KwS6FO5bMXry19orfUppg8KigkRRHlkANyKMN7TDZZlEB3QhfQpZB8WVd0STXYFkApIBxoe/jt4uu41Eb4Zs5MrD14CfCXwtKkddjEwKWQ2Zsp64Uws3YW94Vw8t7O8d8tuUJoczqFdp5dH3ldO44lpw5tSww+YkpEw9F5JUshWeGdm0NR5WBIoGO5jfUUsieW/TybYpNJ7JxbR9b/CJK1/O2V2IL+VkbngpUrUvuvSNhnIFU6g6zxG7Awh/Cz+Q/QLczdgX00wqzfdeHMJE/b4HXByllgO/PKqirKU1Rd55Rz1BniHngYYALZda+SsRDcOM8oYwaeEJa6wk3ap9dSX+2HWXPpgAicVRWK8fakL1wgpu09wW5ixKWkw33JCFdkg96D1ZGI2K9Zsuj+mdfzg8xO8+nK/LEV75D7sA1tfzLMq0nj6a3yFcnJwFXRnTzNF894WFrmE9y6B3hebHpBm/QckpLzqFHhNxS/nLd+RS4n/I5jYtimMp/cLfOn6rpnShUygMTR+YLVO7qZRRU6lLMzl3OGp7cKO4id9/CxEptB76IuPr4Lmboq9FnJrPHzM3eTuPT9MZ9zcDPMRibo8Y5ouzJBwOLutMdvyknhcLNhWw37zVZ9Z6u8RZfpCT2lZ+jZl7RDu7SHE8ID+2f6Ql+Nb8Z344fxU0MXF0qbxzTyGL//AcDcjOU=</latexit><latexit sha1_base64="hQu4FWVilYJ68mOXTqA359+8ff4=">AAAIIHicpVXLThRREC1AbRwfgC7ddJyQuJCxm5DIksQNCxdIGIYEienuuTN0pl/2QySEnUu3+gF+jTvjUr9A/8JTdXuYdw/E7jBU1z116tbrXjcJ/Cy3rF8Li0u3bt8xlu/W7t1/8HBlde3RYRYXqaeaXhzE6ZHrZCrwI9XM/TxQR0mqnNANVMvtveL11geVZn4cHeTniToJnW7kd3zPyaHa3229W61bDUsec1KwS6FO5bMXry19orfUppg8KigkRRHlkANyKMN7TDZZlEB3QhfQpZB8WVd0STXYFkApIBxoe/jt4uu41Eb4Zs5MrD14CfCXwtKkddjEwKWQ2Zsp64Uws3YW94Vw8t7O8d8tuUJoczqFdp5dH3ldO44lpw5tSww+YkpEw9F5JUshWeGdm0NR5WBIoGO5jfUUsieW/TybYpNJ7JxbR9b/CJK1/O2V2IL+VkbngpUrUvuvSNhnIFU6g6zxG7Awh/Cz+Q/QLczdgX00wqzfdeHMJE/b4HXByllgO/PKqirKU1Rd55Rz1BniHngYYALZda+SsRDcOM8oYwaeEJa6wk3ap9dSX+2HWXPpgAicVRWK8fakL1wgpu09wW5ixKWkw33JCFdkg96D1ZGI2K9Zsuj+mdfzg8xO8+nK/LEV75D7sA1tfzLMq0nj6a3yFcnJwFXRnTzNF894WFrmE9y6B3hebHpBm/QckpLzqFHhNxS/nLd+RS4n/I5jYtimMp/cLfOn6rpnShUygMTR+YLVO7qZRRU6lLMzl3OGp7cKO4id9/CxEptB76IuPr4Lmboq9FnJrPHzM3eTuPT9MZ9zcDPMRibo8Y5ouzJBwOLutMdvyknhcLNhWw37zVZ9Z6u8RZfpCT2lZ+jZl7RDu7SHE8ID+2f6Ql+Nb8Z344fxU0MXF0qbxzTyGL//AcDcjOU=</latexit>

H 0W 0<latexit sha1_base64="RyBvzzo2oLyCDLYLhTluRKYkbJU=">AAAIInicpVVLT9RQFD6AWhxfoEs3jROCCxlbQiJLEjcsXKBhGBIkpu3cGZrpy/ZWJISda7f6A/w17owrE3+A/gu/c9ph3h2IbRhOz/3Od+553esmgZ9py/q1sLh04+YtY/l27c7de/cfrKw+PMjiPPVU04uDOD10nUwFfqSa2teBOkxS5YRuoFpu7yWvtz6oNPPjaF+fJeo4dLqR3/E9R7Nqd721/m6lbjUsecxJwS6FOpXPXry69IneUpti8iinkBRFpCEH5FCG94hssiiB7pjOoUsh+bKu6IJqsM2BUkA40Pbw28XXUamN8M2cmVh78BLgL4WlSWuwiYFLIbM3U9ZzYWbtLO5z4eS9neG/W3KF0Go6gXaeXR95VTuORVOHtiUGHzElouHovJIll6zwzs2hqDQYEuhYbmM9heyJZT/PpthkEjvn1pH1P4JkLX97JTanv5XRuWDlitT+KxL2GUiVTiEX+A1YmEP42fz76Bbm7sA+GmEu3jXhzCRP2+B1wcpZYDvz0qoqyhNUvcgp56gzxD3wMMAEsuteJWMuuHGeUcYMPCEsiwo36Q29kvoWfphVSwdE4KyqUIy3J33hAjFt7wl2EyMuJR3uS0a4Ihv0HqyORMR+zZKl6J95PT/I7DSfrswfW/EOuQ/b0PYnw7ycNJ7eKl+RnAxclaKTp/niGQ9LSz3BXfQAz4tNz2mTnkFSch41KvyG4pfz1q/IxYTfcUwM21Tmk7tl/lRd9UypQgaQODpfsMWOrmdRhQ7l7NRyzvD0VmEHsfMePlZiM+hd1MXHdy5TV4U+LZkL/PzMXSeu4v6Yzzm4GWYjE/R4R7RdmSBgcXfa4zflpHCw2bCthv16q76zVd6iy/SYntBT9OwL2qFd2sMJ4SGuz/SFvhrfjO/GD+NnAV1cKG0e0chj/P4Hc3SNRw==</latexit><latexit sha1_base64="RyBvzzo2oLyCDLYLhTluRKYkbJU=">AAAIInicpVVLT9RQFD6AWhxfoEs3jROCCxlbQiJLEjcsXKBhGBIkpu3cGZrpy/ZWJISda7f6A/w17owrE3+A/gu/c9ph3h2IbRhOz/3Od+553esmgZ9py/q1sLh04+YtY/l27c7de/cfrKw+PMjiPPVU04uDOD10nUwFfqSa2teBOkxS5YRuoFpu7yWvtz6oNPPjaF+fJeo4dLqR3/E9R7Nqd721/m6lbjUsecxJwS6FOpXPXry69IneUpti8iinkBRFpCEH5FCG94hssiiB7pjOoUsh+bKu6IJqsM2BUkA40Pbw28XXUamN8M2cmVh78BLgL4WlSWuwiYFLIbM3U9ZzYWbtLO5z4eS9neG/W3KF0Go6gXaeXR95VTuORVOHtiUGHzElouHovJIll6zwzs2hqDQYEuhYbmM9heyJZT/PpthkEjvn1pH1P4JkLX97JTanv5XRuWDlitT+KxL2GUiVTiEX+A1YmEP42fz76Bbm7sA+GmEu3jXhzCRP2+B1wcpZYDvz0qoqyhNUvcgp56gzxD3wMMAEsuteJWMuuHGeUcYMPCEsiwo36Q29kvoWfphVSwdE4KyqUIy3J33hAjFt7wl2EyMuJR3uS0a4Ihv0HqyORMR+zZKl6J95PT/I7DSfrswfW/EOuQ/b0PYnw7ycNJ7eKl+RnAxclaKTp/niGQ9LSz3BXfQAz4tNz2mTnkFSch41KvyG4pfz1q/IxYTfcUwM21Tmk7tl/lRd9UypQgaQODpfsMWOrmdRhQ7l7NRyzvD0VmEHsfMePlZiM+hd1MXHdy5TV4U+LZkL/PzMXSeu4v6Yzzm4GWYjE/R4R7RdmSBgcXfa4zflpHCw2bCthv16q76zVd6iy/SYntBT9OwL2qFd2sMJ4SGuz/SFvhrfjO/GD+NnAV1cKG0e0chj/P4Hc3SNRw==</latexit><latexit sha1_base64="RyBvzzo2oLyCDLYLhTluRKYkbJU=">AAAIInicpVVLT9RQFD6AWhxfoEs3jROCCxlbQiJLEjcsXKBhGBIkpu3cGZrpy/ZWJISda7f6A/w17owrE3+A/gu/c9ph3h2IbRhOz/3Od+553esmgZ9py/q1sLh04+YtY/l27c7de/cfrKw+PMjiPPVU04uDOD10nUwFfqSa2teBOkxS5YRuoFpu7yWvtz6oNPPjaF+fJeo4dLqR3/E9R7Nqd721/m6lbjUsecxJwS6FOpXPXry69IneUpti8iinkBRFpCEH5FCG94hssiiB7pjOoUsh+bKu6IJqsM2BUkA40Pbw28XXUamN8M2cmVh78BLgL4WlSWuwiYFLIbM3U9ZzYWbtLO5z4eS9neG/W3KF0Go6gXaeXR95VTuORVOHtiUGHzElouHovJIll6zwzs2hqDQYEuhYbmM9heyJZT/PpthkEjvn1pH1P4JkLX97JTanv5XRuWDlitT+KxL2GUiVTiEX+A1YmEP42fz76Bbm7sA+GmEu3jXhzCRP2+B1wcpZYDvz0qoqyhNUvcgp56gzxD3wMMAEsuteJWMuuHGeUcYMPCEsiwo36Q29kvoWfphVSwdE4KyqUIy3J33hAjFt7wl2EyMuJR3uS0a4Ihv0HqyORMR+zZKl6J95PT/I7DSfrswfW/EOuQ/b0PYnw7ycNJ7eKl+RnAxclaKTp/niGQ9LSz3BXfQAz4tNz2mTnkFSch41KvyG4pfz1q/IxYTfcUwM21Tmk7tl/lRd9UypQgaQODpfsMWOrmdRhQ7l7NRyzvD0VmEHsfMePlZiM+hd1MXHdy5TV4U+LZkL/PzMXSeu4v6Yzzm4GWYjE/R4R7RdmSBgcXfa4zflpHCw2bCthv16q76zVd6iy/SYntBT9OwL2qFd2sMJ4SGuz/SFvhrfjO/GD+NnAV1cKG0e0chj/P4Hc3SNRw==</latexit><latexit sha1_base64="RyBvzzo2oLyCDLYLhTluRKYkbJU=">AAAIInicpVVLT9RQFD6AWhxfoEs3jROCCxlbQiJLEjcsXKBhGBIkpu3cGZrpy/ZWJISda7f6A/w17owrE3+A/gu/c9ph3h2IbRhOz/3Od+553esmgZ9py/q1sLh04+YtY/l27c7de/cfrKw+PMjiPPVU04uDOD10nUwFfqSa2teBOkxS5YRuoFpu7yWvtz6oNPPjaF+fJeo4dLqR3/E9R7Nqd721/m6lbjUsecxJwS6FOpXPXry69IneUpti8iinkBRFpCEH5FCG94hssiiB7pjOoUsh+bKu6IJqsM2BUkA40Pbw28XXUamN8M2cmVh78BLgL4WlSWuwiYFLIbM3U9ZzYWbtLO5z4eS9neG/W3KF0Go6gXaeXR95VTuORVOHtiUGHzElouHovJIll6zwzs2hqDQYEuhYbmM9heyJZT/PpthkEjvn1pH1P4JkLX97JTanv5XRuWDlitT+KxL2GUiVTiEX+A1YmEP42fz76Bbm7sA+GmEu3jXhzCRP2+B1wcpZYDvz0qoqyhNUvcgp56gzxD3wMMAEsuteJWMuuHGeUcYMPCEsiwo36Q29kvoWfphVSwdE4KyqUIy3J33hAjFt7wl2EyMuJR3uS0a4Ihv0HqyORMR+zZKl6J95PT/I7DSfrswfW/EOuQ/b0PYnw7ycNJ7eKl+RnAxclaKTp/niGQ9LSz3BXfQAz4tNz2mTnkFSch41KvyG4pfz1q/IxYTfcUwM21Tmk7tl/lRd9UypQgaQODpfsMWOrmdRhQ7l7NRyzvD0VmEHsfMePlZiM+hd1MXHdy5TV4U+LZkL/PzMXSeu4v6Yzzm4GWYjE/R4R7RdmSBgcXfa4zflpHCw2bCthv16q76zVd6iy/SYntBT9OwL2qFd2sMJ4SGuz/SFvhrfjO/GD+NnAV1cKG0e0chj/P4Hc3SNRw==</latexit>

H 0W 0⇥H 0W 0<latexit sha1_base64="qCLR11vhe6RI4+XXv+Kf7FL4wqw=">AAAIM3icpVVLT9RQFD6AOji+Blm6KU4ILmToEBJZkrhh4QINw5AwhLSdO9BMX7a3IiHs/B9u9Qf4Y4w740IX/ge/c9ph3h2IbRhOz/3Od+553WtHnpto0/w+N79w5+690uL98oOHjx4/qSw9PUjCNHZUwwm9MD60rUR5bqAa2tWeOoxiZfm2p5p29zWvNz+oOHHDYF9fROrYt04Dt+M6lobqpLK8u9Zca620tOurpLVi8OdJpWrWTHmMcaGeC1XKn71waeETtahNITmUkk+KAtKQPbIowXtEdTIpgu6YLqGLIbmyruiKyrBNgVJAWNB28XuKr6NcG+CbOROxduDFw18MS4NWYRMCF0Nmb4asp8LM2mncl8LJe7vAfzvn8qHVdAbtLLse8qZ2HIumDm1LDC5iikTD0Tk5SypZ4Z0bA1FpMETQsdzGegzZEcteng2xSSR2zq0l678FyVr+dnJsSn8Ko7PByhUp/1ck7NOTKp1DzvDrsDAG8NP599EtzN2BfTDEnL2rwplInrbBa4OVs8B2xrVVUZRnqHqWU85RZ4C776GP8WTX3ULGVHCjPMOMCXh8WGYVbtA7eiP1zfwwq5YOCMBZVKEQb1f6wgZi0t4j7CZEXEo63JWMcEXW6T1YLYmI/Ro5S9Y/s3q+n9lJPm2ZP7biHXIftqHtTYZxPWk8vUW+AjkZuCpZJ0/yxTPu55Z6jDvrAZ6XOm3QJr2EpOQ8qhX49cUv561Xkasxv6OYELaxzCd3y+ypuumZUoT0IHF0rmCzHd3Oogjty9mp5Zzh6S3C9mPnPXwsxCbQ26iLi+9Upq4IfZ4zZ/jZmbtNXNn9MZuzfzNMR0bo8Y5oT2WCgMXdWR+9KceFg81a3azV325Vd7byW3SRntFzeoGefUU7tEt7OCEc7OEzfaGvpW+lH6WfpV8ZdH4ut1mmoaf09x9XaZKT</latexit><latexit sha1_base64="qCLR11vhe6RI4+XXv+Kf7FL4wqw=">AAAIM3icpVVLT9RQFD6AOji+Blm6KU4ILmToEBJZkrhh4QINw5AwhLSdO9BMX7a3IiHs/B9u9Qf4Y4w740IX/ge/c9ph3h2IbRhOz/3Od+553WtHnpto0/w+N79w5+690uL98oOHjx4/qSw9PUjCNHZUwwm9MD60rUR5bqAa2tWeOoxiZfm2p5p29zWvNz+oOHHDYF9fROrYt04Dt+M6lobqpLK8u9Zca620tOurpLVi8OdJpWrWTHmMcaGeC1XKn71waeETtahNITmUkk+KAtKQPbIowXtEdTIpgu6YLqGLIbmyruiKyrBNgVJAWNB28XuKr6NcG+CbOROxduDFw18MS4NWYRMCF0Nmb4asp8LM2mncl8LJe7vAfzvn8qHVdAbtLLse8qZ2HIumDm1LDC5iikTD0Tk5SypZ4Z0bA1FpMETQsdzGegzZEcteng2xSSR2zq0l678FyVr+dnJsSn8Ko7PByhUp/1ck7NOTKp1DzvDrsDAG8NP599EtzN2BfTDEnL2rwplInrbBa4OVs8B2xrVVUZRnqHqWU85RZ4C776GP8WTX3ULGVHCjPMOMCXh8WGYVbtA7eiP1zfwwq5YOCMBZVKEQb1f6wgZi0t4j7CZEXEo63JWMcEXW6T1YLYmI/Ro5S9Y/s3q+n9lJPm2ZP7biHXIftqHtTYZxPWk8vUW+AjkZuCpZJ0/yxTPu55Z6jDvrAZ6XOm3QJr2EpOQ8qhX49cUv561Xkasxv6OYELaxzCd3y+ypuumZUoT0IHF0rmCzHd3Oogjty9mp5Zzh6S3C9mPnPXwsxCbQ26iLi+9Upq4IfZ4zZ/jZmbtNXNn9MZuzfzNMR0bo8Y5oT2WCgMXdWR+9KceFg81a3azV325Vd7byW3SRntFzeoGefUU7tEt7OCEc7OEzfaGvpW+lH6WfpV8ZdH4ut1mmoaf09x9XaZKT</latexit><latexit sha1_base64="qCLR11vhe6RI4+XXv+Kf7FL4wqw=">AAAIM3icpVVLT9RQFD6AOji+Blm6KU4ILmToEBJZkrhh4QINw5AwhLSdO9BMX7a3IiHs/B9u9Qf4Y4w740IX/ge/c9ph3h2IbRhOz/3Od+553WtHnpto0/w+N79w5+690uL98oOHjx4/qSw9PUjCNHZUwwm9MD60rUR5bqAa2tWeOoxiZfm2p5p29zWvNz+oOHHDYF9fROrYt04Dt+M6lobqpLK8u9Zca620tOurpLVi8OdJpWrWTHmMcaGeC1XKn71waeETtahNITmUkk+KAtKQPbIowXtEdTIpgu6YLqGLIbmyruiKyrBNgVJAWNB28XuKr6NcG+CbOROxduDFw18MS4NWYRMCF0Nmb4asp8LM2mncl8LJe7vAfzvn8qHVdAbtLLse8qZ2HIumDm1LDC5iikTD0Tk5SypZ4Z0bA1FpMETQsdzGegzZEcteng2xSSR2zq0l678FyVr+dnJsSn8Ko7PByhUp/1ck7NOTKp1DzvDrsDAG8NP599EtzN2BfTDEnL2rwplInrbBa4OVs8B2xrVVUZRnqHqWU85RZ4C776GP8WTX3ULGVHCjPMOMCXh8WGYVbtA7eiP1zfwwq5YOCMBZVKEQb1f6wgZi0t4j7CZEXEo63JWMcEXW6T1YLYmI/Ro5S9Y/s3q+n9lJPm2ZP7biHXIftqHtTYZxPWk8vUW+AjkZuCpZJ0/yxTPu55Z6jDvrAZ6XOm3QJr2EpOQ8qhX49cUv561Xkasxv6OYELaxzCd3y+ypuumZUoT0IHF0rmCzHd3Oogjty9mp5Zzh6S3C9mPnPXwsxCbQ26iLi+9Upq4IfZ4zZ/jZmbtNXNn9MZuzfzNMR0bo8Y5oT2WCgMXdWR+9KceFg81a3azV325Vd7byW3SRntFzeoGefUU7tEt7OCEc7OEzfaGvpW+lH6WfpV8ZdH4ut1mmoaf09x9XaZKT</latexit><latexit sha1_base64="qCLR11vhe6RI4+XXv+Kf7FL4wqw=">AAAIM3icpVVLT9RQFD6AOji+Blm6KU4ILmToEBJZkrhh4QINw5AwhLSdO9BMX7a3IiHs/B9u9Qf4Y4w740IX/ge/c9ph3h2IbRhOz/3Od+553WtHnpto0/w+N79w5+690uL98oOHjx4/qSw9PUjCNHZUwwm9MD60rUR5bqAa2tWeOoxiZfm2p5p29zWvNz+oOHHDYF9fROrYt04Dt+M6lobqpLK8u9Zca620tOurpLVi8OdJpWrWTHmMcaGeC1XKn71waeETtahNITmUkk+KAtKQPbIowXtEdTIpgu6YLqGLIbmyruiKyrBNgVJAWNB28XuKr6NcG+CbOROxduDFw18MS4NWYRMCF0Nmb4asp8LM2mncl8LJe7vAfzvn8qHVdAbtLLse8qZ2HIumDm1LDC5iikTD0Tk5SypZ4Z0bA1FpMETQsdzGegzZEcteng2xSSR2zq0l678FyVr+dnJsSn8Ko7PByhUp/1ck7NOTKp1DzvDrsDAG8NP599EtzN2BfTDEnL2rwplInrbBa4OVs8B2xrVVUZRnqHqWU85RZ4C776GP8WTX3ULGVHCjPMOMCXh8WGYVbtA7eiP1zfwwq5YOCMBZVKEQb1f6wgZi0t4j7CZEXEo63JWMcEXW6T1YLYmI/Ro5S9Y/s3q+n9lJPm2ZP7biHXIftqHtTYZxPWk8vUW+AjkZuCpZJ0/yxTPu55Z6jDvrAZ6XOm3QJr2EpOQ8qhX49cUv561Xkasxv6OYELaxzCd3y+ypuumZUoT0IHF0rmCzHd3Oogjty9mp5Zzh6S3C9mPnPXwsxCbQ26iLi+9Upq4IfZ4zZ/jZmbtNXNn9MZuzfzNMR0bo8Y5oT2WCgMXdWR+9KceFg81a3azV325Vd7byW3SRntFzeoGefUU7tEt7OCEc7OEzfaGvpW+lH6WfpV8ZdH4ut1mmoaf09x9XaZKT</latexit>

Figure 5: An example for the upsam-ple operation in rl(·).

Notably, Eq. (9) is a simple but flexible formulation. For onething, each self-attention block in the downstream model hasaccess to all the upstream attention heads in both shallow anddeep layers, and hence can be trained to leverage multi-levelrelationship information on its own basis. For another, as thenewly generated attention maps and the reused relationshipsare combined in logits, their relative importance can be auto-matically learned by adjusting the magnitude of logits. It isalso worth noting that the regular upsample operation cannotbe directly applied in rl(·). To illustrate this issue, we takeupsampling a HW×HW (H=W =2) attention map to H ′W ′×H ′W ′ (H ′=W ′=3) for examplein Figure 5. Since each of its rows and columns corresponds to H×W image tokens, we reshape therows or columns back to H×W , scale them to H ′×W ′, and then flatten them to H ′W ′ vectors.

6

3.3 Adaptive InferenceAs aforementioned, the proposed DVT framework progressively increases the number of tokens foreach test sample and performs early-termination, such that “easy” and “hard” images can be processedusing varying tokens with uneven computational cost, improving the overall efficiency. Specifically,at the ith exit that produces the softmax prediction pi, the largest entry of pi, i.e., maxj pij (definedas confidence [23, 52, 47]), is compared with a threshold ηi. If maxj pij ≥ ηi, the inference will stopby adopting pi as the output. Otherwise, the image will be represented using more tokens to activatethe downstream Transformer. We always adopt a zero-threshold for the final Transformer.

The values of {η1, η2, . . .} are solved on the validation set. We assume a budgeted batch classification[23] setting, where DVT needs to recognize a set of samples Dval within a given computationalbudget B > 0. Let Acc(Dval, {η1, η2, . . .}) and FLOPs(Dval, {η1, η2, . . .}) denote the accuracy andcomputational cost on Dval when using the thresholds {η1, η2, . . .}. The optimal thresholds can beobtained by solving the following optimization problem:

maximizeη1,η2,...

Acc(Dval, {η1, η2, . . .}) subject to FLOPs(Dval, {η1, η2, . . .}) ≤ B. (10)

Due to the non-differentiability, we solve this problem with the genetic algorithm [48] in this paper.

4 ExperimentsIn this section, we empirically validate the proposed DVT on ImageNet [9] and CIFAR-10/100[26]. Ablation studies and visualization are presented on ImageNet to give a deeper understandingof our method. Code and pre-trained models based on PyTorch are available at https://github.com/blackfeather-wang/Dynamic-Vision-Transformer. We also provide the implementa-tion using the MindSpore framework and the models trained on a cluster of Ascend AI processors athttps://github.com/blackfeather-wang/Dynamic-Vision-Transformer-MindSpore.

Datasets. (1) ImageNet is a 1,000-class dataset from ILSVRC2012 [9], containing 1.2 million imagesfor training and 50,000 images for validation. (2) CIFAR-10/100 datasets [26] contain 32x32 coloredimages in 10/100 classes. Both of them consist of 50,000 images for training and 10,000 imagesfor testing. For all the three datasets, we adopt the same data pre-processing and data augmentationpolicy as [19, 24, 23]. In addition, we solve the confidence thresholds stated in Section 3.3 on thetraining set, which we find achieves similar performance to adopting cross-validation.

Backbones. Our experiments are based on several state-of-the-art vision Transformers, namelyT2T-ViT-12 [55], T2T-ViT-14 [55], and DeiT-small (w/o distillation) [38]. Unless otherwise specified,we deploy DVT with three exits, corresponding to representing the images as 7x7, 10x10 and 14x14tokens3. For fair comparisons, our implementation exploits the official code of the backbones, andadopts exactly the same training hyper-parameters. More training details can be found in AppendixA. The number of FLOPs is calculated using the fvcore toolkit provided by Facebook AI Research,which is also used in Detectron2 [50], PySlowFast [13], and ClassyVision [1].

Implementation details. For feature reuse, the hidden size and output size of the MLP in fl(·) areset to 128 and 48. In relationship reuse, for implementation efficiency, we share the same hiddenstate across the MLPs of all rl(·), such that rl(Aup), l ∈ {1, . . . , L} can be obtained at one time inconcatenation by implementing a single large MLP, whose hidden size and output size are 3NHL andNHL. Note that NH is the head number of multi-head attention and L is the layer number.

4.1 Main ResultsResults on ImageNet are shown in Figures 6 and 7, where T2T-ViT [55] and DeiT [38] are im-plemented as backbones respectively. As stated in Section 3.3, we vary the average computationalbudget, solve the confidence thresholds, and evaluate the corresponding validation accuracy. Theperformance of DVT is plotted in gray curves, with the best accuracy under each budget plotted inblack curves. We also compare our method with several highly competitive baselines, i.e., TNT [17],LocalViT [28], CrossViT [5], PVT [44], ViT [11] and ResNet [19]. It can be observed that DVTconsistently reduces the computational cost of the backbones. For example, DVT achieves the 82.3%accuracy with 3.6x less FLOPs compared with the vanilla T2T-ViT. When the budget ranges among0.5-2 GFLOPs, DVT has ∼1.7-1.9x less computation than T2T-ViT with the same performance.Notably, our method can flexibly attain all the points on each curve by simply adjusting the values ofconfidence thresholds with a single DVT.

3Although 4x4 tokens are also used as an example in Section 1, we find starting with 7x7 is more efficient.

7

1 2 3 4 5 6

GFLOPs/image

72

73

74

75

76

77

78

79

80

81

82

83

accu

racy

(%)

0 2 4 6 8 10 12 14 16 18

GFLOPs/image

71

72

73

74

75

76

77

78

79

80

81

82

83

84

accu

racy

(%)

DVT

T2T-ViT

TNT

LocalViT

CrossViT

PVT

ResNet

ViT

1.6x

1.7x

1.7x

1.9x

3.6x

Figure 6: Top-1 accuracy v.s. GFLOPs on ImageNet. DVT is implemented on top of T2T-ViT-12/14.

Table 2: The practical speed of DVT.

Models ImageNet (NVIDIA 2080Ti, bs=128)Top-1 Acc. Throughput

T2T-ViT-7 71.68% 1574 img/sDVT 78.48% (↑6.80%) 1574 img/s

T2T-ViT-10 75.15% 1286 img/sDVT 79.74% (↑4.59%) 1286 img/s

T2T-ViT-12 76.74% 1121 img/sDVT 80.43% (↑3.69%) 1128 img/s

T2T-ViT-14 81.50% 619 img/sDVT 81.50% 877 img/s (↑1.42x)

T2T-ViT-19 81.93% 382 img/sDVT 81.93% 666 img/s (↑1.74x)

Table 3: Performance of DVT on CIFAR-10/100.

Models CIFAR-10 CIFAR-100Top-1 Acc. GFLOPs Top-1 Acc. GFLOPs

T2T-ViT-10 97.21% 1.53 85.44% 1.53DVT 97.21% 0.50 (↓3.1x) 85.45% 0.54 (↓2.8x)

T2T-ViT-12 97.45% 1.78 86.23% 1.78DVT 97.46% 0.52 (↓3.4x) 86.26% 0.61 (↓2.9x)

T2T-ViT-14 98.19% 4.80 89.10% 4.80DVT 98.19% 0.77 (↓6.2x) 89.11% 1.62 (↓3.0x)

T2T-ViT-19 98.43% 8.50 89.37% 8.50DVT 98.43% 1.44 (↓5.9x) 89.38% 1.74 (↓4.9x)

T2T-ViT-24 98.53% 13.69 89.62% 13.69DVT 98.53% 1.49 (↓9.2x) 89.63% 1.86 (↓7.4x)

Practical efficiency of DVT. We test the actual speed of DVT on a NVIDIA 2080Ti GPU under abatch inference setting, where a mini-batch of data is fed into the model at a time. After inferring eachTransformer, the samples that meet the early-termination criterion will exit, with the remaining imagesfed into the downstream Transformer. The results are presented in Table 2. Here we adopt a two-exitDVT based on T2T-ViT-12 using 7x7 and 14x14 tokens, which we find more efficient in practice. Allother implementation details remain unchanged. One can observe that DVT improves the accuracy ofsmall models (T2T-ViT-7/10/12) by 3.7-6.8% with the same inference speed, while accelerates theinference of the large T2T-ViT-14/19 models by 1.4-1.7x without sacrificing performance.

Results on CIFAR are presented in Table 3. Following the common practice [11, 55, 17, 38], weresize the CIFAR images to 224x224, and fine-tune the T2T-ViT and DVT models in Figure 6. Theofficial code and training configurations provided by [55] are utilized. We report the computationalcost of DVT when it achieves the competitive performance with baselines. Our proposed method isshown to consume ∼3-9x less computation compared with T2T-ViT.

Table 4: Comparisons between DVT and three state-of-the-art methods for improving the compu-tational efficiency of vision Transformers, i.e., Data distillation [38], DynamicViT [32] and Patchslimming [36]. Both DVT and the baselines are implemented on top of DeiT-small. Since thecomputational cost of DVT can be adjusted online, we match the FLOPs or the accuracy of DVTwith the baselines, respectively, to see the difference.

DeiT-small Data distillation DVT DynamicViT DVT Patch slimming DVT

Top-1 Acc. 79.80% 81.20% 81.20% 81.67% 79.30% 79.30% 80.40% 79.40% 79.40% 79.81%GFLOPs/image 4.61 4.63 3.61 4.63 2.90 2.37 2.90 2.60 2.41 2.60

Comparisons with SOTA baselines. In Table 4, we compare DVT with several recently proposedapproaches for facilitating efficient vision Transformers on ImageNet. Both knowledge distillation(KD) [38] and patch pruning based methods [32, 36] are considered. One can observe that DVToutperforms the baselines with similar FLOPs, or has significantly lower computational cost with thesame accuracy. Besides, one can expect that DVT is compatible with these techniques, i.e., DVT canalso be trained with KD to further boost the accuracy or leverage patch pruning to reduce the FLOPs.

8

0 2 4 6 8 10 12 14 16 18

GFLOPs/image

72

73

74

75

76

77

78

79

80

81

82

83

accu

racy

(%)

DVT

DeiT

ResNet

ViT

1.7x

3.2x

Figure 7: Performance of DeiT-based DVT onImageNet. DeiT-small is used as the backbone.

0.5 1.0 1.5 2.0 2.5

GFLOPs/image

69

70

71

72

73

74

75

76

77

78

79

80

accu

racy

(%)

DVT

DVT (w/o relationship reuse)

DVT (w/o feature reuse)

DVT (w/o reuse)

T2T-ViT

Figure 8: Performance of the DVT based on T2T-ViT-12 with and without the reuse mechanisms.

Table 5: DVT v.s. existing multi-exit models with the same adap-tive inference strategy. The accuracy under each budget is reported.

Networks Top-1 Acc.0.75G 1.00G 1.25G 1.50G 1.75G 2.00G

MSDNet [23] 69.82% 71.24% 72.73% 73.66% 73.99% 74.20%IMTA-MSDNet [27] 70.84% 71.92% 73.41% 74.31% 74.64% 74.94%

RANet [52] 70.48% 72.24% 73.57% 74.56% 75.02% 75.10%DVT (T2T-ViT-12) 73.70% 76.22% 77.89% 78.89% 79.47% 79.75%

Comparisons with existingearly-exiting networks are pre-sented in Table 5. We adoptthe same adaptive strategy (see:Section 3.3) for all the base-lines. The Top-1 accuracy onImageNet under different com-putational budgets is reported.One can observe that DVT significantly outperforms these baselines.4.2 Ablation Study

Table 6: Effects of feature (F) and relationship (R) reuse. Thepercentages in brackets denote the additional computation com-pared to baselines involved by the reuse mechanisms.

Reuse 1st Exit (7x7) 2nd Exit (10x10) 3rd Exit (14x14)F R Top-1 Acc. GFLOPs Top-1 Acc. GFLOPs Top-1 Acc. GFLOPs

70.33% 0.47 73.54% 1.37 76.74% 3.153 69.42% 0.47 75.31% 1.43(4.4%) 79.21% 3.31(5.1%)

3 69.03% 0.47 75.34% 1.41(2.9%) 78.86% 3.34(6.0%)

3 3 69.04% 0.47 75.65% 1.46(6.6%) 80.00% 3.50(11.1%)

Effectiveness of feature and rela-tionship reuse. We conduct exper-iments by ablating one or both ofthe reuse mechanisms. For a clearcomparison, we first deactivate theearly-termination, and report theaccuracy and GFLOPs correspond-ing to each exit in Table 6. Thethree-exit DVT based on T2T-ViT-12 is considered. One can observe that both the two reuse mechanisms are able to significantly boostthe accuracy of DVT at the 2nd and 3rd exits with at most 6% additional computation, while theyare compatible with each other to further improve the performance. We also find that involvingcomputation reusing slightly hurts the accuracy at the 1st exit, which may be attributed to the compro-mise made by the first Transformer for downstream models. However, once the early-termination isadopted, this difference only results in trivial disadvantage when the computational budget is verysmall, as shown in Figure 8. DVT outperforms the baseline significantly in most cases.

Design choices for the reuse mechanisms. Here we study the design of the feature and relationshipreuse mechanisms. For experimental efficiency, we consider a two-exit DVT based on T2T-ViT-12using 7x7 and 10x10 tokens, while enlarge the batch size and the initial learning rate by 4 times. Sucha training setting slightly degrades the accuracy of DVT, but it is still reliable to reflect the differencebetween different design variants. We deactivate early-termination, and report the performance ofeach exit. Notably, as the FLOPs of 1st exit remain unchanged (i.e., 0.47G), we do not present it.

We consider four variants of feature reuse in Table 7: (1) reusing features from the correspondingupstream layer instead of the final layer; (2) reusing classification token; (3) only performing featurereuse in the first layer of the downstream model; (4) removing the LN in fl(·). One can see that takingfinal tokens of the upstream model and reusing them in each downstream layer are both important.

Ablation results for relationship reuse are presented in Table 8. We consider four variants as well:(1) only reusing the attention logits from the corresponding upstream layer; (2) only reusing theattention logits from the final upstream layer; (3) replacing the MLP by a linear layer in rl(·); (4)adopting naive upsample operation instead of what is shown in Figure 5; The results indicate that it isbeneficial to enable each downstream layer to flexibly reuse all upstream attention logits. Besides,naive upsampling significantly hurts the performance.

9

Table 7: Ablation studies for feature reuse.

Ablation 1st Exit (7x7) 2nd Exit (10x10)Top-1 Acc. Top-1 Acc. GFLOPs

w/o reuse 70.08% 73.61% 1.37Layer-wise feature reuse 69.84% 74.31% 1.43

Reuse classification token 69.79% 74.70% 1.43Remove fl(·), l ≥ 2 69.33% 74.73% 1.38Remove LN in fl(·) 69.63% 75.05% 1.42

Ours 69.44% 75.23% 1.43

Table 8: Ablation studies for relationship reuse.

Ablation 1st Exit (7x7) 2nd Exit (10x10)Top-1 Acc. Top-1 Acc. GFLOPs

w/o reuse 70.08% 73.61% 1.37Layer-wise relationship reuse 69.63% 73.89% 1.38Reuse final-layer relationships 69.25% 74.31% 1.39

MLP→Linear 69.20% 73.84% 1.38Naive upsample 69.60% 73.34% 1.41

Ours 69.50% 74.91% 1.41

Cockatoo

“Easy” “Hard”

Balloon

“Easy” “Hard” “Easy” “Hard”

Baseball

“Easy” “Hard”

Beer Bottle

Figure 9: Visualization of the “easy” and “hard” samples in DVT.

0.5 1.0 1.5 2.0 2.5

GFLOPs/image

69

70

71

72

73

74

75

76

77

78

79

80

accu

racy

(%)

DVT (T2T-ViT-12)

0

10000

20000

30000

40000

50000

num

ber

ofim

ages

1st Exit (7x7)

2nd Exit (10x10)

3rd Exit (14x14)

Figure 10: Numbers of images ex-iting at different exits with varyingcomputational budgets.

Table 9: Comparisons of early-termination criteri-ons. The accuracy under each budget is reported.

Ablation Top-1 Acc.0.75G 1.00G 1.25G 1.50G

Randomly Exit 70.19% 71.66% 72.61% 73.59%Entropy-based 73.41% 75.21% 77.08% 78.40%

Confidence-based (ours) 73.70% 76.22% 77.89% 78.89%

Early-termination criterion. We vary the cri-terion for adaptive inference based on DVT(T2T-ViT-12) and report the accuracy under sev-eral computational budgets in Table 9. Two vari-ants are considered: (1) adopting the entropyof softmax prediction to determine whether toexit [37]; (2) performing random exiting withthe same exit proportion as DVT. The simplebut effective confidence-based criterion achieves better performance than both of them.

Table 10: Effects of applying the adaptive depth mechanism tovision Transformers. The accuracy under each budget is reported.

Ablation Top-1 Acc.2.00G 2.50G 3.00G 3.50G 4.00G

T2T-ViT-14 with adaptive depth 76.09% 78.88% 79.46% 79.54% 79.55%DVT (T2T-ViT-14) 79.24% 80.61% 81.47% 82.10% 82.42%

Vision Transformers with adap-tive depth. We test applying theadaptive depth paradigm [23, 52]to ViT in Table 10. Two classi-fiers are uniformly added to the in-termediate layers of T2T-ViT-14,and the early-exiting is performedwith the three exits in the same way as DVT. One can observe that the layer-adaptive T2T-ViTachieves significantly lower accuracy than DVT using identical computational budgets. This inferiorperformance may be alleviated by improving the network architecture, as shown in [23].

4.3 VisualizationFigure 9 shows the images that are first correctly classified at the 1st and 3rd exits of the DVT (T2T-ViT-12). The former are recognized as “easy” samples, while the later are considered to be “hard”.One can observe that “easy” samples usually depict the recognition objectives in clear and canonicalposes and sufficiently large resolution. On the contrary, “hard” samples may contain complex scenesand non-typical poses or only include a small part of the objects, and require a finer representationusing more tokens. Figure 10 presents the numbers of images that exit at different exits when thecomputational budget increases. The plot shows that the accuracy of DVT is significantly improvedwith more images exiting later, which is achieved by changing the confidence thresholds online.

5 ConclusionIn this paper, we sought to optimally configure a proper number of tokens for each individual imagein vision Transformers, and hence proposed the Dynamic Vision Transformer (DVT) framework.DVT processes each test input by sequentially activating a cascade of Transformers using increasingnumbers of tokens, until an appropriate token number is reached (measured by the predictionconfidence). We further introduce the feature and relationship reuse mechanisms to facilitate efficientcomputation reuse. Extensive experiments indicate that DVT significantly improves the computationalefficiency on top of state-of-the-art vision Transformers, both theoretically and empirically.

10

AcknowledgementsThis work is supported in part by the National Science and Technology Major Project of the Ministryof Science and Technology of China under Grants 2018AAA0100701, the National Natural ScienceFoundation of China under Grants 61906106 and 62022048, and Huawei Technologies Ltd. Inparticular, we appreciate the valuable discussions with Chenghao Yang.

References

[1] A. Adcock, V. Reis, M. Singh, Z. Yan, L. van der Maaten, K. Zhang, S. Motwani, J. Guerin,N. Goyal, I. Misra, L. Gustafson, C. Changhan, and P. Goyal. Classy vision. https://github.com/facebookresearch/ClassyVision, 2019.

[2] Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. Layer normalization. arXiv preprintarXiv:1607.06450, 2016.

[3] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal,Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, ArielHerbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler,Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, ScottGray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, IlyaSutskever, and Dario Amodei. Language models are few-shot learners. In H. Larochelle,M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin, editors, NeurIPS, volume 33, pages 1877–1901. Curran Associates, Inc., 2020.

[4] Shijie Cao, Lingxiao Ma, Wencong Xiao, Chen Zhang, Yunxin Liu, Lintao Zhang, Lanshun Nie,and Zhi Yang. Seernet: Predicting convolutional neural network feature-map sparsity throughlow-bit quantization. In CVPR, pages 11216–11225, 2019.

[5] Chun-Fu Chen, Quanfu Fan, and Rameswar Panda. Crossvit: Cross-attention multi-scale visiontransformer for image classification. arXiv preprint arXiv:2103.14899, 2021.

[6] Xiangxiang Chu, Zhi Tian, Bo Zhang, Xinlong Wang, Xiaolin Wei, Huaxia Xia, andChunhua Shen. Conditional positional encodings for vision transformers. arXiv preprintarXiv:2102.10882, 2021.

[7] Ekin D Cubuk, Barret Zoph, Jonathon Shlens, and Quoc V Le. Randaugment: Practicalautomated data augmentation with a reduced search space. In CVPRW, pages 702–703, 2020.

[8] Stéphane d’Ascoli, Hugo Touvron, Matthew Leavitt, Ari Morcos, Giulio Biroli, and LeventSagun. Convit: Improving vision transformers with soft convolutional inductive biases. arXivpreprint arXiv:2103.10697, 2021.

[9] J. Deng, W. Dong, R. Socher, L. Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchicalimage database. In ICML, pages 248–255, 2009.

[10] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-trainingof deep bidirectional transformers for language understanding. In NAACL-HLT (1), pages4171–4186, Minneapolis, Minnesota, June 2019. Association for Computational Linguistics.

[11] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai,Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly,Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for imagerecognition at scale. In ICLR, 2021.

[12] Alaaeldin El-Nouby, Natalia Neverova, Ivan Laptev, and Hervé Jégou. Training vision trans-formers for image retrieval. arXiv preprint arXiv:2102.05644, 2021.

[13] Haoqi Fan, Yanghao Li, Bo Xiong, Wan-Yen Lo, and Christoph Feichtenhofer. Pyslowfast.https://github.com/facebookresearch/slowfast, 2020.

[14] Haoqi Fan, Bo Xiong, Karttikeya Mangalam, Yanghao Li, Zhicheng Yan, Jitendra Malik, andChristoph Feichtenhofer. Multiscale vision transformers. arXiv preprint arXiv:2104.11227,2021.

[15] Michael Figurnov, Maxwell D Collins, Yukun Zhu, Li Zhang, Jonathan Huang, Dmitry Vetrov,and Ruslan Salakhutdinov. Spatially adaptive computation time for residual networks. In CVPR,pages 1039–1048, 2017.

11

[16] Ben Graham, Alaaeldin El-Nouby, Hugo Touvron, Pierre Stock, Armand Joulin, Hervé Jégou,and Matthijs Douze. Levit: a vision transformer in convnet’s clothing for faster inference. arXivpreprint arXiv:2104.01136, 2021.

[17] Kai Han, An Xiao, Enhua Wu, Jianyuan Guo, Chunjing Xu, and Yunhe Wang. Transformer intransformer. arXiv preprint arXiv:2103.00112, 2021.

[18] Yizeng Han, Gao Huang, Shiji Song, Le Yang, Honghui Wang, and Yulin Wang. Dynamicneural networks: A survey. arXiv preprint arXiv:2102.04906, 2021.

[19] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for imagerecognition. In CVPR, pages 770–778, 2016.

[20] Shuting He, Hao Luo, Pichao Wang, Fan Wang, Hao Li, and Wei Jiang. Transreid: Transformer-based object re-identification. arXiv preprint arXiv:2102.04378, 2021.

[21] Andrew Howard, Mark Sandler, Grace Chu, Liang-Chieh Chen, Bo Chen, Mingxing Tan,Weijun Wang, Yukun Zhu, Ruoming Pang, Vijay Vasudevan, et al. Searching for mobilenetv3.In ICCV, pages 1314–1324, 2019.

[22] Andrew G Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, TobiasWeyand, Marco Andreetto, and Hartwig Adam. Mobilenets: Efficient convolutional neuralnetworks for mobile vision applications. arXiv preprint arXiv:1704.04861, 2017.

[23] Gao Huang, Danlu Chen, Tianhong Li, Felix Wu, Laurens van der Maaten, and Kilian QWeinberger. Multi-scale dense networks for resource efficient image classification. In ICLR,2018.

[24] Gao Huang, Zhuang Liu, Geoff Pleiss, Laurens Van Der Maaten, and Kilian Weinberger.Convolutional networks with dense connectivity. IEEE Transactions on Pattern Analysis andMachine Intelligence, pages 1–1, 2019.

[25] Gao Huang, Yu Sun, Zhuang Liu, Daniel Sedra, and Kilian Q Weinberger. Deep networks withstochastic depth. In ECCV, pages 646–661. Springer, 2016.

[26] Alex Krizhevsky and Geoffrey Hinton. Learning multiple layers of features from tiny images.Technical report, Citeseer, 2009.

[27] Hao Li, Hong Zhang, Xiaojuan Qi, Ruigang Yang, and Gao Huang. Improved techniques fortraining adaptive deep networks. In ICCV, pages 1891–1900, 2019.

[28] Yawei Li, Kai Zhang, Jiezhang Cao, Radu Timofte, and Luc Van Gool. Localvit: Bringinglocality to vision transformers. arXiv preprint arXiv:2104.05707, 2021.

[29] Ji Lin, Yongming Rao, Jiwen Lu, and Jie Zhou. Runtime neural pruning. In NeurIPS, pages2181–2191, 2017.

[30] Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and BainingGuo. Swin transformer: Hierarchical vision transformer using shifted windows. arXiv preprintarXiv:2103.14030, 2021.

[31] Ningning Ma, Xiangyu Zhang, Hai-Tao Zheng, and Jian Sun. Shufflenet v2: Practical guidelinesfor efficient cnn architecture design. In ECCV, pages 116–131, 2018.

[32] Yongming Rao, Wenliang Zhao, Benlin Liu, Jiwen Lu, Jie Zhou, and Cho-Jui Hsieh. Dy-namicvit: Efficient vision transformers with dynamic token sparsification. arXiv preprintarXiv:2106.02034, 2021.

[33] Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chieh Chen.Mobilenetv2: Inverted residuals and linear bottlenecks. In CVPR, pages 4510–4520, 2018.

[34] Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. Re-thinking the inception architecture for computer vision. In CVPR, pages 2818–2826, 2016.

[35] Mingxing Tan and Quoc V. Le. Efficientnet: Rethinking model scaling for convolutional neuralnetworks. In ICML, volume 97, pages 6105–6114, 2019.

[36] Yehui Tang, Kai Han, Yunhe Wang, Chang Xu, Jianyuan Guo, Chao Xu, and Dacheng Tao.Patch slimming for efficient vision transformers. arXiv preprint arXiv:2106.02852, 2021.

[37] Surat Teerapittayanon, Bradley McDanel, and Hsiang-Tsung Kung. Branchynet: Fast inferencevia early exiting from deep neural networks. In ICPR, pages 2464–2469. IEEE, 2016.

12

[38] Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, andHervé Jégou. Training data-efficient image transformers & distillation through attention. arXivpreprint arXiv:2012.12877, 2020.

[39] Hugo Touvron, Matthieu Cord, Alexandre Sablayrolles, Gabriel Synnaeve, and Hervé Jégou.Going deeper with image transformers. arXiv preprint arXiv:2103.17239, 2021.

[40] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez,Ł ukasz Kaiser, and Illia Polosukhin. Attention is all you need. In I. Guyon, U. V. Luxburg,S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, NeurIPS, volume 30.Curran Associates, Inc., 2017.

[41] Andreas Veit and Serge Belongie. Convolutional networks with adaptive inference graphs. InECCV, pages 3–18, 2018.

[42] Thomas Verelst and Tinne Tuytelaars. Dynamic convolutions: Exploiting spatial sparsity forfaster inference. In CVPR, pages 2320–2329, 2020.

[43] Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman.GLUE: A multi-task benchmark and analysis platform for natural language understanding. InICLR, 2019.

[44] Wenhai Wang, Enze Xie, Xiang Li, Deng-Ping Fan, Kaitao Song, Ding Liang, Tong Lu, PingLuo, and Ling Shao. Pyramid vision transformer: A versatile backbone for dense predictionwithout convolutions. arXiv preprint arXiv:2102.12122, 2021.

[45] Xin Wang, Fisher Yu, Zi-Yi Dou, Trevor Darrell, and Joseph E Gonzalez. Skipnet: Learningdynamic routing in convolutional networks. In ECCV, pages 409–424, 2018.

[46] Yulin Wang, Zhaoxi Chen, Haojun Jiang, Shiji Song, Yizeng Han, and Gao Huang. Adaptivefocus for efficient video recognition. arXiv preprint arXiv:2105.03245, 2021.

[47] Yulin Wang, Kangchen Lv, Rui Huang, Shiji Song, Le Yang, and Gao Huang. Glance and focus:a dynamic approach to reducing spatial redundancy in image classification. In NeurIPS, 2020.

[48] Darrell Whitley. A genetic algorithm tutorial. Statistics and computing, 4(2):65–85, 1994.

[49] Haiping Wu, Bin Xiao, Noel Codella, Mengchen Liu, Xiyang Dai, Lu Yuan, and Lei Zhang.Cvt: Introducing convolutions to vision transformers. arXiv preprint arXiv:2103.15808, 2021.

[50] Yuxin Wu, Alexander Kirillov, Francisco Massa, Wan-Yen Lo, and Ross Girshick. Detectron2.https://github.com/facebookresearch/detectron2, 2019.

[51] Zuxuan Wu, Tushar Nagarajan, Abhishek Kumar, Steven Rennie, Larry S Davis, KristenGrauman, and Rogerio Feris. Blockdrop: Dynamic inference paths in residual networks. InCVPR, pages 8817–8826, 2018.

[52] Le Yang, Yizeng Han, Xi Chen, Shiji Song, Jifeng Dai, and Gao Huang. Resolution adaptivenetworks for efficient inference. In CVPR, pages 2369–2378, 2020.

[53] Le Yang, Haojun Jiang, Ruojin Cai, Yulin Wang, Shiji Song, Gao Huang, and Qi Tian. Con-densenet v2: Sparse feature reactivation for deep networks. In CVPR, 2021.

[54] Kun Yuan, Shaopeng Guo, Ziwei Liu, Aojun Zhou, Fengwei Yu, and Wei Wu. Incorporatingconvolution designs into visual transformers. arXiv preprint arXiv:2103.11816, 2021.

[55] Li Yuan, Yunpeng Chen, Tao Wang, Weihao Yu, Yujun Shi, Francis EH Tay, Jiashi Feng, andShuicheng Yan. Tokens-to-token vit: Training vision transformers from scratch on imagenet.arXiv preprint arXiv:2101.11986, 2021.

[56] Sangdoo Yun, Dongyoon Han, Seong Joon Oh, Sanghyuk Chun, Junsuk Choe, and YoungjoonYoo. Cutmix: Regularization strategy to train strong classifiers with localizable features. InPICCV, pages 6023–6032, 2019.

[57] Fangao Zeng, Bin Dong, Tiancai Wang, Cheng Chen, Xiangyu Zhang, and Yichen Wei. Motr:End-to-end multiple-object tracking with transformer. arXiv preprint arXiv:2105.03247, 2021.

[58] Hongyi Zhang, Moustapha Cisse, Yann N. Dauphin, and David Lopez-Paz. mixup: Beyondempirical risk minimization. In ICLR, 2018.

[59] Xiangyu Zhang, Xinyu Zhou, Mengxiao Lin, and Jian Sun. Shufflenet: An extremely efficientconvolutional neural network for mobile devices. In CVPR, pages 6848–6856, 2018.

13

[60] Moju Zhao, Kei Okada, and Masayuki Inaba. Trtr: Visual tracking with transformer. arXivpreprint arXiv:2105.03817, 2021.

[61] Zhun Zhong, Liang Zheng, Guoliang Kang, Shaozi Li, and Yi Yang. Random erasing dataaugmentation. In AAAI, volume 34, pages 13001–13008, 2020.

[62] Daquan Zhou, Bingyi Kang, Xiaojie Jin, Linjie Yang, Xiaochen Lian, Zihang Jiang, Qibin Hou,and Jiashi Feng. Deepvit: Towards deeper vision transformer. arXiv preprint arXiv:2103.11886,2021.

14

Appendix

A Training Details

The proposed DVT framework is implemented based on the official code of T2T-ViT4 [55] andDeiT5 [38] when these two models are used as backbones, respectively. The training of DVTfollows exactly the same configurations as the backbones, which are also recommended by theirofficial implementations. The details on optimizer, learning rate schedule, batch size and otherhyper-parameters can be easily found in their papers or code. Note that a number of regularizationand data augmentation techniques are exploited, including RandAugment [7], Random Erasing [61],Label Smoothing [34], Mixup [58], Cutmix [56], and stochastic depth [25]. We train all the modelswith 8 NVIDIA V100 GPUs.

Table 11: Effects when the location for performing feature reuse varies.

Ablation 1st Exit (7x7) 2nd Exit (10x10)Top-1 Acc. Top-1 Acc. GFLOPs

w/o reuse 70.08% 73.61% 1.37Reuse on shallow half of downstream layers 69.67% 75.02% 1.40

Reuse on deep half of downstream layers 69.94% 74.57% 1.40

Reuse on all downstream layers 69.44% 75.23% 1.43

Table 12: Effects when the location for performing relationship reusevaries.

Ablation 1st Exit (7x7) 2nd Exit (10x10)Top-1 Acc. Top-1 Acc. GFLOPs

w/o reuse 70.08% 73.61% 1.37Reuse on shallow half of downstream layers 69.72% 74.89% 1.40

Reuse on deep half of downstream layers 70.00% 73.68% 1.40

Reuse on all downstream layers 69.50% 74.91% 1.41

Table 13: Effects of taking attention logits from varying upstream layers.

Ablation 1st Exit (7x7) 2nd Exit (10x10)Top-1 Acc. Top-1 Acc. GFLOPs

w/o reuse 70.08% 73.61% 1.37Reuse relationships from shallow half of upstream layers 69.93% 74.67% 1.40

Reuse relationships from deep half of upstream layers 69.55% 74.58% 1.40

Reuse relationships from all upstream layers 69.50% 74.91% 1.41

B Additional Results

Which downstream layers benefit more from reuse? To shed light on the layer-wise reuseparadigm in the downstream model, we test performing feature or relationship reuse only in theshallow/deep half of downstream layers. The same experimental protocol as ablating the design ofreuse mechanisms is adopted. The results are shown in Tables 11 and 12. Obviously, it is moreimportant to reuse upstream features/relationships at the shallow layers. This phenomenon indicatesthat the main effects of the proposed reuse mechanisms lie in helping the first several transformerlayers to rapidly extract discriminative representations or to learn accurate attention maps. Thesuccessive layers focus more on further improving the shallow features, while less on leveragingupstream information, which may have been effectively integrated into the shallow layers.

Which upstream layers contribute more to relationship reuse? In Table 13, we further test onlytaking the attention logits from the shallow/deep half of upstream layers in relationship reuse. Onecan observe that both shallow and deep relationships are important for boosting the accuracy of the

4https://github.com/yitu-opensource/T2T-ViT5https://github.com/facebookresearch/deit

15

0.0 0.5 1.0 1.5 2.0 2.5 3.0

average budget (in MUL-ADD)

96.60

96.85

97.10

97.35

97.60

97.85

98.10

98.35

98.60

accu

racy

(%)

DVT

T2T-ViT

0 2 4 6 8 10 12 14

average budget (in MUL-ADD)

96.5

97.0

97.5

98.0

98.5

99.0

accu

racy

(%)

DVT

T2T-ViT

Figure 11: Top-1 accuracy v.s. GFLOPs on CIFAR-10. DVT is implemented on top of T2T-ViT-12/14.

0.0 0.5 1.0 1.5 2.0 2.5 3.0

average budget (in MUL-ADD)

83

84

85

86

87

88

89

90

accu

racy

(%)

DVT

T2T-ViT

0 2 4 6 8 10 12 14

average budget (in MUL-ADD)

82

83

84

85

86

87

88

89

90

91

accu

racy

(%)

DVT

T2T-ViT

Figure 12: Top-1 accuracy v.s. GFLOPs on CIFAR-100. DVT is implemented on top of T2T-ViT-12/14.

400 600 800 1000 1200 1400 1600

throughput (img/s)

68

69

70

71

72

73

74

75

76

77

78

79

80

81

82

83

accu

racy

(%)

DVT

T2T-ViT

Figure 13: Top-1 accuracy v.s. throughput on ImageNet. The results are obtained on NVIDIA 2080TiGPU with a batch size of 128.

downstream model. In addition, it is interesting that reusing more relationships only slightly improvesthe performance. We attribute this to the redundancy within the learned attention logits from differentlayers of the upstream model.

Top-1 accuracy v.s. GFLOPs curves on CIFAR-10/100 are presented in Figures 11 and 12, respec-tively, corresponding to the results reported in Table 3 of the paper.

Top-1 accuracy v.s. throughput curves on ImageNet are presented in Figures 13 corresponding tothe results reported in Table 2 of the paper.

16