• Home

  • Documents

  • Imitation Learning for Human Pose Prediction › pdf › 1909.03449v1.pdf · Imitation Learning for...
10
Imitation Learning for Human Pose Prediction Borui Wang, Ehsan Adeli, Hsu-kuang Chiu, De-An Huang, Juan Carlos Niebles Stanford University {wbr, eadeli, hkchiu, dahuang, jniebles}@cs.stanford.edu Abstract Modeling and prediction of human motion dynamics has long been a challenging problem in computer vision, and most existing methods rely on the end-to-end supervised training of various architectures of recurrent neural net- works. Inspired by the recent success of deep reinforce- ment learning methods, in this paper we propose a new reinforcement learning formulation for the problem of hu- man pose prediction, and develop an imitation learning al- gorithm for predicting future poses under this formulation through a combination of behavioral cloning and genera- tive adversarial imitation learning. Our experiments show that our proposed method outperforms all existing state- of-the-art baseline models by large margins on the task of human pose prediction in both short-term predictions and long-term predictions, while also enjoying huge advantage in training speed. 1. Introduction Modeling the dynamics of human motion and predict- ing human poses is an important and challenging problem in computer vision that has many useful applications in robotics, computer graphics, healthcare, public safety, etc. [14, 20, 21, 42, 43]. Previous work on this subject has mainly been focusing on designing different architectures of recurrent neural networks (RNNs) to model human mo- tion dynamics and adopted a pure supervised-learning ap- proach to train recurrent neural networks to predict future human poses in a sequence [7, 9, 19, 26]. These previous methods based on supervised training of RNN architectures face two main challenges: (1) due to the purely supervised nature of the training methods, the learned RNNs usually do not generalize well to unseen do- mains of the human motion space, which are very likely to appear during test time; (2) since the RNNs are required to generate the whole human pose prediction sequence all to- gether, it is very difficult to keep a good balance between short-term and long-term prediction accuracies. Recently, we have witnessed great success in the devel- ⇡ ✓ Action State Expert Demonstration (ground truth) D w Critic Score Policy Gradient Figure 1: At the core of our imitation learning approach to human pose prediction is a Generative Adversarial Imita- tion Learning (GAIL) [15] process. With the critic function D w and the policy generator π θ , we alternate between up- dating D w (D step) through comparing generated windows of predicted poses with ground truth, and updating π θ (G step) through policy gradient over critic scores from D w . opment of deep reinforcement learning algorithms, which have achieved significantly enhanced state-of-the-art per- formance on many tasks in the areas of game, robotics and control [28, 35, 36]. These recent deep reinforcement learn- ing algorithms, including Generative Adversarial Imitation Learning (GAIL) [15] and Deep Deterministic Policy Gra- dient [23], enjoy the advantages of generalizing well over unseen terrains and maintaining strong sequential correla- tion among local decisions across time. Therefore, in or- der to overcome the above limitations of the previous meth- ods for human pose prediction, a natural question to ask is whether we could build a bridge between reinforcement learning and sequential modeling, such that we can harness the great power of deep reinforcement learning algorithms to help us better model human motion dynamics and pre- dict human poses from sequential observations. In this pa- per, we propose a new modeling framework for human mo- tion dynamics that transforms the task of predicting human poses into a reinforcement learning problem. The environ- ment of this reinforcement learning problem cannot pro- vide feedback signals as the learning agent interacts with it. All we have during the learning process is a training dataset consisting of trajectories of human poses recorded from real human motion. Therefore, we adopt an imitation learning [33, 52] approach and use the training dataset as 1 arXiv:1909.03449v1 [cs.CV] 8 Sep 2019

Imitation Learning for Human Pose Prediction › pdf › 1909.03449v1.pdf · Imitation Learning for Human Pose Prediction Borui Wang, Ehsan Adeli, Hsu-kuang Chiu, De-An Huang, Juan

  • Upload
    others

  • View
    3

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Imitation Learning for Human Pose Prediction › pdf › 1909.03449v1.pdf · Imitation Learning for Human Pose Prediction Borui Wang, Ehsan Adeli, Hsu-kuang Chiu, De-An Huang, Juan

Imitation Learning for Human Pose Prediction

Borui Wang, Ehsan Adeli, Hsu-kuang Chiu, De-An Huang, Juan Carlos NieblesStanford University

{wbr, eadeli, hkchiu, dahuang, jniebles}@cs.stanford.edu

Abstract

Modeling and prediction of human motion dynamics haslong been a challenging problem in computer vision, andmost existing methods rely on the end-to-end supervisedtraining of various architectures of recurrent neural net-works. Inspired by the recent success of deep reinforce-ment learning methods, in this paper we propose a newreinforcement learning formulation for the problem of hu-man pose prediction, and develop an imitation learning al-gorithm for predicting future poses under this formulationthrough a combination of behavioral cloning and genera-tive adversarial imitation learning. Our experiments showthat our proposed method outperforms all existing state-of-the-art baseline models by large margins on the task ofhuman pose prediction in both short-term predictions andlong-term predictions, while also enjoying huge advantagein training speed.

1. IntroductionModeling the dynamics of human motion and predict-

ing human poses is an important and challenging problemin computer vision that has many useful applications inrobotics, computer graphics, healthcare, public safety, etc.[14, 20, 21, 42, 43]. Previous work on this subject hasmainly been focusing on designing different architecturesof recurrent neural networks (RNNs) to model human mo-tion dynamics and adopted a pure supervised-learning ap-proach to train recurrent neural networks to predict futurehuman poses in a sequence [7, 9, 19, 26].

These previous methods based on supervised training ofRNN architectures face two main challenges: (1) due tothe purely supervised nature of the training methods, thelearned RNNs usually do not generalize well to unseen do-mains of the human motion space, which are very likely toappear during test time; (2) since the RNNs are required togenerate the whole human pose prediction sequence all to-gether, it is very difficult to keep a good balance betweenshort-term and long-term prediction accuracies.

Recently, we have witnessed great success in the devel-

⇡✓<latexit sha1_base64="OIbg92+c4CxHGOlWATrm7gBa05w=">AAAB83icbZBNSwMxEIZn61etX1WPXoJF8FR2RdBjwYMeK9ha6C4lm6ZtaDYbklmhLP0bXjwo4tU/481/Y9ruQVtfCDy8M8NM3lhLYdH3v73S2vrG5lZ5u7Kzu7d/UD08ats0M4y3WCpT04mp5VIo3kKBkne04TSJJX+Mxzez+uMTN1ak6gEnmkcJHSoxEIyis8JQi14e4ogjnfaqNb/uz0VWISigBoWavepX2E9ZlnCFTFJru4GvMcqpQcEkn1bCzHJN2ZgOedehogm3UT6/eUrOnNMng9S4p5DM3d8TOU2snSSx60wojuxybWb+V+tmOLiOcqF0hlyxxaJBJgmmZBYA6QvDGcqJA8qMcLcSNqKGMnQxVVwIwfKXV6F9UQ8c31/WGrdFHGU4gVM4hwCuoAF30IQWMNDwDK/w5mXei/fufSxaS14xcwx/5H3+AHRzkfU=</latexit><latexit sha1_base64="OIbg92+c4CxHGOlWATrm7gBa05w=">AAAB83icbZBNSwMxEIZn61etX1WPXoJF8FR2RdBjwYMeK9ha6C4lm6ZtaDYbklmhLP0bXjwo4tU/481/Y9ruQVtfCDy8M8NM3lhLYdH3v73S2vrG5lZ5u7Kzu7d/UD08ats0M4y3WCpT04mp5VIo3kKBkne04TSJJX+Mxzez+uMTN1ak6gEnmkcJHSoxEIyis8JQi14e4ogjnfaqNb/uz0VWISigBoWavepX2E9ZlnCFTFJru4GvMcqpQcEkn1bCzHJN2ZgOedehogm3UT6/eUrOnNMng9S4p5DM3d8TOU2snSSx60wojuxybWb+V+tmOLiOcqF0hlyxxaJBJgmmZBYA6QvDGcqJA8qMcLcSNqKGMnQxVVwIwfKXV6F9UQ8c31/WGrdFHGU4gVM4hwCuoAF30IQWMNDwDK/w5mXei/fufSxaS14xcwx/5H3+AHRzkfU=</latexit><latexit sha1_base64="OIbg92+c4CxHGOlWATrm7gBa05w=">AAAB83icbZBNSwMxEIZn61etX1WPXoJF8FR2RdBjwYMeK9ha6C4lm6ZtaDYbklmhLP0bXjwo4tU/481/Y9ruQVtfCDy8M8NM3lhLYdH3v73S2vrG5lZ5u7Kzu7d/UD08ats0M4y3WCpT04mp5VIo3kKBkne04TSJJX+Mxzez+uMTN1ak6gEnmkcJHSoxEIyis8JQi14e4ogjnfaqNb/uz0VWISigBoWavepX2E9ZlnCFTFJru4GvMcqpQcEkn1bCzHJN2ZgOedehogm3UT6/eUrOnNMng9S4p5DM3d8TOU2snSSx60wojuxybWb+V+tmOLiOcqF0hlyxxaJBJgmmZBYA6QvDGcqJA8qMcLcSNqKGMnQxVVwIwfKXV6F9UQ8c31/WGrdFHGU4gVM4hwCuoAF30IQWMNDwDK/w5mXei/fufSxaS14xcwx/5H3+AHRzkfU=</latexit><latexit sha1_base64="OIbg92+c4CxHGOlWATrm7gBa05w=">AAAB83icbZBNSwMxEIZn61etX1WPXoJF8FR2RdBjwYMeK9ha6C4lm6ZtaDYbklmhLP0bXjwo4tU/481/Y9ruQVtfCDy8M8NM3lhLYdH3v73S2vrG5lZ5u7Kzu7d/UD08ats0M4y3WCpT04mp5VIo3kKBkne04TSJJX+Mxzez+uMTN1ak6gEnmkcJHSoxEIyis8JQi14e4ogjnfaqNb/uz0VWISigBoWavepX2E9ZlnCFTFJru4GvMcqpQcEkn1bCzHJN2ZgOedehogm3UT6/eUrOnNMng9S4p5DM3d8TOU2snSSx60wojuxybWb+V+tmOLiOcqF0hlyxxaJBJgmmZBYA6QvDGcqJA8qMcLcSNqKGMnQxVVwIwfKXV6F9UQ8c31/WGrdFHGU4gVM4hwCuoAF30IQWMNDwDK/w5mXei/fufSxaS14xcwx/5H3+AHRzkfU=</latexit>

Action<latexit sha1_base64="vKwTLujVWtosADprQrUPqsaEl28=">AAAB7XicdVDLSgMxFM34rPVVdekmWARXw2Tqq7uqG5cV7APaoWTSTBubSYYkI5Sh/+DGhSJu/R93/o2ZtoKKHgg5nHMv994TJpxp43kfzsLi0vLKamGtuL6xubVd2tltapkqQhtEcqnaIdaUM0EbhhlO24miOA45bYWjq9xv3VOlmRS3ZpzQIMYDwSJGsLFS84LkX69U9tzqqY9QFXpupXKMPN8Sr4JO/CpErjdFGcxR75Xeu31J0pgKQzjWuoO8xAQZVoYRTifFbqppgskID2jHUoFjqoNsuu0EHlqlDyOp7BMGTtXvHRmOtR7Hoa2MsRnq314u/uV1UhOdBxkTSWqoILNBUcqhkTA/HfaZosTwsSWYKGZ3hWSIFSbGBlS0IXxdCv8nTd9Flt/45drlPI4C2AcH4AggcAZq4BrUQQMQcAcewBN4dqTz6Lw4r7PSBWfeswd+wHn7BNzvj08=</latexit><latexit sha1_base64="vKwTLujVWtosADprQrUPqsaEl28=">AAAB7XicdVDLSgMxFM34rPVVdekmWARXw2Tqq7uqG5cV7APaoWTSTBubSYYkI5Sh/+DGhSJu/R93/o2ZtoKKHgg5nHMv994TJpxp43kfzsLi0vLKamGtuL6xubVd2tltapkqQhtEcqnaIdaUM0EbhhlO24miOA45bYWjq9xv3VOlmRS3ZpzQIMYDwSJGsLFS84LkX69U9tzqqY9QFXpupXKMPN8Sr4JO/CpErjdFGcxR75Xeu31J0pgKQzjWuoO8xAQZVoYRTifFbqppgskID2jHUoFjqoNsuu0EHlqlDyOp7BMGTtXvHRmOtR7Hoa2MsRnq314u/uV1UhOdBxkTSWqoILNBUcqhkTA/HfaZosTwsSWYKGZ3hWSIFSbGBlS0IXxdCv8nTd9Flt/45drlPI4C2AcH4AggcAZq4BrUQQMQcAcewBN4dqTz6Lw4r7PSBWfeswd+wHn7BNzvj08=</latexit><latexit sha1_base64="vKwTLujVWtosADprQrUPqsaEl28=">AAAB7XicdVDLSgMxFM34rPVVdekmWARXw2Tqq7uqG5cV7APaoWTSTBubSYYkI5Sh/+DGhSJu/R93/o2ZtoKKHgg5nHMv994TJpxp43kfzsLi0vLKamGtuL6xubVd2tltapkqQhtEcqnaIdaUM0EbhhlO24miOA45bYWjq9xv3VOlmRS3ZpzQIMYDwSJGsLFS84LkX69U9tzqqY9QFXpupXKMPN8Sr4JO/CpErjdFGcxR75Xeu31J0pgKQzjWuoO8xAQZVoYRTifFbqppgskID2jHUoFjqoNsuu0EHlqlDyOp7BMGTtXvHRmOtR7Hoa2MsRnq314u/uV1UhOdBxkTSWqoILNBUcqhkTA/HfaZosTwsSWYKGZ3hWSIFSbGBlS0IXxdCv8nTd9Flt/45drlPI4C2AcH4AggcAZq4BrUQQMQcAcewBN4dqTz6Lw4r7PSBWfeswd+wHn7BNzvj08=</latexit><latexit sha1_base64="vKwTLujVWtosADprQrUPqsaEl28=">AAAB7XicdVDLSgMxFM34rPVVdekmWARXw2Tqq7uqG5cV7APaoWTSTBubSYYkI5Sh/+DGhSJu/R93/o2ZtoKKHgg5nHMv994TJpxp43kfzsLi0vLKamGtuL6xubVd2tltapkqQhtEcqnaIdaUM0EbhhlO24miOA45bYWjq9xv3VOlmRS3ZpzQIMYDwSJGsLFS84LkX69U9tzqqY9QFXpupXKMPN8Sr4JO/CpErjdFGcxR75Xeu31J0pgKQzjWuoO8xAQZVoYRTifFbqppgskID2jHUoFjqoNsuu0EHlqlDyOp7BMGTtXvHRmOtR7Hoa2MsRnq314u/uV1UhOdBxkTSWqoILNBUcqhkTA/HfaZosTwsSWYKGZ3hWSIFSbGBlS0IXxdCv8nTd9Flt/45drlPI4C2AcH4AggcAZq4BrUQQMQcAcewBN4dqTz6Lw4r7PSBWfeswd+wHn7BNzvj08=</latexit>

State<latexit sha1_base64="R5huLpyHotCvcOLgCdeMQdMZEp4=">AAAB7HicdVBNS8NAEN3Ur1q/qh69LBbBU0japtVb0YvHirYW2lA22027dLMJuxOhhP4GLx4U8eoP8ua/cfshqOiDgcd7M8zMCxLBNTjOh5VbWV1b38hvFra2d3b3ivsHbR2nirIWjUWsOgHRTHDJWsBBsE6iGIkCwe6C8eXMv7tnSvNY3sIkYX5EhpKHnBIwUusGCLB+seTYrlvxzj3s2FWvXvUqhtRrNbfmYdd25iihJZr94ntvENM0YhKoIFp3XScBPyMKOBVsWuilmiWEjsmQdQ2VJGLaz+bHTvGJUQY4jJUpCXiufp/ISKT1JApMZ0RgpH97M/Evr5tCeOZnXCYpMEkXi8JUYIjx7HM84IpREBNDCFXc3IrpiChCweRTMCF8fYr/J+2yCcp2rsulxsUyjjw6QsfoFLmojhroCjVRC1HE0QN6Qs+WtB6tF+t10ZqzljOH6Aest09Iyo7+</latexit><latexit sha1_base64="R5huLpyHotCvcOLgCdeMQdMZEp4=">AAAB7HicdVBNS8NAEN3Ur1q/qh69LBbBU0japtVb0YvHirYW2lA22027dLMJuxOhhP4GLx4U8eoP8ua/cfshqOiDgcd7M8zMCxLBNTjOh5VbWV1b38hvFra2d3b3ivsHbR2nirIWjUWsOgHRTHDJWsBBsE6iGIkCwe6C8eXMv7tnSvNY3sIkYX5EhpKHnBIwUusGCLB+seTYrlvxzj3s2FWvXvUqhtRrNbfmYdd25iihJZr94ntvENM0YhKoIFp3XScBPyMKOBVsWuilmiWEjsmQdQ2VJGLaz+bHTvGJUQY4jJUpCXiufp/ISKT1JApMZ0RgpH97M/Evr5tCeOZnXCYpMEkXi8JUYIjx7HM84IpREBNDCFXc3IrpiChCweRTMCF8fYr/J+2yCcp2rsulxsUyjjw6QsfoFLmojhroCjVRC1HE0QN6Qs+WtB6tF+t10ZqzljOH6Aest09Iyo7+</latexit><latexit sha1_base64="R5huLpyHotCvcOLgCdeMQdMZEp4=">AAAB7HicdVBNS8NAEN3Ur1q/qh69LBbBU0japtVb0YvHirYW2lA22027dLMJuxOhhP4GLx4U8eoP8ua/cfshqOiDgcd7M8zMCxLBNTjOh5VbWV1b38hvFra2d3b3ivsHbR2nirIWjUWsOgHRTHDJWsBBsE6iGIkCwe6C8eXMv7tnSvNY3sIkYX5EhpKHnBIwUusGCLB+seTYrlvxzj3s2FWvXvUqhtRrNbfmYdd25iihJZr94ntvENM0YhKoIFp3XScBPyMKOBVsWuilmiWEjsmQdQ2VJGLaz+bHTvGJUQY4jJUpCXiufp/ISKT1JApMZ0RgpH97M/Evr5tCeOZnXCYpMEkXi8JUYIjx7HM84IpREBNDCFXc3IrpiChCweRTMCF8fYr/J+2yCcp2rsulxsUyjjw6QsfoFLmojhroCjVRC1HE0QN6Qs+WtB6tF+t10ZqzljOH6Aest09Iyo7+</latexit><latexit sha1_base64="R5huLpyHotCvcOLgCdeMQdMZEp4=">AAAB7HicdVBNS8NAEN3Ur1q/qh69LBbBU0japtVb0YvHirYW2lA22027dLMJuxOhhP4GLx4U8eoP8ua/cfshqOiDgcd7M8zMCxLBNTjOh5VbWV1b38hvFra2d3b3ivsHbR2nirIWjUWsOgHRTHDJWsBBsE6iGIkCwe6C8eXMv7tnSvNY3sIkYX5EhpKHnBIwUusGCLB+seTYrlvxzj3s2FWvXvUqhtRrNbfmYdd25iihJZr94ntvENM0YhKoIFp3XScBPyMKOBVsWuilmiWEjsmQdQ2VJGLaz+bHTvGJUQY4jJUpCXiufp/ISKT1JApMZ0RgpH97M/Evr5tCeOZnXCYpMEkXi8JUYIjx7HM84IpREBNDCFXc3IrpiChCweRTMCF8fYr/J+2yCcp2rsulxsUyjjw6QsfoFLmojhroCjVRC1HE0QN6Qs+WtB6tF+t10ZqzljOH6Aest09Iyo7+</latexit>

Expert Demonstration<latexit sha1_base64="hduk12YgwCstIgnWl8autOcieio=">AAAB/XicdVDLSgNBEJyNrxhf8XHzMhgET2E3BhJv4gM8RjBGSBaZnXR0yOzsMtMrxkX8FS8eFPHqf3jzb5yNK6hoQUNR1U13VxBLYdB1353CxOTU9ExxtjQ3v7C4VF5eOTVRojm0eSQjfRYwA1IoaKNACWexBhYGEjrBcD/zO1egjYjUCY5i8EN2ocRAcIZWOi+vHV7HoJEeQBgpgzqXK27VHYO61e1mo17PiFdr7Hge9XKrQnK0zstvvX7EkxAUcsmM6XpujH7KNAou4bbUSwzEjA/ZBXQtVSwE46fj62/pplX6dBBpWwrpWP0+kbLQmFEY2M6Q4aX57WXiX143wUHTT4WKEwTFPxcNEkkxolkUtC80cJQjSxjXwt5K+SXTjKMNrGRD+PqU/k9Oa1XP8uNaZXcvj6NI1skG2SIeaZBdckRapE04uSH35JE8OXfOg/PsvHy2Fpx8ZpX8gPP6ARJelZ8=</latexit><latexit sha1_base64="hduk12YgwCstIgnWl8autOcieio=">AAAB/XicdVDLSgNBEJyNrxhf8XHzMhgET2E3BhJv4gM8RjBGSBaZnXR0yOzsMtMrxkX8FS8eFPHqf3jzb5yNK6hoQUNR1U13VxBLYdB1353CxOTU9ExxtjQ3v7C4VF5eOTVRojm0eSQjfRYwA1IoaKNACWexBhYGEjrBcD/zO1egjYjUCY5i8EN2ocRAcIZWOi+vHV7HoJEeQBgpgzqXK27VHYO61e1mo17PiFdr7Hge9XKrQnK0zstvvX7EkxAUcsmM6XpujH7KNAou4bbUSwzEjA/ZBXQtVSwE46fj62/pplX6dBBpWwrpWP0+kbLQmFEY2M6Q4aX57WXiX143wUHTT4WKEwTFPxcNEkkxolkUtC80cJQjSxjXwt5K+SXTjKMNrGRD+PqU/k9Oa1XP8uNaZXcvj6NI1skG2SIeaZBdckRapE04uSH35JE8OXfOg/PsvHy2Fpx8ZpX8gPP6ARJelZ8=</latexit><latexit sha1_base64="hduk12YgwCstIgnWl8autOcieio=">AAAB/XicdVDLSgNBEJyNrxhf8XHzMhgET2E3BhJv4gM8RjBGSBaZnXR0yOzsMtMrxkX8FS8eFPHqf3jzb5yNK6hoQUNR1U13VxBLYdB1353CxOTU9ExxtjQ3v7C4VF5eOTVRojm0eSQjfRYwA1IoaKNACWexBhYGEjrBcD/zO1egjYjUCY5i8EN2ocRAcIZWOi+vHV7HoJEeQBgpgzqXK27VHYO61e1mo17PiFdr7Hge9XKrQnK0zstvvX7EkxAUcsmM6XpujH7KNAou4bbUSwzEjA/ZBXQtVSwE46fj62/pplX6dBBpWwrpWP0+kbLQmFEY2M6Q4aX57WXiX143wUHTT4WKEwTFPxcNEkkxolkUtC80cJQjSxjXwt5K+SXTjKMNrGRD+PqU/k9Oa1XP8uNaZXcvj6NI1skG2SIeaZBdckRapE04uSH35JE8OXfOg/PsvHy2Fpx8ZpX8gPP6ARJelZ8=</latexit><latexit sha1_base64="hduk12YgwCstIgnWl8autOcieio=">AAAB/XicdVDLSgNBEJyNrxhf8XHzMhgET2E3BhJv4gM8RjBGSBaZnXR0yOzsMtMrxkX8FS8eFPHqf3jzb5yNK6hoQUNR1U13VxBLYdB1353CxOTU9ExxtjQ3v7C4VF5eOTVRojm0eSQjfRYwA1IoaKNACWexBhYGEjrBcD/zO1egjYjUCY5i8EN2ocRAcIZWOi+vHV7HoJEeQBgpgzqXK27VHYO61e1mo17PiFdr7Hge9XKrQnK0zstvvX7EkxAUcsmM6XpujH7KNAou4bbUSwzEjA/ZBXQtVSwE46fj62/pplX6dBBpWwrpWP0+kbLQmFEY2M6Q4aX57WXiX143wUHTT4WKEwTFPxcNEkkxolkUtC80cJQjSxjXwt5K+SXTjKMNrGRD+PqU/k9Oa1XP8uNaZXcvj6NI1skG2SIeaZBdckRapE04uSH35JE8OXfOg/PsvHy2Fpx8ZpX8gPP6ARJelZ8=</latexit>

(ground truth)<latexit sha1_base64="XwHk7jwYR3P1TL0olGzrqgbarBg=">AAAB9XicdVBNS8NAEN3Ur1q/qh69LBahXkpSC623ohePFewHtLFsNtt26WYTdidKCf0fXjwo4tX/4s1/46aNoKIPBh7vzTAzz4sE12DbH1ZuZXVtfSO/Wdja3tndK+4fdHQYK8raNBSh6nlEM8ElawMHwXqRYiTwBOt608vU794xpXkob2AWMTcgY8lHnBIw0m15rMJY+hhUDJPTYbFkV+wFsF05a9RrtZQ41fq542Ans0ooQ2tYfB/4IY0DJoEKonXfsSNwE6KAU8HmhUGsWUTolIxZ31BJAqbdZHH1HJ8YxcejUJmSgBfq94mEBFrPAs90BgQm+reXin95/RhGDTfhMoqBSbpcNIoFhhCnEWCfK0ZBzAwhVHFzK6YToggFE1TBhPD1Kf6fdKoVx/Draql5kcWRR0foGJWRg+qoia5QC7URRQo9oCf0bN1bj9aL9bpszVnZzCH6AevtE10Rkmo=</latexit><latexit sha1_base64="XwHk7jwYR3P1TL0olGzrqgbarBg=">AAAB9XicdVBNS8NAEN3Ur1q/qh69LBahXkpSC623ohePFewHtLFsNtt26WYTdidKCf0fXjwo4tX/4s1/46aNoKIPBh7vzTAzz4sE12DbH1ZuZXVtfSO/Wdja3tndK+4fdHQYK8raNBSh6nlEM8ElawMHwXqRYiTwBOt608vU794xpXkob2AWMTcgY8lHnBIw0m15rMJY+hhUDJPTYbFkV+wFsF05a9RrtZQ41fq542Ans0ooQ2tYfB/4IY0DJoEKonXfsSNwE6KAU8HmhUGsWUTolIxZ31BJAqbdZHH1HJ8YxcejUJmSgBfq94mEBFrPAs90BgQm+reXin95/RhGDTfhMoqBSbpcNIoFhhCnEWCfK0ZBzAwhVHFzK6YToggFE1TBhPD1Kf6fdKoVx/Draql5kcWRR0foGJWRg+qoia5QC7URRQo9oCf0bN1bj9aL9bpszVnZzCH6AevtE10Rkmo=</latexit><latexit sha1_base64="XwHk7jwYR3P1TL0olGzrqgbarBg=">AAAB9XicdVBNS8NAEN3Ur1q/qh69LBahXkpSC623ohePFewHtLFsNtt26WYTdidKCf0fXjwo4tX/4s1/46aNoKIPBh7vzTAzz4sE12DbH1ZuZXVtfSO/Wdja3tndK+4fdHQYK8raNBSh6nlEM8ElawMHwXqRYiTwBOt608vU794xpXkob2AWMTcgY8lHnBIw0m15rMJY+hhUDJPTYbFkV+wFsF05a9RrtZQ41fq542Ans0ooQ2tYfB/4IY0DJoEKonXfsSNwE6KAU8HmhUGsWUTolIxZ31BJAqbdZHH1HJ8YxcejUJmSgBfq94mEBFrPAs90BgQm+reXin95/RhGDTfhMoqBSbpcNIoFhhCnEWCfK0ZBzAwhVHFzK6YToggFE1TBhPD1Kf6fdKoVx/Draql5kcWRR0foGJWRg+qoia5QC7URRQo9oCf0bN1bj9aL9bpszVnZzCH6AevtE10Rkmo=</latexit><latexit sha1_base64="XwHk7jwYR3P1TL0olGzrqgbarBg=">AAAB9XicdVBNS8NAEN3Ur1q/qh69LBahXkpSC623ohePFewHtLFsNtt26WYTdidKCf0fXjwo4tX/4s1/46aNoKIPBh7vzTAzz4sE12DbH1ZuZXVtfSO/Wdja3tndK+4fdHQYK8raNBSh6nlEM8ElawMHwXqRYiTwBOt608vU794xpXkob2AWMTcgY8lHnBIw0m15rMJY+hhUDJPTYbFkV+wFsF05a9RrtZQ41fq542Ans0ooQ2tYfB/4IY0DJoEKonXfsSNwE6KAU8HmhUGsWUTolIxZ31BJAqbdZHH1HJ8YxcejUJmSgBfq94mEBFrPAs90BgQm+reXin95/RhGDTfhMoqBSbpcNIoFhhCnEWCfK0ZBzAwhVHFzK6YToggFE1TBhPD1Kf6fdKoVx/Draql5kcWRR0foGJWRg+qoia5QC7URRQo9oCf0bN1bj9aL9bpszVnZzCH6AevtE10Rkmo=</latexit>

Dw<latexit sha1_base64="ecy/Lu16SXDuJl8ama0PHC3vS6g=">AAAB6nicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRYsAePFe0HtKFstpN26WYTdjdKCf0JXjwo4tVf5M1/47bNQVtfWHh4Z4adeYNEcG1c99sprK1vbG4Vt0s7u3v7B+XDo5aOU8WwyWIRq05ANQousWm4EdhJFNIoENgOxjezevsRleaxfDCTBP2IDiUPOaPGWvf1/lO/XHGr7lxkFbwcKpCr0S9/9QYxSyOUhgmqdddzE+NnVBnOBE5LvVRjQtmYDrFrUdIItZ/NV52SM+sMSBgr+6Qhc/f3REYjrSdRYDsjakZ6uTYz/6t1UxNe+xmXSWpQssVHYSqIicnsbjLgCpkREwuUKW53JWxEFWXGplOyIXjLJ69C66LqWb67rNTqeRxFOIFTOAcPrqAGt9CAJjAYwjO8wpsjnBfn3flYtBacfOYY/sj5/AEnLI20</latexit><latexit sha1_base64="ecy/Lu16SXDuJl8ama0PHC3vS6g=">AAAB6nicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRYsAePFe0HtKFstpN26WYTdjdKCf0JXjwo4tVf5M1/47bNQVtfWHh4Z4adeYNEcG1c99sprK1vbG4Vt0s7u3v7B+XDo5aOU8WwyWIRq05ANQousWm4EdhJFNIoENgOxjezevsRleaxfDCTBP2IDiUPOaPGWvf1/lO/XHGr7lxkFbwcKpCr0S9/9QYxSyOUhgmqdddzE+NnVBnOBE5LvVRjQtmYDrFrUdIItZ/NV52SM+sMSBgr+6Qhc/f3REYjrSdRYDsjakZ6uTYz/6t1UxNe+xmXSWpQssVHYSqIicnsbjLgCpkREwuUKW53JWxEFWXGplOyIXjLJ69C66LqWb67rNTqeRxFOIFTOAcPrqAGt9CAJjAYwjO8wpsjnBfn3flYtBacfOYY/sj5/AEnLI20</latexit><latexit sha1_base64="ecy/Lu16SXDuJl8ama0PHC3vS6g=">AAAB6nicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRYsAePFe0HtKFstpN26WYTdjdKCf0JXjwo4tVf5M1/47bNQVtfWHh4Z4adeYNEcG1c99sprK1vbG4Vt0s7u3v7B+XDo5aOU8WwyWIRq05ANQousWm4EdhJFNIoENgOxjezevsRleaxfDCTBP2IDiUPOaPGWvf1/lO/XHGr7lxkFbwcKpCr0S9/9QYxSyOUhgmqdddzE+NnVBnOBE5LvVRjQtmYDrFrUdIItZ/NV52SM+sMSBgr+6Qhc/f3REYjrSdRYDsjakZ6uTYz/6t1UxNe+xmXSWpQssVHYSqIicnsbjLgCpkREwuUKW53JWxEFWXGplOyIXjLJ69C66LqWb67rNTqeRxFOIFTOAcPrqAGt9CAJjAYwjO8wpsjnBfn3flYtBacfOYY/sj5/AEnLI20</latexit><latexit sha1_base64="ecy/Lu16SXDuJl8ama0PHC3vS6g=">AAAB6nicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRYsAePFe0HtKFstpN26WYTdjdKCf0JXjwo4tVf5M1/47bNQVtfWHh4Z4adeYNEcG1c99sprK1vbG4Vt0s7u3v7B+XDo5aOU8WwyWIRq05ANQousWm4EdhJFNIoENgOxjezevsRleaxfDCTBP2IDiUPOaPGWvf1/lO/XHGr7lxkFbwcKpCr0S9/9QYxSyOUhgmqdddzE+NnVBnOBE5LvVRjQtmYDrFrUdIItZ/NV52SM+sMSBgr+6Qhc/f3REYjrSdRYDsjakZ6uTYz/6t1UxNe+xmXSWpQssVHYSqIicnsbjLgCpkREwuUKW53JWxEFWXGplOyIXjLJ69C66LqWb67rNTqeRxFOIFTOAcPrqAGt9CAJjAYwjO8wpsjnBfn3flYtBacfOYY/sj5/AEnLI20</latexit>

Critic Score<latexit sha1_base64="beih6b2j10ehglSr97kEGWErXXQ=">AAAB83icbZDLTgIxFIbP4A3xhrp000hMXJEZNrokYeMSo1wSmJBOOQMNnXbSdkwI4TXcuNAYt76MO9/GArNQ8E+afPnPOW3PH6WCG+v7315ha3tnd6+4Xzo4PDo+KZ+etY3KNMMWU0LpbkQNCi6xZbkV2E010iQS2IkmjUW984TacCUf7TTFMKEjyWPOqHVWv6G55Yw8MKVxUK74VX8psglBDhXI1RyUv/pDxbIEpWWCGtML/NSGM6rdlQLnpX5mMKVsQkfYcyhpgiacLf88J1fOGZJYaXekJUv398SMJsZMk8h1JtSOzXptYf5X62U2vg1nXKaZRclWD8WZIFaRRQBkyDUyK6YOKFutz8ZUU2ZdTCUXQrC+8ia0a9XA8X2tUq/ncRThAi7hGgK4gTrcQRNawCCFZ3iFNy/zXrx372PVWvDymXP4I+/zB6b3kWg=</latexit><latexit sha1_base64="beih6b2j10ehglSr97kEGWErXXQ=">AAAB83icbZDLTgIxFIbP4A3xhrp000hMXJEZNrokYeMSo1wSmJBOOQMNnXbSdkwI4TXcuNAYt76MO9/GArNQ8E+afPnPOW3PH6WCG+v7315ha3tnd6+4Xzo4PDo+KZ+etY3KNMMWU0LpbkQNCi6xZbkV2E010iQS2IkmjUW984TacCUf7TTFMKEjyWPOqHVWv6G55Yw8MKVxUK74VX8psglBDhXI1RyUv/pDxbIEpWWCGtML/NSGM6rdlQLnpX5mMKVsQkfYcyhpgiacLf88J1fOGZJYaXekJUv398SMJsZMk8h1JtSOzXptYf5X62U2vg1nXKaZRclWD8WZIFaRRQBkyDUyK6YOKFutz8ZUU2ZdTCUXQrC+8ia0a9XA8X2tUq/ncRThAi7hGgK4gTrcQRNawCCFZ3iFNy/zXrx372PVWvDymXP4I+/zB6b3kWg=</latexit><latexit sha1_base64="beih6b2j10ehglSr97kEGWErXXQ=">AAAB83icbZDLTgIxFIbP4A3xhrp000hMXJEZNrokYeMSo1wSmJBOOQMNnXbSdkwI4TXcuNAYt76MO9/GArNQ8E+afPnPOW3PH6WCG+v7315ha3tnd6+4Xzo4PDo+KZ+etY3KNMMWU0LpbkQNCi6xZbkV2E010iQS2IkmjUW984TacCUf7TTFMKEjyWPOqHVWv6G55Yw8MKVxUK74VX8psglBDhXI1RyUv/pDxbIEpWWCGtML/NSGM6rdlQLnpX5mMKVsQkfYcyhpgiacLf88J1fOGZJYaXekJUv398SMJsZMk8h1JtSOzXptYf5X62U2vg1nXKaZRclWD8WZIFaRRQBkyDUyK6YOKFutz8ZUU2ZdTCUXQrC+8ia0a9XA8X2tUq/ncRThAi7hGgK4gTrcQRNawCCFZ3iFNy/zXrx372PVWvDymXP4I+/zB6b3kWg=</latexit><latexit sha1_base64="beih6b2j10ehglSr97kEGWErXXQ=">AAAB83icbZDLTgIxFIbP4A3xhrp000hMXJEZNrokYeMSo1wSmJBOOQMNnXbSdkwI4TXcuNAYt76MO9/GArNQ8E+afPnPOW3PH6WCG+v7315ha3tnd6+4Xzo4PDo+KZ+etY3KNMMWU0LpbkQNCi6xZbkV2E010iQS2IkmjUW984TacCUf7TTFMKEjyWPOqHVWv6G55Yw8MKVxUK74VX8psglBDhXI1RyUv/pDxbIEpWWCGtML/NSGM6rdlQLnpX5mMKVsQkfYcyhpgiacLf88J1fOGZJYaXekJUv398SMJsZMk8h1JtSOzXptYf5X62U2vg1nXKaZRclWD8WZIFaRRQBkyDUyK6YOKFutz8ZUU2ZdTCUXQrC+8ia0a9XA8X2tUq/ncRThAi7hGgK4gTrcQRNawCCFZ3iFNy/zXrx372PVWvDymXP4I+/zB6b3kWg=</latexit>

Policy Gradient<latexit sha1_base64="7EP7U2tO7tpxRVGH69ib9MIncAg=">AAAB+HicdVDLSgMxFM34rPXRUZdugkVwVWbGYrssutBlBfuAdiiZTKYNzSRDkhHGoV/ixoUibv0Ud/6NaTuCih4IHM65h3tzgoRRpR3nw1pZXVvf2Cxtlbd3dvcq9v5BV4lUYtLBggnZD5AijHLS0VQz0k8kQXHASC+YXs793h2Rigp+q7OE+DEacxpRjLSRRnalLRjFGbySKKSE65FddWrOAtCpNZxzz2sa0qzXzxp16BZWFRRoj+z3YShwGpssZkipgesk2s+R1BQzMisPU0UShKdoTAaGchQT5eeLw2fwxCghjIQ0j2u4UL8nchQrlcWBmYyRnqjf3lz8yxukOmr6OeVJqgnHy0VRyqAWcN4CDKkkWLPMEIQlNbdCPEESYW26KpsSvn4K/yddr+YafuNVWxdFHSVwBI7BKXBBA7TANWiDDsAgBQ/gCTxb99aj9WK9LkdXrCJzCH7AevsE1wOTNA==</latexit><latexit sha1_base64="7EP7U2tO7tpxRVGH69ib9MIncAg=">AAAB+HicdVDLSgMxFM34rPXRUZdugkVwVWbGYrssutBlBfuAdiiZTKYNzSRDkhHGoV/ixoUibv0Ud/6NaTuCih4IHM65h3tzgoRRpR3nw1pZXVvf2Cxtlbd3dvcq9v5BV4lUYtLBggnZD5AijHLS0VQz0k8kQXHASC+YXs793h2Rigp+q7OE+DEacxpRjLSRRnalLRjFGbySKKSE65FddWrOAtCpNZxzz2sa0qzXzxp16BZWFRRoj+z3YShwGpssZkipgesk2s+R1BQzMisPU0UShKdoTAaGchQT5eeLw2fwxCghjIQ0j2u4UL8nchQrlcWBmYyRnqjf3lz8yxukOmr6OeVJqgnHy0VRyqAWcN4CDKkkWLPMEIQlNbdCPEESYW26KpsSvn4K/yddr+YafuNVWxdFHSVwBI7BKXBBA7TANWiDDsAgBQ/gCTxb99aj9WK9LkdXrCJzCH7AevsE1wOTNA==</latexit><latexit sha1_base64="7EP7U2tO7tpxRVGH69ib9MIncAg=">AAAB+HicdVDLSgMxFM34rPXRUZdugkVwVWbGYrssutBlBfuAdiiZTKYNzSRDkhHGoV/ixoUibv0Ud/6NaTuCih4IHM65h3tzgoRRpR3nw1pZXVvf2Cxtlbd3dvcq9v5BV4lUYtLBggnZD5AijHLS0VQz0k8kQXHASC+YXs793h2Rigp+q7OE+DEacxpRjLSRRnalLRjFGbySKKSE65FddWrOAtCpNZxzz2sa0qzXzxp16BZWFRRoj+z3YShwGpssZkipgesk2s+R1BQzMisPU0UShKdoTAaGchQT5eeLw2fwxCghjIQ0j2u4UL8nchQrlcWBmYyRnqjf3lz8yxukOmr6OeVJqgnHy0VRyqAWcN4CDKkkWLPMEIQlNbdCPEESYW26KpsSvn4K/yddr+YafuNVWxdFHSVwBI7BKXBBA7TANWiDDsAgBQ/gCTxb99aj9WK9LkdXrCJzCH7AevsE1wOTNA==</latexit><latexit sha1_base64="7EP7U2tO7tpxRVGH69ib9MIncAg=">AAAB+HicdVDLSgMxFM34rPXRUZdugkVwVWbGYrssutBlBfuAdiiZTKYNzSRDkhHGoV/ixoUibv0Ud/6NaTuCih4IHM65h3tzgoRRpR3nw1pZXVvf2Cxtlbd3dvcq9v5BV4lUYtLBggnZD5AijHLS0VQz0k8kQXHASC+YXs793h2Rigp+q7OE+DEacxpRjLSRRnalLRjFGbySKKSE65FddWrOAtCpNZxzz2sa0qzXzxp16BZWFRRoj+z3YShwGpssZkipgesk2s+R1BQzMisPU0UShKdoTAaGchQT5eeLw2fwxCghjIQ0j2u4UL8nchQrlcWBmYyRnqjf3lz8yxukOmr6OeVJqgnHy0VRyqAWcN4CDKkkWLPMEIQlNbdCPEESYW26KpsSvn4K/yddr+YafuNVWxdFHSVwBI7BKXBBA7TANWiDDsAgBQ/gCTxb99aj9WK9LkdXrCJzCH7AevsE1wOTNA==</latexit>

Figure 1: At the core of our imitation learning approach tohuman pose prediction is a Generative Adversarial Imita-tion Learning (GAIL) [15] process. With the critic functionDw and the policy generator πθ, we alternate between up-dating Dw (D step) through comparing generated windowsof predicted poses with ground truth, and updating πθ (Gstep) through policy gradient over critic scores from Dw.

opment of deep reinforcement learning algorithms, whichhave achieved significantly enhanced state-of-the-art per-formance on many tasks in the areas of game, robotics andcontrol [28, 35, 36]. These recent deep reinforcement learn-ing algorithms, including Generative Adversarial ImitationLearning (GAIL) [15] and Deep Deterministic Policy Gra-dient [23], enjoy the advantages of generalizing well overunseen terrains and maintaining strong sequential correla-tion among local decisions across time. Therefore, in or-der to overcome the above limitations of the previous meth-ods for human pose prediction, a natural question to askis whether we could build a bridge between reinforcementlearning and sequential modeling, such that we can harnessthe great power of deep reinforcement learning algorithmsto help us better model human motion dynamics and pre-dict human poses from sequential observations. In this pa-per, we propose a new modeling framework for human mo-tion dynamics that transforms the task of predicting humanposes into a reinforcement learning problem. The environ-ment of this reinforcement learning problem cannot pro-vide feedback signals as the learning agent interacts withit. All we have during the learning process is a trainingdataset consisting of trajectories of human poses recordedfrom real human motion. Therefore, we adopt an imitationlearning [33, 52] approach and use the training dataset as

1

arX

iv:1

909.

0344

9v1

[cs

.CV

] 8

Sep

201

9

Page 2: Imitation Learning for Human Pose Prediction › pdf › 1909.03449v1.pdf · Imitation Learning for Human Pose Prediction Borui Wang, Ehsan Adeli, Hsu-kuang Chiu, De-An Huang, Juan

expert demonstrations for our prediction agent to imitate.This imitation learning problem of predicting human

poses has several key difficulties: (1) the action spaceis continuous-valued and very high-dimensional; (2) thestate space is heterogeneous and encompasses very long se-quences of historical observations with different lengths; (3)the expert demonstrations are performed by different humansubjects and thus exhibit large variance across the under-lying expert policies. To tackle this challenging imitationlearning task, we extend the Generative Adversarial Imita-tion Learning (see Figure 1) framework with sequence-to-sequence architectures [39] and Deep Deterministic PolicyGradient [23] methods to train our prediction agent to makeaccurate predictions of human poses. We also use the ef-ficient behavioral cloning [4] algorithm as pre-training toexpedite the training process.

We evaluate the performance of our proposed imitationlearning algorithm on the popular Human 3.6M dataset[17]. Our experiments demonstrate that our proposed al-gorithm outperforms all the previous methods for humanpose prediction by large margins on both short-term andlong-term predictions and sets the new state-of-the-art per-formance results on the Human 3.6M dataset. The experi-ments also show that our algorithm has huge advantage inspeed and can be trained much more efficiently comparedto previous algorithms.

To summarize, the main contributions of our work are:(1) We propose a new reinforcement learning formulationfor the problem of human pose prediction that supportsmore accurate predictions over both short-term and long-term horizons; (2) We develop an imitation learning algo-rithm for human pose prediction based on this reinforce-ment learning formulation through a combination of behav-ioral cloning and generative adversarial imitation learning.Our algorithm combines the advantages from both learningframeworks and achieves a good balance between sampleefficiency and policy generalizability; (3) We run exten-sive experiments on the challenging Human 3.6M datasetto evaluate the performance of our proposed method, andshow that it outperforms all existing state-of-the-art base-line models by significant margins.

2. Related WorkHuman Motion Prediction: Most of the previous workon video prediction predict future video sequences by re-constructing frames at the pixel level [25, 44], and predictdense trajectories [46], semantic labels [24, 45], or activ-ity labels [3, 22, 38, 47] in the future. However, humanmotion dynamics is better captured by detailed joint loca-tions (i.e., pose) [1, 27, 32], and is often modeled by ei-ther state transition models [48, 49] or recurrent neural net-works [10, 19, 26].

Recently, several works focused on forecasting human

X1 X2 X3 X5X4 X7X6 X9X8 X11X10

X1 X2 X3 X5X4 X7X6 X9X8 X11X10

X1 X2 X3 X5X4 X7X6 X9X8 X11X10

X1 X2 X3 X5X4 X7X6 X9X8 X11X10

t = 3<latexit sha1_base64="g/3iJhDZ2juaBYkMSYtZwaSYv6s=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSFfQiFL14rGBsoQ1ls920SzebsDsRSuhv8OJBEa/+IG/+G7dtDtr6wsLDOzPszBumUhh03W+ntLK6tr5R3qxsbe/s7lX3Dx5NkmnGfZbIRLdDargUivsoUPJ2qjmNQ8lb4eh2Wm89cW1Eoh5wnPIgpgMlIsEoWstHck3Oe9WaW3dnIsvgFVCDQs1e9avbT1gWc4VMUmM6nptikFONgkk+qXQzw1PKRnTAOxYVjbkJ8tmyE3JinT6JEm2fQjJzf0/kNDZmHIe2M6Y4NIu1qflfrZNhdBXkQqUZcsXmH0WZJJiQ6eWkLzRnKMcWKNPC7krYkGrK0OZTsSF4iycvw+NZ3bN8f1Fr3BRxlOEIjuEUPLiEBtxBE3xgIOAZXuHNUc6L8+58zFtLTjFzCH/kfP4AgW+N0A==</latexit><latexit sha1_base64="g/3iJhDZ2juaBYkMSYtZwaSYv6s=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSFfQiFL14rGBsoQ1ls920SzebsDsRSuhv8OJBEa/+IG/+G7dtDtr6wsLDOzPszBumUhh03W+ntLK6tr5R3qxsbe/s7lX3Dx5NkmnGfZbIRLdDargUivsoUPJ2qjmNQ8lb4eh2Wm89cW1Eoh5wnPIgpgMlIsEoWstHck3Oe9WaW3dnIsvgFVCDQs1e9avbT1gWc4VMUmM6nptikFONgkk+qXQzw1PKRnTAOxYVjbkJ8tmyE3JinT6JEm2fQjJzf0/kNDZmHIe2M6Y4NIu1qflfrZNhdBXkQqUZcsXmH0WZJJiQ6eWkLzRnKMcWKNPC7krYkGrK0OZTsSF4iycvw+NZ3bN8f1Fr3BRxlOEIjuEUPLiEBtxBE3xgIOAZXuHNUc6L8+58zFtLTjFzCH/kfP4AgW+N0A==</latexit><latexit sha1_base64="g/3iJhDZ2juaBYkMSYtZwaSYv6s=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSFfQiFL14rGBsoQ1ls920SzebsDsRSuhv8OJBEa/+IG/+G7dtDtr6wsLDOzPszBumUhh03W+ntLK6tr5R3qxsbe/s7lX3Dx5NkmnGfZbIRLdDargUivsoUPJ2qjmNQ8lb4eh2Wm89cW1Eoh5wnPIgpgMlIsEoWstHck3Oe9WaW3dnIsvgFVCDQs1e9avbT1gWc4VMUmM6nptikFONgkk+qXQzw1PKRnTAOxYVjbkJ8tmyE3JinT6JEm2fQjJzf0/kNDZmHIe2M6Y4NIu1qflfrZNhdBXkQqUZcsXmH0WZJJiQ6eWkLzRnKMcWKNPC7krYkGrK0OZTsSF4iycvw+NZ3bN8f1Fr3BRxlOEIjuEUPLiEBtxBE3xgIOAZXuHNUc6L8+58zFtLTjFzCH/kfP4AgW+N0A==</latexit><latexit sha1_base64="g/3iJhDZ2juaBYkMSYtZwaSYv6s=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSFfQiFL14rGBsoQ1ls920SzebsDsRSuhv8OJBEa/+IG/+G7dtDtr6wsLDOzPszBumUhh03W+ntLK6tr5R3qxsbe/s7lX3Dx5NkmnGfZbIRLdDargUivsoUPJ2qjmNQ8lb4eh2Wm89cW1Eoh5wnPIgpgMlIsEoWstHck3Oe9WaW3dnIsvgFVCDQs1e9avbT1gWc4VMUmM6nptikFONgkk+qXQzw1PKRnTAOxYVjbkJ8tmyE3JinT6JEm2fQjJzf0/kNDZmHIe2M6Y4NIu1qflfrZNhdBXkQqUZcsXmH0WZJJiQ6eWkLzRnKMcWKNPC7krYkGrK0OZTsSF4iycvw+NZ3bN8f1Fr3BRxlOEIjuEUPLiEBtxBE3xgIOAZXuHNUc6L8+58zFtLTjFzCH/kfP4AgW+N0A==</latexit>

s1<latexit sha1_base64="b2izBJlX5lULue9NWFfFt+vHo3c=">AAAB6nicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfQkBS8eK9oPaEPZbDft0s0m7E6EEvoTvHhQxKu/yJv/xm2ag7a+sPDwzgw78waJFAZd99spra1vbG6Vtys7u3v7B9XDo7aJU814i8Uy1t2AGi6F4i0UKHk30ZxGgeSdYHI7r3eeuDYiVo84Tbgf0ZESoWAUrfVgBt6gWnPrbi6yCl4BNSjUHFS/+sOYpRFXyCQ1pue5CfoZ1SiY5LNKPzU8oWxCR7xnUdGIGz/LV52RM+sMSRhr+xSS3P09kdHImGkU2M6I4tgs1+bmf7VeiuG1nwmVpMgVW3wUppJgTOZ3k6HQnKGcWqBMC7srYWOqKUObTsWG4C2fvArti7pn+f6y1rgp4ijDCZzCOXhwBQ24gya0gMEInuEV3hzpvDjvzseiteQUM8fwR87nDwLgjZc=</latexit><latexit sha1_base64="b2izBJlX5lULue9NWFfFt+vHo3c=">AAAB6nicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfQkBS8eK9oPaEPZbDft0s0m7E6EEvoTvHhQxKu/yJv/xm2ag7a+sPDwzgw78waJFAZd99spra1vbG6Vtys7u3v7B9XDo7aJU814i8Uy1t2AGi6F4i0UKHk30ZxGgeSdYHI7r3eeuDYiVo84Tbgf0ZESoWAUrfVgBt6gWnPrbi6yCl4BNSjUHFS/+sOYpRFXyCQ1pue5CfoZ1SiY5LNKPzU8oWxCR7xnUdGIGz/LV52RM+sMSRhr+xSS3P09kdHImGkU2M6I4tgs1+bmf7VeiuG1nwmVpMgVW3wUppJgTOZ3k6HQnKGcWqBMC7srYWOqKUObTsWG4C2fvArti7pn+f6y1rgp4ijDCZzCOXhwBQ24gya0gMEInuEV3hzpvDjvzseiteQUM8fwR87nDwLgjZc=</latexit><latexit sha1_base64="b2izBJlX5lULue9NWFfFt+vHo3c=">AAAB6nicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfQkBS8eK9oPaEPZbDft0s0m7E6EEvoTvHhQxKu/yJv/xm2ag7a+sPDwzgw78waJFAZd99spra1vbG6Vtys7u3v7B9XDo7aJU814i8Uy1t2AGi6F4i0UKHk30ZxGgeSdYHI7r3eeuDYiVo84Tbgf0ZESoWAUrfVgBt6gWnPrbi6yCl4BNSjUHFS/+sOYpRFXyCQ1pue5CfoZ1SiY5LNKPzU8oWxCR7xnUdGIGz/LV52RM+sMSRhr+xSS3P09kdHImGkU2M6I4tgs1+bmf7VeiuG1nwmVpMgVW3wUppJgTOZ3k6HQnKGcWqBMC7srYWOqKUObTsWG4C2fvArti7pn+f6y1rgp4ijDCZzCOXhwBQ24gya0gMEInuEV3hzpvDjvzseiteQUM8fwR87nDwLgjZc=</latexit><latexit sha1_base64="b2izBJlX5lULue9NWFfFt+vHo3c=">AAAB6nicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfQkBS8eK9oPaEPZbDft0s0m7E6EEvoTvHhQxKu/yJv/xm2ag7a+sPDwzgw78waJFAZd99spra1vbG6Vtys7u3v7B9XDo7aJU814i8Uy1t2AGi6F4i0UKHk30ZxGgeSdYHI7r3eeuDYiVo84Tbgf0ZESoWAUrfVgBt6gWnPrbi6yCl4BNSjUHFS/+sOYpRFXyCQ1pue5CfoZ1SiY5LNKPzU8oWxCR7xnUdGIGz/LV52RM+sMSRhr+xSS3P09kdHImGkU2M6I4tgs1+bmf7VeiuG1nwmVpMgVW3wUppJgTOZ3k6HQnKGcWqBMC7srYWOqKUObTsWG4C2fvArti7pn+f6y1rgp4ijDCZzCOXhwBQ24gya0gMEInuEV3hzpvDjvzseiteQUM8fwR87nDwLgjZc=</latexit>

a1 = ⇡✓(s1)<latexit sha1_base64="OL9c+85f9QH5SBKQZ89ULcQIsqE=">AAACAHicbZDLSsNAFIYn9VbrrerChZvBItRNSUTQjVJw47KCvUATwmQ6aYdOJmHmRCghG1/FjQtF3PoY7nwbp20W2vrDwMd/zuHM+YNEcA22/W2VVlbX1jfKm5Wt7Z3dver+QUfHqaKsTWMRq15ANBNcsjZwEKyXKEaiQLBuML6d1ruPTGkeyweYJMyLyFDykFMCxvKrR8R38DV2E+5nLowYkLyufefMr9bshj0TXgangBoq1PKrX+4gpmnEJFBBtO47dgJeRhRwKlhecVPNEkLHZMj6BiWJmPay2QE5PjXOAIexMk8Cnrm/JzISaT2JAtMZERjpxdrU/K/WTyG88jIukxSYpPNFYSowxHiaBh5wxSiIiQFCFTd/xXREFKFgMquYEJzFk5ehc95wDN9f1Jo3RRxldIxOUB056BI10R1qoTaiKEfP6BW9WU/Wi/VufcxbS1Yxc4j+yPr8AeiWlU0=</latexit><latexit sha1_base64="OL9c+85f9QH5SBKQZ89ULcQIsqE=">AAACAHicbZDLSsNAFIYn9VbrrerChZvBItRNSUTQjVJw47KCvUATwmQ6aYdOJmHmRCghG1/FjQtF3PoY7nwbp20W2vrDwMd/zuHM+YNEcA22/W2VVlbX1jfKm5Wt7Z3dver+QUfHqaKsTWMRq15ANBNcsjZwEKyXKEaiQLBuML6d1ruPTGkeyweYJMyLyFDykFMCxvKrR8R38DV2E+5nLowYkLyufefMr9bshj0TXgangBoq1PKrX+4gpmnEJFBBtO47dgJeRhRwKlhecVPNEkLHZMj6BiWJmPay2QE5PjXOAIexMk8Cnrm/JzISaT2JAtMZERjpxdrU/K/WTyG88jIukxSYpPNFYSowxHiaBh5wxSiIiQFCFTd/xXREFKFgMquYEJzFk5ehc95wDN9f1Jo3RRxldIxOUB056BI10R1qoTaiKEfP6BW9WU/Wi/VufcxbS1Yxc4j+yPr8AeiWlU0=</latexit><latexit sha1_base64="OL9c+85f9QH5SBKQZ89ULcQIsqE=">AAACAHicbZDLSsNAFIYn9VbrrerChZvBItRNSUTQjVJw47KCvUATwmQ6aYdOJmHmRCghG1/FjQtF3PoY7nwbp20W2vrDwMd/zuHM+YNEcA22/W2VVlbX1jfKm5Wt7Z3dver+QUfHqaKsTWMRq15ANBNcsjZwEKyXKEaiQLBuML6d1ruPTGkeyweYJMyLyFDykFMCxvKrR8R38DV2E+5nLowYkLyufefMr9bshj0TXgangBoq1PKrX+4gpmnEJFBBtO47dgJeRhRwKlhecVPNEkLHZMj6BiWJmPay2QE5PjXOAIexMk8Cnrm/JzISaT2JAtMZERjpxdrU/K/WTyG88jIukxSYpPNFYSowxHiaBh5wxSiIiQFCFTd/xXREFKFgMquYEJzFk5ehc95wDN9f1Jo3RRxldIxOUB056BI10R1qoTaiKEfP6BW9WU/Wi/VufcxbS1Yxc4j+yPr8AeiWlU0=</latexit><latexit sha1_base64="OL9c+85f9QH5SBKQZ89ULcQIsqE=">AAACAHicbZDLSsNAFIYn9VbrrerChZvBItRNSUTQjVJw47KCvUATwmQ6aYdOJmHmRCghG1/FjQtF3PoY7nwbp20W2vrDwMd/zuHM+YNEcA22/W2VVlbX1jfKm5Wt7Z3dver+QUfHqaKsTWMRq15ANBNcsjZwEKyXKEaiQLBuML6d1ruPTGkeyweYJMyLyFDykFMCxvKrR8R38DV2E+5nLowYkLyufefMr9bshj0TXgangBoq1PKrX+4gpmnEJFBBtO47dgJeRhRwKlhecVPNEkLHZMj6BiWJmPay2QE5PjXOAIexMk8Cnrm/JzISaT2JAtMZERjpxdrU/K/WTyG88jIukxSYpPNFYSowxHiaBh5wxSiIiQFCFTd/xXREFKFgMquYEJzFk5ehc95wDN9f1Jo3RRxldIxOUB056BI10R1qoTaiKEfP6BW9WU/Wi/VufcxbS1Yxc4j+yPr8AeiWlU0=</latexit>

a2 = ⇡✓(s2)<latexit sha1_base64="LGWsyXmbZ10TyTinYIjUNFo9Tj0=">AAACAHicbZDLSsNAFIYnXmu9RV24cDNYhLopSREURCi4cVnBXqAJYTKdtEMnkzBzIpTQja/ixoUibn0Md76N0zYLbf1h4OM/53Dm/GEquAbH+bZWVtfWNzZLW+Xtnd29ffvgsK2TTFHWoolIVDckmgkuWQs4CNZNFSNxKFgnHN1O651HpjRP5AOMU+bHZCB5xCkBYwX2MQnq+AZ7KQ9yD4YMyKSqg/p5YFecmjMTXga3gAoq1AzsL6+f0CxmEqggWvdcJwU/Jwo4FWxS9jLNUkJHZMB6BiWJmfbz2QETfGacPo4SZZ4EPHN/T+Qk1noch6YzJjDUi7Wp+V+tl0F05edcphkwSeeLokxgSPA0DdznilEQYwOEKm7+iumQKELBZFY2IbiLJy9Du15zDd9fVBrXRRwldIJOURW56BI10B1qohaiaIKe0St6s56sF+vd+pi3rljFzBH6I+vzB+rMlUw=</latexit><latexit sha1_base64="LGWsyXmbZ10TyTinYIjUNFo9Tj0=">AAACAHicbZDLSsNAFIYnXmu9RV24cDNYhLopSREURCi4cVnBXqAJYTKdtEMnkzBzIpTQja/ixoUibn0Md76N0zYLbf1h4OM/53Dm/GEquAbH+bZWVtfWNzZLW+Xtnd29ffvgsK2TTFHWoolIVDckmgkuWQs4CNZNFSNxKFgnHN1O651HpjRP5AOMU+bHZCB5xCkBYwX2MQnq+AZ7KQ9yD4YMyKSqg/p5YFecmjMTXga3gAoq1AzsL6+f0CxmEqggWvdcJwU/Jwo4FWxS9jLNUkJHZMB6BiWJmfbz2QETfGacPo4SZZ4EPHN/T+Qk1noch6YzJjDUi7Wp+V+tl0F05edcphkwSeeLokxgSPA0DdznilEQYwOEKm7+iumQKELBZFY2IbiLJy9Du15zDd9fVBrXRRwldIJOURW56BI10B1qohaiaIKe0St6s56sF+vd+pi3rljFzBH6I+vzB+rMlUw=</latexit><latexit sha1_base64="LGWsyXmbZ10TyTinYIjUNFo9Tj0=">AAACAHicbZDLSsNAFIYnXmu9RV24cDNYhLopSREURCi4cVnBXqAJYTKdtEMnkzBzIpTQja/ixoUibn0Md76N0zYLbf1h4OM/53Dm/GEquAbH+bZWVtfWNzZLW+Xtnd29ffvgsK2TTFHWoolIVDckmgkuWQs4CNZNFSNxKFgnHN1O651HpjRP5AOMU+bHZCB5xCkBYwX2MQnq+AZ7KQ9yD4YMyKSqg/p5YFecmjMTXga3gAoq1AzsL6+f0CxmEqggWvdcJwU/Jwo4FWxS9jLNUkJHZMB6BiWJmfbz2QETfGacPo4SZZ4EPHN/T+Qk1noch6YzJjDUi7Wp+V+tl0F05edcphkwSeeLokxgSPA0DdznilEQYwOEKm7+iumQKELBZFY2IbiLJy9Du15zDd9fVBrXRRwldIJOURW56BI10B1qohaiaIKe0St6s56sF+vd+pi3rljFzBH6I+vzB+rMlUw=</latexit><latexit sha1_base64="LGWsyXmbZ10TyTinYIjUNFo9Tj0=">AAACAHicbZDLSsNAFIYnXmu9RV24cDNYhLopSREURCi4cVnBXqAJYTKdtEMnkzBzIpTQja/ixoUibn0Md76N0zYLbf1h4OM/53Dm/GEquAbH+bZWVtfWNzZLW+Xtnd29ffvgsK2TTFHWoolIVDckmgkuWQs4CNZNFSNxKFgnHN1O651HpjRP5AOMU+bHZCB5xCkBYwX2MQnq+AZ7KQ9yD4YMyKSqg/p5YFecmjMTXga3gAoq1AzsL6+f0CxmEqggWvdcJwU/Jwo4FWxS9jLNUkJHZMB6BiWJmfbz2QETfGacPo4SZZ4EPHN/T+Qk1noch6YzJjDUi7Wp+V+tl0F05edcphkwSeeLokxgSPA0DdznilEQYwOEKm7+iumQKELBZFY2IbiLJy9Du15zDd9fVBrXRRwldIJOURW56BI10B1qohaiaIKe0St6s56sF+vd+pi3rljFzBH6I+vzB+rMlUw=</latexit>

a3 = ⇡✓(s3)<latexit sha1_base64="qRvkOSw3G3M3Qoa1ydridOzWPTc=">AAACAHicbZDLSsNAFIYn9VbrLerChZvBItRNSaygIELBjcsK9gJNCJPppB06uTBzIpSQja/ixoUibn0Md76N0zYLbf1h4OM/53Dm/H4iuALL+jZKK6tr6xvlzcrW9s7unrl/0FFxKilr01jEsucTxQSPWBs4CNZLJCOhL1jXH99O691HJhWPoweYJMwNyTDiAacEtOWZR8Rr4BvsJNzLHBgxIHlNeY0zz6xadWsmvAx2AVVUqOWZX84gpmnIIqCCKNW3rQTcjEjgVLC84qSKJYSOyZD1NUYkZMrNZgfk+FQ7AxzEUr8I8Mz9PZGRUKlJ6OvOkMBILdam5n+1fgrBlZvxKEmBRXS+KEgFhhhP08ADLhkFMdFAqOT6r5iOiCQUdGYVHYK9ePIydM7rtub7i2rzuoijjI7RCaohG12iJrpDLdRGFOXoGb2iN+PJeDHejY95a8koZg7RHxmfP+3plU4=</latexit><latexit sha1_base64="qRvkOSw3G3M3Qoa1ydridOzWPTc=">AAACAHicbZDLSsNAFIYn9VbrLerChZvBItRNSaygIELBjcsK9gJNCJPppB06uTBzIpSQja/ixoUibn0Md76N0zYLbf1h4OM/53Dm/H4iuALL+jZKK6tr6xvlzcrW9s7unrl/0FFxKilr01jEsucTxQSPWBs4CNZLJCOhL1jXH99O691HJhWPoweYJMwNyTDiAacEtOWZR8Rr4BvsJNzLHBgxIHlNeY0zz6xadWsmvAx2AVVUqOWZX84gpmnIIqCCKNW3rQTcjEjgVLC84qSKJYSOyZD1NUYkZMrNZgfk+FQ7AxzEUr8I8Mz9PZGRUKlJ6OvOkMBILdam5n+1fgrBlZvxKEmBRXS+KEgFhhhP08ADLhkFMdFAqOT6r5iOiCQUdGYVHYK9ePIydM7rtub7i2rzuoijjI7RCaohG12iJrpDLdRGFOXoGb2iN+PJeDHejY95a8koZg7RHxmfP+3plU4=</latexit><latexit sha1_base64="qRvkOSw3G3M3Qoa1ydridOzWPTc=">AAACAHicbZDLSsNAFIYn9VbrLerChZvBItRNSaygIELBjcsK9gJNCJPppB06uTBzIpSQja/ixoUibn0Md76N0zYLbf1h4OM/53Dm/H4iuALL+jZKK6tr6xvlzcrW9s7unrl/0FFxKilr01jEsucTxQSPWBs4CNZLJCOhL1jXH99O691HJhWPoweYJMwNyTDiAacEtOWZR8Rr4BvsJNzLHBgxIHlNeY0zz6xadWsmvAx2AVVUqOWZX84gpmnIIqCCKNW3rQTcjEjgVLC84qSKJYSOyZD1NUYkZMrNZgfk+FQ7AxzEUr8I8Mz9PZGRUKlJ6OvOkMBILdam5n+1fgrBlZvxKEmBRXS+KEgFhhhP08ADLhkFMdFAqOT6r5iOiCQUdGYVHYK9ePIydM7rtub7i2rzuoijjI7RCaohG12iJrpDLdRGFOXoGb2iN+PJeDHejY95a8koZg7RHxmfP+3plU4=</latexit><latexit sha1_base64="qRvkOSw3G3M3Qoa1ydridOzWPTc=">AAACAHicbZDLSsNAFIYn9VbrLerChZvBItRNSaygIELBjcsK9gJNCJPppB06uTBzIpSQja/ixoUibn0Md76N0zYLbf1h4OM/53Dm/H4iuALL+jZKK6tr6xvlzcrW9s7unrl/0FFxKilr01jEsucTxQSPWBs4CNZLJCOhL1jXH99O691HJhWPoweYJMwNyTDiAacEtOWZR8Rr4BvsJNzLHBgxIHlNeY0zz6xadWsmvAx2AVVUqOWZX84gpmnIIqCCKNW3rQTcjEjgVLC84qSKJYSOyZD1NUYkZMrNZgfk+FQ7AxzEUr8I8Mz9PZGRUKlJ6OvOkMBILdam5n+1fgrBlZvxKEmBRXS+KEgFhhhP08ADLhkFMdFAqOT6r5iOiCQUdGYVHYK9ePIydM7rtub7i2rzuoijjI7RCaohG12iJrpDLdRGFOXoGb2iN+PJeDHejY95a8koZg7RHxmfP+3plU4=</latexit>

a4 = ⇡✓(s4)<latexit sha1_base64="rreV7yEbJsbn+ys6jIWjuAU4VL0=">AAACAHicbZDLSsNAFIYnXmu9RV24cDNYhLopiRQURCi4cVnBXqAJYTKdtEMnkzBzIpTQja/ixoUibn0Md76N0zYLbf1h4OM/53Dm/GEquAbH+bZWVtfWNzZLW+Xtnd29ffvgsK2TTFHWoolIVDckmgkuWQs4CNZNFSNxKFgnHN1O651HpjRP5AOMU+bHZCB5xCkBYwX2MQnq+AZ7KQ9yD4YMyKSqg/p5YFecmjMTXga3gAoq1AzsL6+f0CxmEqggWvdcJwU/Jwo4FWxS9jLNUkJHZMB6BiWJmfbz2QETfGacPo4SZZ4EPHN/T+Qk1noch6YzJjDUi7Wp+V+tl0F05edcphkwSeeLokxgSPA0DdznilEQYwOEKm7+iumQKELBZFY2IbiLJy9D+6LmGr6vVxrXRRwldIJOURW56BI10B1qohaiaIKe0St6s56sF+vd+pi3rljFzBH6I+vzB/EGlVA=</latexit><latexit sha1_base64="rreV7yEbJsbn+ys6jIWjuAU4VL0=">AAACAHicbZDLSsNAFIYnXmu9RV24cDNYhLopiRQURCi4cVnBXqAJYTKdtEMnkzBzIpTQja/ixoUibn0Md76N0zYLbf1h4OM/53Dm/GEquAbH+bZWVtfWNzZLW+Xtnd29ffvgsK2TTFHWoolIVDckmgkuWQs4CNZNFSNxKFgnHN1O651HpjRP5AOMU+bHZCB5xCkBYwX2MQnq+AZ7KQ9yD4YMyKSqg/p5YFecmjMTXga3gAoq1AzsL6+f0CxmEqggWvdcJwU/Jwo4FWxS9jLNUkJHZMB6BiWJmfbz2QETfGacPo4SZZ4EPHN/T+Qk1noch6YzJjDUi7Wp+V+tl0F05edcphkwSeeLokxgSPA0DdznilEQYwOEKm7+iumQKELBZFY2IbiLJy9D+6LmGr6vVxrXRRwldIJOURW56BI10B1qohaiaIKe0St6s56sF+vd+pi3rljFzBH6I+vzB/EGlVA=</latexit><latexit sha1_base64="rreV7yEbJsbn+ys6jIWjuAU4VL0=">AAACAHicbZDLSsNAFIYnXmu9RV24cDNYhLopiRQURCi4cVnBXqAJYTKdtEMnkzBzIpTQja/ixoUibn0Md76N0zYLbf1h4OM/53Dm/GEquAbH+bZWVtfWNzZLW+Xtnd29ffvgsK2TTFHWoolIVDckmgkuWQs4CNZNFSNxKFgnHN1O651HpjRP5AOMU+bHZCB5xCkBYwX2MQnq+AZ7KQ9yD4YMyKSqg/p5YFecmjMTXga3gAoq1AzsL6+f0CxmEqggWvdcJwU/Jwo4FWxS9jLNUkJHZMB6BiWJmfbz2QETfGacPo4SZZ4EPHN/T+Qk1noch6YzJjDUi7Wp+V+tl0F05edcphkwSeeLokxgSPA0DdznilEQYwOEKm7+iumQKELBZFY2IbiLJy9D+6LmGr6vVxrXRRwldIJOURW56BI10B1qohaiaIKe0St6s56sF+vd+pi3rljFzBH6I+vzB/EGlVA=</latexit><latexit sha1_base64="rreV7yEbJsbn+ys6jIWjuAU4VL0=">AAACAHicbZDLSsNAFIYnXmu9RV24cDNYhLopiRQURCi4cVnBXqAJYTKdtEMnkzBzIpTQja/ixoUibn0Md76N0zYLbf1h4OM/53Dm/GEquAbH+bZWVtfWNzZLW+Xtnd29ffvgsK2TTFHWoolIVDckmgkuWQs4CNZNFSNxKFgnHN1O651HpjRP5AOMU+bHZCB5xCkBYwX2MQnq+AZ7KQ9yD4YMyKSqg/p5YFecmjMTXga3gAoq1AzsL6+f0CxmEqggWvdcJwU/Jwo4FWxS9jLNUkJHZMB6BiWJmfbz2QETfGacPo4SZZ4EPHN/T+Qk1noch6YzJjDUi7Wp+V+tl0F05edcphkwSeeLokxgSPA0DdznilEQYwOEKm7+iumQKELBZFY2IbiLJy9D+6LmGr6vVxrXRRwldIJOURW56BI10B1qohaiaIKe0St6s56sF+vd+pi3rljFzBH6I+vzB/EGlVA=</latexit>

⇡✓<latexit sha1_base64="BwMQP1r8UE9VGk7lQ2fjI9Bl5KQ=">AAAB83icbZBNSwMxEIZn/az1q+rRS7AInsquCPVY8OKxgv2A7lKyadqGZrMhmRXK0r/hxYMiXv0z3vw3pu0etPWFwMM7M8zkjbUUFn3/29vY3Nre2S3tlfcPDo+OKyenbZtmhvEWS2VqujG1XArFWyhQ8q42nCax5J14cjevd564sSJVjzjVPEroSImhYBSdFYZa9PMQxxzprF+p+jV/IbIOQQFVKNTsV77CQcqyhCtkklrbC3yNUU4NCib5rBxmlmvKJnTEew4VTbiN8sXNM3LpnAEZpsY9hWTh/p7IaWLtNIldZ0JxbFdrc/O/Wi/D4W2UC6Uz5IotFw0zSTAl8wDIQBjOUE4dUGaEu5WwMTWUoYup7EIIVr+8Du3rWuD44abaqBdxlOAcLuAKAqhDA+6hCS1goOEZXuHNy7wX7937WLZueMXMGfyR9/kDb6OR5Q==</latexit><latexit sha1_base64="BwMQP1r8UE9VGk7lQ2fjI9Bl5KQ=">AAAB83icbZBNSwMxEIZn/az1q+rRS7AInsquCPVY8OKxgv2A7lKyadqGZrMhmRXK0r/hxYMiXv0z3vw3pu0etPWFwMM7M8zkjbUUFn3/29vY3Nre2S3tlfcPDo+OKyenbZtmhvEWS2VqujG1XArFWyhQ8q42nCax5J14cjevd564sSJVjzjVPEroSImhYBSdFYZa9PMQxxzprF+p+jV/IbIOQQFVKNTsV77CQcqyhCtkklrbC3yNUU4NCib5rBxmlmvKJnTEew4VTbiN8sXNM3LpnAEZpsY9hWTh/p7IaWLtNIldZ0JxbFdrc/O/Wi/D4W2UC6Uz5IotFw0zSTAl8wDIQBjOUE4dUGaEu5WwMTWUoYup7EIIVr+8Du3rWuD44abaqBdxlOAcLuAKAqhDA+6hCS1goOEZXuHNy7wX7937WLZueMXMGfyR9/kDb6OR5Q==</latexit><latexit sha1_base64="BwMQP1r8UE9VGk7lQ2fjI9Bl5KQ=">AAAB83icbZBNSwMxEIZn/az1q+rRS7AInsquCPVY8OKxgv2A7lKyadqGZrMhmRXK0r/hxYMiXv0z3vw3pu0etPWFwMM7M8zkjbUUFn3/29vY3Nre2S3tlfcPDo+OKyenbZtmhvEWS2VqujG1XArFWyhQ8q42nCax5J14cjevd564sSJVjzjVPEroSImhYBSdFYZa9PMQxxzprF+p+jV/IbIOQQFVKNTsV77CQcqyhCtkklrbC3yNUU4NCib5rBxmlmvKJnTEew4VTbiN8sXNM3LpnAEZpsY9hWTh/p7IaWLtNIldZ0JxbFdrc/O/Wi/D4W2UC6Uz5IotFw0zSTAl8wDIQBjOUE4dUGaEu5WwMTWUoYup7EIIVr+8Du3rWuD44abaqBdxlOAcLuAKAqhDA+6hCS1goOEZXuHNy7wX7937WLZueMXMGfyR9/kDb6OR5Q==</latexit><latexit sha1_base64="BwMQP1r8UE9VGk7lQ2fjI9Bl5KQ=">AAAB83icbZBNSwMxEIZn/az1q+rRS7AInsquCPVY8OKxgv2A7lKyadqGZrMhmRXK0r/hxYMiXv0z3vw3pu0etPWFwMM7M8zkjbUUFn3/29vY3Nre2S3tlfcPDo+OKyenbZtmhvEWS2VqujG1XArFWyhQ8q42nCax5J14cjevd564sSJVjzjVPEroSImhYBSdFYZa9PMQxxzprF+p+jV/IbIOQQFVKNTsV77CQcqyhCtkklrbC3yNUU4NCib5rBxmlmvKJnTEew4VTbiN8sXNM3LpnAEZpsY9hWTh/p7IaWLtNIldZ0JxbFdrc/O/Wi/D4W2UC6Uz5IotFw0zSTAl8wDIQBjOUE4dUGaEu5WwMTWUoYup7EIIVr+8Du3rWuD44abaqBdxlOAcLuAKAqhDA+6hCS1goOEZXuHNy7wX7937WLZueMXMGfyR9/kDb6OR5Q==</latexit>

⇡✓<latexit sha1_base64="BwMQP1r8UE9VGk7lQ2fjI9Bl5KQ=">AAAB83icbZBNSwMxEIZn/az1q+rRS7AInsquCPVY8OKxgv2A7lKyadqGZrMhmRXK0r/hxYMiXv0z3vw3pu0etPWFwMM7M8zkjbUUFn3/29vY3Nre2S3tlfcPDo+OKyenbZtmhvEWS2VqujG1XArFWyhQ8q42nCax5J14cjevd564sSJVjzjVPEroSImhYBSdFYZa9PMQxxzprF+p+jV/IbIOQQFVKNTsV77CQcqyhCtkklrbC3yNUU4NCib5rBxmlmvKJnTEew4VTbiN8sXNM3LpnAEZpsY9hWTh/p7IaWLtNIldZ0JxbFdrc/O/Wi/D4W2UC6Uz5IotFw0zSTAl8wDIQBjOUE4dUGaEu5WwMTWUoYup7EIIVr+8Du3rWuD44abaqBdxlOAcLuAKAqhDA+6hCS1goOEZXuHNy7wX7937WLZueMXMGfyR9/kDb6OR5Q==</latexit><latexit sha1_base64="BwMQP1r8UE9VGk7lQ2fjI9Bl5KQ=">AAAB83icbZBNSwMxEIZn/az1q+rRS7AInsquCPVY8OKxgv2A7lKyadqGZrMhmRXK0r/hxYMiXv0z3vw3pu0etPWFwMM7M8zkjbUUFn3/29vY3Nre2S3tlfcPDo+OKyenbZtmhvEWS2VqujG1XArFWyhQ8q42nCax5J14cjevd564sSJVjzjVPEroSImhYBSdFYZa9PMQxxzprF+p+jV/IbIOQQFVKNTsV77CQcqyhCtkklrbC3yNUU4NCib5rBxmlmvKJnTEew4VTbiN8sXNM3LpnAEZpsY9hWTh/p7IaWLtNIldZ0JxbFdrc/O/Wi/D4W2UC6Uz5IotFw0zSTAl8wDIQBjOUE4dUGaEu5WwMTWUoYup7EIIVr+8Du3rWuD44abaqBdxlOAcLuAKAqhDA+6hCS1goOEZXuHNy7wX7937WLZueMXMGfyR9/kDb6OR5Q==</latexit><latexit sha1_base64="BwMQP1r8UE9VGk7lQ2fjI9Bl5KQ=">AAAB83icbZBNSwMxEIZn/az1q+rRS7AInsquCPVY8OKxgv2A7lKyadqGZrMhmRXK0r/hxYMiXv0z3vw3pu0etPWFwMM7M8zkjbUUFn3/29vY3Nre2S3tlfcPDo+OKyenbZtmhvEWS2VqujG1XArFWyhQ8q42nCax5J14cjevd564sSJVjzjVPEroSImhYBSdFYZa9PMQxxzprF+p+jV/IbIOQQFVKNTsV77CQcqyhCtkklrbC3yNUU4NCib5rBxmlmvKJnTEew4VTbiN8sXNM3LpnAEZpsY9hWTh/p7IaWLtNIldZ0JxbFdrc/O/Wi/D4W2UC6Uz5IotFw0zSTAl8wDIQBjOUE4dUGaEu5WwMTWUoYup7EIIVr+8Du3rWuD44abaqBdxlOAcLuAKAqhDA+6hCS1goOEZXuHNy7wX7937WLZueMXMGfyR9/kDb6OR5Q==</latexit><latexit sha1_base64="BwMQP1r8UE9VGk7lQ2fjI9Bl5KQ=">AAAB83icbZBNSwMxEIZn/az1q+rRS7AInsquCPVY8OKxgv2A7lKyadqGZrMhmRXK0r/hxYMiXv0z3vw3pu0etPWFwMM7M8zkjbUUFn3/29vY3Nre2S3tlfcPDo+OKyenbZtmhvEWS2VqujG1XArFWyhQ8q42nCax5J14cjevd564sSJVjzjVPEroSImhYBSdFYZa9PMQxxzprF+p+jV/IbIOQQFVKNTsV77CQcqyhCtkklrbC3yNUU4NCib5rBxmlmvKJnTEew4VTbiN8sXNM3LpnAEZpsY9hWTh/p7IaWLtNIldZ0JxbFdrc/O/Wi/D4W2UC6Uz5IotFw0zSTAl8wDIQBjOUE4dUGaEu5WwMTWUoYup7EIIVr+8Du3rWuD44abaqBdxlOAcLuAKAqhDA+6hCS1goOEZXuHNy7wX7937WLZueMXMGfyR9/kDb6OR5Q==</latexit>

⇡✓<latexit sha1_base64="BwMQP1r8UE9VGk7lQ2fjI9Bl5KQ=">AAAB83icbZBNSwMxEIZn/az1q+rRS7AInsquCPVY8OKxgv2A7lKyadqGZrMhmRXK0r/hxYMiXv0z3vw3pu0etPWFwMM7M8zkjbUUFn3/29vY3Nre2S3tlfcPDo+OKyenbZtmhvEWS2VqujG1XArFWyhQ8q42nCax5J14cjevd564sSJVjzjVPEroSImhYBSdFYZa9PMQxxzprF+p+jV/IbIOQQFVKNTsV77CQcqyhCtkklrbC3yNUU4NCib5rBxmlmvKJnTEew4VTbiN8sXNM3LpnAEZpsY9hWTh/p7IaWLtNIldZ0JxbFdrc/O/Wi/D4W2UC6Uz5IotFw0zSTAl8wDIQBjOUE4dUGaEu5WwMTWUoYup7EIIVr+8Du3rWuD44abaqBdxlOAcLuAKAqhDA+6hCS1goOEZXuHNy7wX7937WLZueMXMGfyR9/kDb6OR5Q==</latexit><latexit sha1_base64="BwMQP1r8UE9VGk7lQ2fjI9Bl5KQ=">AAAB83icbZBNSwMxEIZn/az1q+rRS7AInsquCPVY8OKxgv2A7lKyadqGZrMhmRXK0r/hxYMiXv0z3vw3pu0etPWFwMM7M8zkjbUUFn3/29vY3Nre2S3tlfcPDo+OKyenbZtmhvEWS2VqujG1XArFWyhQ8q42nCax5J14cjevd564sSJVjzjVPEroSImhYBSdFYZa9PMQxxzprF+p+jV/IbIOQQFVKNTsV77CQcqyhCtkklrbC3yNUU4NCib5rBxmlmvKJnTEew4VTbiN8sXNM3LpnAEZpsY9hWTh/p7IaWLtNIldZ0JxbFdrc/O/Wi/D4W2UC6Uz5IotFw0zSTAl8wDIQBjOUE4dUGaEu5WwMTWUoYup7EIIVr+8Du3rWuD44abaqBdxlOAcLuAKAqhDA+6hCS1goOEZXuHNy7wX7937WLZueMXMGfyR9/kDb6OR5Q==</latexit><latexit sha1_base64="BwMQP1r8UE9VGk7lQ2fjI9Bl5KQ=">AAAB83icbZBNSwMxEIZn/az1q+rRS7AInsquCPVY8OKxgv2A7lKyadqGZrMhmRXK0r/hxYMiXv0z3vw3pu0etPWFwMM7M8zkjbUUFn3/29vY3Nre2S3tlfcPDo+OKyenbZtmhvEWS2VqujG1XArFWyhQ8q42nCax5J14cjevd564sSJVjzjVPEroSImhYBSdFYZa9PMQxxzprF+p+jV/IbIOQQFVKNTsV77CQcqyhCtkklrbC3yNUU4NCib5rBxmlmvKJnTEew4VTbiN8sXNM3LpnAEZpsY9hWTh/p7IaWLtNIldZ0JxbFdrc/O/Wi/D4W2UC6Uz5IotFw0zSTAl8wDIQBjOUE4dUGaEu5WwMTWUoYup7EIIVr+8Du3rWuD44abaqBdxlOAcLuAKAqhDA+6hCS1goOEZXuHNy7wX7937WLZueMXMGfyR9/kDb6OR5Q==</latexit><latexit sha1_base64="BwMQP1r8UE9VGk7lQ2fjI9Bl5KQ=">AAAB83icbZBNSwMxEIZn/az1q+rRS7AInsquCPVY8OKxgv2A7lKyadqGZrMhmRXK0r/hxYMiXv0z3vw3pu0etPWFwMM7M8zkjbUUFn3/29vY3Nre2S3tlfcPDo+OKyenbZtmhvEWS2VqujG1XArFWyhQ8q42nCax5J14cjevd564sSJVjzjVPEroSImhYBSdFYZa9PMQxxzprF+p+jV/IbIOQQFVKNTsV77CQcqyhCtkklrbC3yNUU4NCib5rBxmlmvKJnTEew4VTbiN8sXNM3LpnAEZpsY9hWTh/p7IaWLtNIldZ0JxbFdrc/O/Wi/D4W2UC6Uz5IotFw0zSTAl8wDIQBjOUE4dUGaEu5WwMTWUoYup7EIIVr+8Du3rWuD44abaqBdxlOAcLuAKAqhDA+6hCS1goOEZXuHNy7wX7937WLZueMXMGfyR9/kDb6OR5Q==</latexit>

⇡✓<latexit sha1_base64="BwMQP1r8UE9VGk7lQ2fjI9Bl5KQ=">AAAB83icbZBNSwMxEIZn/az1q+rRS7AInsquCPVY8OKxgv2A7lKyadqGZrMhmRXK0r/hxYMiXv0z3vw3pu0etPWFwMM7M8zkjbUUFn3/29vY3Nre2S3tlfcPDo+OKyenbZtmhvEWS2VqujG1XArFWyhQ8q42nCax5J14cjevd564sSJVjzjVPEroSImhYBSdFYZa9PMQxxzprF+p+jV/IbIOQQFVKNTsV77CQcqyhCtkklrbC3yNUU4NCib5rBxmlmvKJnTEew4VTbiN8sXNM3LpnAEZpsY9hWTh/p7IaWLtNIldZ0JxbFdrc/O/Wi/D4W2UC6Uz5IotFw0zSTAl8wDIQBjOUE4dUGaEu5WwMTWUoYup7EIIVr+8Du3rWuD44abaqBdxlOAcLuAKAqhDA+6hCS1goOEZXuHNy7wX7937WLZueMXMGfyR9/kDb6OR5Q==</latexit><latexit sha1_base64="BwMQP1r8UE9VGk7lQ2fjI9Bl5KQ=">AAAB83icbZBNSwMxEIZn/az1q+rRS7AInsquCPVY8OKxgv2A7lKyadqGZrMhmRXK0r/hxYMiXv0z3vw3pu0etPWFwMM7M8zkjbUUFn3/29vY3Nre2S3tlfcPDo+OKyenbZtmhvEWS2VqujG1XArFWyhQ8q42nCax5J14cjevd564sSJVjzjVPEroSImhYBSdFYZa9PMQxxzprF+p+jV/IbIOQQFVKNTsV77CQcqyhCtkklrbC3yNUU4NCib5rBxmlmvKJnTEew4VTbiN8sXNM3LpnAEZpsY9hWTh/p7IaWLtNIldZ0JxbFdrc/O/Wi/D4W2UC6Uz5IotFw0zSTAl8wDIQBjOUE4dUGaEu5WwMTWUoYup7EIIVr+8Du3rWuD44abaqBdxlOAcLuAKAqhDA+6hCS1goOEZXuHNy7wX7937WLZueMXMGfyR9/kDb6OR5Q==</latexit><latexit sha1_base64="BwMQP1r8UE9VGk7lQ2fjI9Bl5KQ=">AAAB83icbZBNSwMxEIZn/az1q+rRS7AInsquCPVY8OKxgv2A7lKyadqGZrMhmRXK0r/hxYMiXv0z3vw3pu0etPWFwMM7M8zkjbUUFn3/29vY3Nre2S3tlfcPDo+OKyenbZtmhvEWS2VqujG1XArFWyhQ8q42nCax5J14cjevd564sSJVjzjVPEroSImhYBSdFYZa9PMQxxzprF+p+jV/IbIOQQFVKNTsV77CQcqyhCtkklrbC3yNUU4NCib5rBxmlmvKJnTEew4VTbiN8sXNM3LpnAEZpsY9hWTh/p7IaWLtNIldZ0JxbFdrc/O/Wi/D4W2UC6Uz5IotFw0zSTAl8wDIQBjOUE4dUGaEu5WwMTWUoYup7EIIVr+8Du3rWuD44abaqBdxlOAcLuAKAqhDA+6hCS1goOEZXuHNy7wX7937WLZueMXMGfyR9/kDb6OR5Q==</latexit><latexit sha1_base64="BwMQP1r8UE9VGk7lQ2fjI9Bl5KQ=">AAAB83icbZBNSwMxEIZn/az1q+rRS7AInsquCPVY8OKxgv2A7lKyadqGZrMhmRXK0r/hxYMiXv0z3vw3pu0etPWFwMM7M8zkjbUUFn3/29vY3Nre2S3tlfcPDo+OKyenbZtmhvEWS2VqujG1XArFWyhQ8q42nCax5J14cjevd564sSJVjzjVPEroSImhYBSdFYZa9PMQxxzprF+p+jV/IbIOQQFVKNTsV77CQcqyhCtkklrbC3yNUU4NCib5rBxmlmvKJnTEew4VTbiN8sXNM3LpnAEZpsY9hWTh/p7IaWLtNIldZ0JxbFdrc/O/Wi/D4W2UC6Uz5IotFw0zSTAl8wDIQBjOUE4dUGaEu5WwMTWUoYup7EIIVr+8Du3rWuD44abaqBdxlOAcLuAKAqhDA+6hCS1goOEZXuHNy7wX7937WLZueMXMGfyR9/kDb6OR5Q==</latexit>

s2<latexit sha1_base64="OI/WaDoyIo8lh9wHnB4dQAp0xbI=">AAAB6nicbZBNS8NAEIYn9avWr6pHL4tF8FSSItSTFLx4rGg/oA1ls920SzebsDsRSuhP8OJBEa/+Im/+G7dtDtr6wsLDOzPszBskUhh03W+nsLG5tb1T3C3t7R8cHpWPT9omTjXjLRbLWHcDargUirdQoOTdRHMaBZJ3gsntvN554tqIWD3iNOF+REdKhIJRtNaDGdQG5YpbdRci6+DlUIFczUH5qz+MWRpxhUxSY3qem6CfUY2CST4r9VPDE8omdMR7FhWNuPGzxaozcmGdIQljbZ9CsnB/T2Q0MmYaBbYzojg2q7W5+V+tl2J47WdCJSlyxZYfhakkGJP53WQoNGcopxYo08LuStiYasrQplOyIXirJ69Du1b1LN9fVRo3eRxFOINzuAQP6tCAO2hCCxiM4Ble4c2Rzovz7nwsWwtOPnMKf+R8/gAEZI2Y</latexit><latexit sha1_base64="OI/WaDoyIo8lh9wHnB4dQAp0xbI=">AAAB6nicbZBNS8NAEIYn9avWr6pHL4tF8FSSItSTFLx4rGg/oA1ls920SzebsDsRSuhP8OJBEa/+Im/+G7dtDtr6wsLDOzPszBskUhh03W+nsLG5tb1T3C3t7R8cHpWPT9omTjXjLRbLWHcDargUirdQoOTdRHMaBZJ3gsntvN554tqIWD3iNOF+REdKhIJRtNaDGdQG5YpbdRci6+DlUIFczUH5qz+MWRpxhUxSY3qem6CfUY2CST4r9VPDE8omdMR7FhWNuPGzxaozcmGdIQljbZ9CsnB/T2Q0MmYaBbYzojg2q7W5+V+tl2J47WdCJSlyxZYfhakkGJP53WQoNGcopxYo08LuStiYasrQplOyIXirJ69Du1b1LN9fVRo3eRxFOINzuAQP6tCAO2hCCxiM4Ble4c2Rzovz7nwsWwtOPnMKf+R8/gAEZI2Y</latexit><latexit sha1_base64="OI/WaDoyIo8lh9wHnB4dQAp0xbI=">AAAB6nicbZBNS8NAEIYn9avWr6pHL4tF8FSSItSTFLx4rGg/oA1ls920SzebsDsRSuhP8OJBEa/+Im/+G7dtDtr6wsLDOzPszBskUhh03W+nsLG5tb1T3C3t7R8cHpWPT9omTjXjLRbLWHcDargUirdQoOTdRHMaBZJ3gsntvN554tqIWD3iNOF+REdKhIJRtNaDGdQG5YpbdRci6+DlUIFczUH5qz+MWRpxhUxSY3qem6CfUY2CST4r9VPDE8omdMR7FhWNuPGzxaozcmGdIQljbZ9CsnB/T2Q0MmYaBbYzojg2q7W5+V+tl2J47WdCJSlyxZYfhakkGJP53WQoNGcopxYo08LuStiYasrQplOyIXirJ69Du1b1LN9fVRo3eRxFOINzuAQP6tCAO2hCCxiM4Ble4c2Rzovz7nwsWwtOPnMKf+R8/gAEZI2Y</latexit><latexit sha1_base64="OI/WaDoyIo8lh9wHnB4dQAp0xbI=">AAAB6nicbZBNS8NAEIYn9avWr6pHL4tF8FSSItSTFLx4rGg/oA1ls920SzebsDsRSuhP8OJBEa/+Im/+G7dtDtr6wsLDOzPszBskUhh03W+nsLG5tb1T3C3t7R8cHpWPT9omTjXjLRbLWHcDargUirdQoOTdRHMaBZJ3gsntvN554tqIWD3iNOF+REdKhIJRtNaDGdQG5YpbdRci6+DlUIFczUH5qz+MWRpxhUxSY3qem6CfUY2CST4r9VPDE8omdMR7FhWNuPGzxaozcmGdIQljbZ9CsnB/T2Q0MmYaBbYzojg2q7W5+V+tl2J47WdCJSlyxZYfhakkGJP53WQoNGcopxYo08LuStiYasrQplOyIXirJ69Du1b1LN9fVRo3eRxFOINzuAQP6tCAO2hCCxiM4Ble4c2Rzovz7nwsWwtOPnMKf+R8/gAEZI2Y</latexit>

s3<latexit sha1_base64="oob3E0vtozVXlIQwFvfGg118WwA=">AAAB6nicbZBNS8NAEIYn9avWr6pHL4tF8FQSLehJCl48VrQf0Iay2W7apZtN2J0IJfQnePGgiFd/kTf/jds2B219YeHhnRl25g0SKQy67rdTWFvf2Nwqbpd2dvf2D8qHRy0Tp5rxJotlrDsBNVwKxZsoUPJOojmNAsnbwfh2Vm8/cW1ErB5xknA/okMlQsEoWuvB9C/75Ypbdeciq+DlUIFcjX75qzeIWRpxhUxSY7qem6CfUY2CST4t9VLDE8rGdMi7FhWNuPGz+apTcmadAQljbZ9CMnd/T2Q0MmYSBbYzojgyy7WZ+V+tm2J47WdCJSlyxRYfhakkGJPZ3WQgNGcoJxYo08LuStiIasrQplOyIXjLJ69C66LqWb6vVeo3eRxFOIFTOAcPrqAOd9CAJjAYwjO8wpsjnRfn3flYtBacfOYY/sj5/AEF6I2Z</latexit><latexit sha1_base64="oob3E0vtozVXlIQwFvfGg118WwA=">AAAB6nicbZBNS8NAEIYn9avWr6pHL4tF8FQSLehJCl48VrQf0Iay2W7apZtN2J0IJfQnePGgiFd/kTf/jds2B219YeHhnRl25g0SKQy67rdTWFvf2Nwqbpd2dvf2D8qHRy0Tp5rxJotlrDsBNVwKxZsoUPJOojmNAsnbwfh2Vm8/cW1ErB5xknA/okMlQsEoWuvB9C/75Ypbdeciq+DlUIFcjX75qzeIWRpxhUxSY7qem6CfUY2CST4t9VLDE8rGdMi7FhWNuPGz+apTcmadAQljbZ9CMnd/T2Q0MmYSBbYzojgyy7WZ+V+tm2J47WdCJSlyxRYfhakkGJPZ3WQgNGcoJxYo08LuStiIasrQplOyIXjLJ69C66LqWb6vVeo3eRxFOIFTOAcPrqAOd9CAJjAYwjO8wpsjnRfn3flYtBacfOYY/sj5/AEF6I2Z</latexit><latexit sha1_base64="oob3E0vtozVXlIQwFvfGg118WwA=">AAAB6nicbZBNS8NAEIYn9avWr6pHL4tF8FQSLehJCl48VrQf0Iay2W7apZtN2J0IJfQnePGgiFd/kTf/jds2B219YeHhnRl25g0SKQy67rdTWFvf2Nwqbpd2dvf2D8qHRy0Tp5rxJotlrDsBNVwKxZsoUPJOojmNAsnbwfh2Vm8/cW1ErB5xknA/okMlQsEoWuvB9C/75Ypbdeciq+DlUIFcjX75qzeIWRpxhUxSY7qem6CfUY2CST4t9VLDE8rGdMi7FhWNuPGz+apTcmadAQljbZ9CMnd/T2Q0MmYSBbYzojgyy7WZ+V+tm2J47WdCJSlyxRYfhakkGJPZ3WQgNGcoJxYo08LuStiIasrQplOyIXjLJ69C66LqWb6vVeo3eRxFOIFTOAcPrqAOd9CAJjAYwjO8wpsjnRfn3flYtBacfOYY/sj5/AEF6I2Z</latexit><latexit sha1_base64="oob3E0vtozVXlIQwFvfGg118WwA=">AAAB6nicbZBNS8NAEIYn9avWr6pHL4tF8FQSLehJCl48VrQf0Iay2W7apZtN2J0IJfQnePGgiFd/kTf/jds2B219YeHhnRl25g0SKQy67rdTWFvf2Nwqbpd2dvf2D8qHRy0Tp5rxJotlrDsBNVwKxZsoUPJOojmNAsnbwfh2Vm8/cW1ErB5xknA/okMlQsEoWuvB9C/75Ypbdeciq+DlUIFcjX75qzeIWRpxhUxSY7qem6CfUY2CST4t9VLDE8rGdMi7FhWNuPGz+apTcmadAQljbZ9CMnd/T2Q0MmYSBbYzojgyy7WZ+V+tm2J47WdCJSlyxRYfhakkGJPZ3WQgNGcoJxYo08LuStiIasrQplOyIXjLJ69C66LqWb6vVeo3eRxFOIFTOAcPrqAOd9CAJjAYwjO8wpsjnRfn3flYtBacfOYY/sj5/AEF6I2Z</latexit>

s4<latexit sha1_base64="gk0QptPpOcMMdaWam61xgzCT0wo=">AAAB6nicbZBNS8NAEIYn9avWr6pHL4tF8FQSKdSTFLx4rGg/oA1ls920SzebsDsRSuhP8OJBEa/+Im/+G7dtDtr6wsLDOzPszBskUhh03W+nsLG5tb1T3C3t7R8cHpWPT9omTjXjLRbLWHcDargUirdQoOTdRHMaBZJ3gsntvN554tqIWD3iNOF+REdKhIJRtNaDGdQG5YpbdRci6+DlUIFczUH5qz+MWRpxhUxSY3qem6CfUY2CST4r9VPDE8omdMR7FhWNuPGzxaozcmGdIQljbZ9CsnB/T2Q0MmYaBbYzojg2q7W5+V+tl2J47WdCJSlyxZYfhakkGJP53WQoNGcopxYo08LuStiYasrQplOyIXirJ69D+6rqWb6vVRo3eRxFOINzuAQP6tCAO2hCCxiM4Ble4c2Rzovz7nwsWwtOPnMKf+R8/gAHbI2a</latexit><latexit sha1_base64="gk0QptPpOcMMdaWam61xgzCT0wo=">AAAB6nicbZBNS8NAEIYn9avWr6pHL4tF8FQSKdSTFLx4rGg/oA1ls920SzebsDsRSuhP8OJBEa/+Im/+G7dtDtr6wsLDOzPszBskUhh03W+nsLG5tb1T3C3t7R8cHpWPT9omTjXjLRbLWHcDargUirdQoOTdRHMaBZJ3gsntvN554tqIWD3iNOF+REdKhIJRtNaDGdQG5YpbdRci6+DlUIFczUH5qz+MWRpxhUxSY3qem6CfUY2CST4r9VPDE8omdMR7FhWNuPGzxaozcmGdIQljbZ9CsnB/T2Q0MmYaBbYzojg2q7W5+V+tl2J47WdCJSlyxZYfhakkGJP53WQoNGcopxYo08LuStiYasrQplOyIXirJ69D+6rqWb6vVRo3eRxFOINzuAQP6tCAO2hCCxiM4Ble4c2Rzovz7nwsWwtOPnMKf+R8/gAHbI2a</latexit><latexit sha1_base64="gk0QptPpOcMMdaWam61xgzCT0wo=">AAAB6nicbZBNS8NAEIYn9avWr6pHL4tF8FQSKdSTFLx4rGg/oA1ls920SzebsDsRSuhP8OJBEa/+Im/+G7dtDtr6wsLDOzPszBskUhh03W+nsLG5tb1T3C3t7R8cHpWPT9omTjXjLRbLWHcDargUirdQoOTdRHMaBZJ3gsntvN554tqIWD3iNOF+REdKhIJRtNaDGdQG5YpbdRci6+DlUIFczUH5qz+MWRpxhUxSY3qem6CfUY2CST4r9VPDE8omdMR7FhWNuPGzxaozcmGdIQljbZ9CsnB/T2Q0MmYaBbYzojg2q7W5+V+tl2J47WdCJSlyxZYfhakkGJP53WQoNGcopxYo08LuStiYasrQplOyIXirJ69D+6rqWb6vVRo3eRxFOINzuAQP6tCAO2hCCxiM4Ble4c2Rzovz7nwsWwtOPnMKf+R8/gAHbI2a</latexit><latexit sha1_base64="gk0QptPpOcMMdaWam61xgzCT0wo=">AAAB6nicbZBNS8NAEIYn9avWr6pHL4tF8FQSKdSTFLx4rGg/oA1ls920SzebsDsRSuhP8OJBEa/+Im/+G7dtDtr6wsLDOzPszBskUhh03W+nsLG5tb1T3C3t7R8cHpWPT9omTjXjLRbLWHcDargUirdQoOTdRHMaBZJ3gsntvN554tqIWD3iNOF+REdKhIJRtNaDGdQG5YpbdRci6+DlUIFczUH5qz+MWRpxhUxSY3qem6CfUY2CST4r9VPDE8omdMR7FhWNuPGzxaozcmGdIQljbZ9CsnB/T2Q0MmYaBbYzojg2q7W5+V+tl2J47WdCJSlyxZYfhakkGJP53WQoNGcopxYo08LuStiYasrQplOyIXirJ69D+6rqWb6vVRo3eRxFOINzuAQP6tCAO2hCCxiM4Ble4c2Rzovz7nwsWwtOPnMKf+R8/gAHbI2a</latexit>

Step 1:<latexit sha1_base64="uJKmkxctU4hehPr7LHuU2DHXlJQ=">AAAB7nicbZBNSwMxEIZn/az1q+rRS7AInspuL4peCl48VrQf0C4lm2bb0Gw2JLNCKf0RXjwo4tXf481/Y9ruQVtfCDy8M0Nm3khLYdH3v7219Y3Nre3CTnF3b//gsHR03LRpZhhvsFSmph1Ry6VQvIECJW9rw2kSSd6KRrezeuuJGytS9YhjzcOEDpSIBaPorNYDck2C616p7Ff8ucgqBDmUIVe9V/rq9lOWJVwhk9TaTuBrDCfUoGCST4vdzHJN2YgOeMehogm34WS+7pScO6dP4tS4p5DM3d8TE5pYO04i15lQHNrl2sz8r9bJML4KJ0LpDLlii4/iTBJMyex20heGM5RjB5QZ4XYlbEgNZegSKroQguWTV6FZrQSO76vl2k0eRwFO4QwuIIBLqMEd1KEBDEbwDK/w5mnvxXv3Phata14+cwJ/5H3+AFX5jt4=</latexit><latexit sha1_base64="uJKmkxctU4hehPr7LHuU2DHXlJQ=">AAAB7nicbZBNSwMxEIZn/az1q+rRS7AInspuL4peCl48VrQf0C4lm2bb0Gw2JLNCKf0RXjwo4tXf481/Y9ruQVtfCDy8M0Nm3khLYdH3v7219Y3Nre3CTnF3b//gsHR03LRpZhhvsFSmph1Ry6VQvIECJW9rw2kSSd6KRrezeuuJGytS9YhjzcOEDpSIBaPorNYDck2C616p7Ff8ucgqBDmUIVe9V/rq9lOWJVwhk9TaTuBrDCfUoGCST4vdzHJN2YgOeMehogm34WS+7pScO6dP4tS4p5DM3d8TE5pYO04i15lQHNrl2sz8r9bJML4KJ0LpDLlii4/iTBJMyex20heGM5RjB5QZ4XYlbEgNZegSKroQguWTV6FZrQSO76vl2k0eRwFO4QwuIIBLqMEd1KEBDEbwDK/w5mnvxXv3Phata14+cwJ/5H3+AFX5jt4=</latexit><latexit sha1_base64="uJKmkxctU4hehPr7LHuU2DHXlJQ=">AAAB7nicbZBNSwMxEIZn/az1q+rRS7AInspuL4peCl48VrQf0C4lm2bb0Gw2JLNCKf0RXjwo4tXf481/Y9ruQVtfCDy8M0Nm3khLYdH3v7219Y3Nre3CTnF3b//gsHR03LRpZhhvsFSmph1Ry6VQvIECJW9rw2kSSd6KRrezeuuJGytS9YhjzcOEDpSIBaPorNYDck2C616p7Ff8ucgqBDmUIVe9V/rq9lOWJVwhk9TaTuBrDCfUoGCST4vdzHJN2YgOeMehogm34WS+7pScO6dP4tS4p5DM3d8TE5pYO04i15lQHNrl2sz8r9bJML4KJ0LpDLlii4/iTBJMyex20heGM5RjB5QZ4XYlbEgNZegSKroQguWTV6FZrQSO76vl2k0eRwFO4QwuIIBLqMEd1KEBDEbwDK/w5mnvxXv3Phata14+cwJ/5H3+AFX5jt4=</latexit><latexit sha1_base64="uJKmkxctU4hehPr7LHuU2DHXlJQ=">AAAB7nicbZBNSwMxEIZn/az1q+rRS7AInspuL4peCl48VrQf0C4lm2bb0Gw2JLNCKf0RXjwo4tXf481/Y9ruQVtfCDy8M0Nm3khLYdH3v7219Y3Nre3CTnF3b//gsHR03LRpZhhvsFSmph1Ry6VQvIECJW9rw2kSSd6KRrezeuuJGytS9YhjzcOEDpSIBaPorNYDck2C616p7Ff8ucgqBDmUIVe9V/rq9lOWJVwhk9TaTuBrDCfUoGCST4vdzHJN2YgOeMehogm34WS+7pScO6dP4tS4p5DM3d8TE5pYO04i15lQHNrl2sz8r9bJML4KJ0LpDLlii4/iTBJMyex20heGM5RjB5QZ4XYlbEgNZegSKroQguWTV6FZrQSO76vl2k0eRwFO4QwuIIBLqMEd1KEBDEbwDK/w5mnvxXv3Phata14+cwJ/5H3+AFX5jt4=</latexit>

Step 2:<latexit sha1_base64="Z0HSVnJsDJydbuQ5ghdR62WLb0g=">AAAB7nicbZBNSwMxEIazftb6VfXoJVgET2W3F0UvBS8eK9oPaJeSTWfb0Gw2JLNCKf0RXjwo4tXf481/Y9ruQVtfCDy8M0Nm3khLYdH3v7219Y3Nre3CTnF3b//gsHR03LRpZjg0eCpT046YBSkUNFCghLY2wJJIQisa3c7qrScwVqTqEccawoQNlIgFZ+is1gOCptXrXqnsV/y56CoEOZRJrnqv9NXtpzxLQCGXzNpO4GsMJ8yg4BKmxW5mQTM+YgPoOFQsARtO5utO6blz+jROjXsK6dz9PTFhibXjJHKdCcOhXa7NzP9qnQzjq3AilM4QFF98FGeSYkpnt9O+MMBRjh0wboTblfIhM4yjS6joQgiWT16FZrUSOL6vlms3eRwFckrOyAUJyCWpkTtSJw3CyYg8k1fy5mnvxXv3Phata14+c0L+yPv8AVd+jt8=</latexit><latexit sha1_base64="Z0HSVnJsDJydbuQ5ghdR62WLb0g=">AAAB7nicbZBNSwMxEIazftb6VfXoJVgET2W3F0UvBS8eK9oPaJeSTWfb0Gw2JLNCKf0RXjwo4tXf481/Y9ruQVtfCDy8M0Nm3khLYdH3v7219Y3Nre3CTnF3b//gsHR03LRpZjg0eCpT046YBSkUNFCghLY2wJJIQisa3c7qrScwVqTqEccawoQNlIgFZ+is1gOCptXrXqnsV/y56CoEOZRJrnqv9NXtpzxLQCGXzNpO4GsMJ8yg4BKmxW5mQTM+YgPoOFQsARtO5utO6blz+jROjXsK6dz9PTFhibXjJHKdCcOhXa7NzP9qnQzjq3AilM4QFF98FGeSYkpnt9O+MMBRjh0wboTblfIhM4yjS6joQgiWT16FZrUSOL6vlms3eRwFckrOyAUJyCWpkTtSJw3CyYg8k1fy5mnvxXv3Phata14+c0L+yPv8AVd+jt8=</latexit><latexit sha1_base64="Z0HSVnJsDJydbuQ5ghdR62WLb0g=">AAAB7nicbZBNSwMxEIazftb6VfXoJVgET2W3F0UvBS8eK9oPaJeSTWfb0Gw2JLNCKf0RXjwo4tXf481/Y9ruQVtfCDy8M0Nm3khLYdH3v7219Y3Nre3CTnF3b//gsHR03LRpZjg0eCpT046YBSkUNFCghLY2wJJIQisa3c7qrScwVqTqEccawoQNlIgFZ+is1gOCptXrXqnsV/y56CoEOZRJrnqv9NXtpzxLQCGXzNpO4GsMJ8yg4BKmxW5mQTM+YgPoOFQsARtO5utO6blz+jROjXsK6dz9PTFhibXjJHKdCcOhXa7NzP9qnQzjq3AilM4QFF98FGeSYkpnt9O+MMBRjh0wboTblfIhM4yjS6joQgiWT16FZrUSOL6vlms3eRwFckrOyAUJyCWpkTtSJw3CyYg8k1fy5mnvxXv3Phata14+c0L+yPv8AVd+jt8=</latexit><latexit sha1_base64="Z0HSVnJsDJydbuQ5ghdR62WLb0g=">AAAB7nicbZBNSwMxEIazftb6VfXoJVgET2W3F0UvBS8eK9oPaJeSTWfb0Gw2JLNCKf0RXjwo4tXf481/Y9ruQVtfCDy8M0Nm3khLYdH3v7219Y3Nre3CTnF3b//gsHR03LRpZjg0eCpT046YBSkUNFCghLY2wJJIQisa3c7qrScwVqTqEccawoQNlIgFZ+is1gOCptXrXqnsV/y56CoEOZRJrnqv9NXtpzxLQCGXzNpO4GsMJ8yg4BKmxW5mQTM+YgPoOFQsARtO5utO6blz+jROjXsK6dz9PTFhibXjJHKdCcOhXa7NzP9qnQzjq3AilM4QFF98FGeSYkpnt9O+MMBRjh0wboTblfIhM4yjS6joQgiWT16FZrUSOL6vlms3eRwFckrOyAUJyCWpkTtSJw3CyYg8k1fy5mnvxXv3Phata14+c0L+yPv8AVd+jt8=</latexit>

Step 3:<latexit sha1_base64="5ekGCv5L1fE+ycVrMDvjJwgt8qc=">AAAB7nicbZA9SwNBEIb34leMX1FLm8UgWIW7WCjaBGwsI5oPSI6wt5lLluztHbtzQjjyI2wsFLH199j5b9wkV2jiCwsP78ywM2+QSGHQdb+dwtr6xuZWcbu0s7u3f1A+PGqZONUcmjyWse4EzIAUCpooUEIn0cCiQEI7GN/O6u0n0EbE6hEnCfgRGyoRCs7QWu0HhIReXPfLFbfqzkVXwcuhQnI1+uWv3iDmaQQKuWTGdD03QT9jGgWXMC31UgMJ42M2hK5FxSIwfjZfd0rPrDOgYaztU0jn7u+JjEXGTKLAdkYMR2a5NjP/q3VTDK/8TKgkRVB88VGYSooxnd1OB0IDRzmxwLgWdlfKR0wzjjahkg3BWz55FVq1qmf5vlap3+RxFMkJOSXnxCOXpE7uSIM0CSdj8kxeyZuTOC/Ou/OxaC04+cwx+SPn8wdZA47g</latexit><latexit sha1_base64="5ekGCv5L1fE+ycVrMDvjJwgt8qc=">AAAB7nicbZA9SwNBEIb34leMX1FLm8UgWIW7WCjaBGwsI5oPSI6wt5lLluztHbtzQjjyI2wsFLH199j5b9wkV2jiCwsP78ywM2+QSGHQdb+dwtr6xuZWcbu0s7u3f1A+PGqZONUcmjyWse4EzIAUCpooUEIn0cCiQEI7GN/O6u0n0EbE6hEnCfgRGyoRCs7QWu0HhIReXPfLFbfqzkVXwcuhQnI1+uWv3iDmaQQKuWTGdD03QT9jGgWXMC31UgMJ42M2hK5FxSIwfjZfd0rPrDOgYaztU0jn7u+JjEXGTKLAdkYMR2a5NjP/q3VTDK/8TKgkRVB88VGYSooxnd1OB0IDRzmxwLgWdlfKR0wzjjahkg3BWz55FVq1qmf5vlap3+RxFMkJOSXnxCOXpE7uSIM0CSdj8kxeyZuTOC/Ou/OxaC04+cwx+SPn8wdZA47g</latexit><latexit sha1_base64="5ekGCv5L1fE+ycVrMDvjJwgt8qc=">AAAB7nicbZA9SwNBEIb34leMX1FLm8UgWIW7WCjaBGwsI5oPSI6wt5lLluztHbtzQjjyI2wsFLH199j5b9wkV2jiCwsP78ywM2+QSGHQdb+dwtr6xuZWcbu0s7u3f1A+PGqZONUcmjyWse4EzIAUCpooUEIn0cCiQEI7GN/O6u0n0EbE6hEnCfgRGyoRCs7QWu0HhIReXPfLFbfqzkVXwcuhQnI1+uWv3iDmaQQKuWTGdD03QT9jGgWXMC31UgMJ42M2hK5FxSIwfjZfd0rPrDOgYaztU0jn7u+JjEXGTKLAdkYMR2a5NjP/q3VTDK/8TKgkRVB88VGYSooxnd1OB0IDRzmxwLgWdlfKR0wzjjahkg3BWz55FVq1qmf5vlap3+RxFMkJOSXnxCOXpE7uSIM0CSdj8kxeyZuTOC/Ou/OxaC04+cwx+SPn8wdZA47g</latexit><latexit sha1_base64="5ekGCv5L1fE+ycVrMDvjJwgt8qc=">AAAB7nicbZA9SwNBEIb34leMX1FLm8UgWIW7WCjaBGwsI5oPSI6wt5lLluztHbtzQjjyI2wsFLH199j5b9wkV2jiCwsP78ywM2+QSGHQdb+dwtr6xuZWcbu0s7u3f1A+PGqZONUcmjyWse4EzIAUCpooUEIn0cCiQEI7GN/O6u0n0EbE6hEnCfgRGyoRCs7QWu0HhIReXPfLFbfqzkVXwcuhQnI1+uWv3iDmaQQKuWTGdD03QT9jGgWXMC31UgMJ42M2hK5FxSIwfjZfd0rPrDOgYaztU0jn7u+JjEXGTKLAdkYMR2a5NjP/q3VTDK/8TKgkRVB88VGYSooxnd1OB0IDRzmxwLgWdlfKR0wzjjahkg3BWz55FVq1qmf5vlap3+RxFMkJOSXnxCOXpE7uSIM0CSdj8kxeyZuTOC/Ou/OxaC04+cwx+SPn8wdZA47g</latexit>

Step 4:<latexit sha1_base64="gsjxTw7aj3lE1DL/ZrTDxx2EtCA=">AAAB7nicbZBNS8NAEIY39avWr6pHL4tF8FSSIih6KXjxWNF+QBvKZjtpl242YXcilNAf4cWDIl79Pd78N27bHLT1hYWHd2bYmTdIpDDout9OYW19Y3OruF3a2d3bPygfHrVMnGoOTR7LWHcCZkAKBU0UKKGTaGBRIKEdjG9n9fYTaCNi9YiTBPyIDZUIBWdorfYDQkIvrvvlilt156Kr4OVQIbka/fJXbxDzNAKFXDJjup6boJ8xjYJLmJZ6qYGE8TEbQteiYhEYP5uvO6Vn1hnQMNb2KaRz9/dExiJjJlFgOyOGI7Ncm5n/1bophld+JlSSIii++ChMJcWYzm6nA6GBo5xYYFwLuyvlI6YZR5tQyYbgLZ+8Cq1a1bN8X6vUb/I4iuSEnJJz4pFLUid3pEGahJMxeSav5M1JnBfn3flYtBacfOaY/JHz+QNaiI7h</latexit><latexit sha1_base64="gsjxTw7aj3lE1DL/ZrTDxx2EtCA=">AAAB7nicbZBNS8NAEIY39avWr6pHL4tF8FSSIih6KXjxWNF+QBvKZjtpl242YXcilNAf4cWDIl79Pd78N27bHLT1hYWHd2bYmTdIpDDout9OYW19Y3OruF3a2d3bPygfHrVMnGoOTR7LWHcCZkAKBU0UKKGTaGBRIKEdjG9n9fYTaCNi9YiTBPyIDZUIBWdorfYDQkIvrvvlilt156Kr4OVQIbka/fJXbxDzNAKFXDJjup6boJ8xjYJLmJZ6qYGE8TEbQteiYhEYP5uvO6Vn1hnQMNb2KaRz9/dExiJjJlFgOyOGI7Ncm5n/1bophld+JlSSIii++ChMJcWYzm6nA6GBo5xYYFwLuyvlI6YZR5tQyYbgLZ+8Cq1a1bN8X6vUb/I4iuSEnJJz4pFLUid3pEGahJMxeSav5M1JnBfn3flYtBacfOaY/JHz+QNaiI7h</latexit><latexit sha1_base64="gsjxTw7aj3lE1DL/ZrTDxx2EtCA=">AAAB7nicbZBNS8NAEIY39avWr6pHL4tF8FSSIih6KXjxWNF+QBvKZjtpl242YXcilNAf4cWDIl79Pd78N27bHLT1hYWHd2bYmTdIpDDout9OYW19Y3OruF3a2d3bPygfHrVMnGoOTR7LWHcCZkAKBU0UKKGTaGBRIKEdjG9n9fYTaCNi9YiTBPyIDZUIBWdorfYDQkIvrvvlilt156Kr4OVQIbka/fJXbxDzNAKFXDJjup6boJ8xjYJLmJZ6qYGE8TEbQteiYhEYP5uvO6Vn1hnQMNb2KaRz9/dExiJjJlFgOyOGI7Ncm5n/1bophld+JlSSIii++ChMJcWYzm6nA6GBo5xYYFwLuyvlI6YZR5tQyYbgLZ+8Cq1a1bN8X6vUb/I4iuSEnJJz4pFLUid3pEGahJMxeSav5M1JnBfn3flYtBacfOaY/JHz+QNaiI7h</latexit><latexit sha1_base64="gsjxTw7aj3lE1DL/ZrTDxx2EtCA=">AAAB7nicbZBNS8NAEIY39avWr6pHL4tF8FSSIih6KXjxWNF+QBvKZjtpl242YXcilNAf4cWDIl79Pd78N27bHLT1hYWHd2bYmTdIpDDout9OYW19Y3OruF3a2d3bPygfHrVMnGoOTR7LWHcCZkAKBU0UKKGTaGBRIKEdjG9n9fYTaCNi9YiTBPyIDZUIBWdorfYDQkIvrvvlilt156Kr4OVQIbka/fJXbxDzNAKFXDJjup6boJ8xjYJLmJZ6qYGE8TEbQteiYhEYP5uvO6Vn1hnQMNb2KaRz9/dExiJjJlFgOyOGI7Ncm5n/1bophld+JlSSIii++ChMJcWYzm6nA6GBo5xYYFwLuyvlI6YZR5tQyYbgLZ+8Cq1a1bN8X6vUb/I4iuSEnJJz4pFLUid3pEGahJMxeSav5M1JnBfn3flYtBacfOaY/JHz+QNaiI7h</latexit>

Figure 2: Illustration of progressive prediction and our re-inforcement learning formulation of human pose predictionusing an example pose prediction task with length of pastobservations t = 3, length of total future predictions l = 8and size of step-wise prediction windows m = 2.

poses in videos [5–7, 19, 26, 47]. Chao et al. [6] proposeda 3D Pose Forecasting Network on static images; Barsoumet al. [5] took a probabilistic approach for pose predictionusing Wasserstein GAN [2]; Walker et al. [47] proposed avariational autoencoder solution; Fragkiadaki et al. [9] pro-posed two architectures denoted by LSTM-3LR (3 layers ofLSTM cells) and ERD (Encoder-Recurrent-Decoder); Yanet al. [51] and Zhao et al. [53] proposed methods for longertime prediction; Martinez et al. [26] used a carefully tai-lored RNN to learn human motion prediction; and Chiu etal. [7] proposed a multi-layer hierarchical RNN architec-ture (denoted by TP-RNN) to capture human dynamics. Incontrast, instead of training a fully supervised model, in thiswork we introduce the unsupervised GAIL framework intoour training process to enhance the generalizability of ourlearned prediction policy.Reinforcement Learning for Prediction: Methods of re-inforcement learning [23, 28, 29, 40] and imitation learn-ing [15, 33] have been used for predicting future informa-tion in videos in different forms. For instance, DeepMimic[30] proposed a physics-based method for policy genera-tion that is capable of tracking and motion capture. Otherworks such as R2P2 [31] and Tai et al. [41] used generativemodels with Wasserstein reformulation to perform naviga-tion and forward path planning. Differently, in this workwe reformulate sequential pose prediction into a reinforce-ment learning problem and train a prediction policy throughimitation learning approaches.

3. Pose Prediction as Reinforcement LearningIn this section, we introduce how to transform human

pose prediction into a reinforcement learning problem [40].The core task of human pose prediction is the following:given a sequence of past pose observations {x1, x2, . . . , xt}of length t from a human subject, predict the future posesequence {xt+1, xt+2, . . . , xt+l} up to length l. Most pre-vious work adopts an end-to-end approach to train either a

Page 3: Imitation Learning for Human Pose Prediction › pdf › 1909.03449v1.pdf · Imitation Learning for Human Pose Prediction Borui Wang, Ehsan Adeli, Hsu-kuang Chiu, De-An Huang, Juan

single recurrent neural network or a pair of encoder-decoderRNNs to output the whole length-l prediction sequence{xt+1, xt+2, . . . , xt+l} after reading in the historical se-quence {x1, x2, . . . , xt} [7, 19, 26]. In contrast, under ournew reinforcement learning formulation, we evenly breakthe whole prediction sequence {xt+1, xt+2, . . . , xt+l} intomultiple steps. In each step, we only make predictions overa small window of future poses conditioned on all observedand predicted pose information so far. This transformationturns our pose prediction task into a sequential decision-making problem, where reinforcement learning algorithmscan naturally come into play.

More formally, suppose we divide the whole predic-tion sequence into K steps, then each step would cor-respond to a window of m = l

K pose vectors (with-out loss of generality, assume that l is divisible by K).We now define a Markov Decision Process (MDP) [40]to model the generation of the pose prediction sequence.At each step i, where i ∈ {1, 2, . . . ,K}, the state ofthe MDP is defined as the list of all previous pose vec-tors: si = {x1, x2, . . . , xt+(i−1)×m} and the action ofthe MDP is defined as the next length-m window of posevectors that the learning agent needs to predict: ai ={xt+(i−1)×m+1, . . . , xt+i×m}. The transition dynamic ofthis MDP is deterministic, which means that at each stepi, taking an action ai = {xt+(i−1)×m+1, . . . , xt+i×m}at state si = {x1, x2, . . . , xt+(i−1)×m} would determin-istically transition the MDP into the new state si+1 ={x1, x2, . . . , xt+i×m} at step i + 1. The new state si+1 isformed by appending the action ai to the end of the currentstate si. This MDP process of progressive prediction isillustrated in Figure 2.Motivation: There are three key motivations behind this re-inforcement learning formulation of the human pose predic-tion problem. First, through breaking the long prediction se-quence into smaller pieces of pose windows, at each step theprediction agent only needs to focus on learning to predicta much shorter period of time into the future, which greatlyreduces the difficulty of prediction for the agent. Second,the strong sequential correlation across different actions ina MDP guarantees that our agent’s prediction policy willnot only focus on short-term prediction accuracy but alsotakes long-term prediction performance into consideration.Third, only through this formulation can we apply the un-supervised GAIL approach to enhance the generalizabilityof our pose prediction policy over unseen domains.

4. Imitation Learning for Predicting HumanPose Sequences

Now that we have formulated human pose prediction intoa reinforcement learning problem, in this section, we de-velop an imitation learning method to train the predictionpolicy under this new reinforcement learning formulation.

4.1. Deterministic Policy

Under our formulation, the policy of our reinforcementlearning agent would be a mapping from the space of allpossible states S to the space of all possible actionsA. Thispolicy can be either deterministic or stochastic. Under ourcurrent setting of human pose prediction, stochastic policies[40] are not suitable because the dimensionality of the ac-tion spaceA is very high. So we decide to use deterministicpolicies in our imitation learning algorithms for human poseprediction. Formally, the policy of our prediction agent is afunction a = πθ(s) that maps each possible state s in S to asingle action a in A, where θ denotes the parameters of thispolicy mapping function.

4.2. Behavioral Cloning

A classical approach to imitation learning is behavioralcloning (BC) [4, 33], where a policy is trained using super-vised learning methods to minimize the distance betweenits generated actions and the expert’s actions under the statedistribution encountered by the expert. Let the expert policyfunction be πE and the state distribution encountered by theexpert be dπE , then a behavioral cloning algorithm tries tofind an optimal policy πθ∗ such that:

πθ∗ = argminπθ∈Π

Es∼dπE[l(πθ(s), πE(s)

)], (1)

where l is a loss function that measures the distance betweentwo action vectors in the action space A.

BC enjoys the advantage of being highly sample-efficient and fast to compute [34], since it aims to directlymatch the learning agent’s policy with the expert’s pol-icy at every state that appears in the repository of expert-demonstrated trajectories. However, there are some caveatsassociated with BC. The first problem is the generalizationissue — the agent’s policy may be overfitted to the expert’sdemonstrations in the areas of the state space that the experttraversed, and thus may not generalize well to areas outsidethe expert’s experience. Second, behavioral cloning opti-mizes the agent’s policy at each individual state separatelyand ignores the sequential correlation across different stepsin a trajectory, which tends to make the agent’s policy short-sighted. To tackle these problems, we introduce generativeadversarial training into our algorithm and use an adaptedversion of GAIL to further optimize our agent’s policy.

4.3. Generative Adversarial Imitation Learning

Generative Adversarial Imitation Learning (GAIL) is apopular model-free imitation learning framework that wasrecently proposed by Ho and Ermon [15] and is inspiredby the success of Generative Adversarial Networks (GAN)[12]. At its core, GAIL takes an inverse reinforcementlearning [29] approach to the problem of imitation learning,

Page 4: Imitation Learning for Human Pose Prediction › pdf › 1909.03449v1.pdf · Imitation Learning for Human Pose Prediction Borui Wang, Ehsan Adeli, Hsu-kuang Chiu, De-An Huang, Juan

where it aims to first recover an estimate of the reward sig-nals underlying the MDP from the expert’s demonstrationand then learns to optimize the agent’s policy using the re-wards signals that it recovered. Directly estimating rewardfunctions from expert demonstration in high-dimensionaland complex state-action space is intractable, so GAIL turnsthe inverse reinforcement learning problem into its equiva-lent dual problem of occupancy measure matching, wherethe agent seeks to match the distribution of state-action pairs(called occupancy measure) generated by its own policy tothe distribution generated by the expert’s policy [15]. Inits original formulation, GAIL employs a generative adver-sarial training process to iteratively minimize the Jensen-Shannon divergence between the two distributions, and itslearning objective is:

minπθ

maxDw ∈ S×A→(0,1)

(Eπθ [log(Dw(s, a))]

+ EπE [log(1−Dw(s, a))]− λH(πθ)), (2)

where Dw can be interpreted as a discriminative clas-sifier trying to distinguish between state-action pairs gener-ated from the agent’s policy and state-action pairs generatedfrom the expert’s policy, and H(πθ) denotes the discountedcausal entropy of the policy πθ. Then in analogy to the train-ing of GANs, during actual implementation, a GAIL algo-rithm will alternate between a D step that updates the dis-criminator parameter w to maximize Eπθ [log(Dw(s, a))] +EπE [log(1 −Dw(s, a))], and a G step that updates the pa-rameter θ of the policy generator πθ using policy gradientmethods such as Trust Region Policy Optimization (TRPO)[35] or Proximal Policy Optimization (PPO) [36] in orderto minimize Eπθ [log(Dw(s, a))] + λH(πθ), until the sys-tem reaches a saddle point (see Figure 1). Previous workshows that GAIL often tends to generalize better than BCand achieves superior performance on complex tasks.

However, in our human pose prediction scenario, thereare two major difficulties that hinder the effective applica-tion of the original GAIL algorithm. First, the human mo-tion dynamics is highly complex, and thus our randomly ini-tialized policy generator will be vastly different from the ex-pert’s policy at the beginning of GAIL training. This meansthat at the early stage of GAIL training, it is very likely thatthe policy generator manifold would have no non-negligibleintersection with the expert manifold at all. This issue willlet the Jensen-Shannon divergence to saturate quickly andcause the severe problem of vanishing gradients for the dis-criminator [2]. Second, the dimensionality of our actionspaceA is very high, which makes the sampling-based pol-icy gradient reinforcement learning algorithms like TRPOand PPO no longer applicable in practice. Therefore, weadapt GAIL to a specific new form to fit the needs of ourhuman pose prediction task, which we call WGAIL-div.

Xt+Xt+(i-1)mX1 Xt Xt+1

… …GRU GRU GRU GRU

state embedding<latexit sha1_base64="Harz8//p4ZqAzloygK+PhQi+3x0=">AAAB+HicbZBNS8NAEIY39avWj1Y9elksgqeS9KLHghePFewHtKFsNpN26WYTdidCDf0lXjwo4tWf4s1/47bNQVtfWHh4Z4aZfYNUCoOu++2UtrZ3dvfK+5WDw6Pjau3ktGuSTHPo8EQmuh8wA1Io6KBACf1UA4sDCb1geruo9x5BG5GoB5yl4MdsrEQkOENrjWpVgwyBQhxAGAo1HtXqbsNdim6CV0CdFGqPal/DMOFZDAq5ZMYMPDdFP2caBZcwrwwzAynjUzaGgUXFYjB+vjx8Ti+tE9Io0fYppEv390TOYmNmcWA7Y4YTs15bmP/VBhlGN34uVJohKL5aFGWSYkIXKdBQaOAoZxYY18LeSvmEacbRZlWxIXjrX96EbrPhWb5v1lvNIo4yOScX5Ip45Jq0yB1pkw7hJCPP5JW8OU/Oi/PufKxaS04xc0b+yPn8Abkfkw8=</latexit><latexit sha1_base64="Harz8//p4ZqAzloygK+PhQi+3x0=">AAAB+HicbZBNS8NAEIY39avWj1Y9elksgqeS9KLHghePFewHtKFsNpN26WYTdidCDf0lXjwo4tWf4s1/47bNQVtfWHh4Z4aZfYNUCoOu++2UtrZ3dvfK+5WDw6Pjau3ktGuSTHPo8EQmuh8wA1Io6KBACf1UA4sDCb1geruo9x5BG5GoB5yl4MdsrEQkOENrjWpVgwyBQhxAGAo1HtXqbsNdim6CV0CdFGqPal/DMOFZDAq5ZMYMPDdFP2caBZcwrwwzAynjUzaGgUXFYjB+vjx8Ti+tE9Io0fYppEv390TOYmNmcWA7Y4YTs15bmP/VBhlGN34uVJohKL5aFGWSYkIXKdBQaOAoZxYY18LeSvmEacbRZlWxIXjrX96EbrPhWb5v1lvNIo4yOScX5Ip45Jq0yB1pkw7hJCPP5JW8OU/Oi/PufKxaS04xc0b+yPn8Abkfkw8=</latexit><latexit sha1_base64="Harz8//p4ZqAzloygK+PhQi+3x0=">AAAB+HicbZBNS8NAEIY39avWj1Y9elksgqeS9KLHghePFewHtKFsNpN26WYTdidCDf0lXjwo4tWf4s1/47bNQVtfWHh4Z4aZfYNUCoOu++2UtrZ3dvfK+5WDw6Pjau3ktGuSTHPo8EQmuh8wA1Io6KBACf1UA4sDCb1geruo9x5BG5GoB5yl4MdsrEQkOENrjWpVgwyBQhxAGAo1HtXqbsNdim6CV0CdFGqPal/DMOFZDAq5ZMYMPDdFP2caBZcwrwwzAynjUzaGgUXFYjB+vjx8Ti+tE9Io0fYppEv390TOYmNmcWA7Y4YTs15bmP/VBhlGN34uVJohKL5aFGWSYkIXKdBQaOAoZxYY18LeSvmEacbRZlWxIXjrX96EbrPhWb5v1lvNIo4yOScX5Ip45Jq0yB1pkw7hJCPP5JW8OU/Oi/PufKxaS04xc0b+yPn8Abkfkw8=</latexit><latexit sha1_base64="Harz8//p4ZqAzloygK+PhQi+3x0=">AAAB+HicbZBNS8NAEIY39avWj1Y9elksgqeS9KLHghePFewHtKFsNpN26WYTdidCDf0lXjwo4tWf4s1/47bNQVtfWHh4Z4aZfYNUCoOu++2UtrZ3dvfK+5WDw6Pjau3ktGuSTHPo8EQmuh8wA1Io6KBACf1UA4sDCb1geruo9x5BG5GoB5yl4MdsrEQkOENrjWpVgwyBQhxAGAo1HtXqbsNdim6CV0CdFGqPal/DMOFZDAq5ZMYMPDdFP2caBZcwrwwzAynjUzaGgUXFYjB+vjx8Ti+tE9Io0fYppEv390TOYmNmcWA7Y4YTs15bmP/VBhlGN34uVJohKL5aFGWSYkIXKdBQaOAoZxYY18LeSvmEacbRZlWxIXjrX96EbrPhWb5v1lvNIo4yOScX5Ip45Jq0yB1pkw7hJCPP5JW8OU/Oi/PufKxaS04xc0b+yPn8Abkfkw8=</latexit>

(size 1024)<latexit sha1_base64="U/mAFdslXZJ3xroAkY2SP2rzxW4=">AAAB8nicbZDLSgMxFIYz9VbrrerSTbAIdVNmBkGXBTcuK9gLTIeSSc+0oZlkSDJCHfoYblwo4tancefbmLaz0NYfAh//OYec80cpZ9q47rdT2tjc2t4p71b29g8Oj6rHJx0tM0WhTSWXqhcRDZwJaBtmOPRSBSSJOHSjye283n0EpZkUD2aaQpiQkWAxo8RYK6hr9gTYc/2ry0G15jbchfA6eAXUUKHWoPrVH0qaJSAM5UTrwHNTE+ZEGUY5zCr9TENK6ISMILAoSAI6zBcrz/CFdYY4lso+YfDC/T2Rk0TraRLZzoSYsV6tzc3/akFm4pswZyLNDAi6/CjOODYSz+/HQ6aAGj61QKhidldMx0QRamxKFRuCt3ryOnT8hmf53q81/SKOMjpD56iOPHSNmugOtVAbUSTRM3pFb45xXpx352PZWnKKmVP0R87nDyX2j8k=</latexit><latexit sha1_base64="U/mAFdslXZJ3xroAkY2SP2rzxW4=">AAAB8nicbZDLSgMxFIYz9VbrrerSTbAIdVNmBkGXBTcuK9gLTIeSSc+0oZlkSDJCHfoYblwo4tancefbmLaz0NYfAh//OYec80cpZ9q47rdT2tjc2t4p71b29g8Oj6rHJx0tM0WhTSWXqhcRDZwJaBtmOPRSBSSJOHSjye283n0EpZkUD2aaQpiQkWAxo8RYK6hr9gTYc/2ry0G15jbchfA6eAXUUKHWoPrVH0qaJSAM5UTrwHNTE+ZEGUY5zCr9TENK6ISMILAoSAI6zBcrz/CFdYY4lso+YfDC/T2Rk0TraRLZzoSYsV6tzc3/akFm4pswZyLNDAi6/CjOODYSz+/HQ6aAGj61QKhidldMx0QRamxKFRuCt3ryOnT8hmf53q81/SKOMjpD56iOPHSNmugOtVAbUSTRM3pFb45xXpx352PZWnKKmVP0R87nDyX2j8k=</latexit><latexit sha1_base64="U/mAFdslXZJ3xroAkY2SP2rzxW4=">AAAB8nicbZDLSgMxFIYz9VbrrerSTbAIdVNmBkGXBTcuK9gLTIeSSc+0oZlkSDJCHfoYblwo4tancefbmLaz0NYfAh//OYec80cpZ9q47rdT2tjc2t4p71b29g8Oj6rHJx0tM0WhTSWXqhcRDZwJaBtmOPRSBSSJOHSjye283n0EpZkUD2aaQpiQkWAxo8RYK6hr9gTYc/2ry0G15jbchfA6eAXUUKHWoPrVH0qaJSAM5UTrwHNTE+ZEGUY5zCr9TENK6ISMILAoSAI6zBcrz/CFdYY4lso+YfDC/T2Rk0TraRLZzoSYsV6tzc3/akFm4pswZyLNDAi6/CjOODYSz+/HQ6aAGj61QKhidldMx0QRamxKFRuCt3ryOnT8hmf53q81/SKOMjpD56iOPHSNmugOtVAbUSTRM3pFb45xXpx352PZWnKKmVP0R87nDyX2j8k=</latexit><latexit sha1_base64="U/mAFdslXZJ3xroAkY2SP2rzxW4=">AAAB8nicbZDLSgMxFIYz9VbrrerSTbAIdVNmBkGXBTcuK9gLTIeSSc+0oZlkSDJCHfoYblwo4tancefbmLaz0NYfAh//OYec80cpZ9q47rdT2tjc2t4p71b29g8Oj6rHJx0tM0WhTSWXqhcRDZwJaBtmOPRSBSSJOHSjye283n0EpZkUD2aaQpiQkWAxo8RYK6hr9gTYc/2ry0G15jbchfA6eAXUUKHWoPrVH0qaJSAM5UTrwHNTE+ZEGUY5zCr9TENK6ISMILAoSAI6zBcrz/CFdYY4lso+YfDC/T2Rk0TraRLZzoSYsV6tzc3/akFm4pswZyLNDAi6/CjOODYSz+/HQ6aAGj61QKhidldMx0QRamxKFRuCt3ryOnT8hmf53q81/SKOMjpD56iOPHSNmugOtVAbUSTRM3pFb45xXpx352PZWnKKmVP0R87nDyX2j8k=</latexit>

… …

state si<latexit sha1_base64="vbHNn31ZZ33sVPiQnudHXPNWuQU=">AAAB83icbZA9TwJBEIbn8AvxC7W02QgmVuSORu1IbCwxETCBC9lb5mDD3kd250wI4W/YWGiMrX/Gzn/jAlco+CabPHlnJjP7BqmShlz32ylsbG5t7xR3S3v7B4dH5eOTtkkyLbAlEpXox4AbVDLGFklS+Jhq5FGgsBOMb+f1zhNqI5P4gSYp+hEfxjKUgpO1eoY4Iauavqyyfrni1tyF2Dp4OVQgV7Nf/uoNEpFFGJNQ3Jiu56bkT7kmKRTOSr3MYMrFmA+xazHmERp/urh5xi6sM2Bhou2LiS3c3xNTHhkziQLbGXEamdXa3Pyv1s0ovPanMk4zwlgsF4WZYpSweQBsIDUKUhMLXGhpb2VixDUXZGMq2RC81S+vQ7te8yzf1yuNmzyOIpzBOVyCB1fQgDtoQgsEpPAMr/DmZM6L8+58LFsLTj5zCn/kfP4AwdGQyw==</latexit><latexit sha1_base64="vbHNn31ZZ33sVPiQnudHXPNWuQU=">AAAB83icbZA9TwJBEIbn8AvxC7W02QgmVuSORu1IbCwxETCBC9lb5mDD3kd250wI4W/YWGiMrX/Gzn/jAlco+CabPHlnJjP7BqmShlz32ylsbG5t7xR3S3v7B4dH5eOTtkkyLbAlEpXox4AbVDLGFklS+Jhq5FGgsBOMb+f1zhNqI5P4gSYp+hEfxjKUgpO1eoY4Iauavqyyfrni1tyF2Dp4OVQgV7Nf/uoNEpFFGJNQ3Jiu56bkT7kmKRTOSr3MYMrFmA+xazHmERp/urh5xi6sM2Bhou2LiS3c3xNTHhkziQLbGXEamdXa3Pyv1s0ovPanMk4zwlgsF4WZYpSweQBsIDUKUhMLXGhpb2VixDUXZGMq2RC81S+vQ7te8yzf1yuNmzyOIpzBOVyCB1fQgDtoQgsEpPAMr/DmZM6L8+58LFsLTj5zCn/kfP4AwdGQyw==</latexit><latexit sha1_base64="vbHNn31ZZ33sVPiQnudHXPNWuQU=">AAAB83icbZA9TwJBEIbn8AvxC7W02QgmVuSORu1IbCwxETCBC9lb5mDD3kd250wI4W/YWGiMrX/Gzn/jAlco+CabPHlnJjP7BqmShlz32ylsbG5t7xR3S3v7B4dH5eOTtkkyLbAlEpXox4AbVDLGFklS+Jhq5FGgsBOMb+f1zhNqI5P4gSYp+hEfxjKUgpO1eoY4Iauavqyyfrni1tyF2Dp4OVQgV7Nf/uoNEpFFGJNQ3Jiu56bkT7kmKRTOSr3MYMrFmA+xazHmERp/urh5xi6sM2Bhou2LiS3c3xNTHhkziQLbGXEamdXa3Pyv1s0ovPanMk4zwlgsF4WZYpSweQBsIDUKUhMLXGhpb2VixDUXZGMq2RC81S+vQ7te8yzf1yuNmzyOIpzBOVyCB1fQgDtoQgsEpPAMr/DmZM6L8+58LFsLTj5zCn/kfP4AwdGQyw==</latexit><latexit sha1_base64="vbHNn31ZZ33sVPiQnudHXPNWuQU=">AAAB83icbZA9TwJBEIbn8AvxC7W02QgmVuSORu1IbCwxETCBC9lb5mDD3kd250wI4W/YWGiMrX/Gzn/jAlco+CabPHlnJjP7BqmShlz32ylsbG5t7xR3S3v7B4dH5eOTtkkyLbAlEpXox4AbVDLGFklS+Jhq5FGgsBOMb+f1zhNqI5P4gSYp+hEfxjKUgpO1eoY4Iauavqyyfrni1tyF2Dp4OVQgV7Nf/uoNEpFFGJNQ3Jiu56bkT7kmKRTOSr3MYMrFmA+xazHmERp/urh5xi6sM2Bhou2LiS3c3xNTHhkziQLbGXEamdXa3Pyv1s0ovPanMk4zwlgsF4WZYpSweQBsIDUKUhMLXGhpb2VixDUXZGMq2RC81S+vQ7te8yzf1yuNmzyOIpzBOVyCB1fQgDtoQgsEpPAMr/DmZM6L8+58LFsLTj5zCn/kfP4AwdGQyw==</latexit>

Xt+(i-1)m+1

GRU GRU…

… Xt+im

action ai<latexit sha1_base64="L6dz62kMJzbmmyoLciPLPwrRceA=">AAAB9HicbZDNTgIxFIXv4B/iH+rSTSOYuCIzbNQdiRuXmAiYwITcKR1o6HTGtkNCJjyHGxca49aHcefb2AEWCt6k6Zdz7k1vT5AIro3rfjuFjc2t7Z3ibmlv/+DwqHx80tZxqihr0VjE6jFAzQSXrGW4EewxUQyjQLBOML7N/c6EKc1j+WCmCfMjHEoecorGSj7S/CJV7PMq6Zcrbs2dF1kHbwkVWFazX/7qDWKaRkwaKlDrrucmxs9QGU4Fm5V6qWYJ0jEOWdeixIhpP5svPSMXVhmQMFb2SEPm6u+JDCOtp1FgOyM0I73q5eJ/Xjc14bWfcZmkhkm6eChMBTExyRMgA64YNWJqAanidldCR6hsFDankg3BW/3yOrTrNc/yfb3SuFnGUYQzOIdL8OAKGnAHTWgBhSd4hld4cybOi/PufCxaC85y5hT+lPP5A1x5kSA=</latexit><latexit sha1_base64="L6dz62kMJzbmmyoLciPLPwrRceA=">AAAB9HicbZDNTgIxFIXv4B/iH+rSTSOYuCIzbNQdiRuXmAiYwITcKR1o6HTGtkNCJjyHGxca49aHcefb2AEWCt6k6Zdz7k1vT5AIro3rfjuFjc2t7Z3ibmlv/+DwqHx80tZxqihr0VjE6jFAzQSXrGW4EewxUQyjQLBOML7N/c6EKc1j+WCmCfMjHEoecorGSj7S/CJV7PMq6Zcrbs2dF1kHbwkVWFazX/7qDWKaRkwaKlDrrucmxs9QGU4Fm5V6qWYJ0jEOWdeixIhpP5svPSMXVhmQMFb2SEPm6u+JDCOtp1FgOyM0I73q5eJ/Xjc14bWfcZmkhkm6eChMBTExyRMgA64YNWJqAanidldCR6hsFDankg3BW/3yOrTrNc/yfb3SuFnGUYQzOIdL8OAKGnAHTWgBhSd4hld4cybOi/PufCxaC85y5hT+lPP5A1x5kSA=</latexit><latexit sha1_base64="L6dz62kMJzbmmyoLciPLPwrRceA=">AAAB9HicbZDNTgIxFIXv4B/iH+rSTSOYuCIzbNQdiRuXmAiYwITcKR1o6HTGtkNCJjyHGxca49aHcefb2AEWCt6k6Zdz7k1vT5AIro3rfjuFjc2t7Z3ibmlv/+DwqHx80tZxqihr0VjE6jFAzQSXrGW4EewxUQyjQLBOML7N/c6EKc1j+WCmCfMjHEoecorGSj7S/CJV7PMq6Zcrbs2dF1kHbwkVWFazX/7qDWKaRkwaKlDrrucmxs9QGU4Fm5V6qWYJ0jEOWdeixIhpP5svPSMXVhmQMFb2SEPm6u+JDCOtp1FgOyM0I73q5eJ/Xjc14bWfcZmkhkm6eChMBTExyRMgA64YNWJqAanidldCR6hsFDankg3BW/3yOrTrNc/yfb3SuFnGUYQzOIdL8OAKGnAHTWgBhSd4hld4cybOi/PufCxaC85y5hT+lPP5A1x5kSA=</latexit><latexit sha1_base64="L6dz62kMJzbmmyoLciPLPwrRceA=">AAAB9HicbZDNTgIxFIXv4B/iH+rSTSOYuCIzbNQdiRuXmAiYwITcKR1o6HTGtkNCJjyHGxca49aHcefb2AEWCt6k6Zdz7k1vT5AIro3rfjuFjc2t7Z3ibmlv/+DwqHx80tZxqihr0VjE6jFAzQSXrGW4EewxUQyjQLBOML7N/c6EKc1j+WCmCfMjHEoecorGSj7S/CJV7PMq6Zcrbs2dF1kHbwkVWFazX/7qDWKaRkwaKlDrrucmxs9QGU4Fm5V6qWYJ0jEOWdeixIhpP5svPSMXVhmQMFb2SEPm6u+JDCOtp1FgOyM0I73q5eJ/Xjc14bWfcZmkhkm6eChMBTExyRMgA64YNWJqAanidldCR6hsFDankg3BW/3yOrTrNc/yfb3SuFnGUYQzOIdL8OAKGnAHTWgBhSd4hld4cybOi/PufCxaC85y5hT+lPP5A1x5kSA=</latexit>

( RNN Encoder<latexit sha1_base64="Kj5fW4nOXq/nJ9FXx4Pd7wowq3o=">AAAB8nicbZDLSgMxFIYzXmu9VV26CRbBVZnpRt0VRHBVqtgLTIeSyWTa0EwyJGeEMvQx3LhQxK1P4863MW1noa0/BD7+cw455w9TwQ247reztr6xubVd2inv7u0fHFaOjjtGZZqyNlVC6V5IDBNcsjZwEKyXakaSULBuOL6Z1btPTBuu5CNMUhYkZCh5zCkBa/kPzSa+lVRFTA8qVbfmzoVXwSugigq1BpWvfqRoljAJVBBjfM9NIciJBk4Fm5b7mWEpoWMyZL5FSRJmgny+8hSfWyfCsdL2ScBz9/dEThJjJkloOxMCI7Ncm5n/1fwM4qsg5zLNgEm6+CjOBAaFZ/fjiGtGQUwsEKq53RXTEdGEgk2pbEPwlk9ehU695lm+r1cb10UcJXSKztAF8tAlaqA71EJtRJFCz+gVvTngvDjvzseidc0pZk7QHzmfP3P1kKs=</latexit><latexit sha1_base64="Kj5fW4nOXq/nJ9FXx4Pd7wowq3o=">AAAB8nicbZDLSgMxFIYzXmu9VV26CRbBVZnpRt0VRHBVqtgLTIeSyWTa0EwyJGeEMvQx3LhQxK1P4863MW1noa0/BD7+cw455w9TwQ247reztr6xubVd2inv7u0fHFaOjjtGZZqyNlVC6V5IDBNcsjZwEKyXakaSULBuOL6Z1btPTBuu5CNMUhYkZCh5zCkBa/kPzSa+lVRFTA8qVbfmzoVXwSugigq1BpWvfqRoljAJVBBjfM9NIciJBk4Fm5b7mWEpoWMyZL5FSRJmgny+8hSfWyfCsdL2ScBz9/dEThJjJkloOxMCI7Ncm5n/1fwM4qsg5zLNgEm6+CjOBAaFZ/fjiGtGQUwsEKq53RXTEdGEgk2pbEPwlk9ehU695lm+r1cb10UcJXSKztAF8tAlaqA71EJtRJFCz+gVvTngvDjvzseidc0pZk7QHzmfP3P1kKs=</latexit><latexit sha1_base64="Kj5fW4nOXq/nJ9FXx4Pd7wowq3o=">AAAB8nicbZDLSgMxFIYzXmu9VV26CRbBVZnpRt0VRHBVqtgLTIeSyWTa0EwyJGeEMvQx3LhQxK1P4863MW1noa0/BD7+cw455w9TwQ247reztr6xubVd2inv7u0fHFaOjjtGZZqyNlVC6V5IDBNcsjZwEKyXakaSULBuOL6Z1btPTBuu5CNMUhYkZCh5zCkBa/kPzSa+lVRFTA8qVbfmzoVXwSugigq1BpWvfqRoljAJVBBjfM9NIciJBk4Fm5b7mWEpoWMyZL5FSRJmgny+8hSfWyfCsdL2ScBz9/dEThJjJkloOxMCI7Ncm5n/1fwM4qsg5zLNgEm6+CjOBAaFZ/fjiGtGQUwsEKq53RXTEdGEgk2pbEPwlk9ehU695lm+r1cb10UcJXSKztAF8tAlaqA71EJtRJFCz+gVvTngvDjvzseidc0pZk7QHzmfP3P1kKs=</latexit><latexit sha1_base64="Kj5fW4nOXq/nJ9FXx4Pd7wowq3o=">AAAB8nicbZDLSgMxFIYzXmu9VV26CRbBVZnpRt0VRHBVqtgLTIeSyWTa0EwyJGeEMvQx3LhQxK1P4863MW1noa0/BD7+cw455w9TwQ247reztr6xubVd2inv7u0fHFaOjjtGZZqyNlVC6V5IDBNcsjZwEKyXakaSULBuOL6Z1btPTBuu5CNMUhYkZCh5zCkBa/kPzSa+lVRFTA8qVbfmzoVXwSugigq1BpWvfqRoljAJVBBjfM9NIciJBk4Fm5b7mWEpoWMyZL5FSRJmgny+8hSfWyfCsdL2ScBz9/dEThJjJkloOxMCI7Ncm5n/1fwM4qsg5zLNgEm6+CjOBAaFZ/fjiGtGQUwsEKq53RXTEdGEgk2pbEPwlk9ehU695lm+r1cb10UcJXSKztAF8tAlaqA71EJtRJFCz+gVvTngvDjvzseidc0pZk7QHzmfP3P1kKs=</latexit> ( RNN Decoder

<latexit sha1_base64="hOvhECcOY4PaXxP1s7b3viCBmkw=">AAAB8nicbZDLSgMxFIYzXmu9VV26CRbBVZnpRt0VdOGqVLEXmA4lkznThmaSIckIZehjuHGhiFufxp1vY9rOQlt/CHz85xxyzh+mnGnjut/O2vrG5tZ2aae8u7d/cFg5Ou5omSkKbSq5VL2QaOBMQNsww6GXKiBJyKEbjm9m9e4TKM2keDSTFIKEDAWLGSXGWv5Ds4lvgcoI1KBSdWvuXHgVvAKqqFBrUPnqR5JmCQhDOdHa99zUBDlRhlEO03I/05ASOiZD8C0KkoAO8vnKU3xunQjHUtknDJ67vydykmg9SULbmRAz0su1mflfzc9MfBXkTKSZAUEXH8UZx0bi2f04Ygqo4RMLhCpmd8V0RBShxqZUtiF4yyevQqde8yzf16uN6yKOEjpFZ+gCeegSNdAdaqE2okiiZ/SK3hzjvDjvzseidc0pZk7QHzmfP2SakKE=</latexit><latexit sha1_base64="hOvhECcOY4PaXxP1s7b3viCBmkw=">AAAB8nicbZDLSgMxFIYzXmu9VV26CRbBVZnpRt0VdOGqVLEXmA4lkznThmaSIckIZehjuHGhiFufxp1vY9rOQlt/CHz85xxyzh+mnGnjut/O2vrG5tZ2aae8u7d/cFg5Ou5omSkKbSq5VL2QaOBMQNsww6GXKiBJyKEbjm9m9e4TKM2keDSTFIKEDAWLGSXGWv5Ds4lvgcoI1KBSdWvuXHgVvAKqqFBrUPnqR5JmCQhDOdHa99zUBDlRhlEO03I/05ASOiZD8C0KkoAO8vnKU3xunQjHUtknDJ67vydykmg9SULbmRAz0su1mflfzc9MfBXkTKSZAUEXH8UZx0bi2f04Ygqo4RMLhCpmd8V0RBShxqZUtiF4yyevQqde8yzf16uN6yKOEjpFZ+gCeegSNdAdaqE2okiiZ/SK3hzjvDjvzseidc0pZk7QHzmfP2SakKE=</latexit><latexit sha1_base64="hOvhECcOY4PaXxP1s7b3viCBmkw=">AAAB8nicbZDLSgMxFIYzXmu9VV26CRbBVZnpRt0VdOGqVLEXmA4lkznThmaSIckIZehjuHGhiFufxp1vY9rOQlt/CHz85xxyzh+mnGnjut/O2vrG5tZ2aae8u7d/cFg5Ou5omSkKbSq5VL2QaOBMQNsww6GXKiBJyKEbjm9m9e4TKM2keDSTFIKEDAWLGSXGWv5Ds4lvgcoI1KBSdWvuXHgVvAKqqFBrUPnqR5JmCQhDOdHa99zUBDlRhlEO03I/05ASOiZD8C0KkoAO8vnKU3xunQjHUtknDJ67vydykmg9SULbmRAz0su1mflfzc9MfBXkTKSZAUEXH8UZx0bi2f04Ygqo4RMLhCpmd8V0RBShxqZUtiF4yyevQqde8yzf16uN6yKOEjpFZ+gCeegSNdAdaqE2okiiZ/SK3hzjvDjvzseidc0pZk7QHzmfP2SakKE=</latexit><latexit sha1_base64="hOvhECcOY4PaXxP1s7b3viCBmkw=">AAAB8nicbZDLSgMxFIYzXmu9VV26CRbBVZnpRt0VdOGqVLEXmA4lkznThmaSIckIZehjuHGhiFufxp1vY9rOQlt/CHz85xxyzh+mnGnjut/O2vrG5tZ2aae8u7d/cFg5Ou5omSkKbSq5VL2QaOBMQNsww6GXKiBJyKEbjm9m9e4TKM2keDSTFIKEDAWLGSXGWv5Ds4lvgcoI1KBSdWvuXHgVvAKqqFBrUPnqR5JmCQhDOdHa99zUBDlRhlEO03I/05ASOiZD8C0KkoAO8vnKU3xunQjHUtknDJ67vydykmg9SULbmRAz0su1mflfzc9MfBXkTKSZAUEXH8UZx0bi2f04Ygqo4RMLhCpmd8V0RBShxqZUtiF4yyevQqde8yzf16uN6yKOEjpFZ+gCeegSNdAdaqE2okiiZ/SK3hzjvDjvzseidc0pZk7QHzmfP2SakKE=</latexit>

Figure 3: Seq2Seq architecture of our policy generator.

4.4. Adapted WGAIL-div

The problem of non-overlapping manifolds and vanish-ing gradients for the discriminator has long been studiedfor GANs [2], and several new GAN training frameworksbased on Wasserstein distances between distributions hasbeen proposed to overcome this problem. Some of thenotable ones are Wasserstein GAN (WGAN) [2], Wasser-stein GAN with gradient penalty (WGAN-gp) [13], andWasserstein-divergence GAN (WGAN-div) [50]. This se-ries of Wasserstein-based GANs changes the discriminatorof GANs from a classifier that tries to distinguish betweenreal and fake samples and outputs probability values be-tween 0 and 1 into a critic function that directly outputs realnumbers in (−∞,+∞), and uses different methods to en-force the Lipschitz constraint on the critic function. In thiswork, we borrow the formulation of WGAN-div [50] intoGAIL to construct an improved version of GAIL namedWGAIL-div. More specifically, the objective function ofour WGAIL-div algorithm is:

minπθ

maxDw ∈ S×A→R

(Eπθ [Dw(s, a)]− EπE [Dw(s, a)]

− k E(s,a)∼Pu

[‖∇(s,a)Dw((s, a))‖p]), (3)

where Pu is the distribution obtained by sampling uni-formly from the straight lines connecting points of state-action pairs generated by the policy generator and points ofstate-action pairs in expert demonstration in the state-actionspace S × A, and k and p are hyperparameters that adjustthe level of Lipschitz-constraint regularization. Note that inthis WGAIL-div objective function we no longer have thecausal entropy term H(πθ) that appears in (2), since we usedeterministic policies that have constant causal entropies.

In our implementation of WGAIL-div, we alternate be-tween a D step that updates the discriminator param-eter w to maximize Eπθ [Dw(s, a)] − EπE [Dw(s, a)] −k E

(s,a)∼Pu[‖∇(s,a)Dw((s, a))‖p], and a G step that updates

the parameter θ of the policy generator πθ to minimizeEπθ [Dw(s, a)]. As mentioned in the second problem fac-ing GAIL above, for the G step, the classic sampling-based

Page 5: Imitation Learning for Human Pose Prediction › pdf › 1909.03449v1.pdf · Imitation Learning for Human Pose Prediction Borui Wang, Ehsan Adeli, Hsu-kuang Chiu, De-An Huang, Juan

… …LSTM LSTM LSTM LSTM

state embedding<latexit sha1_base64="Harz8//p4ZqAzloygK+PhQi+3x0=">AAAB+HicbZBNS8NAEIY39avWj1Y9elksgqeS9KLHghePFewHtKFsNpN26WYTdidCDf0lXjwo4tWf4s1/47bNQVtfWHh4Z4aZfYNUCoOu++2UtrZ3dvfK+5WDw6Pjau3ktGuSTHPo8EQmuh8wA1Io6KBACf1UA4sDCb1geruo9x5BG5GoB5yl4MdsrEQkOENrjWpVgwyBQhxAGAo1HtXqbsNdim6CV0CdFGqPal/DMOFZDAq5ZMYMPDdFP2caBZcwrwwzAynjUzaGgUXFYjB+vjx8Ti+tE9Io0fYppEv390TOYmNmcWA7Y4YTs15bmP/VBhlGN34uVJohKL5aFGWSYkIXKdBQaOAoZxYY18LeSvmEacbRZlWxIXjrX96EbrPhWb5v1lvNIo4yOScX5Ip45Jq0yB1pkw7hJCPP5JW8OU/Oi/PufKxaS04xc0b+yPn8Abkfkw8=</latexit><latexit sha1_base64="Harz8//p4ZqAzloygK+PhQi+3x0=">AAAB+HicbZBNS8NAEIY39avWj1Y9elksgqeS9KLHghePFewHtKFsNpN26WYTdidCDf0lXjwo4tWf4s1/47bNQVtfWHh4Z4aZfYNUCoOu++2UtrZ3dvfK+5WDw6Pjau3ktGuSTHPo8EQmuh8wA1Io6KBACf1UA4sDCb1geruo9x5BG5GoB5yl4MdsrEQkOENrjWpVgwyBQhxAGAo1HtXqbsNdim6CV0CdFGqPal/DMOFZDAq5ZMYMPDdFP2caBZcwrwwzAynjUzaGgUXFYjB+vjx8Ti+tE9Io0fYppEv390TOYmNmcWA7Y4YTs15bmP/VBhlGN34uVJohKL5aFGWSYkIXKdBQaOAoZxYY18LeSvmEacbRZlWxIXjrX96EbrPhWb5v1lvNIo4yOScX5Ip45Jq0yB1pkw7hJCPP5JW8OU/Oi/PufKxaS04xc0b+yPn8Abkfkw8=</latexit><latexit sha1_base64="Harz8//p4ZqAzloygK+PhQi+3x0=">AAAB+HicbZBNS8NAEIY39avWj1Y9elksgqeS9KLHghePFewHtKFsNpN26WYTdidCDf0lXjwo4tWf4s1/47bNQVtfWHh4Z4aZfYNUCoOu++2UtrZ3dvfK+5WDw6Pjau3ktGuSTHPo8EQmuh8wA1Io6KBACf1UA4sDCb1geruo9x5BG5GoB5yl4MdsrEQkOENrjWpVgwyBQhxAGAo1HtXqbsNdim6CV0CdFGqPal/DMOFZDAq5ZMYMPDdFP2caBZcwrwwzAynjUzaGgUXFYjB+vjx8Ti+tE9Io0fYppEv390TOYmNmcWA7Y4YTs15bmP/VBhlGN34uVJohKL5aFGWSYkIXKdBQaOAoZxYY18LeSvmEacbRZlWxIXjrX96EbrPhWb5v1lvNIo4yOScX5Ip45Jq0yB1pkw7hJCPP5JW8OU/Oi/PufKxaS04xc0b+yPn8Abkfkw8=</latexit><latexit sha1_base64="Harz8//p4ZqAzloygK+PhQi+3x0=">AAAB+HicbZBNS8NAEIY39avWj1Y9elksgqeS9KLHghePFewHtKFsNpN26WYTdidCDf0lXjwo4tWf4s1/47bNQVtfWHh4Z4aZfYNUCoOu++2UtrZ3dvfK+5WDw6Pjau3ktGuSTHPo8EQmuh8wA1Io6KBACf1UA4sDCb1geruo9x5BG5GoB5yl4MdsrEQkOENrjWpVgwyBQhxAGAo1HtXqbsNdim6CV0CdFGqPal/DMOFZDAq5ZMYMPDdFP2caBZcwrwwzAynjUzaGgUXFYjB+vjx8Ti+tE9Io0fYppEv390TOYmNmcWA7Y4YTs15bmP/VBhlGN34uVJohKL5aFGWSYkIXKdBQaOAoZxYY18LeSvmEacbRZlWxIXjrX96EbrPhWb5v1lvNIo4yOScX5Ip45Jq0yB1pkw7hJCPP5JW8OU/Oi/PufKxaS04xc0b+yPn8Abkfkw8=</latexit>

(size 1024)<latexit sha1_base64="U/mAFdslXZJ3xroAkY2SP2rzxW4=">AAAB8nicbZDLSgMxFIYz9VbrrerSTbAIdVNmBkGXBTcuK9gLTIeSSc+0oZlkSDJCHfoYblwo4tancefbmLaz0NYfAh//OYec80cpZ9q47rdT2tjc2t4p71b29g8Oj6rHJx0tM0WhTSWXqhcRDZwJaBtmOPRSBSSJOHSjye283n0EpZkUD2aaQpiQkWAxo8RYK6hr9gTYc/2ry0G15jbchfA6eAXUUKHWoPrVH0qaJSAM5UTrwHNTE+ZEGUY5zCr9TENK6ISMILAoSAI6zBcrz/CFdYY4lso+YfDC/T2Rk0TraRLZzoSYsV6tzc3/akFm4pswZyLNDAi6/CjOODYSz+/HQ6aAGj61QKhidldMx0QRamxKFRuCt3ryOnT8hmf53q81/SKOMjpD56iOPHSNmugOtVAbUSTRM3pFb45xXpx352PZWnKKmVP0R87nDyX2j8k=</latexit><latexit sha1_base64="U/mAFdslXZJ3xroAkY2SP2rzxW4=">AAAB8nicbZDLSgMxFIYz9VbrrerSTbAIdVNmBkGXBTcuK9gLTIeSSc+0oZlkSDJCHfoYblwo4tancefbmLaz0NYfAh//OYec80cpZ9q47rdT2tjc2t4p71b29g8Oj6rHJx0tM0WhTSWXqhcRDZwJaBtmOPRSBSSJOHSjye283n0EpZkUD2aaQpiQkWAxo8RYK6hr9gTYc/2ry0G15jbchfA6eAXUUKHWoPrVH0qaJSAM5UTrwHNTE+ZEGUY5zCr9TENK6ISMILAoSAI6zBcrz/CFdYY4lso+YfDC/T2Rk0TraRLZzoSYsV6tzc3/akFm4pswZyLNDAi6/CjOODYSz+/HQ6aAGj61QKhidldMx0QRamxKFRuCt3ryOnT8hmf53q81/SKOMjpD56iOPHSNmugOtVAbUSTRM3pFb45xXpx352PZWnKKmVP0R87nDyX2j8k=</latexit><latexit sha1_base64="U/mAFdslXZJ3xroAkY2SP2rzxW4=">AAAB8nicbZDLSgMxFIYz9VbrrerSTbAIdVNmBkGXBTcuK9gLTIeSSc+0oZlkSDJCHfoYblwo4tancefbmLaz0NYfAh//OYec80cpZ9q47rdT2tjc2t4p71b29g8Oj6rHJx0tM0WhTSWXqhcRDZwJaBtmOPRSBSSJOHSjye283n0EpZkUD2aaQpiQkWAxo8RYK6hr9gTYc/2ry0G15jbchfA6eAXUUKHWoPrVH0qaJSAM5UTrwHNTE+ZEGUY5zCr9TENK6ISMILAoSAI6zBcrz/CFdYY4lso+YfDC/T2Rk0TraRLZzoSYsV6tzc3/akFm4pswZyLNDAi6/CjOODYSz+/HQ6aAGj61QKhidldMx0QRamxKFRuCt3ryOnT8hmf53q81/SKOMjpD56iOPHSNmugOtVAbUSTRM3pFb45xXpx352PZWnKKmVP0R87nDyX2j8k=</latexit><latexit sha1_base64="U/mAFdslXZJ3xroAkY2SP2rzxW4=">AAAB8nicbZDLSgMxFIYz9VbrrerSTbAIdVNmBkGXBTcuK9gLTIeSSc+0oZlkSDJCHfoYblwo4tancefbmLaz0NYfAh//OYec80cpZ9q47rdT2tjc2t4p71b29g8Oj6rHJx0tM0WhTSWXqhcRDZwJaBtmOPRSBSSJOHSjye283n0EpZkUD2aaQpiQkWAxo8RYK6hr9gTYc/2ry0G15jbchfA6eAXUUKHWoPrVH0qaJSAM5UTrwHNTE+ZEGUY5zCr9TENK6ISMILAoSAI6zBcrz/CFdYY4lso+YfDC/T2Rk0TraRLZzoSYsV6tzc3/akFm4pswZyLNDAi6/CjOODYSz+/HQ6aAGj61QKhidldMx0QRamxKFRuCt3ryOnT8hmf53q81/SKOMjpD56iOPHSNmugOtVAbUSTRM3pFb45xXpx352PZWnKKmVP0R87nDyX2j8k=</latexit>

(size 1024)<latexit sha1_base64="U/mAFdslXZJ3xroAkY2SP2rzxW4=">AAAB8nicbZDLSgMxFIYz9VbrrerSTbAIdVNmBkGXBTcuK9gLTIeSSc+0oZlkSDJCHfoYblwo4tancefbmLaz0NYfAh//OYec80cpZ9q47rdT2tjc2t4p71b29g8Oj6rHJx0tM0WhTSWXqhcRDZwJaBtmOPRSBSSJOHSjye283n0EpZkUD2aaQpiQkWAxo8RYK6hr9gTYc/2ry0G15jbchfA6eAXUUKHWoPrVH0qaJSAM5UTrwHNTE+ZEGUY5zCr9TENK6ISMILAoSAI6zBcrz/CFdYY4lso+YfDC/T2Rk0TraRLZzoSYsV6tzc3/akFm4pswZyLNDAi6/CjOODYSz+/HQ6aAGj61QKhidldMx0QRamxKFRuCt3ryOnT8hmf53q81/SKOMjpD56iOPHSNmugOtVAbUSTRM3pFb45xXpx352PZWnKKmVP0R87nDyX2j8k=</latexit><latexit sha1_base64="U/mAFdslXZJ3xroAkY2SP2rzxW4=">AAAB8nicbZDLSgMxFIYz9VbrrerSTbAIdVNmBkGXBTcuK9gLTIeSSc+0oZlkSDJCHfoYblwo4tancefbmLaz0NYfAh//OYec80cpZ9q47rdT2tjc2t4p71b29g8Oj6rHJx0tM0WhTSWXqhcRDZwJaBtmOPRSBSSJOHSjye283n0EpZkUD2aaQpiQkWAxo8RYK6hr9gTYc/2ry0G15jbchfA6eAXUUKHWoPrVH0qaJSAM5UTrwHNTE+ZEGUY5zCr9TENK6ISMILAoSAI6zBcrz/CFdYY4lso+YfDC/T2Rk0TraRLZzoSYsV6tzc3/akFm4pswZyLNDAi6/CjOODYSz+/HQ6aAGj61QKhidldMx0QRamxKFRuCt3ryOnT8hmf53q81/SKOMjpD56iOPHSNmugOtVAbUSTRM3pFb45xXpx352PZWnKKmVP0R87nDyX2j8k=</latexit><latexit sha1_base64="U/mAFdslXZJ3xroAkY2SP2rzxW4=">AAAB8nicbZDLSgMxFIYz9VbrrerSTbAIdVNmBkGXBTcuK9gLTIeSSc+0oZlkSDJCHfoYblwo4tancefbmLaz0NYfAh//OYec80cpZ9q47rdT2tjc2t4p71b29g8Oj6rHJx0tM0WhTSWXqhcRDZwJaBtmOPRSBSSJOHSjye283n0EpZkUD2aaQpiQkWAxo8RYK6hr9gTYc/2ry0G15jbchfA6eAXUUKHWoPrVH0qaJSAM5UTrwHNTE+ZEGUY5zCr9TENK6ISMILAoSAI6zBcrz/CFdYY4lso+YfDC/T2Rk0TraRLZzoSYsV6tzc3/akFm4pswZyLNDAi6/CjOODYSz+/HQ6aAGj61QKhidldMx0QRamxKFRuCt3ryOnT8hmf53q81/SKOMjpD56iOPHSNmugOtVAbUSTRM3pFb45xXpx352PZWnKKmVP0R87nDyX2j8k=</latexit><latexit sha1_base64="U/mAFdslXZJ3xroAkY2SP2rzxW4=">AAAB8nicbZDLSgMxFIYz9VbrrerSTbAIdVNmBkGXBTcuK9gLTIeSSc+0oZlkSDJCHfoYblwo4tancefbmLaz0NYfAh//OYec80cpZ9q47rdT2tjc2t4p71b29g8Oj6rHJx0tM0WhTSWXqhcRDZwJaBtmOPRSBSSJOHSjye283n0EpZkUD2aaQpiQkWAxo8RYK6hr9gTYc/2ry0G15jbchfA6eAXUUKHWoPrVH0qaJSAM5UTrwHNTE+ZEGUY5zCr9TENK6ISMILAoSAI6zBcrz/CFdYY4lso+YfDC/T2Rk0TraRLZzoSYsV6tzc3/akFm4pswZyLNDAi6/CjOODYSz+/HQ6aAGj61QKhidldMx0QRamxKFRuCt3ryOnT8hmf53q81/SKOMjpD56iOPHSNmugOtVAbUSTRM3pFb45xXpx352PZWnKKmVP0R87nDyX2j8k=</latexit>

action embedding<latexit sha1_base64="TtlC6Vl58zNpjsTqyq+PHy6LPKc=">AAAB+XicbZBNS8NAEIYnftb6FfXoZbEInkrSix4LXjxWsB/QhrLZTNulm03Y3RRK6D/x4kERr/4Tb/4bN20O2vrCwsM7M8zsG6aCa+N5387W9s7u3n7loHp4dHxy6p6dd3SSKYZtlohE9UKqUXCJbcONwF6qkMahwG44vS/q3RkqzRP5ZOYpBjEdSz7ijBprDV2XsgIIxiFGEZfjoVvz6t5SZBP8EmpQqjV0vwZRwrIYpWGCat33vdQEOVWGM4GL6iDTmFI2pWPsW5Q0Rh3ky8sX5No6ERklyj5pyNL9PZHTWOt5HNrOmJqJXq8V5n+1fmZGd0HOZZoZlGy1aJQJYhJSxEAirpAZMbdAmeL2VsImVNk0bFhVG4K//uVN6DTqvuXHRq3ZKOOowCVcwQ34cAtNeIAWtIHBDJ7hFd6c3Hlx3p2PVeuWU85cwB85nz9wk5N2</latexit><latexit sha1_base64="TtlC6Vl58zNpjsTqyq+PHy6LPKc=">AAAB+XicbZBNS8NAEIYnftb6FfXoZbEInkrSix4LXjxWsB/QhrLZTNulm03Y3RRK6D/x4kERr/4Tb/4bN20O2vrCwsM7M8zsG6aCa+N5387W9s7u3n7loHp4dHxy6p6dd3SSKYZtlohE9UKqUXCJbcONwF6qkMahwG44vS/q3RkqzRP5ZOYpBjEdSz7ijBprDV2XsgIIxiFGEZfjoVvz6t5SZBP8EmpQqjV0vwZRwrIYpWGCat33vdQEOVWGM4GL6iDTmFI2pWPsW5Q0Rh3ky8sX5No6ERklyj5pyNL9PZHTWOt5HNrOmJqJXq8V5n+1fmZGd0HOZZoZlGy1aJQJYhJSxEAirpAZMbdAmeL2VsImVNk0bFhVG4K//uVN6DTqvuXHRq3ZKOOowCVcwQ34cAtNeIAWtIHBDJ7hFd6c3Hlx3p2PVeuWU85cwB85nz9wk5N2</latexit><latexit sha1_base64="TtlC6Vl58zNpjsTqyq+PHy6LPKc=">AAAB+XicbZBNS8NAEIYnftb6FfXoZbEInkrSix4LXjxWsB/QhrLZTNulm03Y3RRK6D/x4kERr/4Tb/4bN20O2vrCwsM7M8zsG6aCa+N5387W9s7u3n7loHp4dHxy6p6dd3SSKYZtlohE9UKqUXCJbcONwF6qkMahwG44vS/q3RkqzRP5ZOYpBjEdSz7ijBprDV2XsgIIxiFGEZfjoVvz6t5SZBP8EmpQqjV0vwZRwrIYpWGCat33vdQEOVWGM4GL6iDTmFI2pWPsW5Q0Rh3ky8sX5No6ERklyj5pyNL9PZHTWOt5HNrOmJqJXq8V5n+1fmZGd0HOZZoZlGy1aJQJYhJSxEAirpAZMbdAmeL2VsImVNk0bFhVG4K//uVN6DTqvuXHRq3ZKOOowCVcwQ34cAtNeIAWtIHBDJ7hFd6c3Hlx3p2PVeuWU85cwB85nz9wk5N2</latexit><latexit sha1_base64="TtlC6Vl58zNpjsTqyq+PHy6LPKc=">AAAB+XicbZBNS8NAEIYnftb6FfXoZbEInkrSix4LXjxWsB/QhrLZTNulm03Y3RRK6D/x4kERr/4Tb/4bN20O2vrCwsM7M8zsG6aCa+N5387W9s7u3n7loHp4dHxy6p6dd3SSKYZtlohE9UKqUXCJbcONwF6qkMahwG44vS/q3RkqzRP5ZOYpBjEdSz7ijBprDV2XsgIIxiFGEZfjoVvz6t5SZBP8EmpQqjV0vwZRwrIYpWGCat33vdQEOVWGM4GL6iDTmFI2pWPsW5Q0Rh3ky8sX5No6ERklyj5pyNL9PZHTWOt5HNrOmJqJXq8V5n+1fmZGd0HOZZoZlGy1aJQJYhJSxEAirpAZMbdAmeL2VsImVNk0bFhVG4K//uVN6DTqvuXHRq3ZKOOowCVcwQ34cAtNeIAWtIHBDJ7hFd6c3Hlx3p2PVeuWU85cwB85nz9wk5N2</latexit>

concatenate

…LSTM LSTM

… … …

state si<latexit sha1_base64="vbHNn31ZZ33sVPiQnudHXPNWuQU=">AAAB83icbZA9TwJBEIbn8AvxC7W02QgmVuSORu1IbCwxETCBC9lb5mDD3kd250wI4W/YWGiMrX/Gzn/jAlco+CabPHlnJjP7BqmShlz32ylsbG5t7xR3S3v7B4dH5eOTtkkyLbAlEpXox4AbVDLGFklS+Jhq5FGgsBOMb+f1zhNqI5P4gSYp+hEfxjKUgpO1eoY4Iauavqyyfrni1tyF2Dp4OVQgV7Nf/uoNEpFFGJNQ3Jiu56bkT7kmKRTOSr3MYMrFmA+xazHmERp/urh5xi6sM2Bhou2LiS3c3xNTHhkziQLbGXEamdXa3Pyv1s0ovPanMk4zwlgsF4WZYpSweQBsIDUKUhMLXGhpb2VixDUXZGMq2RC81S+vQ7te8yzf1yuNmzyOIpzBOVyCB1fQgDtoQgsEpPAMr/DmZM6L8+58LFsLTj5zCn/kfP4AwdGQyw==</latexit><latexit sha1_base64="vbHNn31ZZ33sVPiQnudHXPNWuQU=">AAAB83icbZA9TwJBEIbn8AvxC7W02QgmVuSORu1IbCwxETCBC9lb5mDD3kd250wI4W/YWGiMrX/Gzn/jAlco+CabPHlnJjP7BqmShlz32ylsbG5t7xR3S3v7B4dH5eOTtkkyLbAlEpXox4AbVDLGFklS+Jhq5FGgsBOMb+f1zhNqI5P4gSYp+hEfxjKUgpO1eoY4Iauavqyyfrni1tyF2Dp4OVQgV7Nf/uoNEpFFGJNQ3Jiu56bkT7kmKRTOSr3MYMrFmA+xazHmERp/urh5xi6sM2Bhou2LiS3c3xNTHhkziQLbGXEamdXa3Pyv1s0ovPanMk4zwlgsF4WZYpSweQBsIDUKUhMLXGhpb2VixDUXZGMq2RC81S+vQ7te8yzf1yuNmzyOIpzBOVyCB1fQgDtoQgsEpPAMr/DmZM6L8+58LFsLTj5zCn/kfP4AwdGQyw==</latexit><latexit sha1_base64="vbHNn31ZZ33sVPiQnudHXPNWuQU=">AAAB83icbZA9TwJBEIbn8AvxC7W02QgmVuSORu1IbCwxETCBC9lb5mDD3kd250wI4W/YWGiMrX/Gzn/jAlco+CabPHlnJjP7BqmShlz32ylsbG5t7xR3S3v7B4dH5eOTtkkyLbAlEpXox4AbVDLGFklS+Jhq5FGgsBOMb+f1zhNqI5P4gSYp+hEfxjKUgpO1eoY4Iauavqyyfrni1tyF2Dp4OVQgV7Nf/uoNEpFFGJNQ3Jiu56bkT7kmKRTOSr3MYMrFmA+xazHmERp/urh5xi6sM2Bhou2LiS3c3xNTHhkziQLbGXEamdXa3Pyv1s0ovPanMk4zwlgsF4WZYpSweQBsIDUKUhMLXGhpb2VixDUXZGMq2RC81S+vQ7te8yzf1yuNmzyOIpzBOVyCB1fQgDtoQgsEpPAMr/DmZM6L8+58LFsLTj5zCn/kfP4AwdGQyw==</latexit><latexit sha1_base64="vbHNn31ZZ33sVPiQnudHXPNWuQU=">AAAB83icbZA9TwJBEIbn8AvxC7W02QgmVuSORu1IbCwxETCBC9lb5mDD3kd250wI4W/YWGiMrX/Gzn/jAlco+CabPHlnJjP7BqmShlz32ylsbG5t7xR3S3v7B4dH5eOTtkkyLbAlEpXox4AbVDLGFklS+Jhq5FGgsBOMb+f1zhNqI5P4gSYp+hEfxjKUgpO1eoY4Iauavqyyfrni1tyF2Dp4OVQgV7Nf/uoNEpFFGJNQ3Jiu56bkT7kmKRTOSr3MYMrFmA+xazHmERp/urh5xi6sM2Bhou2LiS3c3xNTHhkziQLbGXEamdXa3Pyv1s0ovPanMk4zwlgsF4WZYpSweQBsIDUKUhMLXGhpb2VixDUXZGMq2RC81S+vQ7te8yzf1yuNmzyOIpzBOVyCB1fQgDtoQgsEpPAMr/DmZM6L8+58LFsLTj5zCn/kfP4AwdGQyw==</latexit>

action ai<latexit sha1_base64="L6dz62kMJzbmmyoLciPLPwrRceA=">AAAB9HicbZDNTgIxFIXv4B/iH+rSTSOYuCIzbNQdiRuXmAiYwITcKR1o6HTGtkNCJjyHGxca49aHcefb2AEWCt6k6Zdz7k1vT5AIro3rfjuFjc2t7Z3ibmlv/+DwqHx80tZxqihr0VjE6jFAzQSXrGW4EewxUQyjQLBOML7N/c6EKc1j+WCmCfMjHEoecorGSj7S/CJV7PMq6Zcrbs2dF1kHbwkVWFazX/7qDWKaRkwaKlDrrucmxs9QGU4Fm5V6qWYJ0jEOWdeixIhpP5svPSMXVhmQMFb2SEPm6u+JDCOtp1FgOyM0I73q5eJ/Xjc14bWfcZmkhkm6eChMBTExyRMgA64YNWJqAanidldCR6hsFDankg3BW/3yOrTrNc/yfb3SuFnGUYQzOIdL8OAKGnAHTWgBhSd4hld4cybOi/PufCxaC85y5hT+lPP5A1x5kSA=</latexit><latexit sha1_base64="L6dz62kMJzbmmyoLciPLPwrRceA=">AAAB9HicbZDNTgIxFIXv4B/iH+rSTSOYuCIzbNQdiRuXmAiYwITcKR1o6HTGtkNCJjyHGxca49aHcefb2AEWCt6k6Zdz7k1vT5AIro3rfjuFjc2t7Z3ibmlv/+DwqHx80tZxqihr0VjE6jFAzQSXrGW4EewxUQyjQLBOML7N/c6EKc1j+WCmCfMjHEoecorGSj7S/CJV7PMq6Zcrbs2dF1kHbwkVWFazX/7qDWKaRkwaKlDrrucmxs9QGU4Fm5V6qWYJ0jEOWdeixIhpP5svPSMXVhmQMFb2SEPm6u+JDCOtp1FgOyM0I73q5eJ/Xjc14bWfcZmkhkm6eChMBTExyRMgA64YNWJqAanidldCR6hsFDankg3BW/3yOrTrNc/yfb3SuFnGUYQzOIdL8OAKGnAHTWgBhSd4hld4cybOi/PufCxaC85y5hT+lPP5A1x5kSA=</latexit><latexit sha1_base64="L6dz62kMJzbmmyoLciPLPwrRceA=">AAAB9HicbZDNTgIxFIXv4B/iH+rSTSOYuCIzbNQdiRuXmAiYwITcKR1o6HTGtkNCJjyHGxca49aHcefb2AEWCt6k6Zdz7k1vT5AIro3rfjuFjc2t7Z3ibmlv/+DwqHx80tZxqihr0VjE6jFAzQSXrGW4EewxUQyjQLBOML7N/c6EKc1j+WCmCfMjHEoecorGSj7S/CJV7PMq6Zcrbs2dF1kHbwkVWFazX/7qDWKaRkwaKlDrrucmxs9QGU4Fm5V6qWYJ0jEOWdeixIhpP5svPSMXVhmQMFb2SEPm6u+JDCOtp1FgOyM0I73q5eJ/Xjc14bWfcZmkhkm6eChMBTExyRMgA64YNWJqAanidldCR6hsFDankg3BW/3yOrTrNc/yfb3SuFnGUYQzOIdL8OAKGnAHTWgBhSd4hld4cybOi/PufCxaC85y5hT+lPP5A1x5kSA=</latexit><latexit sha1_base64="L6dz62kMJzbmmyoLciPLPwrRceA=">AAAB9HicbZDNTgIxFIXv4B/iH+rSTSOYuCIzbNQdiRuXmAiYwITcKR1o6HTGtkNCJjyHGxca49aHcefb2AEWCt6k6Zdz7k1vT5AIro3rfjuFjc2t7Z3ibmlv/+DwqHx80tZxqihr0VjE6jFAzQSXrGW4EewxUQyjQLBOML7N/c6EKc1j+WCmCfMjHEoecorGSj7S/CJV7PMq6Zcrbs2dF1kHbwkVWFazX/7qDWKaRkwaKlDrrucmxs9QGU4Fm5V6qWYJ0jEOWdeixIhpP5svPSMXVhmQMFb2SEPm6u+JDCOtp1FgOyM0I73q5eJ/Xjc14bWfcZmkhkm6eChMBTExyRMgA64YNWJqAanidldCR6hsFDankg3BW/3yOrTrNc/yfb3SuFnGUYQzOIdL8OAKGnAHTWgBhSd4hld4cybOi/PufCxaC85y5hT+lPP5A1x5kSA=</latexit>

Dw(si, ai)<latexit sha1_base64="zDTrwCBl0Q366h3rzXHCQdGpgmc=">AAAB9HicbZDLSsNAFIZP6q3WW9Slm8EiVJCSiKDuCrpwWcFeoA1hMp20QyeTODOplNDncONCEbc+jDvfxmmbhbb+MPDxn3M4Z/4g4Uxpx/m2Ciura+sbxc3S1vbO7p69f9BUcSoJbZCYx7IdYEU5E7Shmea0nUiKo4DTVjC8mdZbIyoVi8WDHifUi3BfsJARrI3l3fpPFeWzM4R9durbZafqzISWwc2hDLnqvv3V7cUkjajQhGOlOq6TaC/DUjPC6aTUTRVNMBniPu0YFDiiystmR0/QiXF6KIyleUKjmft7IsORUuMoMJ0R1gO1WJua/9U6qQ6vvIyJJNVUkPmiMOVIx2iaAOoxSYnmYwOYSGZuRWSAJSba5FQyIbiLX16G5nnVNXx/Ua5d53EU4QiOoQIuXEIN7qAODSDwCM/wCm/WyHqx3q2PeWvBymcO4Y+szx873pEO</latexit><latexit sha1_base64="zDTrwCBl0Q366h3rzXHCQdGpgmc=">AAAB9HicbZDLSsNAFIZP6q3WW9Slm8EiVJCSiKDuCrpwWcFeoA1hMp20QyeTODOplNDncONCEbc+jDvfxmmbhbb+MPDxn3M4Z/4g4Uxpx/m2Ciura+sbxc3S1vbO7p69f9BUcSoJbZCYx7IdYEU5E7Shmea0nUiKo4DTVjC8mdZbIyoVi8WDHifUi3BfsJARrI3l3fpPFeWzM4R9durbZafqzISWwc2hDLnqvv3V7cUkjajQhGOlOq6TaC/DUjPC6aTUTRVNMBniPu0YFDiiystmR0/QiXF6KIyleUKjmft7IsORUuMoMJ0R1gO1WJua/9U6qQ6vvIyJJNVUkPmiMOVIx2iaAOoxSYnmYwOYSGZuRWSAJSba5FQyIbiLX16G5nnVNXx/Ua5d53EU4QiOoQIuXEIN7qAODSDwCM/wCm/WyHqx3q2PeWvBymcO4Y+szx873pEO</latexit><latexit sha1_base64="zDTrwCBl0Q366h3rzXHCQdGpgmc=">AAAB9HicbZDLSsNAFIZP6q3WW9Slm8EiVJCSiKDuCrpwWcFeoA1hMp20QyeTODOplNDncONCEbc+jDvfxmmbhbb+MPDxn3M4Z/4g4Uxpx/m2Ciura+sbxc3S1vbO7p69f9BUcSoJbZCYx7IdYEU5E7Shmea0nUiKo4DTVjC8mdZbIyoVi8WDHifUi3BfsJARrI3l3fpPFeWzM4R9durbZafqzISWwc2hDLnqvv3V7cUkjajQhGOlOq6TaC/DUjPC6aTUTRVNMBniPu0YFDiiystmR0/QiXF6KIyleUKjmft7IsORUuMoMJ0R1gO1WJua/9U6qQ6vvIyJJNVUkPmiMOVIx2iaAOoxSYnmYwOYSGZuRWSAJSba5FQyIbiLX16G5nnVNXx/Ua5d53EU4QiOoQIuXEIN7qAODSDwCM/wCm/WyHqx3q2PeWvBymcO4Y+szx873pEO</latexit><latexit sha1_base64="zDTrwCBl0Q366h3rzXHCQdGpgmc=">AAAB9HicbZDLSsNAFIZP6q3WW9Slm8EiVJCSiKDuCrpwWcFeoA1hMp20QyeTODOplNDncONCEbc+jDvfxmmbhbb+MPDxn3M4Z/4g4Uxpx/m2Ciura+sbxc3S1vbO7p69f9BUcSoJbZCYx7IdYEU5E7Shmea0nUiKo4DTVjC8mdZbIyoVi8WDHifUi3BfsJARrI3l3fpPFeWzM4R9durbZafqzISWwc2hDLnqvv3V7cUkjajQhGOlOq6TaC/DUjPC6aTUTRVNMBniPu0YFDiiystmR0/QiXF6KIyleUKjmft7IsORUuMoMJ0R1gO1WJua/9U6qQ6vvIyJJNVUkPmiMOVIx2iaAOoxSYnmYwOYSGZuRWSAJSba5FQyIbiLX16G5nnVNXx/Ua5d53EU4QiOoQIuXEIN7qAODSDwCM/wCm/WyHqx3q2PeWvBymcO4Y+szx873pEO</latexit>

7 fully-connected layers, with Leaky ReLU activation

xt+(i�1)m<latexit sha1_base64="nodAMytCFncG2WkuhXH+1cl/pus=">AAAB83icbZBNSwMxEIZn61etX1WPXhaLUBHLrgj1WPDisYL9gHYp2TRtQ5PsksyKZenf8OJBEa/+GW/+G9N2D9r6QuDhnRlm8oax4AY979vJra1vbG7ltws7u3v7B8XDo6aJEk1Zg0Yi0u2QGCa4Yg3kKFg71ozIULBWOL6d1VuPTBseqQecxCyQZKj4gFOC1uo+9VK8KPNL/1xOe8WSV/HmclfBz6AEmeq94le3H9FEMoVUEGM6vhdjkBKNnAo2LXQTw2JCx2TIOhYVkcwE6fzmqXtmnb47iLR9Ct25+3siJdKYiQxtpyQ4Msu1mflfrZPg4CZIuYoTZIouFg0S4WLkzgJw+1wzimJigVDN7a0uHRFNKNqYCjYEf/nLq9C8qviW769LtWoWRx5O4BTK4EMVanAHdWgAhRie4RXenMR5cd6dj0VrzslmjuGPnM8f1vyQ2g==</latexit><latexit sha1_base64="nodAMytCFncG2WkuhXH+1cl/pus=">AAAB83icbZBNSwMxEIZn61etX1WPXhaLUBHLrgj1WPDisYL9gHYp2TRtQ5PsksyKZenf8OJBEa/+GW/+G9N2D9r6QuDhnRlm8oax4AY979vJra1vbG7ltws7u3v7B8XDo6aJEk1Zg0Yi0u2QGCa4Yg3kKFg71ozIULBWOL6d1VuPTBseqQecxCyQZKj4gFOC1uo+9VK8KPNL/1xOe8WSV/HmclfBz6AEmeq94le3H9FEMoVUEGM6vhdjkBKNnAo2LXQTw2JCx2TIOhYVkcwE6fzmqXtmnb47iLR9Ct25+3siJdKYiQxtpyQ4Msu1mflfrZPg4CZIuYoTZIouFg0S4WLkzgJw+1wzimJigVDN7a0uHRFNKNqYCjYEf/nLq9C8qviW769LtWoWRx5O4BTK4EMVanAHdWgAhRie4RXenMR5cd6dj0VrzslmjuGPnM8f1vyQ2g==</latexit><latexit sha1_base64="nodAMytCFncG2WkuhXH+1cl/pus=">AAAB83icbZBNSwMxEIZn61etX1WPXhaLUBHLrgj1WPDisYL9gHYp2TRtQ5PsksyKZenf8OJBEa/+GW/+G9N2D9r6QuDhnRlm8oax4AY979vJra1vbG7ltws7u3v7B8XDo6aJEk1Zg0Yi0u2QGCa4Yg3kKFg71ozIULBWOL6d1VuPTBseqQecxCyQZKj4gFOC1uo+9VK8KPNL/1xOe8WSV/HmclfBz6AEmeq94le3H9FEMoVUEGM6vhdjkBKNnAo2LXQTw2JCx2TIOhYVkcwE6fzmqXtmnb47iLR9Ct25+3siJdKYiQxtpyQ4Msu1mflfrZPg4CZIuYoTZIouFg0S4WLkzgJw+1wzimJigVDN7a0uHRFNKNqYCjYEf/nLq9C8qviW769LtWoWRx5O4BTK4EMVanAHdWgAhRie4RXenMR5cd6dj0VrzslmjuGPnM8f1vyQ2g==</latexit><latexit sha1_base64="nodAMytCFncG2WkuhXH+1cl/pus=">AAAB83icbZBNSwMxEIZn61etX1WPXhaLUBHLrgj1WPDisYL9gHYp2TRtQ5PsksyKZenf8OJBEa/+GW/+G9N2D9r6QuDhnRlm8oax4AY979vJra1vbG7ltws7u3v7B8XDo6aJEk1Zg0Yi0u2QGCa4Yg3kKFg71ozIULBWOL6d1VuPTBseqQecxCyQZKj4gFOC1uo+9VK8KPNL/1xOe8WSV/HmclfBz6AEmeq94le3H9FEMoVUEGM6vhdjkBKNnAo2LXQTw2JCx2TIOhYVkcwE6fzmqXtmnb47iLR9Ct25+3siJdKYiQxtpyQ4Msu1mflfrZPg4CZIuYoTZIouFg0S4WLkzgJw+1wzimJigVDN7a0uHRFNKNqYCjYEf/nLq9C8qviW769LtWoWRx5O4BTK4EMVanAHdWgAhRie4RXenMR5cd6dj0VrzslmjuGPnM8f1vyQ2g==</latexit>

xt+1<latexit sha1_base64="WKwVmGxB+fiWINGXCkwCFX6rjLM=">AAAB7nicbZBNS8NAEIYn9avWr6pHL4tFEISSSEHBS8GLxwr2A9pQNttNu3SzCbsTsYT+CC8eFPHq7/Hmv3Hb5qCtLyw8vDPDzrxBIoVB1/12CmvrG5tbxe3Szu7e/kH58Khl4lQz3mSxjHUnoIZLoXgTBUreSTSnUSB5OxjfzurtR66NiNUDThLuR3SoRCgYRWu1n/oZXnjTfrniVt25yCp4OVQgV6Nf/uoNYpZGXCGT1Jiu5yboZ1SjYJJPS73U8ISyMR3yrkVFI278bL7ulJxZZ0DCWNunkMzd3xMZjYyZRIHtjCiOzHJtZv5X66YYXvuZUEmKXLHFR2EqCcZkdjsZCM0ZyokFyrSwuxI2opoytAmVbAje8smr0Lqsepbva5X6TR5HEU7gFM7Bgyuowx00oAkMxvAMr/DmJM6L8+58LFoLTj5zDH/kfP4ADXePWA==</latexit><latexit sha1_base64="WKwVmGxB+fiWINGXCkwCFX6rjLM=">AAAB7nicbZBNS8NAEIYn9avWr6pHL4tFEISSSEHBS8GLxwr2A9pQNttNu3SzCbsTsYT+CC8eFPHq7/Hmv3Hb5qCtLyw8vDPDzrxBIoVB1/12CmvrG5tbxe3Szu7e/kH58Khl4lQz3mSxjHUnoIZLoXgTBUreSTSnUSB5OxjfzurtR66NiNUDThLuR3SoRCgYRWu1n/oZXnjTfrniVt25yCp4OVQgV6Nf/uoNYpZGXCGT1Jiu5yboZ1SjYJJPS73U8ISyMR3yrkVFI278bL7ulJxZZ0DCWNunkMzd3xMZjYyZRIHtjCiOzHJtZv5X66YYXvuZUEmKXLHFR2EqCcZkdjsZCM0ZyokFyrSwuxI2opoytAmVbAje8smr0Lqsepbva5X6TR5HEU7gFM7Bgyuowx00oAkMxvAMr/DmJM6L8+58LFoLTj5zDH/kfP4ADXePWA==</latexit><latexit sha1_base64="WKwVmGxB+fiWINGXCkwCFX6rjLM=">AAAB7nicbZBNS8NAEIYn9avWr6pHL4tFEISSSEHBS8GLxwr2A9pQNttNu3SzCbsTsYT+CC8eFPHq7/Hmv3Hb5qCtLyw8vDPDzrxBIoVB1/12CmvrG5tbxe3Szu7e/kH58Khl4lQz3mSxjHUnoIZLoXgTBUreSTSnUSB5OxjfzurtR66NiNUDThLuR3SoRCgYRWu1n/oZXnjTfrniVt25yCp4OVQgV6Nf/uoNYpZGXCGT1Jiu5yboZ1SjYJJPS73U8ISyMR3yrkVFI278bL7ulJxZZ0DCWNunkMzd3xMZjYyZRIHtjCiOzHJtZv5X66YYXvuZUEmKXLHFR2EqCcZkdjsZCM0ZyokFyrSwuxI2opoytAmVbAje8smr0Lqsepbva5X6TR5HEU7gFM7Bgyuowx00oAkMxvAMr/DmJM6L8+58LFoLTj5zDH/kfP4ADXePWA==</latexit><latexit sha1_base64="WKwVmGxB+fiWINGXCkwCFX6rjLM=">AAAB7nicbZBNS8NAEIYn9avWr6pHL4tFEISSSEHBS8GLxwr2A9pQNttNu3SzCbsTsYT+CC8eFPHq7/Hmv3Hb5qCtLyw8vDPDzrxBIoVB1/12CmvrG5tbxe3Szu7e/kH58Khl4lQz3mSxjHUnoIZLoXgTBUreSTSnUSB5OxjfzurtR66NiNUDThLuR3SoRCgYRWu1n/oZXnjTfrniVt25yCp4OVQgV6Nf/uoNYpZGXCGT1Jiu5yboZ1SjYJJPS73U8ISyMR3yrkVFI278bL7ulJxZZ0DCWNunkMzd3xMZjYyZRIHtjCiOzHJtZv5X66YYXvuZUEmKXLHFR2EqCcZkdjsZCM0ZyokFyrSwuxI2opoytAmVbAje8smr0Lqsepbva5X6TR5HEU7gFM7Bgyuowx00oAkMxvAMr/DmJM6L8+58LFoLTj5zDH/kfP4ADXePWA==</latexit>

xt<latexit sha1_base64="oUC4v0BJI9jvdKlaCyKW2ur5V6s=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSERS8FLx4rGDaQhvKZrtpl242YXciltDf4MWDIl79Qd78N27bHLT1hYWHd2bYmTdMpTDout9OaW19Y3OrvF3Z2d3bP6geHrVMkmnGfZbIRHdCargUivsoUPJOqjmNQ8nb4fh2Vm8/cm1Eoh5wkvIgpkMlIsEoWst/6uc47Vdrbt2di6yCV0ANCjX71a/eIGFZzBUySY3pem6KQU41Cib5tNLLDE8pG9Mh71pUNOYmyOfLTsmZdQYkSrR9Csnc/T2R09iYSRzazpjiyCzXZuZ/tW6G0XWQC5VmyBVbfBRlkmBCZpeTgdCcoZxYoEwLuythI6opQ5tPxYbgLZ+8Cq2Lumf5/rLWuCniKMMJnMI5eHAFDbiDJvjAQMAzvMKbo5wX5935WLSWnGLmGP7I+fwBNAuO6A==</latexit><latexit sha1_base64="oUC4v0BJI9jvdKlaCyKW2ur5V6s=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSERS8FLx4rGDaQhvKZrtpl242YXciltDf4MWDIl79Qd78N27bHLT1hYWHd2bYmTdMpTDout9OaW19Y3OrvF3Z2d3bP6geHrVMkmnGfZbIRHdCargUivsoUPJOqjmNQ8nb4fh2Vm8/cm1Eoh5wkvIgpkMlIsEoWst/6uc47Vdrbt2di6yCV0ANCjX71a/eIGFZzBUySY3pem6KQU41Cib5tNLLDE8pG9Mh71pUNOYmyOfLTsmZdQYkSrR9Csnc/T2R09iYSRzazpjiyCzXZuZ/tW6G0XWQC5VmyBVbfBRlkmBCZpeTgdCcoZxYoEwLuythI6opQ5tPxYbgLZ+8Cq2Lumf5/rLWuCniKMMJnMI5eHAFDbiDJvjAQMAzvMKbo5wX5935WLSWnGLmGP7I+fwBNAuO6A==</latexit><latexit sha1_base64="oUC4v0BJI9jvdKlaCyKW2ur5V6s=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSERS8FLx4rGDaQhvKZrtpl242YXciltDf4MWDIl79Qd78N27bHLT1hYWHd2bYmTdMpTDout9OaW19Y3OrvF3Z2d3bP6geHrVMkmnGfZbIRHdCargUivsoUPJOqjmNQ8nb4fh2Vm8/cm1Eoh5wkvIgpkMlIsEoWst/6uc47Vdrbt2di6yCV0ANCjX71a/eIGFZzBUySY3pem6KQU41Cib5tNLLDE8pG9Mh71pUNOYmyOfLTsmZdQYkSrR9Csnc/T2R09iYSRzazpjiyCzXZuZ/tW6G0XWQC5VmyBVbfBRlkmBCZpeTgdCcoZxYoEwLuythI6opQ5tPxYbgLZ+8Cq2Lumf5/rLWuCniKMMJnMI5eHAFDbiDJvjAQMAzvMKbo5wX5935WLSWnGLmGP7I+fwBNAuO6A==</latexit><latexit sha1_base64="oUC4v0BJI9jvdKlaCyKW2ur5V6s=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSERS8FLx4rGDaQhvKZrtpl242YXciltDf4MWDIl79Qd78N27bHLT1hYWHd2bYmTdMpTDout9OaW19Y3OrvF3Z2d3bP6geHrVMkmnGfZbIRHdCargUivsoUPJOqjmNQ8nb4fh2Vm8/cm1Eoh5wkvIgpkMlIsEoWst/6uc47Vdrbt2di6yCV0ANCjX71a/eIGFZzBUySY3pem6KQU41Cib5tNLLDE8pG9Mh71pUNOYmyOfLTsmZdQYkSrR9Csnc/T2R09iYSRzazpjiyCzXZuZ/tW6G0XWQC5VmyBVbfBRlkmBCZpeTgdCcoZxYoEwLuythI6opQ5tPxYbgLZ+8Cq2Lumf5/rLWuCniKMMJnMI5eHAFDbiDJvjAQMAzvMKbo5wX5935WLSWnGLmGP7I+fwBNAuO6A==</latexit>

x1<latexit sha1_base64="JUkHY7vHU848XCTuZdzT8juo0ys=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSERS8FLx4rGDaQhvKZjtpl242YXcjltDf4MWDIl79Qd78N27bHLT1hYWHd2bYmTdMBdfGdb+d0tr6xuZWebuys7u3f1A9PGrpJFMMfZaIRHVCqlFwib7hRmAnVUjjUGA7HN/O6u1HVJon8sFMUgxiOpQ84owaa/lP/dyb9qs1t+7ORVbBK6AGhZr96ldvkLAsRmmYoFp3PTc1QU6V4UzgtNLLNKaUjekQuxYljVEH+XzZKTmzzoBEibJPGjJ3f0/kNNZ6Eoe2M6ZmpJdrM/O/Wjcz0XWQc5lmBiVbfBRlgpiEzC4nA66QGTGxQJnidlfCRlRRZmw+FRuCt3zyKrQu6p7l+8ta46aIowwncArn4MEVNOAOmuADAw7P8ApvjnRenHfnY9FacoqZY/gj5/MHzi2OpQ==</latexit><latexit sha1_base64="JUkHY7vHU848XCTuZdzT8juo0ys=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSERS8FLx4rGDaQhvKZjtpl242YXcjltDf4MWDIl79Qd78N27bHLT1hYWHd2bYmTdMBdfGdb+d0tr6xuZWebuys7u3f1A9PGrpJFMMfZaIRHVCqlFwib7hRmAnVUjjUGA7HN/O6u1HVJon8sFMUgxiOpQ84owaa/lP/dyb9qs1t+7ORVbBK6AGhZr96ldvkLAsRmmYoFp3PTc1QU6V4UzgtNLLNKaUjekQuxYljVEH+XzZKTmzzoBEibJPGjJ3f0/kNNZ6Eoe2M6ZmpJdrM/O/Wjcz0XWQc5lmBiVbfBRlgpiEzC4nA66QGTGxQJnidlfCRlRRZmw+FRuCt3zyKrQu6p7l+8ta46aIowwncArn4MEVNOAOmuADAw7P8ApvjnRenHfnY9FacoqZY/gj5/MHzi2OpQ==</latexit><latexit sha1_base64="JUkHY7vHU848XCTuZdzT8juo0ys=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSERS8FLx4rGDaQhvKZjtpl242YXcjltDf4MWDIl79Qd78N27bHLT1hYWHd2bYmTdMBdfGdb+d0tr6xuZWebuys7u3f1A9PGrpJFMMfZaIRHVCqlFwib7hRmAnVUjjUGA7HN/O6u1HVJon8sFMUgxiOpQ84owaa/lP/dyb9qs1t+7ORVbBK6AGhZr96ldvkLAsRmmYoFp3PTc1QU6V4UzgtNLLNKaUjekQuxYljVEH+XzZKTmzzoBEibJPGjJ3f0/kNNZ6Eoe2M6ZmpJdrM/O/Wjcz0XWQc5lmBiVbfBRlgpiEzC4nA66QGTGxQJnidlfCRlRRZmw+FRuCt3zyKrQu6p7l+8ta46aIowwncArn4MEVNOAOmuADAw7P8ApvjnRenHfnY9FacoqZY/gj5/MHzi2OpQ==</latexit><latexit sha1_base64="JUkHY7vHU848XCTuZdzT8juo0ys=">AAAB7HicbZBNS8NAEIYn9avWr6pHL4tF8FQSERS8FLx4rGDaQhvKZjtpl242YXcjltDf4MWDIl79Qd78N27bHLT1hYWHd2bYmTdMBdfGdb+d0tr6xuZWebuys7u3f1A9PGrpJFMMfZaIRHVCqlFwib7hRmAnVUjjUGA7HN/O6u1HVJon8sFMUgxiOpQ84owaa/lP/dyb9qs1t+7ORVbBK6AGhZr96ldvkLAsRmmYoFp3PTc1QU6V4UzgtNLLNKaUjekQuxYljVEH+XzZKTmzzoBEibJPGjJ3f0/kNNZ6Eoe2M6ZmpJdrM/O/Wjcz0XWQc5lmBiVbfBRlgpiEzC4nA66QGTGxQJnidlfCRlRRZmw+FRuCt3zyKrQu6p7l+8ta46aIowwncArn4MEVNOAOmuADAw7P8ApvjnRenHfnY9FacoqZY/gj5/MHzi2OpQ==</latexit>

xt+(i�1)m+1<latexit sha1_base64="21uFzPqrGXxn9eLAdFgi3ox3+mU=">AAAB9XicbZDLSgMxFIbPeK31VnXpJliESrHMSEGXBTcuK9gLtGPJpJk2NMkMSUYtQ9/DjQtF3Pou7nwb03YW2vpD4OM/53BO/iDmTBvX/XZWVtfWNzZzW/ntnd29/cLBYVNHiSK0QSIeqXaANeVM0oZhhtN2rCgWAaetYHQ9rbceqNIskndmHFNf4IFkISPYWOv+qZeacomde2ei7E16haJbcWdCy+BlUIRM9V7hq9uPSCKoNIRjrTueGxs/xcowwukk3000jTEZ4QHtWJRYUO2ns6sn6NQ6fRRGyj5p0Mz9PZFiofVYBLZTYDPUi7Wp+V+tk5jwyk+ZjBNDJZkvChOOTISmEaA+U5QYPraAiWL2VkSGWGFibFB5G4K3+OVlaF5UPMu31WKtmsWRg2M4gRJ4cAk1uIE6NICAgmd4hTfn0Xlx3p2PeeuKk80cwR85nz+y/ZFH</latexit><latexit sha1_base64="21uFzPqrGXxn9eLAdFgi3ox3+mU=">AAAB9XicbZDLSgMxFIbPeK31VnXpJliESrHMSEGXBTcuK9gLtGPJpJk2NMkMSUYtQ9/DjQtF3Pou7nwb03YW2vpD4OM/53BO/iDmTBvX/XZWVtfWNzZzW/ntnd29/cLBYVNHiSK0QSIeqXaANeVM0oZhhtN2rCgWAaetYHQ9rbceqNIskndmHFNf4IFkISPYWOv+qZeacomde2ei7E16haJbcWdCy+BlUIRM9V7hq9uPSCKoNIRjrTueGxs/xcowwukk3000jTEZ4QHtWJRYUO2ns6sn6NQ6fRRGyj5p0Mz9PZFiofVYBLZTYDPUi7Wp+V+tk5jwyk+ZjBNDJZkvChOOTISmEaA+U5QYPraAiWL2VkSGWGFibFB5G4K3+OVlaF5UPMu31WKtmsWRg2M4gRJ4cAk1uIE6NICAgmd4hTfn0Xlx3p2PeeuKk80cwR85nz+y/ZFH</latexit><latexit sha1_base64="21uFzPqrGXxn9eLAdFgi3ox3+mU=">AAAB9XicbZDLSgMxFIbPeK31VnXpJliESrHMSEGXBTcuK9gLtGPJpJk2NMkMSUYtQ9/DjQtF3Pou7nwb03YW2vpD4OM/53BO/iDmTBvX/XZWVtfWNzZzW/ntnd29/cLBYVNHiSK0QSIeqXaANeVM0oZhhtN2rCgWAaetYHQ9rbceqNIskndmHFNf4IFkISPYWOv+qZeacomde2ei7E16haJbcWdCy+BlUIRM9V7hq9uPSCKoNIRjrTueGxs/xcowwukk3000jTEZ4QHtWJRYUO2ns6sn6NQ6fRRGyj5p0Mz9PZFiofVYBLZTYDPUi7Wp+V+tk5jwyk+ZjBNDJZkvChOOTISmEaA+U5QYPraAiWL2VkSGWGFibFB5G4K3+OVlaF5UPMu31WKtmsWRg2M4gRJ4cAk1uIE6NICAgmd4hTfn0Xlx3p2PeeuKk80cwR85nz+y/ZFH</latexit><latexit sha1_base64="21uFzPqrGXxn9eLAdFgi3ox3+mU=">AAAB9XicbZDLSgMxFIbPeK31VnXpJliESrHMSEGXBTcuK9gLtGPJpJk2NMkMSUYtQ9/DjQtF3Pou7nwb03YW2vpD4OM/53BO/iDmTBvX/XZWVtfWNzZzW/ntnd29/cLBYVNHiSK0QSIeqXaANeVM0oZhhtN2rCgWAaetYHQ9rbceqNIskndmHFNf4IFkISPYWOv+qZeacomde2ei7E16haJbcWdCy+BlUIRM9V7hq9uPSCKoNIRjrTueGxs/xcowwukk3000jTEZ4QHtWJRYUO2ns6sn6NQ6fRRGyj5p0Mz9PZFiofVYBLZTYDPUi7Wp+V+tk5jwyk+ZjBNDJZkvChOOTISmEaA+U5QYPraAiWL2VkSGWGFibFB5G4K3+OVlaF5UPMu31WKtmsWRg2M4gRJ4cAk1uIE6NICAgmd4hTfn0Xlx3p2PeeuKk80cwR85nz+y/ZFH</latexit>

xt+im<latexit sha1_base64="qPg/oPXN6lFFUDTaF6XHwgAak80=">AAAB73icbZBNS8NAEIYn9avWr6pHL4tFEISSiKDeCl48VrAf0Iay2W7apZtN3J2IJfRPePGgiFf/jjf/jds2B219YeHhnRl25g0SKQy67rdTWFldW98obpa2tnd298r7B00Tp5rxBotlrNsBNVwKxRsoUPJ2ojmNAslbwehmWm89cm1ErO5xnHA/ogMlQsEoWqv91MvwTESTXrniVt2ZyDJ4OVQgV71X/ur2Y5ZGXCGT1JiO5yboZ1SjYJJPSt3U8ISyER3wjkVFI278bLbvhJxYp0/CWNunkMzc3xMZjYwZR4HtjCgOzWJtav5X66QYXvmZUEmKXLH5R2EqCcZkejzpC80ZyrEFyrSwuxI2pJoytBGVbAje4snL0DyvepbvLiq16zyOIhzBMZyCB5dQg1uoQwMYSHiGV3hzHpwX5935mLcWnHzmEP7I+fwBLu+QBQ==</latexit><latexit sha1_base64="qPg/oPXN6lFFUDTaF6XHwgAak80=">AAAB73icbZBNS8NAEIYn9avWr6pHL4tFEISSiKDeCl48VrAf0Iay2W7apZtN3J2IJfRPePGgiFf/jjf/jds2B219YeHhnRl25g0SKQy67rdTWFldW98obpa2tnd298r7B00Tp5rxBotlrNsBNVwKxRsoUPJ2ojmNAslbwehmWm89cm1ErO5xnHA/ogMlQsEoWqv91MvwTESTXrniVt2ZyDJ4OVQgV71X/ur2Y5ZGXCGT1JiO5yboZ1SjYJJPSt3U8ISyER3wjkVFI278bLbvhJxYp0/CWNunkMzc3xMZjYwZR4HtjCgOzWJtav5X66QYXvmZUEmKXLH5R2EqCcZkejzpC80ZyrEFyrSwuxI2pJoytBGVbAje4snL0DyvepbvLiq16zyOIhzBMZyCB5dQg1uoQwMYSHiGV3hzHpwX5935mLcWnHzmEP7I+fwBLu+QBQ==</latexit><latexit sha1_base64="qPg/oPXN6lFFUDTaF6XHwgAak80=">AAAB73icbZBNS8NAEIYn9avWr6pHL4tFEISSiKDeCl48VrAf0Iay2W7apZtN3J2IJfRPePGgiFf/jjf/jds2B219YeHhnRl25g0SKQy67rdTWFldW98obpa2tnd298r7B00Tp5rxBotlrNsBNVwKxRsoUPJ2ojmNAslbwehmWm89cm1ErO5xnHA/ogMlQsEoWqv91MvwTESTXrniVt2ZyDJ4OVQgV71X/ur2Y5ZGXCGT1JiO5yboZ1SjYJJPSt3U8ISyER3wjkVFI278bLbvhJxYp0/CWNunkMzc3xMZjYwZR4HtjCgOzWJtav5X66QYXvmZUEmKXLH5R2EqCcZkejzpC80ZyrEFyrSwuxI2pJoytBGVbAje4snL0DyvepbvLiq16zyOIhzBMZyCB5dQg1uoQwMYSHiGV3hzHpwX5935mLcWnHzmEP7I+fwBLu+QBQ==</latexit><latexit sha1_base64="qPg/oPXN6lFFUDTaF6XHwgAak80=">AAAB73icbZBNS8NAEIYn9avWr6pHL4tFEISSiKDeCl48VrAf0Iay2W7apZtN3J2IJfRPePGgiFf/jjf/jds2B219YeHhnRl25g0SKQy67rdTWFldW98obpa2tnd298r7B00Tp5rxBotlrNsBNVwKxRsoUPJ2ojmNAslbwehmWm89cm1ErO5xnHA/ogMlQsEoWqv91MvwTESTXrniVt2ZyDJ4OVQgV71X/ur2Y5ZGXCGT1JiO5yboZ1SjYJJPSt3U8ISyER3wjkVFI278bLbvhJxYp0/CWNunkMzc3xMZjYwZR4HtjCgOzWJtav5X66QYXvmZUEmKXLH5R2EqCcZkejzpC80ZyrEFyrSwuxI2pJoytBGVbAje4snL0DyvepbvLiq16zyOIhzBMZyCB5dQg1uoQwMYSHiGV3hzHpwX5935mLcWnHzmEP7I+fwBLu+QBQ==</latexit>

Figure 4: Architecture of our critic network in WGAIL-div.

approach taken by the original GAIL of using TRPO orPPO to optimize a parametrized stochastic policy fails inthe high-dimensional action space A. Therefore, inspiredby [37] and [23], we use deterministic policy and substitutethe TRPO step in G with a deterministic policy gradientstep that directly use the gradient information from both thecritic function Dw and the policy generating function πθto update θ in order to minimize the expected accumulatedcritic loss value incurred by the agent as it makes predic-tions according to its policy πθ. More specifically, supposethat during the G step we execute our prediction agent’scurrent policy πθ to generate a collection of N samples ofstate-action pairs {(si, ai)}Ni=1, then our current estimate ofthe gradient over the policy parameter θ would be:

∇θ Eπθ [Dw(s, a)]

≈ 1

N

N∑

i=1

∇aDw(s, a)|s=si,a=πθ(si)∇θπθ(s)|s=si (4)

4.5. Model Architectures

Our imitation learning system has two components: apolicy generator network πθ and a critic network Dw.Policy Generator Network: The policy generator networkπθ is at the core of our imitation learning system and is re-sponsible for predicting future human poses by sequentiallygenerating actions (small windows of consecutive pose vec-tors) from states (long sequences of previous pose observa-tions and predictions). It is used in both behavioral cloningand WGAIL-div. At each prediction step i, πθ needs toread in a length-[t + (i − 1) × m] sequence of state si ={x1, x2, . . . , xt+(i−1)×m} and then outputs a length-m se-quence as the action ai = {xt+(i−1)×m+1, . . . , xt+i×m}.To achieve this functionality, we use the sequence-to-sequence (seq2seq) architecture [39] for our policy gen-erator network. We construct a recurrent neural network(RNN) with a Gated Recurrent Unit (GRU) [8] cell of size

1024 as our encoder to read in the state sequence si, andconstruct another RNN with a different GRU cell of size1024 as our decoder to generate the action sequence ai.Similar to [26], we also add an extra layer of linear spatialdecoder on top of each GRU to map its 1024-dimensionaloutput vector down to a 54-dimensional vector (54 is thelength of each pose vector in our dataset, see Section 5 fordetails). The model architecture of πθ is shown in Figure 3.Critic Network: In our WGAIL-div algorithm, be-sides πθ, we also need to construct a critic networkDw that assigns critic values to all state-action pairs(si, ai) in S × A. For this critic network, we builda RNN with a Long Short-Term Memory (LSTM) [16]cell of size 1024 to encode the state sequence si ={x1, x2, . . . , xt+(i−1)×m} into a 1024-dimensional vectorof state embedding h(si), and build another RNN witha different LSTM cell of size 1024 to encode the ac-tion sequence ai = {xt+(i−1)×m+1, . . . , xt+i×m} into a1024-dimensional vector of action embedding u(ai). Wethen concatenate h(si) and u(ai) together into a 2048-dimensional vector embedding, which we feed into a 7-layer fully-connected neural network with the output sizeof each layer equals to 512, 256, 128, 64, 32, 16, 1, re-spectively and the activation functions being Leaky ReLU.This 7-layer fully-connected neural network outputs the fi-nal critic value for the state-action pair that Dw reads in.The model architecture of Dw is depicted in Figure 4.

4.6. Learning Algorithm

Our proposed imitation learning algorithm for humanpose prediction utilizes two methods: BC and WGAIL-div.They both have their respective advantages and disadvan-tages, and their advantages and disadvantages make themhighly complementary to each other. BC has fast train-ing speed and high sample efficiency, but may suffer frombad generalization and error compounding. In contrast,WGAIL-div often produces policies that have better gen-eralization property and pays more attention to the sequen-tial relationship across different time steps, but can be slowand difficult to train during the beginning iterations whenthe freshly initialized agent policy is very far away from theexpert policy. Then under our current setting of using im-itation learning to predict human motion dynamics, thesecharacteristics indicate that BC tends to perform better overshort-term predictions while WGAIL-div tends to be supe-rior over long-term predictions. Therefore, in order to havethe best of both worlds and to let our learning algorithmexcel at both short-term and long-term predictions, we firsttrain our policy generator network πθ using BC, and thenuse the trained parameters to initialize πθ in the WGAIL-div procedure. On one hand, the initialization obtained fromBC would help πθ to quickly move to regions that are closeto the expert policy, which greatly stabilizes and expedites

Page 6: Imitation Learning for Human Pose Prediction › pdf › 1909.03449v1.pdf · Imitation Learning for Human Pose Prediction Borui Wang, Ehsan Adeli, Hsu-kuang Chiu, De-An Huang, Juan

Algorithm 1 WGAIL-div for Human Pose Prediction1: Initialize the policy generator network πθ using the parameterθ trained by BC (Algorithm 2 in Appendix A.1)

2: Randomly initialize the parameter w of the critic network Dw3: for iteration = 1, 2, . . . , T do

D Step:4: Randomly sample a batch of ND length-(t+ l) trajectories

of human pose vectors from the training dataset E5: for j = 1, 2, . . . , ND do6: Take the j-th sampled trajectory {xj,1, xj,2, . . . , xj,t+l}7: for i = 1, 2, . . . ,K do8: sj,i = {xj,1, . . . , xj,t+(i−1)m},9: aj,i = {xj,t+(i−1)m+1, . . . , xj,t+im}

10: if i = 1, then sj,i = sj,i,else sj,i = concatenate[sj,i−1, aj,i−1]

11: aj,i = πθ(sj,i)12: Sample α from the uniform distribution: α ∼ U [0, 1]13: sj,i = α sj,i + (1− α) sj,i,14: aj,i = αaj,i + (1− α) aj,i15: end for16: end for17: Take an Adam step on w to maximize the objective:

w← Adam(∇w

[1

ND×K∑j,iDw(sj,i, aj,i)

−Dw(sj,i, aj,i)−k ‖∇( ˆsj,i, ˆaj,i)Dw(( ˆsj,i, ˆaj,i))‖p])

G Step:18: Randomly sample a batch of NG length-(t+ l) trajectories

of human pose vectors from the training dataset E19: Generate the agent’s and the expert’s state-action pairs

{(sj,i, aj,i)} and {(sj,i, aj,i)} in the same way as in D.20: Take an Adam step on θ to minimize the objective function:

θ← Adam(

1NG×K

∑j,i ∇aDw(s, a)|s=sj,i,a=πθ(sj,i)

∇θπθ(s)|s=sj,i)

21: end for

the training of WGAIL-div. On the other hand, WGAIL-div iterations can be viewed as further optimizing over theshortsighted policy generated from BC to make it generalizebetter to unfamiliar regions of the state space. As a result,our learning algorithm is consisted of two stages:Stage 1: Behavioral Cloning. In the BC stage, we ran-domly sample expert trajectories from the training datasetE , and update the parameter θ of the policy generator net-work πθ in order to match its generated actions with theexpert’s actions over the states that appear in the sampledtrajectories. Here, we use `1-norm to measure the distancebetween two action vectors. Refer to Algorithm 2 in Ap-pendix A.1 for the detailed steps of our BC algorithm.Stage 2: WGAIL-div. In the WGAIL-div stage, our al-gorithm alternates between a D step that updates the criticfunction and a G step that optimizes the policy generatorthrough deterministic policy gradients. The parameter w ofthe critic function is initialized randomly and the parame-ter θ of the policy generator is initialized using the trained

Figure 5: Plot of average mean angle error for different poseprediction models over walking, eating, smoking and dis-cussion in the short-term prediction experiment.

Figure 6: Plot of average mean angle error for different poseprediction models over walking, eating, smoking and dis-cussion in the long-term prediction experiment.

parameter from Stage 1. Our WGAIL-div algorithm for hu-man pose prediction is listed in Algorithm 1.

5. Experiments

To evaluate the performance of our proposed imitationlearning approach on challenging real human pose predic-tion tasks and to have a thorough comparison with previousworks, we run exhaustive experiments to test our algorithmusing the popular Human 3.6M dataset [17] and compareour results with previous benchmarks.The Human 3.6M dataset: The Human 3.6M dataset [17]is currently one of the largest publicly available motion cap-ture dataset, and has been used by many previous paperson human pose prediction. This dataset includes video se-quences recorded from 7 different actors performing 15 dif-ferent categories of human activities, with each actor per-forming each activity in two different trials. The videosare recorded in 50Hz, and for a fair comparison, we fol-low previous papers [19, 26] to downsample the pose se-

Page 7: Imitation Learning for Human Pose Prediction › pdf › 1909.03449v1.pdf · Imitation Learning for Human Pose Prediction Borui Wang, Ehsan Adeli, Hsu-kuang Chiu, De-An Huang, Juan

Table 1: Mean Angle Error for the activities walking, eating, smoking, and discussion in the short-term prediction experiment.The best result in each column is highlighted with boldface.

Walking Eating Smoking Discussionmilliseconds 80 160 320 400 80 160 320 400 80 160 320 400 80 160 320 400ERD [9] 0.93 1.18 1.59 1.78 1.27 1.45 1.66 1.80 1.66 1.95 2.35 2.42 2.27 2.47 2.68 2.76LSTM-3LR [9] 0.77 1.00 1.29 1.47 0.89 1.09 1.35 1.46 1.34 1.65 2.04 2.16 1.88 2.12 2.25 2.23SRNN [19] 0.81 0.94 1.16 1.30 0.97 1.14 1.35 1.46 1.45 1.68 1.94 2.08 1.22 1.49 1.83 1.93Residual [26] 0.28 0.49 0.72 0.81 0.23 0.39 0.62 0.76 0.33 0.61 1.05 1.15 0.31 0.68 1.01 1.09Zero-velocity [26] 0.39 0.68 0.99 1.15 0.27 0.48 0.73 0.86 0.26 0.48 0.97 0.95 0.31 0.67 0.94 1.04TP-RNN [7] 0.25 0.41 0.58 0.65 0.20 0.33 0.53 0.67 0.26 0.47 0.88 0.90 0.30 0.66 0.96 1.04Ours 0.21 0.34 0.53 0.59 0.17 0.30 0.52 0.65 0.23 0.44 0.87 0.85 0.23 0.56 0.82 0.91

Table 2: Mean Angle Error for the activities walking, eating, smoking, and discussion in the long-term prediction experiment.The best result in each column is highlighted with boldface.

Walking Eating Smoking Discussionmilliseconds 80 160 320 560 1000 80 160 320 560 1000 80 160 320 560 1000 80 160 320 560 1000ERD [9] 1.30 1.56 1.84 2.00 2.38 1.66 1.93 2.88 2.36 2.41 2.34 2.74 3.73 3.68 3.82 2.67 2.97 3.23 3.47 2.92LSTM-3LR [9] 1.18 1.50 1.67 1.81 2.20 1.36 1.79 2.29 2.49 2.82 2.05 2.34 3.10 3.24 3.42 2.25 2.33 2.45 2.48 2.93SRNN [19] 1.08 1.34 1.60 1.90 2.13 1.35 1.71 2.12 2.28 2.58 1.90 2.30 2.90 3.21 3.23 1.67 2.03 2.20 2.39 2.43Dropout-AE [11] 1.00 1.11 1.39 1.55 1.39 1.31 1.49 1.86 1.76 2.01 0.92 1.03 1.15 1.38 1.77 1.11 1.20 1.38 1.53 1.73Residual [26] 0.32 0.54 0.72 0.86 0.96 0.25 0.42 0.64 0.94 1.30 0.33 0.60 1.01 1.23 1.83 0.34 0.74 1.04 1.43 1.75Zero-velocity [26] 0.39 0.68 0.99 1.35 1.32 0.27 0.48 0.73 1.04 1.38 0.26 0.48 0.97 1.02 1.69 0.31 0.67 0.94 1.41 1.96TP-RNN [7] 0.25 0.41 0.58 0.74 0.77 0.20 0.33 0.53 0.84 1.14 0.26 0.48 0.88 0.98 1.66 0.30 0.66 0.98 1.39 1.74Ours 0.21 0.34 0.53 0.67 0.69 0.17 0.30 0.52 0.79 1.13 0.23 0.44 0.86 0.95 1.63 0.27 0.56 0.82 1.34 1.81

quence by 2. Each pose in this dataset is represented asan exponential map representation of 32 human joints in3D, and after preprocessing of global translation and rota-tion as described in [7, 9, 19, 26], we adopt the followingevaluation methods for our experiments: during both train-ing and testing, we feed 2000ms (50 frames) of past posevector sequence into the learning system, and the goal is topredict the next 1000ms (25 frames) of future pose vectorsequence. We measure the Euclidean distance between pre-dicted poses and ground truth poses in angle space as theevaluation metric of prediction error, and we report the av-erage prediction error over 6 different timescales: 80, 160,320, 400, 560, and 1000ms. Following previous works, weuse Subjects 1, 6, 7, 8, 9, 11 as the training dataset and useSubject 5 as the testing dataset.Baselines: We compare our experimental results with thefollowing recent state-of-the-art methods for human poseprediction on the Human 3.6M dataset: ERD [9], LSTM-3LR [9], SRNN [19], Dropout-AutoEncoder [11], Residual[26], Zero-velocity [7, 26] and TP-RNN [7].

5.1. Results

The performance results published by previous papersare reported on slightly different prediction timescales(some report on short-term predictions over 400ms and theothers report on long-term predictions over 1000ms) andactivity categories (some only report on 4 different activi-ties). To make a thorough comparison with all these previ-ous methods, we report the prediction results of our modelin Tables 1, 2 and 3, and plot the mean angle error curves inFigures 5 and 6.

Time t (ms)<latexit sha1_base64="0iRWuoxCJXKv3/6DSQvdhksfUII=">AAAB9HicbZDLTgIxFIY7eEO8oS7dNIIJbsgMG10S3bjEhFsCE9IpZ6Ch7Yxth4RMeA43LjTGrQ/jzrexwCwU/JMmX/5zTs7pH8ScaeO6305ua3tndy+/Xzg4PDo+KZ6etXWUKAotGvFIdQOigTMJLcMMh26sgIiAQyeY3C/qnSkozSLZNLMYfEFGkoWMEmMtv8kE4LIp44rQ14Niya26S+FN8DIooUyNQfGrP4xoIkAayonWPc+NjZ8SZRjlMC/0Ew0xoRMygp5FSQRoP10ePcdX1hniMFL2SYOX7u+JlAitZyKwnYKYsV6vLcz/ar3EhLd+ymScGJB0tShMODYRXiSAh0wBNXxmgVDF7K2Yjoki1NicCjYEb/3Lm9CuVT3Lj7VS/S6LI48u0CWqIA/doDp6QA3UQhQ9oWf0it6cqfPivDsfq9ack82coz9yPn8ArDeQtg==</latexit><latexit sha1_base64="0iRWuoxCJXKv3/6DSQvdhksfUII=">AAAB9HicbZDLTgIxFIY7eEO8oS7dNIIJbsgMG10S3bjEhFsCE9IpZ6Ch7Yxth4RMeA43LjTGrQ/jzrexwCwU/JMmX/5zTs7pH8ScaeO6305ua3tndy+/Xzg4PDo+KZ6etXWUKAotGvFIdQOigTMJLcMMh26sgIiAQyeY3C/qnSkozSLZNLMYfEFGkoWMEmMtv8kE4LIp44rQ14Niya26S+FN8DIooUyNQfGrP4xoIkAayonWPc+NjZ8SZRjlMC/0Ew0xoRMygp5FSQRoP10ePcdX1hniMFL2SYOX7u+JlAitZyKwnYKYsV6vLcz/ar3EhLd+ymScGJB0tShMODYRXiSAh0wBNXxmgVDF7K2Yjoki1NicCjYEb/3Lm9CuVT3Lj7VS/S6LI48u0CWqIA/doDp6QA3UQhQ9oWf0it6cqfPivDsfq9ack82coz9yPn8ArDeQtg==</latexit><latexit sha1_base64="0iRWuoxCJXKv3/6DSQvdhksfUII=">AAAB9HicbZDLTgIxFIY7eEO8oS7dNIIJbsgMG10S3bjEhFsCE9IpZ6Ch7Yxth4RMeA43LjTGrQ/jzrexwCwU/JMmX/5zTs7pH8ScaeO6305ua3tndy+/Xzg4PDo+KZ6etXWUKAotGvFIdQOigTMJLcMMh26sgIiAQyeY3C/qnSkozSLZNLMYfEFGkoWMEmMtv8kE4LIp44rQ14Niya26S+FN8DIooUyNQfGrP4xoIkAayonWPc+NjZ8SZRjlMC/0Ew0xoRMygp5FSQRoP10ePcdX1hniMFL2SYOX7u+JlAitZyKwnYKYsV6vLcz/ar3EhLd+ymScGJB0tShMODYRXiSAh0wBNXxmgVDF7K2Yjoki1NicCjYEb/3Lm9CuVT3Lj7VS/S6LI48u0CWqIA/doDp6QA3UQhQ9oWf0it6cqfPivDsfq9ack82coz9yPn8ArDeQtg==</latexit><latexit sha1_base64="0iRWuoxCJXKv3/6DSQvdhksfUII=">AAAB9HicbZDLTgIxFIY7eEO8oS7dNIIJbsgMG10S3bjEhFsCE9IpZ6Ch7Yxth4RMeA43LjTGrQ/jzrexwCwU/JMmX/5zTs7pH8ScaeO6305ua3tndy+/Xzg4PDo+KZ6etXWUKAotGvFIdQOigTMJLcMMh26sgIiAQyeY3C/qnSkozSLZNLMYfEFGkoWMEmMtv8kE4LIp44rQ14Niya26S+FN8DIooUyNQfGrP4xoIkAayonWPc+NjZ8SZRjlMC/0Ew0xoRMygp5FSQRoP10ePcdX1hniMFL2SYOX7u+JlAitZyKwnYKYsV6vLcz/ar3EhLd+ymScGJB0tShMODYRXiSAh0wBNXxmgVDF7K2Yjoki1NicCjYEb/3Lm9CuVT3Lj7VS/S6LI48u0CWqIA/doDp6QA3UQhQ9oWf0it6cqfPivDsfq9ack82coz9yPn8ArDeQtg==</latexit>

1200ms<latexit sha1_base64="VzwQbjr7l4r06rBxahZ9FEpT/1A=">AAAB7XicbZC7SgNBFIbPxluMt1VLm8EgWIXdNFoGbSwjmAskS5idzCZj5rLMzAphyTvYWChi6/vY+TZOki008YeBj/+cw5zzxylnxgbBt1fa2Nza3invVvb2Dw6P/OOTtlGZJrRFFFe6G2NDOZO0ZZnltJtqikXMaSee3M7rnSeqDVPywU5TGgk8kixhBFtntcN6EAgz8KtBLVgIrUNYQBUKNQf+V3+oSCaotIRjY3phkNoox9oywums0s8MTTGZ4BHtOZRYUBPli21n6MI5Q5Qo7Z60aOH+nsixMGYqYtcpsB2b1drc/K/Wy2xyHeVMppmlkiw/SjKOrELz09GQaUosnzrARDO3KyJjrDGxLqCKCyFcPXkd2vVa6Pi+Xm3cFHGU4QzO4RJCuIIG3EETWkDgEZ7hFd485b14797HsrXkFTOn8Efe5w9mlo5X</latexit><latexit sha1_base64="VzwQbjr7l4r06rBxahZ9FEpT/1A=">AAAB7XicbZC7SgNBFIbPxluMt1VLm8EgWIXdNFoGbSwjmAskS5idzCZj5rLMzAphyTvYWChi6/vY+TZOki008YeBj/+cw5zzxylnxgbBt1fa2Nza3invVvb2Dw6P/OOTtlGZJrRFFFe6G2NDOZO0ZZnltJtqikXMaSee3M7rnSeqDVPywU5TGgk8kixhBFtntcN6EAgz8KtBLVgIrUNYQBUKNQf+V3+oSCaotIRjY3phkNoox9oywums0s8MTTGZ4BHtOZRYUBPli21n6MI5Q5Qo7Z60aOH+nsixMGYqYtcpsB2b1drc/K/Wy2xyHeVMppmlkiw/SjKOrELz09GQaUosnzrARDO3KyJjrDGxLqCKCyFcPXkd2vVa6Pi+Xm3cFHGU4QzO4RJCuIIG3EETWkDgEZ7hFd485b14797HsrXkFTOn8Efe5w9mlo5X</latexit><latexit sha1_base64="VzwQbjr7l4r06rBxahZ9FEpT/1A=">AAAB7XicbZC7SgNBFIbPxluMt1VLm8EgWIXdNFoGbSwjmAskS5idzCZj5rLMzAphyTvYWChi6/vY+TZOki008YeBj/+cw5zzxylnxgbBt1fa2Nza3invVvb2Dw6P/OOTtlGZJrRFFFe6G2NDOZO0ZZnltJtqikXMaSee3M7rnSeqDVPywU5TGgk8kixhBFtntcN6EAgz8KtBLVgIrUNYQBUKNQf+V3+oSCaotIRjY3phkNoox9oywums0s8MTTGZ4BHtOZRYUBPli21n6MI5Q5Qo7Z60aOH+nsixMGYqYtcpsB2b1drc/K/Wy2xyHeVMppmlkiw/SjKOrELz09GQaUosnzrARDO3KyJjrDGxLqCKCyFcPXkd2vVa6Pi+Xm3cFHGU4QzO4RJCuIIG3EETWkDgEZ7hFd485b14797HsrXkFTOn8Efe5w9mlo5X</latexit><latexit sha1_base64="VzwQbjr7l4r06rBxahZ9FEpT/1A=">AAAB7XicbZC7SgNBFIbPxluMt1VLm8EgWIXdNFoGbSwjmAskS5idzCZj5rLMzAphyTvYWChi6/vY+TZOki008YeBj/+cw5zzxylnxgbBt1fa2Nza3invVvb2Dw6P/OOTtlGZJrRFFFe6G2NDOZO0ZZnltJtqikXMaSee3M7rnSeqDVPywU5TGgk8kixhBFtntcN6EAgz8KtBLVgIrUNYQBUKNQf+V3+oSCaotIRjY3phkNoox9oywums0s8MTTGZ4BHtOZRYUBPli21n6MI5Q5Qo7Z60aOH+nsixMGYqYtcpsB2b1drc/K/Wy2xyHeVMppmlkiw/SjKOrELz09GQaUosnzrARDO3KyJjrDGxLqCKCyFcPXkd2vVa6Pi+Xm3cFHGU4QzO4RJCuIIG3EETWkDgEZ7hFd485b14797HsrXkFTOn8Efe5w9mlo5X</latexit>

0ms<latexit sha1_base64="ffCJWAy1UH6fgsnnOY4PpZr+Vy4=">AAAB6nicbZBNSwMxEIZn61etX1WPXoJF8FR2e9Fj0YvHivYD2qVk09k2NMkuSVYopT/BiwdFvPqLvPlvTNs9aOsLgYd3ZsjMG6WCG+v7315hY3Nre6e4W9rbPzg8Kh+ftEySaYZNlohEdyJqUHCFTcutwE6qkcpIYDsa387r7SfUhifq0U5SDCUdKh5zRq2zHnxp+uWKX/UXIusQ5FCBXI1++as3SFgmUVkmqDHdwE9tOKXaciZwVuplBlPKxnSIXYeKSjThdLHqjFw4Z0DiRLunLFm4vyemVBozkZHrlNSOzGptbv5X62Y2vg6nXKWZRcWWH8WZIDYh87vJgGtkVkwcUKa525WwEdWUWZdOyYUQrJ68Dq1aNXB8X6vUb/I4inAG53AJAVxBHe6gAU1gMIRneIU3T3gv3rv3sWwtePnMKfyR9/kDFrCNpg==</latexit><latexit sha1_base64="ffCJWAy1UH6fgsnnOY4PpZr+Vy4=">AAAB6nicbZBNSwMxEIZn61etX1WPXoJF8FR2e9Fj0YvHivYD2qVk09k2NMkuSVYopT/BiwdFvPqLvPlvTNs9aOsLgYd3ZsjMG6WCG+v7315hY3Nre6e4W9rbPzg8Kh+ftEySaYZNlohEdyJqUHCFTcutwE6qkcpIYDsa387r7SfUhifq0U5SDCUdKh5zRq2zHnxp+uWKX/UXIusQ5FCBXI1++as3SFgmUVkmqDHdwE9tOKXaciZwVuplBlPKxnSIXYeKSjThdLHqjFw4Z0DiRLunLFm4vyemVBozkZHrlNSOzGptbv5X62Y2vg6nXKWZRcWWH8WZIDYh87vJgGtkVkwcUKa525WwEdWUWZdOyYUQrJ68Dq1aNXB8X6vUb/I4inAG53AJAVxBHe6gAU1gMIRneIU3T3gv3rv3sWwtePnMKfyR9/kDFrCNpg==</latexit><latexit sha1_base64="ffCJWAy1UH6fgsnnOY4PpZr+Vy4=">AAAB6nicbZBNSwMxEIZn61etX1WPXoJF8FR2e9Fj0YvHivYD2qVk09k2NMkuSVYopT/BiwdFvPqLvPlvTNs9aOsLgYd3ZsjMG6WCG+v7315hY3Nre6e4W9rbPzg8Kh+ftEySaYZNlohEdyJqUHCFTcutwE6qkcpIYDsa387r7SfUhifq0U5SDCUdKh5zRq2zHnxp+uWKX/UXIusQ5FCBXI1++as3SFgmUVkmqDHdwE9tOKXaciZwVuplBlPKxnSIXYeKSjThdLHqjFw4Z0DiRLunLFm4vyemVBozkZHrlNSOzGptbv5X62Y2vg6nXKWZRcWWH8WZIDYh87vJgGtkVkwcUKa525WwEdWUWZdOyYUQrJ68Dq1aNXB8X6vUb/I4inAG53AJAVxBHe6gAU1gMIRneIU3T3gv3rv3sWwtePnMKfyR9/kDFrCNpg==</latexit><latexit sha1_base64="ffCJWAy1UH6fgsnnOY4PpZr+Vy4=">AAAB6nicbZBNSwMxEIZn61etX1WPXoJF8FR2e9Fj0YvHivYD2qVk09k2NMkuSVYopT/BiwdFvPqLvPlvTNs9aOsLgYd3ZsjMG6WCG+v7315hY3Nre6e4W9rbPzg8Kh+ftEySaYZNlohEdyJqUHCFTcutwE6qkcpIYDsa387r7SfUhifq0U5SDCUdKh5zRq2zHnxp+uWKX/UXIusQ5FCBXI1++as3SFgmUVkmqDHdwE9tOKXaciZwVuplBlPKxnSIXYeKSjThdLHqjFw4Z0DiRLunLFm4vyemVBozkZHrlNSOzGptbv5X62Y2vg6nXKWZRcWWH8WZIDYh87vJgGtkVkwcUKa525WwEdWUWZdOyYUQrJ68Dq1aNXB8X6vUb/I4inAG53AJAVxBHe6gAU1gMIRneIU3T3gv3rv3sWwtePnMKfyR9/kDFrCNpg==</latexit>

Start<latexit sha1_base64="Y2CFThq7wFBLnY/B2B9DRC9MAW0=">AAAB7HicbVBNT8JAEJ3iF+IX6tHLRmLiibRc9Ej04hGjBRJoyHbZwobtttmdmpCG3+DFg8Z49Qd589+4QA8KvmSTl/dmZmdemEph0HW/ndLG5tb2Tnm3srd/cHhUPT5pmyTTjPsskYnuhtRwKRT3UaDk3VRzGoeSd8LJ7dzvPHFtRKIecZryIKYjJSLBKFrJf0CqcVCtuXV3AbJOvILUoEBrUP3qDxOWxVwhk9SYnuemGOR2lGCSzyr9zPCUsgkd8Z6lisbcBPli2Rm5sMqQRIm2TyFZqL87chobM41DWxlTHJtVby7+5/UyjK6DXKg0Q67Y8qMokwQTMr+cDIXmDOXUEsq0sLsSNqaaMrT5VGwI3urJ66TdqHuW3zdqzZsijjKcwTlcggdX0IQ7aIEPDAQ8wyu8Ocp5cd6dj2VpySl6TuEPnM8f5JiOuA==</latexit><latexit sha1_base64="Y2CFThq7wFBLnY/B2B9DRC9MAW0=">AAAB7HicbVBNT8JAEJ3iF+IX6tHLRmLiibRc9Ej04hGjBRJoyHbZwobtttmdmpCG3+DFg8Z49Qd589+4QA8KvmSTl/dmZmdemEph0HW/ndLG5tb2Tnm3srd/cHhUPT5pmyTTjPsskYnuhtRwKRT3UaDk3VRzGoeSd8LJ7dzvPHFtRKIecZryIKYjJSLBKFrJf0CqcVCtuXV3AbJOvILUoEBrUP3qDxOWxVwhk9SYnuemGOR2lGCSzyr9zPCUsgkd8Z6lisbcBPli2Rm5sMqQRIm2TyFZqL87chobM41DWxlTHJtVby7+5/UyjK6DXKg0Q67Y8qMokwQTMr+cDIXmDOXUEsq0sLsSNqaaMrT5VGwI3urJ66TdqHuW3zdqzZsijjKcwTlcggdX0IQ7aIEPDAQ8wyu8Ocp5cd6dj2VpySl6TuEPnM8f5JiOuA==</latexit><latexit sha1_base64="Y2CFThq7wFBLnY/B2B9DRC9MAW0=">AAAB7HicbVBNT8JAEJ3iF+IX6tHLRmLiibRc9Ej04hGjBRJoyHbZwobtttmdmpCG3+DFg8Z49Qd589+4QA8KvmSTl/dmZmdemEph0HW/ndLG5tb2Tnm3srd/cHhUPT5pmyTTjPsskYnuhtRwKRT3UaDk3VRzGoeSd8LJ7dzvPHFtRKIecZryIKYjJSLBKFrJf0CqcVCtuXV3AbJOvILUoEBrUP3qDxOWxVwhk9SYnuemGOR2lGCSzyr9zPCUsgkd8Z6lisbcBPli2Rm5sMqQRIm2TyFZqL87chobM41DWxlTHJtVby7+5/UyjK6DXKg0Q67Y8qMokwQTMr+cDIXmDOXUEsq0sLsSNqaaMrT5VGwI3urJ66TdqHuW3zdqzZsijjKcwTlcggdX0IQ7aIEPDAQ8wyu8Ocp5cd6dj2VpySl6TuEPnM8f5JiOuA==</latexit><latexit sha1_base64="Y2CFThq7wFBLnY/B2B9DRC9MAW0=">AAAB7HicbVBNT8JAEJ3iF+IX6tHLRmLiibRc9Ej04hGjBRJoyHbZwobtttmdmpCG3+DFg8Z49Qd589+4QA8KvmSTl/dmZmdemEph0HW/ndLG5tb2Tnm3srd/cHhUPT5pmyTTjPsskYnuhtRwKRT3UaDk3VRzGoeSd8LJ7dzvPHFtRKIecZryIKYjJSLBKFrJf0CqcVCtuXV3AbJOvILUoEBrUP3qDxOWxVwhk9SYnuemGOR2lGCSzyr9zPCUsgkd8Z6lisbcBPli2Rm5sMqQRIm2TyFZqL87chobM41DWxlTHJtVby7+5/UyjK6DXKg0Q67Y8qMokwQTMr+cDIXmDOXUEsq0sLsSNqaaMrT5VGwI3urJ66TdqHuW3zdqzZsijjKcwTlcggdX0IQ7aIEPDAQ8wyu8Ocp5cd6dj2VpySl6TuEPnM8f5JiOuA==</latexit>

Residua

l

<latexit sha1_base64="FVAfnrGSn2tNroBd85q8yKklBAQ=">AAAB+nicdVDLSgNBEJz1bXytevQyGARPYTY5JN6CXjxGMVFIljA76dUhsw9metWw5lO8eFDEq1/izb9xEldQ0YKGoqqb7q4gVdIgY+/OzOzc/MLi0nJpZXVtfcPd3OqYJNMC2iJRib4IuAElY2ijRAUXqQYeBQrOg+HRxD+/Bm1kEp/hKAU/4pexDKXgaKW+u9lDuMUgzE/ByEHG1bjvllmFTUFZpeHVa6xhSY0d1FmVeoVVJgVaffetN0hEFkGMQnFjuh5L0c+5RikUjEu9zEDKxZBfQtfSmEdg/Hx6+pjuWWVAw0TbipFO1e8TOY+MGUWB7Yw4Xpnf3kT8y+tmGDb8XMZphhCLz0VhpigmdJIDHUgNAtXIEi60tLdSccU1F2jTKtkQvj6l/5NOteJZflItNw+LOJbIDtkl+8QjddIkx6RF2kSQG3JPHsmTc+c8OM/Oy2frjFPMbJMfcF4/ADgolJ0=</latexit><latexit sha1_base64="FVAfnrGSn2tNroBd85q8yKklBAQ=">AAAB+nicdVDLSgNBEJz1bXytevQyGARPYTY5JN6CXjxGMVFIljA76dUhsw9metWw5lO8eFDEq1/izb9xEldQ0YKGoqqb7q4gVdIgY+/OzOzc/MLi0nJpZXVtfcPd3OqYJNMC2iJRib4IuAElY2ijRAUXqQYeBQrOg+HRxD+/Bm1kEp/hKAU/4pexDKXgaKW+u9lDuMUgzE/ByEHG1bjvllmFTUFZpeHVa6xhSY0d1FmVeoVVJgVaffetN0hEFkGMQnFjuh5L0c+5RikUjEu9zEDKxZBfQtfSmEdg/Hx6+pjuWWVAw0TbipFO1e8TOY+MGUWB7Yw4Xpnf3kT8y+tmGDb8XMZphhCLz0VhpigmdJIDHUgNAtXIEi60tLdSccU1F2jTKtkQvj6l/5NOteJZflItNw+LOJbIDtkl+8QjddIkx6RF2kSQG3JPHsmTc+c8OM/Oy2frjFPMbJMfcF4/ADgolJ0=</latexit><latexit sha1_base64="FVAfnrGSn2tNroBd85q8yKklBAQ=">AAAB+nicdVDLSgNBEJz1bXytevQyGARPYTY5JN6CXjxGMVFIljA76dUhsw9metWw5lO8eFDEq1/izb9xEldQ0YKGoqqb7q4gVdIgY+/OzOzc/MLi0nJpZXVtfcPd3OqYJNMC2iJRib4IuAElY2ijRAUXqQYeBQrOg+HRxD+/Bm1kEp/hKAU/4pexDKXgaKW+u9lDuMUgzE/ByEHG1bjvllmFTUFZpeHVa6xhSY0d1FmVeoVVJgVaffetN0hEFkGMQnFjuh5L0c+5RikUjEu9zEDKxZBfQtfSmEdg/Hx6+pjuWWVAw0TbipFO1e8TOY+MGUWB7Yw4Xpnf3kT8y+tmGDb8XMZphhCLz0VhpigmdJIDHUgNAtXIEi60tLdSccU1F2jTKtkQvj6l/5NOteJZflItNw+LOJbIDtkl+8QjddIkx6RF2kSQG3JPHsmTc+c8OM/Oy2frjFPMbJMfcF4/ADgolJ0=</latexit><latexit sha1_base64="FVAfnrGSn2tNroBd85q8yKklBAQ=">AAAB+nicdVDLSgNBEJz1bXytevQyGARPYTY5JN6CXjxGMVFIljA76dUhsw9metWw5lO8eFDEq1/izb9xEldQ0YKGoqqb7q4gVdIgY+/OzOzc/MLi0nJpZXVtfcPd3OqYJNMC2iJRib4IuAElY2ijRAUXqQYeBQrOg+HRxD+/Bm1kEp/hKAU/4pexDKXgaKW+u9lDuMUgzE/ByEHG1bjvllmFTUFZpeHVa6xhSY0d1FmVeoVVJgVaffetN0hEFkGMQnFjuh5L0c+5RikUjEu9zEDKxZBfQtfSmEdg/Hx6+pjuWWVAw0TbipFO1e8TOY+MGUWB7Yw4Xpnf3kT8y+tmGDb8XMZphhCLz0VhpigmdJIDHUgNAtXIEi60tLdSccU1F2jTKtkQvj6l/5NOteJZflItNw+LOJbIDtkl+8QjddIkx6RF2kSQG3JPHsmTc+c8OM/Oy2frjFPMbJMfcF4/ADgolJ0=</latexit>

BC+W

GAIL

-div

(Our

s)

<latexit sha1_base64="oTg7XQbqoIttD9hNncluhs8k/zo=">AAACIHicdVBNS8MwGE7n15xfU49eikNQxNFOZR7ndlBBcILbhHWMNH27haVpSdLhKPspXvwrXjwoojf9NWZzgoo+EHh4PkjyuBGjUlnWm5Gamp6ZnUvPZxYWl5ZXsqtrdRnGgkCNhCwU1y6WwCiHmqKKwXUkAAcug4bbq4z8Rh+EpCG/UoMIWgHucOpTgpWW2tmi40KH8oQAVyCGjoIb5fpJubLbODk+O9/zaN9xti9iIXeGDnDvK9jO5uy8NYZp5fcP7QOroMlE+bJyaIJqO/vqeCGJA10nDEvZtK1ItRIsFCUMhhknlhBh0sMdaGrKcQCylYw/ODS3tOKZfij04cocq98bCQ6kHASuTgZYdeVvbyT+5TVj5R+1EsqjWAEnnxf5MTNVaI7WMj0qgCg20AQTQfVbTdLFAhO9gcx8H+F/Ui/kbc0vC7lSeTJHGm2gTbSNbFREJXSKqqiGCLpF9+gRPRl3xoPxbLx8RlPGpLOOfsB4/wC64KPs</latexit><latexit sha1_base64="oTg7XQbqoIttD9hNncluhs8k/zo=">AAACIHicdVBNS8MwGE7n15xfU49eikNQxNFOZR7ndlBBcILbhHWMNH27haVpSdLhKPspXvwrXjwoojf9NWZzgoo+EHh4PkjyuBGjUlnWm5Gamp6ZnUvPZxYWl5ZXsqtrdRnGgkCNhCwU1y6WwCiHmqKKwXUkAAcug4bbq4z8Rh+EpCG/UoMIWgHucOpTgpWW2tmi40KH8oQAVyCGjoIb5fpJubLbODk+O9/zaN9xti9iIXeGDnDvK9jO5uy8NYZp5fcP7QOroMlE+bJyaIJqO/vqeCGJA10nDEvZtK1ItRIsFCUMhhknlhBh0sMdaGrKcQCylYw/ODS3tOKZfij04cocq98bCQ6kHASuTgZYdeVvbyT+5TVj5R+1EsqjWAEnnxf5MTNVaI7WMj0qgCg20AQTQfVbTdLFAhO9gcx8H+F/Ui/kbc0vC7lSeTJHGm2gTbSNbFREJXSKqqiGCLpF9+gRPRl3xoPxbLx8RlPGpLOOfsB4/wC64KPs</latexit><latexit sha1_base64="oTg7XQbqoIttD9hNncluhs8k/zo=">AAACIHicdVBNS8MwGE7n15xfU49eikNQxNFOZR7ndlBBcILbhHWMNH27haVpSdLhKPspXvwrXjwoojf9NWZzgoo+EHh4PkjyuBGjUlnWm5Gamp6ZnUvPZxYWl5ZXsqtrdRnGgkCNhCwU1y6WwCiHmqKKwXUkAAcug4bbq4z8Rh+EpCG/UoMIWgHucOpTgpWW2tmi40KH8oQAVyCGjoIb5fpJubLbODk+O9/zaN9xti9iIXeGDnDvK9jO5uy8NYZp5fcP7QOroMlE+bJyaIJqO/vqeCGJA10nDEvZtK1ItRIsFCUMhhknlhBh0sMdaGrKcQCylYw/ODS3tOKZfij04cocq98bCQ6kHASuTgZYdeVvbyT+5TVj5R+1EsqjWAEnnxf5MTNVaI7WMj0qgCg20AQTQfVbTdLFAhO9gcx8H+F/Ui/kbc0vC7lSeTJHGm2gTbSNbFREJXSKqqiGCLpF9+gRPRl3xoPxbLx8RlPGpLOOfsB4/wC64KPs</latexit><latexit sha1_base64="oTg7XQbqoIttD9hNncluhs8k/zo=">AAACIHicdVBNS8MwGE7n15xfU49eikNQxNFOZR7ndlBBcILbhHWMNH27haVpSdLhKPspXvwrXjwoojf9NWZzgoo+EHh4PkjyuBGjUlnWm5Gamp6ZnUvPZxYWl5ZXsqtrdRnGgkCNhCwU1y6WwCiHmqKKwXUkAAcug4bbq4z8Rh+EpCG/UoMIWgHucOpTgpWW2tmi40KH8oQAVyCGjoIb5fpJubLbODk+O9/zaN9xti9iIXeGDnDvK9jO5uy8NYZp5fcP7QOroMlE+bJyaIJqO/vqeCGJA10nDEvZtK1ItRIsFCUMhhknlhBh0sMdaGrKcQCylYw/ODS3tOKZfij04cocq98bCQ6kHASuTgZYdeVvbyT+5TVj5R+1EsqjWAEnnxf5MTNVaI7WMj0qgCg20AQTQfVbTdLFAhO9gcx8H+F/Ui/kbc0vC7lSeTJHGm2gTbSNbFREJXSKqqiGCLpF9+gRPRl3xoPxbLx8RlPGpLOOfsB4/wC64KPs</latexit>

grou

ndtrut

h

<latexit sha1_base64="R2mF6pUEVm/Hl9SkOslHTsj0QTo=">AAAB/nicdVDLSsNAFJ34rPUVFVduBovgqiRV+tgV3bisYB/QhjKZTtqhk0mYuRFLKPgrblwo4tbvcOffOG0jqOiBC4dz7uXee/xYcA2O82EtLa+srq3nNvKbW9s7u/befktHiaKsSSMRqY5PNBNcsiZwEKwTK0ZCX7C2P76c+e1bpjSP5A1MYuaFZCh5wCkBI/Xtwx6wO/CDdKiiRA4wqARG075dcIrlcs11XewUnTkMqZ7VKu45djOlgDI0+vZ7bxDRJGQSqCBad10nBi8lCjgVbJrvJZrFhI7JkHUNlSRk2kvn50/xiVEGOIiUKQl4rn6fSEmo9ST0TWdIYKR/ezPxL6+bQFD1Ui7jBJiki0VBIjBEeJYFHnDFKIiJIYQqbm7FdEQUoWASy5sQvj7F/5NWqegafl0q1C+yOHLoCB2jU+SiCqqjK9RATURRih7QE3q27q1H68V6XbQuWdnMAfoB6+0Tgn6Weg==</latexit><latexit sha1_base64="R2mF6pUEVm/Hl9SkOslHTsj0QTo=">AAAB/nicdVDLSsNAFJ34rPUVFVduBovgqiRV+tgV3bisYB/QhjKZTtqhk0mYuRFLKPgrblwo4tbvcOffOG0jqOiBC4dz7uXee/xYcA2O82EtLa+srq3nNvKbW9s7u/befktHiaKsSSMRqY5PNBNcsiZwEKwTK0ZCX7C2P76c+e1bpjSP5A1MYuaFZCh5wCkBI/Xtwx6wO/CDdKiiRA4wqARG075dcIrlcs11XewUnTkMqZ7VKu45djOlgDI0+vZ7bxDRJGQSqCBad10nBi8lCjgVbJrvJZrFhI7JkHUNlSRk2kvn50/xiVEGOIiUKQl4rn6fSEmo9ST0TWdIYKR/ezPxL6+bQFD1Ui7jBJiki0VBIjBEeJYFHnDFKIiJIYQqbm7FdEQUoWASy5sQvj7F/5NWqegafl0q1C+yOHLoCB2jU+SiCqqjK9RATURRih7QE3q27q1H68V6XbQuWdnMAfoB6+0Tgn6Weg==</latexit><latexit sha1_base64="R2mF6pUEVm/Hl9SkOslHTsj0QTo=">AAAB/nicdVDLSsNAFJ34rPUVFVduBovgqiRV+tgV3bisYB/QhjKZTtqhk0mYuRFLKPgrblwo4tbvcOffOG0jqOiBC4dz7uXee/xYcA2O82EtLa+srq3nNvKbW9s7u/befktHiaKsSSMRqY5PNBNcsiZwEKwTK0ZCX7C2P76c+e1bpjSP5A1MYuaFZCh5wCkBI/Xtwx6wO/CDdKiiRA4wqARG075dcIrlcs11XewUnTkMqZ7VKu45djOlgDI0+vZ7bxDRJGQSqCBad10nBi8lCjgVbJrvJZrFhI7JkHUNlSRk2kvn50/xiVEGOIiUKQl4rn6fSEmo9ST0TWdIYKR/ezPxL6+bQFD1Ui7jBJiki0VBIjBEeJYFHnDFKIiJIYQqbm7FdEQUoWASy5sQvj7F/5NWqegafl0q1C+yOHLoCB2jU+SiCqqjK9RATURRih7QE3q27q1H68V6XbQuWdnMAfoB6+0Tgn6Weg==</latexit><latexit sha1_base64="R2mF6pUEVm/Hl9SkOslHTsj0QTo=">AAAB/nicdVDLSsNAFJ34rPUVFVduBovgqiRV+tgV3bisYB/QhjKZTtqhk0mYuRFLKPgrblwo4tbvcOffOG0jqOiBC4dz7uXee/xYcA2O82EtLa+srq3nNvKbW9s7u/befktHiaKsSSMRqY5PNBNcsiZwEKwTK0ZCX7C2P76c+e1bpjSP5A1MYuaFZCh5wCkBI/Xtwx6wO/CDdKiiRA4wqARG075dcIrlcs11XewUnTkMqZ7VKu45djOlgDI0+vZ7bxDRJGQSqCBad10nBi8lCjgVbJrvJZrFhI7JkHUNlSRk2kvn50/xiVEGOIiUKQl4rn6fSEmo9ST0TWdIYKR/ezPxL6+bQFD1Ui7jBJiki0VBIjBEeJYFHnDFKIiJIYQqbm7FdEQUoWASy5sQvj7F/5NWqegafl0q1C+yOHLoCB2jU+SiCqqjK9RATURRih7QE3q27q1H68V6XbQuWdnMAfoB6+0Tgn6Weg==</latexit>

(a) Taking Photo

Time t (ms)<latexit sha1_base64="0iRWuoxCJXKv3/6DSQvdhksfUII=">AAAB9HicbZDLTgIxFIY7eEO8oS7dNIIJbsgMG10S3bjEhFsCE9IpZ6Ch7Yxth4RMeA43LjTGrQ/jzrexwCwU/JMmX/5zTs7pH8ScaeO6305ua3tndy+/Xzg4PDo+KZ6etXWUKAotGvFIdQOigTMJLcMMh26sgIiAQyeY3C/qnSkozSLZNLMYfEFGkoWMEmMtv8kE4LIp44rQ14Niya26S+FN8DIooUyNQfGrP4xoIkAayonWPc+NjZ8SZRjlMC/0Ew0xoRMygp5FSQRoP10ePcdX1hniMFL2SYOX7u+JlAitZyKwnYKYsV6vLcz/ar3EhLd+ymScGJB0tShMODYRXiSAh0wBNXxmgVDF7K2Yjoki1NicCjYEb/3Lm9CuVT3Lj7VS/S6LI48u0CWqIA/doDp6QA3UQhQ9oWf0it6cqfPivDsfq9ack82coz9yPn8ArDeQtg==</latexit><latexit sha1_base64="0iRWuoxCJXKv3/6DSQvdhksfUII=">AAAB9HicbZDLTgIxFIY7eEO8oS7dNIIJbsgMG10S3bjEhFsCE9IpZ6Ch7Yxth4RMeA43LjTGrQ/jzrexwCwU/JMmX/5zTs7pH8ScaeO6305ua3tndy+/Xzg4PDo+KZ6etXWUKAotGvFIdQOigTMJLcMMh26sgIiAQyeY3C/qnSkozSLZNLMYfEFGkoWMEmMtv8kE4LIp44rQ14Niya26S+FN8DIooUyNQfGrP4xoIkAayonWPc+NjZ8SZRjlMC/0Ew0xoRMygp5FSQRoP10ePcdX1hniMFL2SYOX7u+JlAitZyKwnYKYsV6vLcz/ar3EhLd+ymScGJB0tShMODYRXiSAh0wBNXxmgVDF7K2Yjoki1NicCjYEb/3Lm9CuVT3Lj7VS/S6LI48u0CWqIA/doDp6QA3UQhQ9oWf0it6cqfPivDsfq9ack82coz9yPn8ArDeQtg==</latexit><latexit sha1_base64="0iRWuoxCJXKv3/6DSQvdhksfUII=">AAAB9HicbZDLTgIxFIY7eEO8oS7dNIIJbsgMG10S3bjEhFsCE9IpZ6Ch7Yxth4RMeA43LjTGrQ/jzrexwCwU/JMmX/5zTs7pH8ScaeO6305ua3tndy+/Xzg4PDo+KZ6etXWUKAotGvFIdQOigTMJLcMMh26sgIiAQyeY3C/qnSkozSLZNLMYfEFGkoWMEmMtv8kE4LIp44rQ14Niya26S+FN8DIooUyNQfGrP4xoIkAayonWPc+NjZ8SZRjlMC/0Ew0xoRMygp5FSQRoP10ePcdX1hniMFL2SYOX7u+JlAitZyKwnYKYsV6vLcz/ar3EhLd+ymScGJB0tShMODYRXiSAh0wBNXxmgVDF7K2Yjoki1NicCjYEb/3Lm9CuVT3Lj7VS/S6LI48u0CWqIA/doDp6QA3UQhQ9oWf0it6cqfPivDsfq9ack82coz9yPn8ArDeQtg==</latexit><latexit sha1_base64="0iRWuoxCJXKv3/6DSQvdhksfUII=">AAAB9HicbZDLTgIxFIY7eEO8oS7dNIIJbsgMG10S3bjEhFsCE9IpZ6Ch7Yxth4RMeA43LjTGrQ/jzrexwCwU/JMmX/5zTs7pH8ScaeO6305ua3tndy+/Xzg4PDo+KZ6etXWUKAotGvFIdQOigTMJLcMMh26sgIiAQyeY3C/qnSkozSLZNLMYfEFGkoWMEmMtv8kE4LIp44rQ14Niya26S+FN8DIooUyNQfGrP4xoIkAayonWPc+NjZ8SZRjlMC/0Ew0xoRMygp5FSQRoP10ePcdX1hniMFL2SYOX7u+JlAitZyKwnYKYsV6vLcz/ar3EhLd+ymScGJB0tShMODYRXiSAh0wBNXxmgVDF7K2Yjoki1NicCjYEb/3Lm9CuVT3Lj7VS/S6LI48u0CWqIA/doDp6QA3UQhQ9oWf0it6cqfPivDsfq9ack82coz9yPn8ArDeQtg==</latexit>

1200ms<latexit sha1_base64="VzwQbjr7l4r06rBxahZ9FEpT/1A=">AAAB7XicbZC7SgNBFIbPxluMt1VLm8EgWIXdNFoGbSwjmAskS5idzCZj5rLMzAphyTvYWChi6/vY+TZOki008YeBj/+cw5zzxylnxgbBt1fa2Nza3invVvb2Dw6P/OOTtlGZJrRFFFe6G2NDOZO0ZZnltJtqikXMaSee3M7rnSeqDVPywU5TGgk8kixhBFtntcN6EAgz8KtBLVgIrUNYQBUKNQf+V3+oSCaotIRjY3phkNoox9oywums0s8MTTGZ4BHtOZRYUBPli21n6MI5Q5Qo7Z60aOH+nsixMGYqYtcpsB2b1drc/K/Wy2xyHeVMppmlkiw/SjKOrELz09GQaUosnzrARDO3KyJjrDGxLqCKCyFcPXkd2vVa6Pi+Xm3cFHGU4QzO4RJCuIIG3EETWkDgEZ7hFd485b14797HsrXkFTOn8Efe5w9mlo5X</latexit><latexit sha1_base64="VzwQbjr7l4r06rBxahZ9FEpT/1A=">AAAB7XicbZC7SgNBFIbPxluMt1VLm8EgWIXdNFoGbSwjmAskS5idzCZj5rLMzAphyTvYWChi6/vY+TZOki008YeBj/+cw5zzxylnxgbBt1fa2Nza3invVvb2Dw6P/OOTtlGZJrRFFFe6G2NDOZO0ZZnltJtqikXMaSee3M7rnSeqDVPywU5TGgk8kixhBFtntcN6EAgz8KtBLVgIrUNYQBUKNQf+V3+oSCaotIRjY3phkNoox9oywums0s8MTTGZ4BHtOZRYUBPli21n6MI5Q5Qo7Z60aOH+nsixMGYqYtcpsB2b1drc/K/Wy2xyHeVMppmlkiw/SjKOrELz09GQaUosnzrARDO3KyJjrDGxLqCKCyFcPXkd2vVa6Pi+Xm3cFHGU4QzO4RJCuIIG3EETWkDgEZ7hFd485b14797HsrXkFTOn8Efe5w9mlo5X</latexit><latexit sha1_base64="VzwQbjr7l4r06rBxahZ9FEpT/1A=">AAAB7XicbZC7SgNBFIbPxluMt1VLm8EgWIXdNFoGbSwjmAskS5idzCZj5rLMzAphyTvYWChi6/vY+TZOki008YeBj/+cw5zzxylnxgbBt1fa2Nza3invVvb2Dw6P/OOTtlGZJrRFFFe6G2NDOZO0ZZnltJtqikXMaSee3M7rnSeqDVPywU5TGgk8kixhBFtntcN6EAgz8KtBLVgIrUNYQBUKNQf+V3+oSCaotIRjY3phkNoox9oywums0s8MTTGZ4BHtOZRYUBPli21n6MI5Q5Qo7Z60aOH+nsixMGYqYtcpsB2b1drc/K/Wy2xyHeVMppmlkiw/SjKOrELz09GQaUosnzrARDO3KyJjrDGxLqCKCyFcPXkd2vVa6Pi+Xm3cFHGU4QzO4RJCuIIG3EETWkDgEZ7hFd485b14797HsrXkFTOn8Efe5w9mlo5X</latexit><latexit sha1_base64="VzwQbjr7l4r06rBxahZ9FEpT/1A=">AAAB7XicbZC7SgNBFIbPxluMt1VLm8EgWIXdNFoGbSwjmAskS5idzCZj5rLMzAphyTvYWChi6/vY+TZOki008YeBj/+cw5zzxylnxgbBt1fa2Nza3invVvb2Dw6P/OOTtlGZJrRFFFe6G2NDOZO0ZZnltJtqikXMaSee3M7rnSeqDVPywU5TGgk8kixhBFtntcN6EAgz8KtBLVgIrUNYQBUKNQf+V3+oSCaotIRjY3phkNoox9oywums0s8MTTGZ4BHtOZRYUBPli21n6MI5Q5Qo7Z60aOH+nsixMGYqYtcpsB2b1drc/K/Wy2xyHeVMppmlkiw/SjKOrELz09GQaUosnzrARDO3KyJjrDGxLqCKCyFcPXkd2vVa6Pi+Xm3cFHGU4QzO4RJCuIIG3EETWkDgEZ7hFd485b14797HsrXkFTOn8Efe5w9mlo5X</latexit>

0ms<latexit sha1_base64="ffCJWAy1UH6fgsnnOY4PpZr+Vy4=">AAAB6nicbZBNSwMxEIZn61etX1WPXoJF8FR2e9Fj0YvHivYD2qVk09k2NMkuSVYopT/BiwdFvPqLvPlvTNs9aOsLgYd3ZsjMG6WCG+v7315hY3Nre6e4W9rbPzg8Kh+ftEySaYZNlohEdyJqUHCFTcutwE6qkcpIYDsa387r7SfUhifq0U5SDCUdKh5zRq2zHnxp+uWKX/UXIusQ5FCBXI1++as3SFgmUVkmqDHdwE9tOKXaciZwVuplBlPKxnSIXYeKSjThdLHqjFw4Z0DiRLunLFm4vyemVBozkZHrlNSOzGptbv5X62Y2vg6nXKWZRcWWH8WZIDYh87vJgGtkVkwcUKa525WwEdWUWZdOyYUQrJ68Dq1aNXB8X6vUb/I4inAG53AJAVxBHe6gAU1gMIRneIU3T3gv3rv3sWwtePnMKfyR9/kDFrCNpg==</latexit><latexit sha1_base64="ffCJWAy1UH6fgsnnOY4PpZr+Vy4=">AAAB6nicbZBNSwMxEIZn61etX1WPXoJF8FR2e9Fj0YvHivYD2qVk09k2NMkuSVYopT/BiwdFvPqLvPlvTNs9aOsLgYd3ZsjMG6WCG+v7315hY3Nre6e4W9rbPzg8Kh+ftEySaYZNlohEdyJqUHCFTcutwE6qkcpIYDsa387r7SfUhifq0U5SDCUdKh5zRq2zHnxp+uWKX/UXIusQ5FCBXI1++as3SFgmUVkmqDHdwE9tOKXaciZwVuplBlPKxnSIXYeKSjThdLHqjFw4Z0DiRLunLFm4vyemVBozkZHrlNSOzGptbv5X62Y2vg6nXKWZRcWWH8WZIDYh87vJgGtkVkwcUKa525WwEdWUWZdOyYUQrJ68Dq1aNXB8X6vUb/I4inAG53AJAVxBHe6gAU1gMIRneIU3T3gv3rv3sWwtePnMKfyR9/kDFrCNpg==</latexit><latexit sha1_base64="ffCJWAy1UH6fgsnnOY4PpZr+Vy4=">AAAB6nicbZBNSwMxEIZn61etX1WPXoJF8FR2e9Fj0YvHivYD2qVk09k2NMkuSVYopT/BiwdFvPqLvPlvTNs9aOsLgYd3ZsjMG6WCG+v7315hY3Nre6e4W9rbPzg8Kh+ftEySaYZNlohEdyJqUHCFTcutwE6qkcpIYDsa387r7SfUhifq0U5SDCUdKh5zRq2zHnxp+uWKX/UXIusQ5FCBXI1++as3SFgmUVkmqDHdwE9tOKXaciZwVuplBlPKxnSIXYeKSjThdLHqjFw4Z0DiRLunLFm4vyemVBozkZHrlNSOzGptbv5X62Y2vg6nXKWZRcWWH8WZIDYh87vJgGtkVkwcUKa525WwEdWUWZdOyYUQrJ68Dq1aNXB8X6vUb/I4inAG53AJAVxBHe6gAU1gMIRneIU3T3gv3rv3sWwtePnMKfyR9/kDFrCNpg==</latexit><latexit sha1_base64="ffCJWAy1UH6fgsnnOY4PpZr+Vy4=">AAAB6nicbZBNSwMxEIZn61etX1WPXoJF8FR2e9Fj0YvHivYD2qVk09k2NMkuSVYopT/BiwdFvPqLvPlvTNs9aOsLgYd3ZsjMG6WCG+v7315hY3Nre6e4W9rbPzg8Kh+ftEySaYZNlohEdyJqUHCFTcutwE6qkcpIYDsa387r7SfUhifq0U5SDCUdKh5zRq2zHnxp+uWKX/UXIusQ5FCBXI1++as3SFgmUVkmqDHdwE9tOKXaciZwVuplBlPKxnSIXYeKSjThdLHqjFw4Z0DiRLunLFm4vyemVBozkZHrlNSOzGptbv5X62Y2vg6nXKWZRcWWH8WZIDYh87vJgGtkVkwcUKa525WwEdWUWZdOyYUQrJ68Dq1aNXB8X6vUb/I4inAG53AJAVxBHe6gAU1gMIRneIU3T3gv3rv3sWwtePnMKfyR9/kDFrCNpg==</latexit>

Start<latexit sha1_base64="Y2CFThq7wFBLnY/B2B9DRC9MAW0=">AAAB7HicbVBNT8JAEJ3iF+IX6tHLRmLiibRc9Ej04hGjBRJoyHbZwobtttmdmpCG3+DFg8Z49Qd589+4QA8KvmSTl/dmZmdemEph0HW/ndLG5tb2Tnm3srd/cHhUPT5pmyTTjPsskYnuhtRwKRT3UaDk3VRzGoeSd8LJ7dzvPHFtRKIecZryIKYjJSLBKFrJf0CqcVCtuXV3AbJOvILUoEBrUP3qDxOWxVwhk9SYnuemGOR2lGCSzyr9zPCUsgkd8Z6lisbcBPli2Rm5sMqQRIm2TyFZqL87chobM41DWxlTHJtVby7+5/UyjK6DXKg0Q67Y8qMokwQTMr+cDIXmDOXUEsq0sLsSNqaaMrT5VGwI3urJ66TdqHuW3zdqzZsijjKcwTlcggdX0IQ7aIEPDAQ8wyu8Ocp5cd6dj2VpySl6TuEPnM8f5JiOuA==</latexit><latexit sha1_base64="Y2CFThq7wFBLnY/B2B9DRC9MAW0=">AAAB7HicbVBNT8JAEJ3iF+IX6tHLRmLiibRc9Ej04hGjBRJoyHbZwobtttmdmpCG3+DFg8Z49Qd589+4QA8KvmSTl/dmZmdemEph0HW/ndLG5tb2Tnm3srd/cHhUPT5pmyTTjPsskYnuhtRwKRT3UaDk3VRzGoeSd8LJ7dzvPHFtRKIecZryIKYjJSLBKFrJf0CqcVCtuXV3AbJOvILUoEBrUP3qDxOWxVwhk9SYnuemGOR2lGCSzyr9zPCUsgkd8Z6lisbcBPli2Rm5sMqQRIm2TyFZqL87chobM41DWxlTHJtVby7+5/UyjK6DXKg0Q67Y8qMokwQTMr+cDIXmDOXUEsq0sLsSNqaaMrT5VGwI3urJ66TdqHuW3zdqzZsijjKcwTlcggdX0IQ7aIEPDAQ8wyu8Ocp5cd6dj2VpySl6TuEPnM8f5JiOuA==</latexit><latexit sha1_base64="Y2CFThq7wFBLnY/B2B9DRC9MAW0=">AAAB7HicbVBNT8JAEJ3iF+IX6tHLRmLiibRc9Ej04hGjBRJoyHbZwobtttmdmpCG3+DFg8Z49Qd589+4QA8KvmSTl/dmZmdemEph0HW/ndLG5tb2Tnm3srd/cHhUPT5pmyTTjPsskYnuhtRwKRT3UaDk3VRzGoeSd8LJ7dzvPHFtRKIecZryIKYjJSLBKFrJf0CqcVCtuXV3AbJOvILUoEBrUP3qDxOWxVwhk9SYnuemGOR2lGCSzyr9zPCUsgkd8Z6lisbcBPli2Rm5sMqQRIm2TyFZqL87chobM41DWxlTHJtVby7+5/UyjK6DXKg0Q67Y8qMokwQTMr+cDIXmDOXUEsq0sLsSNqaaMrT5VGwI3urJ66TdqHuW3zdqzZsijjKcwTlcggdX0IQ7aIEPDAQ8wyu8Ocp5cd6dj2VpySl6TuEPnM8f5JiOuA==</latexit><latexit sha1_base64="Y2CFThq7wFBLnY/B2B9DRC9MAW0=">AAAB7HicbVBNT8JAEJ3iF+IX6tHLRmLiibRc9Ej04hGjBRJoyHbZwobtttmdmpCG3+DFg8Z49Qd589+4QA8KvmSTl/dmZmdemEph0HW/ndLG5tb2Tnm3srd/cHhUPT5pmyTTjPsskYnuhtRwKRT3UaDk3VRzGoeSd8LJ7dzvPHFtRKIecZryIKYjJSLBKFrJf0CqcVCtuXV3AbJOvILUoEBrUP3qDxOWxVwhk9SYnuemGOR2lGCSzyr9zPCUsgkd8Z6lisbcBPli2Rm5sMqQRIm2TyFZqL87chobM41DWxlTHJtVby7+5/UyjK6DXKg0Q67Y8qMokwQTMr+cDIXmDOXUEsq0sLsSNqaaMrT5VGwI3urJ66TdqHuW3zdqzZsijjKcwTlcggdX0IQ7aIEPDAQ8wyu8Ocp5cd6dj2VpySl6TuEPnM8f5JiOuA==</latexit>

Residua

l

<latexit sha1_base64="FVAfnrGSn2tNroBd85q8yKklBAQ=">AAAB+nicdVDLSgNBEJz1bXytevQyGARPYTY5JN6CXjxGMVFIljA76dUhsw9metWw5lO8eFDEq1/izb9xEldQ0YKGoqqb7q4gVdIgY+/OzOzc/MLi0nJpZXVtfcPd3OqYJNMC2iJRib4IuAElY2ijRAUXqQYeBQrOg+HRxD+/Bm1kEp/hKAU/4pexDKXgaKW+u9lDuMUgzE/ByEHG1bjvllmFTUFZpeHVa6xhSY0d1FmVeoVVJgVaffetN0hEFkGMQnFjuh5L0c+5RikUjEu9zEDKxZBfQtfSmEdg/Hx6+pjuWWVAw0TbipFO1e8TOY+MGUWB7Yw4Xpnf3kT8y+tmGDb8XMZphhCLz0VhpigmdJIDHUgNAtXIEi60tLdSccU1F2jTKtkQvj6l/5NOteJZflItNw+LOJbIDtkl+8QjddIkx6RF2kSQG3JPHsmTc+c8OM/Oy2frjFPMbJMfcF4/ADgolJ0=</latexit><latexit sha1_base64="FVAfnrGSn2tNroBd85q8yKklBAQ=">AAAB+nicdVDLSgNBEJz1bXytevQyGARPYTY5JN6CXjxGMVFIljA76dUhsw9metWw5lO8eFDEq1/izb9xEldQ0YKGoqqb7q4gVdIgY+/OzOzc/MLi0nJpZXVtfcPd3OqYJNMC2iJRib4IuAElY2ijRAUXqQYeBQrOg+HRxD+/Bm1kEp/hKAU/4pexDKXgaKW+u9lDuMUgzE/ByEHG1bjvllmFTUFZpeHVa6xhSY0d1FmVeoVVJgVaffetN0hEFkGMQnFjuh5L0c+5RikUjEu9zEDKxZBfQtfSmEdg/Hx6+pjuWWVAw0TbipFO1e8TOY+MGUWB7Yw4Xpnf3kT8y+tmGDb8XMZphhCLz0VhpigmdJIDHUgNAtXIEi60tLdSccU1F2jTKtkQvj6l/5NOteJZflItNw+LOJbIDtkl+8QjddIkx6RF2kSQG3JPHsmTc+c8OM/Oy2frjFPMbJMfcF4/ADgolJ0=</latexit><latexit sha1_base64="FVAfnrGSn2tNroBd85q8yKklBAQ=">AAAB+nicdVDLSgNBEJz1bXytevQyGARPYTY5JN6CXjxGMVFIljA76dUhsw9metWw5lO8eFDEq1/izb9xEldQ0YKGoqqb7q4gVdIgY+/OzOzc/MLi0nJpZXVtfcPd3OqYJNMC2iJRib4IuAElY2ijRAUXqQYeBQrOg+HRxD+/Bm1kEp/hKAU/4pexDKXgaKW+u9lDuMUgzE/ByEHG1bjvllmFTUFZpeHVa6xhSY0d1FmVeoVVJgVaffetN0hEFkGMQnFjuh5L0c+5RikUjEu9zEDKxZBfQtfSmEdg/Hx6+pjuWWVAw0TbipFO1e8TOY+MGUWB7Yw4Xpnf3kT8y+tmGDb8XMZphhCLz0VhpigmdJIDHUgNAtXIEi60tLdSccU1F2jTKtkQvj6l/5NOteJZflItNw+LOJbIDtkl+8QjddIkx6RF2kSQG3JPHsmTc+c8OM/Oy2frjFPMbJMfcF4/ADgolJ0=</latexit><latexit sha1_base64="FVAfnrGSn2tNroBd85q8yKklBAQ=">AAAB+nicdVDLSgNBEJz1bXytevQyGARPYTY5JN6CXjxGMVFIljA76dUhsw9metWw5lO8eFDEq1/izb9xEldQ0YKGoqqb7q4gVdIgY+/OzOzc/MLi0nJpZXVtfcPd3OqYJNMC2iJRib4IuAElY2ijRAUXqQYeBQrOg+HRxD+/Bm1kEp/hKAU/4pexDKXgaKW+u9lDuMUgzE/ByEHG1bjvllmFTUFZpeHVa6xhSY0d1FmVeoVVJgVaffetN0hEFkGMQnFjuh5L0c+5RikUjEu9zEDKxZBfQtfSmEdg/Hx6+pjuWWVAw0TbipFO1e8TOY+MGUWB7Yw4Xpnf3kT8y+tmGDb8XMZphhCLz0VhpigmdJIDHUgNAtXIEi60tLdSccU1F2jTKtkQvj6l/5NOteJZflItNw+LOJbIDtkl+8QjddIkx6RF2kSQG3JPHsmTc+c8OM/Oy2frjFPMbJMfcF4/ADgolJ0=</latexit>

BC+W

GAIL

-div

(Our

s)

<latexit sha1_base64="oTg7XQbqoIttD9hNncluhs8k/zo=">AAACIHicdVBNS8MwGE7n15xfU49eikNQxNFOZR7ndlBBcILbhHWMNH27haVpSdLhKPspXvwrXjwoojf9NWZzgoo+EHh4PkjyuBGjUlnWm5Gamp6ZnUvPZxYWl5ZXsqtrdRnGgkCNhCwU1y6WwCiHmqKKwXUkAAcug4bbq4z8Rh+EpCG/UoMIWgHucOpTgpWW2tmi40KH8oQAVyCGjoIb5fpJubLbODk+O9/zaN9xti9iIXeGDnDvK9jO5uy8NYZp5fcP7QOroMlE+bJyaIJqO/vqeCGJA10nDEvZtK1ItRIsFCUMhhknlhBh0sMdaGrKcQCylYw/ODS3tOKZfij04cocq98bCQ6kHASuTgZYdeVvbyT+5TVj5R+1EsqjWAEnnxf5MTNVaI7WMj0qgCg20AQTQfVbTdLFAhO9gcx8H+F/Ui/kbc0vC7lSeTJHGm2gTbSNbFREJXSKqqiGCLpF9+gRPRl3xoPxbLx8RlPGpLOOfsB4/wC64KPs</latexit><latexit sha1_base64="oTg7XQbqoIttD9hNncluhs8k/zo=">AAACIHicdVBNS8MwGE7n15xfU49eikNQxNFOZR7ndlBBcILbhHWMNH27haVpSdLhKPspXvwrXjwoojf9NWZzgoo+EHh4PkjyuBGjUlnWm5Gamp6ZnUvPZxYWl5ZXsqtrdRnGgkCNhCwU1y6WwCiHmqKKwXUkAAcug4bbq4z8Rh+EpCG/UoMIWgHucOpTgpWW2tmi40KH8oQAVyCGjoIb5fpJubLbODk+O9/zaN9xti9iIXeGDnDvK9jO5uy8NYZp5fcP7QOroMlE+bJyaIJqO/vqeCGJA10nDEvZtK1ItRIsFCUMhhknlhBh0sMdaGrKcQCylYw/ODS3tOKZfij04cocq98bCQ6kHASuTgZYdeVvbyT+5TVj5R+1EsqjWAEnnxf5MTNVaI7WMj0qgCg20AQTQfVbTdLFAhO9gcx8H+F/Ui/kbc0vC7lSeTJHGm2gTbSNbFREJXSKqqiGCLpF9+gRPRl3xoPxbLx8RlPGpLOOfsB4/wC64KPs</latexit><latexit sha1_base64="oTg7XQbqoIttD9hNncluhs8k/zo=">AAACIHicdVBNS8MwGE7n15xfU49eikNQxNFOZR7ndlBBcILbhHWMNH27haVpSdLhKPspXvwrXjwoojf9NWZzgoo+EHh4PkjyuBGjUlnWm5Gamp6ZnUvPZxYWl5ZXsqtrdRnGgkCNhCwU1y6WwCiHmqKKwXUkAAcug4bbq4z8Rh+EpCG/UoMIWgHucOpTgpWW2tmi40KH8oQAVyCGjoIb5fpJubLbODk+O9/zaN9xti9iIXeGDnDvK9jO5uy8NYZp5fcP7QOroMlE+bJyaIJqO/vqeCGJA10nDEvZtK1ItRIsFCUMhhknlhBh0sMdaGrKcQCylYw/ODS3tOKZfij04cocq98bCQ6kHASuTgZYdeVvbyT+5TVj5R+1EsqjWAEnnxf5MTNVaI7WMj0qgCg20AQTQfVbTdLFAhO9gcx8H+F/Ui/kbc0vC7lSeTJHGm2gTbSNbFREJXSKqqiGCLpF9+gRPRl3xoPxbLx8RlPGpLOOfsB4/wC64KPs</latexit><latexit sha1_base64="oTg7XQbqoIttD9hNncluhs8k/zo=">AAACIHicdVBNS8MwGE7n15xfU49eikNQxNFOZR7ndlBBcILbhHWMNH27haVpSdLhKPspXvwrXjwoojf9NWZzgoo+EHh4PkjyuBGjUlnWm5Gamp6ZnUvPZxYWl5ZXsqtrdRnGgkCNhCwU1y6WwCiHmqKKwXUkAAcug4bbq4z8Rh+EpCG/UoMIWgHucOpTgpWW2tmi40KH8oQAVyCGjoIb5fpJubLbODk+O9/zaN9xti9iIXeGDnDvK9jO5uy8NYZp5fcP7QOroMlE+bJyaIJqO/vqeCGJA10nDEvZtK1ItRIsFCUMhhknlhBh0sMdaGrKcQCylYw/ODS3tOKZfij04cocq98bCQ6kHASuTgZYdeVvbyT+5TVj5R+1EsqjWAEnnxf5MTNVaI7WMj0qgCg20AQTQfVbTdLFAhO9gcx8H+F/Ui/kbc0vC7lSeTJHGm2gTbSNbFREJXSKqqiGCLpF9+gRPRl3xoPxbLx8RlPGpLOOfsB4/wC64KPs</latexit>

grou

ndtrut

h

<latexit sha1_base64="R2mF6pUEVm/Hl9SkOslHTsj0QTo=">AAAB/nicdVDLSsNAFJ34rPUVFVduBovgqiRV+tgV3bisYB/QhjKZTtqhk0mYuRFLKPgrblwo4tbvcOffOG0jqOiBC4dz7uXee/xYcA2O82EtLa+srq3nNvKbW9s7u/befktHiaKsSSMRqY5PNBNcsiZwEKwTK0ZCX7C2P76c+e1bpjSP5A1MYuaFZCh5wCkBI/Xtwx6wO/CDdKiiRA4wqARG075dcIrlcs11XewUnTkMqZ7VKu45djOlgDI0+vZ7bxDRJGQSqCBad10nBi8lCjgVbJrvJZrFhI7JkHUNlSRk2kvn50/xiVEGOIiUKQl4rn6fSEmo9ST0TWdIYKR/ezPxL6+bQFD1Ui7jBJiki0VBIjBEeJYFHnDFKIiJIYQqbm7FdEQUoWASy5sQvj7F/5NWqegafl0q1C+yOHLoCB2jU+SiCqqjK9RATURRih7QE3q27q1H68V6XbQuWdnMAfoB6+0Tgn6Weg==</latexit><latexit sha1_base64="R2mF6pUEVm/Hl9SkOslHTsj0QTo=">AAAB/nicdVDLSsNAFJ34rPUVFVduBovgqiRV+tgV3bisYB/QhjKZTtqhk0mYuRFLKPgrblwo4tbvcOffOG0jqOiBC4dz7uXee/xYcA2O82EtLa+srq3nNvKbW9s7u/befktHiaKsSSMRqY5PNBNcsiZwEKwTK0ZCX7C2P76c+e1bpjSP5A1MYuaFZCh5wCkBI/Xtwx6wO/CDdKiiRA4wqARG075dcIrlcs11XewUnTkMqZ7VKu45djOlgDI0+vZ7bxDRJGQSqCBad10nBi8lCjgVbJrvJZrFhI7JkHUNlSRk2kvn50/xiVEGOIiUKQl4rn6fSEmo9ST0TWdIYKR/ezPxL6+bQFD1Ui7jBJiki0VBIjBEeJYFHnDFKIiJIYQqbm7FdEQUoWASy5sQvj7F/5NWqegafl0q1C+yOHLoCB2jU+SiCqqjK9RATURRih7QE3q27q1H68V6XbQuWdnMAfoB6+0Tgn6Weg==</latexit><latexit sha1_base64="R2mF6pUEVm/Hl9SkOslHTsj0QTo=">AAAB/nicdVDLSsNAFJ34rPUVFVduBovgqiRV+tgV3bisYB/QhjKZTtqhk0mYuRFLKPgrblwo4tbvcOffOG0jqOiBC4dz7uXee/xYcA2O82EtLa+srq3nNvKbW9s7u/befktHiaKsSSMRqY5PNBNcsiZwEKwTK0ZCX7C2P76c+e1bpjSP5A1MYuaFZCh5wCkBI/Xtwx6wO/CDdKiiRA4wqARG075dcIrlcs11XewUnTkMqZ7VKu45djOlgDI0+vZ7bxDRJGQSqCBad10nBi8lCjgVbJrvJZrFhI7JkHUNlSRk2kvn50/xiVEGOIiUKQl4rn6fSEmo9ST0TWdIYKR/ezPxL6+bQFD1Ui7jBJiki0VBIjBEeJYFHnDFKIiJIYQqbm7FdEQUoWASy5sQvj7F/5NWqegafl0q1C+yOHLoCB2jU+SiCqqjK9RATURRih7QE3q27q1H68V6XbQuWdnMAfoB6+0Tgn6Weg==</latexit><latexit sha1_base64="R2mF6pUEVm/Hl9SkOslHTsj0QTo=">AAAB/nicdVDLSsNAFJ34rPUVFVduBovgqiRV+tgV3bisYB/QhjKZTtqhk0mYuRFLKPgrblwo4tbvcOffOG0jqOiBC4dz7uXee/xYcA2O82EtLa+srq3nNvKbW9s7u/befktHiaKsSSMRqY5PNBNcsiZwEKwTK0ZCX7C2P76c+e1bpjSP5A1MYuaFZCh5wCkBI/Xtwx6wO/CDdKiiRA4wqARG075dcIrlcs11XewUnTkMqZ7VKu45djOlgDI0+vZ7bxDRJGQSqCBad10nBi8lCjgVbJrvJZrFhI7JkHUNlSRk2kvn50/xiVEGOIiUKQl4rn6fSEmo9ST0TWdIYKR/ezPxL6+bQFD1Ui7jBJiki0VBIjBEeJYFHnDFKIiJIYQqbm7FdEQUoWASy5sQvj7F/5NWqegafl0q1C+yOHLoCB2jU+SiCqqjK9RATURRih7QE3q27q1H68V6XbQuWdnMAfoB6+0Tgn6Weg==</latexit>

(b) Walking

Figure 7: Visualization of long-term human pose predictionresults for taking photo (top) and walking (bottom). Thepurple poses are ground truth, the green poses are predictedby the Residual [26] model and the orange poses are ours.

From the experimental results, we see that the predictionperformance of our proposed imitation learning approach(BC+WGAIL-div) surpasses all the baseline models by sig-nificant margins on almost all 15 human activity categoriesacross all different prediction timescales. Our prediction re-sults set the new state-of-the-art performance for the task ofpredicting human motion on the Human 3.6M dataset overboth short-term and long-term predictions.

Page 8: Imitation Learning for Human Pose Prediction › pdf › 1909.03449v1.pdf · Imitation Learning for Human Pose Prediction Borui Wang, Ehsan Adeli, Hsu-kuang Chiu, De-An Huang, Juan

Table 3: Mean Angle Error for the remaining 11 actions in Human 3.6M in the long-term prediction experiment. The bestresult in each column is highlighted with boldface.

Purchases Sitting Sitting down Taking photomillisec 80 160 320 400 560 1000 80 160 320 400 560 1000 80 160 320 400 560 1000 80 160 320 400 560 1000Residual 0.60 0.86 1.24 1.30 1.58 2.26 0.44 0.74 1.19 1.40 1.57 2.03 0.51 0.93 1.44 1.65 1.94 2.55 0.33 0.65 0.97 1.09 1.19 1.47TP-RNN 0.59 0.82 1.12 1.18 1.52 2.28 0.41 0.66 1.07 1.22 1.35 1.74 0.41 0.79 1.13 1.27 1.47 1.93 0.26 0.51 0.80 0.95 1.08 1.35Ours 0.54 0.78 1.07 1.14 1.46 2.23 0.29 0.48 0.87 1.04 1.21 1.58 0.34 0.68 1.01 1.14 1.34 1.78 0.17 0.37 0.60 0.72 0.84 1.06

Directions Greeting Talking on the phone Posingmillisec 80 160 320 400 560 1000 80 160 320 400 560 1000 80 160 320 400 560 1000 80 160 320 400 560 1000Residual 0.44 0.69 0.83 0.94 1.03 1.49 0.53 0.88 1.29 1.45 1.72 1.89 0.61 1.12 1.57 1.74 1.59 1.92 0.47 0.87 1.49 1.76 1.96 2.35TP-RNN 0.38 0.59 0.75 0.83 0.95 1.38 0.51 0.86 1.27 1.44 1.72 1.81 0.57 1.08 1.44 1.59 1.47 1.68 0.42 0.76 1.29 1.54 1.75 2.47Ours 0.27 0.46 0.81 0.89 0.95 1.41 0.43 0.75 1.17 1.33 1.62 1.72 0.54 1.05 1.40 1.56 1.52 1.67 0.27 0.55 1.16 1.41 1.83 2.69

Waiting Walking dog Walking together Average over all 15 actionsmillisec 80 160 320 400 560 1000 80 160 320 400 560 1000 80 160 320 400 560 1000 80 160 320 400 560 1000Residual 0.34 0.65 1.09 1.28 1.61 2.27 0.56 0.95 1.28 1.39 1.68 1.92 0.31 0.61 0.84 0.89 1.00 1.43 0.43 0.75 1.11 1.24 1.42 1.83TP-RNN 0.30 0.60 1.09 1.31 1.71 2.46 0.53 0.93 1.24 1.38 1.73 1.98 0.23 0.47 0.67 0.71 0.78 1.28 0.37 0.66 0.99 1.11 1.30 1.71Ours 0.25 0.53 0.96 1.19 1.59 2.39 0.46 0.80 1.12 1.31 1.65 1.86 0.19 0.41 0.61 0.64 0.67 1.15 0.31 0.57 0.90 1.02 1.23 1.65

Table 4: Ablation Study: Mean Angle Error of BehavioralCloning (BC) alone, WGAIL-div alone, and WGAIL-divwith Behavioral Cloning as pre-training (BC+WGAIL-div)averaged over the four activities walking, eating, smoking,and discussion in the human pose prediction experiment onthe Human 3.6M dataset. The best result in each column ishighlighted with boldface.

millisec 80 160 320 400 560 1000BC 0.23 0.43 0.72 0.78 0.96 1.36WGAIL-div 0.23 0.44 0.74 0.80 0.98 1.33BC+WGAIL-div 0.21 0.41 0.69 0.75 0.94 1.32

We also plot visualization of the long-term human poseprediction results in Figure 7 for two representative humanactivities: taking photo and walking. From the visualiza-tion we can see that the predictions made by our imitationlearning algorithm look very similar to the ground truth andare much better and more natural than the predictions madeby the baseline model Residual [26]. See Appendix A.3 formore visualization of our prediction results.Training Speed: During our experiments, we observedthat the training speed of our proposed imitation learningmethod is much faster than previous methods. In our exper-iments using 4 NVIDIA Titan GPUs, on average, our BC+ WGAIL-div algorithm finishes training within 20 min-utes, while the baseline models Residual and TP-RNN fin-ish training in more than 8 hours. There is at least a ×24speedup on training using our proposed method.

5.2. Ablation Study

Our proposed imitation learning algorithm is composedof two components: (1) a BC component (Algorithm 2 inAppendix A.1) as pretraining; and (2) a WGAIL-div com-ponent (Algorithm 1). In our ablation study, in order to testthe importance of each component, we train our human poseprediction agent on the Human 3.6M dataset [18] using: (a)

BC alone; (b) WGAIL-div alone; and (c) WGAIL-div withBC as pretraining. The results of our ablation study ex-periments are summarized in Table 4. As we can see fromTable 4, removing either BC or WGAIL-div from our imi-tation learning algorithm would worsen the prediction per-formance. Therefore, our ablation study shows that bothcomponents of our imitation learning algorithm are neces-sary in order to achieve optimal performance.

5.3. Discussion

In this work, all the pose predictions are solely basedon past pose observations. In our future work, we planto extend our work to tackle more sophisticated scenariosof human motion prediction. First, we plan to extend ourWGAIL-div framework to multi-agent WGAIL-div in or-der to take into account other humans in the surroundingswhen performing group human motion prediction. Second,we plan to introduce latent variables into WGAIL-div in or-der to effectively model other factors such as environments,objects, activities, and intentions that might also be involvedin human motion prediction.

6. Conclusion

In this work, we proposed a new reinforcement learn-ing formulation of the human pose prediction problem, anddeveloped an imitation learning algorithm for pose predic-tion under this new formulation. Our experiments showedthat our proposed method can effectively learn to make ac-curate human pose predictions over both short terms andlong terms with very fast training speed. Our work alsohas great potential to be generalized to other tasks of se-quential modeling and time-series predictions in computervision and machine learning, which will also be the direc-tion of our future work.Acknowledgments: We would like to thank Mindtree andPanasonic for their support.

Page 9: Imitation Learning for Human Pose Prediction › pdf › 1909.03449v1.pdf · Imitation Learning for Human Pose Prediction Borui Wang, Ehsan Adeli, Hsu-kuang Chiu, De-An Huang, Juan

References[1] Mykhaylo Andriluka, Stefan Roth, and Bernt Schiele.

Monocular 3d pose estimation and tracking by detection.In Computer Vision and Pattern Recognition (CVPR), 2010IEEE Conference on, pages 623–630. IEEE, 2010.

[2] Martin Arjovsky, Soumith Chintala, and Leon Bottou.Wasserstein gan. arXiv preprint arXiv:1701.07875, 2017.

[3] Mohammad M Arzani, Mahmood Fathy, Hamid Aghajan,Ahmad A Azirani, Kaamran Raahemifar, and Ehsan Adeli.Structured prediction with short/long-range dependencies forhuman activity recognition from depth skeleton data. In In-telligent Robots and Systems (IROS), 2017 IEEE/RSJ Inter-national Conference on, pages 560–567. IEEE, 2017.

[4] Michael Bain and Claude Sommut. A framework for be-havioural cloning. Machine intelligence, 15(15):103, 1999.

[5] Emad Barsoum, John Kender, and Zicheng Liu. Hp-gan:Probabilistic 3d human motion prediction via gan. arXivpreprint arXiv:1711.09561, 2017.

[6] Yu-Wei Chao, Jimei Yang, Brian Price, Scott Cohen, and JiaDeng. Forecasting human dynamics from static images. InCVPR, 2017.

[7] Hsu-Kuang Chiu, Ehsan Adeli, Borui Wang, De-An Huang,and Juan Carlos Niebles. Action-agnostic human pose fore-casting. IEEE Winter Conference on Applications of Com-puter Vision (WACV), 2019.

[8] Kyunghyun Cho, Bart Van Merrienboer, Dzmitry Bahdanau,and Yoshua Bengio. On the properties of neural machinetranslation: Encoder-decoder approaches. arXiv preprintarXiv:1409.1259, 2014.

[9] Katerina Fragkiadaki, Sergey Levine, Panna Felsen, and Ji-tendra Malik. Recurrent network models for human dynam-ics. In Computer Vision (ICCV), 2015 IEEE InternationalConference on, pages 4346–4354. IEEE, 2015.

[10] Partha Ghosh, Jie Song, Emre Aksan, and Otmar Hilliges.Learning human motion models for long-term predictions.arXiv preprint arXiv:1704.02827, 2017.

[11] Partha Ghosh, Jie Song, Emre Aksan, and Otmar Hilliges.Learning human motion models for long-term predictions.arXiv preprint arXiv:1704.02827, 2017.

[12] Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, BingXu, David Warde-Farley, Sherjil Ozair, Aaron Courville, andYoshua Bengio. Generative adversarial nets. In Advancesin neural information processing systems, pages 2672–2680,2014.

[13] Ishaan Gulrajani, Faruk Ahmed, Martin Arjovsky, VincentDumoulin, and Aaron C Courville. Improved training ofwasserstein gans. In Advances in neural information pro-cessing systems, pages 5767–5777, 2017.

[14] Ankur Gupta, Julieta Martinez, James J Little, and Robert JWoodham. 3d pose from motion for cross-view action recog-nition via non-linear circulant temporal encoding. In Pro-ceedings of the IEEE conference on computer vision and pat-tern recognition, pages 2601–2608, 2014.

[15] Jonathan Ho and Stefano Ermon. Generative adversarial im-itation learning. In Advances in neural information process-ing systems, pages 4565–4573, 2016.

[16] Sepp Hochreiter and Jurgen Schmidhuber. Long short-termmemory. Neural computation, 9(8):1735–1780, 1997.

[17] Catalin Ionescu, Dragos Papava, Vlad Olaru, and Cristian

Sminchisescu. Human3.6M: Large scale datasets and predic-tive methods for 3d human sensing in natural environments.IEEE transactions on pattern analysis and machine intelli-gence, 36(7):1325–1339, 2014.

[18] Catalin Ionescu, Dragos Papava, Vlad Olaru, and CristianSminchisescu. Human3.6M: Large scale datasets and predic-tive methods for 3d human sensing in natural environments.IEEE transactions on pattern analysis and machine intelli-gence, 36(7):1325–1339, 2014.

[19] Ashesh Jain, Amir R Zamir, Silvio Savarese, and AshutoshSaxena. Structural-rnn: Deep learning on spatio-temporalgraphs. In Proceedings of the IEEE Conference on ComputerVision and Pattern Recognition, pages 5308–5317, 2016.

[20] Hema Koppula and Ashutosh Saxena. Learning spatio-temporal structure from rgb-d videos for human activity de-tection and anticipation. In International conference on ma-chine learning, pages 792–800, 2013.

[21] Lucas Kovar, Michael Gleicher, and Frederic Pighin. Motiongraphs. In ACM SIGGRAPH 2008 classes, page 51. ACM,2008.

[22] Junwei Liang, Lu Jiang, Juan Carlos Niebles, Alexander GHauptmann, and Li Fei-Fei. Peeking into the future: Predict-ing future person activities and locations in videos. In Pro-ceedings of the IEEE Conference on Computer Vision andPattern Recognition, pages 5725–5734, 2019.

[23] Timothy P Lillicrap, Jonathan J Hunt, Alexander Pritzel,Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, andDaan Wierstra. Continuous control with deep reinforcementlearning. arXiv preprint arXiv:1509.02971, 2015.

[24] Pauline Luc, Natalia Neverova, Camille Couprie, Jakob Ver-beek, and Yann LeCun. Predicting deeper into the futureof semantic segmentation. In of: ICCV 2017-InternationalConference on Computer Vision, page 10, 2017.

[25] Reza Mahjourian, Martin Wicke, and Anelia Angelova.Geometry-based next frame prediction from monocularvideo. In Intelligent Vehicles Symposium (IV), 2017 IEEE,pages 1700–1707. IEEE, 2017.

[26] Julieta Martinez, Michael J Black, and Javier Romero. Onhuman motion prediction using recurrent neural networks. InCVPR, 2017.

[27] Dushyant Mehta, Srinath Sridhar, Oleksandr Sotnychenko,Helge Rhodin, Mohammad Shafiei, Hans-Peter Seidel,Weipeng Xu, Dan Casas, and Christian Theobalt. Vnect:Real-time 3d human pose estimation with a single rgb cam-era. ACM Transactions on Graphics (TOG), 36(4):44, 2017.

[28] Volodymyr Mnih, Koray Kavukcuoglu, David Silver, AlexGraves, Ioannis Antonoglou, Daan Wierstra, and MartinRiedmiller. Playing atari with deep reinforcement learning.arXiv preprint arXiv:1312.5602, 2013.

[29] Andrew Y Ng, Stuart J Russell, et al. Algorithms for inversereinforcement learning. In Icml, volume 1, page 2, 2000.

[30] Xue Bin Peng, Pieter Abbeel, Sergey Levine, and Michielvan de Panne. Deepmimic: Example-guided deep reinforce-ment learning of physics-based character skills. ACM Trans-actions on Graphics (TOG), 37(4):143, 2018.

[31] Nicholas Rhinehart, Kris M Kitani, and Paul Vernaza. R2p2:A reparameterized pushforward policy for diverse, precisegenerative path forecasting. In Proceedings of the EuropeanConference on Computer Vision (ECCV), pages 772–788,

Page 10: Imitation Learning for Human Pose Prediction › pdf › 1909.03449v1.pdf · Imitation Learning for Human Pose Prediction Borui Wang, Ehsan Adeli, Hsu-kuang Chiu, De-An Huang, Juan

2018.[32] Gregory Rogez, Jonathan Rihan, Srikumar Ramalingam,

Carlos Orrite, and Philip HS Torr. Randomized trees for hu-man pose detection. In Computer Vision and Pattern Recog-nition, 2008. CVPR 2008. IEEE Conference on, pages 1–8.IEEE, 2008.

[33] Stephane Ross, Geoffrey Gordon, and Drew Bagnell. A re-duction of imitation learning and structured prediction to no-regret online learning. In Proceedings of the fourteenth inter-national conference on artificial intelligence and statistics,pages 627–635, 2011.

[34] Caude Sammut. Behavioral cloning. Encyclopedia of Ma-chine Learning, pages 93–97, 2010.

[35] John Schulman, Sergey Levine, Pieter Abbeel, Michael Jor-dan, and Philipp Moritz. Trust region policy optimization. InInternational conference on machine learning, pages 1889–1897, 2015.

[36] John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Rad-ford, and Oleg Klimov. Proximal policy optimization algo-rithms. arXiv preprint arXiv:1707.06347, 2017.

[37] David Silver, Guy Lever, Nicolas Heess, Thomas Degris,Daan Wierstra, and Martin Riedmiller. Deterministic policygradient algorithms. In ICML, 2014.

[38] Khurram Soomro, Haroon Idrees, and Mubarak Shah. On-line localization and prediction of actions and interactions.IEEE Transactions on Pattern Analysis and Machine Intelli-gence, 2018.

[39] Ilya Sutskever, Oriol Vinyals, and Quoc V Le. Sequence tosequence learning with neural networks. In Advances in neu-ral information processing systems, pages 3104–3112, 2014.

[40] Richard S Sutton and Andrew G Barto. Reinforcement learn-ing: An introduction. MIT press, 2018.

[41] Lei Tai, Jingwei Zhang, Ming Liu, and Wolfram Burgard.Socially compliant navigation through raw depth inputs withgenerative adversarial imitation learning. In 2018 IEEE In-ternational Conference on Robotics and Automation (ICRA),pages 1111–1117. IEEE, 2018.

[42] Nikolaus F Troje. Decomposing biological motion: A frame-work for analysis and synthesis of human gait patterns. Jour-nal of vision, 2(5):2–2, 2002.

[43] Raquel Urtasun, David J Fleet, and Pascal Fua. 3d peopletracking with gaussian process dynamical models. In 2006IEEE Computer Society Conference on Computer Vision andPattern Recognition (CVPR’06), volume 1, pages 238–245.IEEE, 2006.

[44] Carl Vondrick, Hamed Pirsiavash, and Antonio Torralba.Generating videos with scene dynamics. In Advances InNeural Information Processing Systems, pages 613–621,2016.

[45] Jacob Walker, Carl Doersch, Abhinav Gupta, and MartialHebert. An uncertain future: Forecasting from static imagesusing variational autoencoders. In European Conference onComputer Vision, pages 835–851. Springer, 2016.

[46] Jacob Walker, Abhinav Gupta, and Martial Hebert. Denseoptical flow prediction from a static image. In Computer Vi-sion (ICCV), 2015 IEEE International Conference on, pages2443–2451. IEEE, 2015.

[47] Jacob Walker, Kenneth Marino, Abhinav Gupta, and Mar-tial Hebert. The pose knows: Video forecasting by generat-

ing pose futures. In 2017 IEEE International Conference onComputer Vision (ICCV), pages 3352–3361. IEEE, 2017.

[48] Jack M Wang, David J Fleet, and Aaron Hertzmann. Gaus-sian process dynamical models for human motion. IEEEtransactions on pattern analysis and machine intelligence,30(2):283–298, 2008.

[49] Di Wu and Ling Shao. Leveraging hierarchical parametricnetworks for skeletal joints based action segmentation andrecognition. In Proceedings of the IEEE Conference on Com-puter Vision and Pattern Recognition, pages 724–731, 2014.

[50] Jiqing Wu, Zhiwu Huang, Janine Thoma, Dinesh Acharya,and Luc Van Gool. Wasserstein divergence for gans. In Pro-ceedings of the European Conference on Computer Vision(ECCV), pages 653–668, 2018.

[51] Xinchen Yan, Akash Rastogi, Ruben Villegas, KalyanSunkavalli, Eli Shechtman, Sunil Hadap, Ersin Yumer, andHonglak Lee. Mt-vae: Learning motion transformations togenerate multimodal human dynamics. In Proceedings of theEuropean Conference on Computer Vision (ECCV), pages265–281, 2018.

[52] Kuo-Hao Zeng, William B Shen, De-An Huang, Min Sun,and Juan Carlos Niebles. Visual forecasting by imitating dy-namics in natural sequences. In Proceedings of the IEEEInternational Conference on Computer Vision, pages 2999–3008, 2017.

[53] Yi Zhou, Zimo Li, Shuangjiu Xiao, Chong He, Zeng Huang,and Hao Li. Auto-conditioned recurrent networks for ex-tended complex human motion synthesis. In ICLR, 2018.


Imitation Analysis

Imitation Analysis

Documents
Susanto Imitation, Technology and the Western Worldsummit.sfu.ca/system/files/iritems1/15264/Imitation, Technology... · Imitation, Technology, and the Western World ... Aristotle,

Susanto Imitation, Technology and the Western Worldsummit.sfu.ca/system/files/iritems1/15264/Imitation, Technology... · Imitation, Technology, and the Western World ... Aristotle,

Documents
Emulation, imitation, over-imitation and the scope of ...rstb.royalsocietypublishing.org/content/royptb/364/1528/2417.full.pdf · Emulation, imitation, over-imitation and the scope

Emulation, imitation, over-imitation and the scope of ...rstb.royalsocietypublishing.org/content/royptb/364/1528/2417.full.pdf · Emulation, imitation, over-imitation and the scope

Documents
Reinforcement Learning in Robotics: ASurveyReinforcement learning embraces the full complexity of these problems by requiring both interactive, sequential prediction as in imitation

Reinforcement Learning in Robotics: ASurveyReinforcement learning embraces the full complexity of these problems by requiring both interactive, sequential prediction as in imitation

Documents
Dissecting the Imitation Faculty: The Multiple Imitation Mechanisms (MIM) Hypothesis

Dissecting the Imitation Faculty: The Multiple Imitation Mechanisms (MIM) Hypothesis

Science
Flowing ConvNets for Human Pose Estimation in Videos · Flowing ConvNets for Human Pose Estimation in Videos ... for localisation the prediction targets are positions in the ... modal,

Flowing ConvNets for Human Pose Estimation in Videos · Flowing ConvNets for Human Pose Estimation in Videos ... for localisation the prediction targets are positions in the ... modal,

Documents
Brand Imitation

Brand Imitation

Documents
Key-Pose Prediction in Cyclic Human Motion€¦ · Key-Pose Prediction in Cyclic Human Motion ... ming motions in one of the four swimming styles, ... of characteristic parts of the

Key-Pose Prediction in Cyclic Human Motion€¦ · Key-Pose Prediction in Cyclic Human Motion ... ming motions in one of the four swimming styles, ... of characteristic parts of the

Documents
An Articulated Structure-aware Network for 3D Human Pose ...proceedings.mlr.press/v101/tang19a/tang19a.pdfapproaches generally estimate the 3D human pose by holistic prediction, which

An Articulated Structure-aware Network for 3D Human Pose ...proceedings.mlr.press/v101/tang19a/tang19a.pdfapproaches generally estimate the 3D human pose by holistic prediction, which

Documents
watch faces Imitation Rhodium - Colered Face with LoopsWFS-94 Imitation Rhodium WFS-78 Imitation Rhodium w Item! w Item! w Item! Imitation Rhodium WFS-7 Imitation Rhodium WFS-62 Imitation

watch faces Imitation Rhodium - Colered Face with LoopsWFS-94 Imitation Rhodium WFS-78 Imitation Rhodium w Item! w Item! w Item! Imitation Rhodium WFS-7 Imitation Rhodium WFS-62 Imitation

Documents
Supplementary: Exploiting View-Specific …...Multi-metric pose and class model (MMJ-VC) In case of multi-task multi-metric formulation (Eq. (7)-(9) of the main paper), the pose prediction

Supplementary: Exploiting View-Specific …...Multi-metric pose and class model (MMJ-VC) In case of multi-task multi-metric formulation (Eq. (7)-(9) of the main paper), the pose prediction

Documents
Structured Prediction Techniques for Imitation Learning · new domains unseen during training. We can further classify generalizable imitation learning into two subcategories. The

Structured Prediction Techniques for Imitation Learning · new domains unseen during training. We can further classify generalizable imitation learning into two subcategories. The

Documents
Imitation and Gender Insubordinationpcnw.org/files/Butler-ImitationandGenderInsubordination.pdf · Imitation and Gender Insubordination

Imitation and Gender Insubordinationpcnw.org/files/Butler-ImitationandGenderInsubordination.pdf · Imitation and Gender Insubordination

Documents
Imitation Magazin

Imitation Magazin

Documents
Imitation Article

Imitation Article

Documents
john-russell.orgjohn-russell.org/Texts/7op.pdf · ocean pose. ocean pose. cxean pose. ocean pose _ ocean pose. ocean pose. ocean pose. ocean pose. ocean pose starting in the south

john-russell.orgjohn-russell.org/Texts/7op.pdf · ocean pose. ocean pose. cxean pose. ocean pose _ ocean pose. ocean pose. ocean pose. ocean pose. ocean pose starting in the south

Documents
Imitation and Innovation

Imitation and Innovation

Spiritual
Imitation and Autism.final.2 - msu.eduingers19/lab/Imitation and Autism.pdf · Imitation and Autism 1 In press in Infants & Young Children The social role of imitation in autism:

Imitation and Autism.final.2 - msu.eduingers19/lab/Imitation and Autism.pdf · Imitation and Autism 1 In press in Infants & Young Children The social role of imitation in autism:

Documents
The Imitation Game

The Imitation Game

Documents
Multi-Task Hierarchical Imitation Learning for Home Automationronberenstein.com/papers/CASE19_Multi-Task Hierarchical Imitation... · Multi-Task Hierarchical Imitation Learning for

Multi-Task Hierarchical Imitation Learning for Home Automationronberenstein.com/papers/CASE19_Multi-Task Hierarchical Imitation... · Multi-Task Hierarchical Imitation Learning for

Documents
Structured Output Prediction and Learning for Deep ... · exploiting the structure of the human pose in 3D yields systematic gains. 1 Introduction Human pose estimation has made rapid

Structured Output Prediction and Learning for Deep ... · exploiting the structure of the human pose in 3D yields systematic gains. 1 Introduction Human pose estimation has made rapid

Documents
Learning by imitation

Learning by imitation

Education
Imitation And Instruction

Imitation And Instruction

Education
Supervised Learning of Behaviorsrail.eecs.berkeley.edu/deeprlcourse-fa18/static/slides/lec-2.pdfOther topics in imitation learning •Structured prediction •Inverse reinforcement

Supervised Learning of Behaviorsrail.eecs.berkeley.edu/deeprlcourse-fa18/static/slides/lec-2.pdfOther topics in imitation learning •Structured prediction •Inverse reinforcement

Documents
On-line Human Gait Stability Prediction using LSTMs …On-line Human Gait Stability Prediction using LSTMs for the fusion of Deep-based Pose Estimation and LRF-based Augmented Gait

On-line Human Gait Stability Prediction using LSTMs …On-line Human Gait Stability Prediction using LSTMs for the fusion of Deep-based Pose Estimation and LRF-based Augmented Gait

Documents
InfoGAIL: Interpretable Imitation Learning from Visual ...papers.nips.cc/paper/6971-infogail-interpretable-imitation-learning... · InfoGAIL: Interpretable Imitation Learning from

InfoGAIL: Interpretable Imitation Learning from Visual ...papers.nips.cc/paper/6971-infogail-interpretable-imitation-learning... · InfoGAIL: Interpretable Imitation Learning from

Documents
imitation is imitation

imitation is imitation

Education
From Imitation to InnovationFrom Imitation to Innovation · From Imitation to InnovationFrom Imitation to Innovation ... Instead, a brandInstead, a brand-new road should benew road

From Imitation to InnovationFrom Imitation to Innovation · From Imitation to InnovationFrom Imitation to Innovation ... Instead, a brandInstead, a brand-new road should benew road

Documents
Imitation, Empathy,

Imitation, Empathy,

Documents
Laws of Imitation

Laws of Imitation

Documents