Lecture #10 – Modeling the Physical World

Aykut Erdem // Hacettepe University // Spring 2019

CMP722ADVANCED COMPUTER VISION

Video: The Jenga-playing robot (MIT)

• graph structured data

• graph neural nets (GNNs)

• GNNs for ”classical network

problems”

Previously on CMP722Illustration: Kevin Hong // Quanta Magazine

Lecture overview• physical scene understanding• intuitive physics• interaction networks• relation networks• visual interaction networks• learning physics engines via graph networks

• Disclaimer: Much of the material and slides for this lecture were borrowed from —Peter Battaglia’s slides on “Structure in physical intelligence”

3

How do you understand a scene?

4

How do you understand a scene?1. Parse it into physical objects and relations

2. Reason about the objects and their interactions

5

Attached? Fall?

Support

"Precarious"

“Infinite use of finite means”- von Humboldt, on the productivity of language

6

"Precarious"

Kenneth Craik, “The Nature of Explanation”, 1943: "If the organism carries a 'small-scale model’ of externalreality and of its own possible actions within its head, it is able to try out various alternatives, conclude which is thebest of them, react to future situations before they arise, utilize the knowledge of past events in dealing with thepresent and future, and in every way to react in a muchfuller, safer, and more competent manner to theemergencies which face it." (pg 61)

"This concept of 'thinghood' is of fundamental importancefor any theory of thought." (pg 77)

7

Claim: Human intelligence is structured

• Objects and relations reflect decisions made by evolution, experience, and task demands about how to represent the world in an efficient anduseful way

• Structure in our core cognitive knowledge evident very early in infancy(Spelke)

• Model-building over recognizing patterns (Tenenbaum)

• Combinatorial generalization via compositionality ("infinite use of finitemeans”)

8

Founded on objects, relations, reasoning

What is the mechanism of human intuitive physics?

9

Intuitive Physics Engine: the "physics engine in the head"

Battaglia, Hamrick, Tenenbaum, 2013, PNAS

Experiments: What will happen? Why?

10

Will it fall? In which direction? Different masses

Comples scenes Infer the mass Predict fluids

Battaglia et al., 2013 Hamrick et al., 2016 Bates et al., 2015, 2018

Message from cognitionHumans use richly structured representations of objects and relations

to reason about, and interact with, their everyday environment.

What insights does humans’ structured intelligence offer AI?

11

We need better object- and relation-centricmodels in AIA graph is a natural way to represent entities and their relations: • “Nodes“ correspond to entities, objects, events, etc.• “Edges“ correspond to their relations, interactions, transitions, etc. • Inferences about entities and relations respect the graphical structure.

Graphs can capture data from many complex systems:

12

• Physical systems• Scene graphs• Social networks• Linguistic structure• Programs

• Search trees• Communication networks• Transportation networks• Chemical structure• Phylogenetic trees

Intuitive physics as reasoning about graphs

13

Intuitive physics as reasoning about graphs

14

Interaction Network

Strong relational inductive bias: Deep learning architecture which operates on graphs

Related to the broad family of "Graph Neural Networks" (Scarselli et al, 2009; Li et al, 2015) and "Message-Passing Neural Networks" (Gilmer et al., 2017).Chang et al. (2016) also proposed a similar version in parallel.

15Battaglia et al., 2016, NeurIPS

Interaction Network

16Battaglia et al., 2016, NeurIPS

Interaction NetworkCan learn a general-purpose physics engine, simulating future states from initial ones

17Battaglia et al., 2016, NeurIPS

Gravitational forces Rigid collisions betweenwalls and balls

Springs and rigid collisions

1000-step rollouts from 1-step supervised training

18Battaglia et al., 2016, NeurIPS

n-body

Gro

und

trut

hM

odel

Balls Strings

Zero-shot generalization to larger systems

19Battaglia et al., 2016, NeurIPS

Ballsn-body

Gro

und

trut

hM

odel

Strings

Interaction Network for system-level predictionsA "global model" can be added, which aggregates the per-object outputs to make predictions.

Can be trained to predict potential energy of a system, outperforming MLP baselines

20Battaglia et al., 2016, NeurIPS

Relation NetworkRemove “object model” and predict global outputs only using “relation model”’s output

21

Raposo et al., 2017, ICLR workshop; Santoro et al., 2017, NeurIPS

Relation Networks can infer relations in dot motion

22Santoro et al., 2017, NeurIPS

Trained on mass-spring systems

Generalizes to point-light walkers

Input Model Ground truth

Input Model Ground truth

"Visual interaction network"

23

An interaction network augmented with a learnable perception system

"Visual interaction network"

24

Multi-frame encoder (conv net-based) Interaction network

Watters et al., 2017, NeurIPS

"Visual interaction network"

25Watters et al., 2017, NeurIPS

Can even predictinvisible objects, inferred fromhow they affectvisible ones

Spring Gravity Magnetic Billiards Billiards Drift

Exa

mple

Frame

Truth

Prediction

Exa

mple

Frame

Truth

Prediction

Table 1: Rollout Trajectories. For each of our datasets, we show an example frame, an example truefuture trajectory, and a corresponding predicted rollout trajectory (for 40-60 timesteps, depending onthe dataset). This top half shows the 3-object regime and the bottom half shows the 6-object regime.For visual clarity, all objects are rendered at a higher resolution here than in the training input.

8

Spring Gravity Magnetic Billiards Billiards Drift

Learning to simulate more complex robotic systems

26

Alvaro Sanchez-Gonzalez, Nicolas Heess, Tobi Springenberg, Josh Merel, Martin Riedmiller, Raia Hadsell, Peter Battaglia

ICML, 2018

Systems: "DeepMind Control Suite" (Mujoco) & real JACO

27

JACO Arm

DeepMind Control Suite (Tassa et al., 2018)

28

Systems: "DeepMind Control Suite" (Mujoco) & real JACO

JACO Arm

Kinematic tree of the actuated systemas a graphRepresenting physical system as a graph: • Bodies → Nodes

• Joints → Edges

• Global properties

Similar representation to: • Interaction Networks (Battaglia et al. 2016)

• NerveNet (Wang et al. 2018) (graph-structured policy, rather than model)

29

Graph Network (GN)

30

Battaglia et al., 2018 Graph Network (GN)

Graph-to-graph, modular block design

Battaglia et al., 2018g g*

g

{ni}

{ej}

g*

{n*i}

{e*j}

Edge update

Node update

Global update

Graph-to-graph, modular block design

Edgeupdate

Nodeupdate

Globalupdate

Forward model: supervised, 1-step training w/ random control inputs

31

Forward model: supervised, 1-step training w/ random control inputs

Chained 100-step predictions

Input graph (t) Next graph (t+1)

Sanchez-Gonzalez et al., 2018, ICML

Sanchez-Gonzalez et al., 2018, ICML

Chained

100-step predictions

Results: Graph Net (GN) vs MLP forward models

32Sanchez-Gonzalez et al., 2018, ICML

More repeated structure:Better performance over MLP

Better test generalization, within and outside of the

training distribution

GN forward model: Multiple systems & zero-shot generalization

Single model trained:

• Pendulum, Cartpole, Acrobot, Swimmer6 & Cheetah

Zero-shot generalization: Swimmer

• # training links: {3,4,5,6, -,8,9, -, -, ...}

• # testing links: {-, -, -, -, 7, -, -, 10-14}

33Sanchez-Gonzalez et al., 2018, ICML

GN forward model: Real JACO data

34Sanchez-Gonzalez et al., 2018, ICML

GN forward model: Real JACO data

(Real JACO trajectories, rendered using Mujoco)

Recurrent graph network

Sanchez-Gonzalez et al., 2018, ICML

Recurrent graph network

(Real JACO trajectories, rendered using Mujoco)

System identification: GN-based inference, under diagnostic control inputs

35

System identification: GN-based inference, under diagnostic control inputs

Unobserved system parameters (e.g. mass, length) are implicitly inferred

Sanchez-Gonzalez et al., 2018, ICML

Unobserved system parameters (e.g. mass, length) are implicitly inferred

Sanchez-Gonzalez et al., 2018, ICML

Using learned models for control

36

Control: Model-based planningTrajectory optimization: the GN-based forward model is differentiable, so we can backpropagate through it, and find a sequence of actions that maximize reward

37Sanchez-Gonzalez et al., 2018, ICML

Control: Multiple systems via a single model

38Sanchez-Gonzalez et al., 2018, ICML

Control: Zero-shot control

39Sanchez-Gonzalez et al., 2018, ICML

Control: Multiple reward functions

40Sanchez-Gonzalez et al., 2018, ICML

Learning to use mental simulation

41

Learning to use mental simulation"Imagination-based metacontroller"

42

"Spaceship task":• Navigate to your home planet by choosing a force vector• Challenging because the planets exert gravity

The agent learns 3 components: 1. Action policy (via stochastic value gradients (Heess et al.

2015)) 2. GN-based forward model (via supervised 1-step training) 3. Internal strategy for using imagination to test potential

actions before selecting one to execute (via REINFORCE)

Hamrick et al., 2017, ICLR

Learning to use mental simulation"Imagination-based planner"

43Pascanu et al., 2017, arXiv

Learning to use mental simulation "Imagination-based planner"

Pascanu et al., 2017, arXiv

• Red: real actions • Blue: 1 step of imagination • Green: 2+ steps of imagination

Graph-structured model-free policies

44

Graph-structured model-free policies forphysical construction in humans and AI

45Hamrick et al., 2018, Proc Cog Sci

Jess Hamrick, Kelsey Allen, Victor Bapst, Tina Zhu, Kevin McKee,

Josh Tenenbaum, Peter BattagliaProc Cog Sci, 2018

The "glue task" Goal: Glue blocks together to make the towerstable, using the minimum amount of glue.

46

Structured model-free policies for physicalconstruction in humans and AI

Hamrick et al., 2018, Proc Cog Sci

47

GM<latexit sha1_base64="vjTCpRgsPEJfhljVzwQb7AFhV5c=">AAAB6nicbVA9SwNBEJ2LXzF+RS1tFoNgFe5E0MIiYKGNENF8QHKEvc1csmRv79jdE8KRn2BjoYitv8jOf+MmuUITHww83pthZl6QCK6N6347hZXVtfWN4mZpa3tnd6+8f9DUcaoYNlgsYtUOqEbBJTYMNwLbiUIaBQJbweh66reeUGkey0czTtCP6EDykDNqrPRw07vrlStu1Z2BLBMvJxXIUe+Vv7r9mKURSsME1brjuYnxM6oMZwInpW6qMaFsRAfYsVTSCLWfzU6dkBOr9EkYK1vSkJn6eyKjkdbjKLCdETVDvehNxf+8TmrCSz/jMkkNSjZfFKaCmJhM/yZ9rpAZMbaEMsXtrYQNqaLM2HRKNgRv8eVl0jyrem7Vuz+v1K7yOIpwBMdwCh5cQA1uoQ4NYDCAZ3iFN0c4L8678zFvLTj5zCH8gfP5A+mdjYU=</latexit><latexit sha1_base64="vjTCpRgsPEJfhljVzwQb7AFhV5c=">AAAB6nicbVA9SwNBEJ2LXzF+RS1tFoNgFe5E0MIiYKGNENF8QHKEvc1csmRv79jdE8KRn2BjoYitv8jOf+MmuUITHww83pthZl6QCK6N6347hZXVtfWN4mZpa3tnd6+8f9DUcaoYNlgsYtUOqEbBJTYMNwLbiUIaBQJbweh66reeUGkey0czTtCP6EDykDNqrPRw07vrlStu1Z2BLBMvJxXIUe+Vv7r9mKURSsME1brjuYnxM6oMZwInpW6qMaFsRAfYsVTSCLWfzU6dkBOr9EkYK1vSkJn6eyKjkdbjKLCdETVDvehNxf+8TmrCSz/jMkkNSjZfFKaCmJhM/yZ9rpAZMbaEMsXtrYQNqaLM2HRKNgRv8eVl0jyrem7Vuz+v1K7yOIpwBMdwCh5cQA1uoQ4NYDCAZ3iFN0c4L8678zFvLTj5zCH8gfP5A+mdjYU=</latexit><latexit sha1_base64="vjTCpRgsPEJfhljVzwQb7AFhV5c=">AAAB6nicbVA9SwNBEJ2LXzF+RS1tFoNgFe5E0MIiYKGNENF8QHKEvc1csmRv79jdE8KRn2BjoYitv8jOf+MmuUITHww83pthZl6QCK6N6347hZXVtfWN4mZpa3tnd6+8f9DUcaoYNlgsYtUOqEbBJTYMNwLbiUIaBQJbweh66reeUGkey0czTtCP6EDykDNqrPRw07vrlStu1Z2BLBMvJxXIUe+Vv7r9mKURSsME1brjuYnxM6oMZwInpW6qMaFsRAfYsVTSCLWfzU6dkBOr9EkYK1vSkJn6eyKjkdbjKLCdETVDvehNxf+8TmrCSz/jMkkNSjZfFKaCmJhM/yZ9rpAZMbaEMsXtrYQNqaLM2HRKNgRv8eVl0jyrem7Vuz+v1K7yOIpwBMdwCh5cQA1uoQ4NYDCAZ3iFN0c4L8678zFvLTj5zCH8gfP5A+mdjYU=</latexit><latexit sha1_base64="vjTCpRgsPEJfhljVzwQb7AFhV5c=">AAAB6nicbVA9SwNBEJ2LXzF+RS1tFoNgFe5E0MIiYKGNENF8QHKEvc1csmRv79jdE8KRn2BjoYitv8jOf+MmuUITHww83pthZl6QCK6N6347hZXVtfWN4mZpa3tnd6+8f9DUcaoYNlgsYtUOqEbBJTYMNwLbiUIaBQJbweh66reeUGkey0czTtCP6EDykDNqrPRw07vrlStu1Z2BLBMvJxXIUe+Vv7r9mKURSsME1brjuYnxM6oMZwInpW6qMaFsRAfYsVTSCLWfzU6dkBOr9EkYK1vSkJn6eyKjkdbjKLCdETVDvehNxf+8TmrCSz/jMkkNSjZfFKaCmJhM/yZ9rpAZMbaEMsXtrYQNqaLM2HRKNgRv8eVl0jyrem7Vuz+v1K7yOIpwBMdwCh5cQA1uoQ4NYDCAZ3iFN0c4L8678zFvLTj5zCH8gfP5A+mdjYU=</latexit>

GN1<latexit sha1_base64="oAmr7/S238q10w2wEvXkfEGmAr8=">AAAB9HicbVDLSgMxFL1TX7W+qi7dBIvgqsyIUBcuCi50JRXsA9qhZNJMG5pkxiRTKEO/w40LRdz6Me78GzPtLLT1QOBwzr3ckxPEnGnjut9OYW19Y3OruF3a2d3bPygfHrV0lChCmyTikeoEWFPOJG0aZjjtxIpiEXDaDsY3md+eUKVZJB/NNKa+wEPJQkawsZLfE9iMlEhv72d9r1+uuFV3DrRKvJxUIEejX/7qDSKSCCoN4VjrrufGxk+xMoxwOiv1Ek1jTMZ4SLuWSiyo9tN56Bk6s8oAhZGyTxo0V39vpFhoPRWBncxC6mUvE//zuokJr/yUyTgxVJLFoTDhyEQoawANmKLE8KklmChmsyIywgoTY3sq2RK85S+vktZF1XOr3sNlpX6d11GEEziFc/CgBnW4gwY0gcATPMMrvDkT58V5dz4WowUn3zmGP3A+fwCg3ZH4</latexit><latexit sha1_base64="oAmr7/S238q10w2wEvXkfEGmAr8=">AAAB9HicbVDLSgMxFL1TX7W+qi7dBIvgqsyIUBcuCi50JRXsA9qhZNJMG5pkxiRTKEO/w40LRdz6Me78GzPtLLT1QOBwzr3ckxPEnGnjut9OYW19Y3OruF3a2d3bPygfHrV0lChCmyTikeoEWFPOJG0aZjjtxIpiEXDaDsY3md+eUKVZJB/NNKa+wEPJQkawsZLfE9iMlEhv72d9r1+uuFV3DrRKvJxUIEejX/7qDSKSCCoN4VjrrufGxk+xMoxwOiv1Ek1jTMZ4SLuWSiyo9tN56Bk6s8oAhZGyTxo0V39vpFhoPRWBncxC6mUvE//zuokJr/yUyTgxVJLFoTDhyEQoawANmKLE8KklmChmsyIywgoTY3sq2RK85S+vktZF1XOr3sNlpX6d11GEEziFc/CgBnW4gwY0gcATPMMrvDkT58V5dz4WowUn3zmGP3A+fwCg3ZH4</latexit><latexit sha1_base64="oAmr7/S238q10w2wEvXkfEGmAr8=">AAAB9HicbVDLSgMxFL1TX7W+qi7dBIvgqsyIUBcuCi50JRXsA9qhZNJMG5pkxiRTKEO/w40LRdz6Me78GzPtLLT1QOBwzr3ckxPEnGnjut9OYW19Y3OruF3a2d3bPygfHrV0lChCmyTikeoEWFPOJG0aZjjtxIpiEXDaDsY3md+eUKVZJB/NNKa+wEPJQkawsZLfE9iMlEhv72d9r1+uuFV3DrRKvJxUIEejX/7qDSKSCCoN4VjrrufGxk+xMoxwOiv1Ek1jTMZ4SLuWSiyo9tN56Bk6s8oAhZGyTxo0V39vpFhoPRWBncxC6mUvE//zuokJr/yUyTgxVJLFoTDhyEQoawANmKLE8KklmChmsyIywgoTY3sq2RK85S+vktZF1XOr3sNlpX6d11GEEziFc/CgBnW4gwY0gcATPMMrvDkT58V5dz4WowUn3zmGP3A+fwCg3ZH4</latexit><latexit sha1_base64="oAmr7/S238q10w2wEvXkfEGmAr8=">AAAB9HicbVDLSgMxFL1TX7W+qi7dBIvgqsyIUBcuCi50JRXsA9qhZNJMG5pkxiRTKEO/w40LRdz6Me78GzPtLLT1QOBwzr3ckxPEnGnjut9OYW19Y3OruF3a2d3bPygfHrV0lChCmyTikeoEWFPOJG0aZjjtxIpiEXDaDsY3md+eUKVZJB/NNKa+wEPJQkawsZLfE9iMlEhv72d9r1+uuFV3DrRKvJxUIEejX/7qDSKSCCoN4VjrrufGxk+xMoxwOiv1Ek1jTMZ4SLuWSiyo9tN56Bk6s8oAhZGyTxo0V39vpFhoPRWBncxC6mUvE//zuokJr/yUyTgxVJLFoTDhyEQoawANmKLE8KklmChmsyIywgoTY3sq2RK85S+vktZF1XOr3sNlpX6d11GEEziFc/CgBnW4gwY0gcATPMMrvDkT58V5dz4WowUn3zmGP3A+fwCg3ZH4</latexit>

GN2<latexit sha1_base64="pet508CCa1uIZM8cv8xqNGylB9w=">AAAB9HicbVDLSgMxFL1TX7W+qi7dBIvgqswUQRcuCi50JRXsA9qhZNJMG5pkxiRTKEO/w40LRdz6Me78GzPtLLT1QOBwzr3ckxPEnGnjut9OYW19Y3OruF3a2d3bPygfHrV0lChCmyTikeoEWFPOJG0aZjjtxIpiEXDaDsY3md+eUKVZJB/NNKa+wEPJQkawsZLfE9iMlEhv72f9Wr9ccavuHGiVeDmpQI5Gv/zVG0QkEVQawrHWXc+NjZ9iZRjhdFbqJZrGmIzxkHYtlVhQ7afz0DN0ZpUBCiNlnzRorv7eSLHQeioCO5mF1MteJv7ndRMTXvkpk3FiqCSLQ2HCkYlQ1gAaMEWJ4VNLMFHMZkVkhBUmxvZUsiV4y19eJa1a1XOr3sNFpX6d11GEEziFc/DgEupwBw1oAoEneIZXeHMmzovz7nwsRgtOvnMMf+B8/gCiYZH5</latexit><latexit sha1_base64="pet508CCa1uIZM8cv8xqNGylB9w=">AAAB9HicbVDLSgMxFL1TX7W+qi7dBIvgqswUQRcuCi50JRXsA9qhZNJMG5pkxiRTKEO/w40LRdz6Me78GzPtLLT1QOBwzr3ckxPEnGnjut9OYW19Y3OruF3a2d3bPygfHrV0lChCmyTikeoEWFPOJG0aZjjtxIpiEXDaDsY3md+eUKVZJB/NNKa+wEPJQkawsZLfE9iMlEhv72f9Wr9ccavuHGiVeDmpQI5Gv/zVG0QkEVQawrHWXc+NjZ9iZRjhdFbqJZrGmIzxkHYtlVhQ7afz0DN0ZpUBCiNlnzRorv7eSLHQeioCO5mF1MteJv7ndRMTXvkpk3FiqCSLQ2HCkYlQ1gAaMEWJ4VNLMFHMZkVkhBUmxvZUsiV4y19eJa1a1XOr3sNFpX6d11GEEziFc/DgEupwBw1oAoEneIZXeHMmzovz7nwsRgtOvnMMf+B8/gCiYZH5</latexit><latexit sha1_base64="pet508CCa1uIZM8cv8xqNGylB9w=">AAAB9HicbVDLSgMxFL1TX7W+qi7dBIvgqswUQRcuCi50JRXsA9qhZNJMG5pkxiRTKEO/w40LRdz6Me78GzPtLLT1QOBwzr3ckxPEnGnjut9OYW19Y3OruF3a2d3bPygfHrV0lChCmyTikeoEWFPOJG0aZjjtxIpiEXDaDsY3md+eUKVZJB/NNKa+wEPJQkawsZLfE9iMlEhv72f9Wr9ccavuHGiVeDmpQI5Gv/zVG0QkEVQawrHWXc+NjZ9iZRjhdFbqJZrGmIzxkHYtlVhQ7afz0DN0ZpUBCiNlnzRorv7eSLHQeioCO5mF1MteJv7ndRMTXvkpk3FiqCSLQ2HCkYlQ1gAaMEWJ4VNLMFHMZkVkhBUmxvZUsiV4y19eJa1a1XOr3sNFpX6d11GEEziFc/DgEupwBw1oAoEneIZXeHMmzovz7nwsRgtOvnMMf+B8/gCiYZH5</latexit><latexit sha1_base64="pet508CCa1uIZM8cv8xqNGylB9w=">AAAB9HicbVDLSgMxFL1TX7W+qi7dBIvgqswUQRcuCi50JRXsA9qhZNJMG5pkxiRTKEO/w40LRdz6Me78GzPtLLT1QOBwzr3ckxPEnGnjut9OYW19Y3OruF3a2d3bPygfHrV0lChCmyTikeoEWFPOJG0aZjjtxIpiEXDaDsY3md+eUKVZJB/NNKa+wEPJQkawsZLfE9iMlEhv72f9Wr9ccavuHGiVeDmpQI5Gv/zVG0QkEVQawrHWXc+NjZ9iZRjhdFbqJZrGmIzxkHYtlVhQ7afz0DN0ZpUBCiNlnzRorv7eSLHQeioCO5mF1MteJv7ndRMTXvkpk3FiqCSLQ2HCkYlQ1gAaMEWJ4VNLMFHMZkVkhBUmxvZUsiV4y19eJa1a1XOr3sNFpX6d11GEEziFc/DgEupwBw1oAoEneIZXeHMmzovz7nwsRgtOvnMMf+B8/gCiYZH5</latexit>

GNM<latexit sha1_base64="1uUQuLnXmq2FrQq5fvsHBbzm7v8=">AAAB9HicbVDLSgMxFL1TX7W+qi7dBIvgqsyIoAsXBRe6USrYB7RDyaSZNjTJjEmmUIZ+hxsXirj1Y9z5N2baWWjrgcDhnHu5JyeIOdPGdb+dwsrq2vpGcbO0tb2zu1feP2jqKFGENkjEI9UOsKacSdowzHDajhXFIuC0FYyuM781pkqzSD6aSUx9gQeShYxgYyW/K7AZKpHe3E97d71yxa26M6Bl4uWkAjnqvfJXtx+RRFBpCMdadzw3Nn6KlWGE02mpm2gaYzLCA9qxVGJBtZ/OQk/RiVX6KIyUfdKgmfp7I8VC64kI7GQWUi96mfif10lMeOmnTMaJoZLMD4UJRyZCWQOozxQlhk8swUQxmxWRIVaYGNtTyZbgLX55mTTPqp5b9R7OK7WrvI4iHMExnIIHF1CDW6hDAwg8wTO8wpszdl6cd+djPlpw8p1D+APn8wfLTZIU</latexit><latexit sha1_base64="1uUQuLnXmq2FrQq5fvsHBbzm7v8=">AAAB9HicbVDLSgMxFL1TX7W+qi7dBIvgqsyIoAsXBRe6USrYB7RDyaSZNjTJjEmmUIZ+hxsXirj1Y9z5N2baWWjrgcDhnHu5JyeIOdPGdb+dwsrq2vpGcbO0tb2zu1feP2jqKFGENkjEI9UOsKacSdowzHDajhXFIuC0FYyuM781pkqzSD6aSUx9gQeShYxgYyW/K7AZKpHe3E97d71yxa26M6Bl4uWkAjnqvfJXtx+RRFBpCMdadzw3Nn6KlWGE02mpm2gaYzLCA9qxVGJBtZ/OQk/RiVX6KIyUfdKgmfp7I8VC64kI7GQWUi96mfif10lMeOmnTMaJoZLMD4UJRyZCWQOozxQlhk8swUQxmxWRIVaYGNtTyZbgLX55mTTPqp5b9R7OK7WrvI4iHMExnIIHF1CDW6hDAwg8wTO8wpszdl6cd+djPlpw8p1D+APn8wfLTZIU</latexit><latexit sha1_base64="1uUQuLnXmq2FrQq5fvsHBbzm7v8=">AAAB9HicbVDLSgMxFL1TX7W+qi7dBIvgqsyIoAsXBRe6USrYB7RDyaSZNjTJjEmmUIZ+hxsXirj1Y9z5N2baWWjrgcDhnHu5JyeIOdPGdb+dwsrq2vpGcbO0tb2zu1feP2jqKFGENkjEI9UOsKacSdowzHDajhXFIuC0FYyuM781pkqzSD6aSUx9gQeShYxgYyW/K7AZKpHe3E97d71yxa26M6Bl4uWkAjnqvfJXtx+RRFBpCMdadzw3Nn6KlWGE02mpm2gaYzLCA9qxVGJBtZ/OQk/RiVX6KIyUfdKgmfp7I8VC64kI7GQWUi96mfif10lMeOmnTMaJoZLMD4UJRyZCWQOozxQlhk8swUQxmxWRIVaYGNtTyZbgLX55mTTPqp5b9R7OK7WrvI4iHMExnIIHF1CDW6hDAwg8wTO8wpszdl6cd+djPlpw8p1D+APn8wfLTZIU</latexit><latexit sha1_base64="1uUQuLnXmq2FrQq5fvsHBbzm7v8=">AAAB9HicbVDLSgMxFL1TX7W+qi7dBIvgqsyIoAsXBRe6USrYB7RDyaSZNjTJjEmmUIZ+hxsXirj1Y9z5N2baWWjrgcDhnHu5JyeIOdPGdb+dwsrq2vpGcbO0tb2zu1feP2jqKFGENkjEI9UOsKacSdowzHDajhXFIuC0FYyuM781pkqzSD6aSUx9gQeShYxgYyW/K7AZKpHe3E97d71yxa26M6Bl4uWkAjnqvfJXtx+RRFBpCMdadzw3Nn6KlWGE02mpm2gaYzLCA9qxVGJBtZ/OQk/RiVX6KIyUfdKgmfp7I8VC64kI7GQWUi96mfif10lMeOmnTMaJoZLMD4UJRyZCWQOozxQlhk8swUQxmxWRIVaYGNtTyZbgLX55mTTPqp5b9R7OK7WrvI4iHMExnIIHF1CDW6hDAwg8wTO8wpszdl6cd+djPlpw8p1D+APn8wfLTZIU</latexit>

. . .<latexit sha1_base64="Gj7yv98SlyD93Ghofp+NnyXvd2c=">AAAB7HicbVBNS8NAFHypX7V+VT16WSyCp5KIUA8eCl48VjBtoQ1ls920SzebsPsilNDf4MWDIl79Qd78N27bHLR1YGGYecO+N2EqhUHX/XZKG5tb2zvl3cre/sHhUfX4pG2STDPus0QmuhtSw6VQ3EeBkndTzWkcSt4JJ3dzv/PEtRGJesRpyoOYjpSIBKNoJb8/TNAMqjW37i5A1olXkBoUaA2qXzbHspgrZJIa0/PcFIOcahRM8lmlnxmeUjahI96zVNGYmyBfLDsjF1YZkijR9ikkC/V3IqexMdM4tJMxxbFZ9ebif14vw+gmyIVKM+SKLT+KMkkwIfPLyVBozlBOLaFMC7srYWOqKUPbT8WW4K2evE7aV3XPrXsP17XmbVFHGc7gHC7BgwY04R5a4AMDAc/wCm+Ocl6cd+djOVpyiswp/IHz+QPvmo68</latexit><latexit sha1_base64="Gj7yv98SlyD93Ghofp+NnyXvd2c=">AAAB7HicbVBNS8NAFHypX7V+VT16WSyCp5KIUA8eCl48VjBtoQ1ls920SzebsPsilNDf4MWDIl79Qd78N27bHLR1YGGYecO+N2EqhUHX/XZKG5tb2zvl3cre/sHhUfX4pG2STDPus0QmuhtSw6VQ3EeBkndTzWkcSt4JJ3dzv/PEtRGJesRpyoOYjpSIBKNoJb8/TNAMqjW37i5A1olXkBoUaA2qXzbHspgrZJIa0/PcFIOcahRM8lmlnxmeUjahI96zVNGYmyBfLDsjF1YZkijR9ikkC/V3IqexMdM4tJMxxbFZ9ebif14vw+gmyIVKM+SKLT+KMkkwIfPLyVBozlBOLaFMC7srYWOqKUPbT8WW4K2evE7aV3XPrXsP17XmbVFHGc7gHC7BgwY04R5a4AMDAc/wCm+Ocl6cd+djOVpyiswp/IHz+QPvmo68</latexit><latexit sha1_base64="Gj7yv98SlyD93Ghofp+NnyXvd2c=">AAAB7HicbVBNS8NAFHypX7V+VT16WSyCp5KIUA8eCl48VjBtoQ1ls920SzebsPsilNDf4MWDIl79Qd78N27bHLR1YGGYecO+N2EqhUHX/XZKG5tb2zvl3cre/sHhUfX4pG2STDPus0QmuhtSw6VQ3EeBkndTzWkcSt4JJ3dzv/PEtRGJesRpyoOYjpSIBKNoJb8/TNAMqjW37i5A1olXkBoUaA2qXzbHspgrZJIa0/PcFIOcahRM8lmlnxmeUjahI96zVNGYmyBfLDsjF1YZkijR9ikkC/V3IqexMdM4tJMxxbFZ9ebif14vw+gmyIVKM+SKLT+KMkkwIfPLyVBozlBOLaFMC7srYWOqKUPbT8WW4K2evE7aV3XPrXsP17XmbVFHGc7gHC7BgwY04R5a4AMDAc/wCm+Ocl6cd+djOVpyiswp/IHz+QPvmo68</latexit><latexit sha1_base64="Gj7yv98SlyD93Ghofp+NnyXvd2c=">AAAB7HicbVBNS8NAFHypX7V+VT16WSyCp5KIUA8eCl48VjBtoQ1ls920SzebsPsilNDf4MWDIl79Qd78N27bHLR1YGGYecO+N2EqhUHX/XZKG5tb2zvl3cre/sHhUfX4pG2STDPus0QmuhtSw6VQ3EeBkndTzWkcSt4JJ3dzv/PEtRGJesRpyoOYjpSIBKNoJb8/TNAMqjW37i5A1olXkBoUaA2qXzbHspgrZJIa0/PcFIOcahRM8lmlnxmeUjahI96zVNGYmyBfLDsjF1YZkijR9ikkC/V3IqexMdM4tJMxxbFZ9ebif14vw+gmyIVKM+SKLT+KMkkwIfPLyVBozlBOLaFMC7srYWOqKUPbT8WW4K2evE7aV3XPrXsP17XmbVFHGc7gHC7BgwY04R5a4AMDAc/wCm+Ocl6cd+djOVpyiswp/IHz+QPvmo68</latexit>G1

<latexit sha1_base64="YNShseMoKm2HdChKvcjMRmoBu5o=">AAAB6nicbVA9SwNBEJ2LXzF+RS1tFoNgFe5EiIVFwELLiOYDkiPsbfaSJXt7x+6cEI78BBsLRWz9RXb+GzfJFZr4YODx3gwz84JECoOu++0U1tY3NreK26Wd3b39g/LhUcvEqWa8yWIZ605ADZdC8SYKlLyTaE6jQPJ2ML6Z+e0nro2I1SNOEu5HdKhEKBhFKz3c9r1+ueJW3TnIKvFyUoEcjX75qzeIWRpxhUxSY7qem6CfUY2CST4t9VLDE8rGdMi7lioaceNn81On5MwqAxLG2pZCMld/T2Q0MmYSBbYzojgyy95M/M/rphhe+ZlQSYpcscWiMJUEYzL7mwyE5gzlxBLKtLC3EjaimjK06ZRsCN7yy6ukdVH13Kp3f1mpX+dxFOEETuEcPKhBHe6gAU1gMIRneIU3RzovzrvzsWgtOPnMMfyB8/kDvy2NaQ==</latexit><latexit sha1_base64="YNShseMoKm2HdChKvcjMRmoBu5o=">AAAB6nicbVA9SwNBEJ2LXzF+RS1tFoNgFe5EiIVFwELLiOYDkiPsbfaSJXt7x+6cEI78BBsLRWz9RXb+GzfJFZr4YODx3gwz84JECoOu++0U1tY3NreK26Wd3b39g/LhUcvEqWa8yWIZ605ADZdC8SYKlLyTaE6jQPJ2ML6Z+e0nro2I1SNOEu5HdKhEKBhFKz3c9r1+ueJW3TnIKvFyUoEcjX75qzeIWRpxhUxSY7qem6CfUY2CST4t9VLDE8rGdMi7lioaceNn81On5MwqAxLG2pZCMld/T2Q0MmYSBbYzojgyy95M/M/rphhe+ZlQSYpcscWiMJUEYzL7mwyE5gzlxBLKtLC3EjaimjK06ZRsCN7yy6ukdVH13Kp3f1mpX+dxFOEETuEcPKhBHe6gAU1gMIRneIU3RzovzrvzsWgtOPnMMfyB8/kDvy2NaQ==</latexit><latexit sha1_base64="YNShseMoKm2HdChKvcjMRmoBu5o=">AAAB6nicbVA9SwNBEJ2LXzF+RS1tFoNgFe5EiIVFwELLiOYDkiPsbfaSJXt7x+6cEI78BBsLRWz9RXb+GzfJFZr4YODx3gwz84JECoOu++0U1tY3NreK26Wd3b39g/LhUcvEqWa8yWIZ605ADZdC8SYKlLyTaE6jQPJ2ML6Z+e0nro2I1SNOEu5HdKhEKBhFKz3c9r1+ueJW3TnIKvFyUoEcjX75qzeIWRpxhUxSY7qem6CfUY2CST4t9VLDE8rGdMi7lioaceNn81On5MwqAxLG2pZCMld/T2Q0MmYSBbYzojgyy95M/M/rphhe+ZlQSYpcscWiMJUEYzL7mwyE5gzlxBLKtLC3EjaimjK06ZRsCN7yy6ukdVH13Kp3f1mpX+dxFOEETuEcPKhBHe6gAU1gMIRneIU3RzovzrvzsWgtOPnMMfyB8/kDvy2NaQ==</latexit><latexit sha1_base64="YNShseMoKm2HdChKvcjMRmoBu5o=">AAAB6nicbVA9SwNBEJ2LXzF+RS1tFoNgFe5EiIVFwELLiOYDkiPsbfaSJXt7x+6cEI78BBsLRWz9RXb+GzfJFZr4YODx3gwz84JECoOu++0U1tY3NreK26Wd3b39g/LhUcvEqWa8yWIZ605ADZdC8SYKlLyTaE6jQPJ2ML6Z+e0nro2I1SNOEu5HdKhEKBhFKz3c9r1+ueJW3TnIKvFyUoEcjX75qzeIWRpxhUxSY7qem6CfUY2CST4t9VLDE8rGdMi7lioaceNn81On5MwqAxLG2pZCMld/T2Q0MmYSBbYzojgyy95M/M/rphhe+ZlQSYpcscWiMJUEYzL7mwyE5gzlxBLKtLC3EjaimjK06ZRsCN7yy6ukdVH13Kp3f1mpX+dxFOEETuEcPKhBHe6gAU1gMIRneIU3RzovzrvzsWgtOPnMMfyB8/kDvy2NaQ==</latexit>

G0<latexit sha1_base64="vj48jMMQe2f55rU6zb6RZp+K9y4=">AAAB6nicbVA9SwNBEJ2LXzF+RS1tFoNgFe5EiIVFwELLiOYDkiPsbfaSJXt7x+6cEI78BBsLRWz9RXb+GzfJFZr4YODx3gwz84JECoOu++0U1tY3NreK26Wd3b39g/LhUcvEqWa8yWIZ605ADZdC8SYKlLyTaE6jQPJ2ML6Z+e0nro2I1SNOEu5HdKhEKBhFKz3c9t1+ueJW3TnIKvFyUoEcjX75qzeIWRpxhUxSY7qem6CfUY2CST4t9VLDE8rGdMi7lioaceNn81On5MwqAxLG2pZCMld/T2Q0MmYSBbYzojgyy95M/M/rphhe+ZlQSYpcscWiMJUEYzL7mwyE5gzlxBLKtLC3EjaimjK06ZRsCN7yy6ukdVH13Kp3f1mpX+dxFOEETuEcPKhBHe6gAU1gMIRneIU3RzovzrvzsWgtOPnMMfyB8/kDvamNaA==</latexit><latexit sha1_base64="vj48jMMQe2f55rU6zb6RZp+K9y4=">AAAB6nicbVA9SwNBEJ2LXzF+RS1tFoNgFe5EiIVFwELLiOYDkiPsbfaSJXt7x+6cEI78BBsLRWz9RXb+GzfJFZr4YODx3gwz84JECoOu++0U1tY3NreK26Wd3b39g/LhUcvEqWa8yWIZ605ADZdC8SYKlLyTaE6jQPJ2ML6Z+e0nro2I1SNOEu5HdKhEKBhFKz3c9t1+ueJW3TnIKvFyUoEcjX75qzeIWRpxhUxSY7qem6CfUY2CST4t9VLDE8rGdMi7lioaceNn81On5MwqAxLG2pZCMld/T2Q0MmYSBbYzojgyy95M/M/rphhe+ZlQSYpcscWiMJUEYzL7mwyE5gzlxBLKtLC3EjaimjK06ZRsCN7yy6ukdVH13Kp3f1mpX+dxFOEETuEcPKhBHe6gAU1gMIRneIU3RzovzrvzsWgtOPnMMfyB8/kDvamNaA==</latexit><latexit sha1_base64="vj48jMMQe2f55rU6zb6RZp+K9y4=">AAAB6nicbVA9SwNBEJ2LXzF+RS1tFoNgFe5EiIVFwELLiOYDkiPsbfaSJXt7x+6cEI78BBsLRWz9RXb+GzfJFZr4YODx3gwz84JECoOu++0U1tY3NreK26Wd3b39g/LhUcvEqWa8yWIZ605ADZdC8SYKlLyTaE6jQPJ2ML6Z+e0nro2I1SNOEu5HdKhEKBhFKz3c9t1+ueJW3TnIKvFyUoEcjX75qzeIWRpxhUxSY7qem6CfUY2CST4t9VLDE8rGdMi7lioaceNn81On5MwqAxLG2pZCMld/T2Q0MmYSBbYzojgyy95M/M/rphhe+ZlQSYpcscWiMJUEYzL7mwyE5gzlxBLKtLC3EjaimjK06ZRsCN7yy6ukdVH13Kp3f1mpX+dxFOEETuEcPKhBHe6gAU1gMIRneIU3RzovzrvzsWgtOPnMMfyB8/kDvamNaA==</latexit><latexit sha1_base64="vj48jMMQe2f55rU6zb6RZp+K9y4=">AAAB6nicbVA9SwNBEJ2LXzF+RS1tFoNgFe5EiIVFwELLiOYDkiPsbfaSJXt7x+6cEI78BBsLRWz9RXb+GzfJFZr4YODx3gwz84JECoOu++0U1tY3NreK26Wd3b39g/LhUcvEqWa8yWIZ605ADZdC8SYKlLyTaE6jQPJ2ML6Z+e0nro2I1SNOEu5HdKhEKBhFKz3c9t1+ueJW3TnIKvFyUoEcjX75qzeIWRpxhUxSY7qem6CfUY2CST4t9VLDE8rGdMi7lioaceNn81On5MwqAxLG2pZCMld/T2Q0MmYSBbYzojgyy95M/M/rphhe+ZlQSYpcscWiMJUEYzL7mwyE5gzlxBLKtLC3EjaimjK06ZRsCN7yy6ukdVH13Kp3f1mpX+dxFOEETuEcPKhBHe6gAU1gMIRneIU3RzovzrvzsWgtOPnMMfyB8/kDvamNaA==</latexit>

GM<latexit sha1_base64="vjTCpRgsPEJfhljVzwQb7AFhV5c=">AAAB6nicbVA9SwNBEJ2LXzF+RS1tFoNgFe5E0MIiYKGNENF8QHKEvc1csmRv79jdE8KRn2BjoYitv8jOf+MmuUITHww83pthZl6QCK6N6347hZXVtfWN4mZpa3tnd6+8f9DUcaoYNlgsYtUOqEbBJTYMNwLbiUIaBQJbweh66reeUGkey0czTtCP6EDykDNqrPRw07vrlStu1Z2BLBMvJxXIUe+Vv7r9mKURSsME1brjuYnxM6oMZwInpW6qMaFsRAfYsVTSCLWfzU6dkBOr9EkYK1vSkJn6eyKjkdbjKLCdETVDvehNxf+8TmrCSz/jMkkNSjZfFKaCmJhM/yZ9rpAZMbaEMsXtrYQNqaLM2HRKNgRv8eVl0jyrem7Vuz+v1K7yOIpwBMdwCh5cQA1uoQ4NYDCAZ3iFN0c4L8678zFvLTj5zCH8gfP5A+mdjYU=</latexit><latexit sha1_base64="vjTCpRgsPEJfhljVzwQb7AFhV5c=">AAAB6nicbVA9SwNBEJ2LXzF+RS1tFoNgFe5E0MIiYKGNENF8QHKEvc1csmRv79jdE8KRn2BjoYitv8jOf+MmuUITHww83pthZl6QCK6N6347hZXVtfWN4mZpa3tnd6+8f9DUcaoYNlgsYtUOqEbBJTYMNwLbiUIaBQJbweh66reeUGkey0czTtCP6EDykDNqrPRw07vrlStu1Z2BLBMvJxXIUe+Vv7r9mKURSsME1brjuYnxM6oMZwInpW6qMaFsRAfYsVTSCLWfzU6dkBOr9EkYK1vSkJn6eyKjkdbjKLCdETVDvehNxf+8TmrCSz/jMkkNSjZfFKaCmJhM/yZ9rpAZMbaEMsXtrYQNqaLM2HRKNgRv8eVl0jyrem7Vuz+v1K7yOIpwBMdwCh5cQA1uoQ4NYDCAZ3iFN0c4L8678zFvLTj5zCH8gfP5A+mdjYU=</latexit><latexit sha1_base64="vjTCpRgsPEJfhljVzwQb7AFhV5c=">AAAB6nicbVA9SwNBEJ2LXzF+RS1tFoNgFe5E0MIiYKGNENF8QHKEvc1csmRv79jdE8KRn2BjoYitv8jOf+MmuUITHww83pthZl6QCK6N6347hZXVtfWN4mZpa3tnd6+8f9DUcaoYNlgsYtUOqEbBJTYMNwLbiUIaBQJbweh66reeUGkey0czTtCP6EDykDNqrPRw07vrlStu1Z2BLBMvJxXIUe+Vv7r9mKURSsME1brjuYnxM6oMZwInpW6qMaFsRAfYsVTSCLWfzU6dkBOr9EkYK1vSkJn6eyKjkdbjKLCdETVDvehNxf+8TmrCSz/jMkkNSjZfFKaCmJhM/yZ9rpAZMbaEMsXtrYQNqaLM2HRKNgRv8eVl0jyrem7Vuz+v1K7yOIpwBMdwCh5cQA1uoQ4NYDCAZ3iFN0c4L8678zFvLTj5zCH8gfP5A+mdjYU=</latexit><latexit sha1_base64="vjTCpRgsPEJfhljVzwQb7AFhV5c=">AAAB6nicbVA9SwNBEJ2LXzF+RS1tFoNgFe5E0MIiYKGNENF8QHKEvc1csmRv79jdE8KRn2BjoYitv8jOf+MmuUITHww83pthZl6QCK6N6347hZXVtfWN4mZpa3tnd6+8f9DUcaoYNlgsYtUOqEbBJTYMNwLbiUIaBQJbweh66reeUGkey0czTtCP6EDykDNqrPRw07vrlStu1Z2BLBMvJxXIUe+Vv7r9mKURSsME1brjuYnxM6oMZwInpW6qMaFsRAfYsVTSCLWfzU6dkBOr9EkYK1vSkJn6eyKjkdbjKLCdETVDvehNxf+8TmrCSz/jMkkNSjZfFKaCmJhM/yZ9rpAZMbaEMsXtrYQNqaLM2HRKNgRv8eVl0jyrem7Vuz+v1K7yOIpwBMdwCh5cQA1uoQ4NYDCAZ3iFN0c4L8678zFvLTj5zCH8gfP5A+mdjYU=</latexit>

G0<latexit sha1_base64="vj48jMMQe2f55rU6zb6RZp+K9y4=">AAAB6nicbVA9SwNBEJ2LXzF+RS1tFoNgFe5EiIVFwELLiOYDkiPsbfaSJXt7x+6cEI78BBsLRWz9RXb+GzfJFZr4YODx3gwz84JECoOu++0U1tY3NreK26Wd3b39g/LhUcvEqWa8yWIZ605ADZdC8SYKlLyTaE6jQPJ2ML6Z+e0nro2I1SNOEu5HdKhEKBhFKz3c9t1+ueJW3TnIKvFyUoEcjX75qzeIWRpxhUxSY7qem6CfUY2CST4t9VLDE8rGdMi7lioaceNn81On5MwqAxLG2pZCMld/T2Q0MmYSBbYzojgyy95M/M/rphhe+ZlQSYpcscWiMJUEYzL7mwyE5gzlxBLKtLC3EjaimjK06ZRsCN7yy6ukdVH13Kp3f1mpX+dxFOEETuEcPKhBHe6gAU1gMIRneIU3RzovzrvzsWgtOPnMMfyB8/kDvamNaA==</latexit><latexit sha1_base64="vj48jMMQe2f55rU6zb6RZp+K9y4=">AAAB6nicbVA9SwNBEJ2LXzF+RS1tFoNgFe5EiIVFwELLiOYDkiPsbfaSJXt7x+6cEI78BBsLRWz9RXb+GzfJFZr4YODx3gwz84JECoOu++0U1tY3NreK26Wd3b39g/LhUcvEqWa8yWIZ605ADZdC8SYKlLyTaE6jQPJ2ML6Z+e0nro2I1SNOEu5HdKhEKBhFKz3c9t1+ueJW3TnIKvFyUoEcjX75qzeIWRpxhUxSY7qem6CfUY2CST4t9VLDE8rGdMi7lioaceNn81On5MwqAxLG2pZCMld/T2Q0MmYSBbYzojgyy95M/M/rphhe+ZlQSYpcscWiMJUEYzL7mwyE5gzlxBLKtLC3EjaimjK06ZRsCN7yy6ukdVH13Kp3f1mpX+dxFOEETuEcPKhBHe6gAU1gMIRneIU3RzovzrvzsWgtOPnMMfyB8/kDvamNaA==</latexit><latexit sha1_base64="vj48jMMQe2f55rU6zb6RZp+K9y4=">AAAB6nicbVA9SwNBEJ2LXzF+RS1tFoNgFe5EiIVFwELLiOYDkiPsbfaSJXt7x+6cEI78BBsLRWz9RXb+GzfJFZr4YODx3gwz84JECoOu++0U1tY3NreK26Wd3b39g/LhUcvEqWa8yWIZ605ADZdC8SYKlLyTaE6jQPJ2ML6Z+e0nro2I1SNOEu5HdKhEKBhFKz3c9t1+ueJW3TnIKvFyUoEcjX75qzeIWRpxhUxSY7qem6CfUY2CST4t9VLDE8rGdMi7lioaceNn81On5MwqAxLG2pZCMld/T2Q0MmYSBbYzojgyy95M/M/rphhe+ZlQSYpcscWiMJUEYzL7mwyE5gzlxBLKtLC3EjaimjK06ZRsCN7yy6ukdVH13Kp3f1mpX+dxFOEETuEcPKhBHe6gAU1gMIRneIU3RzovzrvzsWgtOPnMMfyB8/kDvamNaA==</latexit><latexit sha1_base64="vj48jMMQe2f55rU6zb6RZp+K9y4=">AAAB6nicbVA9SwNBEJ2LXzF+RS1tFoNgFe5EiIVFwELLiOYDkiPsbfaSJXt7x+6cEI78BBsLRWz9RXb+GzfJFZr4YODx3gwz84JECoOu++0U1tY3NreK26Wd3b39g/LhUcvEqWa8yWIZ605ADZdC8SYKlLyTaE6jQPJ2ML6Z+e0nro2I1SNOEu5HdKhEKBhFKz3c9t1+ueJW3TnIKvFyUoEcjX75qzeIWRpxhUxSY7qem6CfUY2CST4t9VLDE8rGdMi7lioaceNn81On5MwqAxLG2pZCMld/T2Q0MmYSBbYzojgyy95M/M/rphhe+ZlQSYpcscWiMJUEYzL7mwyE5gzlxBLKtLC3EjaimjK06ZRsCN7yy6ukdVH13Kp3f1mpX+dxFOEETuEcPKhBHe6gAU1gMIRneIU3RzovzrvzsWgtOPnMMfyB8/kDvamNaA==</latexit>

GNcore<latexit sha1_base64="sfcetjjriA53KVhP8LRkSGs9KNA=">AAAB+3icbVDLSsNAFJ34rPUV69LNYBFclUQEXbgouNCVVLAPaEOYTCft0HmEmYlYQn7FjQtF3Poj7vwbJ20W2npg4HDOvdwzJ0oY1cbzvp2V1bX1jc3KVnV7Z3dv3z2odbRMFSZtLJlUvQhpwqggbUMNI71EEcQjRrrR5Lrwu49EaSrFg5kmJOBoJGhMMTJWCt3agCMzVjy7ucvDDEtF8tCtew1vBrhM/JLUQYlW6H4NhhKnnAiDGdK673uJCTKkDMWM5NVBqkmC8ASNSN9SgTjRQTbLnsMTqwxhLJV9wsCZ+nsjQ1zrKY/sZJFUL3qF+J/XT018GWRUJKkhAs8PxSmDRsKiCDikimDDppYgrKjNCvEYKYSNratqS/AXv7xMOmcN32v49+f15lVZRwUcgWNwCnxwAZrgFrRAG2DwBJ7BK3hzcufFeXc+5qMrTrlzCP7A+fwBopiUyw==</latexit><latexit sha1_base64="sfcetjjriA53KVhP8LRkSGs9KNA=">AAAB+3icbVDLSsNAFJ34rPUV69LNYBFclUQEXbgouNCVVLAPaEOYTCft0HmEmYlYQn7FjQtF3Poj7vwbJ20W2npg4HDOvdwzJ0oY1cbzvp2V1bX1jc3KVnV7Z3dv3z2odbRMFSZtLJlUvQhpwqggbUMNI71EEcQjRrrR5Lrwu49EaSrFg5kmJOBoJGhMMTJWCt3agCMzVjy7ucvDDEtF8tCtew1vBrhM/JLUQYlW6H4NhhKnnAiDGdK673uJCTKkDMWM5NVBqkmC8ASNSN9SgTjRQTbLnsMTqwxhLJV9wsCZ+nsjQ1zrKY/sZJFUL3qF+J/XT018GWRUJKkhAs8PxSmDRsKiCDikimDDppYgrKjNCvEYKYSNratqS/AXv7xMOmcN32v49+f15lVZRwUcgWNwCnxwAZrgFrRAG2DwBJ7BK3hzcufFeXc+5qMrTrlzCP7A+fwBopiUyw==</latexit><latexit sha1_base64="sfcetjjriA53KVhP8LRkSGs9KNA=">AAAB+3icbVDLSsNAFJ34rPUV69LNYBFclUQEXbgouNCVVLAPaEOYTCft0HmEmYlYQn7FjQtF3Poj7vwbJ20W2npg4HDOvdwzJ0oY1cbzvp2V1bX1jc3KVnV7Z3dv3z2odbRMFSZtLJlUvQhpwqggbUMNI71EEcQjRrrR5Lrwu49EaSrFg5kmJOBoJGhMMTJWCt3agCMzVjy7ucvDDEtF8tCtew1vBrhM/JLUQYlW6H4NhhKnnAiDGdK673uJCTKkDMWM5NVBqkmC8ASNSN9SgTjRQTbLnsMTqwxhLJV9wsCZ+nsjQ1zrKY/sZJFUL3qF+J/XT018GWRUJKkhAs8PxSmDRsKiCDikimDDppYgrKjNCvEYKYSNratqS/AXv7xMOmcN32v49+f15lVZRwUcgWNwCnxwAZrgFrRAG2DwBJ7BK3hzcufFeXc+5qMrTrlzCP7A+fwBopiUyw==</latexit><latexit sha1_base64="sfcetjjriA53KVhP8LRkSGs9KNA=">AAAB+3icbVDLSsNAFJ34rPUV69LNYBFclUQEXbgouNCVVLAPaEOYTCft0HmEmYlYQn7FjQtF3Poj7vwbJ20W2npg4HDOvdwzJ0oY1cbzvp2V1bX1jc3KVnV7Z3dv3z2odbRMFSZtLJlUvQhpwqggbUMNI71EEcQjRrrR5Lrwu49EaSrFg5kmJOBoJGhMMTJWCt3agCMzVjy7ucvDDEtF8tCtew1vBrhM/JLUQYlW6H4NhhKnnAiDGdK673uJCTKkDMWM5NVBqkmC8ASNSN9SgTjRQTbLnsMTqwxhLJV9wsCZ+nsjQ1zrKY/sZJFUL3qF+J/XT018GWRUJKkhAs8PxSmDRsKiCDikimDDppYgrKjNCvEYKYSNratqS/AXv7xMOmcN32v49+f15lVZRwUcgWNwCnxwAZrgFrRAG2DwBJ7BK3hzcufFeXc+5qMrTrlzCP7A+fwBopiUyw==</latexit>

⇥M<latexit sha1_base64="xCEPSgjeJaAOppNxwTZXrwRukIg=">AAAB73icbVA9SwNBEJ2LXzF+RS1tFoNgFe5E0MIiYGMjRDAfkBxhb7OXLNnbu+zOCeHIn7CxUMTWv2Pnv3GTXKGJDwYe780wMy9IpDDout9OYW19Y3OruF3a2d3bPygfHjVNnGrGGyyWsW4H1HApFG+gQMnbieY0CiRvBaPbmd964tqIWD3iJOF+RAdKhIJRtFK7iyLihtz3yhW36s5BVomXkwrkqPfKX91+zNKIK2SSGtPx3AT9jGoUTPJpqZsanlA2ogPesVRRu8bP5vdOyZlV+iSMtS2FZK7+nshoZMwkCmxnRHFolr2Z+J/XSTG89jOhkhS5YotFYSoJxmT2POkLzRnKiSWUaWFvJWxINWVoIyrZELzll1dJ86LquVXv4bJSu8njKMIJnMI5eHAFNbiDOjSAgYRneIU3Z+y8OO/Ox6K14OQzx/AHzucPqJqPrw==</latexit><latexit sha1_base64="xCEPSgjeJaAOppNxwTZXrwRukIg=">AAAB73icbVA9SwNBEJ2LXzF+RS1tFoNgFe5E0MIiYGMjRDAfkBxhb7OXLNnbu+zOCeHIn7CxUMTWv2Pnv3GTXKGJDwYe780wMy9IpDDout9OYW19Y3OruF3a2d3bPygfHjVNnGrGGyyWsW4H1HApFG+gQMnbieY0CiRvBaPbmd964tqIWD3iJOF+RAdKhIJRtFK7iyLihtz3yhW36s5BVomXkwrkqPfKX91+zNKIK2SSGtPx3AT9jGoUTPJpqZsanlA2ogPesVRRu8bP5vdOyZlV+iSMtS2FZK7+nshoZMwkCmxnRHFolr2Z+J/XSTG89jOhkhS5YotFYSoJxmT2POkLzRnKiSWUaWFvJWxINWVoIyrZELzll1dJ86LquVXv4bJSu8njKMIJnMI5eHAFNbiDOjSAgYRneIU3Z+y8OO/Ox6K14OQzx/AHzucPqJqPrw==</latexit><latexit sha1_base64="xCEPSgjeJaAOppNxwTZXrwRukIg=">AAAB73icbVA9SwNBEJ2LXzF+RS1tFoNgFe5E0MIiYGMjRDAfkBxhb7OXLNnbu+zOCeHIn7CxUMTWv2Pnv3GTXKGJDwYe780wMy9IpDDout9OYW19Y3OruF3a2d3bPygfHjVNnGrGGyyWsW4H1HApFG+gQMnbieY0CiRvBaPbmd964tqIWD3iJOF+RAdKhIJRtFK7iyLihtz3yhW36s5BVomXkwrkqPfKX91+zNKIK2SSGtPx3AT9jGoUTPJpqZsanlA2ogPesVRRu8bP5vdOyZlV+iSMtS2FZK7+nshoZMwkCmxnRHFolr2Z+J/XSTG89jOhkhS5YotFYSoJxmT2POkLzRnKiSWUaWFvJWxINWVoIyrZELzll1dJ86LquVXv4bJSu8njKMIJnMI5eHAFNbiDOjSAgYRneIU3Z+y8OO/Ox6K14OQzx/AHzucPqJqPrw==</latexit><latexit sha1_base64="xCEPSgjeJaAOppNxwTZXrwRukIg=">AAAB73icbVA9SwNBEJ2LXzF+RS1tFoNgFe5E0MIiYGMjRDAfkBxhb7OXLNnbu+zOCeHIn7CxUMTWv2Pnv3GTXKGJDwYe780wMy9IpDDout9OYW19Y3OruF3a2d3bPygfHjVNnGrGGyyWsW4H1HApFG+gQMnbieY0CiRvBaPbmd964tqIWD3iJOF+RAdKhIJRtFK7iyLihtz3yhW36s5BVomXkwrkqPfKX91+zNKIK2SSGtPx3AT9jGoUTPJpqZsanlA2ogPesVRRu8bP5vdOyZlV+iSMtS2FZK7+nshoZMwkCmxnRHFolr2Z+J/XSTG89jOhkhS5YotFYSoJxmT2POkLzRnKiSWUaWFvJWxINWVoIyrZELzll1dJ86LquVXv4bJSu8njKMIJnMI5eHAFNbiDOjSAgYRneIU3Z+y8OO/Ox6K14OQzx/AHzucPqJqPrw==</latexit>

⇡(E)<latexit sha1_base64="jh7gUoALVOlBmg3iPkbszI9veEE=">AAAB7XicbVBNSwMxEJ3Ur1q/qh69BItQL2VXBD0WROixgq2FdinZNNvGZpMlyQpl6X/w4kERr/4fb/4b03YP2vpg4PHeDDPzwkRwYz3vGxXW1jc2t4rbpZ3dvf2D8uFR26hUU9aiSijdCYlhgkvWstwK1kk0I3Eo2EM4vpn5D09MG67kvZ0kLIjJUPKIU2Kd1O4lvHp73i9XvJo3B14lfk4qkKPZL3/1BoqmMZOWCmJM1/cSG2REW04Fm5Z6qWEJoWMyZF1HJYmZCbL5tVN85pQBjpR2JS2eq78nMhIbM4lD1xkTOzLL3kz8z+umNroOMi6T1DJJF4uiVGCr8Ox1POCaUSsmjhCqubsV0xHRhFoXUMmF4C+/vEraFzXfq/l3l5V6I4+jCCdwClXw4Qrq0IAmtIDCIzzDK7whhV7QO/pYtBZQPnMMf4A+fwCn5I6H</latexit><latexit sha1_base64="jh7gUoALVOlBmg3iPkbszI9veEE=">AAAB7XicbVBNSwMxEJ3Ur1q/qh69BItQL2VXBD0WROixgq2FdinZNNvGZpMlyQpl6X/w4kERr/4fb/4b03YP2vpg4PHeDDPzwkRwYz3vGxXW1jc2t4rbpZ3dvf2D8uFR26hUU9aiSijdCYlhgkvWstwK1kk0I3Eo2EM4vpn5D09MG67kvZ0kLIjJUPKIU2Kd1O4lvHp73i9XvJo3B14lfk4qkKPZL3/1BoqmMZOWCmJM1/cSG2REW04Fm5Z6qWEJoWMyZF1HJYmZCbL5tVN85pQBjpR2JS2eq78nMhIbM4lD1xkTOzLL3kz8z+umNroOMi6T1DJJF4uiVGCr8Ox1POCaUSsmjhCqubsV0xHRhFoXUMmF4C+/vEraFzXfq/l3l5V6I4+jCCdwClXw4Qrq0IAmtIDCIzzDK7whhV7QO/pYtBZQPnMMf4A+fwCn5I6H</latexit><latexit sha1_base64="jh7gUoALVOlBmg3iPkbszI9veEE=">AAAB7XicbVBNSwMxEJ3Ur1q/qh69BItQL2VXBD0WROixgq2FdinZNNvGZpMlyQpl6X/w4kERr/4fb/4b03YP2vpg4PHeDDPzwkRwYz3vGxXW1jc2t4rbpZ3dvf2D8uFR26hUU9aiSijdCYlhgkvWstwK1kk0I3Eo2EM4vpn5D09MG67kvZ0kLIjJUPKIU2Kd1O4lvHp73i9XvJo3B14lfk4qkKPZL3/1BoqmMZOWCmJM1/cSG2REW04Fm5Z6qWEJoWMyZF1HJYmZCbL5tVN85pQBjpR2JS2eq78nMhIbM4lD1xkTOzLL3kz8z+umNroOMi6T1DJJF4uiVGCr8Ox1POCaUSsmjhCqubsV0xHRhFoXUMmF4C+/vEraFzXfq/l3l5V6I4+jCCdwClXw4Qrq0IAmtIDCIzzDK7whhV7QO/pYtBZQPnMMf4A+fwCn5I6H</latexit><latexit sha1_base64="jh7gUoALVOlBmg3iPkbszI9veEE=">AAAB7XicbVBNSwMxEJ3Ur1q/qh69BItQL2VXBD0WROixgq2FdinZNNvGZpMlyQpl6X/w4kERr/4fb/4b03YP2vpg4PHeDDPzwkRwYz3vGxXW1jc2t4rbpZ3dvf2D8uFR26hUU9aiSijdCYlhgkvWstwK1kk0I3Eo2EM4vpn5D09MG67kvZ0kLIjJUPKIU2Kd1O4lvHp73i9XvJo3B14lfk4qkKPZL3/1BoqmMZOWCmJM1/cSG2REW04Fm5Z6qWEJoWMyZF1HJYmZCbL5tVN85pQBjpR2JS2eq78nMhIbM4lD1xkTOzLL3kz8z+umNroOMi6T1DJJF4uiVGCr8Ox1POCaUSsmjhCqubsV0xHRhFoXUMmF4C+/vEraFzXfq/l3l5V6I4+jCCdwClXw4Qrq0IAmtIDCIzzDK7whhV7QO/pYtBZQPnMMf4A+fwCn5I6H</latexit>

Structured model-free policies for physical construction in humans and AI

Hamrick et al., 2018, Proc Cog Sci

Structured model-free policies for physicalconstruction in humans and AI

Graph-structured representations for model-free RL

48

Zambaldi et al., 2018, arXiv/under review

Relational deep reinforcement learningUnderlying graphObservation

Bra

nch

leng

th =

1B

ranc

h le

ngth

= 3

0 1 2 3 4 5 6 7 8Environment steps 1e8

1e8

0.0

0.2

0.4

0.6

0.8

1.0

Frac

tion

solv

ed

Environment steps

Frac

tion

solv

ed

0 2 4 6 8 10 12 140.0

0.2

0.4

0.6

0.8

1.0

Relational (1 block)Relational (2 blocks)Baseline (3 blocks)Baseline (6 blocks)

Relational (2 block)Relational (4 blocks)

Box-World:• Acquire gem (white) by

opening a sequence of locked boxes

• Model-free (A2C) with self-attention/GN state representation, and message-passing

Relational deep reinforcement learning

Box-World: • Acquire gem (white) by

opening a sequence of locked boxes

• Model-free (A2C) withself-attention / GN state representation, and message-passing

49Zambaldi et al., 2018, arXiv/under reviewZambaldi et al., 2018, arXiv/under review

Relational deep reinforcement learning

Box-World:• Acquire gem (white) by

opening a sequence of locked boxes

• Model-free (A2C) with self-attention/GN state representation, and message-passing

ConclusionsHuman use richly structured generative knowledge• Combinatorial generalization: “Infinite use of finite means”

• Object- and relation-centric representations

• Structured mental simulation

Graph Networks: strong relational inductive bias• Naturally support combinatorial generalization via compositional sharing

• Graph-structured representations and policies

• Open-source library: github.com/deepmind/graph_nets (with demos, includingphysics!)

50

Reject false choices

• The “bias versus variance trade-off” is real—however the emphasis shouldn’t be on “versus", but rather on “trade-off”.

• Biology doesn’t choose between nature versus nurture. It uses nature andnurture jointly, to build wholes which are greater than the sums of their parts.

• There’s great promise in synthesizing new techniques by drawing on the full AI toolkit and marrying the best approaches from today with those which wereessential during times when data and computation were at a premium.

51

• Nature and Nurture

• Structure and Flexibility

• Symbolic and Connectionist

• Hand-engineered and End-to-end