THE ROLE OF AI IN A VR WORLD -...

In partnership with

THE ROLE OF AI IN A VR WORLDNirmal Mehta - @normalfaults - Bayesian by BirthDrew Farris - @drewfarris – Grudgingly Bayesian Cameron Kruse - @camkruse –Bayesian by Default

OCTOBER 2018

@BoozAllen | #AIVR | #GTC18

INTRODUCTIONS

1

Nirmal Mehta:Chief Technologist. He has 10 years of Distributed Application architectures andemerging technology research, prototype development and implementation.He leads the firm’s efforts in Immersive Machine Intelligence and emergingtechnology strategy. He focuses on bringing leading edge technologies toenterprise systems for commercial and public-sector clients.

Cameron Kruse:Lead Technologist. He works within Booz Allen’s Strategic Innovation Groupworking on projects at the intersection of AI and immersive technology. Helikes working in this space as he sees technology as something that should helphumans better explore and connect with the world around us.

Drew Farris:Chief Technologist. He is one of Booz Allen’s machine learning subject matterexperts. He mostly focused on Information Retrieval and Natural LanguageProcessing, he has a latent love of Virtual Reality, with an undergrad degree inComputer Graphics.

Booz Allen Hamilton GTC DC 2018 @BoozAllen | #AIVR | #GTC18

IMMERSIVE TECHNOLOGY NEEDS AI AI NEEDS IMMERSIVE TECHNOLOGY

2

ALL EMERGING TECHNOLOGY GAIN PREVALENCE AS AN ECOSYSTEM OF CONNECTED TECHNOLOGIES. TO CHART A PATH INTO THE FUTURE, IT IS BETTER TO LOOK AT THE CONNECTIONS BETWEEN TECHNOLOGIES.

• We see the key convergences creating theconnections that will shape the future to beCloud Computing, AI, and ImmersiveTechnology.

• Both AI and Immersive are diverse sets oftechnologies. Today we will put most of ourfocus on the intersection of AI and VR.

• The convergence of AI and AR is also veryinteresting, but for the sake of clarity we’llleave it for a future conversation.

Cloud

AI Immersive

Booz Allen Hamilton GTC DC 2018 @BoozAllen | #AIVR | #GTC18

AI 3d Model Video RobotsImmersive

SMART AGENTS: REINFORCEMENT LEARNING

3

LEARNING STRATEGIES WITH REWARD FUNCTIONS

David Busch (@HappySlice) – Booz Allen Hamilton

Booz Allen Hamilton GTC DC 2018 @BoozAllen | #AIVR | #GTC18

SMART AGENTS: DATA-DRIVEN ANIMATION

4

LEARNING COMPLEX KINEMATIC SYSTEMS

CrowdAI, NIPS 2017 Learning to Run Challenge

Booz Allen Hamilton GTC DC 2018 @BoozAllen | #AIVR | #GTC18

SMART AGENTS: MULTI-AGENT LEARNING

5

AGENTS DEVELOP STRATEGIES WHEN TRAINING AGAINST HUMANS IN MULTI-PLAYER GAMES

OpenAI Five: Dota Gameplay

Booz Allen Hamilton GTC DC 2018 @BoozAllen | #AIVR | #GTC18

POSE ESTIMATION FOR INTERACTIVITY

6

2-D TO 3-D POINT DETECTION

OpenPose

Booz Allen Hamilton GTC DC 2018 @BoozAllen | #AIVR | #GTC18

POSE ESTIMATION FOR INTERACTIVITY

7

MOTION TRANSFER

Chan, et al. – Everybody Dance Now

Booz Allen Hamilton GTC DC 2018 @BoozAllen | #AIVR | #GTC18

HUMAN MESH RECOVERY

8

Peng, et al.

Kanazawa, et al

Booz Allen Hamilton GTC DC 2018 @BoozAllen | #AIVR | #GTC18

POSE ESTIMATION TECHNIQUES

9

Peng, et al.

Booz Allen Hamilton GTC DC 2018 @BoozAllen | #AIVR | #GTC18

LEARNING MOTION FROM VIDEO

10

COMBINING POSE ESTIMATION TECHNIQUES AND DATA TO DEVELOP ANIMATION BEHAVIORS

Peng, et. al. – Reinforcement Learning of Physical Skills from Videos

Booz Allen Hamilton GTC DC 2018 @BoozAllen | #AIVR | #GTC18

USING GENERATIVE ADVERSARIAL NETWORKS TO CREATE 3D CONTENT

11Booz Allen Hamilton GTC DC 2018

CONTENT CREATION IS THE MOST EXPENSIVE AND TIME CONSUMING PART OF DEVELOPING IMMERSIVE EXPERIENCES. USING AI TO CREATE 3D CONTENT COULD CUT COSTS AND MAKE ARTISTS FASTER AT THEIR JOBS.

Wu, Jiajun, et al. - Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling

@BoozAllen | #AIVR | #GTC18

USING GENERATIVE ADVERSARIAL NETWORKS TO CREATE 3D CONTENT

12Booz Allen Hamilton GTC DC 2018

CONTENT CREATION IS THE MOST EXPENSIVE AND TIME CONSUMING PART OF DEVELOPING IMMERSIVE EXPERIENCES. USING AI TO CREATE 3D CONTENT COULD CUT COSTS AND MAKE ARTISTS FASTER AT THEIR JOBS.

Wu, Jiajun, et al. - Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling

@BoozAllen | #AIVR | #GTC18

USING GENERATIVE ADVERSARIAL NETWORKS TO CREATE 3D CONTENT

13Booz Allen Hamilton GTC DC 2018

CONTENT CREATION IS THE MOST EXPENSIVE AND TIME CONSUMING PART OF DEVELOPING IMMERSIVE EXPERIENCES. USING AI TO CREATE 3D CONTENT COULD CUT COSTS AND MAKE ARTISTS FASTER AT THEIR JOBS.

Wu, Jiajun, et al. - Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling

@BoozAllen | #AIVR | #GTC18

TRAINING AI ALGORITHMS IN 3D ENVIRONMENTS

14

THERE ARE MANY TIMES WHEN TRAINING DATA IS TOO COSTLY OR CANNOT BE OBTAINED. WE HAVE SEEN SUCCESS TRAINING AI IN 3D ENVIRONMENTS.

Andre Nguyen – Booz Allen Hamilton

Booz Allen Hamilton GTC DC 2018 @BoozAllen | #AIVR | #GTC18

TRAINING AI ALGORITHMS IN 3D ENVIRONMENTS

15

THERE ARE MANY TIMES WHEN TRAINING DATA IS TOO COSTLY OR CANNOT BE OBTAINED. WE HAVE SEEN SUCCESS TRAINING AI IN 3D ENVIRONMENTS.

Andre Nguyen – Booz Allen Hamilton

Booz Allen Hamilton GTC DC 2018 @BoozAllen | #AIVR | #GTC18

TRAINING ROBOTS THROUGH IMITATION USING VIRTUAL REALITY AS THE INTERFACE

16Booz Allen Hamilton GTC DC 2018

TRAINING ROBOTS TO PERFORM HUMAN TASKS IS USUALLY COSTLY AND HARD. INSTEAD OF TRYING TO PROGRAM A ROBOT WE COULD TRAIN ROBOTS THROUGH IMITATION.

OpenAI – Teaching Robots to learn

@BoozAllen | #AIVR | #GTC18

TRAINING ROBOTS THROUGH IMITATION USING VIRTUAL REALITY AS THE INTERFACE

17Booz Allen Hamilton GTC DC 2018

OpenAI – Teaching Robots to learn

TRAINING ROBOTS TO PERFORM HUMAN TASKS IS USUALLY COSTLY AND HARD. INSTEAD OF TRYING TO PROGRAM A ROBOT WE COULD TRAIN ROBOTS THROUGH IMITATION.

@BoozAllen | #AIVR | #GTC18

TRAINING ROBOTS THROUGH IMITATION USING VIRTUAL REALITY AS THE INTERFACE

18Booz Allen Hamilton GTC DC 2018

OpenAI – Teaching Robots to learn

TRAINING ROBOTS TO PERFORM HUMAN TASKS IS USUALLY COSTLY AND HARD. INSTEAD OF TRYING TO PROGRAM A ROBOT WE COULD TRAIN ROBOTS THROUGH IMITATION.

Vision Network

@BoozAllen | #AIVR | #GTC18

TRAINING ROBOTS THROUGH IMITATION USING VIRTUAL REALITY AS THE INTERFACE

19Booz Allen Hamilton GTC DC 2018

OpenAI – Teaching Robots to learn

TRAINING ROBOTS TO PERFORM HUMAN TASKS IS USUALLY COSTLY AND HARD. INSTEAD OF TRYING TO PROGRAM A ROBOT WE COULD TRAIN ROBOTS THROUGH IMITATION.

Vision Network

Imitation Network

@BoozAllen | #AIVR | #GTC18

TRAINING ROBOTS THROUGH IMITATION USING VIRTUAL REALITY AS THE INTERFACE

20Booz Allen Hamilton GTC DC 2018

TRAINING ROBOTS TO PERFORM HUMAN TASKS IS USUALLY COSTLY AND HARD. INSTEAD OF TRYING TO PROGRAM A ROBOT WE COULD TRAIN ROBOTS THROUGH IMITATION.

OpenAI – Teaching Robots to learn

@BoozAllen | #AIVR | #GTC18

In partnership with

PANEL DISCUSSIONNirmal Mehta - @normalfaults - Bayesian by BirthDrew Farris - @drewfarris – Grudgingly Bayesian Cameron Kruse - @camkruse –Bayesian by Default

OCTOBER 2018

@BoozAllen | #AIVR | #GTC18

CONTINUE THE CONVERSATION

22

Find out more about BAH immersive studio at immersive.bah.comGame Engines:- https://unity3d.com/machine-learning- https://unrealcv.org/

Generative Adversarial Networks:- http://3dgan.csail.mit.edu/- http://papers.nips.cc/paper/5423-generative-adversarial-nets- https://arxiv.org/abs/1610.07584

Pose Estimation, Human Mesh Recovery, Everybody Dance- https://xbpeng.github.io/projects/SFV/index.html- https://akanazawa.github.io/hmr/- https://arxiv.org/abs/1808.07371

Open AI Five (DOTA): https://blog.openai.com/openai-five/

Open AI (Teaching Robots to learn): https://blog.openai.com/robots-that-learn/

Booz Allen Hamilton GTC DC 2018

IF YOU HAVE A USE CASE OR ARE INTERESTED IN COLLABORATING COME TALK TO US!

@BoozAllen | #AIVR | #GTC18