Embed Size (px)
Citation preview
Copyright © 2017 1
Using AI to Read Human Body
Language In Real-Time from Standard
Video
Copyright © 2017 2
Current SolutionsKeyboardsVoiceSpecialty Sensors / Cameras
ShortcomingsExpensive to own / operateTime consuming to set up / useRequire specialty sensors / suits
How do we enable computers to communicate with us in a natural way
Friction: Humans are Physical / Computers are Digital
Copyright © 2017 3
Humans are Visual Creatures80% of language is non-verbal80% of that is body language
Dogs have co-evolved with humans to read human body language
Inspired by Nature
Security Work Companionship Entertainment Navigation
Copyright © 2017 4
Solution: Teach Computers to Read Human Body Language
How do we make the digital version of “Man’s Best Friend”?Solution:
Ordinary cameras + GPUs + deep learning Value Proposition
Natural InteractivityConsumer Hardware keeps getting cheaper and betterDeep Learning Brains keep getting smarter
Copyright © 2017 5
Why Now? Peace Dividends of the Game Console Wars
Kinect paved the wayGPUs enable deep learningGame Engines enable synthetic data and thousands of new interactive applications
Copyright © 2017 6
wrnch™: Human Pose Estimation Engine
User Application
wrnch™
In: Video
Out: Humans
Copyright © 2017 7
Deep LearningAccurateRobustKeeps getting smarter
Real-timeEnables interactivity
No Specialty HardwareConsumer grade cameras & GPUsTake as input: any video feed from anywhere
Wrnch™ Engine: Unique Features
Copyright © 2017 8
Live Demos
Copyright © 2017 9
Applications: AR / VR
Copyright © 2017 10
Applications: Robot Interaction
Copyright © 2017 11
In City for safetyIn Home for elder care, etc.In Store for retailIn Vehicle for awarenessIn Factories for accidents, etc.
Applications: Security / Human Monitoring
Copyright © 2017 12
Applications: Health & Wellness
Copyright © 2017 13
Applications: Eyes for Virtual Assistants
Copyright © 2017 14
Applications: Sport Analytics
Copyright © 2017 15
Fast allowing real-time interactivityAccurate tracking of 63 body parts per person including fingersRobust across large numbers of people in crowded conditions
BodySLAM™: Unique Features
Copyright © 2017 16
Wrnch Deep Learning Training Pipeline
Real Data Synthetic Data DL Models
wrBrain
Wrnch DL Training Framework
Copyright © 2017 17
Wrnch Deep Learning Inferencing Architecture
2D Pose
3D Pose
Tracker Activity Recognition
Gesture Recognition
Segmentation 2D Skeletons
3D Skeletons IDs Locations Activities Gestures
2D Video
2D Video
2D Video
3D Shape
3D Meshes
Identifier
TensorRT
Copyright © 2017 18
GPU CPU OS Total(mSec)
2D CNN 3D CNN Misc
Jetsen TX2 n/a Linux 297 200 84 13K80 Intel Xeon E5-
2686v4 2.3GHz Linux 131 90 36 51080 Intel i7-5930K
3.5GHz Linux 37 22 12 31080 Intel i7-5930K
3.5GHz Windows 41 24 13 4
Titan XP Intel i7-5930K 3.5GHz Linux 30 16 11 3
Titan XP Intel i7-5930K 3.5GHz Windows 34 18 12 4
Runtime Performance
Copyright © 2017 19
Gesture RecognitionPoint AtThumbs upFist
Activity RecognitionFall detectionPick up itemPut down item
Multiple Camera SupportTriangulationPersistent tracking3D
Roadmap
Copyright © 2017 20
Questions?
