Embed Size (px)
Citation preview
Defense Systems and AI-Enabled Vision Computing
Kevin Moran ([email protected])
WE INNOVATE. WE DELIVER.YOU SUCCEED.
Abaco Systems advances the capabilities of the warfighter by providing game changing mission ready embedded systems, components and technologies to defense contractors.
Our products reduce program risk, allow technology insertion with affordable readiness, and ultimately help platforms reach deployment sooner with lower cost
2
3
Highly Experienced Team of 800+ Professionals with Global Reach
4
Our Portfolio
5
ABACO SYSTEMS SMALL FORM FACTOR RUGGED BOXGVC1000
Rugged TX2 SoM
Digital Protocols
MilCAN / CAN
High Speed 10 Gig
Ethernet
Integrated SATA Storage
Expandable Future IO
Military Connectors
Designed for Rugged applications for use in Harsh environments
including Military Vehicles, UAVs, Robotics, Avionics and Industrial.
Aligned to military environmental specifications
-40°C to +71°C
6
ABACO SYSTEMS SMALL FORM FACTOR RUGGED BOXGVC2000
Designed for Rugged applications for use in Harsh environments
including Military Vehicles, UAVs, Robotics, Avionics and Industrial.
Aligned to military environmental specifications
-40°C to +71°C
Rugged GM107
Maxwell ADC
DAC
ARINC429 (October)
1553
28V I/O
Digital I/O
Audio
7
Vision functions supported by TX2-integrated GVC1000/2000
• Deep Learning, inference at the edge.
• Advanced image processing ISP and compression
• Data parallelism using CUDA and OpenCL
8
What is Autonomous?From the Oxford Dictionary…
• Freedom to govern itself or control its own affairs
• Freedom to act independently
• Device capable of operating without direct
human control
It is not simply…
• Automated,
• Remotely operated (unmanned),
• Guided.
For our purposes: Able to conduct complex, cooperative, extended missions
within broad objectives, principles and guidelines.
9
Modern Origin: ALVINN 1985-88
ALVINN Road Test - 1988
10
11
Vehicle AI: abridged chronology – personal / commercial
Chevy Tahoe: DARPA
Urban Challenge 2007Tesla S Autopilot 2014
ALVINN 1988
Robo-tramRobo-taxi Robo-pizza Robo-beer
12
Vehicle AI: abridged chronology – military
ALVINN 1988
Oshkosh TerraMax: DARPA
Urban Challenge 2007 Lockheed Martin F-35A 2014
Autonomous Platform Autonomous Underwater Vehicle Boeing QF-16 Robot Pack Mule
Tremendous variety of applications, missions, environments!
13
AI will serve many purposes
Logistics
AR displays, wearables
ISR
Cyber security, resiliency
War gaming/planning
Big data, cloud computing
Collaborative missions
Electronic warfare
Missiles
14
DoD invests in Autonomous and AI
Excerpted from roadmap:
• Autonomous system is self-directed to reach a human-directed goal.
• Machine learning…autonomous systems can develop modified strategies.
• In unforeseen situations the autonomous system finds the optimal solution.
• Smart teams of unmanned systems operating autonomously…conduct operations in
contested environments.
• Industry and academic partnerships will be critical.
DoD Unmanned Systems Roadmap
15
All Branches active with Autonomous R&DLand AirSea
Boeing QF-16
S-76 Helicopter - SARA
Cessna 208 – DARPA ALIAS
Sea Hunter
Swarm Patrol
UUV
Robotic Sentries
Cooperative Task Trucks
Legged Squad Support
16
Swarm capabilities across all branches
Air Force: Predator swarms Navy: Swarm Patrol BoatsArmy: Robotic Sentries
Common core:• Swarm is given broad cooperative mission.• Control is decentralized.• Vehicles extend communications beyond line of sight.• On-guard and patrolling 24x7.• Decision ability to move from defensive to offensive posture.
MDARS
SWARM
17
Autonomous vehicles and AR require vision
Both demand fidelity, accuracy and low latency!
…add registration and stabilization
Target classification, tracking, fusion, perception
18
Demanding visual environments and challenges
• 2D localization • Lack of intentionally active threats• Stable terrain• Blue skies
On-road: planar field
• 3D localization• Contested space: omni-directional threats • Explosions: altered terrain, EMPs• Degraded visual environments
“Off-road”: spherical field
Software - AXIS Enabled Middleware for High Performance
Image processing,
visualization & graphics
AXIS ImageFlex
19
AXIS ImageFlexImage processing and visualization toolkit
20
Vision solutions:General
22
Machine Vision examples Neural Network Classification
Neural Network Object Detection
CUDA Interoperability
Adaptive Image Fusion
GPU Stabilization
Image Morphing
Situational Awareness
23
Neural Network Classification
(CAFFE (Convolutional Architecture for Fast Feature Embedding) is a deep learning framework, originally developed at UC Berkeley.)
• Demonstrates integration of AI classifier into an
ImageFlex application.
• Uses a pre-trained CAFFE-based Googlenet neural network.
• Converted to an optimized inference engine via ImageFlex conversion
utility, this, in turn, uses NVIDIA’s TensorRT.
• ImageFlex also provides a AI annotator tool to assist labelling images to
prepare for neural net training for frameworks such as CAFFE using
NVIDIA’s DIGITs.
24
Neural Network Classification
0:17
25
Neural Network Object Detection
• Demonstrates integration of AI object detection
into an ImageFlex application
• Uses NVIDIA’s DetectNet object detection neural network.
• As per the simpler classifier, this is converted to an optimized inference
engine via ImageFlex conversion utility, that uses NVIDIA’s TensorRT.
• ImageFlex is used to draw the bounding boxes based on the detections
and certainty threshold.
26
Neural Network Object Detection
0:20
27
CUDA Interoperability
• Utilizes ImageFlex simple API to facilitate CUDA
and ImageFlex interoperability.
• Allows user to integrate their CUDA processing into an ImageFlex application.
• Eliminates need for complex and confusing OpenGL and CUDA interop code.
• API also provides OpenCL interoperability support.
28
CUDA Interoperability
0:20
29
Adaptive Image Fusion
• Two types of fusion mode available:
o Alpha blend – simple weighted
combination of two input images.
o Adaptive fusion – designed to maintain
high-resolution detail of both images.
• Provides per-pixel frequency cut & other fusion weightings. Enables tailoring to best
meet fusion demands of application and sensor attributes.
• Leverages GPU via OpenGL ‘shader’ language - OpenGLSL. GPU agnostic. High
performance, minimal CPU overhead.
30
Adaptive Image Fusion
0:32
Vision solutions:Augmented Reality / Displays
32
GPU Stabilization
• Use a CUDA-optimized algorithm to stabilize video,
correcting for shake in horizontal and vertical
planes, plus rotations zoom.
• Algorithm is based on the Lucas-Kanade Method.
33
GPU Stabilization
0:20
34
Distortion Correction and Image Morphing
Provides simple barrel / pin
cushion distortion correction.
Complex adaptive correction via “morphing” per grid.
More complex dense grid can morph to arbitrary
shape. E.g. to compensate for HMD eye pieces or
complex optics.
Each sub-rectangle in grid can morph in three
different ways to suite requirements.
35
Distortion Correction and Image Morphing
0:17
36
Situational Awareness
• ImageFlex provides API to create and draw cube maps.
• Provides the capability to generate a full or partial ‘skybox’ from live
camera array or equirectangular videos files.
• Facilities real-time 360 or full “spherical” situational awareness
applications via camera stitching.
• Configuration utility enable set-up of stitched “skybox” panoramic from
camera array and capture configuration parameters for skybox
application.
37
Situational Awareness
0:27
38
Conclusion and final messages
• Vital Defense is key to security. • Autonomous will transform Defense -
Artificial Intelligence as a linchpin.• GPGPU-enabled deep-learning
graphics/vision/compute solutions are critical.
• Opportunities for developers are incredible and incredibly important!
