Image Processing from UAS Sensor Data: Prospects and

Brainlike, Inc. 1

Image Processing from UAS Sensor Data: Prospects and Challenges*

Robert J. (Bob) Jannarone, Gregory Schaefer, and Ciani Sparks, Brainlike, Inc.

Darren Ireland, Kathleen Leonard, and Dale Funk,

LGL Alaska Research Associates

* Some results are based on images that were kindly supplied by Resource Management, LLC and Shell Exploration & Science.

Brainlike, Inc. 2

Presentation Roadmap

1) Wildlife Detection Scenario 2) Added Value Analysis : two examples 3) Operational Prospects 4) Product Operation: a peak under the hood

(time permitting)

Brainlike, Inc. 3

Can you find caribou in this image?

Brainlike, Inc. 4

Can you find them in these “chips?”

Brainlike, Inc. 5

Can you identify them in these chips?

Brainlike, Inc. 6

Caribou Image: detail location

Area of Detail (next slide)

Brainlike, Inc. 7

Caribou Image: detail

Brainlike, Inc. 8

Caribou Detection: “alert map”

Brainlike, Inc. 9

Can you find whales in these images?

A.

B.

C.

Brainlike, Inc. 10

Can you find them in these image chips?

A.

B.

C.

Brainlike, Inc. 11

Can you identify them in these chips?

Brainlike, Inc. 12

Whale Image “Alert Map”

Brainlike, Inc. 13

Independent Validation (LGL et. al: Society for Marine Mammalogy 2015)

“Testing of the detection software occurred on images from surveys that contained tens-of-thousands of images without whales as well as tens of images with whales that had been withheld from the detector development phase. The overall successful detection rate was approximately 70%, with 100% of whales being detected on many surveys. The time required to manually review the automated processing output was less than 2% of the full manual analysis time.

Brainlike, Inc. 14

Presentation Roadmap

1) Wildlife Detection Scenario 2) Added Value Analysis: two examples 3) Operational Prospects 4) Product Operation: a peak under the hood

(time permitting)

Brainlike, Inc. 15

Effort Reduction Example (36 Mpix RGB)

Brainlike, Inc. 16

Continuous Monitoring Example: monetized benefit analysis:

Brainlike, Inc. 17

Monetized Benefit Analysis Detail

Likelihoods

Component Costs

Overall Risk/Utility

Brainlike, Inc. 18

Presentation Roadmap

1) Wildlife Detection Scenario 2) Added Value Analysis: two examples 3) Operational Prospects 4) Product Operation: a peak under the hood

(time permitting)

Brainlike, Inc.

Triage Extended Definition

19

3. (in data processing) The rapid reduction of massive data streams to actionable information.

Brainlike, Inc.

Whale Detection Triage (and beyond)

20

2014 Product: • Locates most targets automatically • Reduces analysis time by over 90% • Processes images in near real-time

2014 Objective: Deliver an automated alternative to manual, post-flight methods for detecting marine mammals from airborne images in a cluttered environment

2015-16 Goals: • General-purpose image–based target detection • Fully automated in-flight/post-flight detection • On-board, real-time processing • Full resolution, real-time “chip” (VGA-sized sub-frame) transmission • Small packet size • Low SWaP profile

Payoffs: • Reduced analyst/operator effort • Real-time, high resolution chip display • Low transmission bandwidth • Low transmission power • High sensor persistence • Low development/integration cost • Big return on investment

Brainlike, Inc. 21

On-Board Triage: UAS “Vision”

21

High Resolution

Sensors

A P I

D

r i v e r

Triage

Processor Low Bandwidth Information

High Bandwidth

Data

Telemetry Control

Sensor Control

Brainlike, Inc. 22

Presentation Roadmap

1) Wildlife Detection Scenario 2) Added Value Analysis: two examples 3) Operational Prospects 4) Product Operation: a peak under the hood

(time permitting)

Brainlike, Inc.

Data Layout

23

Brainlike, Inc.

Shortcut

24

Brainlike, Inc.

Output Structure, 1

25

Brainlike, Inc.

Output Structure, 2

26

Brainlike, Inc.

Bat File Structure

27

Brainlike, Inc.

Whale Templates: two simple examples

28

Brainlike, Inc.

Template Matching: rotation example

29

Brainlike, Inc.

Template Matching: rescaling example

30

Brainlike, Inc. 31

Caribou Template Matching

Brainlike, Inc. 32

Caribou Template Matching

(cont.)

Brainlike, Inc. 33

Presentation Summary

1) Wildlife Detection Scenario 2) Added Value Analysis: two examples 3) Operational Prospects 4) Product Operation: a peak under the hood

(time permitting)

Brainlike, Inc. 34

Thank you very much for your interest!

Robert J. (Bob) Jannarone bobjannarone@brainlike.com

(619) 887-1153 (cell)