Upload
embedded-vision-alliance
View
78
Download
1
Embed Size (px)
Citation preview
Copyright © 2017 Camio 1
real-time video search
Carter Maslan
May 2017
Blending Cloud and Edge Machine Learning to Deliver Real-time
Video Monitoring
Copyright © 2017 Camio 3
In deluge of video, less is more
Copyright © 2017 Camio 4
sport utility vehicle
Camio makes video simple & useful
cameras that learn
cloud service
client firmware
works with any camera on any network
Copyright © 2017 Camio 5
• Compute Power
deep learning +
inexpensive cameras +
cloud-connected compute capacity
• Social Expectation
transparency & accountability
+ security threats +
always-connected mobility
Why now? First time feasible & expected
Copyright © 2017 Camio 7
Historical challenges
Problem
Limited edge compute power
Expensive cloud compute
Limited Internet bandwidth
False positives
Diverse workloads
Mitigation
RTSP streams via Camio Box
Real-time event ranking
Metadata-first pipeline
Adaptive motion filters
Hooks for callback extensions
Copyright © 2017 Camio 10
• Uploads metadata [and video]
• Modular cloud / edge compatible components
• General-purpose task managers
• Validated firmware updates
• Remote logging
Plumbing is important
Effort for Edge + Cloud is 90% plumbing
Copyright © 2017 Camio 11
The Camio Box and Cloud Pipelines
Removal of
spurious motion
Motion
Detection
Events, time
compression
Box
Cloud
Indexing,
storage,
search
ML:
important for
user?
ML: scene
recognition
ML: object
recognition
Current
Copyright © 2017 Camio 12
The Camio Box and Cloud Pipelines
Removal of
spurious motion
Motion
Detection
Events, time
compression
Box
Cloud
Indexing,
storage,
search
ML:
important for
user?
ML: scene
recognition
ML: object
recognition
Next release
Copyright © 2017 Camio 13
The Camio Box and Cloud Pipelines
Removal of
spurious motion
Motion
Detection
Events, time
compression
Box
Cloud
Indexing,
storage,
search
ML:
important for
user?
ML: scene
recognition
ML: object
recognition
~ June
Copyright © 2017 Camio 14
The Camio Box and Cloud Pipelines
Removal of
spurious motion
Motion
Detection
Events, time
compression
Box
Cloud
Indexing,
storage,
search
ML:
important for
user?
ML: scene
recognition
ML: object
recognition
~ Summer
Copyright © 2017 Camio 15
• Edge
• Fat content ML
• High frame rate ML
• Sequential operations
• Use-it-or-lose it state
• Counters (and math)
How to decide what’s Edge vs. Cloud?
• Cloud
• Parallelizable spikes
• Long async operations
• Context & feedback loops
• Shared ML models
• Services & hooks
Driven by available hardware, quality, cost and bandwidth
Copyright © 2017 Camio 16
• User-specific query parsers
• Parser is augmented by labels observed in user’s own history.
• Precedence and boolean logic inferred from known cameras, zones, colors, direction
of movement, etc.
• Unrecognized query terms still applied as filters (e.g., OCR text).
• Natural Language Parser is used for dates, times, relative ranges.
• Minimizes need for search operators (only “v:{{view_token}}”).
• /api/search
• Text-focused search is voice-input friendly.
• E.g., https://camio.com/api/search?text=people+8am+to+10am
• JSON supports flexible metadata/media responses.
How Camio Search Works
Copyright © 2017 Camio 17
Example hardware
VM
Box
● 3 1080p cams
● ARM A53
● Quad-core
● 2 GHz
● 2 GB RAM
● Gbit Ethernet
Box Pro
● 16 1080p cams
● Intel i5-5200U
● Dual core (8 equiv)
● 2.2 GHz
● 8 GB RAM
● Gbit Ethernet
Box Virtual
● Linux VM
● Deploy on local
hardware
● Grow with the need
● Download and go!
Copyright © 2017 Camio 18
• https://www.embedded-vision.com/platinum-
members/embedded-vision-alliance/embedded-vision-
training/videos/pages/feb-2017-member-meeting-camio
• https://camio.com/metadata
• https://api.camio.com
Resources