Embed Size (px)
Suzanne Little Dublin City University
Insight Centre for Data Analytics
Cloud Large Scale Video Analysis
H2020-ICT-2015 Cloud-LSVA
Big Data - research
Understanding high-volume, high-speed
multimedia data – a computer scientist’s view
• Big data in ADAS means manual annotation is
infeasible
• Data that’s not annotated is not usable
• Annotation via Machine Learning
• Deep learning: a step change in computer vision
• Semantics: formal meaning
~15-20 TB/day
~300 hr (2014)
Lossy content
Large amount of files
Worldwide upload points
~10-50 TB/day/vehicle
~8 hr collection window
Lossless content
Reduced amount of files
Limited upload points
ADAS Context
Open-road Acquisition
Big Data
Volume
Velocity
Variety
Storage Processing
Annotating video
Road scene Ground-truth Camera setup
Scene level
Static Objects
Dynamic Objects
Background
Actors
Vehicles
Classes
CamSeq dataset
• 101 original frames
• 101 ground truth frames
Teaching Computers to “see”
• Supervised Machine Learning uses labelled examples to train
computer systems
• Large numbers of examples are required for every concept
Annotation 4564 654654 06465 46546 54604640
4892 1894 24087 5469876 5868765
4840 6847684 68406 8484 65
41847 21098 98065 4 98 406546984
6046 84065 484 0 54 8406 4868 46
5180 2 3210684 05418 940 6541
08 405 4198 4 0 541 98 40 65 498
40 65 41 98 40 654 9840 65403
216984 06 54 98406 54 9840 6 541
98 405418 40 6 549 804 6 54 098
40 654 984
Campo dei Miracoli
La Torre di Pisa
Field of Miracles, Pisa, Italy
Leaning Tower
Pisa, Italy
Sunny day in Pisa
My holidays
magic
What about Deep Learning?
A deep convolutional network
Conv1
Input
Conv2 Conv3
Max
pooling
Max
pooling
Max
pooling
Max
pooling
Conv4 Conv5
Output
Deep Learning
• Rebranding of an old idea?
• Now possible because of powerful hardware (GPUs)
• Requires large quantities of unlabelled input
• Can require a lot of working memory
• Configuring, optimising complexity and performance is a
bit of an art at present
Semantics
• The meaning and context of annotations
• An ontology formally specifies the terms and their
relationships
• Can be used for:
– Query expansion (car is a vehicle)
– Inference (cars have doors)
– Vocabulary (sidewalk == footpath)
Object
Child
Child Child
Child
Child
Semantics
Annotation for ADAS is more than labels
Volume
Velocity Variety Veracity Value
Cloud-LSVA
Move from manual to (semi-) automatic video annotation - Scale
- Accuracy
• Aiding human annotators
• Supporting data growth
• Adapting computer vision models
• Operating on small (vehicle) and large (cloud) scale
Value
Acknowledgments
• Cloud-LSVA is funded by the EU H2020 framework under grant
number 688099.
• Specific contributions to this talk were made by Dr Kevin
McGuinness (DCU), Dr Houssem Chatbri (DCU) and Manuel Reis-
Monteiro (Valeo), Dr Marcos Nieto (Vicomtech)
• For more information on Cloud-LSVA see http://cloud-lsva.eu
• Contact: [email protected]
