20
AWASS 2013 Case Study Computational Self-awareness in Smart-Camera Networks Lukas Esterle | LakesideLabs, Klagenfurt University [email protected] Peter R. Lewis | CERCIA, University of Birmingham

Computational Self-awareness in Smart-Camera Networks - Lukas Esterle

Embed Size (px)

DESCRIPTION

Lukas Esterle's talk from AWASS 2013

Citation preview

Page 1: Computational Self-awareness in Smart-Camera Networks - Lukas Esterle

AWASS 2013 Case Study

Computational Self-awareness in Smart-Camera Networks

Lukas Esterle | LakesideLabs, Klagenfurt [email protected]

Peter R. Lewis | CERCIA, University of Birmingham

Page 2: Computational Self-awareness in Smart-Camera Networks - Lukas Esterle

• Overview of the EPiCS Project

• Surveillance and Camera Networks

• Smart-Cameras

• Multi-Camera Tracking of Objects

• Challenges in Smart-Camera Networks for the week

• Self-Awareness in Smart-Camera Networks

• Prerequisites

2AWASS 2013 Case Study | Lukas Esterle | June 24, 2013

Outline of this talk

Page 3: Computational Self-awareness in Smart-Camera Networks - Lukas Esterle

The EPiCS Project: Motivation

3AWASS 2013 Case Study | Lukas Esterle | June 24, 2013

• What are the characteristics of future complex systems?

Large Heterogeneous

Uncertain

Dynamic Decentralised

Page 4: Computational Self-awareness in Smart-Camera Networks - Lukas Esterle

The EPiCS Project: Motivation

4AWASS 2013 Case Study | Lukas Esterle | June 24, 2013

• We are increasingly faced with systems that are …

… and often have conflicting requirements!

Large Heterogeneous

Uncertain

Dynamic

Decentralised

Page 5: Computational Self-awareness in Smart-Camera Networks - Lukas Esterle

• Requirements of the application domain can conflict in functionality, performance, resource usage, costs, reliability, safety and security.

• Design systems as collections of self-aware and self-expressive nodes.

– Use online learning and adapt to specific scenario

– Algorithm selection during runtime focused on goals of node

– Interaction between nodes to drive global behaviour.

The EPiCS Project: Motivation

5AWASS 2013 Case Study | Lukas Esterle | June 24, 2013

Page 6: Computational Self-awareness in Smart-Camera Networks - Lukas Esterle

6AWASS 2013 Case Study | Lukas Esterle | June 24, 2013

• High performance computing cluster for financial applications

• Multi-camera networks for multi-object tracking

• Hypermusic

Page 7: Computational Self-awareness in Smart-Camera Networks - Lukas Esterle

Surveillance and Camera Networks

7AWASS 2013 Case Study | Lukas Esterle | June 24, 2013

Image: http://commons.wikimedia.org/wiki/File:Three_Surveillance_cameras.jpg

up to

4.2 million

CCTV cameras in the UK

McCahill, Michael, and Clive Norris. "CCTV

in Britain." Center for Criminology and

Criminal Justice-University of Hull-United

Kingdom (2002): 1-70.

Page 8: Computational Self-awareness in Smart-Camera Networks - Lukas Esterle

• Use of Camera Systems

– Traffic monitoring

– Crime prevention

– Crowd control

– Surveillance production lines

– Building monitoring

– Person & Object tracking

Surveillance Cameras

9AWASS 2013 Case Study | Lukas Esterle | June 24, 2013

Image: http://www.fh-wien.ac.at/uploads/pics/080611_Asfinag.JPG

Page 9: Computational Self-awareness in Smart-Camera Networks - Lukas Esterle

• Tracking is the process of identifying and locating a (moving) predefined ‘model’ within consecutive frames of a video.

Person & Object Tracking

10AWASS 2013 Case Study | Lukas Esterle | June 24, 2013

Page 10: Computational Self-awareness in Smart-Camera Networks - Lukas Esterle

• Tracking is the process of identifying and locating a (moving) predefined ‘model’ within consecutive frames of a video.

• Drawbacks of using ‘dumb’ cameras

– Raw video data

– Data has to be transmitted & stored

– Data has to be processed

– Privacy issues

Person & Object Tracking

11AWASS 2013 Case Study | Lukas Esterle | June 24, 2013

Page 11: Computational Self-awareness in Smart-Camera Networks - Lukas Esterle

• Smart cameras combine processing unit with image sensor

– General purpose smart-cameras from off-the-shelf components

– Special purpose smart-cameras for specific applications

• Allows to process images on the camera

– Raw image data does not need to leave the camera

– Operator only gets relevant and/or aggregated data

– Operator only gets information about certain events

Smart-Cameras

12AWASS 2013 Case Study | Lukas Esterle | June 24, 2013

Page 12: Computational Self-awareness in Smart-Camera Networks - Lukas Esterle

• Flexible high-performance platform running Linux

– PandaBoardes as main board with USB / CSI / GPMC connected components

– TI OMAP 4460 processor, 2x ARM Cortex A9 @ 1.2 GHz, 2x Cortex M3

– 1GB DDR2 SDRAM

– SD-Card, USB, DVI, HDMI, audio

– 802.11 b/g/n

– Bluetooth 2.1 + Bluetooth 4 Low Energy

– Ubuntu 12.04 Linux

• Camera modules

– USB

– Camera Serial Interface (CSI)

– External module connected via GPMC using FPGA

Smart-Cameras

13AWASS 2013 Case Study | Lukas Esterle | June 24, 2013

Page 13: Computational Self-awareness in Smart-Camera Networks - Lukas Esterle

Multi-Camera Tracking of Objects

15AWASS 2013 Case Study | Lukas Esterle | June 24, 2013

Camera 1 Camera 2 Camera 3

?

Camera 4

• Which camera to continue tracking?

• Identify next camera a priori or use a server or operator.

• What about dynamics / uncertainties?

Page 14: Computational Self-awareness in Smart-Camera Networks - Lukas Esterle

Distributed Tracking

16AWASS 2013 Case Study | Lukas Esterle | June 24, 2013

• Use auctions for exchanging tracking responsibilities – Cameras act as self-interested agents, i.e., maximize their own utility

– Selling camera (currently tracking object) opens the auction

– Other cameras return bids with price corresponding to “tracking” confidence

– Camera with highest bid continues tracking; trading based on Vickrey auction Bid C4

Camera 1 Camera 2

Camera 3

Camera 4

Initauction

Bid C3

Fully distributed approach

no a-priori topology knowledge required

Page 15: Computational Self-awareness in Smart-Camera Networks - Lukas Esterle

• On the node level

– How maximise utility while minimise communication

– When and how to advertise objects to other cameras

– How to value objects within the field of view of a camera

• On the network level

– How to define good camera strategies to optimise the network-wide outcome

Challenges in Multi-Camera Tracking

17AWASS 2013 Case Study | Lukas Esterle | June 24, 2013

Page 16: Computational Self-awareness in Smart-Camera Networks - Lukas Esterle

• Cameras build models/knowledge about the behaviour of the objects to be tracked

• Cameras derive models/knowledge from the objects behaviour about themselves

• Cameras create models/knowledge about their immediate environment

• Cameras derive models/knowledge about their neighbouring cameras

Self-awareness in Multi-Camera Systems

18AWASS 2013 Case Study | Lukas Esterle | June 24, 2013

Page 17: Computational Self-awareness in Smart-Camera Networks - Lukas Esterle

• You will be provided:

– The open-source simulation environment CamSimallowing you to simulate object tracking in multi-camera networks

– Various scenarios for the simulation tool

– Matlab scripts for evaluation of your simulation results

Main objective for the week:

Come up with a distributed way to assign tracking responsibilities in a smart camera network; preserving resources and keeping high tracking utility alike.

Self-awareness in Multi-Camera Systems

19AWASS 2013 Case Study | Lukas Esterle | June 24, 2013

Page 18: Computational Self-awareness in Smart-Camera Networks - Lukas Esterle

• During the week, you can expect to do the following:

– Learn about the difficulties in camera coordination for distributed tracking

– Work with provided strategies for distributed coordination of tracking responsibilities

– Build your own scenarios with the provided simulation tool

– Come up with your own strategy or modify one of the existing strategies to assign tracking responsibilities in a distributed camera network

– Optionally, you may want to relax one of the assumptions made in the simulation tool or come up with your own project for self-awareness in distributed smart-camera networks

Self-awareness in Multi-Camera Systems

20AWASS 2013 Case Study | Lukas Esterle | June 24, 2013

Page 19: Computational Self-awareness in Smart-Camera Networks - Lukas Esterle

• Proficiency in Java

• At least one member in the team with some experience with machine learning tools and techniques (the more the better).

• Knowledge of Matlab is a plus

Prerequisites

21AWASS 2013 Case Study | Lukas Esterle | June 24, 2013

Page 20: Computational Self-awareness in Smart-Camera Networks - Lukas Esterle

• HOW?

Compose a self-contained, short video explaining self-awareness accessible to a broad audience

• WHEN?

Submission Deadline:

July 31, 2013

• DETAILS

www.epics-project.eu/contest