Smart cameras as embedded systems

presented by preethi.m.v

• Smart cameras are equipped with a high-performance onboard computing and communication infrastructure, combining in a single embedded device–video sensing–processing

–communications

• Networks of embedded cameras can potentially support more complex and challenging applications:_Surveillance–Tracking–Motion analysis

• “From Analog to Digital Cameras”:• 1st generation surveillance: analog equipment

(closed circuit TV cameras transmitted video signal over analog lines)

• 2nd generation: digital back-end components; allow real time automated analysis of incoming data

• 3rd generation: complete digital transformation; video converted in digital domain at the camera and transmitted via a computer network; cameras can also compress video to save bandwidth

Evolution

• 4th generation: intelligent cameras; perform low-level image processing operations on the captured frames onboard to improve video compression and intelligent host efficiency – however most of the processing is done at a central unit

• But “smart cameras” – directly perform highly sophisticated video analysis– video sensing– video processing – communication – designed as reconfigurable and flexible processing nodes with

self-reconfiguration, self-monitoring, and self diagnosis capabilities.

• Shift from a central to a distributed control surveillance system– Increase the surveillance system’s functionality,

availability, and autonomy– Can react autonomously to changes in the system’s

environment – Can detected events in the monitored scenes.

• A static surveillance system configuration is no longer feasible!

• Sensing unit– Monochrome CMOS image sensor– delivers images with VGA resolution at up to 30 fps– transfers images via a first-in, first-out (FIFO) memory to the PU

• Processing unit (PU)– Up to 10 Texas Instruments TMS320C64x DSPs

• can deliver an aggregate performance of up to 80 GIPS while keeping the power consumption low

– PCI bus couples the DSPs and connects them to the network processor• Communication unit

– network processor: Intel XScale IXP425– establishes the connection between the processing and communication units– controls internal and external communication– currently supports two interfaces for IP-based external communication:

– Wired Ethernet – wireless Global System for Mobile Communications/general packet radio service (GSM/GPRS)

• Commercial off the shelf hardware components to test and evaluate the video surveillance system

• 1 cam consists of:–network processor–several DSPs–a CMOS image sensor

• Multicamera object-tracking application• Multicamera system instantiates only a single tracker (agent) task• The agent follows the tracked object migrating to the SmartCam that should

next observe the object• Tracking agent based on a Kanade-Lucas-Tomasi feature tracker• Main advantage is its short initialization time

– Applicable for multicamera object tracking by mobile agents– Tracking agents control the handover process, using predefined migration regions – When the tracked object enters a migration region, the tracker initiates handover

to the next SmartCam– Each migration region assigned to one or more possible next SmartCams– Motion vectors help distinguish among several SmartCams assigned to the same

migration migration region– Motion vectors check whether the object moves in the correct direction– A master-slave approach for the tracked object handover

• Tracking agent’s migration between SmartCams takes up to 1 second• Task-allocation system’s setup time—approximately 190 milliseconds

Conclusions• Keys to successful deployment of smart

cameras are:–the integration of sensing, computing, and

communication in a small, power-aware embedded device

–the availability of high-level image/video processing algorithms

• System usage: –traffic surveillance–detection of stationary vehicles –detection of wrong-way drivers–computation of average speed,lane occupancy