Some Applications of Distributed Architectures in Image-based

Surveillance SystemsGraduate Seminar in CIS 750 Video Processing and Mining Spring 2003

Presented by:

Benjamin Garrett

Agenda

Distributed Multi-Sensor Surveillance: Issues and Recent AdvancesPramod K. Varshney and Ioana L. Coman

– Sensing Devices– Architectural Issues– Information Processing– Case study: concealed weapons detection

Distributed Multi-Sensor Surveillance

Primary aspects to take into consideration:– Sensing device development: some added

processing capabilities needed.– System and network design: global considerations

and architectural issues need attention.– Information processing tasks: the abundance of

data requires elaborate information fusion techniques.

Distributed Multi-Sensor Surveillance

Intelligent Distributed Systems (IDS)– Includes some work on specialized OS

Intelligence Surveillance and Reconnaissance (ISR)– Data processing issues arise due to large quantities

of data Intelligent Real-time Integrated Sensor (IRIS)

systems– Redundant sensors for added reliability

Sensing Devices

Early efforts were dedicated to the development of various types of sensing devices – acoustic/sonar, IR, seismic, magnetic.

One example is the Remote Battlefield Acoustic Sensor System (REMBASS)

REMBASS

Ground-based, all-weather, day-and-night, battlefield surveillance, target development, and early warning system capable of remote operation under field conditions.

Basic purpose of REMBASS is to detect, locate, classify, and report personnel and vehicular (wheeled and tracked) activities in real-time within the area of deployment.

It uses remotely monitored sensors placed along likely enemy avenues of approach.

REMBASS

Sensors respond to seismic-acoustic energy, IR energy, and magnetic field changes to detect enemy activities.

The sensors process the data and provide detection of classification information which is incorporated into digital messages and transmitted through short burst transmission to the system sensor monitor programmer set.

The messages are demodulated, decoded, displayed, and recorded to provide a time-phased record of enemy activity.

REMBASS - Problems

Sensors had limited processing power and over-loaded the central unit with data.

Limited bandwidth and information fusion capabilities at the central unit did not allow optimum utilization of the retrieved data.

MEMS

Micro-Electro-Mechanical Systems (MEMS) – integration of mechanical elements, sensors,

actuators, and electronics on a common silicon substrate through micro-fabrication technology.

MEMS

Promises systems-on-a-chip capabilities.

Microelectronic integrated circuits can be thought of as the "brains" of a system and MEMS augments this decision-making capability with "eyes" and "arms", to allow microsystems to sense and control the environment.

Sensors gather information from the environment through measuring mechanical, thermal, biological, chemical, optical, and magnetic phenomena.

The electronics then process the information derived from the sensors activate mechanical devices.

The Georgia Tech Wearable Motherboard

Promises a multitude of applications in sports medicine, advanced health care, and monitoring of astronauts, law enforcement personnel, and combat soldiers.

Optical fibers can detect bullet holes, and special sensors and interconnects monitor vital signs of the body.

The Georgia Tech Wearable Motherboard

I. Plastic optical fibers woven throughout the fabric of the shirt.

II. Flexible data bus transmitting information from sensors mounted on an inside shirt.

III. Bus also transmits information to the sensors (and hence, the wearer) from external sources.

IV. The optical fiber can be used to pinpoint the location of a bullet penetration in combat causality care.

Distributed Multi-Sensor System Architecture

Operational Independence Managerial Independence Evolutionary Independence Emergent Behavior Geographic Distribution

Distributed Multi-Sensor System Architecture

Sensor-level intelligent subsystems – one or a few devices configured for fast reaction time.

Regional or local subsystems – where data fusion takes place.

Central Intelligence Units – usually few if not only one. Makes complex decisions and can override decisions of lower level units.

Distributed Multi-Sensor System Architectural Issues - IDS

Encompasses a wide range of activities. Intelligent Interactive Distributed Systems

group - Vrije Universiteit (VU) in Amsterdam – Agent Operating System: a platform for managing

mobile processes.

Information Processing

Refers to effective means for coordinating the data coming from multiple sensors.

Data/image/information fusion is a vast research field with many open projects in progress.

Video Surveillance and Monitoring Team at CMU

VSAM at Carnegie Mellon

VSAM at Carnegie Mellon

Data fusion: Every observed object is positioned in a 3D geodetic coordinate system using geolocation.

Sensor Tracking: Sensors considered as precious resource to be allocated according to user-specified tasks.

Scene Visualization: Employs a GUI giving a synthetic view of the environment.

VSAM at Carnegie Mellon

Case study: concealed weapons

Uses two different types of sensors: MMW and IR wave sensors.

Infrared waves give better resolution. Millimeter waves penetrate better.

IR waves and Millimeter waves

Case study: concealed weapons

Image Registration – The process of finding the corresponding points from two or more images.

IR image is superimposed over the MMW to evaluate the accuracy of registration task.

Distributed Surveillance Systems – Concluding remarks

High amounts of funding being invested in distributed multi-sensor surveillance systems.

Many of the issues presented are open research problems, some of which are still in their initial stages of development.

Encompasses a wide variety of disciplines and fields.

Sources Consulted

[1] Bult K. et. al. “Low Power Systems for Wireless Microsensors”, Proc. of the 1996 Intl. Symposium on Low Power Electronics and Design, Monterey, CA, Aug. 1996, pp. 17-22

[2] Lin T.-H., Sanchez H., Kaiser W. J. and Marcy H. O. “Wireless Integrated Network Sensors (WINS) for Tactical Information Systems”, Proc. of the 1998 Government Microcircuit Applications Conference.

[3] Sungmee Park, Kenneth Mackenzie, Sundaresan Jayaraman. “The wearable motherboard: a framework for personalized mobile information processing (PMIP). 170-174 Electronic Edition (DOI: 10.1145/513918.513961)

[4] Babak Firoozbakhsh, Nikil Jayant, Sungmee Park, and Sundaresan Jayaraman. “Wireless Communication of Vital Signs Using the Georgia Tech Wearable Motherboard”, IEEE Intl. Conference on Multimedia & Expo. 2000, New York, NY, Electronic Proceedings.

Sources Consulted cont.

[5] Y. Wang and B. Lohmann. Multisensor image fusion: concept, method and applications. Technical report, University of Bremen, 2000.

[6] H. Qi, X. Wang, S. S. Iyengar, and K. Chakrabarty, “Multisensor data fusion in distributed sensor networks using mobile agents”, Proc. Intl. Conf. Information Fusion, pp. 11-16, August 2001.

[7] R. Collins, A. Lipton, H. Fujiyoshi, and T. Kanade. “Algorithms for cooperative multisensor surveillance. Proceedings of the IEEE, Vol. 89, No. 10, October, 2001, pp. 1456 – 1477.

[8] The Intelligent Interactive Distributed Systems group web site: http://www.iids.org/.

[9] The Remote Battlefield Acoustic Sensor System web site: http://www.fas.org/man/dod-101/sys/land/rembass.htm

[10] The Micro-Electro-Mechanical Systems Clearinghouse web site: http://www.memsnet.org/mems/