Project #12:  Feature Extraction from Multimedia Databases

SURF-IT

By:Isaac MahgreftehMentor: Prof. Sharad Mehrotra

Introduction

The rapid deployment of cameras and a variety of  other sensors in smart buildings opens the doors to a new age of application programming.

Fraud detection, crime prevention, power utilization, occupancy monitoring, and social network analysis are just a small fraction of such application.

At UCI, as part of "Responsphere", many buildings on campus are fitted with  variety of sensors including video, audio, RFID, people-counters and environmental sensors.  My grand goal: To automatically detect key identities in a  in multimedia database created from the Responsphere video cameras.

How Will This Extracted Information be Used?

• A fundamental challenge is being able to search and visualize the information collected. 

• Once the algorithms are in place we will use them to automatically, annotate, structure and present meaningful information about the data collected.

 • This will be used as part of StreamViewer - A tool for

the visualization and synchronization of multimedia streams.

First Step at Addressing the Problem

In order to process a multimedia database, the first step is to extract meaningful information from it: •  Is there an object in the foreground?

 • Which direction is the object traveling?

 • Attributes specific to that object such as: height, size, color.

    

TOOLKIT

OPENCV: a C library developed by Intel which contains over 500 functions for computer vision. Eclipse: Integrated Design Environment.

Small data set of images of people walking in Bren Hall

A Sample Image

How To Detect If  There Is an Object In The Frame?

For static cameras the process is relatively simple.   1) Feed images from the same camera to the program, so that the computer can learn a background model.

2) Once we have accumulated enough images, we can  subtract the current frame from the background model created.  3) If the current frame contains information that is significantly different from the background - trigger: "motion detected". 

The math. Looking  Deeper Into Image Subtraction.

S(x,y) = sum of the pixel values  in a location N=number of frames collected M(x,y)= S(x,y) /N     Sq(x,y)= (S(x,y))^2  Standard deviation α(x,y)= sqrt((Sq(x,y)/N)-( S(x,y)/N)2)After that the pixel in a certain pixel location in certain frame is regarded as belonging to a moving object if the following condition is met

The math. Looking  Deeper Into Image Subtraction.

Abs(m(x,y)-p(x,y))> C α(x,y)

why should C = 3? due to the “three sigmas” rule. That states states that for a normal distribution, almost all values lie within 3 standard deviations of the mean.

The math. Looking  Deeper Into Image Subtraction.

The Future And Beyond

I am really enjoying the Feature Extraction project, and I hope to continue with this project  throughout the upcoming school year.  The features that I am planning to add through the summer, and continue through the year, are the following: Detecting if a person has backpack.

Which way is the person walking.

Color of clothing .

Hair color.

If You Like This then you will Love These

opencv reference manual: http://worldlibrary.net/eBooks/Give-Away/Technical_eBooks/OpenCVReferenceManual.pdf

Learning Opencv Vision With The Opencv Library, which is a free. ebook for ANTPAC users :http://search.safaribooksonline.com/9780596516130

For more infomation on Stream Viewer and Responsphere please vist Ronen Vaisenberg website www.ics.uci.edu/~ronen/

Thank You For Everything!!!

I would like to thank SURF-IT  and everyone for talking time out of their day to learn about feature extraction.  I would also like to thank my mentor Proffsor Sharad Mehrotra. for this   amazing opportunity .    Ronen Vaisenberg, who got me interested  in the project, and the idea of higher education.     Jay Lickfett, helps me with debugging, and anything i can think of relating to computers.      SATViewer team,   I have learned more about computer science in a couple of weeks then I could have imagin