Localization System Using Video and PIR Information Fusion

Barna Cornel Aurel Vlaicu University of Arad

B-dul Revolutiei, Nr. 77, Arad, Romania barnac@rdslink.ro,

Abstract In the paper is presented a localization system composed of video cameras and infrared detectors. First, a localization is done by using the information obtained from video cameras, using a proper set of attributes. An uncertainty measurement is performed based on the similarity with the nearest prototype and the dissimilarity from the rest of prototypes. The best granularity is also determined. The information delivered by the infrared detectors establish an estimation of the object’s localization and the heating surface. These reinforce the video information, by using an information fusion.

Key words: uncertainty, fusion, localization, video cameras, PIR detectors.

1. INTRODUCTION

The complexity of the captured images, and the numerous parameters that can influence the localization in images determined the appearance of numerous articles, which propose a vast number of methods. In this paper a new view is presented, based on a fusion of video data with information obtained from inexpensive infrared detectors. Using only PIR infrared detectors (which have binary outputs), it will have only one dimension for variation, namely the time line, and the localization will be rough. However, for two or more video cameras, there will be a three dimension variation in space coordinates, one variation in time dimensionality, one in brightness dimension and one in color variation, which leads to a vary large variability of information. Also the complexity of the problem is increase by the lights reflections produced by different sources. All this information is transmitted throw an optic medium to the captor, inducing noises and faulty data. From these facts it can be concluded that:- the information obtained by the video sensors has a large range of uncertainty- the localization efficiency can be improved by using different types of sensors for obtaining data from the same environment, reducing in this way the variability of the searched space. In this article, a localization system that takes into consideration these two observations will be presented. For the first requirement a system formed by two video cameras is

proposed. The two cameras permit a triangulation and the recognition of the features; afterwards, the corresponding granularity pattern and the degree of uncertainty are determined. The conclusion of the first level of surveillance is reinforced by a second detection system formed by a net of infrared detectors, which evaluates the localization and the estimated heating surface. The degree of uncertainty is, hence, reduced this way.

2. VIDEO LOCALIZATION

To determine the degree of confidence for the estimated characteristics is one of the most important issues in feature discovery and labeling. It has often been underlined [1], [5] that crisp interpretation of classification results is causing the loss of an important part of the initial information, and for this reason it is preferable to either use a fuzzy form for out coming data or to establish a continence measure. In these cases, an uncertainty factor can be used. Equivalence classes imply three main proprieties: reflexibility, symmetry and transitivity. In numerous applications, the transitivity doesn’t hold (see Luce paradox), fact determined by the approximation of the observation (two characteristic measurements seam to be equal, but in reality there is little difference between them). For this reason, the use of a more general relationship like similarity, in which the transitive propriety is not required, is more adequate. This is especially true for granular computing and fuzzy logic, where the observations are rough. For this reason, the fuzzy similarity relationship proposed in [2] will be used. This relationship is obtained by overlaying two fuzzy relationships, that of inclusion and that of dominance. The first one implies a fuzzy implication relationship, and establishes the degree of inclusion of a variable x in an interval boarded by a constant value a. The second one implies the fuzzy implications between a boarded constant b and the variable x. Thus, the similarity relationship is:

)()(),,( bxxabaxsim (1)

Thus, the label of the unknown feature is determined by the nearest prototype[7], which is also the most similar to it:

S(x,a,b)=max(sim(x,a,b)) (2)

21978-1-4244-1608-0/07/$25.00 ©2007 IEEE.

The feature is also compared to the next nearest prototype. The dissimilarity is then determined from this reference point. Dissimilarity is considered the complementary measure of similarity, and its relationship is determined by using the negation of the latter one. In the present paper, the classical negation function will be used. The resulting relationship for this measure is:

D(x,a,b)=Ni ,1

min (1-sim )),,( ii bay (3)

From the relationship between these two measurements the measure of uncertainty, expressed as a difference between the similarity with the closest prototype and the dissimilarity from the second closest prototype can be determined:

),,(),,(),,( baxDbaxSbaxU (4)

If the result of this measurement is positive, then small values indicate small uncertainty, because the labeled feature is similar to the designed prototype, and is weakly similar to the next closest one, but if the result is large, it can be concluded that uncertainty is important, because the difference of similarity between the feature and the next nearest prototypes is small, and the possibility of it belonging to each of these two classes is high, resulting that the probability of making a mistake is considerable. If the difference has a negative value, it can be concluded that the feature does not belong to the investigated class, because it is more similar to another prototype, or it can be labeled on the information provided by the existent prototypes.

3. INFRARED LOCALIZATION

Today, in alarm systems, the most frequently used sensors are the PIR type infrared detectors. Each of these detector’s outputs are binary signals provided by a reed relay. Thus, the information obtained from one sensor is quite limited, indicating the presence or absence of an object. Analyzing the infrared detector’s working principle, it can be seen that the output signal is a threshold function on some ambiguous parameters, like distance, temperature gradient, radiation surface and speed of object:

),,,(, SvlTfDR ji (5)where:

jiDR , is an output signal of the detection of the i sensor, corresponding to the j location intrusion,

T is the variation of temperature between the local environment and the intrusion object l is the distance between the detector and the intrusion object v is the speed of the intrusion object and S is the heating surface of the intrusion object.

In real conditions, the elements of the function are only

roughly estimated because of the uncertain conditions and the possible large range of intrusion objects. Unfortunately, the detectors are also provided with a quite summary technical presentation, underlining especially the mounting specifications of the sensor and not mentioning the function or the characteristics [6]. This low granularity of the obtained information concludes to a fuzzy approach of the problem, using also Pawlak’s rough set theory. For completing the proposed task the following steps will be passed: Step1. From the function (5) it can be concluded that for determining the localization, a function of the distance is necessary. To establish this issue, the fuzzy relationship between the distance from the detector and the rest of the parameters will be determined. Hence, the relationship is:

ji DvSTfl )( (6)

where is a t-norm. By observing the reaction of the sensor to different distances, the characteristics of the detector will be determined [3],[4]. It must be observed that the distances are nested in each experiment; small heating surfaces can be detected from short distances and large heating surfaces can be detected from large distances. Step 2.For large surveyed areas, the detectors are usually placed in a grid. Thus, such a distribution of the sensors will be considered, with the same distance between every detector. In this case, the granularity of the observation is a rectangular area centered on each detector, so that the delimitation between two adjacent sensors will be at half distance. This is justified by the fuzzy membership of the best detection discrimination with a triangular form, which has the maximum uncertainty for ½, (having value 1 in the sensor location, and 0 in next sensor location). Step 3. Measuring the temperature of the area and estimating the outside temperature determined the temperature variation. The set of characteristics established in step 1 which corresponds to the estimated fuzzy temperature variation is then selected. The set of sensors, which are activated when an object is moving in an observed area, is also observed. For each granule of the area, the type of object which caused the activation will be determined. For this, the characteristic vector of the sensor and the granule will be overlapped. If there is more than one characteristic overlapping the granule, the lower and the upper approximation on that granule will be determined, according to rough set theory. For active sensors, zero membership for granules out of range will be considered. These types will be determined for all characteristic vectors, which correspond to the fuzzy temperature variation. Step 4.In this step the size of the estimated location is determined by joint operation of all granules with no zero values. Being a lower and an upper approximation of the granules, it will be a lower and an upper approximation of the location hh AA , . This result will be obtained for each

characteristic vector. The most likely estimation of the object is in the middle of the determined area.

SOFA 2007 • 2nd IEEE International Workshop on Soft Computing Applications • 21–23 August, 2007 • Gyula, Hungary – Oradea, Romania

22

Step 5.The evolution of the object’s location in time is observed by periodically repeating steps 1-4. Knowing the movement’s space and time, the speed of the object can be estimated.Step 6.From the speed estimation obtained in step 5 the characteristic vector, which corresponds to that speed, will be selected, resulting in the estimated heating surface, the estimated area and localization of the object.

4. INFORMATION FUSION

The last phase implied an information fusion between the results obtained from the two types of captors. Because the information obtain from the video cameras is more significant than the information from the infrared detectors, the latter will only reinforce the conclusion of the former. Thus, one will operate on the distance and surface obtained from both methods in order to increase or decrease the uncertainty obtained from the video cameras. The following relationship will be operated:

),1min( 1UU F (7) and

)||

||||

||1(11

V

IV

v

IV

SSS

lll

UK

U

where:

IV ll , are the distances obtained by video,

respectively by infrared sensors, IV SS , are the estimated surfaces obtained by video, respectively by infrared sensors,

]1,0[, are weigh coefficients, and K is a normalization factor.It must be observed that because the estimation of the surfaces is less accurate than the results obtained for the distances. Choosing for example

21K (8)

it can result into a bigger confidence in the obtained result if , are small, or it can increase the degree of uncertainty if

the errors of distance and surface estimation are significant. In the end, by applying a threshold to the final result FU the estimations can be accepted or not.

5. CONCLUSIONS

Localization techniques are mostly based on information provided by video cameras. In this article, a granularity-based method is presented. The result is obtained by applying three procedures.First a video recognition is used by establishing the best granularity for one of the attributes sets corresponding to the distance range, and then determining an uncertainty degree

for these features. This method is based on two fuzzy relationships: the similarity and dissimilarity of the unknown feature, which is required to be labeled, and the two most nearest prototypes. It had been proved that the method is sound, and leads to good results. The method can be generalized by taking into consideration the k-nearest neighboring prototypes for determining the dissimilarity measure. Further investigation will be done in this direction. In the second stage, a granularity-based localization and heating surface estimation is performed using the infrared detectors. The information provided by this type of sensors has a rough granularity, and the estimation is based on observing a sequence of simultaneously activated outputs. Previously, a fuzzy characteristic determination is necessary to be performed for the detectors. Finally, is performed an information fusion by enforcing the uncertainty degree proved by the video system with the information obtained from the infrared detectors. This operation is based on the presumption that video provided information is more significant than that obtained from the infrared detectors.

REFERENCES

[1] F.Alkoot,J.Kittler: Modified product fusion, Pattern Recognition Letters Vol 23,Nr.8 2002

[2] A.Bargiera, W,Pedrycz, K.Hirota Logic-based granular prototyping 26-th Annual International Computer Software COMPSAC 2002

[3] C.Barna An Alarm System Design with PIR using Fuzzy Sensor Fusion and Genetic Algoritm Program IEEE SOFA 2005 Szeged-Arad pag.195-199

[4] C.Barna A testing method for PIR detections system CIMCA 2005 Inter.Conf. on Computational Intelligence Viena pag.199-204

[5] R.Yager: A General approach to the Fusion of Imprecise Information, Inter’l Journal of Intelligent system Vol.12 1997

[6] ---- Technical reference for PIR Digital Security Controls Ltd

[7] C.Barna, Measuring Feature uncertainty by using similarity 7-th WSEAS Conference on Fuzzy Systems 2006, Dubrovnik, Croatia, pag.39-42, ISBN: 960-8457-46-7

C. Barna • Localization System Using Video and PIR Information Fusion

23