Proposed Technique for Aircraft Recognition in Intelligent Video Automatic Target Recognition

System (IVATRs)

Abstract—The main objective of this research is to automate WEFT (Wings, Engine, Fuselage, Tail) technique as input to aid VATR (Visual Automatic Target Recognition) system. VATR systems are relatively mobile in nature and easy to use in field. Earlier aircrafts have been observed using traditional binocular techniques, which are still in use along with some early warning system. The prime advantage to use VATR is to make the system more flexible and portable. It also helps to automate traditional technique with latest technology. Many techniques, currently in use, to recognize aircrafts are the same as used in World War I, and II. Although these traditional techniques are good but have limited scope in terms of its applicability and generality. WEFT is well known technique for recognizing the four major parameters of shape to recognize an aircraft. It can be applied using video camcorders to facilitate military surveillance. This paper is discussing some of the important shape feature extraction using WEFT techniques. These extracted features become an input to our Intelligent Video Automatic Target Recognition System (IVATRs) for further processing in feature extraction techniques.

Keywords-component; WEFT, Feature Extraction, Segmentation, Silhoutte, CBIR

I. INTRODUCTION Recognition of modern world aircrafts with the use of

manual binoculars in real time is a difficult and risky task. People use to recognize different objects on the basis of their engine sound and shapes. In World War I, and II many of the visible aid techniques for tracking military planes were used. The sound and shape are two basic parameters to recognize any high speed flying object. With the help of sound we can find the direction. Trained Air Defense personnel can identify an aircraft and its direction on the basis of its sound. It is possible that suspected aircraft cannot directly approach to the targeted area but it will take a long route.

The importance of using manual binocular in field cannot be denied. Tracking the maneuvering of aircraft manual observation is still in use. Multiple intelligent sensors like range finders, Global Positioning Systems (GPS) and Infrared (IR) are in use to eliminate the error due to manual inputs. Video surveillance is growing field which caters the need of security and close monitoring for small ranges, whereas; the

Automatic Target Recognition (ATR) is another technique for surveillance for wide ranges.

Traditional Automatic Target Recognition (ATRs) system refers to the task of selecting, detecting and successfully identifying different potential targets from simple scenes to complex scenes. An ATR system performs as a backbone for electronic warfare. It becomes critical element in early warning (EW), reconnaissance systems and smart weapons. Desert Strom is an example that shows the importance of ATRs in day light missions.

With the growing needs of technology the mobility and portability become essential for robust system. Especially in close range targets, the use of RADAR, LiDAR and other input sensors with static base stations are not possible. Instead of using long range sensors for mobility, the video camcorder is being used for short range observations that improve the traditional binoculars with more powerful and enhanced zooms.

Different ATRs can be classified on the basis of parameters such as input data, sensors and its applications. Input sensors are Video Automatic Target Recognition System (VATR)[1], Synthetic Aperture Radar (SAR)[2], Forward Looking Infrared (FLIR)[3], etc. The applications may be submarine periscope system, naval navigational system [4], tunneling system, air traffic controlling and others.

Concept of gathering input using video camcorder is introduced in 2001 [1]. Depending upon image extraction and retrieval, there are two broader categories of ATRs as follows:

a. Statistical ATRs b. Model-based ATRs Both categories differ mainly in verification phase. The

statistical ATRs use empirical data to perform different types of statistical tests to verify signature of target. Based on verification results, it identifies the object. Whereas, the model-based ATRs require different faces of target and perform one to one mapping. In statistical ATRs huge amount of data related to single targeted object is required. Hence multiple mathematical and statistical methods have been used for identification while small database will be helpful in model-

Syed Faisal Ali Computer & Information Sciences,

Universiti Teknologi Petronas (UTP) Tronoh, Malaysia

Email: sfaisalaliutp@gmail.com,

Aamir Saeed Malik Electrical & Electronics Engineering, Universiti Teknologi Petronas (UTP)

Tronoh, Malaysia Email: aamir_saeed@petronas.com.my

Jafreezal Jaafar Computer & Information Sciences,

Universiti Teknologi Petronas (UTP) Tronoh, Malaysia

Email: jafreez@petronas.com.my

2010 International Conference on Computer Applications and Industrial Electronics (ICCAIE 2010), December 5-7, 2010, Kuala Lumpur, Malaysia

978-1-4244-9055-4/10/$26.00 ©2010 IEEE 174

based ATRs. The basic components for any ATRs are detect, select and identify.

II. FACTORS AFFECTING AIRCRAFT DETECTION AND RECOGNITION

There are many factors due to which an aircraft cannot be clearly observed through manual techniques. These factors are as follows:

A. Complex Backgrounds In manual observation technique it is difficult to identify

the aircraft from a larger distance because of complex background. Complex background means that, too many moving objects are observed in targeted window, so the aircraft with a longer distance cannot be classified. For example, an aircraft with a distance of 10 km seems to be a spot and if there is flock of birds moving also then it is difficult to extract the exact size of birds and the aircraft. Similarly if the line of sight is not clear and disrupt with mountains, high-rise buildings, trees. In both of the cases it is difficult to identify object till it actually come near to the observation point (OP). If there are too many objects in the observation window it is possible that every time they are moving the computational algorithms will start calculating them which is false alarm.

B. Natural Camouflage In bright day, if the observational window is capturing

every movement (sensitive to motion) then it is hard to concentrate on one target. Moreover, if the segmentation only depends on color then it is possible for a large distance objects to have a color similar to the background that is difficult to observed and recognize till it comes closer to the range. Even the camouflage color of the objects may be difficult to observe, if it resembles with the backgrounds objects. In case of night, direct visibility and recognition of objects is almost impossible by using video camcorder alone due to darkness.

C. Visibility of Objects due to Sun light During sunny day, if the aircraft is approaching towards the

observation area with sun in behind than it is difficult to predict the targeted object. The reason is that the aperture of the video camcorder will take excess amount of sunlight and can disseminate the objects at larger distances.

D. Failure of Early Warning (EW) Equipments Aircrafts flying at high altitude having the stealth characteristics cannot be detected till they physically reach to the targeted grounds. Sometime the weather conditions are also interfering with the early warning signals. In any case, if an aircraft cannot be detect from a safe distance it will count as the failure of EW equipment.

E. Size of Targeted Objects Size of different targeted objects also helps to maintain

their stealth properties. Normally, the fighter plane with the role of intercept are of small size, even the remotely piloted vehicles (RPV’s) - drones that are made for different missions are so small like starting from 3 feet to 9 feet that they cannot be easily observed from larger distances. Similarly their engine

noise is also less as compare to real fighter planes so they cannot be easily detected by means of RADARs. The observation window requires an early warning to pick drones because normally they maintain low level of flights, as they are remotely controlled their normal range is 1 Km ~ 3 Km altitude. The transportation and sometimes utility aircrafts are big in size which may visually observable from larger distance like Hercules C-130 etc.

F. Low Level of Flights During day night missions, the fighter planes either make

their cruise flight too high till they reach to targeted area or too low to avoid interruptions of their presence on RADARs. In case of too high altitude their features cannot be extracted by visual support system. As they are in low level of flight then only with the proper distance they visibility of features may clear to extract.

G. Distance and Aspect Ratio As the distance is closer with the observational window the

features of the targeted object may become clearer to identify and verify. As shown in Fig 1.

Figure 1: Distance and safe distance to observe features of aircraft.

III. AUTOMATING TRADITIONAL (WEFT ) TECHNIQUE For visual observations traditionally a powerful binocular is

used to examine the object and extract the features like shape, color, signs on aircrafts, engine sound, etc.

With the change of computational power and blessings of powerful computer hardware and algorithms it is possible to perform traditional powerful techniques to embed with more powerful computation to minimize the time duration of target detection which is always a positive point to recognize any aircraft.

During our research, we are likely adopting the WEFT technique as input to the Intelligent Video Automatic Target Recognition System (IVATRs).

A. Method of Extracting Features We use digital video camera initially to extract the

observational window. Once the scene is captured series of filters are used to perform different operations of preprocessing such as edge detection, blob finding, active contours, cluster size (max ~ min), image silhouettes, histogram and image segmentation.

175

With the help of edge detection techniques such as Canny we find the edges of objects which are easy to calculate the static and dynamic objects from the scene. The blob filter is use to calculate the number of blobs clusters in the scene once the blob find we can calculate histogram for further processing and this histogram is also used during the image retrieval process; in our case we are using content based image retrieval technique (CBIR) [5] to match and extract the images from previously train and stored database. The active contours are used to find the direction of object which helps us in finding the behavior of object and tracking the path, similarly it will also help us in finding the direction of object as it is approaching or leaving from the targeted area. Silhouette [6] of images will help us in recognizing the shape of the object which will be extracted and segmented using multiple segmentation techniques such as clustering method, histogram-method, region growing method and watershed transformation. We use multiple methods for segmentation as it is the most interesting and important phase for our system. The comparisons of multiple segmentation methods will lead our result with high accuracy and which limits false alarm ratio.

B. Image Matching and Retrieval For the retrieval of images the CBIR technique is used.

Back propagation neural network is used as learning phase of the intelligent system. Classification of aircrafts can be done by using multi class support vector machine. All the segmented images will be tagged and store in CBIR database after learning phase ends. During the execution of system soon after the video frames will be analyzed preprocessing steps completed. The segmented tagged images of the in the form of silhouette will feed to CBIR system as query. The CBIR will perform statistical data to extract the elliptical ratio and send it to query and individual feature extraction. The segmented parts of an aircraft will be machined and complete aircraft will be recognized. The complete procedure of CBIR is shown in Figure 2.

Figure 2: General content based image retrieval (CBIR)

system [4]

IV. WEFT FEATURES As discussed earlier that WEFT is one of the most

successful ways to identify and manually extract the features of aircraft. Some of the features that come under WEFT technique are shown in Fig. 3. There are common features which every aircraft does have. They are common to each but the parametric value of each aircraft is different with another.

Figure 3: WEFT features at a glance.

A. Wings/ Rotars Every aircraft can easily be detected on the basis of shape

even though it is coming from large distance as shown in Fig 4. Normally the wings can be identified by the location where

they are installed with the body of the aircraft and the shape they bear. Wings can be categorized as:

• Slant • Taper • Wingtip

Figure 4: Different aircrafts shape from distance the position of wings;

engine, fuselage and tail are different with each other.

As in Fig 4; the position of the wings are high wing, mid wing and low wing. Similarly in Fig 5, the slant positions of wings are highlighted.

Figure 5: Slant positions of wings [9]

176

The wing tapper position is shown in Fig 6. It is easy to make a silhouette of aircraft and the identification of aircraft will be easy by using support vector machines [7].

Figure 6: Wings tapper types [8][9]

B. Engine Aircraft engines can be identified using the same method

as we have used in recognition of wings. The number of engines installed in the aircraft, their position, dimensions their size, length, the air intake inlets all these parameters and sounds will lead us to the recognition of aircraft type. Normally the sound parameter is difficult to identify, especially if many of objects are coming from the same direction. Even engine can also be divided into jet engines and propeller engines in which the engine is either installed on side or at the tail; in case of propeller the engine is installed on nose or on the wings. Different engine positions mounted on aircraft as shown in Figure 7.

Figure 7: Engine(s) position on aircraft

C. Fuselage Fuselage is the lower area of aircraft which is distributed in

three major areas; the nose, the mid, and the rear. The nose area is attached with the canopy of the aircraft and then mid and rear portion of the aircraft hold cabin if the aircraft is transportation, utility or surveillance aircraft. If the aircraft primary role is interception or bomber then the crew consists of one or two members only and there is no cabin for passengers. Figure 8 show different views of nose, mid and rear with

canopy of intercept aircraft. The number of aircraft tail(s) is also seen from large distance.

Figure 8: Different views of F14 tomcat

The nose is also recognizable if the side view of the targeted aircraft is visible clearly. The shape of nose can be pointed straight, pointed down, curve, or blunt nose. Pointed nose are conical shape nose like F16, Mirage F1 and Concord. Blunt nose aircrafts are Su-7B, and MiG-17. As shown in Figure 9.

Figure 9: Different shapes of aircraft nose

D. Tail Number of tails, position, tail fin shape, and slant or angled

are the features from tail that can be observed from distance and based on these we can identify the aircraft. Figure 10, shows different types of tail positions mounted on fuselage.

Figure 10: Tail Positions and mounted position on fuselage.[9]

Similarly, different types of aircraft base on their role have different types of tail which help in their role, as shown in figure 11.

177

Figure 11: Tail shapes of aircraft

V. EXPERIMENT To prove this theory we implement a CBIR database. Our

database consists of fighter and commercial planes. We have implemented morphological watershed [10] image segmentation techniques to extract the wing shape from the fighter plane.

a) Original Image b) Texture Gradient c) Modulated Intensity

e) Total Gradient f) Segmented Wing Image

Figure 12: Segmentation of wing

Similarly, for testing the CBIR system the test image dataset was initially consists of 201 images of aircrafts in which 71 are fighter plane and 131 are commercial plane.

Figure 13: Query Image for CBIR

Figure 14: Retrieved Images from CBIR

Figure 15: Precision Recall Curve for the CBIR Results

VI. CONSTRAINTS Every system has some constraints based on their design

structures and limitations to the time, resources and research methodology. Some of the initial assumptions which is required to highlight before making the experiment successful are as follows:

• Statistical ATRs works on large number of empirical data, therefore we use some model toys to perform initial research work. The results may differ and have higher error due to less number of datasets.

• Still many universities are using toy models to emulate the results.

• Designing of large number of different objects with their images are not possible (3D objects modelings and their mesh structures).

• For testing purpose we will use some toy models and perform test on them.

178

• Only vehicles and flying objects will be the part of learning, validation and verification.

• Multiple objects will be entertained but the error impact on result for support vector machine is initially not known.

• Weather conditions will affect the objects visibility for large distances initially this cannot be entertained without the help of other sensors (RADAR, FLIR, and SAR).

• Small toy models with good light and visibility may initially be use to train neural networks.

VII. CONCLUSION We have presented an old technique which is manual and

still useful for visual aircraft recognition. In this research work we consider high speed fighter planes as True Positive, and all the other commercials, cargo and other planes as True Negative. We have highlighted the technique by which we can make it intelligent and automate WEFT technique after getting input using video camcorder to our system. We used washer method to extract wing, initially, because wings and tails can be seen and recording with a large distance. Currently, we are in the process of capturing the data from video camcorder and process it in our CBIR system.

Our next step is to design a SVM for the classification of aircrafts and other vehicles. Further we analyze both manual and video camcorder techniques by using active contours and then process it with segmentation algorithms. This process will give us both statistical and analytical results which will be litmus test for us to use the hybrid technique of WEFT with IVATRs.

ACKNOWLEDGMENT We are thankful for the Universiti Teknologi PETRONAS (UTP) for their support and help and for their research funds for this project.

REFERENCES

[1] W.H. Licata, S. Principal, and S. Engineer, Automatic Target Recognition (ATR) Beyond the Year 2000, 2001.

[2] B. Bhanu and Y. Lin, "Genetic algorithm based feature selection for target detection in SAR images," Image and Vision Computing, vol. 21, 2003, pp. 591-608.

[3] B. Li, R. Chellappa, and Q. Zheng, "Experimental Evaluation of FLIR ATR Approaches—A Comparative Study," Computer Vision and Image Understanding, vol. 24, 2001, pp. 5-24.

[4] G. Pasquariello, G. Satalino, and F. Spilotros, "Automatic target recognition for naval traffic control using NN," Image (Rochester, N.Y.), vol. 16, 1998, pp. 67-73.

[5] R.S. Choras, "Image Feature Extraction Techniques and Their Applications for CBIR and Biometrics Systems," International Journal of Biology and Biomedical Engineering, vol. 1, 2007.

[6] N.R. Howe and C. Science, "Silhouette Lookup for Automatic Pose Tracking," Pattern Recognition, 2004, pp. 0-7.

[7] O.L. Mangasarian, "A Feature Selection Newton Method for Support Vector Machine Classification," Sciences-New York, 2004, pp. 185-202.

[8] T. Auxiliarymen and H. March, "Aircraft recognition," 2008, pp. 1-10.

[9] Field Manual 44-80, Department of Army, Washington DC 30th September 1996. [10] A. Toumi, B. Hoeltzener, A. Khenchaf, and B. France, “Using

Watersheds segmentation on ISAR image for automatic target recognition,” Image (Rochester, N.Y.), 2007, pp. 1-6.

179