LOW -COST UAV FOR THE ENVIRONMENTAL EMERGENCY MANAGEMEN T. PHOTOGRAMMETRIC PROCEDURES FOR RAPID MAPPING ACTIVI TIES

D. Marenchino a

a Politecnico di Torino, DITAG, Torino, Italy – davide.marenchino@polito.it

KEY WORDS: UAV, Photogrammetric processing, SIFT, DSM extraction, Solid True Orthophoto ABSTRACT: The main research topic of this PhD thesis is the analysis and the development of photogrammetric procedures for the processing of digital images acquired by an Unmanned Aerial Vehicle (UAV). The mini-UAV “Pelican” is a low-cost aerial platform capable of autonomous flight and equipped with a photogrammetric payload for rapid mapping purposes. It has been developed by ITHACA (Information Technology for Humanitarian Assistance, Cooperation and Action) - a non-profit association with the main goal to carry on operational and research activities in the field of geomatics for analysis, evaluation and mitigation of natural and manmade hazards. The UAV “Pelican” is capable to perform a complete automatic image acquisition, according to a predefined flight plan specifications. This work was aimed at developing an automatic approach for the digital image processing. The photogrammetric processing led to the automatic extraction of Digital Surface Models (DSMs) and the generation of Solid True Orthophotos (STOP). Robust and reliable feature extraction and matching techniques were developed and implemented in Matlab code. In particular SIFT operator, was employed for the automatic extraction of tie points. Moreover, a multi-image matching method (Multi-Image Geometrical Constraint Cross-Correlation, MIGC3) and a robust filter (Self-tuning Standard deviation Median Filter, S2MF) were developed for the automatic DSM extraction. The algorithms were validated and tested on aerial images and on images acquired by the UAV “Pelican” over an archaeological area. The experimental results confirm the high performance of these techniques and the suitability of the photogrammetric process for rapid map production, according to the requirements of the ITHACA project.

1. INTRODUCTION

The technological progress in the electronical and aerospace engineering fields allowed the development of low-cost UAVs characterized by a small size (mini-UAVs), that can carry onboard imaging or non-imaging sensors. These advantageous features led the civil community to have an increasing interest in mini-UAVs. The photogrammetric community took part to the research issues concerning the use of UAVs for map production. The research group of Geomatics of the Politecnico di Torino, in cooperation with ITHACA (Information Technology for Humanitarian Assistence, Cooperation and Action) and the Department of Aerospace Engineering (DIASP) of the Politecnico di Torino, developed a mini-UAV devoted to emergency management in case of environmental disasters. ITHACA association aims at conducting an intense operational and research activity in the field of geomatics for the analysis, evaluation and mitigation of damages caused by natural or anthropic catastrophes. Since 2006, ITHACA has been involved in many initiatives, in cooperation with WFP (World Food Programme), the largest United Nations operational agency. Currently, the association produces basic and thematic maps for the monitoring and the assessment of areas subject to catastrophic events, mainly exploiting remote sensing techniques. Nevertheless, the satellite images are usually unable to quickly acquire data relative to a catastrophic event, therefore in situ missions have to be foreseen. To this purpose an UAV, called “Pelican”, has been developed. “Pelican” is a low-cost mini-UAV equipped with photogrammetric sensors and capable of autonomous navigation (GPS/IMU) and automatic digital image acquisition (characterized by a suitable geometric and radiometric quality). The aim of “Pelican” is to perform photogrammetric surveys in remote and disaster-affected areas, where it is not possible to carry out traditional photogrammetric flights, allowing to update existing maps.

In this paper, a methodology for the photogrammetric processing of images acquired by the UAV “Pelican” is proposed. The main goal is to develop an automatic approach, which involves the stereoscopic acquisition of digital images and the development of algorithms for the digital image processing devoted to the production of Digital Surface Models (DSM) and orthophotos. First, a procedure for the analysis of the image acquisition phase was developed. The validation of the stereoscopic coverage and the detection of shooting problems were carried out, according to the flight plan specifications. Then, an in-depth research about the techniques developed in photogrammetry and Computer Vision (CV) communities was performed, in order to define the state of the art in the automation of the digital image process. Hence, algorithms related to digital image pre-processing, automatic feature extraction and matching, automatic DSM generation and orthophoto production were implemented and tested by the author. The final product of the photogrammetric process is the Solid True Orthophoto (STOP). A STOP supplies not only the geometric and radiometric information of a true-orthophoto, but also the altimetric data provided by the DSM. Therefore, it can be suitable for the rapid assessment of natural and manmade hazards on remote areas, according to the goal of the Early Impact project leaded by ITHACA. The complete photogrammetric procedure was tested on both close-range images and aerial images, the latter acquired by the UAV “Pelican”. At this purpose, many flight tests were performed over countryside areas and archaeological sites.

2. UNMANNED AERIAL VEHICLE: OVERVIEW

2.1 UAV definition and classification

The word “UAV” (Unmanned Aerial Vehicle) is well known in the research community, and in the last years, it has been more

and more used in the ordinary language too. In literature many definitions of UAV can be received. The American Institute of Aeronautics and Astronautics (AIAA), Committee of Standards “Lexicon of UAV/ROA Terminology” (2004), defines an UAV as: “an aircraft which is designed or modified, not to carry a human pilot and is operated through electronic input initiated by the flight controller or by an onboard autonomous flight management control system that does not require flight controller intervention”. Other terms are usually adopted to indicate this category of vehicles, such as: Drone, ROA (Remotely Operated Aircraft), UVS (Unmanned Vehicle System). Currently, UAVs are classified according to different specifications. This is due to the lack of an international certification. It follows that different groups characterised and classified UAVs in different ways. The most effective classification has been drafted by the Unmanned Vehicles Systems International Association (UVS, 2008): UAVs are split in three main categories (Tactical, Strategic, Special Purpose) with respect to their possible usage. Besides, each typology of aerial vehicle is subdivided into subcategories, according to their features and performances; particular reference is made to the vehicle’s range, maximum climb rate, endurance and weight. Table 1 refers to the tactic group, that encompasses the UAV “Pelican”.

Table 1. UAV classification – Tactic Group (UAV association). 2.2 UAV developments and applications

The first UAV flight go back to the middle XIX century, when the Austrian troops attacked Venice with 200 unmanned balloons launched from a ship. Nevertheless, the technological development of UAVs started in the 50’s for military purposes. During the cold war, different countries started projects with the aim of producing vehicles devoted to reconnaissance, surveillance and penetration of hostile territories missions, without the presence of an onboard pilot. At present, the market for UAVs has been rapidly growing both in military and civil applications. The UVS international association (UVS, 2008) has estimated the trend of the development and commercialization of UASs in all over the world. The results show that the number of UASs produced from 2004 to 2008 is doubled. These statistics demonstrate that although UAS represents a relatively small segment of the aerospace market, it constitutes one of the more dynamic areas of the industry. The growing interest in the UAV technology is mainly due to the high versatility of these systems to many kind of missions.

The rapid development of the UASs in the military and civil field is due to: • their flight performance: UASs can operate in a wide range

of operational altitudes (from 100 m to over 30000 m) and have an elevated range of endurance (1-48 hours);

• their adaptability to various typologies of missions; • their inexpensiveness: the possibility of designing aerial

vehicles with variable dimensions, relatively reduced weight, and no onboard personnel allows to carry out flight operations at lower costs compared to the ones required by traditional aircrafts.

However, several pre-requisites must be satisfied to render UAS a viable, cost-effective and regulated alternative to existing resources. Major civil and commercial market barriers include: � the total costs of UAS, in particular the high costs of the

assurances for civil missions; � the lack of sure communication frequencies; � the lack of an airspace regulation that covers all types of

UAS, encompassing the airspace integration and the airworthiness issues;

� the capacity for payload flexibility. Currently, UAV are employed in many civil applications such as land monitoring and remote sensing, agriculture, public security. In the Photogrammetry community, many research teams are developing UAVs. The main goal of these researches is to develop low-cost platforms, equipped with electro optic or infrared sensors, which allow photogrammetric surveys in an automatic way. Furthermore, the challenge of the researches is the development of an automatic photogrammetric processing, for the production of rectified images or orthophoto mosaics.

3. UAV “PELICAN”

In situation of natural emergencies, high-resolution images become a fundamental tool for the assessment of damages impacts on infrastructures and strategic areas. The knowledge of the road network conditions, the evaluation of the damage in infrastructures, which could potentially increase the hazard (dams, dikes, bridges), is not assured by high-resolution satellite images. Moreover, the definition of people and facilities involved in the surrounding areas cannot be quickly estimated by remote sensing techniques. Hence, an alternative image acquisition system must be employed. The adaptability of UAVs for various typologies of missions, in particular for monitoring operations in remote areas has interested the group of Geomatics of DITAG. The researchers have carried out some investigations in the aerospace engineering field (DIASP, Department of Aerospace Engineering) in order to design and develop a low cost mini-UAV. At the moment, after almost two years of researches and tests, three autonomous platforms (wood prototype DC engine, carbon fiber DC engine, carbon fiber ICE engine) are available by ITHACA (Figure 1). The specifications of each platform are summarized in Table 2. Each platform can be easily transportable on normal aircrafts and usable on the field by a couple of operators and it is equipped with different digital sensors which allow to acquire both high resolution frame images and low resolution movie clips. Currently, the digital image acquisition is carried out by the semi-professional RICOCH GR camera, which is characterised by a geometric resolution of 8 Mpixel and a focal length of 5.9 mm. Furthermore, a new pod for a semi-

UAV Categories Acronym Range (km)

Climb rate (m)

Endurance (hours)

Mass (kg)

Tactic Micro µ (Micro) < 10 250 1 <5 Mini Mini < 10 150 to 300 < 2 150

Close Range CR 10 a 30 3000 2 to 4 150 Short Range SR 30 a 70 3000 3 to 6 200

Medium Range MR 70 a 200

5000 6 to 10 1250

Medium Range Endurance

MRE > 500 8000 10 to 18 1250

Low Altitude Deep Penetration

LADP > 250 50 to 9000 0.5 to 1 350

Low Altitude Long Endurance

LALE > 500 3000 >24 < 30

Medium Altitude Long Endurance

MALE > 500 14000 24 to 48 1500

professional digital camera (Canon Eos 5D) is in the prototyping stage.

(a) (b) Figure 1. UAV “Pelican”. (a) Carbon fiber, ICE engine version.

(b) Carbon fiber, DC engine version.

Propulsion DC ICE Wing span (m) 2 2

Wing surface (m2) 2.1 2.1

Length (fuselage, m) 1.75 1.75

Width (fuselage, m) 1.43 1.43

Weight (body, g) 7500/8150 7500/8150

Fuel weight (g) - 1500/1000

Payload capacity (g) 2500/2000 1500/1000

Flight envelope@sea level (m/s)

10-20 10-20

Cruise Altitude (m) 120 120

Cruise speed (m/s) 15 15

Optimum range limits 15 Km 25 Km

Optimum endurance limits (h)

0.5@15m/s 1@15m/s

Table 2. UAV “Pelican” technical features.

The platforms are equipped with a micro-autopilot unit (the MP2128g autopilot of the Micro Pilot company), that enables the automated navigation (except taking-off and landing) and a real-time monitoring of the overlooked areas. The MP2128g is composed by an electronic circuit board and a Ground Control Software (GCS), called HORIZONmp. The navigation system includes a GPS/DGPS unit, three-axis gyroscope and accelerometer (IMU), relative airspeed probe, pressure altitude transducer, AGL ultrasonic altitude sensor, and external servo board. The control software HORIZONmp provides flight path and current sensor values in real-time. The operator can also define a flight plan (up to 1000 waypoints) on a preloaded map and update it during the flight. Besides the system can be connected with the payload cameras, making possible to schedule an automatic shooting time. Therefore, it is possible to carry out automated photogrammetric flights at half of the optimum range limits. The operations of take-off and landing must be accomplished manually due to the insufficient GPS’s in-flight accuracy.

4. PHOTOGRAMMETRIC PROCESSING OF UAV IMAGE SEQUENCES

The photogrammetric processing of UAV images acquired by the UAV “Pelican” was developed according to the workflow in Figure 2. Image sequences acquired with mini-UAVs generally do not respect the stereoscopic normal case acquisition, mainly due to the platform instability. Therefore, the classical techniques used in Photogrammetry for the automatic homologous point extraction, relative orientation and DSM generation usually fail. Hence, an accurate research of the feature extraction and image matching techniques employed in the Photogrammetry and CV communities allowed the analysis of algorithms, which assure invariance to geometric and radiometric transformations.

Figure 2. Photogrammetric process of images acquired by the

UAV.

Scale Invariant Feature Transform (SIFT, Lowe 2004) and Multi-Image Geometrically Constraint Cross Correlation (MIGC3, Zhang 2005) were implemented for the automatic homologous point extraction and the DSM generation. Furthermore, a new procedure for the blunder detection in raw DSMs, called Self-tuning Standard deviation Median Filter (S2MF) was proposed. The algorithms were implemented in Matlab code and they were validated on aerial images acquired by metric digital camera and medium format images acquired by the UAV “Pelican”. Then, accuracy tests were performed in order to validate the performances of the proposed method and to evaluate the automation level of the processes. 4.1 Image acquisition

Digital image acquisition is the first step of the photogrammetric processing. Currently, in the aerial platforms, the digital cameras are electronically connected with PCs and synchronized with the Electronic Control System (ECS). Therefore, a complete automatic image acquisition can be carried out, according to the flight plan indications. The UAV system “Pelican” is not a professional aerial platform, therefore the ECS systems, which are used for the aerial photogrammetric surveys, cannot be installed on the unmanned platform. This is due to the very small dimensions of “Pelican”, to the very limited payload weight and to the limited budget for the system development. Anyway, the MP2128g autopilot and the imaging payload allow to acquire a sequence of stereoscopic digital images in a complete automatic way, according to the flight plan specifications. The flight plan is saved in a file .log and uploaded into the GCS software before the flight. During the flight the GCS sends in real time the flight information to the autopilot, for the automatic navigation and image acquisition. Moreover, the autopilot records the measurements of its sensors, the telemetry of the flights. Finally, after the flight operations, the telemetry and all the information recorded by the autopilot are downloaded into the GCS. Hence, it is possible to carry out an analysis of the automatic acquisition, in order to detect possible problems and to check the stereoscopic coverage of the area. This operation is a fundamental step of the photogrammetric processing, because the low stability of mini-UAV platforms

Image acquisition

Digital image pre-processing

Wallis filter

Tie point extraction

BBA

SIFT/LSM

GPS/IMU pseudo-obs.

GCPs/CPs

DSM extraction

STOP generation

Aerial Triangulation

MIGC3/S2MF

orthoprojection

Automatic operation Commercial software operation Manual operation

does not assure image acquisitions with high geometric accuracy. 4.2 Digital image pre-processing

In the photogrammetric process, the radiometric content of the images has to be improved in order to increase the dynamic range of the images and optimise them for subsequent interest point extraction and image matching. At this purpose, the Wallis filter (Wallis, 1976) was implemented in Matlab. It has been tested on aerial images acquired with the Intergraph Z/I Imaging DMC camera over a mountain area in Calabria (Italy). The main goal of these tests was the definition of an optimized set of the Wallis’ filter parameters for the image enhancement. First of all the algorithm was tested by visual control of the filtered images, in order to check the correct implementation. Afterwards, some tests were carried out by changing the filter parameters (Figure 3).

(a) (b) (c)

Figure 3. Wallis filter’s application on an aerial image. (a) original image. (b-c) filtered images with different setting

parameters. 4.3 Automatic tie point extraction. SIFT – CC, LSM

Feature extraction and image matching procedures are two fundamental steps in the digital photogrammetric workflow. These techniques allow to carry out tie points extraction operations without manual collimations. Most of the commercial photogrammetric software implement interest point extractors (Forstner, Harris operators) and Area Based Matching (ABM) techniques (Cross-Correlation, Least Square Matching) for the automatic collimations of homologous points or automated DSM extraction. These techniques work very well on high-textured images and whenever the acquisition conditions are quite near to the normal one. Nevertheless, they can easily fail when the geometric conditions during the image acquisition do not comply with the classical approach. An example is the UAV application, where image sequences are acquired with narrow or large baseline, illumination changes, camera rotations, 3D viewpoint changes. Therefore, some investigation were carried out on feature extraction and matching operators, called region detectors, which are invariant to geometric and radiometric distortions. Although these techniques were developed for CV applications, the high performances of these operators led to an increasing interest of the photogrammetric community. In particular, the SIFT operator was implemented and validated on terrestrial and aerial images. SIFT is a region detector/descriptor (Lowe, 2004) which extracts image features invariant to image scaling and rotation and partially invariant to change in illumination and camera 3D viewpoint (affine transformation). At each feature is associated a vector of dimension 128, called descriptor (Figure 4).

Figure 4. SIFT descriptor (Lowe, 2004). The correspondence between two candidate points is found by the evaluation of minimum Euclidean distance between the descriptor. The performance of SIFT operator were compared with the algorithms used in photogrammetry (Forstner extractor, Cross-Correlation and LSM). 4.3.1 Validation test: aerial images. The implemented algorithms were validated on a couple of aerial images acquired with the Intergraph Z/I Imaging DMC camera over a mountain area in Lauria (Calabria, Italy). The performance evaluation of Forstner, Cross Correlation, LSM (ForLSM) and SIFT algorithms was carried out through the definition of the total number (Mth) and the rate of correctly identified homologous pairs (Mthok). For this purpose, the robust estimation of the symmetric relative orientation between the two images was carried out, by means of the Least Median Square (LMS) technique (Rousseau, 1987). The congruency of the relative orientation parameters was checked by a statistical comparison with a set of reference parameters. For this purpose, a relative orientation was carried out by manual stereo plotting of 10 high visible points, with a well distributed geometry. Many experimental tests were carried out changing some matching parameters and the rejection threshold L of the LMS estimator. In particular, the Normalized Cross Correlation NCC parameter was changed in ForLSM. Then, the threshold Th_eu, which defines the minimum difference in Euclidean distance between two candidates was varied in SIFT. The experimental results, which are shown in Table 3, confirm the high performances of SIFT.

Table 3. Aerial images. ForLSM, SIFT operators. Rate of homologous points and maximum parallaxes in relation to the

thresholds parameters. The number of corresponding points produced by SIFT and ForLSM (Mthok) is similar for low rejection thresholds, but the difference rises rapidly with the growing of L values. Therefore we can say that SIFT matches more points than ForLSM. The maximum residual parallaxes (pmax) of the relative orientation obtained by SIFT and ForLSM are comparable and in both cases have a sub-pixel value. Although this result is quite divergent from previous works (Remondino, 2006), it underlines the good performance of region detectors also in accuracy.

W

W

W’

ϑ

W’

4.3.2 Validation test: terrestrial images. SIFT and ForLSM techniques were tested on close-range images with high geometrical distortions. The investigated stereoscopic pair was acquired in the Misericordia church in Turin (Italy) with Canon EOS 5D camera (Figure 5). Canon EOS 5D was calibrated with the software Calibra implemented by the research group of the Politecnico di Torino (Marenchino, 2004). The acquisition was performed in good illumination conditions, nevertheless the images have an geometrical distortion due to a viewpoint change of about 25°.

Figure 5. Image correspondences extracted by SIFT on the altar of the Misericordia church.

The evaluation of the matching methods was performed by fixing the NCC and Th_eu thresholds to 0.9 and 2, respectively. The experimental results are shown in Table 4.

Table 4. Misericordia images. ForLSM, SIFT operators. Rate of homologous points and maximum parallaxes in relation to the

thresholds parameters. It can be noticed that SIFT offers an excellent performance in terms of number of pairs extracted (643), a number which is almost five times higher than the number of pairs extracted using ForLSM (144). Furthermore, the rate of correctly matched points (Mthok) is quite stable in SIFT compared to the previous test, while it is rather reduced in ForLSM. The maximum parallaxes maintain sub-pixel values in SIFT for high rejection thresholds, while they are drastically increased in ForLSM. Furthermore, the statistical check of the ForLSM relative orientation has confirmed that the robust estimation is still affected by outliers; in fact the angular parameters do not satisfy the statistical equality with a reference set, for rejection threshold higher than 5.

4.3.3 Validation test: UAV images. A first test of evaluation of SIFT and ForLSM techniques, was carried out on an image pair, acquired with the RICOH camera installed onboard the UAV “Pelican”. The images were taken over a country-area in the surroundings of the flight site in Villareggia (TO, Italy), at a relative altitude of 100 m (GSD=0.04 m). The covered area is made up by fields and meadows. The same procedure used for the tests carried out on close-range images was followed. The results provided by SIFT and ForLSM methods are quite different as to the ones of the previous tests (Table 5). First of all, the number of pairs (Mth) extracted by SIFT (40), is slower than the one detected in Misericordia (1500). This difference is due to the setting of the threshold parameters of the detector.

Table 5. UAV images, Test 1. ForLSM, SIFT operators. Rate of homologous points and maximum parallaxes in relation to the

thresholds parameters. The thresholds recommended by Lowe give good results in terrestrial applications, where images are well-textured; but must be carefully set in aerial cases, such as this one, in which repetitive patterns or lack of texture are usually possible. Therefore, many tests were performed on these images, in order to define and propose more suitable thresholds. In particular, the performance of the detector in relation to the Th_key threshold was investigated (Figure 6).

(a) (b)

(a) (b)

Figure 6. UAV images, Test 1. (a-b) Image correspondences automatically matched with SIFT (256 points, Th_key=0.0005, L=10). (c-d) homologous points extracted with ForLSM (100

points, NCC=0.8, L=10).

Experimental tests led to vary these parameters from the Lowe’s standard value (0.001) to 0.0005, allowing the detection of 478 image pairs, with a sub-pixel accuracy. A second test was performed on a stereoscopic pair acquired over a ploughed field and a meadow. In this case, the high repetitive patterns and a drag effect on one of the two images caused many problems in the image matching for both techniques. For this reason, the ForLSM did not managed to identify homologous points in the ploughed area, due to the bad texture. SIFT, instead, furnished interesting results after a careful setting of the threshold parameters. In the first set of experiments, the Th_key threshold was varied over a wide range, but no interesting results were obtained. Other investigations on the contrast value Dˆx were therefore performed. As already mentioned, this parameter allows points to be removed in poor-contrast regions and it was set at 0.03, according to Lowe. In this application (Table 6), the reduction of Dˆx to 0.01 allowed a high number of points (457) to be extracted and matched with sub-pixel accuracy, in comparison to ForLSM (102).

Table 6. UAV images, Test 2. ForLSM, SIFT operators. Rate of homologous points and maximum parallaxes in relation to the

contrast threshold.

Furthermore, SIFT was able to detect some homologous points in the ploughed field (Figure 7), a situation where the collimation of correspondences is a very difficult task, even in manual mode.

Figure 7. UAV images, Test 2. Homologous points detected by

the SIFT operator on a ploughed area.

The tests have experimentally proved that SIFT provides very robust and accurate results, which can be compared to the ForLSM ones. The number of point extracted and matched is very high (up to 600 on medium-format cameras) and the accuracy is good enough for quickly photogrammetric applications. Moreover, SIFT assures good performances also on image pairs with high geometric and photometric distortions, in which the classical image matching techniques usually fail. In these tests, SIFT give very good results on images with repetitive pattern or lack of texture. On the other hand, SIFT requires the a priori setting of thresholds parameters. This operation is computationally expensive and may condition the results. In this dissertation, some empirically estimated thresholds for aerial applications are proposed, with particular attention to the images acquired by the mini-UAV “Pelican” over non-urban areas. Nevertheless, in order to improve the performances of the method, a self-adaptive approach is recommended.

4.4 Aerial triangulation

The main goal of the UAV “Pelican” project is the development of a platform, which performs automatic flights and image acquisitions. Furthermore, the navigation system could measure the attitude parameters, in order to provide the raw GPS/IMU data for the DG approach. Nevertheless, the raw parameters must be enough accurate, due to the mathematical low stability of the collinearity equations. Therefore, the accuracy analysis of the raw exterior orientation parameters provided by the MP2128g was carried out. Some tests were carried out in DGPS way and in classical way. The experimental results underline the improvement of the accuracy using the DGPS approach (RMSEGPS=2.2 m; RMSEDGPS=0.4 m), nevertheless they are not enough accurate for a Direct Georeferencing approach. In conclusion, the receivers installed on the MP2128g do not supply enough accurate attitude data for the computation of a

BBA with pseudo-observation equations. The bundle block operations in the experimental tests were carried out with the Leica LPS software. Now, an automatic tool for the import into Leica LPS software of the tie points extracted with SIFT, is under development. Furthermore, a Matlab script was implemented in order to produce approximate DSM, with the feature points extracted with SIFT for each image pair and the external orientation parameters assessed with LPS. 4.5 DSM extraction

A reliable and accurate technique for the automatic DSM extraction was developed and implemented in Matlab code, according to the workflow in Figure 8.

Figure 8. Workflow of the automatic DSM generation

technique. The input data are a set of stereoscopic images with the external orientation, and the approximate DSM of the surveyed area. The process is set up by four main steps. After a geometrical image pre-processing (distortion-free image generation) and a feature extraction with Forstner operator, a multi-image matching approach (Multi-Image Geometrically Constraint Cross Correlation, MIGC3) is carried out. This method follows the procedure proposed by Zhang (Zhang, 2005). The original algorithm, developed for pushbroom images, was adapted to frame camera images. Furthermore, the matching strategies have been modified, in order to speed-up the process and to allow quick DSMs production. Finally, a filtering algorithm let to detect and remove 3D points affected by gross errors. The method, called Self-tuning Standard deviation Median Filter, (S2MF) was developed by the author. The three steps are completely automatic. The final DSM can be manually edited by the user. This operation is recommended if the final products are STOPs, due to the intrinsic 3D information of these maps. On the other hand, the manual editing can be skipped, if classic orthophotos have to be produced. 4.5.1 Multi Image Geometrically Constraint Cross-Correlation (MIGC3). The MIGC3 procedure is based on the concept of multi-image matching guided from the object space, therefore any number of images can be matched simultaneously and the geometrical constraints (height, epipolar line) are implicitly integrated. Together with an adaptive determination of the correlation parameter, it has the ability to reduce the problems caused by surface discontinuities, occlusions,

DS

M c

rea

tion

pro

cess

Interior/Exterior Orientation

Stereoscopic Images

Approximate DSM

IMAGE ACQUISITION BBA SIFT/BBA

Geometric image pre-processing

Multi Image Geometrically Constraint Cross-Correlation

Self Tuning Standard Deviation Median Filter

Feature extraction (Forstner)

DSM Editing (optional)

Final DSM

Automatic operation Manual operation

INP

UT

da

ta

repetitive structures (Zhang, 2005). The use of an approximate terrain height constraint has been proposed, which can be obtained without further automatic computations, and that allows to accelerate the DSM generation procedure. We exploited the performances of SIFT operator, which is employed for the automatic tie point extraction. SIFT produces a high number of image correspondences (up to 600 for stereoscopic image pair), which assures a quite good initial approximated DSM. The method works as follows (Figure 9): one of the stereoscopic images is chosen as the reference image, and others serve as the search images. In the reference image, the feature extraction with Forstner operator is carried out. Then, for each interest point, the image matching procedure is performed. An image point is projected in the object space and the approximate DSM allows to define a first 3D position of the point that, according to a height constraint in the object space, should lie between Pmax and Pmin. If these two points are back-projected onto the search images, the bounds of the epipolar segment will be defined.

Figure 9. Image matching with the MIGC3 technique. After the application of the geometric constraints, the research of the homologous points along the epipolar segment is performed using the function called SNCC (Sum of Normalized Cross-Correlation).

( ) ( )∑=

=n

1i

00 Z,pNCCn

1Z,pSNCC (1)

Where NCC is the Normalized Cross-Correlation parameter of each stereo-pair. The auto-adaptive correlation window warping and size, and the dynamic parallax compensation allow the extraction of homologous points on areas with poor texture, repetitive structures or geometric discontinuities (Figure 10).

Search image 06 Reference image 07 Search image 08

NCC (image 07-06) NCC (image 07-08) SNCC (image 06-07-08)

Figure 10. Application of the MIGC3 algorithm.

4.5.2 Self-tuning Standard deviation Median Filter (S2MF). The elimination of the gross errors from the raw DSM supplied by the MIGC3 technique is carried out through a robust filtering algorithm called Self-tuning Standard deviation Median Filter (S2MF). The elimination of blunders is carried out by means of a comparison between the height of each point and a parameter (distance function) which describes the altitudes of a set of neighbour points. If the difference of altitude is below a threshold, the point will belong to the terrain. The planimetric DSM area is split in nb bins and each point is classified in the respective bin, which is denoted by a row and column index (i,j). The bin size is a fundamental parameter, which affects the performances of the filter. On one hand, small bins allow to perform a local analysis of the object, increasing the probability to avoid surface discontinuities. Nevertheless, it is more likely that some bins could have no enough points for a statistically analysis. On the other hand, big bins let to increase the number of points within each one, but the hypothesis of a local planar surface could not be easily respected. Hence, the square bin size ab is defined in relation to the density of the points in the object space. Moreover, for each bin the median, medi,j and the standard deviation, σ i,j of the heights are computed. The median is a robust estimator of the mean value of a random variable; therefore, it is not sensitive to the presence of outliers in the data set. Otherwise, the standard deviation of the data, computed as to the median is sensitive to the measures affected by gross errors. Thus the distance function, which compares the elevations of each point with the altimetric trend of the surrounding bins, is sensitive to the gross errors too. In order to avoid these problems, a self-tuning approach for the computation of the standard deviation of each bin has been developed. Given a bin with s points, the standard deviation σ1 of the whole data set of heights and the median value are computed. Then, the minimum and maximum altitudes are removed and a new value of standard deviation σ2 is estimated. The process is iterated until the difference of the standard deviations between the iterations will be smaller than the threshold ∆st, defined by the user (Figure 11).

stiter1iter ∆σσ ≤−− (2)

The process allows removing the points that are likely affected by gross errors from the computation of the standard deviation with an automated robust approach.

Figure 11. Self-tuning standard deviation computation of a bin. The distance function is defined by means of the bilinear interpolation of the median medi,j and the standard deviation σ i,j of the bins around each point. Therefore, given a point with bin index ( )djj,dii ++ .

j,1i

j,i1j,1i1j,idjj,dii

med)dj1(di

med)di1)(dj1(meddjdimeddj)di1(med

+

+++++

−++−−+⋅⋅+⋅⋅−=

(3)

p

j,1i

j,i1j,1i1j,idjj,dii

)dj1(di

)di1)(dj1(didjdj)di1(

+

+++++

−++−−++−=

σσσσσ

(4)

Therefore the distance function can be expressed as:

djj,diidjj,diikdjj,diidjj,dii nmedQnmed ++++++++ +<<− σσ (5)

where n is a multiplicative factor of the standard deviation, which defines the width of the confidence interval. The distance function is applied for each point of the raw DSM and the ones which do not respect the inequality (4.92) are classified as outliers. The S2MF is a quick and reliable method, which assures good performances for the filter operations on DSMs automatically extracted by photogrammetric techniques. The algorithm does not require a priori knowledge of a set of error-free points; it works very quickly, and it is able to discard gross errors in a complete automatic mode. 4.5.3 Validation test: aerial images. A set of validation tests of the MIGC3 and S2MF algorithms was carried out on three Z/I Imaging DMC images, taken over the countryside of Lauria, in order to exploit a band with triple stereoscopic coverage. The original images (average altitude of 960 m, GSD = 0.10 m) were resampled (equivalent relative altitude: 3400 m; equ. GSD: 0.34 m) at the aim of speed up the automatic process. The images could be considered as a central perspective of the object, therefore the distortion-free resampling was not required. The external orientation parameters were computed with the Leica Photogrammetry Suite (LPS) package. Then, the 700 homologous points extracted with SIFT operator were exploited for the approximate DSM generation. The test underlines the high performances of the MIGC3 technique in many different textured areas. The rate of feature matched is higher than the 90%. Furthermore, the rate of the outliers, which were manually detected, is very low. Moreover, the test confirms that the number of correct matches is higher than the one extracted with a single image pair, especially in bad textured images. Some analyses were carried out in order to assess the performance of the S2MF in relation to the setting of the threshold parameters. Figure 12 shows the influence of the ∆st threshold on the smoothing properties of the filter.

(a) (b) (c)

Figure 12. S2MF. (a) raw DSM. (b-c) DSMs filtered with different ∆st thresholds.

(a) (b)

Figure 13. (a) 3D visualization of the final DSM. (b) Performance analysis of the MIGC3/S2MF techniques.

The setting of the thresholds of S2MF allowed the generation of a final DSM with more than 34000 points and a mean density step of 5 m (Figure 13). The accuracy of the final DSM was estimated according to the Italian technical specifications, edited by the CISIS (Centro Interregionale di Coordinamento e documentazione per le Informazioni Territoriali; Brovelli et al., 2008). This regulation classifies a digital model in level of precision through the assessment of the total linear error LE95Q of homogeneous areas. Therefore, the area was divided in 20 homogeneous zones with two features: open field and wooded area. For each area, a set of CPs was measured on the originally stereo pairs by means of stereoscopic collimations. This operation allowed to obtain altitude values with a theoretical accuracy (LE95CP) about four times higher than the one of the resampled images. The total Linear Error (LE95Q), was computed by means of the relations:

( )n

QQ96.1RSME96.195LE

n

1i

2i,CPi,AM

AM

∑=

−=⋅= (6)

Q

2

CP

2

AMQ T95LE95LE95LE ≤+= (7)

Where LE95AM is the Linear Error of the model and Tq the tolerance. The accuracy check provided astonishing results, which allowed to classify the DSM in Level 4 (Table 7). Consequently, MIGC3 and S2MF algorithms supplies a set of 3D points with accuracy and density suitable for automatic production of DSMs in emergency application.

Table 7. Estimated accuracy of the DSM (CISIS regulation).

4.5.4 Orthoprojection. In the last years, the researches concerning geomatics dealt with the integration between digital images and 3D models automatically extracted with Lidar or photogrammetric techniques. In particular, the research group of Geomatics of the Politecnico di Torino, developed and implemented an innovative product, called Solid True Orthophoto (STOP; Bornaz et al., 2006) that combines the high radiometric resolution of orthophotos with the 3D information of DSMs. This is a cheap and efficient product used to represent the correct shape of any 3D object in photographic form. STOP allows to examine the investigated areas by means of measurements of 3D points, distance, areas and volumes; therefore, it is the ideal cartographic support for emergency operations in damaged areas. The TIN produced with MIGC3 and filtered with S2MF on the area of Lauria, was employed for the orthoprojection. In this case, the TIN is not suitable for the production of a true orthophoto, due to the lack of dense data. Therefore, an ordinary Solid Orthophoto (SOP) was created. The SOP, which has a pixel size of 0.50 m can be represented in the range of scales 1:5000-1:10000. Nevertheless, the production of quick maps allows us to fix 1:5000 as the maximum scale of representation. The metrical congruence of the 1:5000 scale representation was checked by means of an accuracy analysis, proposed by CISIS regulations. The accuracy of the SOP was estimated through the computation of the total Circular Error at 95% of probability:

( ) ( )n

NNEE

7308.1RSME7308.195CE

n

1i

n

1i

2

i,CPi,OP

2

i,CPi,OP

OP

∑ ∑= =

−+−=⋅=

(8)

MIGC3/S2MF - LAURIA TEST

Area (pixels) 3800x2000 Feat. Extracted 43358 Feat. Matched 40164 (92.6 %) No/Multi match 3194 (7.3 %)

Outliers 5400 (13.4 %) Final 3D points 34764 (86.6 %)

Density (m2/point) 25 Plan. Step (m) 5

OPEN FIELD WOOD LE95CP (m) 0.25 0.60 LE95AM (m) 0.45 0.74 LE95Q (m) 0.50 0.95

TQ level 4 (m) 0.60 1.20

N,E

2

CP

2

OPEN T95CE95CE95CE ≤+= (9)

TERRAIN RAISED ENTITIES

CE95CP (m) 0.25 0.60 CE95OP (m) 0.81 1.54 CE95EN (m) 0.84 1.65 TEN 1:5000 2.60 7.80

Table 8. Estimated accuracy of the SOP (CISIS regulation).

The results (Table 8) show that the Solid Orthophoto (Figure 14), produced with the DSM automatically extracted and filtered with MIGC3 and S2MF algorithms can be represented at the scale 1:5000. The CE95 errors are very low and they could be compared with higher scales.

Figure 14. Planimetric and 3D visualization of the SOP.

5. UAV “PELICAN”. PHOTOGRAMMETRIC TESTS

Three photogrammetric tests were performed to evaluate the flight capabilities of the UAV “Pelican” and the geometric and radiometric performances of the sensors for photogrammetric purposes. The first test was carried out in the Villareggia test site (Turin - Italy), while the others tests were carried out on the archaeological area of Augusta Bagiennorum, which is situated a few kilometres away from the actual city of Bene Vagienna (CN, Italy). In the last application, two flights were performed over the Theatre and the Amphitheatre ruins of Augusta Bagiennorum (Bendea et al., 2007). Over the amphitheatre an automatic acquisition of four images with a medium stereoscopic coverage of the 60% and a relative altitude of 100 m (GSD = 0.04 m) was carried out (Figure 15). The aerial triangulation was performed using the Leica Photogrammetric Suite software. The results of the BBA were very satisfying. The RSME of the GCPs were lower than 0.08 m.

Figure 15. Amphitheatre area. Image acquisition. The DSM extraction and the orthoprojection were performed with the algorithms developed by the author. The SIFT algorithm and the robust relative orientation with LMS were used for the approximate DSM generation (about 950 points). The MIGC3 was applied twice, using two images as reference. The raw TINs were filtered with S2MF, which removed about the 15% of the points extracted. In comparison with the

performance on Lauria images, the MIGC3/S2MF algorithms provided, as expected, worse results (Figure 16). The rate of point matched was about the 50%. This was due to the lack of a suitable triple stereoscopic coverage, which is indispensable for a multi-image matching approach. Furthermore, the repetitive structure of the texture in the fields next to the archaeological area reduced the performance of the single-pair matching technique.

(a) (b) Figure 16. Amphitheatre area. (a) 3D visualization of the DSM. (b) Performance analysis of the MIGC3/S2MF technique. Nevertheless, on the whole area a DSM with more than 34000 and 0.6 m of planimetric step was extracted. The orthoprojection was carried out using the raw DSM (no editing), in order to check the performances of a completely automatic orthophoto production. The buildings of the farm were excluded from the orthoprojection area, due to the lack of editing on the data. The accuracy of the final product was checked according to the CISIS regulations. The results underlined the suitability of the orthophoto for 1:500 scale mapping. Over the theatre, only three images were suitable for a photogrammetric processing. They were acquired at a different flight altitudes (from 60 to 70 m) and in non-normal conditions (Figure 17).

Figure 17. Theatre area. Image acquisition. The image processing was the same as the previous one. The aerial triangulation provided good results (RMSE on CPs lower than 0.07 m), furthermore the MIGC3/S2MF algorithms allowed the automatic DSM generation also on stereoscopic images with high geometric distortions (scale, rotation, affine transform).

(a) (b)

Figure 18. Theatre area. (a) 3D visualization of the SOP. (b) Performance analysis of the MIGC3/S2MF technique.

MIGC3-S2MF Amphi 06-07-08

Area (pixels) 3050x2280 Feat. Extracted 43286 Feat. Matched 21479 (49.6 %) No/Multi match 21907 (51.4 %)

Outliers 3226 (15.0 %) Final 3D points 18253 (85.0%)

Density (m2/point) 0.36 Plan. Step (m) 0.6

MIGC3-S2MF Theatre 38-62-39

Area (pixels) 3240x2300 Feat. Extracted 38182 Feat. Matched 19451 (50.9 %) No/Multi match 18731 (49.1 %)

Outliers 520 (2.6 %) Final 3D points 18931 (97.4%)

Density (m2/point) 0.36 Plan. Step (m) 0.6

In fact, almost 40000 points were detected (Figure 18-b). The high rate of correct match in the area with triple stereoscopic coverage confirms the potentiality of the multi-image matching approach. Finally a SOP (Figure 18-a) at 1:500 map scale was produced and checked (CISIS regulations). The photogrammetric tests on the archaeological ruins of Augusta Bagiennorum underlined the potential suitability of the aerial surveys performed with UAV for mapping purposes. A semi-automatic approach allows to produce orthoimages characterised by an information content and a nominal map scale suitable for archaeological analysis.

6. CONCLUSIONS AND FUTURE WORKS

In the presented dissertation, the potentialities in the use of UAVs for photogrammetric purposes were investigated. In particular the performances of the low-cost mini UAV “Pelican” were analysed and tested, in order to assess the suitability of this system for the production of quick maps in areas affected by catastrophic events. Particular attention was turned to the development of a photogrammetric workflow for the automatic image data processing. The goal of the process is the automatic production of the Solid True OrthoPhoto (STOP), a cartographic product, which integrates the radiometric content of digital images with the 3D information supplied by DSMs. STOP is the ideal product for the early impact analysis of areas affected by natural and manmade hazards. To achieve this goal, different algorithms devoted to the image pre-processing, the feature extraction and the DSM production were developed and implemented in Matlab code. The non-conventional image acquisition of mini-UAV platforms led the author to perform a deep analysis of the feature and image matching techniques developed by Photogrammetry and Computer Vision communities. In particular the SIFT algorithm was implemented and tested on different image pairs. The experimental tests and the comparison with algorithms widely used in photogrammetry (Forstner extractor, Cross-correlation, LSM) confirmed the good performance of the operator, which can extract many homologous points also in bad image acquisition condition. Then, the high number of homologous points extracted by SIFT allowed the automatic generation of approximate TINs, which are the fundamental data for the automatic extraction of DSMs. The Multi-Image Geometrical Constraint Cross Correlation (MIGC3) and the Self-tuning Standard-deviation Median Filter (S2MF) techniques were developed and implemented in order to extract and filter DSM in a complete automatic way. The validation tests confirmed the high performance of these techniques, especially if a multi-image stereoscopic coverage is available. Furthermore, the accuracy and density checks of the final DSM and of the STOP (according to the CISIS regulations) showed that a complete automatic approach for the DSM extraction and orthoprojection provides astonishing results, which are suitable for the rapid map production at medium scale. The photogrammetric processing implemented by the author was employed on images acquired by the UAV “Pelican” over an archaeological area. Although the image acquisition has not provided a correct stereoscopic coverage of the areas, the results are encouraging. The DSMs and STOPs have been produced in an automatic way, providing good results also on bad textured areas, such as cultivations or ploughed fields. In conclusion, the photogrammetric workflow presented in this dissertation is suitable for the production of quick maps, according to the requirements of the UAV ITHACA project. Future developments concerning this work are mainly related to the improvement of the automatic data acquisition phase and to

the development of automatic techniques to extract buildings and road networks based on SOP processing.

REFERENCES

Bendea H., Chiabrando F., Giulio Tonolo F., Marenchino D., 2007. Mapping of archaeological areas using a low-cost uav. The Augusta Bagiennorum test site. XXI CIPA International symposium. Athens, Greece, 1-6 October 2007.

Boccardo. P., Dequal S., Giulio Tonolo F., Marenchino D., 2006. ITHACA: un progetto innovativo per la gestione delle emergenze ambientali. Proceeding ASITA 2006 Bolzano, Italy. Bornaz L., Dequal S., Lingua A., 2006. L’ortofoto solida di precisione: un prodotto innovativo per la rappresentazione e la gestione dei dati tridimensionali. Proceeding Conferenza nazionale SIFET, Taranto, Italy. Brovelli M., Cilloccu F., Crespi M., Dequal S., Lingua A., 2008. Ortoimmagini 10K e modelli altimetrici – Specifiche tecniche. CISIS, Centro Interregionale di Coordinamento e documentazione per le informazioni territoriali.

Lowe D.G., 1999. Object recognition from local scale-invariant features. International conference in Computer Vision, Curfu, Greece, pp. 1150-1157. Lowe D.G., 2004. Distinctive image Features from Scale-Invariant Keypoints. International Journal of Computer Vision.

Forstner W., Gulch E., 1987. A fast operator for detection and precise location of distinct points, corners and circular features. Proc. Inter. Conference on fast Processing of photogrammetric data.

Marenchino D., 2004. Calibrazione fotogrammetrica di camere digitali amatoriali. Degree thesis, Politecnico di Torino, Italy. Remondino F., 2006. Detectors and descriptors for photogrammetric applications. Photogrammetric and computer vision ISPRS symposium, Bonn, Germany. Rousseuw P. J., Leroy A. M., 1987. Robust regression and outlier detection., John Wiley & Sons, Inc. Unmanned Aircraft System. The Global Perspective 2008/2009. International Unmnanned Aerial System Community, 2008. Unmanned Systems Roadmap 2007-2032 report. Department of Defence of the United States of America, 2007. Zhang L., 2005. Automatic Digital Surface Model (DSM) generation from Linear Array Images. PhD. dissertation, ETH Zurich, Switzerland. Wallis R., 1976 An approach to the space variant restoration and enhancement of images. In Proceedings of Symosium on Current Mathematical Problems in Image Science. Postgraduate School, Monterey CA, USA.