fHyark

d Pho

ed fo

human interaction and can be available for users within tion of sufficiently large CCD arrays is impossible at themoment, so large-format digital cameras are built eitheras multi-head systems by fusing several smaller CCD

ISPRS Journal of Photogrammetry & Remote1. Introduction

Photogrammetric imaging techniques are changingover from analog to digital. The advantages of digitaltechniques over analog are undeniable: superior radio-metric quality, lower material costs, and shorter produc-tion cycles, to name but a few. By using digital imagingand direct georeferencing (DG), up-to-date georefer-enced imagery can be produced practically without

hours of image acquisition. On the application side,improved image quality makes it possible to automateimage measurement tasks, improves interpretability, andenables interpretation of images using remote sensingmethods. The performance of the sensors must be knownand sensor calibration must be accurately determined inorder to fully meet expectations.

Conventional photogrammetric mapping is based on23 cm23 cm large-format film cameras. The produc-Test field system calibration will be a fundamental part of the future photogrammetric production line. Accurate calibration andperformance evaluations are necessary for fully assessing the stability and accuracy of digital sensing techniques. In this paper, amethod of comprehensive geometric calibration in a test field has been developed and empirically tested using eight image blockscollected with three UltraCamD digital large format photogrammetric cameras. Permanent photogrammetric test fields form the basisof the method. Important components of the method are determination of system parameters, evaluation of systematic errors, andassessment of geometric accuracy. The results showed that UltraCamD images contained systematic deformations that could not bemodeled with single lens additional parameter models. Good point determination accuracy was obtained despite the systematic errors;the typical accuracy was 23 m in image space in the horizontal coordinates and 0.050.09 of the object distance in height. One ofthe cameras had significantly poorer performance. In the worst cases, the horizontal accuracy was 5 m in image space and the heightaccuracy was 0.18 of the object distance. The analog cameras gave better results than the UltraCamD, but the development ofappropriate mathematical models for UltraCamD as well as improvements in digital sensors may change the situation in the near future. 2006 International Society for Photogrammetry and Remote Sensing, Inc. (ISPRS). Published by Elsevier B.V. All rightsreserved.

Keywords: accuracy; aerial; Calibration; camera; distortion; geometric; orientationAbstractAvailable online 21 June 2006Geometric test field calibration o

Eija Honkavaara , Eero Ahokas, JuhaRisto Kuittinen, Lauri M

Finnish Geodetic Institute, Department of Remote Sensing an

Received 31 August 2005; received in revisCorresponding author. Tel.: +358 9 295 55 202; fax: +358 9 295 55211.

E-mail address: Eija.Honkavaara@fgi.fi (E. Honkavaara).

0924-2716/$ - see front matter 2006 International Society for PhotogramAll rights reserved.doi:10.1016/j.isprsjprs.2006.04.003digital photogrammetric sensors

ypp, Juha Jaakkola, Harri Kaartinen,elin, Kimmo Nurminen

togrammetry, Geodeetinrinne 2, FIN-02430 Masala, Finland

rm 13 April 2006; accepted 13 April 2006

Sensing 60 (2006) 387399www.elsevier.com/locate/isprsjprsarrays and cameras or by using linear CCD arrays(Cramer, 2004; Petrie, 2003, 2005). The design principle

metry and Remote Sensing, Inc. (ISPRS). Published by Elsevier B.V.

ogramof high-end systems has been that they should be as goodas conventional analog cameras and in many aspectseven better (Drstel, 2003; Fricker, 2001; Leberl andGruber, 2003).

The geometric accuracy and stability of complicateddigital large-format sensors is an important issue. Eventhough published results indicate the high quality ofdigital cameras (e.g. Alhamlan et al., 2004; Drstel,2003; Kremer and Gruber, 2004), more detailedstudies are needed to draw further conclusions andmake recommendations about the use of new sensorsin both conventional and new applications. Theoptimal use of sensors requires exact knowledge oftheir performance.

The Finnish Geodetic Institute (FGI) has maintainedpermanent test fields for photogrammetric systemcalibration and testing since 1994 (Ahokas et al.,2000; Kuittinen et al., 1994). Recently, the test fieldshave been used to calibrate for DG (Honkavaara, 2004;Honkavaara et al., 2003) and to calibrate and test digitalphotogrammetric sensors (Honkavaara et al., 2005). Theuse of these test fields for geometric calibration isdescribed in this article.

The calibration issue will be investigated only from atechnical point of view; formal aspects such ascalibration standards are not considered. Exact mathe-matical formulations and tolerances go beyond thescope of this article. The principles of cameracalibration can be found in Eisenhart (1963) andLivingston et al. (1980). Studies on test field calibrationfor DG and the calibration of digital cameras haverecently been reported by Cramer (2004, 2005), Cramerand Stallman (2002), Heipke et al. (2002), Honkavaara(2004), Honkavaara et al. (2003, 2005), Jacobsen(2004) and Merchant et al. (2004). Several internationalgroups are working on the area of field calibration andtesting of the geometry and radiometry of airborne andspaceborne imaging systems. Important working groupsinclude the CEOS/ISPRS calibration and validation taskforce, the USGS Digital Camera Characterizationinitiative, and the EuroSDR network on Digital CameraCalibration (see Cramer, 2005 for further details ofthese groups).

The objectives of this study were to develop amethod for geometric test field calibration of digitalphotogrammetric sensors and by using the method toempirically evaluate the performance of the VexcelUltraCamD digital photogrammetric large-formatcamera. The calibration method is outlined in Section2. Test materials and methods are described inSection 3 and the results are presented and discussed

388 E. Honkavaara et al. / ISPRS Journal of Photin Section 4.2. Calibration method

2.1. Outline of the method

The suggested method includes the following steps:(1) image collection of a calibration block in an accuratetest field, (2) automatic tie point measurement andinteractive control point measurement, (3) self-calibrat-ing and non-self-calibrating bundle block adjustment,(4) analysis of the calibrated parameters, (5) analysis ofresiduals from various observations (image, GPS, GPS/IMU), (6) geometric accuracy assessment and (7) otherevaluations (e.g. stability, previous calibrations etc.). Inthis study, the calibration quantities, calibration blocks,and test fields are briefly described and the results ofempirical calibration blocks from three UltraCamDs arepresented.

2.2. The quantities to be determined

In the proposed geometric calibration procedure, thesystem parameters are determined and the geometricaccuracy is evaluated. These tasks are performed bymeans of an appropriate calibration block (Section 2.3),a geometric test field (Section 2.4), and a self-calibratingbundle block adjustment (Frstner et al., 2004).

The following quantities characterize the geometricperformance of the system:

The fundamental lens parameters, i.e. principal point(x0, y0) and principal distance (c),

The geometric distortion characteristics of thecamera system; distortions can be modeled usingphysical, empirical or mixed additional parameters(Frstner et al., 2004),

The systematics of the image observation residualsbefore and after self-calibration,

Misalignments of the sub-systems (camera, GPS,IMU): lever arms and boresight misalignments,

The errors in GPS and IMU observations, Point determination and back projection accuracybefore and after self-calibration.

The geometric distortion characteristic of digitallarge format systems is an important issue. Typically, thecamera manufacturers claim that images are distortion-free (e.g. Krpfl et al., 2004), but this should be verifiedby test flights. Well-established physical deformationmodels are the most attractive option for test fieldcalibration, but in the end the selection of theappropriate model should be based on a detailed

metry & Remote Sensing 60 (2006) 387399analysis of the entire system. Important considerations

ograminclude the correlation of the parameters and the manyphysical factors resulting in additional distortions(Frstner et al., 2004; Section 3.3).

2.3. Calibration block

The fundamental parameters of the calibration blockare its structure and flying height. The block structureaffects the accuracy and determinability of variousparameters. Three appropriate block structures weresuggested by Honkavaara (2004) for integrated GPS/IMU/camera-system calibration: a comprehensive block(e.g. 2 to 4 parallel flight lines and 2 to 4 crossing flightlines), a cross block (2 perpendicular bidirectional flightlines), and a line block (single bidirectional flight line).The blocks should have large side and forward overlaps(6080%). A comprehensive block structure gives thebest accuracy and should be used for the mostdemanding tasks, but since the number of flight linesis also a cost factor, the two other block structures wereconsidered feasible in several applications. To be able todetermine the principal distance accurately, either theblock must contain extensive height differences, or theimage centres must have been accurately determined byGPS and two image blocks with different scales must beadjusted simultaneously (Jacobsen, 2004).

Another important block parameter is the flyingheight. In the case of analog cameras, the calibration isnormally determined for each mapping scale, becausedifferent environmental conditions (pressure, tempera-ture, etc.) at different flying heights affect the systemcalibration (Heipke et al., 2002; Honkavaara et al., 2003,Jacobsen, 2004; Merchant et al., 2004). A recommendedapproach is to simultaneously process two flyingheights, which makes the calibration valid at these twoscales and at all the scales between them (assuming thata linear interpolation scheme is appropriate to model theeffect of flying height on the parameters). In the case ofdigital cameras, the stability with respect to flying heightis an important research topic that should be studiedwith various types of survey aircraft.

2.4. Geometric test field

The test field for the geometric calibration consists ofa network of targeted benchmarks. The FGI hasestablished the following permanent test fields to enablecalibration at various scales (Ahokas et al., 2000;Kuittinen et al., 1994):

The large-scale test field (ground sample distance

E. Honkavaara et al. / ISPRS Journal of Phot(GSD)

calibration target with 240 control points is used. A setof 84 calibration images is taken of the calibration targetfrom 3 fixed camera stations using various orientations.The control points are measured automatically andspecific UltraCamD geometric parameters are deter-mined with a self-calibrating bundle block adjustment.The UltraCamD geometric parameters include cameraparameters for each cone (principal distance, principalpoint and lens distortions), parameters of CCD posi-tions, look-up tables of the remaining displacements,and the approximate transformation parameters betweenthe cones. The calibration process is described in detailby Krpfl et al. (2004) and an example of the

3.2. Flight missions

The parameters of the missions were the following:

Several flying heights to evaluate the performance ofthe sensor with respect to the flying height,

Several flight dates to evaluate the stability of thesystem,

Several cameras to evaluate the performance of thecamera type,

Comparison with conventional analog cameras torelate the new results to conventional methods.

ying h

50500010300000501010

390 E. Honkavaara et al. / ISPRS Journal of Photogrammetry & Remote Sensing 60 (2006) 387399comprehensive calibration protocol is provided byCramer (2004).

In the image post-processing phase the correctionsdetermined in the laboratory are applied to the sub-images and the images are stitched together automat-ically using software provided by the manufacturer. Inthe stitching process, the slave images are fitted to themaster image using projective transformation; theparameters are based on laboratory calibration and areimproved by automatically measuring tie points be-tween overlapping image areas. The results of thestitching process are distortion-free, large-format, high-resolution, panchromatic, color, and color-infraredimages. Low-resolution color and color-infrared imagescan also be calculated. For the data user, the necessarycamera parameters are the principal distance, principalpoint coordinates, and pixel size (Krpfl et al., 2004;Leberl and Gruber, 2003).

The correctness of this image formation process hasto be tested in practice. A key issue is the degree towhich laboratory-calibrated values are valid in airborneconditions and the level of stability of these parameters.If the calibration is not valid, systematic errors canappear in various sub-images.

Table 1Blocks used in geometric quality evaluation

Block Date d.m.y Test field GSD [cm] Optic [mm] Fl

u1_g4 11.10.04 Sjkulla 4 101.4 4u2_g4 14.10.04 Sjkulla 4 101.4 4u2_g8a 14.10.04 Sjkulla 8 101.4 9u2_g25 14.10.04 Jms 25 101.4 28u2_g50 14.10.04 Jms 50 101.4 56u2_g8b 15.10.04 Sjkulla 8 101.4 9u2_g8c 15.10.04 Sjkulla 8 101.4 9u3_g4 14.05.05 Sjkulla 4 101.4 4r3300 24.04.02 Sjkulla 6.6 214.108 7r4000 25.04.02 Sjkulla 8 153.030 6

r31000 09.09.04 Jms 62 153.030 4743Three technically similar UltraCamD cameras (UC1,UC2 and UC3) were calibrated in test fields. The imageblocks were collected at Sjkulla at large scales (GSD4 cm and 8 cm) and at Jms at small scales (GSD 25 cmand 50 cm). Details of the test fields were provided inSection 2.4. Comprehensive blocks with four parallelflight lines and two cross-strips, and cross blocks withtwo perpendicular bi-directional flight lines werecollected (Table 1 and Fig. 1).

The UC1 mission took place at Sjkulla on October11, 2004. The aperture was 5.6 and exposure times were1/1251/175 s. The flying speed was approximately80 m/s. GPS data were acquired during the mission, butunfortunately their quality was not sufficient foraccurate analysis. In this mission, a cross-shapedimage block with a 4 cm GSD was collected (u1_g4).

The UC2 mission was performed with a hiredUltraCamD on October 1415, 2004. The camera wasinstalled in the aircraft OH-ACN (Rockwell TurboCommander 690A with a pressurised cabin and twocamera holes) belonging to the National Land Survey ofFinland (NLS). The aperture was 5.6 and the exposuretimes were 1/901/125 s. The flying speed was 80 m/sand more. The gyro-stabilized mount (GSM3000) did

eight [m] Scale Swath [m] Overlaps [%] Num. images

1:4440 460 p: 60 321:4440 460 p, q: 60 321:8880 920 p: 60 321:27780 2875 p: 60 1181:55560 5750 p: 60 641:8880 920 p: 60 271:8880 920 p: 60 261:4440 460 p: 80, q: 60 601:3300 759 p, q: 80 201:4000 920 p, q: 60 24

1:31000 7130 p: 60 34

not function correctly, which resulted in fairly largeimage tilts (typically 02, in maximum 5) and crabangles. The Jms blocks were collected late in theevening in inadequate light conditions and the borderregions of the images did not have sufficient light. Thiscaused difficulties with tie point measurements in theborder areas. Accurate GPS/IMU processing also failedon this second mission. Image blocks with a 4 cm, 8 cm,25 cm and 50 cm GSD were collected (u2_g4, u2_g8a,u2_g25, u2_g50, u2_g8b, u2_g8c). Furthermore, twolarge-scale (u2_g4, u2_g8a) and two small-scale(u2_g25, u2_g50) blocks were collected consecutivelyto allow full system calibration, but unfortunately due tothe lack of accurate GPS/IMU data, this part of thesystem calibration could not be calculated. Three similarblocks were collected with an 8 cm GSD to evaluatestability aspects.

The UC3 mission was performed in Sjkulla on 14May, 2005. The aircraft used was a Piper Cheyenne 2with a pressurised cabin. The flying speed wasapproximately 60 m/s and the system did not have a

3.3. Methods

The images were processed with the image producersusing UltraCamD Office Processing Centre software. 8-bit/channel high-resolution RGB-images were used inthe geometric quality analysis.

The image measurements for aerial triangulationwere performed by several companies and with differentsoftware. The FGI measured the UC1 and UC3 blocksand two UC2 blocks (u1_g4, u2_g25, u2_g50, u3_g4)using Intergraph ISAT software. FM-Kartta Ltd mea-sured the remaining four UC2 blocks (u2_g4, u2_g8a,u2_g8b, u2_g8c) using Inpho MATCH-AT-software.The RC20-blocks were measured by the NLS usingSocetSet and Orima. The blocks were analyzed at theFGI using FGIAT and Inpho InBlock block adjustmentsoftware. All the calculations were performed in thelocal tangential coordinate system. A priori standarddeviations for the tie point observations were set at 3 mand for the GCP observations X=Y=1 cm, Z=2 cmat Sjkulla and X=Y=Z=2 cm at Jms.

391E. Honkavaara et al. / ISPRS Journal of Photogrammetry & Remote Sensing 60 (2006) 387399stabilized mount. One block with a 4 cm GSD and acomprehensive structure was collected (u3_g4).

Three older analog image blocks taken by the NLSusing an RC20 were used as reference material (Table 1:r3300, r4000, r31000). The large-scale blocks with acomprehensive structure were collected at Sjkulla. Thesmall-scale block (r31000) with a cross block structurewas collected at Jms.Fig. 1. Test block structures. GCPs are marked wThe calibration processing was performed using theprinciples presented in Section 2.2. GPS/IMU data werenot available, so this part of the calibration could not beperformed. It was assumed that the UltraCamD imageswere stable and distortion free, and hence a single lensphysical distortion model was used (see below).Residuals of image observations were examined forsystematic behavior by calculating average residuals inith triangles and checkpoints with squares.

ograma 1515 grid. All the GCPs and complete blocks wereused for determining the system parameters. Theempirical accuracy of the point determination andback-projection were investigated using some of thetargeted points as GCPs (Jms: 9, Sjkulla: 12) and therest were used as checkpoints (Jms: 10, Sjkulla: 2529). Sub-blocks consisting of four parallel strips wereused for the geometric accuracy assessment in the casesof comprehensive blocks (u2_g4, u3_g4, r3300, r4000).

The well-known eight-parameter physical model wasused to model the image distortions:

Dx Dx0x=cDc k1xr2 k2xr4 k3xr6 p1r2 2x2 2p2xy 1

Dy Dy0y=cDc k1yr2 k2yr4 k3yr6 2p1xy p2r2 2y2 2

The model includes corrections for the principaldistance (c), principal point (x0, y0), radialdistortions (k1, k2, k3), and decentering distortions(p1, p2); r

2 =x2 +y2; x and y are image coordinates withthe principal point as the origin of the coordinatesystem. The principal distance could not be evaluatedwith the test data. In some cases, the model wassupplemented with in-plane corrections (x=b1x+b2y). It is well known that correlations exist betweenmany of these parameters. The most significant onesare the mutual correlation of the radial distortionparameters and the correlation of the principal pointcoordinates with the decentering distortion parameters.Practical experience with analog photogrammetriccameras and digital close range cameras has shownthat despite the correlations, good parameter stabilitycan be obtained if the interior orientation is stable(Frstner et al., 2004).

4. Results and discussion

4.1. Theoretical expectations

Kraus (1993, p. 284) gives the expected values forthe point determination accuracy of targeted points innormal mapping blocks (p=60%, q=20%) with addi-tional parameters. The expected height accuracy is0.03 of the object distance and the expectedhorizontal accuracy is 3 m in image space. Looseraccuracy expectations (5 m in horizontal coordinatesand 0.05 of the object distance in height) are alsooften considered to be reasonable (e.g. Drstel, 2003).

The FOVof the UltraCamD is significantly smaller in

392 E. Honkavaara et al. / ISPRS Journal of Photthe flying direction than that of conventional framecameras. In practice, this is seen as an inferior B/H-ratioand height accuracy. The difference arising from usingthe rectangular image format of the UltraCamD insteadof a square image format was evaluated with idealsimulated data. The same area was covered both bysquare (103.5 mm103.5 mm) and rectangular(103.5 mm67.5 mm) images; the focal length was100 mm. The block had four image strips and the scalewas 1:4000. The length of the strips was fixed by 9square images with a 60% forward overlap. 81 regularlydistributed tie points (99 grid) were generated in eachimage (tie=2 m). Both block types had the same 13GCPs, which were created for the square block in thestrip overlap areas with 4 base distances and at the blockcorners (GCP_X,Y=1 cm, GCP_Z=2 cm). In somecalculations GPS and IMU observations were alsoused (GPS=10 cm, =0.005, =0.008). Threedifferent overlap configurations were used (p=q=60%;p=60%, q=20%; p=80%, q=20%). This setup allowstheoretical evaluation of the deteriorating impact ofrectangular image format in the block geometry, anddesign methods to compensate for this effect.

Standard deviations of point and orientation un-knowns (Fig. 2) were used to evaluate various blockconfigurations. The analysis concentrated on the rectan-gular image format; only a few results for the squareformat are given for reference. The rectangular imageformat yielded poorer accuracy than the square format.This was especially apparent in the orientationunknowns and the height component of the pointunknowns. The difference in accuracy of the twoimage formats was greatest in the case of the blockwith 20% side overlap and no GPS/IMU support (forinstance, the square block had 40% better object heightaccuracy). The use of 80% forward overlap improved theaccuracy of all the unknowns. When GPS/IMU supportwas used, the rectangular and square blocks gave almostthe same orientation accuracy, but the square block had30% better object height accuracy. In the case of therectangular image format, the horizontal accuracy wastypically approx. 2 m in image space and the heightaccuracy was 0.050.1 of the object distance.

4.2. General results

The 0 values were 1.32.6 m (Table 2). Theexpected value is 2 m or less; blocks u1_g4 andu2_g25 exceeded this value. Additional parameters(x0, y0, k1, k2, k3, p1, p2; Section 3.3) had differenteffects on the object and image residuals. The use ofadditional parameters improved the RMSEs of image

metry & Remote Sensing 60 (2006) 387399residuals by only 48%. Instead, the use of additional

the case of UC2, the correction for x0 (flying direction)was approx. 65 m in the 4 cm GSD flight, 4055 m inthe 8 cm GSD flights, and approx. 35 m in the 25 and50 cm flights. Similar values were obtained fromconsecutive calibrations; the flying height appeared tohave only a slight effect on the values.

The effects of radial and decentering parameters areshown in Fig. 3 and the maximum corrections areprovided in Table 2. The symmetric radial and thedecentering distortions were significant for all threecameras. Corrections in two of the consecutively flownblocks (u2_g4 and u2_g8a and u2_g8b and u2_g8c)were similar. In addition, the corrections of u2_g8a andu2_g8c were similar. The maximum values were 4

393ogrammetry & Remote Sensing 60 (2006) 387399E. Honkavaara et al. / ISPRS Journal of Photparameters (especially radial corrections) affected theheight residuals of GCPs in many blocks. The largestheight improvements (>50%) appeared in the UC3block (u3_g4) and UC2 blocks with 4 cm, 25 cm and50 cm GSD (u2_g4, u2_g25 and u2_g50). Improve-ments in horizontal components were smaller, typicallyless than 10%. Free network adjustments wereperformed in order to ensure that the GCPs did notcause the behavior; additional parameters similar tothose in the constrained adjustment were obtained.

4.3. Calibration parameters

Principal point corrections are shown in Table 2.Fairly large values were obtained for UC1 and UC2. In

14 m (Table 2). Corrections were the largest in theblocks u1_g4 and u2_g8b.

Fig. 2. Simulation results on comparison of rectangular and squareimage formats (FOV in the flying direction: square: 55, UltraCamD:37). Accuracy of point determination, perspective centres androtations. (uc = UltraCamD; sq: square; g: GPS; i: IMU; A: p=80%,q=20%; B: p=60%, q=20%; C: p=q=60%).The correlations of the additional parameters werediscussed in Section 3.3. The significance of theparameters and the similarity of the corrections inseveral blocks indicated good determinability; this resultis consistent with the expectations.

4.4. Systematic image errors

Plots of image residuals (average residuals of thecells in the 1515 grid) for the three cameras are shownin Fig. 4. The nine separate CCD arrays can be visuallydetected from the residual plots. It appeared that thesystematics of the residuals of UC1 were larger thanthose of the other two cameras. The use of physicalimage deformation parameters did not significantlydecrease the systematics of the residuals. The reason forthis is that the applied additional parameters did notmodel the existing errors correctly.

Table 2Test field calibration of UltraCamD: 0: standard error of unit weight,PP: principal point corrections, PP stdev: standard deviation of PP,Max Cor: maximum corrections of radial and decentering parameters(Fig. 3), Max Sys: maximum systematic error at vector plots (Figs. 4and 5)

Block 0[m]

PP [m] PP stdev[m]

Max. [m]

dx0 dy0 dx0 dy0 Cor. Sys.

u1_g4 2.6 29.4 47.6 4.1 4.2 14.3 4.5u2_g4 1.9 63.8 10.3 2.3 2.6 7.1 2.2u2_g8a 1.4 42.0 23.5 3.0 3.1 7.7 2.4u2_g25 2.3 34.2 1.2 3.8 4.0 4.8 1.7u2_g50 1.8 34.5 1.4 4.6 5.1 7.3 1.7u2_g8b 2.0 48.7 12.7 4.6 4.9 10.0 4.0u2_g8c 1.5 52.7 29.0 4.0 4.2 4.6 2.6

u3_g4 1.8 4.5 7.2 2.8 3.0 4.2 4.1

394 E. Honkavaara et al. / ISPRS Journal of Photogrammetry & Remote Sensing 60 (2006) 387399Fig. 5 shows the residual plots of 6 blocks collectedwith UC2. The systematics were clearly smaller in theblocks collected on the first day than in those collectedon the second day. The poor illumination conditionsduring the collection of blocks u2_g25 and u2_g50resulted in the absence of tie points in the image borders(Section 3.2). The correlation of the vector plots wasevaluated by calculating correlation coefficients in the xand y directions. On the basis of both the numbers andthe visual analysis, it could be concluded that thesystematics of the consecutively flown image blocks

Fig. 4. Plots of average image observation residuals

Fig. 3. Effects of test field calibrated radial and decentring disto(u2_g4 and u2_g8a, u2_g25 and u2_g50, u2_g8b andu2_g8c) were significantly correlated (correlation coef-ficients >0.5). In addition, the 8 cm blocks collected onOctober 14 and October 15 (u2_g8a and u2_g8c) had asignificant correlation coefficient of 0.60.7.

The evaluation showed that the UltraCamD hadsystematic errors caused by the special compositestructure of the images. The use of the single-lensadditional parameter model is not sufficient for this typeof images. This is an important topic for future researchand development work.

in 1515 grid for three UltraCamD cameras.

rtion parameters in image coordinates on various blocks.

515

395E. Honkavaara et al. / ISPRS Journal of Photogrammetry & Remote Sensing 60 (2006) 3873994.5. Geometric accuracy assessment

Fig. 5. Plots of average image observation residuals in 14.5.1. Accuracy of point determinationThe accuracy of point determination is shown in

Table 3 and Fig. 6. The results from adjustments withoutadditional parameters are shown for all blocks. Theresults of adjustments with additional parameters areshown if their use significantly improved the accuracy.Theoretical values (RMSEs of standard deviations of

Table 3Point determination statistics (: additional parameters were used)

Block Theoretical [cm] Empirical [cm]

RMSE RMSE

X Y Z X Y Z

u1_g4 1.3 1.3 4.6 2.5 2.1 7.9u2_g4 0.9 0.6 2.9 1.1 1.5 6.9u2_g4 0.6 0.5 2.2 0.9 1.3 4.1u3_g4 0.7 0.6 2.2 1.2 1.2 7.2u3_g4 0.7 0.6 2.6 1.0 1.0 2.1u2_g8a 0.9 0.8 3.6 1.0 2.1 5.8u2_g8b 1.1 1.2 4.5 3.1 1.7 5.9u2_g8c 0.8 1.0 4.3 3.1 1.7 6.8u2_g50 9.6 9.0 39.7 8.4 7.4 76.3u2_g50 9.2 8.7 38.7 8.9 7.7 29.0r3300 1.5 1.2 3.7 1.2 1.2 3.1r4000 1.4 1.4 3.0 1.0 1.1 1.9r31000 23.1 22.3 56.3 10.8 6.8 19.3point unknowns) and empirical values (RMSE, standard

grid for one UltraCamD with 6 repetitive calibrations.deviation and average) are given.All three cameraswere testedwith aGSDof 4 cm. The

performance of the cameras was different. The previousanalysis has indicated that the UC1 had geometricproblems. This was also apparent in the point determi-nation accuracy; the empirical horizontal RMSE wasapprox. 2 cm (5 m in image) and the empirical heightRMSE was approx. 8 cm (0.18 of H). Additional

Stdev Average

X Y Z X Y Z

2.5 2.0 7.5 0.3 0.5 2.91.1 1.2 5.4 0.3 0.9 4.40.9 1.1 3.5 0.1 0.8 2.31.2 1.2 6.0 0.3 0.3 4.11.0 1.0 2.2 0.2 0.3 0.01.1 1.4 4.7 0.1 1.6 3.51.8 1.7 6.0 2.5 0.4 0.82.0 1.8 6.8 2.4 0.1 1.48.2 7.5 63.0 3.2 2.2 47.48.3 7.9 27.1 4.1 1.4 13.51.2 1.1 2.7 0.0 0.4 1.51.0 1.1 2.0 0.2 0.3 0.110.5 7.2 20.0 4.3 0.8 3.1

ogram396 E. Honkavaara et al. / ISPRS Journal of Photparameters did not affect the results for UC1, but with theother two cameras the use of radial and decenteringcorrections significantly improved the height accuracy (asimilar performance was also observed in the analysis ofheight residuals of GCPs; Section 4.2). The empiricalhorizontal RMSE of the UC2 and UC3 blocks wasapprox. 11.5 cm (23 m in image space). At its best,the empirical height RMSE of UC2 was approx. 4 cm(0.09 of H) and of UC3 approx. 2 cm (0.05 of H)when additional parameters were applied.

The accuracy of three blocks with an 8 cm GSD wasfairly similar. The empirical horizontal RMSE wasapprox. 13 cm (13 m in image space) and theempirical height RMSE was approx. 67 cm (0.060.08 of H). Additional parameters did not affect theresults significantly.

Fig. 6. Empirical point determination RMSE for X, Y and Z.Horizontal accuracy scaled to image. Height accuracy in of theobject distance (H) (Theor.: theoretical value, no: no additionalparameters, add: additional parameters).The empirical horizontal RMSE of the block with50 cm GSD was less than 10 cm (1.5 m in imagespace). The empirical height RMSE was 75 cm(0.13 of H) if additional parameters were not usedand 30 cm (0.05 of H) if additional parameters wereused.

Theoretical expectations for the point determinationaccuracy of the block types used were 12 m forhorizontal coordinates in the image and 0.040.06 ofthe object distance in height (Fig. 6). In two of thecameras tested, the empirical point determination RMSEwas 13 m in the image in horizontal coordinates and0.050.09 of the object distance in height when themost appropriate geometric model was used. TheRMSEs of the third camera were almost twice aslarge. The empirical RMSEs were worse than thetheoretical RMSEs, but the empirical standard deviations(Table 3) were in many cases closer to the theoreticalvalues; this indicates the presence of systematic blockdeformations.

The studies were performed using GSDs of 4 cm,8 cm and 50 cm. The accuracy of point determination forvarious GSDs was fairly consistent. Consistent resultswere obtained even with a high-speed aircraft flying atlow altitude.

4.5.2. Back projection accuracyBack projection accuracy was evaluated by calculat-

ing the image coordinates of checkpoints and comparingthese values with the measured values. With this methodit is possible to evaluate the potential of UltraCamDimages in orthophoto production, for instance. Blockswith a 4 cm GSD (u1_g4, u2_g4 and u3_g4) wereevaluated.

Additional parameters affected the results. If addi-tional parameters were not used, the 2D back projectionRMSE was approx. 8 m (in image space). Whenadditional parameters were used, the RMSE of UC2 andUC3 was 5 m (in image space); the RMSE of UC1 was7 m (in image space). All cases were appropriate fororthophoto production.

4.6. Stability

The stability of UC2 was evaluated by comparing thecalibration parameters (Section 4.3), the systematicerrors (Section 4.4) and the empirical accuracy (Section4.5). The calibration parameters and systematic imageerrors of consecutively flown blocks that had differentGSDs were correlated. This means that the system canbe insensitive to flying height. Blocks with an 8 cm

metry & Remote Sensing 60 (2006) 387399GSD flown on different days also showed similar

ogrambehavior. However, the principal distance performancecould not be evaluated in this study. It is also possiblethat the use of incorrect mathematical models couldhave affected the results. After developing appropriatemodels, it is necessary to repeat the stability analysis.The most extensive data available to perform thisanalysis are mapping blocks collected by mappingcompanies. If the sensor is not geometrically stable, itsuse in DG is questionable. This topic needs further studyusing GPS/IMU observations.

4.7. Comparison with the analog camera

Point determination accuracy statistics for threeRC20 blocks are given in Table 3. The performance ofthe cameras was stable. The empirical horizontal RMSEwas approx. 24 m in image space and the empiricalheight RMSE was 0.020.04 of the objectdistance. This means a horizontal accuracy of 1 cmand a height accuracy of 23 cm for the large-scaleblocks and a horizontal accuracy of 10 cm and a heightaccuracy of 20 cm for the small-scale block. The use ofadditional parameters did not significantly affect theaccuracy of the RC20 blocks. These results prove thequality of the test fields and the stability of the standardframe camera.

On the basis of the results presented, the importantdesign principle of the UltraCamD can be evaluated,i.e. whether the UltraCamD is geometrically as goodas an analog camera. The FOV of the UltraCamD inthe cross-flight direction corresponds with that of afilm camera with a focal length of 210 mm. In flightdirection the FOV is smaller, which reduces the heightaccuracy (Section 4.1). With regard to technicaltesting alone (point determination accuracy withtargeted points), the analog camera yielded betterresults than the UltraCamD: a similar or a better pointdetermination accuracy was obtained with the analogcameras (Table 3) even though their swaths wereconsiderably wider (Table 1). The radiometric qualityof the UltraCamD, however, is superior to that of theanalog camera, and these radiometric properties willalso improve the geometric results from the Ultra-CamD e.g. in DEM generation, measurement of lowcontrast objects, and stereo mapping. This is animportant issue that must be taken into account inpractical studies. The improvement of self-calibrationmodels can also change the situation (Section 4.8).

The results indicated that the UltraCamD couldreplace analog cameras. However, for the systems andmethods used, it is advisable to check the results

E. Honkavaara et al. / ISPRS Journal of Photcarefully if accuracy requirements are high (e.g. betterthan 5 m in horizontal coordinates (in the image) and0.15 of the object distance).

4.8. Treatment of the multi-head distortions

Based on the results it is clear that multi-headparameters are needed for the highest accuracy. Anexperiment was performed by implementing a multi-head option to the FGIAT software of the FGI; thephysical parameters supplemented with in-plane distor-tions (Section 3.3) were calculated for each of the fourpanchromatic camera heads. The adjustments wereperformed with 4 and 8 cm blocks. The results withthe multi-head parameters were in general slightly betterthan results with the single-head parameters. The mostremarkable improvement was obtained with the prob-lematic camera UC1 (block u1_g4). With this block thepoint determination RMSEs with multi-head parameterswere 1.7 cm in horizontal coordinates (4.0 m in imagespace) and 3.8 cm in height (0.08H); this result isalmost as good as the results of the other two cameras.Analysis of the image residuals showed that thesystematic errors of image residuals had decreased,but not completely disappeared. The results showed thatan improvement could be gained with multi-headparameters. In the tested model there were highcorrelations between many parameters (e.g. correlationsbetween the 4 sets of radial distortion parameters), so thedevelopment of a better model is necessary; this issue,however goes beyond the scope of this article. In orderto develop an optimal model, detailed information onthe camera's construction is necessary; unfortunatelysuch information is typically considered a businesssecret.

The most convenient way to deal with deforma-tions would be to perform the self-calibrating bundleblock adjustment with an appropriate sensor modeland correct the distortions in the UltraCamD post-processing software. With this approach the post-processed images would be distortion free and thecomplicated distortion processing could be eliminatedfrom the subsequent parts of the production line. Asimilar approach is already used with pushbroomsensors (Fricker, 2001). Another approach, i.e. imple-menting and maintaining the multi-head parametersfor each multi-head sensor type and in all blockadjustment software, is difficult and time-consuming.Furthermore, when dealing with the determinedcorrections, e.g. performing another resampling, useof the determined parameters directly or use of look-up tables complicates or even distorts the data

397metry & Remote Sensing 60 (2006) 387399processing.

ogram5. Conclusions

This study developed a method for geometricallycalibrating any type of airborne camera in a test field. Tobe fully operational, exact specifications and tolerancesmust be determined for the various components of themethod. Rigorous empirical investigations with variouscamera types, the development of appropriate sensormodels, support of camera manufacturers and interna-tional co-operation are necessary to complete thesetasks.

Future experiments will show the role of the test fieldcalibration. At the moment, it seems that calibrationprocessing of digital photogrammetric systems willinvolve component calibration in a laboratory, fieldcalibration in a test field and self-calibration of everyimage block. The difference to the current practises isthat the role of the field calibration is going to be moreimportant. Probably the most important function of thetest field calibration will be to test and validate thesystems. The degree to which the parameters determinedin the test field can be used in further processes isdependent on the stability of the systems. This is oneimportant topic for future research. The test fieldcalibration is particularly important in the context ofdirect georeferencing.

Eight image blocks collected with the three Ultra-CamD digital photogrammetric cameras were geomet-rically calibrated and tested with the method. Somesystematic errors appeared and distorted the results;future studies will show whether these problems can becompletely offset by using more appropriate geometricmodels. It appeared that a multi-head additionalparameter model would be necessary for the highestaccuracy; the determination of appropriate models is animportant topic for future research. It was concluded thatthe most convenient approach would be to determineand correct the multi-head distortions in the image post-processing software. Two of the tested cameras showedgood results; one of the cameras appeared to haveserious geometric problems.

To conclude, the results show that the geometricperformance of the first generation UltraCamD cameraswas surprisingly good. Typical empirical point determi-nation accuracy values were 23 m in horizontalcoordinates in the image space and 0.050.09 of theobject distance in height. Since the insufficiency of themathematical modeling particularly affected the heightcomponent, some improvement in the height accuracy ispossible. The empirical results were worse than thetheoretical results, which also indicated the presence of

398 E. Honkavaara et al. / ISPRS Journal of Photuncompensated systematic errors in image coordinates.It must be borne in mind that the test field calibrationshows the performance of the entire image productionline, so some of the errors detected can be assigned tothe other components of the system. This is oneimportant property of field calibration and testing; italso teaches image producers how to use the systemsand points out the weak links in the production line.

Acknowledgements

The authors are grateful to Finnmap Ltd, FM-KarttaLtd, Geotec Vermessungsgesellschaft mbH, MeixnerConsulting Engineers, and the National Land Survey ofFinland for their contribution in acquiring the imagedata, for performing a large proportion of the imagemeasurements, and for giving valuable commentsconcerning the test setup and results. The assistance ofseveral individuals at the FGI is particularly appreciated.

The roles played by of the authors in the study wereas follows: Eija Honkavaara (test design, testing,analysis); Eero Ahokas, Harri Kaartinen, KimmoNurminen (field surveys); Lauri Markelin (field surveys,image processing); Juha Hyypp and Risto Kuittinen(general guidance); Juha Jaakkola (discussions, assis-tance in graphics).

References

Ahokas, E., Kuittinen, R., Jaakkola, J., 2000. A system to control thespatial quality of analogue and digital aerial images. InternationalArchives of Photogrammetry and Remote Sensing 33 (Part B4),4552.

Alhamlan, S., Mills, J.P., Walker, A.S., Saks, T., 2004. The influenceof ground control points in the triangulation of Leica ADS40 Data.International Archives of Photogrammetry Remote Sensing andSpatial Information Sciences 35 (Part B1), 495500.

Cramer, M., 2004. EuroSDR network on digital camera calibration.Report Phase I (Status Oct 26, 2004). http://www.ifp.uni-stuttgart.de/eurosdr/EuroSDR-Phase1-Report.pdf (accessed March 31,2006).

Cramer, M., 2005. Digital airborne cameras status and future.Proceedings of ISPRS Hannover Workshop 2005: High-Resolution Earth Imaging for Geospatial Information. 8 p., onCD-ROM.

Cramer, M., Stallman, D., 2002. System calibration for directgeoreferencing. International Archives of the Photogrammetry,Remote Sensing and Spatial Information Sciences 34 (Part 3A),7984.

Drstel, C., 2003. DMC practical experiences and photogrammetricsystem performance. In: Fritsch (Ed.), Photogrammetric Week2003. Wichmann Verlag, pp. 5965.

Eisenhart, C., 1963. Realistic evaluation of the precision and accuracyof instrument calibration systems. Journal of Research, NationalBureau of Standards 67C (2).

Frstner, W., Wrobel, B., Paderes, F., Craig, R., Fraser, C., Dolloff, J.,

metry & Remote Sensing 60 (2006) 3873992004. In: McGlone, J.C., Mikhail, E., Bethel, J. (Eds.), Analyticalphotogrammetric operations, ASPRS Manual of Photogrammetry,

5th edition. American Society for Photogrammetry and RemoteSensing, pp. 763936.

Fricker, P., 2001. ADS40 progress in digital aerial data collection.In: Fritsch, D., Spiller, R. (Eds.), Photogrammetric Week 2001.Wichmann Verlag, pp. 6776.

Heipke, C., Jacobsen, K., Wegmann, H., 2002. Analysis of the Resultsof the OEEPE Test Integrated Sensor Orientation. In: Heipke, C.,Jacobsen, K., Wegmann, H. (Eds.), OEEPE Official Publication,vol. 43, pp. 3149.

Honkavaara, E., 2004. In-flight camera calibration for directgeoreferencing. International Archives of Photogrammetry, Re-mote Sensing and Spatial Information Sciences 35 (Part B1),166171.

Honkavaara, E., Ilves, R., Jaakkola, J., 2003. Practical results of GPS/IMU/Camera-system calibration. Proceedings of InternationalWorkshop: Theory, Technology and Realities of Inertial/GPSSensor Orientation, Castelldefels, Spain, 22.23.9.2003, 10 pp.CD-ROM.

Honkavaara, E., Markelin, L., Ilves, R., Savolainen, P., Vilhomaa, J.,Ahokas, E., Jaakkola, J., Kaartinen, H. 2005. In-flight perfor-mance evaluation of digital photogrammetric sensors. Proceed-ings of ISPRS Hannover Workshop, 1720 May 2005. 6 pp., onCD-ROM.

Jacobsen, K., 2004. Direct integrated sensor orientation pros andcons. International Archives of Photogrammetry, Remote Sensingand Spatial Information Sciences 35 (Part B3), 829835.

Kraus, K., 1993. Photogrammetry, Volume 1: Fundamentals andStandard Processes. ISBN: 3-427-78684-6.

Kremer, J., Gruber, M., 2004. Operation of UltraCamD together withCCNS/Aerocontrol first experiences and results. InternationalArchives of Photogrammetry, Remote Sensing and SpatialInformation Sciences 35 (Part B1), 172177.

Krpfl, M., Kruck, E., Gruber, M., 2004. Geometric calibration of thedigital large format camera UltraCamD. International Archives ofPhotogrammetry, Remote Sensing and Spatial InformationSciences 35 (Part B1), 4244.

Kuittinen, R., Ahokas, E., Hgholen, A., Laaksonen, J., 1994. Test-field for aerial photography. The Photogrammetric Journal ofFinland 14 (1), 5362.

Leberl, F., Gruber, M., 2003. Flying the new large format digital aerialcamera Ultracam. In: Fritsch (Ed.), Photogrammetric Week 2003.Wichmann Verlag, pp. 6776.

Livingston, R.G., Berndsen, C.E., Ondrejka, R., Spriggs, R.M.,Kosofsky, L.J., Van Steenbrugh, D., Norton, C., Brown, D., 1980.Chapter 4.8: Camera Calibration, In: Slama, C.C., Theurer, C.,Henriksen, S.W. (Eds.), Manual of Photogrammetry, FourthEdition. American Society of Photogrammetry and RemoteSensing, pp. 232277.

Merchant, D.C., Schenk, A., Habib, A., Yoon, T., 2004. USGS/OSUprogress with digital camera in situ calibration methods.International Archives of Photogrammetry Remote Sensing andSpatial Information Sciences, vol. 35 (Part B2), pp. 1924.

Petrie, G., 2003. Airborne digital frame cameras. Geoinformatics 7 (6),1827.

Petrie, G., 2005. Airborne pushbroom line scanners. Geoinformatics 1(8), 5057.

399E. Honkavaara et al. / ISPRS Journal of Photogrammetry & Remote Sensing 60 (2006) 387399

Geometric test field calibration of digital photogrammetric sensorsIntroductionCalibration methodOutline of the methodThe quantities to be determinedCalibration blockGeometric test field

Materials and methodsUltraCamDFlight missionsMethods

Results and discussionTheoretical expectationsGeneral resultsCalibration parametersSystematic image errorsGeometric accuracy assessmentAccuracy of point determinationBack projection accuracy

StabilityComparison with the analog cameraTreatment of the multi-head distortions

ConclusionsAcknowledgementsReferences