9

Worth a thousand words – Photogrammetry for archaeological 3D surveying

Fabio Remondino

After a period during which laser scanners have dominated surveying applications, photogrammetry is slowly coming back, both at research and commercial level. Indeed recent developments and improvements of hardware and software are drawing attention back to the image-based approach for many surveying and modelling purposes. This article reports the state-of-the art of photogrammetry for archaeological 3D mapping applications with examples at different scale.

Keywords: photogrammetry, image-based modelling, automation, lidar

“It is quite evident that even with our past progress, we have only scratched the surface of the possibilities in the use of photogrammetry” (George D. Hardy, 1973)

IntroductionA decade’s experience of recording and creating digital 3D datasets such as point clouds, digital surfaces or solid models has demonstrated that these data can be a truly valuable resource for many heritage applications and activities like interpretation, analyses, computer-aided restoration, cross-comparison, monitoring of shape and colour, the creation of 3D repositories and catalogues, and multimedia exhibitions; all this is in addition to the more traditional activities

of historical documentation, digital conservation and interactive visualization. Unfortunately examples of research applications which move beyond visualizations and metrics, however sophisticated, remain limited (e.g. Figure 9.1). Indeed 3D models often do not move beyond serving as very detailed documentation of an object, site or landscape and their aesthetic value is sometimes treated as an end in itself. In too many cases archaeologists do not succeed in extracting new archaeological information from 3D data which may have a real impact on knowledge and allow us to address larger research questions. This gap between the creation of a model and its interpretation and use in upstream research can be traced, at least in part, to both a

Figure 9.1: Examples of 3D modelling from terrestrial images found on the Internet or acquired during holiday trips

Fabio Remondino118

communication gap between the archaeological and surveying communities and also to the absence of proper tools (mainly software) for non-experts that are able to produce detailed information appropriate for archaeological needs.

This gap in communication and expertise sharing and the lack of effective tools is present at each stage in the digital recording and 3D modelling process. The complete process is multi-faceted, involving data collection, processing, visualization, interpretation, reprocessing, manipulation and cleaning, and more visualization and interpretation, with deliberate choices made at every stage. The manner of data acquisition and processing, the nature of the surveyed subject, and the intended purpose of the 3D recording all have significant impacts on the overall process and the final result. The steps of digital data capture, data processing, archiving and long-term data management, visualization and data reproduction form a chain of procedures and data transformations performed to derive new products, losing and altering elements of the original data along the way. To create high-fidelity digital models, the operator should make choices and employ algorithms that ensure that any processes where large amounts of information are lost occur as late as possible in the processing chain. This paper focuses specifically on the choice between laser scanning and photogrammetric approaches made at the initial data collection step of the 3D digital modelling process, emphasizing the potential and value of photogrammetric approaches, and comments on the impact of this choice on later stages. Examples throughout the text illustrate the potential of photogrammetry and the entire image-based modelling procedure.

Photogrammetry (Mikhail et al. 2001; Luhmann et al. 2007) is a robust technique for processing image data to create 3D models. Starting from corresponding features in images (tie points), metric and detailed 3D information with statistical estimates of unknown parameters are derived. Photogrammetry developed from its origin as an analogue process to a fully digital modelling technique in the 1980s. While satellite imagery is now of a suitable resolution, aerial imagery is still the most common source of data for large scale mapping. In recent years developments in photogrammetry have been complemented and strengthened by those in the fields of computer vision, image processing

and 3D reconstruction- most importantly those developing ‘structure from motion’ procedures. Computer vision researchers have focused on developing a fully automated processing pipeline based on projective geometry. These fully automated approaches may reduce the accuracy of the results, but can be useful for visualization, object-based navigation, location based services, robot control, shape recognition, augmented reality, annotation transfer or image browsing purposes- and hence are of interest for some archaeological applications. The greatest benefit of advances in computer vision to photogrammetry is the continuous development of new image analysis algorithms and 3D reconstruction methods. These have been adopted by the photogrammetric community in order to automate most of the steps of the 3D modelling pipeline

Collecting 3D dataThe creation of a 3D model begins with data collection. Reality-based 3D surveying and modelling of heritage sites and objects may be achieved using passive sensors and image data (Remondino and El-Hakim 2006), active sensors and range data (Vosselman and Maas 2010), classical surveying (e.g. total stations or GNSS), extrusion functions from existing 2D drawings and maps (Yin et al. 2009) or an integration of the aforementioned techniques, as is common in cases of large and complex sites (Stamos et al. 2008; Guidi et al. 2009; Remondino et al. 2009). In general 3D surveying involves the acquisition of unstructured 3D data (e.g. point clouds) using a chosen sensor and technique, while 3D modelling denotes the procedure of converting those unstructured data into structured 3D data (e.g. polygonal meshes). The surveying technique (e.g. 3D scanning or imaging) is selected according to the object’s dimensions, location constraints, instrument portability and usability, surface characteristics, working team experience, budget, final goal of the survey and so on. An image-based survey is defined as a survey where photographic images, together with an element of scale or some ground control points, are processed (e.g. with photogrammetry) to produce 3D reconstructions and line drawings. Passive, image-based sensors (e.g. digital cameras) deliver image data which are processed using mathematical formulae and photogrammetric techniques to infer 3D

1199 Worth a thousand words – Photogrammetry for archaeological 3D surveying

information from 2D measurements. A range-based survey is defined as a survey where range sensors (e.g. terrestrial or airborne laser scanning) are employed to directly record an unstructured 3D point cloud of the surveyed scene. To date, many projects have elected to use (airborne or terrestrial) laser scanning due to the high reliability and ease of use of such instruments, in spite of challenges posed by time consuming post-processing and comparatively bulky and expensive equipment. Recent (i.e. post-2008) developments in automated and dense 3D reconstruction from images (Hirschmueller 2008; Remondino et al. 2008; Hiep et al. 2009; Furukawa and Ponce 2010; Verhoeven 2010; Pierrot-Deseilligny et al. 2011), based on photogrammetry and computer vision methods have shown very promising results. This development, coupled with the release of web-based, automated processing tools such as ARC3D and Photosynth (Vergauwen and Van Gool 2006; Snavely et al. 2008) , for example, and open-source image processing algorithms (Bundler, Apero, MicMac, PMVS, etc.) have renewed attention on image-based 3D modelling as an inexpensive, robust and practical alternative to 3D scanning.

A 3D surveying and modelling process, in addition to delivering metric and possibly geo-referenced results, should quantify accuracy (the closeness between measured values and reference values) and precision (the consistency with which a measurement or a set of measurements can be repeated), and provide good value in performance, completeness, portability, flexibility and, ideally for archaeologists, should also be low cost. While it is not easy to achieve all this in a single technique – hence sensor and data integration, in particular for large and complex sites – thanks to recent developments in processing algorithms and software, images can be used as a unique data source to produce very satisfactory 3D models for many applications. To achieve good results experience is still required for image acquisition and processing, but increasing levels of automation are allowing non-expert surveyors to achieve aesthetically pleasing 3D models. Increases in automation and consequent ease of use should not, in theory, come at the price of the geometric quality of the digital model. The quantitative comparison of the results of automated processes in comparison with manual, expert processing methods is ongoing and a topic beyond the scope of this paper, rather the usability (for archaeologists)

of the results of automated processes is the focus here.

Recording optical sensors and imaging platformsA variety of image-based sensors are available which may be used for 3D photogrammetric documentation (Remondino 2011). Images can be acquired with terrestrial, aerial or satellite sensors. Terrestrial digital cameras include CCD/CMOS sensors, frame, linear, multiple head, SLR-type, industrial, off-the-shelf, high- speed, panoramic head cameras, and still-video. Common terrestrial cameras have at least 10–12 Megapixels at very low price while high-end digital cameras feature more than 40–50 Megapixel imaging sensors. Airborne cameras for photogrammetric surveying are generally classified as small, medium and large format cameras (Sandau 2009), they feature frame or linear array sensors and they are generally coupled with GNSS/IMU (Global Navigation Satellite System/Inertial Measurement Unit) systems. In addition to available aerial acquisition platforms, particular attention has been devoted to UAVs (Unmanned Autonomous Vehicles) like low-altitude model helicopters or drones. These platforms can fly in an autonomous mode, using an integrated GNSS/IMU which facilitates many procedures in the data acquisition and processing chain, a stabilizer platform to achieve precise, high-quality image overlap and digital cameras. UAVs can be used to collect image data from otherwise difficult to access areas (Figure 9.2) or integrated with terrestrial surveying, and applications are increasing rapidly due to their flexibility, the ability to fly in restricted areas, ease of manoeuvrability and transport, flexible working heights and the possibility to shoot in all directions (vertically down, but also in convergent and horizontal modes). The images acquired can be transmitted to a ground station and processed in situ to ensure complete coverage and provide some quality control for the final models (Figure 9.3).

While much of the recent development in photogrammetric documentation has used close-range ground based techniques or UAVs, satellite imagery has the potential in the near future to provide high quality models for large landscape areas (Dowman et al. 2011). Many optical satellite sensors have a return-to-location time of less than

Fabio Remondino120

Figure 9.5: Archaeological artefact (a, b) surveyed and modelled in 3D using images. To derive metric and scaled results, a known distance (here on a scaled bar) is necessary (b). Results achieved with interactive measurements (c, 260 polygons) and automated 3D processing (d, 1.9 Mil. polygons)

Figure 9.2: A set of images acquired with an UAV platform over a temple (a), the image orientation results (b) and the surface model produced (c) for orthoimage and site map generation

Figure 9.3: A mosaic of UAV images (left) acquired above the excavation area of Pava (Siena, Italy). The DSM, derived for exaction documentation, monitoring, GIS and archiving purposes, is shown in shaded (centre) and textured mode (right)

Figure 9.4: Dense 3D reconstruction of a temple in Paestum from about 150 images for documentation, conservation and communication purposes

Figure 9.6: Image-based surveying of a Buddha statue and generation of a 3D point cloud, shown in colour-code and textured mode respectively

1219 Worth a thousand words – Photogrammetry for archaeological 3D surveying

three days, providing close temporal resolution. Large archives of imagery are available, often providing stereo-pairs, providing historical depth unavailable from other sensors. In addition to time-depth, the spatial resolutions achieved are comparable to those from laser scanning. The smallest geometric element which is visible in the images, generally called the Ground Sample Distance (GSD) or footprint can reach 45 cm for civil applications using high resolution satellites, while airborne sensors can deliver images with a 10–20 cm GSD and UAV flights can produce images with very high GSD of about 1–5 cm (Figure 9.4) and terrestrial close-range imagery can produce a GSD of less than 1 mm.

Data processing and point cloud generationThe typical image processing pipeline consists of the following steps:

• Sensor calibration to determine the internal deformities and parameters of the employed sensor;

• Sensor orientation (or image triangulation) to retrieve the 6 DoF camera poses (i.e. positions where the images have been acquired), normally using a bundle adjustment method;

• Surface measurement to derive a dense or sparse point cloud of the surveyed scene using multi-image matching algorithms in the former case, or manual measurement in the latter;

• Feature extraction to create vector data or layers for GIS applications;

• Digital Surface Model (DSM) and Digital Terrain Model (DTM) generation for site documentation, volume computation, or thophoto creat ion, etc . for other applications;

• Orthophoto production (e.g. precisely rectified images where scale error and perspective distortion have been removed) and texture mapping.

All these steps can be performed in most commercial photogrammetric software, using both manual and automated procedures. The automated processing of multiple images is the key feature that brings photogrammetry away from the (inefficient) old days of manual point identification and stereo-pair creation and makes it a pragmatic choice for an archaeological project. User interaction is still required for scaling and geo-referencing, and unless the image processing is performed simply to derive a nice-looking 3D model without any metric requirement this step will require the most interaction and effort (Figure 9.5). At each step in the processing chain the choice between manual and automated approaches, and between technically correct and approximating techniques, affects the quality of the final result and the time needed to achieve it.

In the case of terrestrial acquisitions, digital cameras must be accurately calibrated, preferably in a controlled laboratory environment, with a 3D testfield and a bundle adjustment solution with additional parameters to fully compensate for systematic errors (Remondino and Fraser 2006). Generally a target-based approach is used in order to automatically identify and precisely measure the match points in the images. When archive images are used (Gruen et al. 2004), camera calibration becomes tricky, requiring assumptions and hypotheses to correctly process the images and derive an accurate 3D model.images and derive an accurate 3D model.model.

Once a basic model has been created from the image data, further reconstruction of 3D scenes and model refinement will depend on the project requirements. In the case of complex architectural scenes, man-made objects, detailed city modelling and cartographic applications at large scale, manual or semi-automated approaches are generally the preferred tools for surface measurement and feature extraction, as they remain more precise and reliable (Figures 9.6 and 9.7). On the other hand, small free-form

Figure 9.7: Aerial oblique survey over a mediaeval village in Tuscany (a) to produce a dense and detailed 3D point cloud of the heritage (b, c)

Fabio Remondino122

objects, detailed ornaments or small/medium scale DSM can be automatically reconstructed using advanced matching techniques in order to produce dense 3D point clouds and textured polygonal models (Hirschmueller 2008; Remondino et al. 2008; Haala, 2009; Hiep et al. 2009; Gehrke et al. 2010).

Methods based on structure-from-motion procedures combine the sensor orientation and 3D reconstruction in a single phase, leaving the scaling and geo-referencing, if required, to the end of the procedure. The scaling and geo-referencing is accomplished with a simple Helmert transformation. While structure from motion techniques make for very efficient processing pipelines, this aspect of the technique is less than ideal; the technically correct approach requires the introduction of the datum information (e.g. Ground Control Points) in the orientation phase.

The lidar alternativeLidar is probably the most significant measurement technology introduced in the topographic mapping field in the last decade (Shan and Toth 2008). The term lidar (Light Detection And Ranging) is a general term used to indicate laser scanning surveying. Laser scanners can record the intensity of the surveyed scene, and are effective even on textureless areas (in contrast to photogrammetry).

Since 2000 Airborne Laser Scanning (ALS) and Terrestrial Laser Scanning (TLS) have been used in various applications, with continuous scientific investigations and improvements in

both hardware and software (Opitz this volume). Commercial ALS systems are based on the Time of Flight (ToF) measurement principle with near infrared (NIR) laser light and commercial TLS are based on ToF and triangulation measurement principles with laser light in the range 450–1500 nm. The main advantages of laser scanning is the direct creation of a dense 3D point cloud and, in some cases, the possibility to map structures hidden by vegetation thanks to the multiple return or full waveform digitization. Penetration through the vegetation canopy is a key factor in the wide acceptance of lidar as a tool for landscape archaeology.

Ten years ago, as laser scanning was growing in popularity as a means to produce dense point clouds for 3D documentation, mapping and visualization purposes at various scales, photogrammetry could not efficiently deliver results similar to those achieved with lidar instruments. Consequently lidar became the dominant technology in 3D recording in archaeology, and replaced photogrammetry in many application areas. Further, many photogrammetric researchers shifted their research interests to lidar, resulting in further decline in advancements in image-based photogrammetric techniques. Over the past four years improvements in hardware and software, primarily from the computer vision community, have improved photogrammetry-based tools and algorithms to the point that lidar and photogrammetry now can deliver comparable geometrical 3D results for many terrestrial applications (Figure 9.8; Remondino et al. 2008; Pierrot-Deseilligny et al. 2011)

Figure 9.8: A burial excavation (a) reconstructed in 3D (b, c) from a set of terrestrial images for metric documentation and physical replica purposes

1239 Worth a thousand words – Photogrammetry for archaeological 3D surveying

Lidar vs. PhotogrammetryAs terrestrial scanning and photogrammetry can produce comparable results, the choice between the two depends primarily on project constraints and requirements. Terrestrial laser scanning instruments are still relatively cumbersome and expensive compared to terrestrial digital cameras (off-the-shelf or SLR-type) and their bulkiness might be problematic in some field campaigns or research projects. The point clouds recorded with lidar instruments are already metrically correct, but they are not based on redundant measurements which may be problematic for projects concerned with absolute accuracy. Typical photogrammetric measures derived in the adjustment procedures (variance estimations and other statistical matrices) are not available to evaluate the quality of the lidar point clouds produced; laser scanners provide just a few statistics to describe errors for the entire dataset (normally these numbers are provided by the vendors). TLS errors are normally treated as ‘black boxes’ as they lack well-defined procedures to assess per-project quality, and because internal procedures are hidden, raw data are normally not accessible and calibration or correction procedures are not clear. Photogrammetric processing, while more labour intensive in this case, can be carried out so that calibration procedures, corrections, and error metrics are explicitly stated.

Photogrammetric processing algorithms can suffer from problems with the initial image quality (noise, low radiometric quality, shadows, etc.) or certain surface materials (shiny or texture-less objects), resulting in a noisy DSM or difficulties in feature extraction. Furthermore, in order to derive metric 3D results from images a known distance or ground control points are required. On the other hand, for aerial applications photogrammetric approaches can be advantageous in open areas. The typical point density of lidar datasets is around 1–20 points/m2 while an aerial photogrammetric

image typically has a GSD on the order of 10 cm and could theoretically be used to produce a DSM with 100 points/m2.

ConclusionsThe article has described the state-of-the art of photogrammetry for archaeological 3D surveying and modelling. The richness of image content information cannot be matched by any other data acquisition device and examples demonstrate the potential of an image-based methodology. Image-based 3D modelling is a cheap but powerful alternative to 3D scanning. Although experience in image acquisition and 3D modelling is still needed, various open-source and web-based tools are available to create 3D results from images with little expertise required (Figure 9.9). Although these automated processes are rapidly being adopted by archaeological projects, the user should always keep in mind the goal of the 3D reconstruction and be able to distinguish between nice-looking 3D models and accurate metric results, and which is important for any given application.

From a research point of view, the great scientific and technological challenges today are no longer related to sensor and system development, but rather to algorithm design and improvement. Indeed, having a large variety of imaging sensors (from satellite to underwater) at our disposal, the obvious question is not the source of data or the processing technique, but the final accuracy of the 3D data that can be derived. So there is a clear future for photogrammetry at least in geo-spatial and precision mapping applications. Photogrammetry is definitively out of the shadow of lidar and is once again an active research area. Of course, the two techniques should not be considered as competing, but as complementary given all the technical differences between them.

Despite all the potential of photogrammetry

Figure 9.9: 3D modelling of the lower part of a column for documentation and virtual reconstruction purposes. (a–b) images of the acquired sequence; (c) produced shaded 3D model; (d, e) final textured 3D model

Fabio Remondino124

(and lidar) and the constant pressure of inter-national heritage organizations, a systematic and targeted use of 3D surveying and modelling in the Cultural Heritage field is still not yet employed as a default approach. Furthermore when a 3D model is produced, it is often subsampled or reduced to a 2D drawing due to a lack of software or knowledge for the proper handling of 3D data by non-experts. It should be clear that the availability and use of 3D metric data opens a wide spectrum of further applications and allows new analyses, studies, interpretations, conservation policies or digital restoration. Thus 3D virtual heritage should be used more frequently, taking advantage of all that remote sensing technologies and the third dimension offer to the heritage world.

ReferencesAgarwal, S. Snavely, N., Simon, I., Seitz, S.M.,

Szeliski, R., 2009. Building Rome in a day. Proc. Int. Conference on Computer Vision. Kyoto: Japan.

Dowman, I., Jacobsen, K., Konecny, G., Sandau, R., 2011. High Resolution Optical Satellite Imagery. Whittles Publishing.

Furukawa, Y. and Ponce, J., 2010. Accurate, dense and robust multiview stereopsis. IEEE Transactions on Pattern Analysis and Machine Intelligence 32(8), 1362–76.

Gehrke, S., Morin, K., Downey, M., Boehrer, N. and Fuchs, T., 2010. Semi-global matching: an alternative to LiDAR for DSM generation? International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences 38(1), Calgary, Canada (on CD-. ROM).

Gruen, A., Remondino, F. and Zhang, L., 2004. Photo-grammetric Reconstruction of the Great Buddha of Bamiyan, Afghanistan. The Photogrammetric Record 19(107), 177–99.

Guidi, G., Remondino, F., Russo, M., Menna, F., Rizzi, A. and Ercoli, S., 2009. A Multi-ResolutionA Multi-Resolution methodology for the 3D modeling of large and complex archaeological areas. International Journal of Architectural Computing 7(1), 39–55.

Haala, N., 2009. Comeback of digital image matching. Photogrammetric Week, 289–301.

Hirschmueller, H., 2008. Stereo processing by semi-global matching and mutual information. IEEE Transactions on Pattern Analysis and Machine Intelligence 30(2), 328–41.

Hiep, V.H., Keriven, R., Labatut, P. and Pons, J.P., 2009. Towards high-resolution large-scale multi-view stereo. Proc. Computer Vision and Pattern Recognition. Kyoto: Japan

Luhmann, T., Robson, S., Kyle, S. and Hartley, I., 2007. Close Range Photogrammetry: Principles,

Techniques and Applications. Whittles: Dunbeath, UK.

Mikhail, E.M., Bethel, J.S. and McGlone, J.C., 2001. Introduction to modern photogrammetry. Wiley.

Pierrot-Deseilligny, M., De Luca, L. and Remondino, F., 2011. Automated image-based procedures for accurate artifacts 3D modeling and orthoimage generation. 23th Int. CIPA (International Scientific Committee for Documentation of Cultural Heritage) Symposium, Prague, Czech Republic (on CD-ROM).

Remondino, F. and El-Hakim, S., 2006. Image-based 3d modelling: a review. The Photogrammetric Record 21(115), 269–91.

Remondino, F. and Fraser, C., 2006. Digital camera calibration methods: considerations and com-parisons. International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences 36(5), 266–72.

Remondino, F., El-Hakim, S., Gruen, A. and Zhang, L., 2008. Development and performance analysis of image matching for detailed surface reconstruction of heritage objects. IEEE Signal Processing Magazine 25(4), 55–65.

Remondino, F., El-Hakim, S., Girardi, S., Rizzi, A., Benedetti, S. and Gonzo, L., 2009. 3D Virtual3D Virtual reconstruction and visualization of complex architectures – The 3D-ARCH project. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences 38(5/W1).

Remondino, F., 2011. Heritage Recording and 3D Modeling with Photogrammetry and 3D Scanning. Remote Sensing 3(6), 1104–38.

Sandau, R., 2009. Digital Airborne Camera: Intro-duction and Technology. Springer.

Shan, J. and Toth, C., 2008. Topographic Laser Ranging and Scanning: principles and processing. CRC Press.

Snavely, N., Seitz, S. M. and Szeliski, R., 2008. Modeling the world from internet photo collec-tions. International Journal of Computer Vision 80(2), 189–210.

Stamos I., Liu L., Chen C., Woldberg G., Yu, G. and Zokai S., 2008. Integrating automatedIntegrating automated range registration with multiview geometry for photorealistic modelling of large-scale scenes. International Journal of Computer Vision 78(2–3), 237–60.

Vergauwen, M. and Van Gool, L., 2006. Web-Based 3D Reconstruction Service. Machine Vision Applications 17, 411–26.

Verhoeven, G., 2010. Taking computer vision aloft – archaeological three-dimensional reconstruction from aerial photographs with PhotoScan. Archaeo-logical Prospection 18(1), 67–73. DOI: 10.1002/arp.399.

Vosselman, G. and Maas, H.-G., 2010. Airborne and terrestrial laser scanning. CRC Press.

Yin, X., Wonka, P. and Razdan, A., 2009. Generating 3d building models from architectural drawings: A survey. IEEE Computer Graphics and Applications 29(1), 20–30.