Cultural Heritage 16

OPTICAL MEASUREMENT TECHNIQUES FOR THE DIGITAL PRESERVATION, DOCUMENTATION AND

ANALYSIS OF CULTURAL HERITAGE

A. Rizzi1, F. Voltolini1, F. Remondino2, S. Girardi1, L. Gonzo1 1Fondazione B.Kessler-IRST, Trento, Italy

<rizziale><fvoltolini><girardi><lgonzo>@itc.it, www.itc.it 2Institute of Geodesy and Photogrammetry – ETH Zurich, Switzerland

fabio@geod.baug.ethz.ch, www.photogrammetry.ethz.ch KEY WORDS: Cultural Heritage, 3D Modeling, Photogrammetry, Range Sensors, Comparison. ABSTRACT Nowadays the digital documentation and 3D modeling of Cultural Heritage objects and sites is receiving much more attention. This is due from one side to the large amount of technologies and data available for fast, detailed and quick documentation and from the other side to the great interest of restorators and archaeologists towards instruments able to provide innovative inspections and diagnostic results. The correct selection, combination and integration of the available techniques requires great experience and the need to combine them comes from the complexity of some heritage structures and the lack of a single method capable of providing satisfactory results in all measuring conditions. In this work we report our use of consumer digital cameras, range sensors and infrared thermography data for the documentation of heritage objects. The objective is to give an overview of the different techniques and the used methodology for the digital recording of detailed objects, monuments and large structures.

1 INTRODUCTION

Many different documentation methods are available nowadays, ranging from hand drawings to scanning, from surveying to radar, from satellite to underwater acquisitions. A method is generally selected according to the project requirements, the available budget and the experiences of the investigator. The need for (digital) documentation, preservation and conservation is given by the constant threat and danger that are affecting our Cultural Heritage. Currently we can distinguish three main approaches for the optical recording, documentation and visualisation of heritage sites, monuments and finds: 1. Image-based methods (e.g. photogrammetry): these methods (Remondino and El-Hakim, 2006)

are widely used for the 3D reconstruction of architectural objects (Debevec et al., 1996; El-Hakim, 2002; Voltolini et al., 2006), for the precise modelling of terrain and cities (Grün 2000) or monuments and statues (Visnovcova et al. 2001, Gruen et al., 2004) and lately also for precise and detailed modeling of complex objects using consumer-grade digital cameras (Remondino and Zhang, 2006; Chandler et al., 2007). Image-based methods use projective geometry (Pollefeys et al. 2004) or a perspective camera model. They are highly portable and the sensors are often low priced.

2. Range-based methods (e.g. laser scanning): these techniques (Blais, 2004) are based on active sensors that directly capture the geometric 3D information of an object using artificial laser light (e.g. ShapeGrabberTM, RieglTM) or projecting a pattern (e.g. BreukmannTM) without requiring a mathematical model to derive 3D information from 2D observations. Applying different measurement principles, they capture millions of points in relatively short time but require great

Rizzi, Voltolini, et al.

17

editing efforts to correctly model the recorded 3D data. They are pretty simple to be used even if still quite expensive and often unpractical in some field campaigns.

3. Combination of image- and range-based methods: in many applications, a single 3D modelling method that satisfies all project requirements is not yet available. Photogrammetry and active sensors have been often combined in particular for the recording of large architectural objects or complex archaeological sites, where no technique by itself can efficiently and quickly provide a complete and detailed model (Bernardini et al. 2002; El-Hakim et al. 2004; Lambers et al. 2007; Voltolini et al., 2007).

Comparisons between image-based and range-based modeling are reported in (Böhler and Marbs, 2004; Kadobayashi et al., 2004; Böhler, 2005; Remondino et al., 2005). We can safely say that at the moment, for all types of objects and sites, there is no single modeling technique able to satisfy all requirements of high geometric accuracy, portability, full automation, photo-realism and low costs as well as flexibility and efficiency. In this paper we report our experiences in modeling heritage, using both images and range sensors. After a review of both techniques, we will illustrate some examples, commenting also the accuracy of the used approaches.

2 TERRESTRIAL IMAGE-BASED MODELING

Among the available optical measurement techniques, images are able to provide for both 3D geometric and textural information about an objects or contexts under documentation. Image-Based Modeling (IBM) methods (like photogrammetry, shape from shading, shape from texture, shape from specularity, shape from contour, shape from 2D edge gradients, etc) use 2D image measurements to recover 3D object information through a mathematical model. Generally at least two images are required, even if it is possible to infer 3D data using a single image. Data acquisition is quick and image analysis may be undertaken parallel to ongoing survey, excavation or restoration. Inexpensive systems are available for image acquisition and old images may be also used to document lost objects (Grün et al., 2004). Portability, flexibility and the possibility to acquire image data from ground level or at different altitude and different position are just few advantages of IBM approach. These peculiarities are important when one deals with complex heritage or large architectures because many parts of the model could be occluded by some obstructing objects, as well as narrow paths (Voltolini et al., 2006). Between IBM methods, photogrammetry is a highly accurate and flexible approach, suited for different scales. While photogrammetric image analysis is not a trivial task, with the advent of automated procedures and fully digital products in the 90’s it has become easier to use, cheaper and a wide range of commercial software is available. Photogrammetric packages allow, after the (manual) tie points measurement and bundle adjustment phase, to obtain sensor calibration and orientation data, three-dimensional object point coordinates from multi-image networks, as well as wireframe or textured 3D models. At the moment no commercial solution is able to perform an automated markerless image orientation while calibration based on coded-target is an issue already solved since some years (Gangi and Hanley, 1998; Cronk et al., 2006). Furthermore, there is no package able to automatically reconstruct a complex surface model employing more than two images: indeed commercial photogrammetric software (e.g. LPS - Leica Geosystem, PI-3000 - Topcon) have only a matching tool able to provide dense surface model from ‘convergent’ stereo-pairs (Kadobayashi et al., 2004; Chandler et al., 2007) while a multi-photo approach would be more reliable and efficient. In order to obtain accurate and correct results, the employed cameras have to be calibrated (Remondino and Fraser, 2006). It is generally better first to calibrate the internal camera parameters with a given setting and using the most appropriate image network and then recover the object geometry using those calibration parameters at the same camera setting.

Cultural Heritage 18

The entire photogrammetric workflow used to derive metric and reliable information of a scene from a set of images consists of (i) calibration and orientation, (ii) 3D measurements via image matching, (iii) structuring and modeling, (iv) texture mapping and visualization. For a review of the entire image-based modeling pipeline we refer to (Remondino and El-Hakim, 2006). Even though the terrestrial photogrammetry packages are generally user friendly, it is still necessary to have a good photogrammetric background to use them in a correct way. Moreover, in order to produce a detailed and accurate model of a complex building or monument, the task can be cumbersome and time consuming. Therefore we believe that site managers, archaeologists, restorators, conservators and the whole heritage community need more simple, cost-effective and fast methods to record and document heritage from images. The latest developments, in particular related to surface reconstruction, are going in this direction (El-Hakim, 2006; Remondino and Zhang, 2006). Therefore, photogrammetry is acquiring a growing important role in the digital cultural heritage documentation and preservation fields, where detailed, precise and reliable results are needed. In Figure 1 we report some examples of objects surveyed with photogrammetry.

Figure 1: Two examples of 3D models created by means of image-based modeling technique (Voltolini et al., 2007; Remondino, 2007).

3 TERRESTRIAL RANGE-BASED MODELING

Range-based approaches are often complementary or combined to image-based ones (El-Hakim et al. 2004; Voltolini et al., 2007). The use of active sensors has become a very common source of documentation data in recent years, in particular for non-expert users, as they provide range data of surfaces in high resolution and generally with high accuracy. In many cases there aren’t good conditions to acquire images due to constraints or occlusions in the acquisition and it is impossible to work with a good intersection angle, baseline and overlap (Voltolini et al., 2006). These are the major obstacles that lead up to use in some applications active sensors instead of images. Also (i) the easiness of use of range sensor, (ii) the possibility to reach high level of detail generally with good accuracy and (iii) the fast acquisition time, suggest the use of scanners, even though the last researches in the photogrammetric field are changing this trend. Like photogrammetry, active sensors are suitable for different scales. While the recording devices are still expensive, important progress has been made in recent years towards an efficient processing and analysis of the range data. Range-based systems (Blais, 2004) are mainly based on two principles: triangulation and time delay. Triangulation-based systems work on short distance (between 0.1 and 500 cm) and can reach very high accuracy. The considered object is pointed by a light beam that is seen by a CCD camera. A triangle is formed by the point source of the light spot, the light point on the surface of the object

Rizzi, Voltolini, et al.

19

and the projected point on the CCD camera. The triangle in uniquely defined by two angles and the distance can be measured. The process has to be repeated for all points of the object. In many cases, to decrease the amount of scanning time, stripe light is projected instead of a single dot. In these systems, called slit laser scanners, the laser line is projected on the object and imaged at an angle. The deformation of the profile is a direct function of range. In our research work, triangulation-based active sensors have been used when high resolution was necessary. The high accuracy in the 3D measurements is in fact essential for 3D documentation of very detailed object as sculptures, low reliefs and other complex small objects. In Figure 2 we report some examples of small objects surveyed with a Shape Grabber SG1000 and a Breuckmann OptoTOP-SE.

Figure 2: Some detailed 3D models of cultural heritage objects created with a triangulation-based laser scanner and using a stripe projection system (right) (Akca et al., 2006).

Active sensors that use pattern projection technique are also based on the triangulation principle. A grid of lines is imaged from a viewpoint that is in a different position from the source spot viewpoint. The pattern is equivalent to a second camera used in the stereo-photogrammetry and the position of a point is determined through the triangulation principle. These systems have usually a field of view smaller than triangulation-based laser scanner but can generally rich a higher accuracy. Time delay active sensors are instead employed for large objects, as for example architectures, rooms, walls and archaeological sites. These sensors work on longer distances (i.e. 2 – 1000 m) but are less accurate than triangulation systems. The systems measure the time that light needs to propagate from laser source to the object and back. Assuming a constant speed light the distance to the object can be calculated. There are two main methods to calculate the distance. The first one is based on pulse principle that detects the time a laser is reflected back to the receiving detector, usually a photodiode. The second method is amplitude modulation of the optical carrier that measures the range from the phase variation between transmitted and received signal. This second method gives better resolution but works until a predefined range. Figure 3 shows some 3D objects modeled with a Leica HDS 3000 laser scanner. Other types of active sensors are conoscopic systems that have better precision than triangulation ones (few microns) but a smaller field of view. For these reasons they are generally used for planar object, as for example paintings and small coins. These systems are based on structured light projected on the object. The measurement process calculates the distance between the illuminated point and a fixed reference plane. It’s impossible to give a general rule to decide which technique has to be used in a certain situation and with which instrument. In addition to the geometric information acquired with active sensors, digital images have to be used (and generally separately acquired) for the texture mapping purpose to get a photo-realistic 3D model (Bernardini and Rushmeier, 2002; El-Hakim at al., 2003).

Cultural Heritage 20

Figure 3. Medium and large size objects modeled with a TOF laser scanner.

4 APPLICATIONS

Optical measurements are a key aspect in Cultural Heritage documentation. Which technique is needed to recover 3D information depends on many factors, as explained in the previous sections. Hereafter we report some image-based terrestrial modeling of heritages, discussing also the accuracy of the measurements by comparing the image- and range-based approaches. In Figure 4 is reported the 3D modeling of a small relief (ca 80x50 cm) performed using six images acquired with a 13,5 Mega pixel SRL Kodak camera equipped with an 18 mm objective, previously calibrated in the lab. After the automated tie point extraction (Remondino and Ressl, 2006) and the successive bundle adjustment to retrieve the exterior orientation parameters, a dense and detailed surface model was reconstructed with the image matcher presented in (Remondino and Zhang, 2006). The average image footprint is ca 0.5 mm and approximately 2M points were recovered, converted into a mesh and afterwards textured for a photo-realistic visualization of the virtual 3D model. As most of the acquisitions are done in non-controlled environment (i.e. not in a lab), often the 3D models that we want to compare are in different reference systems and possibly with a different scale. Therefore, having at our disposal range data as ‘ground truth’, to asses the discrepancies between range- and image-based approaches, we performed the following operations: - registration of the models: it gives already a global estimation of the deviation between the two approaches (for example in terms of sigma0 or standard deviation); - graphical visualization of the deviations (error map) between the two model. Figure 5 shows an ornament located in the courtyard of the Buonconsiglio Castle, Trento, Italy. In this case, for the 3D modeling, five images were acquired with a 13,5 Mega pixel SRL Kodak camera equipped with 135 mm objective. The average footprint of the images is ca 1 mm while the diameter of the detail is ca 1.6 m. The image matcher (Remondino and Zhang, 2006) produced a surface model with almost 3M points, afterward textured for photo-realistic visualization. For the accuracy evaluation of the ornament model, a Time-of-Flight (TOF) laser scanner (Leica HDS 3000, 6 mm positional accuracy at 50m) has been used as ground truth. The distance between the laser scanner and the object during the scanning was about 20 m. To compare the photogrammetric models with the ground truth data, we used PolyWorks Inspector, Rapid Form and Geomagic Studio software. The derived error colour-coded maps of the distances between the models present all the same behaviour and also the statistics confirmed a standard deviation between the two techniques of ca 3 mm.

Rizzi, Voltolini, et al.

21

Figure 4. Small heritage object located in the Buonconsiglio castle (Trento, Italy) modeled with photogrammetry using 6 images. Shaded and textured results are presented.

Figure 5: 3D model of a relief generated with a matching procedure on 5 images, visualized in shaded and textured mode (above and middle images). 3D comparison results (below) between the photogrammetric model and the same object digitized with a TOF laser scanner: using three different modeling packages, the same behaviour is reported as well as the computed standard deviation (ca 3 mm).

Cultural Heritage 22

In Cultural Heritage documentation, in particular for artworks and frescos, the visible information are generally not enough and it is often necessary to join extra information provided by other sensors working in different spectral bands (e.g. IR, UV) in order to study deeper the object. For example, in case of paintings, IR and UV sensors are able to reveal unexpected information (Pelagotti et al., 2007). For building façades, IR thermography can provide a state of conservation of the walls or reveal padding and older layers (Moropoulo et al., 2001). Some examples of additional information obtained using an infrared camera on the Buonconsiglio castle façades are reported in the next figure.

Figure 6: IR images of parts of the Buonconsiglio Castle (Trento, Italy). The colours maps represent the temperature of the walls revealing information of possible padding or old layers.

5 CONSIDERATIONS AND CONCLUSIONS

In this contribution we reported optical measurement techniques generally used for the documentation of cultural heritage. In the terrestrial case, we have at our disposal a great amount of technologies and methodologies to acquire data, ranging from off-the-shelf digital cameras to expensive active sensors. With a thermal camera it’s possible to document the details that are generally invisible to standard CCD/CMOS cameras. This is really useful in the analysis of cultural heritages and it allows the historians to study deeper buildings, their history and state of conservation and restoration. When comparing photogrammetry and active sensors, the former is generally well suited to record edges and textures, whereas the latter is usually better suited to record surfaces. In recent years, both techniques have been successfully combined in a variety of applications, allowing the use of the inherent strength of both approaches. Nevertheless, as reported in the examples and in the 3D comparison results, recent developments in image matching have demonstrated the potentiality of photogrammetry to derive, from a relatively small number of images, with very little interaction and quite accurate results, all the fine details of an object with results very similar to active sensors, but in a fraction of time costs. Therefore we can safely say that with the appropriate surface modeler algorithm, there are no differences between image- or range-based approaches and that the aspect that is no more decisive in the choice of the modeling technique is the accuracy and details of the final 3D model, at least in most of the terrestrial applications. Typical factors still involved in the choice of the technology are the costs, ease of use, portability and usability. Nice-looking 3D models (e.g. derived with ‘structure from motion’ approaches) are not of great interest for precise and detailed heritage documentations, but mainly useable in quick visualization applications. Indeed 3D models without fine geometric details and surface irregularities, even if presented with texture details, will exhibit smoothness of the surface, flat-looking and not recovered details Therefore the selection of the documentation and modeling technique should be performed considering all the variables of a project, e.g. budget, location constraints, time needed for the acquisition, final goal of the model, etc.

Rizzi, Voltolini, et al.

23

REFERENCES

1. Akca, D., Remondino, F., Novàk, D., Hanusch, T., Schrotter, G., and Gruen, A., 2006: Recording and modeling of cultural heritage objects with coded structured light projection systems. 2nd International Conference on Remote Sensing in Archaeology, pp.375-382, Rome, Italy, December 4-7

2. Bernardini, F., Rushmeier, H., 2002: The 3D Model Acquisition Pipeline. Computer Graphics Forum, 21(2):149-172

3. Blais, F., 2004: A review of 20 years of Range Sensors Development. J. of Electronic Imaging, 13(1): 231-240

4. Böhler, W. and Marbs, A., 2004. 3D scanning and photogrammetry for heritage recording: a comparison. Proceedings of 12th Int. Conf. on Geoinformatics, 7-9 June 2004, Gävle, Sweden, 291-298

5. Böhler, W., 2005: Comparison of 3D laser scanning and other 3D measurement techniques. Recording, Modelling and Visualisation of Cultural Heritage, Baltsavias et al. (Eds), Taylor & Francis Group, pp. 89-99

6. Chandler, J., Bryan, P., Freyer, J., 2007: The development and application of a simple methodology for recording rock art using consumer-grade digital cameras. Photogrammetric Record, 22(117), pp.10-21

7. Cronk, S., Fraser, C., Hanley, H., 2006: Automatic metric calibration of colour digital cameras. Photogrammetric Record, 21(116), pp. 355-372

8. Debevec, P.E., Taylor, C.J. and J. Malik, 1996: Modelling and rendering architecture from photographs: a hybrid geometry and image-based approach. ACM Proceedings of SIGGRAPH ’96, pp. 11-20

9. El-Hakim, S.F., 2002: Semi-automatic 3D reconstruction of occluded and unmarked surfaces from widely separated views. IAPRS&SIS, Vol. 34(5), pp. 143-148, Corfu, Greece

10. El-Hakim, S.F., Gonzo, L., Picard, M., Girardi, S., Simoni, A., Paquet, E., Viktor, H., Brenner C., 2003: Visualisation of highly textured surfaces. 4th Int. Symp. VAST2003, pp. 231-240, Brighton, UK.

11. El-Hakim, S.F., Beraldin, J.A., Picard, M. and Godin, G., 2004: Detailed 3D reconstruction of large-scale heritage sites with integrated techniques. IEEE Computer Graphics and Application, 24(3): 21-29

12. El-Hakim, S., 2006: A sequential approach to capture fine geometric details from images. IAPRS&SIS, Vol. 36(5), pp. 97-102, Dresden, Germany

13. Ganci, G., Hanley, H., 1998: Automation in videogrammetry. IAPRS, 32(5), pp.53-58 14. Grün, A., 2000: Semi-automated approaches to site recording and modelling. IAPRS&SIS,

33(5/1): 309-318. 15. Grün, A., Remondino, F. and Zhang, L., 2004: Photogrammetric Reconstruction of the

Great Buddha of Bamiyan, Afghanistan. Photogrammetric Record, 19(107): 177-199 16. Kadobayashi, R., Kochi, N., Otani, H. and Furukawa, R., 2004: Comparison and evaluation

of laser scanning and Photogrammetry and their Combined Use for Digital Recording of Cultural Heritage. IAPRS&SIS, 35(5): 401-406, Istanbul, Turkey

17. Lambers, K. Eisenbeiss, H., Sauerbier, M., Kupferschmidt, D., Gaisecker, T., Sotoodeh, S., Hanusch, T., 2007: Combining photogrammetry and laser scanning for the recording and modelling of the Late Intermediate Period site of Pinchango Alto, Palpa, Peru. Journal of Archaeological Science 34 (in press)

18. Moropoulo, A., Avdelidis, N., Koui, M., Delegou, E., Tsiourva, T., 2001: Infrared thermographic assessment of materials and techniques for the protection of cultural heritage. Multispectral and Hyperspectral Image Acquisition and Processing, SPIE Vol. 4548

19. Pelagotti, A., Del Mastio, A., Razionale, A., 2007: Active and passive sensors for art works analysis and investigations. Videometrics IX, Proc. SPIE-IS&T Electronic Imaging, Vol. 6491

Cultural Heritage 24

20. Pollefeys, M., Van Gool, L., Vergauwen, M., Verbiest, F., Cornelis, K., Tops, J. and Koch, R., 2004: Visual modelling with a hand-held camera. International Journal of Computer Vision, 59(3): 207-232.

21. Remondino, F., Guarnieri, A. and Vettore, 2005. 3D modelling of close-range objects: photogrammetry or laser scanning? Videometrics VIII, Proc. SPIE-IS&T Electronic Imaging, Vol. 5665, pp. 216-225

22. Remondino, F., El-Hakim, S., 2006: Image-based 3D modelling: a review. Photogrammetric Record, 21(115), pp. 269-291

23. Remondino, F., Fraser, C., 2006: Digital Camera Calibration Methods: Considerations and Comparisons. IAPRS&SIS, Vol. 36(5), pp. 266-272, Dresden, Germany

24. Remondino, F., Ressl, C., 2006: Overview and experience in automated markerless image orientation. IAPRS&SIS, Vol. 36(3), pp. 248-254, Bonn, Germany

25. Remondino, F., Zhang, L., 2006: Surface reconstruction algorithms for detailed close-range object modeling. IAPRS&SIS, Vol. 36(3), pp. 117-121, Bonn, Germany

26. Remondino, F., 2007: Image-based detailed 3D geometric reconstruction of heritage objects. SGPBF Symposium (joint meeting with DGPF and OVG), Basel, Switzerland (in press)

27. Visnovcova (Niederöst), J., Zhang, L. and Grün, A., 2001: Generating a 3D model of a Bayon tower using non-metric imagery. IAPRS, 34(5/W1):30-39, Ayutthaya, Thailand, on CD-ROM

28. Voltolini, F., Remondino, F., Pontin, M., Gonzo, L., 2006: Experiences and considerations in image-based-modeling of complex architectures. IAPRS&SIS, Vol. 36(5), pp. 309-314

29. Voltolini, F., El-Hakim, S., Remondino, F., Girardi, S., Rizzi, A., Pontin, M., and Gonzo, L., 2007: Digital Documentation of complex architectures by integration of multiple techniques – The case study of Valer Castle. Videometrics IX, Proc. SPIE-IS&T Electronic Imaging, Vol. 6491