Principles and Practices of Robust, Photography-based Digital

  • View

  • Download

Embed Size (px)

Text of Principles and Practices of Robust, Photography-based Digital

  • Principles and Practices of Robust, Photography-based Digital Imaging Techniques for Museums

    Mark Mudge, Carla Schroer, Graeme Earl, Kirk Martinez, Hembo Pagi, Corey Toler-Franklin, Szymon Rusinkiewicz, Gianpaolo Palma, Melvin Wachowiak, Michael Ashley, Neffra Matthews, Tommy Noble, Matteo Dellepiane

    AbstractThis full day tutorial will use lectures and demonstrations from leading researchers and museum practitioners to

    present the principles and practices for robust photography-based digital techniques in museum contexts. The tutorialwill present many examples of existing and cutting-edge uses of photography-based imaging including ReflectanceTransformation Imaging (RTI), Algorithmic Rendering (AR), camera calibration, and methods of imaged-basedgeneration of textured 3D geometry. The tutorial will also explore a framework for

    Leading museums are now adopting the more mature members of this family of robust digital imaging practices. Thesepractices are part of the emerging science known as Computational Photography (CP). The imaging familys commonfeature is the purpose-driven selective extraction of information from sequences of standard digital photographs. Theinformation is extracted from the photographic sequences by computer algorithms. The extracted information is thenintegrated into a new digital representations containing knowledge not present in the original photogs, examined eitheralone or sequentially.

    The tutorial will examine strategies that promote widespread museum adoption of empirical acquisition technologies,generate scientifically reliable digital representations that are born archival, assist this knowledges long-term digitalpreservation, enable its future reuse for novel purposes, aid the physical conservation of the digitally representedmuseum materials, and enable public access and research.

    Keywords: Reflectance transformation imaging, empirical provenance, photogrammetry, non-photorealistic rendering,digital preservation, cultural heritage

    The 11th International Symposium on Virtual reality, Archaeology and Cultural Heritage VAST (2010) pp A. Artusi, M. Joly-Parvex,G. Lucet, A. Ribes, and D. Pitzalis (Editors)

    1. Tutorial Overview

    Today, leading museums are adopting a new family ofrobust digital imaging practices. This imaging familyscommon feature is the purpose-driven selective extraction ofinformation from a sequence of standard digitalphotographs. The information extracted from thephotographic sequence is selected by computer algorithms.The extracted information is then integrated into a newdigital representation containing knowledge not present inthe original photographs, examined either alone orsequentially. These practices are part of the emergingscience known as Computational Photography.

    The algorithms are embedded in software tools that keepthe computer science under the hood and allow the user todrive the tools in service of their customary workingculture. No ongoing assitance from outside digital imagingtechnologists is necessary.

    The imaging family is able to process the information fromthe photographs with only minor user involvement. Thishighly automatic operation permits the writing of a scientificlab notebook chronicling each of the means andcircumstances of the new digital representationsgeneration. This machine readable log permits qualitativeevaluation of the representations reliability and suitabilityfor its original and potential novel purposes both now and inthe future.

    Following international metadata standards, the labnotebook, bundled with the original photographs and thenewly generated representations form a born archivalpackage ready for ingest into the world knowledge base andthe museum-library-archive long-term digital preservationenvironment.

    The following presentations describe the practices ofReflectance Transformation Imaging, AlgorithmicRendering, dense, close range Photogrammetry, semanticknowledge management, long term digital preservation, andthe application of these tools within museums and culturalheritage environments.

    1.1 Sequence of Presentations

    Mark Mudge and Carla Schroer from Cultural HeritageImaging will present an overview of the themes uniting thetutorials presentations. They will explore issues thatinfluence technology adoption decisions and the advantagesthat can be realized when image-based empiricalinformation acquisition is organized in conformance withthe fundamental principles of science. They will also presenta unified photographic data capture strategy that acquires allthe information necessary to enable ReflectanceTransformation Imaging, Algorithmic Rendering andPhotogrammetry.

  • M. Mudge, C. Schroer, et al. / Photography-based Digital Imaging Techniques for Museums

    Graeme Earl, Kirk Martinez, and Hembo Pagi fromSouthampton University will provide a summary of theiruses of reflectance transformation imaging in archaeologicalcontexts. they will also introduce the UK Arts andHumanities Research Council funded ReflectanceTransformation Imaging (RTI) System for AncientDocumentary Artefacts project. The AHRC RTI project is acollaboration with Alan Bowman, Charles Crowther andJacob Dahl at the University of Oxford.

    Corey Toler-Franklin and Szymon Rusinkiewicz fromPrinceton University will discuss Algorithmic Rendering(AR). Their AR work takes photographic image sequencescontaining reflective spheres, such as the RTI data set, andgenerates RGBN images with per-pixel color and surfaceshape information, in the form of surface normals. TheseRGBN images are powerful tools for documenting complexreal-world objects because they are easy to capture at a highresolution, and readily extendible to processing toolsoriginally developed for full 3D models. Most state-of-the-art nonphotorealistic rendering algorithms are simplyfunctions of the surface normal, lighting and viewingdirections. Simple extensions to signal processing tools canpreserve the integrity of the normals, while introducing awide range of control for a variety of stylistic effects. RGBNimages are more efficient to process than full 3D geometry,requiring less storage and computation time. Functions arecomputed in image space producing powerful 3D resultswith simpler 2D methods.

    Gianpaolo Palma from the Visual Computing Lab, fromthe Italian National Research Councils (CNR) Institute forInformation Science and Technology (ISTI) will present twotools to visualize and analyze RTI images in an interactiveway. The first one is a multi-platform viewer, RTIViewer,developed also to work remotely through HTTP, that allowsthe user to apply a set of new shading enhancementtechniques improving the virtual examination andinterpretation of details of the artifact. The second is a webapplication based on SpiderGL [DBPGS10], a JavaScript3D graphics library which relies on WebGL, which permitsthe realtime rendering of huge RTIs with a multi-resolutionencoding in the next generation of web browser.

    Mel Wachowiak from the Smithsonian InstitutionsMuseum Conservation Institute (MCI) will describe somemuseum uses of RTI and its place among photographiccapture and 3D scanning at the Smithsonian Institution (SI).MCI has a central role as a research unit and collaborator inanalysis of heritage objects and sites. MCIs part in thedigitization of collections is to offer an expanded vision ofthe application of appropriate technologies. He will showhow RTI fills a niche that other imaging solutions cant fillby offering an immersive, near 3D experience and imageprocessing tools as well as accurately document features thatare impossible to acquire with 3D scanning. He will alsoshow a broad range of RTI projects. These have ranged insize and scope from tiny natural history specimens to largeartworks, both in the studio and on location. Buttons,

    jewelry, fossils, prehistoric stone tools and many othermaterials will demonstrate the strengths and weaknesses ofthe current RTI technology and software.

    Michael Ashley from Cultural Heritage Imaging willdiscuss and demonstrate practical digital preservationframeworks that protect images throughout the entireproduction life-cycle. Using off the shelf and open sourcesoftware coupled with a basic understanding of metadata, hewill show it is possible to produce and manage high valuedigital representations of physical objects that are bornarchive-ready and long-term sustainable. He will alsodemystify the alphabet soup of file formats, data standards,and parametric imaging, and demonstrate proven workflowsthat can be deployed in any museum productionenvironment, scalable from the individual part time shooterto full fledged imaging departments.

    Neffra Matthews and Tommy Noble from the U.S.Department of the Interior, Bureau of Land Managements,National Operations Center will present the principles ofphotogrammetry, deriving measurements from photographs.They will demonstrate that by following thephotogrammetric fundamentals, mathematically sound andhighly accurate textured 3D geometric results may beachieved. they will also show how technological advancesin digital cameras, computer processors, and computationaltechniques, such as sub-pixel image matching, makephotogrammetry an even more portable and powerful tool.Extremely dense and accurate 3D surface data can becreated with a limited number of photos, equipment, andimage capture time.

    Matteo Dellepiane from the Visual Computing Lab of theItalian National Research Councils (CNR) Institute forInformation Science and Technology (ISTI) will present 2applications. The first is an alternate method for generatingtextured 3D geometry for interpretive purposes using theARC3D web service. The Arc3D web service, inputs useruploaded uncalibrated photographic sequences to generateand then return the 3D model. The second application,Mesh