Principles and Practices of Robust, Photography-based Digital

  • Published on

  • View

  • Download

Embed Size (px)


  • Principles and Practices of Robust, Photography-based Digital Imaging Techniques for Museums

    Mark Mudge, Carla Schroer, Graeme Earl, Kirk Martinez, Hembo Pagi, Corey Toler-Franklin, Szymon Rusinkiewicz, Gianpaolo Palma, Melvin Wachowiak, Michael Ashley, Neffra Matthews, Tommy Noble, Matteo Dellepiane

    AbstractThis full day tutorial will use lectures and demonstrations from leading researchers and museum practitioners to

    present the principles and practices for robust photography-based digital techniques in museum contexts. The tutorialwill present many examples of existing and cutting-edge uses of photography-based imaging including ReflectanceTransformation Imaging (RTI), Algorithmic Rendering (AR), camera calibration, and methods of imaged-basedgeneration of textured 3D geometry. The tutorial will also explore a framework for

    Leading museums are now adopting the more mature members of this family of robust digital imaging practices. Thesepractices are part of the emerging science known as Computational Photography (CP). The imaging familys commonfeature is the purpose-driven selective extraction of information from sequences of standard digital photographs. Theinformation is extracted from the photographic sequences by computer algorithms. The extracted information is thenintegrated into a new digital representations containing knowledge not present in the original photogs, examined eitheralone or sequentially.

    The tutorial will examine strategies that promote widespread museum adoption of empirical acquisition technologies,generate scientifically reliable digital representations that are born archival, assist this knowledges long-term digitalpreservation, enable its future reuse for novel purposes, aid the physical conservation of the digitally representedmuseum materials, and enable public access and research.

    Keywords: Reflectance transformation imaging, empirical provenance, photogrammetry, non-photorealistic rendering,digital preservation, cultural heritage

    The 11th International Symposium on Virtual reality, Archaeology and Cultural Heritage VAST (2010) pp A. Artusi, M. Joly-Parvex,G. Lucet, A. Ribes, and D. Pitzalis (Editors)

    1. Tutorial Overview

    Today, leading museums are adopting a new family ofrobust digital imaging practices. This imaging familyscommon feature is the purpose-driven selective extraction ofinformation from a sequence of standard digitalphotographs. The information extracted from thephotographic sequence is selected by computer algorithms.The extracted information is then integrated into a newdigital representation containing knowledge not present inthe original photographs, examined either alone orsequentially. These practices are part of the emergingscience known as Computational Photography.

    The algorithms are embedded in software tools that keepthe computer science under the hood and allow the user todrive the tools in service of their customary workingculture. No ongoing assitance from outside digital imagingtechnologists is necessary.

    The imaging family is able to process the information fromthe photographs with only minor user involvement. Thishighly automatic operation permits the writing of a scientificlab notebook chronicling each of the means andcircumstances of the new digital representationsgeneration. This machine readable log permits qualitativeevaluation of the representations reliability and suitabilityfor its original and potential novel purposes both now and inthe future.

    Following international metadata standards, the labnotebook, bundled with the original photographs and thenewly generated representations form a born archivalpackage ready for ingest into the world knowledge base andthe museum-library-archive long-term digital preservationenvironment.

    The following presentations describe the practices ofReflectance Transformation Imaging, AlgorithmicRendering, dense, close range Photogrammetry, semanticknowledge management, long term digital preservation, andthe application of these tools within museums and culturalheritage environments.

    1.1 Sequence of Presentations

    Mark Mudge and Carla Schroer from Cultural HeritageImaging will present an overview of the themes uniting thetutorials presentations. They will explore issues thatinfluence technology adoption decisions and the advantagesthat can be realized when image-based empiricalinformation acquisition is organized in conformance withthe fundamental principles of science. They will also presenta unified photographic data capture strategy that acquires allthe information necessary to enable ReflectanceTransformation Imaging, Algorithmic Rendering andPhotogrammetry.

  • M. Mudge, C. Schroer, et al. / Photography-based Digital Imaging Techniques for Museums

    Graeme Earl, Kirk Martinez, and Hembo Pagi fromSouthampton University will provide a summary of theiruses of reflectance transformation imaging in archaeologicalcontexts. they will also introduce the UK Arts andHumanities Research Council funded ReflectanceTransformation Imaging (RTI) System for AncientDocumentary Artefacts project. The AHRC RTI project is acollaboration with Alan Bowman, Charles Crowther andJacob Dahl at the University of Oxford.

    Corey Toler-Franklin and Szymon Rusinkiewicz fromPrinceton University will discuss Algorithmic Rendering(AR). Their AR work takes photographic image sequencescontaining reflective spheres, such as the RTI data set, andgenerates RGBN images with per-pixel color and surfaceshape information, in the form of surface normals. TheseRGBN images are powerful tools for documenting complexreal-world objects because they are easy to capture at a highresolution, and readily extendible to processing toolsoriginally developed for full 3D models. Most state-of-the-art nonphotorealistic rendering algorithms are simplyfunctions of the surface normal, lighting and viewingdirections. Simple extensions to signal processing tools canpreserve the integrity of the normals, while introducing awide range of control for a variety of stylistic effects. RGBNimages are more efficient to process than full 3D geometry,requiring less storage and computation time. Functions arecomputed in image space producing powerful 3D resultswith simpler 2D methods.

    Gianpaolo Palma from the Visual Computing Lab, fromthe Italian National Research Councils (CNR) Institute forInformation Science and Technology (ISTI) will present twotools to visualize and analyze RTI images in an interactiveway. The first one is a multi-platform viewer, RTIViewer,developed also to work remotely through HTTP, that allowsthe user to apply a set of new shading enhancementtechniques improving the virtual examination andinterpretation of details of the artifact. The second is a webapplication based on SpiderGL [DBPGS10], a JavaScript3D graphics library which relies on WebGL, which permitsthe realtime rendering of huge RTIs with a multi-resolutionencoding in the next generation of web browser.

    Mel Wachowiak from the Smithsonian InstitutionsMuseum Conservation Institute (MCI) will describe somemuseum uses of RTI and its place among photographiccapture and 3D scanning at the Smithsonian Institution (SI).MCI has a central role as a research unit and collaborator inanalysis of heritage objects and sites. MCIs part in thedigitization of collections is to offer an expanded vision ofthe application of appropriate technologies. He will showhow RTI fills a niche that other imaging solutions cant fillby offering an immersive, near 3D experience and imageprocessing tools as well as accurately document features thatare impossible to acquire with 3D scanning. He will alsoshow a broad range of RTI projects. These have ranged insize and scope from tiny natural history specimens to largeartworks, both in the studio and on location. Buttons,

    jewelry, fossils, prehistoric stone tools and many othermaterials will demonstrate the strengths and weaknesses ofthe current RTI technology and software.

    Michael Ashley from Cultural Heritage Imaging willdiscuss and demonstrate practical digital preservationframeworks that protect images throughout the entireproduction life-cycle. Using off the shelf and open sourcesoftware coupled with a basic understanding of metadata, hewill show it is possible to produce and manage high valuedigital representations of physical objects that are bornarchive-ready and long-term sustainable. He will alsodemystify the alphabet soup of file formats, data standards,and parametric imaging, and demonstrate proven workflowsthat can be deployed in any museum productionenvironment, scalable from the individual part time shooterto full fledged imaging departments.

    Neffra Matthews and Tommy Noble from the U.S.Department of the Interior, Bureau of Land Managements,National Operations Center will present the principles ofphotogrammetry, deriving measurements from photographs.They will demonstrate that by following thephotogrammetric fundamentals, mathematically sound andhighly accurate textured 3D geometric results may beachieved. they will also show how technological advancesin digital cameras, computer processors, and computationaltechniques, such as sub-pixel image matching, makephotogrammetry an even more portable and powerful tool.Extremely dense and accurate 3D surface data can becreated with a limited number of photos, equipment, andimage capture time.

    Matteo Dellepiane from the Visual Computing Lab of theItalian National Research Councils (CNR) Institute forInformation Science and Technology (ISTI) will present 2applications. The first is an alternate method for generatingtextured 3D geometry for interpretive purposes using theARC3D web service. The Arc3D web service, inputs useruploaded uncalibrated photographic sequences to generateand then return the 3D model. The second application,Meshlab is and open source tool for processing 3D datafrom a wide variety of 3D scanning and image-basedsources into high quality 3D geometric models.

    The tutorial will also include a live demonstration by MarkMudge and Carla Schroer of the Highlight RTI imageacquisition process along with the capture of a cameracalibration and photogrammetric image sequences.


    2. Integrated Capture Methods for the Generation of Multiple Scientifically Reliable Digital Representations for Museums

    Tutorial Presenters: Mark Mudge, Carla SchroerAdditional Author: Marlin LumCultural Heritage Imaging, USA

  • M. Mudge, C. Schroer, et al. / Photography-based Digital Imaging Techniques for Museums

    Adoption of RTI tools is underway at leading museumsincluding the Smithsonian Institution, the Museum ofModern Art, the Metropolitan Museum, the Fine ArtsMuseums of San Francisco, the Los Angeles CountyMuseum of Art, and the Worcester Art Museum. Thelessons learned by CHI and its collaborators, whichestablished the sufficient conditions for this adoption, canguide the development of emerging technologies and theadaptation of existing technologies to the adoptionrequirements of the museum community and culturalheritage activities generally.

    Figure 1: Unified photo sequence capture of theSennenjem Lintel from the collection of the Phoebe A.Hearst Museum of Anthropology at the University ofCalifornia Berkeley. Data to generate RTIs, ARs, anddense textured 3D geometry was acquired during thesession

    2.1 Factors influencing widespread adoption of digital imaging practices

    CHI and our collaborators have extensively discussed theobstacles to widespread adoption of robust digitaldocumentary technologies by cultural heritage professionalsand the means to remove these obstacles in prior literature.[MMSL06] [MMC*08] [RSM09]. The following materialreviews the central themes of this analysis.

    2.1.1 Ease of use for museum professionals

    Designed from the beginning through intensivecollaboration with cultural heritage practitioners,Reflectance Transformation Imaging (RTI), and relatedemerging technologies such as Algorithmic Rendering (AR)along with its next generation Collaborative AlgorithmicRendering Engine (CARE) tool, are crafted to be compatiblewith current working cultures and digital-imaging skill sets.The goal is to democratize technology and foster widespreadadoption of robust digital documentary methods by greatlyreducing the barriers of cost and technological complexitythat characterize many current 3D methodologies.

    Until recently, adoption of robust digital practices wasslow in museum contexts, largely because many of todays

    legacy digital practices, required museum workers to seekhelp from expensive digital imaging experts, or to learncomplex computer programs themselves. For successfulwidespread adoption, practices must be not requireextensive technical re-education, and must remain withinthe scope of restrictive budgets.

    The key design insight behind Cultural Heritage Imagings(CHIs) international RTI software research developmentcollaborations and now the AR-based emerging CARE toolis that automation of digital processing tasks can put thecomputer-science complexity and genius under the hood,leaving humanities users free to explore in the direction thataccomplishes their primary objectives, using theirknowledge more effectively. This strategy overcomes thethe hard to learn hard to use obstacles to digitaltechnology adoption and greatly enhances the effective useof work and research time among domain experts.

    2.1.2 Scientific reliability

    Over the past eight years, CHI's discussions withnumerous humanities and natural science professionalsrevealed that widespread adoption of digital representationsin all fields, including the multi-disciplinary study ofcultural heritage, requires confidence that the data theyrepresent are reliable. This confidence requires means toqualitatively evaluate the digital representation. For scholarsto use digital representations built by someone else, theyneed to know that what is represented in the digitalsurrogate is truly what is observed on the physical original.If archaeologists are relying on digital representations tostudy Paleolithic stone tools, they must be able to judge thelikelihood that a feature on the representation is also on theoriginal and vice versa. For scholars to adopt widespreaduse of digital representations, they must be able to haveabsolute trust in the representations quality andauthenticity.

    RTIs and the CARE tool are designed to record the sameinformation that a scientist records in a lab notebook or anarchaeologist records in field notes. The RTI and CAREtools are and will be based on digital photography, capableof automatic post-processing and automatic recording ofimage generation process history in a machine readable log.

    Additional software features are under construction. Theywill automatically map this log to a semantically robustinformation architecture. Once the mapping process hasbeen completed, digital processing can automatically recordempirical provenance information into these semanti...


View more >