AIIM Ottawa presentation by Ron Surette, DG DIgital Preservation and CIO Library and Archives Canada
Citation preview
Digital Preservation…a wicked problem
Ronald SuretteDG Digital Preservation and CIOLibrary and Archives [email protected]
Outline
Digital Preservation Key Questions Integrity and Authenticity LAC Strategy Whole of Society Model State of the DP world Conclusion
2
Presenter
Presentation Notes
Going to provide some initial thoughts about whether digital is different Then explore some key questions that we have been thinking about Will make some observations about the nature of integrity and authenticity in a digital world Next, I will explore where all of the above takes us in terms of LAC’s current approach to digital preservation And I will put that in the context of our broader approach to acquisiton, preservation and access – which we call a “Whole of Society Model” And that should be enough Highlight that we see this as a rapidly moving area in which change is constant so we expect our thinking to adapt and evolve.
Wicked problem• Originally coined by Professors Rittel and Webber in a seminal paper in 1973. • From Wikipedia
– A problem that is difficult or impossible to solve because of incomplete, contradictory, and changing requirements that are often difficult to recognize. Moreover, because of complex interdependencies, the effort to solve one aspect of a wicked problem may reveal or create other problems.
• They are complex, ever changing problems that you haven’t been able to treat with much success (or “solve” in the traditional sense of the word), because they won’t keep still. They are “messy”.
• They cannot be solved in a traditional linear fashion (as you would a “tame” problem) as the problem definition itself evolves as new possible solutions are considered and/or implemented.
• We just have to learn to live with them and manage them. There is no “SOLUTION”• Digital preservation fits that description and, importantly, some of the approaches to “wicked
problems” become useful as strategies for Digital Preservation as I hope you will see later in the presentation.
3
There is no solution to a wicked problem
We just have to learn to live with them and manage them. Evolve with time…
But you can’t SOLVE them
4
Digital is different than analog LAC has three core business functions Acquisition Preservation Access
Across three primary sources: Government records via Records Disposition Authorities (RDAs) Published material via legal deposit/purchases Private records via donations/purchases
IN ALL BUSINESS LINES AND ACROSS ALL SOURCES, DIGITAL IS DIFFERENT
5Digital is different
Presenter
Presentation Notes
We believe digital is different, no matter which parts of our holdings or which of our principal activities we are thinking about.
A Word from Daniel Caron,Librarian and Archivist of Canada
“. . . the face of information has changed substantially in the last decade: superabundance; rapid creation, sharing and remixing by individuals; multiple formats; unprecedented access; ever-present and expanding user influence, points of view, skills and engagement. This picture is in direct contrast to that of the past, which was characterised by limited creation and quantity; mediated access and decisions; authoritative sources; specialist interventions; limited number of fixed formats; limited sharing; and fewer players.”
– Daniel J. Caron, Shaping our Continuing Memory Collectively: A Representative Documentary Heritage
6Digital is different
Presenter
Presentation Notes
The Librarian and Archivist of Canada has been very active at IFLA, at ICA, and in many other fora, to confirm and describe our view that digital is different.
A “Wicked Problem”: Attributes of the challenge
Profound Digital objects are now the dominant way of creating, managing,
exchanging and accessing information Changing architecture of social and business processes Shifting institutional structures, boundaries and relational
configurations
Complex Content: distributed network-based digital objects, the
emergence of shared digital stewardship Technical: a shape-shifting, ever-changing landscape dominated
by persistent uncertainty Social: emergence of new actors, new and more complex social
and economic dynamics
7Digital is different
Presenter
Presentation Notes
The term “wicked problem” was originally coined by Professors Rittel and Webber in a seminal paper in 1973. According to Wikipedia, a "wicked problem" is a phrase originally used in social planning to describe a problem that is difficult or impossible to solve because of incomplete, contradictory, and changing requirements that are often difficult to recognize. Moreover, because of complex interdependencies, the effort to solve one aspect of a wicked problem may reveal or create other problems. Typically, the wicked problems are those complex, ever changing societal and organizational policy and planning problems that you haven’t been able to treat with much success (or “solve” in the traditional sense of the word), because they won’t keep still. They are messy, devious, and they fight back when you try to “solve” them. So you just have to learn to live with them because they cannot be solved in a traditional linear fashion (as you would a “tame” problem) as the problem definition itself evolves as new possible solutions are considered and/or implemented. In our view the challenge of digital preservation fits that description and, importantly, some of the approaches to “wicked problems” become useful as strategies for Digital Preservation as I hope you will see later in the presentation.
A “Wicked Problem”: Attributes of the challenge . . . Cont’d Distributed Increasingly distributed, network-based organization of digital
information and its life-cycle management Shared among a broad range of societal actors in all sectors of
society; global, trans-national in scope Requires national and trans-national frameworks and
governance
“Wicked” Continuous uncertainty and shifting solution domains Traditional assumptions and approaches to problem resolution
do not work Requires new thinking, innovation, continuous experimentation;
new skills and governance
8Digital is different
Digital Preservation: 4 Key Questions
1. In what ways is digital preservation NOT the same as analog preservation? Digital objects are less fixed (photograph vs website; book vs
blog) Digital objects will be acquired based on analysis of value not
evidence of value: more chaff with the wheat: continual re-appraisal?
Total cost of ownership shifts: digital costs more to preserve than to get – more interventions required
All of the above are true in both migration and emulation scenarios
IF THE PROBLEMS ARE NOT THE SAME, WHY WOULD WE EXPECT THE SOLUTIONS TO BE?
9Key Questions
Presenter
Presentation Notes
Our efforts to respond to the “wicked” nature of digital presentation have led us to explore a number of key questions and what follows are some of our important insights to date from that exploration. I am sure that many of you will have thought about these issues, none of them are unique to LAC, and we would welcome your analysis and advice. Capturing digital assets “in case” they are important because they will disappear. Continue to review their relevance as time goes on and deleting irrelevant information captured “in case” Create models that would support a quantitative analysis of the assets and liabilities we hand off to the next generation One of the emerging distinctions between digital and analogue material in our minds is that digital objects are increasingly becoming “dynamically generated” and fixed documents that exist in digital format. I am not talking here about the ease of editing and changing a digital object, I am reflecting that “webpages” are no longer “pages”, stored as such and served up the user as a whole. “Web views” are rather a set of components, visual, structural, data, that exist entirely separately from each other and only exist together when a user calls for a particular view. How to acquire and preserve such dynamic content? Costs associated with the acquisition, maintenance, and providing access to assets are very different than those of physical assets. The decision to acquire and the decision to dispose of digital assets must be made using a value model based on the value/cost ratio. In most cases The initial costs of acquiring and storing of digital assets are lower than physical assets. The costs of maintaining usable copies of digital assets may become significant over time. The costs of disposing digital assets is low. The costs of maintaining digital assets is not linear but increases in quantum leaps when technology changes occur. Digital assets may be very short lived and may need to be acquired on their projected or potential value The use of specific review process to test the value/cost ration for disposition is required. This often occurs when significant technology investments are required.
Digital Preservation: 4 Key Questions
2. Should we take a “generational” approach to digital preservation? Continual refresh of tools and practices: a
long-term solution = short-term solution + next short-term solution + next short-term solution . . . . 10 years at a time? Requires new approach to services (not
systems) development
10Key Questions
Presenter
Presentation Notes
So, here is one solution that comes to us from “wicked” thinking. If permanent solutions are not possible then can we create the effect of permanance by linking together a continual set of short-term solutions? Our view is that doing so will require us to change how we plan, build and maintain our technology tools for digital preservation. Without delving into the depths of this, it is for us a very significant shift in how we work.
Digital Preservation: 4 Key Questions
3. Should we develop and use TDR models that support “levels” of preservation, accountability, and metadata requirements? Specialized preservation; common access Shift metadata focus to information “about-what-the-
item-is-about” not “about-the-item” Store and access metadata in structures that
support digital discovery
11Key Questions
Presenter
Presentation Notes
We began our digital preservation journey trying to apply a single, unified approach to all of our digital holdings. This was driven by two key concepts: That because its all bits and bytes, an equal degree of preservation could and should be applied to all digital content; To improve access we wanted to be able to ensure equal capability to discover material in our holdings – and to make them equally available to users subject to the applicable intellectual property rights regimes. We now believe that we can in fact mirror some long-standing practices in the analogue world. Digital may be different but there are still ideas and approaches that work across the two! So, just as we provide different levels of preservation to different parts of our physical holdings – a simple example would be differences in temperature and relative humidity when storing different formats – we are now planning on having specialized digital preservation repositories to reflect difference in digital content. Feature films and other audio-visual projects would be good examples here. But we remain fixed about the second concept – a common access platform. And that takes us to ideas about new approaches to metadata – both in terms of what metadata we need to support access and how best to house and utilize that data to enable discovery. More on this later.
Digital Preservation: 4 Key Questions
4. How could we implement and manage a networked operationalization of the first three concepts? Appraisal decisions before-or-at the point of
creation Sharing information on what’s being collected
and why Transfer ownership and control over bits and
bytes rather than transfer the digital objects themselves
12Key Questions
Presenter
Presentation Notes
As with our challenges with respect to acquisition and access, we have decided that we cannot be successful in digital preservation by trying to do it alone. The task is too big, too complex and too subject to change. We must, in our view, work together with documentary heritage institutions, with other information-rich elements of our society, and with those who create and use the holdings that we must preserve. In a world of super-abundance of information, the decisions on what to preserve need to made by the many and not just the few and need to be made transparently. And in a break from the analogue notion of physically safeguarding items within the walls of our institution, we may need to take ownership and control over digital items that remain outside of our technical infrastructure. Can this work and if so how? Government documents provide a good illustration.
Analogue meanings: Tied to concept of
“original”: unique version of an item
Risk of alteration or alternate versions
Demonstrated via provenance and “chain of control”
Digital meanings: Original no longer exists
– email/tweet Reduced risk to alteration
of “initial” version; access is to copies
Multiple copies allows validation
Demonstrated via provenance and chain of control
Integrity and Authenticity
13Integrity and Authenticity
Presenter
Presentation Notes
One of the key concepts behind libraries and archives is that we provide not just access to content but access to a particular type of content. Content that has a degree of reliability as to its authenticity and in some cases originality. Others have begun to explore and write about how these important concepts link across the digital divide. For our purposes, there are a number of key shifts that we see happening.
Integrity and Authenticity
Thoughts, insights, issues, actions Chain of control requires documenting
changes . . . Same as analogue Demonstrate this chain of control for digital
items at three levels: “l’objet physique, l’objetlogique, et l’objet conceptuel” (Frey) Reformatting/migrating the object versus
emulating the device
14Integrity and Authenticity
Presenter
Presentation Notes
First of all, 3 key principles/ideas: 1. We can assert the "authenticity" of a item of documentary heritage when we can demonstrate a clear management of the documentary heritage from its creation to the present day. "Management" does not imply unaltered. The ink-blot version of the proclamation of the constitution is an authentic document but certainly one very much altered since its creation. In fact, "management" implies/accepts a degree of alteration as inevitable (although I am sure that some exceptions can be found, I believe that they will be exceptions. A hundred-year old government record is not the same as the record the day it was created (see: The Declaration of Independence). We treat and restore works of art, films, etc without their losing authenticity. 2. LAC's use of the term authenticity does not relate to the veracity of the information contained/captured/represented by the documentary heritage. BUT, we cannot walk away from this completely - our task is to select and acquire documentary heritage that documents the Canadian experience. In deciding which material is "significant" we are making some judgement, perhaps implicitly, about the veracity of the information itself. We would not, I believe, keep a copy of the 1861 Census if we did not have some degree of confidence in its accuracy. 3. Authenticity is, for LAC, fundamentally a question of process control rather than preventing change. To return to point 1, if we can demonstrate that our processes for storing and treating (ie changing) maintain the essential character and contents of the documentary heritage over time, then we can declare it authentic. We can keep some items, particularly digital ones, completely unaltered but in doing so we may lose the ability to authenticate them if they can no longer be viewed, heard, etc.
The OAIS Model
15
OAIS Model Digital?
16
Receiving Shipping
Digital Preservation Tenants: A Re-cap
1. Digital archives are significantly different from physical archives with respect to acquisition, preservation and discovery.
2. Digital archives are assets held in trust for use by our citizens: they have measurable values and costs that change with time. The efforts associated to preserving digital assets should be proportional to their value.
3. Digital archives need to be preserved for an extended period but the preservation technology and strategy will be achieved one generation at a time.
4. Because of their unstable nature, digital assets must be captured close to creation and should be disposed of later if their value is not sustained during reviews.
5. Digital archives need to be federated using a shared data model.
17LAC Strategy
Presenter
Presentation Notes
So, to try and pull the threads of ideas from the previous pages, we arrive at a number of digital preservation tenants. This is our current list and we will remain absolutely, fixed, firm and focused about them until we decide they need to change. And they will need to change. But until then, they will guide us forward. Don’t hesitate to let me know if you think they need to change now.
LAC Strategy
Describes our current, emergent approach Consider it one way not the only way Expect continual change and refinement Open to challenge, confirmation and
change Welcome dialogue and discussion on all of
these issues What is missing? Alternative approaches?
18LAC Strategy
Presenter
Presentation Notes
Drilling down from those we come to our strategy for digital preservation. And again, I will emphasize that it is based on a “learn-as-we-go” approach that underlines the importance of constantly assessing and re-assessing our plans in light of experience and new knowledge.
LAC Strategy
One solution does not fit . . . (. . . Any/All) The only durable solutions are those that can be
Current thinking suggests that digital continuity will require a continual migration of digital documentary heritage to new formats. It is impossible to know what migration technologies will exist 20, 50 and 100 years from now. But they won’t be the ones that exist today. Digital continuity cannot be assured by building a system today that will migrate and maintain digital items in the same way that we can build an analogue preservation centre and know that it will safeguard material for generations to come. Rather, our digital continuity strategy must be built around the capability for change; our ability to constantly innovate and change the way in which we migrate digital documentary heritage to not-yet-developed formats. We must make such change commonplace: affordable, easy to implement and low-risk. This calls for a dis-aggregated approach to systems; the creation of a loosely-coupled set of application services and tools any one of which can be easily replaced without impacting on the entire set of services. “Replacability” must become the first criteria for any preservation functionality. We can provide assurance of this capability for change in two ways over the short-term. One, in the manner in which we conceive and design the digital preservation “toolset” – the services that we will need on an on-going basis – such that they are as changeable and adaptable as possible and do not create obstacles to the integration of new tools and services. Two, by defining the specifications of a successful migration – what outcome must a successful migration of digital document heritage produce? Such specifications will cover the desired authenticity and reliability of the migrated documentary heritage and the evidentiary trail required to demonstrate them. This second step will provide an assurance that we can make sound preservation choices in the future because we have defined clearly the digital continuity outcome we want. And the first step will provide assurance that we are capable of implementing those choices on a continual basis that is affordable, predictable and manageable.
LAC Strategy . . . More details
Use different repositories for different kinds of digital objects Have built a repository to meet our most stringent
requirements. Examining commercial products to hold specialized
records with different requirements With a common integration layer to support
control, discovery and access
20Social Data Model
LAC Strategy . . . More details
All objects, including access versions, held in the TDRs (DAMs)
Moving to digital service delivery model where access to LAC holdings is via digital version regardless of original format
Digital objects served from TDRs to the user via common search layer and according to the applicable rights management regime (access is legislation/policy driven not technology-driven)
21LAC Strategy
Agile Digital Preservation Strategies
22
Preservation Requirements
Met
adat
a R
equi
rem
ents
Low
Hig
hLo
w
High
High descriptionLow preservation
High descriptionHigh preservation
Low descriptionLow preservation
Low descriptionHigh preservation
Current Strategy
TDR/DAM Architecture
23
LAC is using a whole of society approach for the selection of items to be acquired.
This approach uses a domains social model to focus our attention on areas of importance within Canadian society, using the concept of fundamental discourses. (More at: http)
The Whole of Society Data Model is a semantic representation the domains social model and is used for the management of the metadata. This data model can also be shared with other documentary heritage organizations (DHOs), thereby significantly enhancing the “findability” experience across DHOs.
Whole of Society Model: “About-ness”
24Whole of Society Model
The model uses several dimensions of facets:
People: Relevant people and their attributes including biographies
Artists, Authors, Politicians, Celebrities Terms: Roles that link people to organizations for a time
period Prime Minister, President, CEO, Owner, Member
Organizations Political, Economic, Government, Social entities
Eras Formal: centuries, decades, years Informal: related to events (Depression, Victorian,
Edwardian) Social domains Health, Military, Science, Environment
Whole of Society Model: Facets (cont’d)
26Social Data Model
The Whole of Society Model metadata is used to describe the assets and is used to compliment and extend provenance and preservation metadata
The Whole of Society Model metadata is used to provide the multifaceted discovery function that is difficult to achieve with traditional metadata structures. The relationship between assets and the model provides the new “finding aids”
These relationships need to be captured in new data structures and tools
Whole of Society Model
27Whole of Society Model
• List of Digital Preservation Initiatives– http://en.wikipedia.org/wiki/List_of_digital_preservation_initiatives
– Planets – European Community– NDIIP – Library of Congress– DPE – Digital Preservation Europe– CASPAR – (Cultural, Artistic and Scientific
knowledge for Preservation, Access and Retrieval– NARA – National Archives and Record
Assurance of long-term digital preservation derives from LAC’s change capability; meaning, its ability to successfully and consistently implement change on a frequent, rapid and effective basis.
This means digital is different from analogue The way forward will be highlighted by
ongoing learning; we welcome input and comment to ensure that we learn from others
Conclusion
29Conclusion
The preceding pages describe Library and Archives Canada’s current understanding of, and strategies and plans for, digital preservation, a rapidly changing field. LAC expects its way forward to continually evolve and welcomes perspectives, challenges, and comments that will help it identify and implement necessary revisions to its approach.
