Pulling it all together…
with thanks to Sheila Anderson
Pulling it all together
Think of it as a continuum…………………..
2
Su b m is s io n
3 4
R e vis io n (s )
Qu a lity As s e s s m e n ta n d Pu b lica tio n
6
Te ch n ica lOb s o le s ce n ce
Res ourc e Dis c ov ery MetadataTec hnic al MetadataRights MetadataFile Format Conv ers ionUnique, Pers is tent Identif ierV ers ion Contro l
Migration, EmulationOther Pres erv ationA c tion
C re a tio n
1
File Format & ContentTy pes Determined
5
R e vie wR e te n tio n
4
7
W ith d ra w
The OAIS Model
Vocabulary for Digital Repository practitioners
Sub-divides a complex task into related functions
Provides logic and structure to allow digital holdings to be managed and processed
Six Primary Functions
IngestArchival StoreAccessData ManagementAdministrationPreservation Planning
OAIS Functional Model
Some thoughts. . .
OAIS is a conceptual reference model and not a system design modelFunctions in the OAIS Reference Model may not necessarily correspond with the functional modules of a system that would implement the model. (Organisational needs vary!)Functionality of an actual digital repository may be much more limited than in the OAIS Reference Model, and may include additionsTake from it only what you need to manage your digital collections
Process and Activities
Business and strategic planning, including costs and fund raising
Resource creation (not included in OAIS model)Resource preparation and enhancement (not included in OAIS)Resource presentation: access and delivery (Access) Resource curation / Data Management (Ingest)Preservation planning (Preservation Planning)Archival storage (Archival Store)
Management and administration (Admin + bits of data management)
Process and Activities
Strategic and business planning; fund raisingWhat is the purpose of your data creation efforts?Are you undertaking a one-off project or establishing something longer termIf the latter is it short, medium or long-term?Will you be providing a service for others to use?What skills will you needWhat infrastructure? What will it cost?How will you raise the money?Do you want to do everything in-house or outsource some or all of the tasks?
Process and Activities
Resource creation: the methods used to convert analogue information into raw digital data
ScanningDigital photographyTranscriptionOCREtc. etc.
Process and Activities
Resource preparation and enhancement: methods used to transform raw data into information
Modeling and design of the resource’s information structureNormalisation and enhancement of data StructuringEncodingManipulationAdding and creating metadata
Process and Activities
Resource presentation: providing access, delivery and use of the resource
Static or interactive web-based services; Off-line servicesResource discoveryOn-line browsing and selectionRequests from users Coordinates requests and execution of requests Applies controls to limit access as neededGenerates information for disseminationDelivers information (data) to users
Process and Activities
Resource curation: management of the resource over the long-term
Maintaining schema, view definitions, referential integrityReceiving/managing updates & additions to data & metadataVersion ControlPerforming quality assuranceGenerating archive versions to archive’s data formatting and documentation standardsExtracting descriptive information for data management and to support search and retrieval functionsCoordinating updates and transfer of the data and metadata to appropriate places
Process and Activities
Preservation planningEvaluating contents of archive; periodically recommending updates to migrate current holdingsDeveloping and maintaining recommendations for archive standards and policiesMonitoring changes in technology environment, users’ service requests, and knowledge baseDeveloping and maintaining detailed migration plans, software prototypes and test plans
Process and Activities
Archival storageReceiving data from curation process and adding to permanent storageManaging storage hierarchyRefreshing/replacing mediaPerforming error checksProviding disaster recovery capabilitiesDuplicating contents for off-site storage
Process and Activities
Management and administrationNegotiating agreements and licencesManaging IPR, copyright issuesManaging quality control and audit processMonitoring systems operationsProviding inventories, reports, updatesEstablishing and maintaining archive standards and policiesProviding user support
Requirements
OAIS-Type Repository
CreationPreparation
Access Curation
Preservation Planning
Archival store
Management and Administration
Money
Skills
Strategic Business Planning
Infrastructure
Infrastructure
SpaceEquipmentFacilitiesProject Management
Systems and toolsSoftware and hardwareStandards – technical, metadata etc.
Skills
Scholars / Curators / digital librarians (creation, acquisition and curation)Scanner operatives, photographers etc. etc.Information professionals (metadata management & creation)Computer scientists (systems design and development, management tools)Preservation professionals (preservation planning and management, migration and emulation services)Systems administrators (systems management, disc partitioning, refreshment, back-up)Managers and administrators: project, process, strategic and financial etc.
Costs
Staff salaries and benefits
Operating expenses
Software
System equipment
Other?
Solutions: In-house Repository
“….is a set of services that a organisation or institution offers to the members of its community for the creation, management and dissemination of digital materials created by the organisation or institution and its community members. It is most essentially an organisational commitment to the stewardship of these digital materials, including long-term preservation where appropriate, as well as organisation and access or distribution.”
Lynch, C., ARL Bimonthly Report 226, http://www.arl.org/newsltr/226/ir.htm
A very long quote……
As with computing, the cost of data repositories (done correctly) will be dominated by the recurring costs of personnel performing curation, maintenance and upgrade,and providing user advice, assistance, and support. The most sophisticated of these personnel need professional skills in the relevant aspects of information management and information technology (e.g. databases, archival file systems, building portals),and will be developing and maintaining custom software. By using a combination of high-speed networks and local high-speed caches, there is no hard requirement to co-locate professional staff with physical storage particularly staff performing data acquisition and curation functions as opposed to disk partitioning, regeneration, and backup functions. As with computing, there is need for support personnel at local institutions, in discipline-specific groups (often located in centers),and centralized in centers. Although further analysis is needed, we expect that the most efficient approach will be to have relatively centralized storage hardware (with supporting staff) but distributed data acquisition and curation personnel.
http://www.communitytechnology.org/nsf_ci_report/report.pdf (page 77)
Solutions: Sharing; Outsourcing
Sharing infrastructure and costsInstitutional consortiaRegional consortiaLocal/national alliances
Federated servicesLocal/national/regional
Outsourcing some or all of the tasksCreationPreservation
Example: Shared Archives Model
Institutional Repository
Accessions institutional collections and provides access and delivery services
Metadata integrated into preservation metadata store and on-line catalogue facilities
Preservation System
Harvest preservation (and discovery) metadata
Capture preservation copies
Users
Preservation ServicesRisk assessment
process
Preservation Service
Benefits and Challenges
BenefitsShared infrastructureShared expertiseEffective use of resources – human, technical and financialShared costsExperts doing what they are best at
ChallengesCreating partnerships (partnerships or outsourcing?)Agreement on policy and proceduresEstablishing relationships of trustConsistency of practice and outputs
Now it’s your turn…….
Recap: Questions to Ask What is the purpose of your data creation efforts?Are you undertaking a one-off project or establishing something longer termIf the latter is it short, medium or long-term?Will you be providing a service for others to use?What skills will you needWhat infrastructure? What will it cost?Do you want to do everything in-house or outsource some or all of the tasks?
Resource creation
Skills?
Infrastructure?
Likely costs (high, medium, low)?
Possible solutions (in-house, outsource, other)?
Other issues?
Resource enhancement
Skills?
Infrastructure?
Likely costs (high, medium, low)?
Possible solutions (in-house, outsource, other)?
Other issues?
Resource presentation
Skills?
Infrastructure?
Likely costs (high, medium, low)?
Possible solutions (in-house, outsource, other)?
Other issues?
Resource curation
Skills?
Infrastructure?
Likely costs (high, medium, low)?
Possible solutions (in-house, outsource, other)?
Other issues?
Preservation planning
Skills?
Infrastructure?
Likely costs (high, medium, low)?
Possible solutions (in-house, outsource, other)?
Other issues?
Archival storage
Skills?
Infrastructure?
Likely costs (high, medium, low)?
Possible solutions (in-house, outsource, other)?
Other issues?
Management and administration
Skills?
Infrastructure?
Likely costs (high, medium, low)?
Possible solutions (in-house, outsource, other)?
Other issues?