A/V Digitization:How do people really do this?VENDOR BASED DIGITIZATION AND QUALITY CONTROL AT THE ARCHIVES OF THE UNIVERSITY OF WISCONSIN-MILWAUKEE

Charles HosaleA/V Project Archivist

UWM Archives

Background

In 2015 UWM is taking focused actions on our aging audio and audiovisual holdingsCollections containing physical audio/audiovisual media that have high

significance, those that are facing information loss, or those for which special funding sources exist were identified for digitization.

The project necessitated codification of workflows and procedures, to ensure the integrity of the digitizations and efficiency of the project.

We developed a process for ingestion and quality control of digitized files that is based off of current A/V best practices - significantly those of the California Audiovisual Preservation Project (CAVPP).Created a standard for file names, standard forms for inventorying and

ingesting, and a workflow with step by step procedures.

Vendor-based digitization◦ Definition: Digitization of the physical media where the creation of

digital files is not performed by the controlling organization.◦ The controlling organization contracts with digitization vendors, who create

digital surrogates of physical media for the controlling organization.

◦ Reasons for engaging in vendor-based digitization:◦ The organization doesn’t have access to necessary equipment, and

possessing that equipment is not part of the institution’s mission.◦ Digitization requires format or equipment expertise that the organization’s

staff does not possess.

Selecting Collections for Digitization

Three factors influence selection: Condition, Significance, and Cost.

Condition refers to the physical characteristics of the resource. All physical records have a life expectancy based on format and storage conditions. If a record is degrading and losing information with age than it’s prioritization for digitization rises. Format obsolescence is another consideration when evaluating a record’s condition.

Collections that are Significant are those that have a high potential for research or generate lots of use and public interest. An archive’s collection policy should aid in determining significance, and the determination process should by systematic. Access is an aspect of significance – if a record is permanently restricted than it might not be considered as significant as if you could present the digitization on the internet.

Cost refers to the actual cost of digitization. While this consideration should not be weighted higher than condition or significance, it is important to spread your budget widely. If digitizing one motion picture takes up the entirety of your digitization allotment for the year, it may be worth looking at other collections. Conversely, considering the “cost” of total information loss can also help when selecting records for digitization.

Planning your request: Knowing and being able to communicate your needs to vendors is essential.◦ Establish what file formats and qualities you will request be created.

◦ NARA digitization guidelines are extremely helpful, but unrealistic for some institutions◦ Are there limitations or realities at your institution that necessitate unique or

special requests? Communicate those considerations to the vendor. ◦ Create an initial inventory of physical formats – how many items of each format

type, including physical duration. Vendors are not concerned with content, but will need to know the physical formats to be able to generate a project quote.

◦ Create a budget – know what quotes you can afford.

Desired File Formats

Preservation Master: A high quality, lossless or uncompressed digitization created in a non-proprietary file format. This file is for archival storage and only used in creation of new access copies.

UWM Audio Specs: 96kHz 24-bit WAV

UWM VHS Video Specs: uncompressed 720x486 4:2:2 .avi

Access Copy: A medium quality compressed file meant for patron use.

UWM Audio Specs: 48kHz 16-bit mp3

UWM VHS Video Specs: 720x480 4:3 compressed .mp4

Get quotes and choose a vendor◦ Approach multiple vendors with

your request.◦ Evaluate which vendor is best for

you by examining the quotes and contracts

◦ Establish a total estimated cost for the project

◦ Make sure that your requests, including special requests, are established in writing.

What you should be receiving

One preservation master per item

One access copy per item(optional)

A digitization project report, likely in the form of a spreadsheet, with item level metadata for each record digitized◦ Should at minimum include original item

number, digitization filename, duration, and transfer/condition notes.

Checksums for each file (recommended)

Preparing records for digitization◦ Create an item level inventory of the selected records for every collection.

◦ Record at minimum the item number, type, format, format duration, and desired file name for each record. This could also include brief item level descriptions.

◦ Make sure each item is rewound to its starting position and is clearly marked with an item number.

◦ Package the items carefully, according to preservation standards and paying attention to fragile formats.

◦ If possible/practical at your institution, make a note in the finding aid or catalog record that indicates that the records are out to digitization. Vendor-based digitization projects can take months to fully complete, and researchers will not be able to immediately access the records for a majority of that time. Also be sure that there are appropriate call slips, out cards ect.

QC: Ingesting the digitized files

Ingesting is the first step in the quality control process. After digitization is complete and you receive the digital files from the vendor:◦ Verify that the requested deliverables exist for each item. ◦ Check that a digitization project report was provided, and that it contains all the

information requested of the vendor.◦ Quickly confirm that the filenames appear to be the same as the filenames you

requested.◦ If your institution is set up for checksum verification, validate the checksums provided

by the vendor against those of the files your institution received. At UWM we use Fixity to generate a checksum report and check those against the vendor’s.

◦ Also! Create a preservation backup of the files provided by the vendor. The preservation backup is separate from the set on which you do your QC, and is intended as a failsafe.

Fixity Fixity is a digital records management tool that uses checksums (a cryptographic hash expression that textually expresses the data in a file) to verify the integrity of records over time.

Fixity If the encoded data in a record changes, either by human manipulation or mechanical error, the record’s checksum value will also change.

Fixity generates checksums for records, and checks the new values against the old values. If anything has changed the program alerts the user.

Unexpected changes can indicate a failure in the storage media, or unauthorized manipulation of the record, amongst other possibilities.

Quality Control Workflow Quality control at UWM is an eight-step process we conduct on each preservation master and accompanying access copy.

1) Verify that the file is named correctly, if it is not, record the correct file name.

2) Verify that the file is correctly encoded according to the project’s specifications. At UWM we use MediaInfo to examine A/V files’ technical metadata.

3) Examine the file’s waveform for errors, such as dropout or signal irregularities. Waveforms visually represent the data in the file on a timeline, so instead of needing to watch or listen to the whole file, one can see it all at once. UWM uses QCTools to examine video waveforms, and SonicVisualizer for audio.

Quality Control Workflow 4) Examine the beginning, middle, and end of the recording for quality and consistency.

5) Check for completeness of the file. Most recordings have a fairly natural beginning and ending.

6) Verify that the content in the recording corresponds to its descriptive metadata. Is the content present in the recording what it is supposed to be? If not, files may have been switched

7) Check that sound is being played correctly in the expected speaker configuration.

8) Briefly compare the recording length and quality of the access copy to that of the master, verifying that they are the same.

If any of these steps raised concerns, contact the vendor to clear up issues.

MediaInfoMediaInfo is a GUI that displays technical metadata for digital video and audio.

Sonic Visualizer for Audio QC Sonic Visualizer is an audio waveform viewer and player.

QCTools for Video QC QCTools is a video waveform viewer and player

QCTools

QCTools

Following up with the vendor If any errors were encountered during the QC process contact the vendor with descriptions of each error.

Don’t be afraid to contact the vendor! They are interested in creating the best product they can and, from experience, carefully answer your questions and work towards a resolution.

Completing the Project◦ When the project is complete and the invoice is closed the vendor will return

the physical records. Once the records are received check that each item was returned, is in acceptable condition, and is rewound to it’s starting position.

◦ Return the records to their permanent locations.◦ Update any catalog records or finding aids as necessary.◦ Currently UWM restricts access to the physical original of any A/V record that

has been digitized. We place a restriction note in the catalog record, finding aid, location database, and on the physical item.

◦ Place the QC Report and the vendor-provided digitization project report into the appropriate collection’s casefile.

◦ Ensure that the access copies are made accessible, and the preservation masters made secure and inaccessible to the public.

Benefits of QC and Digitization

Since Fall 2014:◦ 174 audio cassettes digitized, totaling 173 hours.◦ 73 video cassettes digitized, totaling 47 hours.

◦ 8 collections A/V have been digital preserved

QC so far has resulted in:◦ - identifying two records that was mistakenly not digitized by the vendor◦ - identifying two files that were digitized by the vendor but not delivered to us◦ - identifying four records that were mistook for one another by the vendor◦ - having three videos re-digitized for better quality◦ - identifying one hidden recording, which was present at the end of another recording.◦ - identifying one collection that had hidden privacy issues in the minimally processed A/V

Resources for you◦ UWM’s A/V QC Workflow◦ A/V QC documentation form, for recording the eight-step checklist◦ UWM’s standard file-naming convention for A/V preservation and

access copies.◦ A spreadsheet for inventorying records when planning a project.

◦ Contact: Charles Hosale - A/V Project Archivist, UWM Archivescmhosale@uwm.edu