1 Bridging the Worlds of Analogue & Digital e stablishing strategic policy framework for creating & preserving digital archives in National Archives of

1

Bridging the Worlds of Analogue & Digital

establishing strategic policy framework for creating &

preserving digital archives inNational Archives of Singapore

Pitt Kuan Wah

National Archives of Singapore 20 Feb 2001

2

The Pagodas of Bagan

3

The Pagodas of Bagan

Stonehenge of Great BritainStonehenge of Great Britain

5

We 'know' the past by the records people have left us.

Future generations, who may well have 'known' the 21st Century as silence and absence, unless they are able to access the records (many of which would have existed only in digital form) we leave behind.

6

““Saving Bits and Bytes for History”Saving Bits and Bytes for History”

7 August 2000

(The faith of Harvard Mark III computer)(The faith of Harvard Mark III computer)

7

William Shakespeare had written William Shakespeare had written HamletHamlet on a on a word word processorprocessor... ...

Thomas Jefferson had saved his drafts of the Thomas Jefferson had saved his drafts of the Declaration of IndependenceDeclaration of Independence with a with a computer text computer text editoreditor......

Alexander Graham Bell had documented his Alexander Graham Bell had documented his experiments with the telephone on experiments with the telephone on floppy disksfloppy disks … …

Leonardo da Vinci had used a Leonardo da Vinci had used a computer graphics computer graphics system system to create the to create the Mona LisaMona Lisa......

Would their great achievements still be available to us today?

What if….

8

Role of the Archives... As society's guarantee of an intellectual audit trail. This trail could be fragmented or broken by relying

exclusively on digital storage.

9

Durability of Digital Media In theory, optical storage media should be

more durable than magnetic tapes because the former is a non-contacting read/write technology

Actual life expectancy affected by:– manufactured quality

– quality of recording/writing equipment

– frequency of access/use & handling

– storage environment - temperature, humidity & cleanliness

10

Accessibility of digitised records`

Durable digital format is crucial in determining accessibility of records over time

11

NationalComputerisation

Plan1980 - 1985

Environmental Scan… IT Development in Singapore

1986 - 1991

1992 -

2000 -

12

Use of IT in Singapore Over 90% of businesses computerised, Internet-

ready. In a Y2000 survey conducted by

Infocommunications Development Authority, 70% of all households owned PC and 59% of them connected to Internet.

As of Aug 2000, Singapore achieved a mobile penetration rate of 63% - among the highest in the world.

On 14 Feb 2001, DTV broadcast commenced for public buses.

13

e-Citizen Centree-Citizen Centre

13 Confidential© IDA Singapore 2000

www.ida.gov.sg

14

E-Govt Structures

Committee of Permanent Secretaries

E-Government Policy Committee

Public Service Infocomm Steering Committee

Managing for Excellence Office

Government Chief Information Office

15

Revised National Archives and Record Centre Act (1967)– Electronic Records included in NHB Act 1993

Reviewed recordKeeping policy & preservation guidelines– Expanded existing IM on housekeeping records– New IM on Infocommunication Technology

Management (archiving aspect)– New guideline on management of email records

Establishing Policy FrameworkEstablishing Policy Framework

16

Civil Service-wide Email Usage

17

Appraisal and Preservation of Backlog Application Systems (databases)

– recent inventory exercise listed some 700 application systems, 40% of housekeeping nature

Formulate Practical GoalsFormulate Practical Goals

18

File/Document Management

Document Imaging

Workflow/Task Management

Search & Retrieval

Security & Access Controls & Role Based management

ERS ERS FunctionalitiesFunctionalities

Backfile Conversion

Archiving

Electronic Faxing

Digital Signature/Encryption

• KKey role in developing Civil Service-wide Electronic Registry System

19

Mapping Preservation Technologies

20

Breaking the HourglassBreaking the Hourglass

21

In search of an “archival medium” Unknown at this point in time All are software and hardware dependent,

need periodic migration/refreshment Existing choices:

– Magnetic tape (eg 9-track tape, 3590 cartridge)– CD-R (ISO 9660 compatible) and WORM– MO disk– Optical tape 5?50?10?

22

Durable Open-ended Universal Versatile Cost-effective Legally binding

23

Selecting Media Beyond technical

specifications and capability Market penetration and

competition– Super floppy drive– CD-R vs DVD-R

Avoid leading edge technologies

24

We need not be exclusive in selecting imaging technologies!– Technologies Can Co-

exist

Hybrid Systems are still more cost effective today– Digital tech is not cheap– Digital tech is fast

changing, thus not lasting

INFOGUARD T M

WRITABLECDK do ak

T M

dis cC M PAC TO

PRO TE CTIO N SYS T E M

Kodak

25

#1 Outputting the records to human-readable media

Preservation Technologies – Options

26

“Download and print out the pretty version of TIME DIGITAL 2026 here and save it for the future.”

27

Why Microfilm Still?

Stable, matured technology Creation & preservation

processes guided by international and national standards

Film medium not susceptible to technology changes

VIRUS ALERT !

Bye-Bye No HELP

VIRUS FOUND !

Your Data is infected with the GOTCHA virus. This file is now UNUSABLE!!

You will NOT see this on microfilm!

29

Singapore Standards on Microfilming– CP 69:1998

Microfilming of documents on 16 mm black and white, silver-gelatin type microfilm/planetary and rotary camera filming – Operating procedures

– CP 70:1998 Microfilming of technical drawings and other drawing office

documents on 35 mm black and white, silver-gelatin type microfilm/planetary camera filming – Operating procedures

– CP 71:1998 Quality control inspection of 16 mm and 35 mm black and

white, silver-gelatin type microfilm

– CP 72:1998 Microfilming of newspapers on 35 mm black and white, silver-

gelatin type microfilm/planetary camera filming – Operating procedures

ImagesfromHost

xxxxxxlllmmxx

xxxxxxlllmmxx

xxxxxxlllmmxx

loan 12.TIF

loan 12-1.TIF

loan 12-2.TIF

Loan 116/1/93

Loan 117/6/93

Loan 119/23/93

Loan 1111/19/93

Loan 124/15/93

Loan 124/23/93

Loan 1211/23/93

R5 F5.0 R5 F5.1 R5 F5.2 R5 F5.3 R5 F6.0 R5 F6.1 R5 F6.2

xxxxxxlllmmxx

xxxxxxlllmmxx

xxxxxxlllmmxx

xxxxxxlllmmxx

loan 11.TIF

loan 11-1.TIF

loan 11-2.TIFloan 11-3.TIF

TIFF to Writer Log File

transfer to host, CAR

loan 11.TIF R5 F5.0loan 11-1.TIF R5 F5.1loan 11-2.TIF R5 F5.2loan 11-3.TIF R5 F5.3loan 12.TIF R5 F6.0loan 12-1.TIF R5 F6.1loan 12-3.TIF R5 F6.2loan 13.TIF R5 F7.0

IMAGES

Transfer

to Host

ARCHIVE SERVER

Direct Digital-to-Film Solution

31

Key Findings Electronic mail with attachments (in

HDB and RCB electronic records) can be opened up and print to film

Electronic data from mainframe systems like pay records from AGD - can be printed to film, bypassing conventional paper process

Pilot Tests

32

Future Directions ... Need for NAS to re-engineering

its preservation processes – Replacement of small microfilm

cameras with scanners– Improve output quality and

accessibility – Improve productivity

Output critical electronic records directly onto film medium

33

#2 Preserving records in their native format as an interim


34

#3 Applying available techniques to overcome technological obsolescence emulation

migration


35

““Exterminate! Exterminate!!”Exterminate! Exterminate!!” This was the infamous statement made by the Daleks - Dr Who’s most persistent enemies...But for archivists who concern with preservation of e-archives, our statement would have to be: “Migrate! Migrate!!”“Migrate! Migrate!!”

36

#4 Converting the Records to Standard Formats:

A question of riskASCII

Unicode

XML

PDF

PCD


37

Stable & mature technology developed by world’s leader in film industry:– Conceptualized in 1989, released 1992

Adopted by many libraries, museums and archives (More than 60 million PCD images worldwide)

Direct input from source (film)– System able to differentiate film types (brands) &

their characteristics, thus accurate conversion– Opportunity to correct colour & density errors in

original image - occurred either in camera exposure or film processing

Photo-CD for Photographic & Cartographic Images

38

High quality image, low storage cost– US$1 per 2000 dpi scan (35mm film)– U$$6 per 4000 dpi scan (4”x5” film)

Meet multiple needs for Internet display, desktop printing, exhibition and high quality publishing– A 4000 dpi single scan produces six multiple optical (true)

resolutions: Base/16 (128 x 192 pixels) Base/4 (256 x 384 pixels) Base (512 x 768 pixels) 4xBase (1024 x 1536 pixels) 16xBase (2048 x 3072 pixels) 64xBase (6144 x 4096 pixels)

39

How does NAS deal with proprietary problem?– deposit source code

or make an ESCROW arrangement to deposit with 3rd party

– sign MOU, help us to migrate when technology is replaced

200% Zoom

NAS pilot tested 500,000 documents between 1996 and 2000

PDF for Compound documents & Graphic Texts

41

Our views on PDF…...

Not yet ISO standard but fulfils many archival requirements:– enable exchange of graphical documents over

network, independent of hardware platform, operating system and even font characters.

PDF Reference Manual, Nov 1996 (ISBN 0-201-62628-4)

– faithful rendition of documents during exchange, contents alteration almost impossible

PDF documents accepted in Singapore courts

– free viewer enable viewing across almost any platform

42

#5 Persistent Object Preservation for Archives (POPA)

NARA’s joint project with San Diego Supercomputer Centre

Based on archival science: provenance & original order– Tested on several classes of electronic records in NARA, eg

email, databases, GIS– More tests on preservation of web pages

Infrastructure independent


43

– Two archivists consultants from NAC (1995) and NAA (1997)

– In 1999, Dr Kenneth Thibodeau, Director, Electronic Records Archives Programme, US National Archives (NARA) reviewed and recommended:

inventory & appraise all ER set up ER programme modeled after NARA for

management of ER with a core staff adopt NARA’s long term ER Archives Technology for

preservation of complex records eg. Multimedia & Geographical Information System

Need for Collaboration

44

QuickTime™ and aPhoto - JPEG decompressor

are needed to see this picture.

Local Effort

NAS fostered a tripartite MOU was signed in Feb 2001 to develop a eArchiving module for the Civil Service-wide ERSERS (Electronic Registry System)

45

Leverage on Expertise/Resources NAS: As national custodian of archives,

expertise in recordkeeping & long term access

SQLV: As developer of the generic Civil Service-wide ERS, expertise in“knowledge management” solutions

Kodak: As leader in imaging industry (since 1916) & strong commitment in film technology and other forms of digital storage media

46

Selection of Folders/Files

Archiving Engine

Extraction Decryption process if file is

encrypted. Document in native format. Converts file to TIFF format.

A. List of FilesB. Conversion of files to XML: Native Document* TIFF Format (Binary)

*(with Profile, Activity History& Access List).

Extraction ofList of

Documents

Directory: Document with Profile Activity History Access List

Extract Documents with Profile, ActivityHistory & Access List to a Directory

Decryption of XML File with Private Key: Native Document TIFF File (send to Digital Archival

Writer for microfilm processing)

Files to NationalArchives

National Archives of Singapore (NAS)Ministry

Overview of Electronic Archiving System for ERS

User

ERS Server

Database(Profile, Activity

History & AccessList)

ERS(where e-documents

reside)

Extraction results can bereturned to user

Offload XML files to aCD-Rom, ZIP, removable media or

secure online transfer with appointedPublic Key Encryption.

Tape drive Optical driveRemovable storage

Document Archive Writer

DAW Server Storage Server

Secure Storage Disk array

Archiving tomicrofilm

Digital Archiving inXML

microfilm

47

a science fiction adventure series follows four explorers who discover a way to travel to parallel universes.

48

Episode: NEW GODS FOR OLD

QuinnQuinn Mallory: Mallory: “You are telling me that I have to choose?”

Maggie Beckett: Maggie Beckett: “That is what life is all about too. It’s about choices and possibilities.”

Rembrandt Brown:Rembrandt Brown: “Choices are never easy, otherwise we would never know how powerful we are!”

49

The Future

No single strategy can address preservation and access issues adequately

information value

technology

opportunity cost

Living with risk….

Playing it safe in today’s volatile digital environment isn’t necessary less risky.

Sometimes taking a calculated risk is wiser!

51

“Perfecting the Perfecting the knownknown”

“Imperfectly Imperfectly seizing the seizing the unknownunknown”

52

Thank you