A centre of expertise in data curation and preservation RMS Annual ConferenceBrighton01 May 2007 Funded by: This work is licensed under the Creative Commons

a centre of expertise in data curation and preservation

RMS Annual Conference Brighton 01 May 2007

Funded by:This work is licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 2.5 UK: Scotland License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-sa/2.5/scotland/ ; or, (b) send a letter to Creative Commons, 543 Howard Street, 5th Floor, San Francisco, California, 94105, USA. Excludes images.

Scaling the e-mail mountainA records manager’s guide to e-mail curation

Maureen Pennock

Digital Curation Centre, UKOLN, University of Bath



The first telegram…

Source: The American Memory collection @ the Library of Congress



The first telephone call…

Source: American Treasures collection @ the Library of Congress



The first email…

… ‘QWERTYIOP’?

… ‘ASDFGHJK’?[But it included information on using the ‘@’

symbol]

Announcing the availability of

networked mail… … exact words

unknown



Contemporary use

• 100% of businesses use email for business purposes (AIIM Survey 2003)• Administrative and commercial activities• Contracts and agreement negotiations• Financial issues• Legal issues• Tendering processes• Recruitment activities

• Both records and non-records



Legal drivers

• Data Protection Act (1998)• Freedom of Information Act (2000)• Regulation of Investigatory Powers Act (2000)• Human Rights Act (1998)• Intellectual property legislation

• Trade Secrets/Law of Confidentiality

• Copyright Law

• Database Rights



Risk management & business drivers

• Risks of failure to curate:• Legal consequences

• Financial consequences

• Loss of public credibility

• Loss of organisational memory

• Loss of accountability

• Loss of transparency

• Reduced efficiency

• Should be addressed by risk management framework• Based on information compliance & risk assessment exercises



Contemporary challenges (I)• Unclear responsibilities:

• Email as personal domain, not organisational property

• User reluctance to ‘manage’ inbox• Clash between IT and RM policies (if any)

• Enforcement problems• Inappropriate content• Undesirable content• Embarrassing content



Contemporary challenges (II)• E-mail discovery carries inherent problems…

• Dispersed locations: IMAP; POP; Forwarded messages; mailbox; shared drive; ERMS; EDMS…

• Some may be inaccessible to systems admin or records managers

• Relevant data may (or may not) be • In subject line• In message content• In message attachment (difficult to search)

• Sheer scale of challenge• Not indexed for fast searching

• … high discovery costs



Contemporary challenges (III)• Coherence of message threads• Admissibility of printed versions of e-

mails?• Improper deletion/destruction procedures

• This is not just an issue for ‘records’• Information (mis)management • Risk (mis)management

• Can lead to all sorts of problems...



TV Chief lets rip in snappy birthday email

‘A good day to bury bad news…’



1. Digital Curation & E-mails

What is digital curation?How does it apply to e-mails?

Life cycle roles & responsibilities



What is digital curation?

• “Digital Curation: …The activity of, managing and promoting the use of data from its point of creation, to ensure it is fit for contemporary purpose, and available for discovery and re-use.”

Lord & MacDonald (2003)



Why curate e-mails?• All records must be curated to ensure they remain fit for

use and re-use: including e-mails

• To combat:• Technological obsolescence

• Bad creation & management practices

• To help differentiate between records and non-records (and treat accordingly)

• To help meet legal obligations

• To be cost effective and efficient

• To secure viability and reliability of messages for future



Storage & Preserv-

ation

Transfer

Active Use

Appraisal &

Selection

Access & Re-use

Creation

Curation & the life-cycle

• Meaningful chain of custody

• Requires compatibility of different stages

• Requires input from range of stakeholders

• Takes control over the records throughout lifetime

Records

Disposal?

Disposal?



Stakeholders & Roles• The range of stakeholders that affect the

survival of digital material cuts across the whole lifecycle; everyone plays an important role• Management & policy-makers• Users - creators & receivers of e-mail messages• Records Managers• IT staff

• System & mail-server administration• LAN Manager

• Archivists • Re-users

'Curators'



Issues for Creators & Recipients • E-mails must be:

• Well-formed• Well-managed (even sent items!)• Accessible

• Important elements:• Good creation/response practices

• Inserting metadata• Headers – subject line, addresses• Message body - context

• Message formats• Attachments• Complying with house-style

• Good inbox management• Compliance with organisational policy



Issues for Curators• E-mails must be:

• Whole - comprising message body, headers & attachments• Captured as appropriate into organisational

document/records/archives management system• Destroyed as appropriate

• Important policy and practical elements:• Identification & selection of e-mail records from non-records• Proper filing & integration of e-mail records• Deletion of transient/unnecessary e-mails• Saving e-mail records independently of e-mail client• Determining authenticity requirements• Archiving & preservation

• Guidance and Training for users ( at all levels)• Communication across the board



Issues for Re-users• E-mails must be:

• Accessible for appropriate re-users• Exported in an appropriate and usable format

• Things to consider:• Legal access and re-use restrictions may be different & must

be observed• Appropriate re-use software may be needed

• Resource Discovery• Access Rights Management

• Different re-users may have different re-use requirements• E-mails can be re-used for very different purposes to why

they were originally created



2. Practical Steps



Develop an e-mail policy• One or more policies to cover:

• Creation practices• Using business e-mail accounts for private

use & vice versa• Responsibilities & shared access• Levels of organisational monitoring• Legal issues• Integrated records retention & preservation• Disposal



Provide education & training• On composition –

• Meaningful messages• Formats• Attachments• Context

• On storage & transfer• Both sent and received mails• Relationship between inbox management and

records management

• On legal responsibilities



Implement a solution• Consistent application of appropriate strategy

• May be best determined by strategic working group with representatives from different stakeholder groups

• Should take entire life-cycle into account

• Should be integrated with organisation RM – not a stand alone solution

• Must consider issues raised in this presentation

• Requires consistency across life-cycle stages

• Assess compliance regularly!

• Revisit regularly to keep up-to-date



3. Management & Preservation Options



The First Solution



The First Solution… ?

???

? ?

Doesn’t solve all the problems… … and may

even create extra ones!



Coming a close second…

[…also known as the ‘do-nothing’ approach]



A more realistic option

• Convert or save in Standard formats• ‘The best thing about standards is that there

are so many to choose from!’• Which are most appropriate for e-mails?

• PDF?• TIFF?• RFC 2822/Plain text?• XML?



Other issues to consider

• What about attachments?• Dealing with digital signatures• Preservation metadata• Long term storage and archiving• Audit and certification



The future?• Changes to e-mail interface

• New applications• Alternative search functionality• Visualisation for re-use

• Alternative technologies• Instant Messaging• SKYPE• RSS (Really Simple Syndication)• Discussion forums• Blogs, wiki’s, texting…



Further reading & references (I)• Policy:

• Institutional Records Management & E-mails project (2003)

• http://www.lboro.ac.uk/computing/irm

• Sample University Guidelines & Advice• Edinburgh University – exemplar

• Developing a policy for managing e-mail (2004)• Guidelines from TNA• http:///www.nationalarchives.gov.uk



Further reading & references (II)• Technical solutions for preservation:

• Digital Preservation Testbed XMaiL and preservation recommendations (2003)

• http://www.digitaleduurzaamheid.nl

• Antwerp City Archives e-mail preservation template & advice (2003 – 2006)

• http://www.expertisecentrumdavid.be

• National Archives of Australia: XENA (2004 +)• http://www.naa.gov.au

• San Diego Super Computer Centre (1999 +)• http://www.sdsc.edu



Further reading & references (III)• DCC Digital Curation Manual

Instalment on ‘Curating E-mail Messages: A life-cycle approach to the management and preservation of e-mail messages’

Maureen PennockDigital Curation Centre & UKOLN2006

http://www.dcc.ac.uk



Thank you.

Questions?

Maureen [email protected]

(Join the DCC Associates Network at http://www.dcc.ac.uk)

Documents

A centre of expertise in data curation and preservation RMS Annual ConferenceBrighton01 May 2007 Funded by: This work is licensed under the Creative Commons