Embed Size (px)
Citation preview
Heads of Knowledge and Information Management and Departmental Record Officer Training 2019
25th November 2019
Delivered and supported by a variety of teams across The National ArchivesFacilitator - Katie Kinkead
Welcome to the Learning Room
Objectives
By the end of the training you will be able to:
• Explain how your role and TNA roles and responsibilities
are affected by PRA and Information Rights Legislation.
• Recognise how to further and better collaborate with TNA.
• Understand and clarify TNA’s key functions that affect your role.
Public Records Act 1958
The National Archives
Public Record Bodies
Responsibilities under Public Records Act:
The National Archives
Provide guidance and supervision to publicrecord bodies on the safekeeping and selectionof public records.
Preserve transferred records. Provide facilities for the public to see and
obtain copies of transferred records, unlessthe records are closed or retained because anexemption in the Freedom of Information Actapplies.
Oversee the place of deposit system on behalfof the Secretary of State for Digital, Culture,Media and Sport.
Return records temporarily at the request ofthe transferring organisation.
Care and preserve records.
Responsibilities under Public Records Act:
Public Record Bodies
Selection of records for permanent preservationunder the guidance and supervision of the Keeper of Public Records.
Safe-keeping of records. Transfer of records to The National Archives or
an approved place of deposit by the due date unless they need to be retained, in which case the Secretary of State for Digital, Culture, Media and Sport’s approval must be obtained.
Carry out formal applications for retention and closure through The National Archives which are then reviewed by The Advisory Council on National Archives and Records.
Disposal of records not selected for preservation, by destruction or presentation to another institution.
Department for Digital, Culture, Media and Sport
DCMS – Secretary of State
The National Archives
Jeff James - Chief Executive and Keeper
Lucy Fletcher - Director for Government
John Sheridan - Digital Director
Government Audience
Digital Archiving
Public Record Bodies
How we all fit together…
Cabinet Office (CO)
Department for Digital, Culture, Media and Sport
DCMS – Secretary of State
The National Archives
Public Record Bodies
• Information Management Report (IMR)• Information Management Assessment
(IMA)• Training• Advice and Guidance• Events/User-groups
How we all fit together…
Department for Digital, Culture, Media and Sport
DCMS – Secretary of State
The National Archives
Public Record Bodies
The Advisory Council on National
Records and Archives (ACNRA)
Places of Deposit (POD)
Information Commissioner’s
Office (ICO)
GKIM Professions
What can support you
• Office 365 GKIM user group – [email protected]
• Google GKIM user group – [email protected]
• ADRO – [email protected]
• KIM Leaders – [email protected]
• IRMS – irms.org.uk
• ARA – archives.org.uk
• CILIP – cilip.org.uk
• Civil Service Learning – https://civilservicelearning.civilservice.gov.uk/
• Crown Commercial Service –https://www.crowncommercial.gov.uk/agreements/rm3781
• The National Archives Training and Events –http://www.nationalarchives.gov.uk/information-management/training/
FOI - Responding to historical information requests (Access to Public Records)
Facilitated by Conor Gregan and Christine Peters, FOI Managers, The National Archives
GDPR from the archival perspective
Linda Stewart
Data Protection Officer
A long tradition of making data available
1981,1984, 1998, 2018
Research, history 33.—(1) In this section—
and statistics. “research purposes” includes statistical or historical
purposes; “the relevant conditions”, in relation to any processing of
personal data, means the conditions—
(a) that the data are not processed to support measures or
decisions with respect to particular individuals, and (b) that the data
are not processed in such a way that
substantial damage or substantial distress is, or is likely to be,
caused to any data subject….19Processing for archiving, research and statistical purposes: safeguards
This section has no associated Explanatory Notes
(1)This section makes provision about—
(a)processing of personal data that is necessary for archiving purposes in the public interest,
(b)processing of personal data that is necessary for scientific or historical research purposes, and
(c)processing of personal data that is necessary for statistical purposes.
(2)Such processing does not satisfy the requirement in Article 89(1) of the GDPR for the processing to be subject to appropriate safeguards
for the rights and freedoms of the data subject if it is likely to cause substantial damage or substantial distress to a data subject.
2005: Access under
PRA becomes access
under FOIA = access
via DPA 1998
European
Convention
108 1981
Data Protection Act 1984
Mention Archiving in Privacy Notice
The GDPR has extra and more granular requirements to inform data subjects of the processing undertaken.
Departments are encouraged to mention transfer to the National Archives in their privacy notices, and to provide a link to the National Archives’ website.
Why has this not happened?
John Smith
Note that the name of a person may not in itself beenough to make the person identifiable and it usuallydepends on the context in which it appears or thepresence of supplementary information enabling aperson to be identified.
However if the record is being made available online and will be exposed to search engines, the name will enable linking to other information that could identify the person, and should therefore be redacted.
John Smith’s Agricultural Census Return
Any records (paper or digital) that are indexed byname or other identifier - generally case files,should be closed until DPA no longer applies.
Life expectancyremainsat 100 years…
…or earlier if events intervene
The hundred year ‘rule’
How do you approach sensitivity review?
Closed or redacted
Generally for 100 years
Finding aids, generally open, with caveats:
How do you approach sensitivity review?
Where ?
Who?
Why?
What happens to personal data in your records once they are here?
Archival records can still be used to take decisions about people
Data subject’s information rights
Sharing with other government departments
Digitisation and online access
GDPR,AIPI and BAU
Qualifications -
- Evidence of status
Rights – Subject Access Requests
Open records - we have to supply if we’re given enough information to locate –not just everything about me in the archives
Closed records – no obligation to supply but we usually do; we were set up to provide evidence.
Right to be forgotten
Erasure
Never destroy
Remove from access under our takedown and reclosure policy
Most people just don’t want their name to come up in search engine returns out of context.
If we decide not to remove online content we advise people of their right to make a separate erasure request to the search engines.
Right to Rectification
Corrections
The National Archives cannot alter an archived public record
S 9 of the PRA 1958 – record is what it is
Evidence that a government department had wrong information about them is of value to the data subject.
We will consider annotating the record, leaving the original unchanged - if it is technically possible
Disclosing to police
Can see open archival documents - just like anyone else
Can see who has accessed records if we are satisfied it is necessary
If they wish to see a closed record, we ask them to ask their DRO to contact you. If you’re satisfied, you request the closed record back in the normal way.
s. 29 DPA 1998 is now Sch 2. part 1. para 2.(1) DPA 2018
The Crown is not indivisible
Personal Data Online - The privacy deathstar
Personal Data in the Archives
Engage with government
Transfer to TNA
Online
‘Personal Data’ Online
Records about living people
not publically available online
Known to be
deceased
100 year
‘rule’
Limited access –
academic paywall
Researchers’ DP responsibilities
Researchers are Data Controllers for any personal data they access – we have notices in reading rooms and in our rules this effect.
Although the data they see is open, they still have a duty not to use it in a manner that would cause substantial damage or substantial distress
The National Archives does not give privileged access to closed personal data (permitted under DP but not allowed under FOI)
Recap GDPR mostly BAU for transferring personal data to TNA
It is still transferring dept’s responsibility to locate and reviewpersonal data that could identify living individuals
Data subjects have limited rights: redaction not erasure;annotation not correction
‘Crown Indivisibility’ does not apply to data protection: TNAwill not share details of people who have accessed archivalrecords without statutory authority or a specific court order.
Police must access closed records via transferring department
Availability of personal data impacts likelihood of distress
Researchers responsible as Data Controllers for any use theymake of personal data
Intellectual Property
Lynn Swyny
Copyright Manager
Only one thing is
impossible to God - to
find any sense in any
copyright law on the
planet.
What do you know about…
CopyrightLicensing
Re-use
CopyrightCrown copyright
Non-crown copyright
Third party copyright
DRO responsibilities
Understand your organisation’s copyright situation, including any delegation
Maintain information about 3rd party copyright that you hold
If you have one, make contact with your organisation’s commercial licensing team
Familiarise yourself with available guidance from TNA and IPO
Copyright 2TNA responsibilities
The Keeper manages Crown copyright and database right under Letters Patent from the Queen
Authorises all licensing under Open Government Licence terms
In rare circumstances, can approve assignment of Crown copyright to a non-Crown body, or assignment to the Crown
Will give a Delegation of Authority to a Crown body if it is able to license its own material
Advice and guidance on copyright and licensing
Re-use of Public Sector InformationRe-use of PSI Regulations 2015
Managing Public Money
DRO Responsibilities
Generally, comply with the Regulations
Understand what material you hold that is available for re-use, & how it is being re-used
Have in place a complaints handling process
Understand your department’s public task
If you have one, make contact with your organisation’s commercial licensing team
Re-use of Public Sector Information 2
TNA responsibilities
Lead department for re-use – source of guidance on re-use for public sector bodies
Guidance on defining your public task
Advice from the policy team
Licensing
Open Government Licence – default for Crown bodies, recommended for all UK public sector bodies
Crown bodies can only vary from OGL if they have a delegation
Other government licences are available if you have a delegation or are a non-Crown body
But remember, Government policy remains that wherever possible, public sector information should be made available for re-use under OGL terms; and the Regulations allow relatively little scope for restrictive licensing
Licensing 2DRO responsibilities
If you have one, make contact with your organisation’s commercial licensing team
Ensure that your information is available for re-use under OGL terms wherever possible
TNA responsibilities
Maintaining the UK Government Licensing Framework
Advice and guidance
Any Questions?
Mark Twain image by skeeze from PixabayAll other images © Crown copyright
UK Government Web Archive
Tom Storrar
Head of Web Archiving
Who has heard of web archiving?Who has heard of UK Government Web
Archive?
1996…
…2019
Collecting the government
web estate…
…including socialmedia
The National Archives and Web Archiving
Comprises more than 25,000 crawls of over 4,500 websites (1996-present)
Is approximately 155tb in size, nearly 6 billion resources
Has a powerful full text search tool
Is fully open online and accessed by millions of users.
It is a vehicle for Collection, Preservation and for Access for digital records.
It is also a tool for contextualisation of records past, present and future.
The UK Government Web Archive
What do we capture?>800 distinct websites and social media accounts are regularly archived: those of central govt. public bodies, delivery channels (e.g. GOV.UK) national NHS, public inquiries, some inquests.
On these, we take everything we can on the target website: Publications, datasets, other documents Images Video, animations And all other files that make up the website
…but, web archiving operates within technical constraints. Content must be: Publicly-available Reachable by robots / crawlers
For more information, see our guidance at http://www.nationalarchives.gov.uk/webarchive/guidance/
Finding content in the web archiveSearch (https://webarchive.nationalarchives.gov.uk/search/)
- Search across the full text of UKGWA, with features such as
restrict to website, date range, format
A to Z (http://www.nationalarchives.gov.uk/webarchive/atoz/)
- Most of the websites we have archived are listed here
Know the URL? Use the /*/ index
- e.g. https://webarchive.nationalarchives.gov.uk/*/https://www.gov.uk/
- Shows all the dates at which a resource was captured.
Discovery (https://discovery.nationalarchives.gov.uk/)
- Most websites are catalogued at series level
How do you think we can work together?
Consider the web archive as a vehicle for digital transfer.
Make sure that the content is “crawlable”! Check our guidance and, if in doubt, ask our advice.
Copyright: ensure that the website copyright statement is clear. This supports future reuse and we need to know of any 3rd party content.
Review our Takedown Policy:
nationalarchives.gov.uk/legal/takedown-and-reclosure-policy/
Tell us about any new websites/web platforms/social media.
Leave content available on the web for at least six months and consider archiving timescales – it is not an instantaneous process!
Check capture before removing it from the live web.
How we can work together
Introduction to Transfer Digital Records (TDR)
Kirsten Arnold, Service Owner: Digital Selection & TransferNovember 2019
User engagement
Guidance for ongoing and upcoming transfers
Establishing & delivering service improvements
What we do in Digital Selection & Transfer:
Current Digital Transfer Journey
Formed of a number of steps:
Appraisal & Selection
Digital Sensitivity Review
Transfer(records reach TNA)
Ingest into TNA Systems
Preservation
Access (via Discovery)
Transfer Steps
We currently ask the transferring body to:
1. Complete a digital transfer form
2. Install and utilise three software tools
3. Generate a number of reports and files including a metadata schema which is bespoke to each transferring department
4. Place a copy of their files on an encrypted hard drive and send to TNA
Why do we need to improve on what we are currently offering?
NEED VALUE
Statutory Trust and reputation
Users Easier and simpler transfers
Digital by design Improves efficiency
TDR AGILE APPROACH
https://govdesign.tumblr.com/image/164061029093
TDR GOVERNANCE – BALANCE & CHECKS
BUSINESS OBJECTIVES
USER NEEDS
TDR REQUIREMENTS
Security (Cloud)
Data Protection (GDPR)
Accessibility (WCAG)
GDS Guidance
Audits (BDO)
Agile
However…TDR is not a new concept!
TB1 TB2 TB3 TB4 TB5 TB6 TB7 TB8 TB9 TB10 TB11
Stage Min Max Average TDR
Tools installation (DROID etc.)
Digital Transfer Form
request & completion
Technical checkpoint:
(DROID export)
Metadata for records:
Interim metadata CSV
Metadata for series
(creation/check)
Summary for closure to
Advisory Council
Advisory Council meeting:
(and approval)
Metadata for
series:(validation/confirmation)
Metadata for records:
Final metadata CSV
"Physical" transfer
Subtotals of Stages
Duration (days)
STAGE DURATIONS IN CURRENT PROCESS
STAGE DURATIONS IN CURRENT PROCESSTB1 TB2 TB3 TB4 TB5 TB6 TB7 TB8 TB9 TB10 TB11
Stage Min Max Average TDR
Tools installation (DROID etc.) 337.00 87.00 13.00 67.00 390.00 273.00 404.00 0.50 0.50 0.50 0.50 404.00 157.25 YES
Digital Transfer Form
request & completion 0.50 0.50 0.50 17.00 1.00 0.50 17.00 3.90 YES
Technical checkpoint:
(DROID export) 36.00 0.50 50.00 0.50 0.50 0.50 0.50 21.00 21.00 1.00 0.50 50.00 13.15 YES
Metadata for records:
Interim metadata CSV 27.00 0.50 21.00 59.00 40.00 39.00 310.00 10.00 10.00 4.00 0.50 310.00 52.05 YES
Metadata for series
(creation/check) 43.00 0.50 0.50 41.00 0.50 62.00 0.50 4.00 5.00 5.00 76.00 0.50 62.00 21.64 YES
Summary for closure to
Advisory Council 305.00 0.50 0.50 0.50 52.00 52.00 0.50 37.00 0.50 305.00 56.00 NO
Advisory Council meeting:
(and approval) 50.00 37.00 0.50 68.00 52.00 13.00 63.00 13.00 63.00 33.00 5.00 0.50 68.00 36.14 NO
Metadata for
series:(validation/confirmation) 1.00 2.00 0.50 88.00 0.50 0.50 0.50 0.50 0.50 0.50 88.00 10.44 YES
Metadata for records:
Final metadata CSV 11.00 134.00 0.50 245.00 0.50 7.00 0.50 0.50 0.50 245.00 49.88 YES
"Physical" transfer 7.00 4.00 0.50 0.50 36.00 27.00 57.00 0.50 57.00 18.86 YES
Subtotals of Stages 817.50 266.50 87.00 570.00 589.00 422.00 831.00 50.50 157.50 80.50 81.00 419.30
Duration (days)
DIG
Gateways’ structure
Interaction start point
Transferring bodies and selection and appraisal
BREAKING ASSUMPTIONS
Transfer Digital Records (TDR) is a TNA service for transferring bodies who have selected their born digital records for transfer.
TDR is a cloud based collection, preparation and transfer service that will enable the upload of these records and facilitate a more streamlined transfer, as an improvement to the current manual process.
USER PERSONAS
TDR BRAINKirsten Arnold Product OwnerService Owner (Digital Selection & Transfer)
Laura DamianLead Developer
Ian HoyleSnr Developer
Sam Palmer Snr Developer
Suzanne Hamilton Snr Developer
Tom KingDeveloper
Sarra Hamdi Lead User Researcher
David Clipsham Technical Architect
Pia Lembke UI Designer/Developer
Linda Dodd Delivery Manager (Interim)
Mark Kingsbury Developer
TDR ALPHA STAGE – June – October 2019
Opportunity to try out different solutions
Building prototypes and testing different ideas
Challenge the status quo – explore new approaches
Produced throw away code
Test riskiest assumptions
Focus on the areas which will bring the most challenges
By the end of Alpha we decided which of the ideas we’ve tested we want to take forward as an MVP for the Beta stage
TDR ALPHA SCOPE
To demonstrate how technical design, infrastructure and tools can resolve these key user needs with a working proof-of-concept prototype to represent three of the four defined TDR Agile Epics:
UPLOAD – I as a User can get my content into TDR ‘easily’
PROCESSING –TDR will add data & complete specified checks for me
TRANSFER/EXPORT – My content/data reaches TNA with its integrity intact
AUDIT – data model approach only; out of Alpha scope for prototyping
PAIN POINTS
1.Complete a digital transfer form
Integrated into TDR
2. Install and utilise three software tools
TDR will remove the need for users to install most if not all of these tools including DROID
3. Generate a number of reports and files including a metadata schema which is bespoke to each transferring department
TDR will reduce the need for the exchange of spreadsheets
4. Place a copy of their files on an encrypted hard drive and send to TNA
Users will use a web based browser to upload their transfer consignments
It does not include appraisal, selection, sensitivity review or any other aspects related to ‘pre’ transfer.
Rather it looks to automate operational aspects of the transfer process itself
Remove the need for tool installation and editing of metadata spreadsheets in Excel.
IN SUMMARY
No more tools to download
Cloud-based (AWS)
Automated metadata creation
Automated ‘sanitisation’ –antivirus, file format identification
Interactive, guided ‘fix-up’ to finalise your transfers
TDR DELIVERS …
Your Time
Thank you for attending the training
Explain how your role and TNA roles and responsibilities are affected by PRA and Information Rights Legislation.
Recognise how to further and better collaborate with TNA.
Understand and clarify some of TNA’s key functions that affect your role.
By the end of the training you will be able to:
[email protected]@nationalarchives.gov.uk
Behind the Scenes Tour
