David Underdown @davidunderdown9

22 November 2013

Digital Records and theCounty Durham Home Guard

Catalogue Day 2013

About me

• Worked here 8 ½ years

• Involved in our digital preservation activities throughout

• A techie

• But –

• Also carry out my own family history and other research as a reader

Project background – digital records

• Initially we had focused on “born digital” records

• Increasing realisation that we also needed to protect better the investment represented by digitisation projects

• How to deal with the set of MOD personnel records, covering post First World War service

• Large collection and personal nature of information challenging

• Some previous examples of us taking material in forms that would normally be used only as surrogates

• “Digitised records” concept

About the project

• Home Guard personnel records initially selected as a priority for early transfer

• Comprise around 4.5 million individual records, 720m of shelving

• County Durham selected as pilot, approx 2% of total

• Model for overall population, variety of industrial areas/urban/suburban/rural, also coastal and inland areas

• Sampling from whole series had suggested only about 10% would be closed

The Home Guard

• Organised by county

• 1 or more battalion each

• Local companies or platoons

• Large employers had own units

• Duties carried out on top of usual work

• Active 14 May 1940–3 December 1944 (disbanded 31 December 1945)

6

Challenges and issues 1

• Piloting digitised record process, but also infrastructure upgrade and new version of digital repository software

• Record volume still potentially a problem, even if challenge is storing digital files not paper

• To store County Durham set as Tiff images, approx 7TB, 2% of total Home Guard collection (and we make 3 copies)

• Therefore entire Home Guard around 380TB

• JPEG2000 with lossless compression roughly halves storage required

Challenges and issues 2

• Closure: record for any person under 100 closed under data protection legislation and code of practice for archives

o Can be opened via FOI request if proof of death provided

• Some records also contain medical information – closed for 100 years from date of record

• What information to transcribe for Discovery and supporting FOI work?

• Also provenance information about digitisation process

Break down by birth yearYear No. of DoBs % of Total Records Cumulative total Cumulative %

Before 1913 40,925 49.43 40,925 49.41913 1,943 2.35 42,868 51.81914 2,106 2.54 44,974 54.31915 1,678 2.03 46,652 56.31916 1,650 1.99 48,302 58.31917 1,389 1.68 49,691 60.01918 1,233 1.49 50,924 61.51919 1,658 2.00 52,582 63.51920 2,947 3.56 55,529 67.11921 3,971 4.80 59,500 71.91922 4,978 6.01 64,478 77.91923 6,065 7.32 70,543 85.21924 5,742 6.93 76,285 92.11925 4,200 5.07 80,485 97.21926 1,730 2.09 82,215 99.31927 264 0.32 82,479 99.61928 21 0.03 82,500 99.61929 4 0.00 82,504 99.6

No DoB 296 0.36 82,800 100.0Total 82,800

After matching with death registersYear No. of DoBs Death confirmed by match Remainder

Before 1913 40,925 n/a n/a1913 1,943 1,221 722 62,073 75.01914 2,106 1,259 847 62,920 76.01915 1,678 1,006 672 63,592 76.81916 1,650 980 670 64,262 77.61917 1,389 812 577 64,839 78.31918 1,233 714 519 65,358 78.91919 1,658 966 692 66,050 79.81920 2,947 1,613 1,334 67,384 81.41921 3,971 2,037 1,934 69,318 83.71922 4,978 2,391 2,587 71,905 86.81923 6,065 2,649 3,416 75,321 91.01924 5,742 2,279 3,463 78,784 95.11925 4,200 1,706 2,494 81,278 98.21926 1,730 684 1,046 82,324 99.41927 264 100 164 82,488 99.61928 21 7 14 82,502 99.61929 4 2 2 82,504 99.6

No DoB 296 0.0Total 82,800 20,426 Total matched

61,351 Total open

Where next?

• Developed Scanning and Transcription framework

o Also sets our basic standards for digitisation projects, even those outside framework

o Not intended to set standards for others, necessarily

• Naval personnel records

• 1939 National Registration records

QUESTIONS?

Further details of our image and metadata specifications can be found on our website at: http://nationalarchives.gov.uk/documents/information-management/digitisation-at-the-national-archives.pdf