Digitisation
Mick Eadie
Visual Arts Data Service
The ‘input channels’ of digitisation (keyboard, scanner etc.)
are narrow and can only capture a partial representation of the original source
chose data model
digital objectschose digitisation method
identify sources to digitise
Source – Digitisation - Resource
PhotocopyPhotographRecording
OriginalSource
Copyof
Source
Item toDigitise
Sound,Movingimage
DigitalObject
2D Image3D Model
DigitalResource
Digital audio/movie recording
ScanDigital Camera3D Scan
OCRLine tracing
Digitisation Pathways
Users
Knowledge
Experience
Culture
Environment
HardwareSoftware(OS)(Network)
Digital ObjectsBinary Data
Data ModelsRelationships
The environment of a digital resource often receives the most attention, but it is the users and digital objects that are most important
Hardware and software selection should be based on the needs of the users and the types of digital objects to be used
Fit for Purpose: Digital objects must be created with their intended use/purpose of paramount importance
Elements of a Digital Resource
Digital Objects
• Text– Data stored as a stream of characters (numbers, letters, etc.)
• Image– Data primarily understood as a spatial pattern or shape– Bitmap and vector images/raster (bitmap) and vector spatial
data
• Time– Data primarily understood as a sequence through time– Audio and/or video (multimedia)
Text
• Essentially, numeric codes used by the computer to represent specific characters– Fonts must be designed to provide a visual image for each
code– Software must be designed to interpret the codes
• ASCII is the most well known text encoding scheme– 1 byte per character = 256 unique characters, primarily the
Latin alphabet– Other characters are handled by having multiple code pages– Each code page uses the same codes to represent different
characters• UNICODE is the replacement for ASCII
– 2 bytes to store each character = 60,000+ codes– Can represent characters from different alphabets
simultaneously as each character has a unique code
Text Transcription
• Advantages:– Low overhead to start transcription: person, keyboard,
document– Hand-written documents can be transcribed– A transcriber can follow complex disorganised
documents• Issues:
– Slow and expensive– Human error
• Good practice:– Double entry (two transcribers both enter the same
document and the transcriptions are checked for differences)
– Keep copies of originals with transcriptions (preferably as digital images as this make post-transcription checking simple and quick)
Optical Character Recognition
• Advantages:– Automatic, suitable for digitising large numbers of
documents– Highly accurate for clean, clear type written documents
• Issues:– Current technology is very poor on hand-writing– Complex document layout can become scrambled
• Good practice:– Proof-read, spell check OCR output for errors– Provide image of page with text so users can check the
text themselves
Bitmap (Raster) Images
• The image is made up of many pixels• Each pixel stores information about its colour• The standard archival file format is uncompressed TIFF
Resolution
• Resolution is often expressed as dots per inch (dpi)• More accurately pixels per inch (ppi)• The ‘frequency’ at which samples are taken by the capture
device from the original source
Common misconceptions about ppi• Not an indicator of image size or quality• Unless we know the size (inches, cms) of the original• A better guide to digital image size is pixel dimensions e.g.
2000 x 3000 pixels, which allows us to work out the size of the image we will output to monitor or printer
• No of pixels/output res = output size
Scanners and Digital Cameras
• Advantages:– Accurate(?) visual representation of the source
• Issues:– Text and logical structure of a document is not captured
(can be through OCR or line tracing)• Good practice:
– Capture master images at appropriate resolution and bit depth
– Check the optical resolution of the scanner (avoid interpolated resolution)
– Check the colour resolution (bit depth)– Check scanning time– Record details of scanner settings and any image editing
done afterwards
Vectors
• A point represents an exact location in two or three dimensional space
• Two points define a line
• A series of connected lines define an area
x,y x,y,z
Vector Data
• Advantages:– Can be zoomed (c.f. bitmap images)– Allows spatial analysis (spatial statistics, network
analysis)• Issues:
– Precision versus accuracy (detail versus truthfulness)– Scale versus resolution
• Good practice:– Ensure polygon topology (the polygons each line
belongs to) is stored
Digital Audio
• Human hearing– Frequency (pitch) - 20Khz to 20,000Khz– Intensity (loudness) - 0 and 120Db
• Full sound reproduction requires digitisation at more than 40,000 samples a second (44,100 is a common standard)– NYQUIST rate: for lossless digitisation, the sampling
rate should be at least twice the maximum audio frequency
• One second of good quality uncompressed digital sound is equivalent to ¼ of the Complete plays of Shakespeare– MP3 offers good quality compressed (lossy) files
• Midi: not a digital recording of actual sounds, but a digital sample ‘library’ of how musical instruments sound
Digital Moving Images
• 1 second of uncompressed good quality digital video (without sound) is equivalent to about ¾ of the complete plays of Shakespeare
• MPEG - The Motion Pictures Experts Group standards are the most popular compression standards– The three standards, MPEG-1, MPEG-
2, MPEG-4• Compression basically works by selecting
key frames and only recording changes between the frames (but it gets a lot more complicated!)
Data Models
• A data model is a set of rules that defines a particularly way
of organising a collection of digital objects
• List, one item follows another
• Tree, each item can have several children
• Sets, items belong to one or more groups
• Geography/geometry, items are located using a co-ordinate system
Selecting a Data Model
• To be useful, digital objects must be:– Arranged according to the rules of an appropriate
data model– Stored in a file format that can represent the data
model– Accessed with software that understands the file
format and the data model, and can present the data in an appropriate way
• When selecting a data model– Consider the ‘natural’ organisation of your source– Consider what method of organisation will be
familiar to your users– Consider the method of organisation that best fits
your purposes• Then seek specialist advice if you need it!
Selecting Software
• Selecting the right data model is more important than selecting a particular piece of software
• Pick software that works with your preferred data model (can perform the right tasks)– Don’t use a webpage editor as a database– Don’t use a word processor as a spreadsheet
• Avoid little-used software with proprietary features• Look for software with lots of export and import options• Look for software that supports important standards
– Trees markup XML (SGML)– Sets relational databases SQL– Coordinates CAD or GIS less clear, use file formats
like DXF, ESRI shape files
Digitisation: a Balancing Act
• Successful digitisation involves several trade-offs:– Amount and detail versus time and cost of digitisation– Complexity of the digital resource versus ease of use– Flexibility of the digital resource versus suitability for a
specific use– Digitisation with current technology versus future
possibilities
• Your project should be guided by a firm understanding of the source and the intended purpose of the digital resource– Do not exceed available support (financial, technical,
labour)– Minimise the loss of information from the original during
the digitisation process– Keep information that tracks the origin and history of the
digital resource with the digital resource
Where to get more advice
AHDS Guides to Good Practice serieshttp://vads.ahds.ac.uk/guides/index.html
Technical Advisory Service for Images (TASI)http://www.tasi.ac.uk
Text Encoding Workshopshttp://www.ota.ahds.ac.uk
BUFVC Workshopshttp://www.bufvc.ac.uk