View
219
Download
0
Category
Tags:
Preview:
Citation preview
Danish Legal Deposit on the Internet:
Current Solutions and Approaches for the Future
ECDL, September 2001by Birgit N. Henriksen
Head of Digitization and Web DepartmentThe Royal Library, Denmark
bnh@kb.dk
Presentation outline
•Since 1998 selection based archiving (production)
•netarchive.dk (new project, multiple archiving strategies , 2001-2002)
•Nordic Web Archive (project 2000-2001, access to web archives)
Three different initiatives:
The Danish Legal Deposit Law
•1697: All printers in royal and ducal lands must deposit
•1703: Only printers in Copenhagen have to deposit
•1781: All printers in royal and ducal lands must deposit
•1902: All printed materials to be deposited
•1927: Posters and some types of ephemera excluded
•1997: All published works to be deposited
The law from 1997 covers
any work published in Denmark regardless of medium
“work”: a delimited quantity of information which must be considered a final and independent unit
“published”: when … copies of the work have been placed on sale or otherwise distributed to the public
Types of Net Publications
Static included (only periodically updated) •monographs•periodicals
Dynamic excluded (continuously updated) •Databases•homepages
How do we get the material?
•Download based on notification
NOT
•Harvesting the Danish domain•Delivery of works (a collection of files) from the individual publishers
Registration
•WHOthe person in charge of the technical completion of the digital copy
•HOW by filling out a form at http://www.pligtaflevering.dk
Download - workflow
The staff at the Danish Department, The Royal Library
•determines whether a publication is covered by the law
•if yes, downloads all files belonging to the work•checks downloaded work•catalogues and classifies the work in the OPAC (only periodicals)
•transfers work to archival server
(server mirrored nightly to State and University Library, Århus)
System Environment
Server witharchive
Web-server withRegistration system
Firewall/Router
Internet
Collecting System
Pc’s with restricted access to archive
KB’sLAN
REX
Free access
Controlledaccesss throughaccessthrough firewall
Server witharchive
Firewall/Router
Internet
PCs with restricted access to archivepligtafleveret
SB’sLAN
SOL Controlled accessadgangthrough firewall
Daily mirror from KB to SB
SB
KBRoyalLibrary
Web-server
Free access
Domain names in .dk domain
# of sub-domains
Registered in .dk May 12’th 1999
96.371
Registered in .dkJune 12’th 2001
301.730
Represented in archive June 12’th 2001
< 1000
Volume in archived material
June 1999 June 2000 June 2001
# net publications
958 5424 9175
#RepresentationsRepr./net pub.
13061.36
66191.22
116071.26
#Files – total Files/net pub.
87.88667.3
346.68552.4
569.15049.0
#Bytes – total 1,66 Gbyte 12,0 Gbyte 18,2 Gbyte
Monographs vs Periodicals
BeforeJuly 1st 1999
BeforeJuly 1st 2000
BeforeJuly 1st 2001
# % # % # %
Monographs
642 67 1594 29 2850 31
Periodicals(issues)
316 33 3830 71 6325 69
Public vs. Private Publishers
June 1999 June 2000 June 2001
# % # % # %
Public 648 68 3985 71 6200 67,5
Private 304 32 1430 26.4 2975 32,5
Staff resources
Man Years Paid hours per publication
Comments
1998 2,3 12,75 System being developed and set up
1999 1,9 1,2 Downloading, cataloguing and classifying all publications
2000 1,3 0,6 Downloading all, cataloguing and classifying periodicals
MimeType Statistics – % of collected files
June 1999
June 2000
June 2001
TEXT/HTML
56,0 58,6 % 59,3 %
Image (GIF, JPEG, PNG)
41,8 % 38,4 % 37,9 %
PDF 1,3 % 1,6 % 1,7 %
Other formats
0,9% 1,4 % 1,1 %
Three generations using the internet
1st (age 74) 2th (age 40) 3th (age 10-15)
Professionel life (Work/ school related)
Professional online periodicals /portals
Professional online periodicals /portals
Product informationInstitutions and organisations Newsgroups
Uncritical all available material
Entertainment
Just surfing around
AuctionsGame servicesBizarre websitesNewsgroups
EventsGame servicesGimmicksChat services
Searching for information
Search enginesNewsMunicipal sites
Search engines (including cashed web pages)News and media/portalsState- and municipal sitesProduct databases
Search engines
Special interests
HomebankingStock exchange
Homebanking and info related to family economyE-commerceOrganisationsSeasonal interests
Sport clubs (results)Live role play
The modifications from 1902
•Brochures and advertisements•Catalogues•Election campaign material•Club/organisation magazines •Songs•Scouting magazines, church newsletters•Maps•Portraits•Art prints
Brouchers
Online services like krak.dk
Organisation websites
Newsletters/minuts on websites
Product databases/portals
Net Art
Problems related to the notification concept
•Lack of notification of multiple representations of a publication
•Lack of notification of new versions
Problems related to technical issues•Errors or inconsistencies in the published files
•Java applets – no solution at the moment
•Found solutions on previous problems:•Documents with java scripts •Data behind forms•Data behind username/password logins
•Cookies-based session handling•SSL encryption
Gains if harvesting is used
•Better coverage of Denmark outside the public sphere
•Updated versions – also for static publications
• New trends on the net as soon as they appear
Why not only harvesting?
•Programs and plug-ins are difficult to keep track on
•Harvesting is not always possible (e.g.. streamed and web casted material)
•Harvesting may not give a useful result- technical problems (java, interactive sites)
- personalised sites•Harvesting may produce a collection of documents that have never existed on the net
•Harvesting may not always give the best format for long-time preservation
Collections made by harvesting
•Are not complete – previous slides
•No robot will never be able to make a ’true’ snapshot – the snapshot contains a mix of documents that have never been published together at the same time – a ’fake’
Archive for Danish Literature
• www.adl.dk from 1. October 2001
•All full texts are structured in XML on work level
•The XML is loaded to a database•The database performs the web publishing in well-formed HTML on a page level
What do we prefer to archive and for what purpose?
Birte Christensen-Dalsgaard: Archive Experience, not Data
User Interface
Service Layer
Data Layer
Library SystemXML parserChatEtc.
Databases:CataloguePapers and articlesFinansial InformationEtc.
Database publishing,
Web Archiving Conference, CPH June 2001
•Focus: User Expectations to webarchiving in DK
•Brought together :•members of the user community, scholars as well as scientits
•member from the organisations traditionally in charge of preserving oral and written material
•members with technical knowledge
•Proceedings (UK) – netarchive.dk
Web Archiving Conference, CPH June 2001
•Sholars & scientist: •Archive the dynamic part of the web•Focus on archiving
• the content• the context• the evidence of use
•Archivists: •Use different archiving approaches •New methods for archiving dynamic material
•Budgets for making snapshots and making selective collections are comparable
Birte Christensen-Dalsgaard: 3 dimensions - duration
Real time dialog
Published, static
Sig
nal l
ifet
ime
Hourly Update
• Book-like publications• Scientific Journals• News-sites• Chat
Birte Christensen-Dalssgard: 3 dimensions - Permanent value
Transient
Persistent
Perman
ent V
alue
• What is worth preserving?• Quality vs. Representative
Birte Christensen-Dalsgaard: Background - Nature of Information
InteractivityStatic Dynamic
Transient
Persistent
Perman
ent V
alue
Real time dialog
Published, static
Sig
nal l
ifet
ime
Hourly Update
Birte Christensen-Dalsgaard: Domain of different harvesting methods
InteractivityStatic Dynamic
Transient
Persistent
Perman
ent V
alue
Real time dialog
Published, static
Sig
nal l
ifet
imeLegal Deposit, DK
Hourly Update
Accumulative harvestingSnapshot
Birte Christensen-Dalsgaard: What is missing?
InteractivityStatic Dynamic
Transient
Persistent
Perman
ent V
alue
Real time dialog
Published, static
Sig
nal l
ifet
imeLegal Deposit, DK
Hourly Update
Accumulative harvestingSnapshot
Accumulative
Snapshot
netarchive.dk (1)
InteractivityStatic Dynamic
Transient
Persistent
Perman
ent V
alue
Real time dialog
Published, static
Sig
nal l
ifet
ime
Process
Test different archival approaches and the subsequent usability of the archived material for research
netarchive.dk (2)
•Pilot project testing different archival approaches and the subsequent usability of the archived material for research
•Project partners:•State and University Library, Aarhus •Centre for Internet Research •The Royal Library
•With economic support from the Danish Electronic Research Library (DEF)
•Period: August 2001 – July 2002•Case: Danish municipal elections November
2001
netarchive.dk (3)
•Which materials with •What frequency?•Collection method?•Which software?
•How should the collection of materials be organized and how should it be stored?
•How should obsolescence of data formats be dealt with?
•How should access be given?•Budgets for collecting and storing
netarchive.dk (4)
Net material covered by netarchive.dk
• net activities from existing news media (newspapers, radio, TV (both national, regional and local media))
• political parties official pages, national and local• individual politicians’ personal pages• official (county) municipal pages• voters’ personal pages• »local themes«- pages• special interest organisations• portals in the broadest sense• opinion polling firms• public emails/ press releases• news groups / usenet• net-conferences and chat
How do we catch the missing part?
Process rather than material – ‘Filming’ the net through a browser
Goal:
Catch chronological series of displayed WebPages
Tools to take into consideration:
•Business intelligence tools•Tools used in usability laboratories
…
Nordic Web Archive (NWA)
•Establish a Danish test archive in order to participate in NWA
•Software: NEDLIB robot•Status 1/9 2001:
•Archiving started 20/8 2001•1.9 mio documents •43 GB uncompressed data
Recommended