Oh Time, Thy Pyramids! The Biodiversity Heritage Library and the Unchaining of the Universal Library(?)

MARTIN R. KALFATOVIC :: SMITHSONIAN INSTITUTION LIBRARIES :: INFORMATION FUTURES INSTITUTE :: 12 APRIL 2008

Oh Time, Thy Pyramids!

Martin R. KalfatovicSmithsonian Institution Libraries8 April 2008

The Biodiversity Heritage Library and the Unchaining of the Universal Library(?)


The cultivation of natural science cannot be efficiently carried on without reference to an extensive library

Charles Darwin, et al. (1847)

Darwin, C. R. et al. 1847. Copy of Memorial to the First Lord of the Treasury [Lord John Russell], respecting the Management of the British Museum. Parliamentary Papers, Accounts and Papers 1847, paper number (268), volume XXXIV.253 (13 April): 1-3. [Complete Works of Charles Darwin Online]


Over 250 years of systematic description of life

Systema naturae (10th ed. 1758) by Carl von Linné

Taxonomic Literature


Taxonomic descriptions must be published for the name to be valid

Publications must be available to the public through trusted sources

Libraries have been the traditional place



The cited half-life of publications in taxonomy is longer than in any other scientific discipline

* * * The decay rate is longer than in any scientific discipline

~ Macro-economic case for open accessTom Moritz



2003. Telluride. Encyclopedia of Life meeting

February 2005. London. Library and Laboratory: the Marriage of Research, Data and Taxonomic Literature

May 2005. Washington. Ground work for the Biodiversity Heritage Library

June 2006. Washington. Organizational and Technical meeting

August 2006. New York Botanical Garden. BHL Director’s Meeting.

October 2006. St. Louis/San Francisco. Technical meetings

February 2007. Museum of Comparative Zoology. Organizational meeting

May 2007. Encyclopedia of Life and BHL Portal Launch. Washington DC.

BHL Timeline



Structure of the Encyclopedia of Life

Serine Molecule


Serine Molecule

Synthesis CenterField Museum

BiodiversityHeritageLibrary

Species Pages & SecretariatSmithsonian

Education &Outreach

Smithsonian/Harvard

InformaticsMarine Biological

Laboratory & MOBOT


American Museum of Natural History (New York)

Field Museum (Chicago)

Natural History Museum (London)

Smithsonian Institution (Washington)

Missouri Botanical Garden (St. Louis)

New York Botanical Garden (New York)

BHL Members


Royal Botanic Garden, Kew

Botany Libraries, Harvard University

Ernst Mayr Library of the Museum of Comparative Zoology, Harvard University

Marine Biological Laboratory / Woods Hole Oceanographic Institution

BHL Members


BHL Members

University of Illinois, Urbana-Champaign (contributing member)

Scheme for addition of European and Asian partners underway

Additional categories of membership under consideration


BHL Focus: Literature




• Core literature pre-1923: 100 million pages (?)

• All pre-1923: 120-150 million pages

• All literature: 280-320 million pages



• 1.3 million catalogue records • 73% are monographs

(remainder are serials at title-level)

• 63% is English language material

• The next most popular language (9%) is German

• About 30% of material was published before 1923

BHL Collections


Vast, But Not Infinite

So, yes, there's a not insignificant amount of content, by some accounts it's even vast, but by

no means infinite!


Vast, But Not Infinite• How vast COULD it be ?

• 100 characters (Western European languages, plus spaces and some punctuation)

• Each line has 50 spaces• Each page is 40 lines long• Each book is 500 pages long• Total Books: 100 1,000,000

• Googolplex: 1 followed by a googol (10 100) zeros


Vast, But Not Infinite“I assumed you packed the library in 1,000 volume boxes, each box having a capacity of precisely one cubic meter. All space to the farthest known spiral galaxies would not hold the Universal Library. In fact, you would need this volume of space so often that the number of packed universes would be a figure with only some 60 zeros less than the figure for the number of volumes… The figure is not infinite, it is a finite figure.”

- Kurd Lasswitz, “The Universal Library.” 1901


Vast, But Not Infinitedsoydrebnefdsellnhvaaipevls oadisaedhoholnarmea dlhecrositnsoritln gdustoaeaeb ossibrseltlmlie ey f tkstmdd eoiicai ytjtymniemruincrerlrc kym sdgklemy ecc poslr nnpz lrmpdcgeebf tetlealnm ceekctn sa oseesese erlsede c rotoire nhndlidd irb nwsrpeedsrc tey a lbneeu epon e cv mm tjn tarreohaaosimnicuse lvyubsu gurec egrll gnruveielrreuie w izbegii tckrne aseieoogrmia hmiuhtydaesdat uketwe te mnha egs foueirr tirstnnyriytuodnar inaon itaonlaai osn ral fd nurroysceo rteepcrn nrswrdstlr dt beeod sracpope Oh tiempo tus pirámides irhesrksesan ueada cte osamnec u gn bri seo aaircdgir kgielddaltneasfsdi n rwtlira etgtu erdoi nnte poycd rmogn t rdaebodpc ha tcl asrlceao usc tiosdrbaesci retiigvnooneeoe e m a b y fcoiihlha dcguo cdiem o pnz aitfehn sigtssipc isa seemue edk aerbgelgninbe n d aboaarlsdpd ele felr

“La biblioteca de Babel” by Jorge Luis Borges, 1941


In any well-appointed Natural History Library there should be found every book and every edition of every book dealing in the remotest way with the subjects concerned.

Charles Davies Sherborn, Epilogue to Index Animalium,

March 1922

A Universal Library for Life


Uh, well, we did it. We're done. Glad to have been of assistance

Thanks for the fish!



Status Update: Statistics 3,047 titles completed 7,669 volumes 2,945,143 pages in portal ~5.5 million pages scanned Three 10 Scribe station

centers in operation (Boston, New York, Washington)

Two 1 to 2 Scribe stations operations (Smithsonian, London, Urbana-Champaign)


Status Update: Encomiums

It's beautiful, with intimidating scope, and it makes me feel optimistic just musing on the implications of how librarians and scientists and IT wizards and artists have gathered valuable images and text (multilingual) and made them freely available in this elegant interface. http://tinyurl.com/5mdtyq


Status Update: Encomiums

I’ve been trying for weeks to think of something interesting and relevant to say about this way cool database, the Biodiversity Heritage Library, but I haven’t really come up with anything more than “this is way cool! Go look at it!”. http://tinyurl.com/6ctfgz


Status Update: Today1) Proven the concept of mass scanning of general collections

2) Proof of concept of automated structural markup done in collaboration with Penn State and the Internet Archive

3) Built proof of concept portal on proprietary (.Net) environment

4) High levels of OCR accuracy in late 19th and 20th century printing

5) Applied taxonomic intelligence (species name finding) across millions of pages against nearly 11 million names in NameBank

6) Data mining BHL for other bioinformatics projects (EOL)

7) Obtained buy-in from a diverse group of learned societies for the BHL opt-in copyright model

8) Support and encouragement from our traditional scientific and bibliophile audiences

9) Collaboration with a international group of competitive organizations


Status Update: Tomorrow + n1) Get equal cost efficiencies and speed for special collections

2) Nail down automated structural markup to a high level of accuracy

3) Port the portal from .Net to Fedora

4) Improve OCR for legacy publications in multiple languages with a minimum of human intervention

5) Broaden the use of the taxonomic intelligence algorithm and develop the iterative processing of BHL text

6) Data mining BHL for other bioinformatics projects (?????)

7) Work with commercial publishers for fair and equitable use of their publications

8) Expand audiences through social networking and repurposing content for new audiences

9) Expand the consortium to bring in more partners, and more partners in Europe, Asia and the developing world


So, uh, where's the Librarian?

The dodo is an extinct flightless bird, once a native of the island of Mauritius. The dodo was brilliantly adapted to its environment.

... but the environment changed ...

First described in 1598, it was gone by 1681. Rapid change without adaptation leads to extinction.



Changed Priorities AheadFor the Snark's a peculiar creature, that won't

Be caught in a commonplace way.

Do all that you know, and try all that you don't:

Not a chance must be wasted to-day!

~ Lewis Carroll, The Hunting of the Snark (1874)


Change: Reasonable or Unreasonable?

The reasonable man adapts himself to the world; the unreasonable man persists in trying to adapt the world to himself. Therefore, all progress depends on the unreasonable manGeorge Bernard ShawMan and Superman, “Maxims for Revolutionists” #23


What Should a Library Be?

“Ubiquitous and transparent”~ Tom Garnett


What Should a Librarian Be? Data curator. Just as

librarians have traditionally carried forward the printed text and helped to see it through the centuries, this role must now include the more transient data files generated by scholarly output

Application developer. If the library is ubiquitous and transparent, how does that data interact with the ambient world?


What Should a Librarian Be? Bibliophilic

Fetishist. Many books do have an intrinsic tactile value, they need to be seen, touched, preserved as objects of desire, not containers of information. There will be a need for the Librarian as curators in the Museum of the Book


What Should a Librarian Be?In the anticipated symbiotic partnership, men will set the goals, formulate the hypotheses, determine the criteria, and perform the evaluations. Computing machines will do the routinizable work that must be done to prepare the way for insights and decisions in technical and scientific thinking. Preliminary analyses indicate that the symbiotic partnership will perform intellectual operations much more effectively than man alone can perform them. ~ J.C.R. Licklider. 1960. Man-Comptuer Symbiosis


The world has arrived at an age of cheap complex devices of great reliability; and something is bound to come of it

- Vannevar Bush (1945)

And so ...






Thanks ... for sticking around!


Biodiversity Heritage Libraryhttp://www.biodiversitylibrary.org/

Biodiversity Heritage Library Bloghttp://biodiversitylibrary.blogspot.com

Encyclopedia of Lifehttp://www.eol.org/

Smithsonian Institution Librarieshttp://www.sil.si.edu/

Universal Biological Indexer and Organizerhttp://www.ubio.org/

LINKS


Thanks to:

Chris Freeland, Missouri Botanical Garden

Cathy Norton, MBL/WHOI

Tom Garnett, The Biodiversity Heritage Library

The staff at the Internet Archive

Images from

Map of Online Communities from XKCD (http://xkcd.com/256/)

The Galaxy of Images, Smithsonian Libraries (www.sil.si.edu/imagegalaxy)

Visualization of the Library of Babel(http://jubal.westnet.com/hyperdiscordia/library_of_babel.html)

CREDITS

Technology

Oh Time, Thy Pyramids! The Biodiversity Heritage Library and the Unchaining of the Universal Library(?)