CLiMB: Computational Linguistics for

Metadata Building

Center for Research on Information Access

Columbia University Libraries

Judith L. Klavans 3

Four areas

• Collections

• Technology

• Users and Uses

• Interface Tools

November - Waters/Lodato for next steps

June 2003 to November 2003

Judith L. Klavans 4

Goals of December Meeting

• Conclusion from last meeting: CLiMB still a research platform

• Explore potential platforms for testing CLiMB tools

• What is the possibility of working with ArtStor?

• Changes in Personnel

Judith L. Klavans 5

Problems in Image Access

Cataloging digital images Traditional approach:

manual expertise labor intensive expensive

Can automated techniques help?

Judith L. Klavans 6

CLiMB Technical ContributionCLiMB will identify and extract

• proper nouns• terms and phrases

from text related to an image:

September 14, 1908, the basis of the Greenes' final design had been worked out. It featured a radically informal, V-shaped plan (that maintained the original angled porch) and interior volumes of various heights, all under a constantly changing roofline that echoed the rise and fall of the mountains behind it. The chimneys and foundation would be constructed of the sandstone boulders that comprised the local geology, and the exterior of the house would be sheathed in stained split-redwood shakes. —Edward R. Bosley. Greene & Greene. London : Phaidon, 2000. p. 127

Judith L. Klavans 7

Can we harvest image descriptors?

Judith L. Klavans 8

• Collections

• Technology

• Users and Uses

• Interface Tools

Progress and Planning

Judith L. Klavans 11

Judith L. Klavans 21

Progress and Planning

• Collections

• Technology

• Users and Uses

• Interface Tools

Judith L. Klavans 22

Text Analysis and Filtering

1. Divide text into words and phrases

2. Gather features for each word and phrase • E.g. Is it in the AAT? Is it very frequent?

3. Develop formulae using this information

4. Use formulae to rank for usefulness as potential metadata

Judith L. Klavans 23

What Features do we Track?

• Lexical features– Proper noun, common noun

• Relevancy to domain– Text Object Identifier (TOI)– Presence in the Art & Architecture Thesaurus– Presence in the back-of-book index

• Statistical observations– Frequency in the text– Frequency across a larger set of texts, within and

outside the domain

Judith L. Klavans 24

Problem: Too much Data!

• How should the output be filtered?

• What filtering helps additional text

processing (e.g. for text segmentation)?

• What filtering matches what users think?

Judith L. Klavans 25

Techniques for Filtering

1. Take an initial guess• Collect input from users

• Alter formulae based on feedback

2. Use automatic techniques to guess (machine-learning)

• Collect input from users

• Run programs to make predictions based on given opinions (Bayesian networks, classifiers, decision trees)

3. The CLiMB approach: Use both techniques!

Judith L. Klavans 28

Next Steps

• Filter “given” information (already in catalogue record if you are lucky enough to have one!)

• What does CLiMB get that is new?

• How much is useful?

• What is the “cost”?

Judith L. Klavans 30

Progress and Planning

• Collections

• Technology

• Users and Uses

• Interface Tools

Judith L. Klavans 31

Formative Evaluation Meeting

• At the advice of External Advisory Board

• October 17, 2003

• Goals:– Get early feedback from many user types– Incorporate that feedback into CLiMB toolset– Help shape next steps

Judith L. Klavans 32

Formative Evaluation - Attendees

• CLiMB Project Team

- Judith Klavans - Roberta Blitz - Rebecca Passonneau - Angela Giral - Vera Horvath - David Elson - Bob Wolven - Stephen Davis - Mark Weber

• CLiMB: External Advisory Board - Jeff Cohen (Bryn Mawr) - Carl Lagoze (Cornell) - Merrilee Proffitt (RLG)

• Invitees - Robert Carlucci (Columbia) - Terry Catapano (Columbia) - Paula Gabbard (Columbia) - Deborah Kempe (Frick) - Doug Oard (UMd)

• Could not Attend– Tony Gill (Mellon)– Abby Goodrum (Syracuse)– Elisa Lanzi (Smith)

Judith L. Klavans 33

Results from Formative Evaluation

• Best – Humans select, CLiMB selects– Cordelia A. Culbertson

• Better - Humans select, CLiMB might not– Ludowici-Celadon Company

• Better – Humans might not but CLiMB selects– house, Tichenor house, most significant house

• Good – Humans do not select, CLiMB does not– problem, time

Judith L. Klavans 34

Research Questions

• Will CLiMB metadata help users get access to the digital images they want?

• Will these terms help catalogers provide this access?

• How well are the CLiMB tools performing in providing required metadata?

Judith L. Klavans 37

Use Results for Improvement

• Determine ways to better filter CLiMB

results

• Use input for improving ranking

Judith L. Klavans 40

Progress and Planning

• Collections

• Technology

• Users and Uses

• Interface Tools

Judith L. Klavans 41

Interface Tools

• Planning the new interface for image professionals to prepare CLiMB metadata from texts

• For catalogers / metadata specialists and visual resources professionals

• Goals– to provide a platform for a wider community

– to be able to collect feedback on CLiMB at a wider level

– to complete the CLiMB interface “deliverable”

Judith L. Klavans 42

Interface Tools – Stay Tuned!

• CLiMB toolset currently implemented with textual interface– Fully-functional shell

• New graphical user interface (GUI) can be built on top of existing codebase– Perl/Tk

• Design– Initiating design phase now– Consulting metadata and image specialists

Judith L. Klavans 43

Next Steps

• External Advisory Board– June 2004

• Select project directions

• Potential partners

Thank you!

www.columbia.edu/cu/cria