Metadata Interoperability Framework (MIF)

ELAG 2014

Naeem Muhammad

Sam Alloing

Agenda

• Who are we?• Europeana inside• Mapping and transformation service• Demo• Future

LIBIS

• Service provider for libraries, museums and archives• Part of university of Leuven (Belgium), but also

external partners• 19 team members

Europeana Inside• Goal is to facilitate the content delivery to Europeana

through CMS (=ECK)– Europeana is a portal for aggregation of cultural heritage objects

• 26 partners (Cultural institutions, software providers ( Adlib, Zetcom,… ))

• 30 months (April 2012 – September 2014)• Deliver 960000 records to Europeana• Special topic content re-ingestion

– Europeana enrichments send back to the CMS

Europeana Inside

Europeana Connection Kit (ECK)ECKECK local

Core ECK services• Metadata Definition• Set Manager• Statistics• PID Generation• Preview• Validation• Data push (Sword)/ data pull (OAI-PMH)• Mapping and transformation

Europeana Connection Kit (ECK)ECKECK local

LibisCoDe – example of ECK local

• Our version of the ECK local• A plugin for CollectiveAccess (

http://collectiveaccess.org/)– CollectiveAccess is an open source CMS – LibisCode is also open source

• Supports a LIDO, MARC and (future?) EAD installation profile.

LibisCoDe – How to use it

CollectiveAccess or other CMS

CollectiveAccess with LIBIScoDe Europeana

EuropeanaYour CollectiveAccess with LIBIScoDe

Convert your format to MARC/LIDO

Convert EDM to your format

ECK core

ECK core

Mapping and transformation service• Transforms one metadata format in another

– You can use a default mapping, this is only the transformation service

– Or create a mapping file with a custom mapping (mapping and transformation service)

– Implemented are MARC to EDM, LIDO to EDM• LIDO = XML format used by museum• EDM = RDF format from Europeana

• Why we created the service?– Service-oriented architecture– Different metadata formats needed to be supported

Input• MARC XML or LIDO• Mapping rule file

Output• Only EDM is supported

– RDF based format from Europeana– Core classes are supported (ProvidedCHO, Aggregation,

WebResource)– We will be adding Contextual classes shortly (Place, Agent,

Concept,…)

Mapping rules

• Example MARC– [command],[marc tag + subfield],[edm field]– Doesn’t use indicators at this moment, could change

Mapping rules

• Example MARC (2)– You only need to know the EDM field not the format

• For example edm:rights can be in multiple classes, but you just put edm:rights, the EDM class is derived

ProvidedCHO

WebResource

Mapping rules• Example LIDO

– No concise way to select the correct LIDO field, so we use XPATH-like expression

– [command],[LIDO path],[edm field]• Attributes are supported

Supported commands (1/2)

Action Definition Example

COPY Copies the value from a source field to a target field. If different source fields are defined to be copied to the same target field a new occurrence of the target field will be created

COPY,/lidoRecID,dc:identifier

APPEND Append a source field to a target field. The specified source field will be appended to the target field. If there is already metadata in a field. The information will be added to that field and not replaced. At the time of append extra information can be added to the value of the source field. In case no extra information is needed an empty space can be left.

APPEND,/category/term, some text ,dc:title

or

APPEND,/category/term, ,dc:title

SPLIT The source field will be split on the defined character and put in the target field. (In the example it will be split on space)

SPLIT,/descriptiveMetadata/eventWrap/eventSet/event/eventMaterialsTech/displayMaterialsTech, ,dcterms:medium

Action Definition ExampleCOMBINE Multiple source fields can be combined in one

target field. The fields are combined with a spaceCOMBINE, /category/term;/lidoRecID, dc:title

LIMIT Limit the number of characters in a source field LIMIT,/descriptiveMetadata/objectIdentificationWrap/inscriptionsWrap/inscriptions/inscriptionTranscription,9,edm:description

PUT Add a value to a target field. The value can contain commas, but no 2 pipes (||), because this is used as replacement character

PUT,"Koninklijke Musea voor Kunst en Geschiedenis, Brussel",edm:dataProvider

REPLACE Replaces a value in the source field with a replace string in the target Field.

In the example d'Histoire will be replaced by History in the edm:provider element

REPLACE,/administrativeMetadata/rightsWorkWrap/rightsWorkSet/rightsHolder/legalBodyName/appellationValue,d'Histoire,History,edm:provider

CONDITION With condition you can combine different actions and use a conditional flow. IF can be used to define conditional action. With nested conditions also ELSE IF can be used.The CONDITION actions starts and ends with curly brackets ‘{ ’. A new line is also mandatory.

CONDITION,{

IF[/descriptiveMetadata/eventWrap/eventSet/event/eventType/term,EQUALS,Production,DO(COMBINE, /category/term;/lidoRecID, dc:title)]

}

Supported commands (2/2)

Web service

• No user interface• Meant to be integrated in CMS or use a REST client• dmt.php/DataMapping/<provider>/<batch>/Transform

– POST request– Provider: This is the name of the organization making the

request.– Batch: This is the name of the batch submitted for

transformation.

Webservice - Parameters• records or record

– Zip file with multiple XML files or one XML with multiple records are supported (the mimetype needs to be correct)

• mappingRulesFile– Needs to have mimetype text/csv– Without this parameter you use the Transformation service

• sourceFormat– Needs to be LIDO or MARC

• targetFormat– Needs to be EDM

Web service - Response• Returns a request_id• Use it to download finished record(s)• Use status:

– 0: no request exist with this id– 1: not yet mapped/transformed– 2: ready to be fetched

Web service – Other (GET) requests• Status• Fetch

– Download records– Parameter is request_id

• List– List of supported metadata formats

Demo

• Three possibilities– Rest client tool (e.g. restclient)– Libiscode with input records generation + mapping– Libiscode with external records + mapping

Future?

• Add input formats (CSV, Filemaker XML, some custom XMLs…)

• Update/add output formats (add EDM contextual classes, add other formats)

• Extend/add/update actions• Add queuing (near future)• Add mapping interface (or integrate with MINT?)

Information.

Inspiration.Innovation.

LIBIS into-info@libis.be – www.libis.be+32 (0) 16 32 22 66De Croylaan 54 – PB 5592 – B-3001 Heverlee

Thank you !Questions ?