Formats, metadata, standards and vocabularies for …...2019/06/03 · Introduction Good practices Metadata mapping Conclusion Formats, metadata, standards and vocabularies for national

IntroductionGood practices

Metadata mappingConclusion

Formats, metadata, standards andvocabularies for national bibliographic

databases

Ivanovic Dragan

University of Novi Sad

ENRESSH Training school

[email protected] Formats, metadata, standards and vocabularies



Outline

1 IntroductionMy UniversityQuestions/challenges

2 Good practicesDesignVocabularies, authority control and identifiersData use

3 Metadata mappingIntegrated European Publication Information ServiceMapping processMapping tools

4 Conclusion




My UniversityQuestions/challenges

Outline




4 Conclusion






The first faculty in Novi Sad was founded in 1954The University of Novi Sad was founded on 28th of June1960Today, UNS represents an autonomous institution foreducation, science and arts

















Rectorate building





University of Novi Sad Cities





Outline




4 Conclusion





Metadata vs data

Metadata commonly are understood as ‘data about data’The content of bibliographic databases are bibliographicmetadata referring to research outputResearch outputs (pdf, xls, etc) represent data, whilebibliographic databases store metadata - data aboutresearch outputsThat is especially case if you are looking at bibliographicdatabase as source for publications discovery (informationretrieval)However, if you are looking at bibliographic database assource for bibliometrics analysis or research evaluation,then content of database could be called data





Metadata vs data






Metadata vs data






Metadata vs data






Metadata vs data






Which metadata vs in which format

Which metadata should be preserved in bibliographicdatabase is one question

purposeneedsnational evaluation rule-booksmandatory vs optionalrich vs light

In which format metadata should be preserved is theanother question

how to select best format for preservation?structured database vs csv vs xml vs json, etcmetadata schema




































































Standards

Which standard formats to be supported for export?Which protocol should be implemented for harvestingmetadata from/to the system?OAI-PMH, OpenAIRE guidelines, SRU/W, etc.





Standards






Standards






Vocabularies

Not related to the structure of metadata or formatRelated to the content - allowed values/terms for metadataPublication types?Question very important for interoperability of systemsIf we speak languages which have similar rules andstructures (nouns, verbs, etc), but we use different terms -can we communicate?





Vocabularies






Vocabularies






Vocabularies






Vocabularies





DesignVocabularies, authority control and identifiersData use

Outline




4 Conclusion





Recommendation 3

Define the data model and/or metadata schema, takinginto account the database’s purpose and recognizedstandardsEnsures that the system can fulfil its purpose, whilefollowing recognized standards simplifies the work and canbenefit interoperabilityMajority of bibliographic standards do not take evaluationpurposes into account





Recommendation 3






Recommendation 3






Recommendation 4

Select a suitable technical solution and design thetechnical structure of the databaseContributes to the functionality, performance, andmaintainability of the databasePurpose, budget, the estimated number ofrecords/requests, contemporary technologies/databases,experience of staff - technicians and librarians should betaken into account





Recommendation 4






Recommendation 4






Outline




4 Conclusion





Recommendation 12

Maintain authority lists for publication channelsContributes to the accuracy of data on publicationchannels and the functionality of the databaseJournals, conferences, publishers, etclocal/external identifiers, title, etc.





Recommendation 12






Recommendation 12






Recommendation 12






Recommendation 13

Maintain authority lists for authors and organisationsContributes to the accuracy of data on authors andorganisations and the functionality of the databaselocal/external (ORCID) identifiers, inside/outside databasescope (national/international person/organization), name,history/variations of names, etc.





Recommendation 13






Recommendation 13






Recommendation 14

Use international persistent identifiers where possibleIncreases interoperability with other national andinternational databases and systemsORCID, DOI, ISSN, ISBN, etc





Recommendation 14






Recommendation 14






Recommendation 15

Use as much as possible terms from well-known andstandardized vocabulariesEnhances the interoperability and functionality of thedatabaseLanguages’ and countries’ codes, publication types andscientific fields (problematic!)





Recommendation 15






Recommendation 15






Recommendation 16

When developing own vocabulary, consultstakeholders and relevant expertsEnsures that the vocabulary is usable and captures all usecases"From scratch" or extended stadard vocabulary,human/machine readable vocabulary, SKOS semanticrelations vocabulary





Recommendation 16






Recommendation 16






Outline




4 Conclusion





Recommendation 22

Specify procedures for data accessEnhances the usability of the databaseuser interface, API, protocol(s) for harvesting/federatedsearch, etc.Take into account licences (GDPR), needs of differentusers, different ways to transfer data





Recommendation 22






Recommendation 22






Recommendation 22






Recommendation 23

Offer research output metadata in multiplerepresentationsEnsures that users with different needs and preferencescan efficiently use the datauser profiles and preferences are different, option tocustomize the display and format, export to standardizedformats, XML, Bibtex, JSON, RDF - semantic web (FAIRprinciples)





Recommendation 23






Recommendation 23






Recommendation 24

Provide access to the data through a functional userinterfaceEnables consulting the database in various ways andincreases transparencySearching (basic and advance), browsing, downloading





Recommendation 24






Recommendation 24






Recommendation 25

Facilitate automated access to the data through an APIor a metadata harvesting protocolEnables automated and efficient use of the databaseREST, JSON vs XML, authentication and authorization(A1.2 FAIR principle), OAI-PMH, OAI-ORE, etc.





Recommendation 25






Recommendation 25






Recommendation 26

Enable crawling of bibliographic records by websearch enginesEnsures that database content can be found throughacademic search enginesCrawlers, Robots Exclusion Protocol (robots.txt), specificcrawling guidelines (Google Scholar)





Recommendation 26






Recommendation 26





Integrated European Publication Information ServiceMapping processMapping tools

Outline




4 Conclusion





Why

EU services - evaluation for EU funded projects, reporting,etcPublications/outputs discovery





Why

EU services - evaluation for EU funded projects, reporting,etcPublications/outputs discovery





Approaches

Distributed vs CentralizedThe distributed approach makes it easier to have completeinformation in real-time, since it does not requirepropagation of updates to the central catalogue - federatedsearch SRU/WHowever, for data-intensive operations, the centralized approach doesn’t have the problem of querying multiple sites,and has more complete overview of the data availablewhen executing operations - harvesting data OAI-PMH





Approaches






Approaches






Distributed SRU/W based approach





Centralized OAI-PMH based approach





Centralized approach

Data provider (nodes) and Service provider (IntegratedEuropean Publication Information Service)Protocols for harvesting metadata should be implementedon both side (OAI-PMH, ResourceSync, etc.)Target metadata format(s) should be selectedAll nodes (partner systems) have to export metadata to (atleast one) target metatada formatAll nodes (data providers) have to map its metadata totarget metadata format





























Outline




4 Conclusion





Process

Matching source schema entities to target schema entitiesMatching source attributes to target attributesExpressing the mapping in some format/languageImplementation of mappings rules in source system





Process






Process






Process






Actors

1 Expert(s) for source schema2 Expert(s) for target schema3 Expert(s) for source/target vocabularies4 Software developers





Actors






Actors






Actors






Collaboration

Collaboration between schema/vocabularies experts isusually not a problemHowever, collaboration between those experts andsoftware developers could be a problem

Don’t "speak" the same languageThe process of implementation of mappings rules in sourcesystem is error-prone and time-consumingCan we automate the process? Can complete process beperformed by schema/vocabularies experts?





Collaboration







Collaboration







Collaboration







Collaboration







Outline




4 Conclusion





Why

The process of matching and mapping implies a lot of timeand effort from experts on the source and target schemataTo simplify and accelerate the process, a tool needs to beadopted for automationBesides enhancement of mapping development, such atool should make the implementation of mappings moreeffective and shareable





Why






Why






X3ML toolkit

The X3ML toolkit with the 3M editor could be used toautomate the mappingsThis toolkit allows several steps and tasks of the process ofharvesting, matching, mapping and integrating the datafrom the sources to the target catalogue3M (one component of X3ML toolkit) guides the user tospecify the schemata matchings and the instancesgeneratorsX3ML engine (the another X3ML toolkit component)automatically transforms the source data into target format





X3ML toolkit






X3ML toolkit






X3ML toolkit






X3ML toolkit





3M

3M eases the process of matching by parsing andanalyzing the source and target schemata, thus allowingauto-completion when selecting the entities and propertiesto be matchedThis mechanism speeds the matching process and allowsnon-expert users (users that do not have an extendedknowledge of the whole schema) to define a matching





3M

3M eases the process of matching by parsing andanalyzing the source and target schemata, thus allowingauto-completion when selecting the entities and propertiesto be matchedThis mechanism speeds the matching process and allowsnon-expert users (users that do not have an extendedknowledge of the whole schema) to define a matching





3M

The description of the matching is homogenized, whichreduces the misunderstandings between experts andsoftware developers3M also includes a versioning mechanism that allowsstorage of different versions of the matchingsThe X3ML engine can be used exhaustively to test anyversion of the matching at any time just by providing asample of data and applying the transformationThe result is immediately available and can be analysed tocheck for defaults or implemented corrections





3M






3M






3M






3M demo

Dublin Core is the source - linkCERIF RDF should be the resulthttps://isl.ics.forth.gr/3MMapping Project - ENRESSH Dublin Core to CERIF 1.6


https://www.amazon.com/Through-My-Eyes-Ruby-Bridges/dp/0590189239

https://isl.ics.forth.gr/3M




3M demo








3M demo








3M demo







Summary

Part of Manuel of good practices: CREATING ANDMAINTAINING A NATIONAL BIBLIOGRAPHIC DATABASEFOR RESEARCH OUTPUT has been presentedIn order to improve reusability of metadata, the systemcould be a data provider and could export metadata tosome Service Provider(s)Source metadata schemata should be mapped to targetmetadata schemata




Summary





Summary





Questions

Thank you for your attention!!!If you have any questions, please do not hesitate to

ask me during the schoolcontact me by email - [email protected]


Documents

Formats, metadata, standards and vocabularies for …...2019/06/03 · Introduction Good practices Metadata mapping Conclusion Formats, metadata, standards and vocabularies for national