Formats, metadata, standards and vocabularies for …...2019/06/03 · Introduction Good practices...

IntroductionGood practices

Metadata mappingConclusion

Formats, metadata, standards andvocabularies for national bibliographic

databases

Ivanovic Dragan

University of Novi Sad

ENRESSH Training school

dragan.ivanovic@uns.ac.rs Formats, metadata, standards and vocabularies

Outline

1 IntroductionMy UniversityQuestions/challenges

2 Good practicesDesignVocabularies, authority control and identifiersData use

3 Metadata mappingIntegrated European Publication Information ServiceMapping processMapping tools

4 Conclusion

My UniversityQuestions/challenges

Outline

4 Conclusion

The first faculty in Novi Sad was founded in 1954The University of Novi Sad was founded on 28th of June1960Today, UNS represents an autonomous institution foreducation, science and arts

Rectorate building

University of Novi Sad Cities

Outline

4 Conclusion

Metadata vs data

Metadata commonly are understood as ‘data about data’The content of bibliographic databases are bibliographicmetadata referring to research outputResearch outputs (pdf, xls, etc) represent data, whilebibliographic databases store metadata - data aboutresearch outputsThat is especially case if you are looking at bibliographicdatabase as source for publications discovery (informationretrieval)However, if you are looking at bibliographic database assource for bibliometrics analysis or research evaluation,then content of database could be called data

Metadata vs data

Which metadata vs in which format

Which metadata should be preserved in bibliographicdatabase is one question

purposeneedsnational evaluation rule-booksmandatory vs optionalrich vs light

In which format metadata should be preserved is theanother question

how to select best format for preservation?structured database vs csv vs xml vs json, etcmetadata schema

Standards

Which standard formats to be supported for export?Which protocol should be implemented for harvestingmetadata from/to the system?OAI-PMH, OpenAIRE guidelines, SRU/W, etc.

Standards

Vocabularies

Not related to the structure of metadata or formatRelated to the content - allowed values/terms for metadataPublication types?Question very important for interoperability of systemsIf we speak languages which have similar rules andstructures (nouns, verbs, etc), but we use different terms -can we communicate?

Vocabularies

DesignVocabularies, authority control and identifiersData use

Outline

4 Conclusion

Recommendation 3

Define the data model and/or metadata schema, takinginto account the database’s purpose and recognizedstandardsEnsures that the system can fulfil its purpose, whilefollowing recognized standards simplifies the work and canbenefit interoperabilityMajority of bibliographic standards do not take evaluationpurposes into account

Recommendation 3

Recommendation 4

Select a suitable technical solution and design thetechnical structure of the databaseContributes to the functionality, performance, andmaintainability of the databasePurpose, budget, the estimated number ofrecords/requests, contemporary technologies/databases,experience of staff - technicians and librarians should betaken into account

Recommendation 4

Outline

4 Conclusion

Recommendation 12

Maintain authority lists for publication channelsContributes to the accuracy of data on publicationchannels and the functionality of the databaseJournals, conferences, publishers, etclocal/external identifiers, title, etc.

Recommendation 12

Recommendation 13

Maintain authority lists for authors and organisationsContributes to the accuracy of data on authors andorganisations and the functionality of the databaselocal/external (ORCID) identifiers, inside/outside databasescope (national/international person/organization), name,history/variations of names, etc.

Recommendation 13

Recommendation 14

Use international persistent identifiers where possibleIncreases interoperability with other national andinternational databases and systemsORCID, DOI, ISSN, ISBN, etc

Recommendation 14

Recommendation 15

Use as much as possible terms from well-known andstandardized vocabulariesEnhances the interoperability and functionality of thedatabaseLanguages’ and countries’ codes, publication types andscientific fields (problematic!)

Recommendation 15

Recommendation 16

When developing own vocabulary, consultstakeholders and relevant expertsEnsures that the vocabulary is usable and captures all usecases"From scratch" or extended stadard vocabulary,human/machine readable vocabulary, SKOS semanticrelations vocabulary

Recommendation 16

Outline

4 Conclusion

Recommendation 22

Specify procedures for data accessEnhances the usability of the databaseuser interface, API, protocol(s) for harvesting/federatedsearch, etc.Take into account licences (GDPR), needs of differentusers, different ways to transfer data

Recommendation 22

Recommendation 23

Offer research output metadata in multiplerepresentationsEnsures that users with different needs and preferencescan efficiently use the datauser profiles and preferences are different, option tocustomize the display and format, export to standardizedformats, XML, Bibtex, JSON, RDF - semantic web (FAIRprinciples)

Recommendation 23

Recommendation 24

Provide access to the data through a functional userinterfaceEnables consulting the database in various ways andincreases transparencySearching (basic and advance), browsing, downloading

Recommendation 24

Recommendation 25

Facilitate automated access to the data through an APIor a metadata harvesting protocolEnables automated and efficient use of the databaseREST, JSON vs XML, authentication and authorization(A1.2 FAIR principle), OAI-PMH, OAI-ORE, etc.

Recommendation 25

Recommendation 26

Enable crawling of bibliographic records by websearch enginesEnsures that database content can be found throughacademic search enginesCrawlers, Robots Exclusion Protocol (robots.txt), specificcrawling guidelines (Google Scholar)

Recommendation 26

Integrated European Publication Information ServiceMapping processMapping tools

Outline

4 Conclusion

EU services - evaluation for EU funded projects, reporting,etcPublications/outputs discovery

Approaches

Distributed vs CentralizedThe distributed approach makes it easier to have completeinformation in real-time, since it does not requirepropagation of updates to the central catalogue - federatedsearch SRU/WHowever, for data-intensive operations, the centralized approach doesn’t have the problem of querying multiple sites,and has more complete overview of the data availablewhen executing operations - harvesting data OAI-PMH

Approaches

Distributed SRU/W based approach

Centralized OAI-PMH based approach

Centralized approach

Data provider (nodes) and Service provider (IntegratedEuropean Publication Information Service)Protocols for harvesting metadata should be implementedon both side (OAI-PMH, ResourceSync, etc.)Target metadata format(s) should be selectedAll nodes (partner systems) have to export metadata to (atleast one) target metatada formatAll nodes (data providers) have to map its metadata totarget metadata format

Outline

4 Conclusion

Process

Matching source schema entities to target schema entitiesMatching source attributes to target attributesExpressing the mapping in some format/languageImplementation of mappings rules in source system

Process

Actors

1 Expert(s) for source schema2 Expert(s) for target schema3 Expert(s) for source/target vocabularies4 Software developers

Actors

Collaboration

Collaboration between schema/vocabularies experts isusually not a problemHowever, collaboration between those experts andsoftware developers could be a problem

Don’t "speak" the same languageThe process of implementation of mappings rules in sourcesystem is error-prone and time-consumingCan we automate the process? Can complete process beperformed by schema/vocabularies experts?

Collaboration

Outline

4 Conclusion

The process of matching and mapping implies a lot of timeand effort from experts on the source and target schemataTo simplify and accelerate the process, a tool needs to beadopted for automationBesides enhancement of mapping development, such atool should make the implementation of mappings moreeffective and shareable

X3ML toolkit

The X3ML toolkit with the 3M editor could be used toautomate the mappingsThis toolkit allows several steps and tasks of the process ofharvesting, matching, mapping and integrating the datafrom the sources to the target catalogue3M (one component of X3ML toolkit) guides the user tospecify the schemata matchings and the instancesgeneratorsX3ML engine (the another X3ML toolkit component)automatically transforms the source data into target format

X3ML toolkit

3M eases the process of matching by parsing andanalyzing the source and target schemata, thus allowingauto-completion when selecting the entities and propertiesto be matchedThis mechanism speeds the matching process and allowsnon-expert users (users that do not have an extendedknowledge of the whole schema) to define a matching

The description of the matching is homogenized, whichreduces the misunderstandings between experts andsoftware developers3M also includes a versioning mechanism that allowsstorage of different versions of the matchingsThe X3ML engine can be used exhaustively to test anyversion of the matching at any time just by providing asample of data and applying the transformationThe result is immediately available and can be analysed tocheck for defaults or implemented corrections

3M demo

Dublin Core is the source - linkCERIF RDF should be the resulthttps://isl.ics.forth.gr/3MMapping Project - ENRESSH Dublin Core to CERIF 1.6

3M demo

Summary

Part of Manuel of good practices: CREATING ANDMAINTAINING A NATIONAL BIBLIOGRAPHIC DATABASEFOR RESEARCH OUTPUT has been presentedIn order to improve reusability of metadata, the systemcould be a data provider and could export metadata tosome Service Provider(s)Source metadata schemata should be mapped to targetmetadata schemata

Summary

Questions

Thank you for your attention!!!If you have any questions, please do not hesitate to

ask me during the schoolcontact me by email - dragan.ivanovic@uns.ac.rs

Formats, metadata, standards and vocabularies for …...2019/06/03 · Introduction Good practices...

Documents

Towards an Interoperability Framework for Metadata Standardskmr.nada.kth.se/papers/SemanticWeb/TowardsAFramework.pdf · The model thus incorporates the concepts of metadata vocabularies,

4 1 Recommendations for metadata and data formats …succeed-project.eu/sites/default/files/deliverables/Succeed_600555...metadata and data formats for online availability and

Modeling the Complexity of Music Metadata in Semantic ... · In this section, we describe a data model for music metadata – the DOREMUS ontology, and a set of controlled vocabularies

Esri Geoportal Server Got Metadata? · PDF fileGeoportal 2.5.x Enable discovery across ArcGIS platform •Output formats: DCAT, Sitemap, CSV, KML, HTML •Preview, metadata details

Lecture 3: Vocabularies & Data Formats on the Social Web (2014)

RDA linked data vocabularies data management and use … presentation RDA Linked... · Open Metadata Registry. RDA/RDF. RDA Vocabularies (GitHub) RDA Toolkit. content. RDA Registry

Creating and Maintaining Metadata Vocabularies for Network

Open Government Vocabularies and Metadata - … · Vocabularies In Statistics, vocabularies are Code sets Classifications and Taxonomies Database models XML schemas Questionnaires

Construction of Marine Vocabularies in the Marine Metadata Interoperability Project Luis Bermudez, John Graybeal, MBARI Anthony Isenor, Defence R&D Canada

User-generated metadata: Boon or bust for indexing and controlled vocabularies?

© Metadata Technology ESCWA SDMX Workshop Session: Data Formats

Interoperability in Science Data: Stories from the Trenches...Interoperability Framework Standard Services Vocabularies Metadata catalogs Data archives Community data models Applications

April 2008 1 Uncorking the Varietals: Social Tagging, Folksonomies & Controlled Vocabularies Margaret Maurer Head, Catalog and Metadata Kent State University

Ch 6. More Metadata Vocabularies

Guidelines for using subject metadata and controlled vocabularies …docs.niwa.co.nz/library/public/subjectmetadata.pdf · Guidelines for using subject metadata and controlled vocabularies

Core vocabularies and Metadata Sets for Governments

Data Management CCTS...Metadata formats Metadata comes in a variety of formats but the most common are 1.A text or HTML document This will record all the information necessary and

Three SGML metadata formats: TEI, EAD, and CIMIarchive.ifla.org/documents/libraries/cataloging/metadata/biblink2.pdf · Standard Generalized Markup Language (SGML) is a well-established

4 1 Recommendations for metadata and data formats for

Semantic Technologies for Spatial Infrastructures · Ontologies Vocabularies . Metadata . Data & Metadata High Performance Computing Private Sector ABS GA BoM … Big Data co -located