Embed Size (px)
DESCRIPTION
Towards a collaborative metadata management system for sensor networks using SMW. Steps taken and lessons learned
Citation preview
SWMCon Fall 2013 | 29 October 2013
Effective metadata management
of sensor networks using SMW
Jan Willem Noteboom29 October 2013
SWMCon Fall 2013 | 29 October 2013
2
Outline
Need for effective metadata management
Towards a collaborative environment
Experiences with Semantic MediaWiki (SMW)
SWMCon Fall 2013 | 29 October 2013
3
Need for effective metadata managementSituation• sensor networks more popular, bigger and complex• access to many metadata sources needed for interpretation• consistency of metadata sources is demanding
Needs• integrated access to all sensor network metadata• linked sources of sensor network metadata• collaborative management of metadata• interfaces for scientists, field inspectors and applications• using international standards and open technology
SWMCon Fall 2013 | 29 October 2013
4
Need for effective metadata management What metadata
Measurement System infoMeasurement Process info
Measurement Data info
Collection & Discovery metadata
Provenance & Lineage metadata
Interpretationmetadata
Platform & instrument metadata
Metadatain
scope
Phenomenaproperties(dynamic nature)
Measuringcharacteristics
(“static”nature)
SWMCon Fall 2013 | 29 October 2013
5
Need for effective metadata management What services
DiscoveryDiscovery, ViewView and RetrievalRetrieval of sensor network information
AnnotationAnnotation of sensor network information
NotificationNotification about sensor network information updates
UpdateUpdate of sensor network information
AdministrationAdministration of users, information and components
Discovery View Retrieval Update
Services
Annotation Notification Administration
Users & Applications
Contentmanagers
Servicemanagers
(linked) Sensor Network Metadata(processes, systems)
SWMCon Fall 2013 | 29 October 2013
6
Towards a collaborative environmentPhase I: Inventory & Analysis
collaborative data model for sensor network metadatacollaborative platform/architectuur
Phase II: Definition & PoC & Migrationdefinition of requirements, architecture and data modelproof of concept (PoC) using SMWdata migration toolkit
Phase III: Realization, Cleansing & OperationalizationRealization of version 1.0 Migration and cleansing of migrated data (automated and manually)Implementation in the operational environment
SWMCon Fall 2013 | 29 October 2013
7
Towards a collaborative environment (meta)Data model
Property-oriented (linked data)
Linked data with time validation and status
KNMI terminology and taxonomy
“Based on” international standards (ISO 19115, SensorML)
Systems and processes
Annotations (commentary)Events of several types
Links to media (docs, pictures) and people/organisations Example: Element T(temperature) is measured
according to measurement process P using instrument I that is deployed at location L in field F of station S (managed by person P) and part of network N
SWMCon Fall 2013 | 29 October 2013
8
Towards a collaborative environment Collaborative platform
Linked Sensor Data(systems, processes)
Metadata sources
Services
Meta
Data
la
yer
Serv
ice
layer
Ap
pl la
yer
Repository
Existing Services
External
sources
Migrationof
metadata
Exte
rnal
Meta
Data
(Geo)Portal
Applications, UsersData managers
Discovery View Retrieval
Update
Annotation
Notification Administration
Internal
sources
Other collaborative environments
SWMCon Fall 2013 | 29 October 2013
9
Experiences with Semantic MediaWikiContext:
MW 1.21.2, SMW 1.8, 30 extensions, no SPARQL endpoint, 23410 pages, 25(50) namespaces, 190 properties (611,171 property values), 150 templates, 25 forms, 12 categories (classes)….
Experiences:
In general
Data modelling or structuring
Data discovery (searching/browsing)
Data view & retrieval (displaying/encoding/querying)
Data update (creating/editing/transferring)
Data annotation (commenting)
Data notification (watching)
Administration (user management, access control)
SWMCon Fall 2013 | 29 October 2013
10
Experiences with Semantic MediaWiki General
Well-known environment to most users
Powerful collaborative environment
Steep learning curve
Many useful extensions
Documentation is getting better (hint: “Working with Mediawiki” Yaron Korn)
NoteDifferences in the quality of extensions(error messaging, documentation, governance)
SWMCon Fall 2013 | 29 October 2013
11
Experiences with Semantic MediaWiki Data structure
Easy implementation of the data model using properties, templates, categories and pages
Usage of several namespaces
Usage of semantic internal objects to add time validation to links (property qualifier)
NoteImport of ontologies (foaf, skos, ssn, dc, etc) considered but not applied.
SWMCon Fall 2013 | 29 October 2013
12
Experiences with Semantic MediaWiki Data discovery
Searching/Browsing:Query forms via site-bar menu
very flexible filtering and pre-loadabletabs for standard vs advance filtersno linkage of filters…
Standard search/browse
Semantic maps & timelines
NoteNo use of drilldown forperformance reasons
SWMCon Fall 2013 | 29 October 2013
13
Experiences with Semantic MediaWiki Data View & RetrievalDisplaying/encoding/query:
Pages with Infoboxes
Several Result Formats and custom templates
Collapsible sections/infoboxes
Custom templates to construct “complex” ask queries (increased $smwgQMaxDepth, $maxRecursionDepth)
Concepts to ease retrieval
NotesRestricting “view” access of page parts with Userfunctions (weak).
SWMCon Fall 2013 | 29 October 2013
14
Experiences with Semantic MediaWiki Data update & editCreating/editing/transferring:
Namespaces and Usergroups to restrict (create/edit) accessForms with HeadertabsFormlink buttons for creationSeveral form input types (e.g. combobox, two listboxes)
• Data transfer for data migration (e.g. xml import)
NoteForms not well suited to handle of many instances (>50) of a multiple instance template (e.g. internal objects/subobjects)
SWMCon Fall 2013 | 29 October 2013
15
Experiences with Semantic MediaWiki Data AnnotationPage commentingNotesEvents
permanenttemporary (period)planned/unplanned
AttachmentsTalk (discussion)
NoteClear policy needed for event registration
SWMCon Fall 2013 | 29 October 2013
16
Experiences with Semantic MediaWiki Data Notification
Watchlist (pages)
Notification (Echo)
NoteNeed for summary reports of changes for dedicated user groups….
SWMCon Fall 2013 | 29 October 2013
17
Experiences with Semantic MediaWiki Data Administration
Special page utilitiesUser managementData housekeepingStatistics
Maintenance scriptsDaily backupRebuilding MWRefreshing SMW
SWMCon Fall 2013 | 29 October 2013
18
To summarizeSemantic Mediawiki (SWM) is a powerful collaborative environment to manage sensor network metadata effectively.
Notes/NeedsExtensions differs in quality (error handling, documentation, governance). Documentation has improvedNeed for forms that can handle many instances (>50) of a multiple instance templateNeed for summary reporting of changes (dedicated for user groups)
SWMCon Fall 2013 | 29 October 2013
19
Many thanks for your attention!
