® Reading meeting. December 12-14th, 2011 QUAlity aware VIsualisation for the Global Earth Observation system of systems University of Reading meeting

®

QUAlity aware VIsualisation for the Global Earth QUAlity aware VIsualisation for the Global Earth Observation system of systemsObservation system of systems

Reading meeting. December 12-14th, 2011

University of Reading meeting

December 12-14th, 2011 Reading UK

[email protected]

Reading meeting. December 12-14th,

2011

2

University of Reading

• 2 mornings+ 2 afternoons in a 3 day meeting

DecemberMonth 11

July 2011Month 6

February 2011Month 1

®



Introduction.


2011

4

UCGIS Research priorities

• GIScience research priorities identified by the University Consortium of Geographic Information Science (UCGIS)

Parr D. A., Lu Y.(2010) The Landscape of GIScience Publications 1997–2007: An Empirical Investigation with Latent Semantic Analysis, Transactions in GIS, 2010, 14(5): 689–708

0

2

4

68

10

12

14

Per

cen

tag

e o

f G

IS

pu

blic

atio

ns

1997

1998

1999

2000

2001

2002

2003

2004

2005

2006

2007

Spatial Data Infrastructure

Uncertainty

Visualization

Dynamics of the UCGIS research


2011

5

This is still a GEOSS project

• New on the GCI– Discovery and download of broader and more heterogeneous

datasets by connecting existing data sharing systems (i.e. EuroGEOSS and GENESI DEC/FedEO) with the GCI that allows• transverse data discovery from data catalogues outside of GEOSS

registries (supported by the broker functionality and the Earth Observation Common Vocabulary (EOCV))

• direct download of data sets.

• GEO is in a transition to a new structure. All committees and tasks are gone (ADC, STC, ST-09-02) and a new structure is emerging.


2011

6

GEOSS Infrastructure Enhancements

Services: Catalog, Access,

Processing

Inventory Access*

EO Data Applications: Web and PC

Brokered Search:EuroGEOSS*,

GENESI*, CWIC*

Brokered Search:EuroGEOSS*,

GENESI*, CWIC*

GEOSS Registration

Service (CSR)

GEOSS Registration

Service (CSR)Standards and Interoperability

Registry

Standards and Interoperability

Registry

Best Practices

Wiki

Best Practices

Wiki

ClearinghouseClearinghouseEO Vocabulary*EO Vocabulary*

GEOSSCommonInfrastructure

* Prototype capabilities

GEO WebPortal

Service Monitoring*

Service Monitoring*


2011

7

Growth of Registered/Accessible Resources

DataData products*products*

year

Number

Earth Earth Observation Observation SystemsSystems

ServicesServices

2011(Nov.)2011 (Sept.)2010

500

141

90,909

28,000,000+

1,000

1,500

191

1492

1500+

43

500+

381

10,000

1,000,000

100,000

10,000,000*via CEOS CWIC client integration

100,000,000

Data SetsData Sets


2011

8

A 3 days event. Main Objectives

• Second phase on GeoViQua Quality Model

• Start almost everything– Components for Quality elicitation– Components for Search– Components for Visualization

• Close the main phase of requirements

• Definition of scenarios


2011

9

A 3 days event. Timeline

• Day 12 Afternoon ESSC Conference Room(Focus on preparing the work to do and inspiration for the next day)– Quality model, Requirements– Visualization, GEOLabel– GCI (GEO Portal duplicate and EuroGEOSS broker– Introduction to scenarios

• Day 13 Morning Meterology Dept. (Split sesions)– Quality inference ROOM_A(1L43) or Quality extraction ROOM_B(1L61)

– Quality visualisation ROOM_A(1L43) or Quality search ROOM_B(1L61)

– 3 Scenarios 1L43 plenary+split– 3 Scenarios ROOM_B(1L61) or Philosophical Transactions Paper Room A (1L36) – Components design revisited based on scenarios 1L43

• Day 14 Morning Henley Business School, Room 102 – Current status of the GEOSS quality Metadata– Requirements – Review strategy for next deliverables and final comments


2011

10

Day 12 Afternoon. - ESSC Conference Room

• 14:00_14:15 Jon Blower Welcome • 14:15_14:25 Joan Masó Introduction • 14:25_14:45 Joan Masó Review of the project WP by active tasks, next

deliverables, next milestones, small teams • 14:45_15:15 Kevin Yang Quality data model definition for GeoViQua and GEOSS

Discussions on the Quality data model that will be adopted in the project • 15:15_15:30 Jon Blower Visualization in GeoViQua • 15:30_16:05 SC-ESA The copy of the GCI for our eyes only

• 16:05_16:20 Coffee break

• 16:20_16:50 Lorenzo Bigagli GEO User interfaces (search, visualization, and user feedback). (Based on an EuroGEOSS? broker demonstration and more ideas)

• 16:50_17:20 Victoria GEO Label. Review of the questionnaire and measures to stimulate participation

• 17:20_17:35 Simon Thum Requirements; current status and Wednesday preparation

• 17:35_18:00 Eva Sevillano Introduction of the GeoViQua 3 Scenarios • • Evening Table booked for 17 people at Sizzling Spice at 19:30 (very close to University)


2011

11

Day 13 Morning - Meterology Dept. 1L43

• 09:00_09:20 Joan Masó Components model presentation 1L43 • 09:30_10:50 Dan Cornford Quality inference ROOM_A(1L43)• 09:30_10:50 Joan Masó Quality extraction ROOM_B(1L61)


• 11:10_12:30 Jon Blower Quality visualisation ROOM_A(1L43)• 11:10_12:30 Lorenzo Bigagli Quality search ROOM_B(1L61)

• 12:30_13:30 Lunch


2011

12

Day 13 Afternoon - Met. Dept. 1L43

• 13:30_15:30 Eva Sevillano GeoViQua 3 Scenarios 1L43 Plenary• 15:30_15:45 Coffee break • 15:45_17:00 Eva Sevillano GeoViQua 3 Scenarios 1L43 Plenary

• 17:00_17:10 Break to change rooms

• 17:10_18:00 Kevin Progres on Royal Society Philosophical Transactions Paper Room A(1L36) Split

• 17:10_18:00 Eva Sevillano GeoViQua 3 Scenarios Room B(1L43) Split• 18:00_18:30 Joan Masó Components design revisited based on

scenarios and assign responsibilities 1L43 Plenary

• Evening Dinner


2011

13

Day 14 Morning - Henley Business School, Room 102

• 09:00_09:20 Many Short report on the split sessions focussing in coordination aspects

• 09:20_09:45 Eva Sevillano Current status of the GEOSS quality Metadata

• 09:45_10:55 Thorsten Reitz Requirements communicate conclusions and organize some corrections Cluster adaptors,


• 11:15_12:00 Simon Discard requirements that are interesting but we will not be able to address (pragmatic choices and decisions)

• 12:00_12:30 Joan Masó Review strategy for next deliverables and final comments

• 12:30_14:00 Lunch


2011

14

Agenda last minute items

• The agenda for today:– http://twiki.geoviqua.org/twiki/bin/view/ GeoViQuaIntranet / PTBMeeting03

• Is there any last minute items to be included in the agenda?

• Who is taking minutes?

®



The project

Joan Masó (CREAF)


2011

16

The aim

GeoViQua will provide a set of scientifically developed software components and services that facilitate the creation, search and visualization of quality information on EO data integrated and validated in the GEOSS Common Infrastructure.

Pilot case studies

CC RR OO SS SS

SS BB AA

Communitybuilding

GEO S&T Label


2011

17

Project objectives

• 1. Provision of innovative tools to enhance the current infrastructure capability. – GeoViQua’s major technical innovation is search and visualization tools for

the community which communicate and exploit data quality information from GEOSS catalogues.

• 2. Development of the GEOLabel. – GEOLabel requirements are identified (Work Package WP2), integrated with

components (WP6), validated and applied into pilot cases (WP7), and disseminated to the community (WP8). It will be completed in collaboration with the GEO task ST-09-02.

• 3. Harmonization, exploitation and dissemination of project outputs. – A careful validation process is conducted in collaboration with a number of

communities of practice and standards committees to ensure that the project contributes effectively to the GCI architecture. Collaboration to AIP will be continuous.


2011

18

• Enhanced geo-search tools– Quality-aware catalogue service – Quality-aware catalogue client

• Quality aware visualisation components– Integrate of quality information – Show the quality information

• This main 3 WP have started or are starting today.

Project structure

Delivery of solutions to end users

Data quality elicitation mechanism

Enhanced geo-search

tools

Quality aware visualisation

tools

®



Some informative topics


2011

20

EuroGEOSS conference

• Advancing the vision for GEOSS• January 25th to 27th. Madrid• ~40 presentations

• Two GeoViQua presentations– Friday January 27, 2012 – Salón de Actos– Session 10: Data Quality And Biodiversity- Chair Javier Nogueras

• GEOSS Quality Framework and Quality Information Encoding Xiaoy Yang et al. (11:20)

• Analysis of the Quality Metadata in GEOSS ClearinghouseEva Sevillano et al. (11:40)


2011

21

Arch. Implementation Pilot 4 and 5 participation

• Participation in AIP4 as a coordinated effort to produce a tutorial on data quality. The team was formed by GeoViQua, QA4EO and ESIP IQ cluster. – Currently this means: CREAF (Joan Masó), ASTON (Dan Cornford) and

NASA (Gregory Leptoukh) with Veronica Guidetti (as an observer)

• We fail to generate momentum and to write the tutorials for different reasons

• The SIF is still interested in having our tutorials in the BPW– http://wiki.ieee-earth.org/Documents/GEOSS_Tutorials

• We will be able to reformat the deliverable D6.1 Data quality encoding as a best practice paper Month 15: April 2012

• AIP4 has ended.

• Participation in AIP4 as a coordinated effort to produce a tutorial on data quality. The team was formed by GeoViQua, QA4EO and ESIP IQ cluster. – Currently this means: CREAF (Joan Masó), ASTON (Dan Cornford) and

NASA (Gregory Leptoukh) with Veronica Guidetti (as an observer)

• We fail to generate momentum and to write the tutorials for different reasons

• The SIF is still interested in having our tutorials in the BPW– http://wiki.ieee-earth.org/Documents/GEOSS_Tutorials

• We will be able to reformat the deliverable D6.1 Data quality encoding as a best practice paper Month 15: April 2012

• AIP4 has ended.


2011

22

AIP-5 Planning

• Scenario based development based on requests – Energy, Health, Water, Disaster

• Develop Information Schemas for GEOSS • Single sign-on • Draft schedule

– CFP development - Dec to Feb 2012 – Kick-off workshop - April or May 2012 – Development - 4 months – Complete by 2012 GEO plenary (Brazil)

• Deliverables – Components, ERs, Deployed Scenarios


2011

23

Possible OWS-9 activities

• Quality encodings– Producer created quality information is supposed to be encoded on metadata as

quality measures as well as conformity to some rules using 19139 XML encodings.– An alternative encoding is provided by UncertML (An old public discussion paper in

OGC)– Provenance is seen as part of the quality information (ISO19115 Lineage is an

alternative, W3Cprov is another alternative)– User feedback about datasets and peer review are other information that could be

included as quality.

• The state of the art in OGC– Almost no reference in service standards about how to relate data and quality. Links to metadata

elements is a possibility but no real examples are provided.– Situation in encoding standards like GML or KML.– Some profiles for WMS or other OWS services but there is no profile on how to communicate

data with data quality.– WCS 2.0 general structure mentioned an extension for an uncertainty coverage in WCS but this

is not available– WPS implementations provide access to any kind of models but no reference on how to

communicate provenance on results or how to input and propagate uncertainties to the outputs (Some old OWS-x edition experimented with this)

– OWS Context standard provide another way to associate data to other information in the same view but no quality scenario has been considered so far.

OGC®


2011

24

Quality viewer scenario

• A integrated client wants to show data and quality/provenance at the same time.– Data quality indicators and flags

• Product/dataset/sensor level quality as text data• Pixel/feature quality in the map view

– As extra style– In a synchronized window

– Provenance as a tree or network• It is possible to show/switch the source data from the provenance tree.

• Relevant services– WMS, KML clients

• Deliverables– Quality encoding for WMS and KML ER– Quality enable integrated client able to render data, quality indicators,

quality flag parallel datasets, provenance tree/network representation (workflow)

OGC®


2011

25

Quality propagation scenario

• The central element or the scenario is a WPS chain that is able to propagate quality in processes execution.– Reads data quality from other services– Encodes output resulting data quality– Involves multipart input and output

• data + metadata + quality + provenance

• Services involved– WFS that could encode quality in GML or in independent metadata– WCS (uncertainty extension) as input– WPS quality enabled

• Deliverables– WCS uncertainty extension ER– Quality enable WFS (+GML schema) ER– WPS quality propagation + provenance implementation with multipart files.

OGC®


2011

26

Participation in OWS-9 (or OWS-10)

• GeoViQua aims to provide some tools on quality visualization with the data and data search filtered and raged by quality, but first there is a need have a clear way to easily expose data quality in OGC services and encodings.

• We offer a small monetary sponsorship and also one person of the CREAF staff to participate/lead some activity on data quality in-kind. Other partners in the project could also participate in-kind but this has to be confirmed.

• NRCan, NGA and FAA has expressed interest in the provenance/propagation scenario and could finance some participation.

OGC®


2011

27

OWS-9 Preliminary ScheduleOWS-9 Preliminary Schedule

© 2011 Open Geospatial Consortium© 2011 Open Geospatial Consortium

• Contact: –Nadine Alameh, nalameh at opengeospatial.org –George Percivall, gpercivall at opengeospatial.org

OGC®

®



Where we are


2011

29

Starting an important period

Start PrototypesValidation

Mobile Solutions

Search & Visualization

Data ready

Quality recommendations

Quality parameterization

Pilot cases

User & technical requirements to CoP

User & technical solutions to CoP

Workshops

Proposals evaluation Final documentGeoLabel

Metadata extraction

Best practices quality encoding

Direct extraction from continuous variables

Quality elicitation User feedbackExtraction from categorical variables

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36

Febr

uary

201

1

Janu

ary

2012

Janu

ary

2013

Dec

embe

r 201

3

2014


2011

30

Next deliverables

• D3.1 Metadata extraction quality component – Month 12 January 2012 (CREAF)

• D7.2 Report on current GCI and identification of issues in integration with GEOSS – Month 12 January 2012 (UAB)

• D2.1 User requirements document – Month 13 February 2012 (FRAUN)

• D2.2 Technical requirements document – Month 14 March 2012 (FRAUN)

• D6.1 Data quality encoding as a best practice paper – Month 15 April 2012 (ASTON)– Connected to BPW tutorials.


2011

31

Next Milestones

• MS03 User and stakeholder workshop (revision on the user and technical requirements)– Month 11 December 2011!!!

• MS11 Encoding mechanism for quality information and linkage defined and agreed – Month 15 April 2012

• MS4 Prototype quality elicitation components ready to validate in pilot case studies

• MS6 Prototype quality geo-search component ready to validate in pilot case studies

• MS8 Prototype quality visualisation component ready to validate in pilot case studies– Month 18 July 2012


2011

32

Motion to delay MS03: workshop

• The PTB approves the activities (consultation process) done as a way to pass: "MS03 User and stakeholder workshop (revision on the user and technical requirements)"

– Currently there are general felling that requirements process has been too long and we have to start producing things.

– Some conclusions of the requirements/interviews where already presented at QA4EO as a poster– The GEO Label questionnaire and the quality model will generate new requirements– GEO is in the middle of a committee restructure process (not the best time)

• Discussion: DC: Refrase that we already made it. JS: good. LB: OWS-9 is a possibility. JB: Milestone is already passed.

• Motion: Joan Masó (CREAF)• Second: Dan Cornford

• Is there any objection to the unanimous consent?• Passes

• For the next PMB: Consider a "MS03-2 User and stakeholder workshop (revision on the user and technical requirements based on first GeoViQua prototypes)" to Month 24 where first prototypes will be available.

7 partners is quorum in the PTB

®



Let's start!

Thank you

®



Components model presentation


2011

35

®



Quality Extraction quality component

Task 3.1


2011

37

Original DoW Description

• This task develops a tool to identify quality indicators from existing metadata documents and will be able to read information from different file formats and process this information. Since quality parameters were left open in 19115, this tool will integrate quality description into the unified model specified in Task 6.1.

• This will make possible to create a catalogue of quality metadata in a common format. Also, provenance information in 19115 metadata documents (LI_Lineage) can be used to infer quality information from the data sources. [CREAF, UAB, S&T and CNR]


2011

38

Two possible interpretations

class DQ_Element classification, lineage and citation

DQ_DataQuality

+ scope :DQ_Scope

«Abstract»DQ_Element

+ dateTime :DateTime [0..*]

«moved»+ evaluationMethodDescription :CharacterString [0..1]+ evaluationMethodType :DQ_EvaluationMethodTypeCode [0..1]+ evaluationProcedure :DQM_SourceReference [0..1]+ measureDescription :CharacterString [0..1]+ measureIdentification :MD_Identifier [0..1]+ nameOfMeasure :CharacterString [0..*]

MD_Metadata

LI_Source LI_ProcessStep

LI_Lineage

+resourceLineage0..*

+processStep 0..*+source 0..*

+source

0..* +sourceStep

0..*

+dataQualityInfo

0..*

+derivedElement

0..*report

Quality model Task 6.1

Quality Extraction

Quality Presentatio

n

Catalogue service Task 4.1

CommunityCatalogues


2011

39

Adapt data to EuroGEOSS broker model

• Tool to transform quality data to a common format (Kevin's quality model based on ISO 19115-1 and 19157)– Tool to transform from other formats

• NetCDF

• HDF

• …

– Tool to transform form other standards used in capacity catalogues• (needed CNR contribution here)


DQ_DataQuality

+ scope :DQ_Scope




MD_Metadata


LI_Lineage



+source

0..* +sourceStep

0..*

+dataQualityInfo

0..*

+derivedElement

0..*report


Quality Extraction



2011

40

Example of NetCDF ->ISO 19115

• https://geo-ide.noaa.gov/wiki/index.php?title=NetCDF_Attribute_Convention_for_Dataset_Discovery


2011

41

To the EuroGEOSS broker

• If there are no quality parameters, a tool to extract data quality from provenance data (in theory)

– In order this to be possible, we need a link between the resource and its sources. This is possible if there is a link to the source metadata document.

– But links are poor populated and they can not function as real links• 59FR13375• NF-05-05 GSF Multibeam Data• J:\Albuq_orig_meta\PAIS\pais_valley_pipl.xml• Server=204.47.240.122; Service=esri_sde; User=geo_pol; Version=SDE.DEFAULT• drto_DOQQ.sid, Projection: NAD_ 83_UTM_Zone 16N, Meters.• Stark County, Ohio GIS Parcel Data (http://downloads.ddti.net/starkoh/)• BLM Digital GCDB Flat Files (http://www.blm.gov/ca/st/en/prog/cadastral/gcdbpage.html)

– They are not useful to link to source data in the GCI or in other catalogues.– What are we going to do?


DQ_DataQuality

+ scope :DQ_Scope




MD_Metadata


LI_Lineage



+source

0..* +sourceStep

0..*

+dataQualityInfo

0..*

+derivedElement

0..*report


Quality Extraction



2011

42

To the GEOPortal

• A XSL transformation that allows better visibility of the quality indicators, lineage/provenance etc.

• Ted portal and the metadata punctuation adapted for DataQuality

Quality Presentatio

n



2011

43

NOAA metadata records

http://www.ngdc.noaa.gov/metadata/published/NOAA/NESDIS/CLASS/iso/


2011

44

Rubric XSL

• Ted Habermann

®



Wednesday


2011

46

Day 14 Morning - Henley Business School, Room 102

• 09:00_09:20 Many Short report on the split sessions focussing in coordination aspects

• 09:20_09:45 Eva Sevillano Current status of the GEOSS quality Metadata

• 09:45_10:55 Thorsten Reitz Requirements communicate conclusions and organize some corrections Cluster adaptors,


• 11:15_12:00 Simon Discard requirements that are interesting but we will not be able to address (pragmatic choices and decisions)

• 12:00_12:30 Joan Masó Review strategy for next deliverables and final comments

• 12:30_14:00 Lunch

®



Reports on split sessions

Focus on actions


2011

48

Quality Extraction


DQ_DataQuality

+ scope :DQ_Scope




MD_Metadata


LI_Lineage



+source

0..* +sourceStep

0..*

+dataQualityInfo

0..*

+derivedElement

0..*report


Quality Extraction

Quality Presentatio

n


CommunityCatalogues


2011

49

Actions

• We will study the way netCDF and HDF quality metadata can be imported

• We will consider some capacity catalogues metadata as candidates to import

• XSL will be provided to transform metadata schemas into readable forms to illustrate how to present quality metadata to the user

• Close collaboration with WP4 (CNR) and data quality model (WP6.1).

®



Review strategy for next deliverables and final

comments


2011

51

Next deliverables

• D3.1 Metadata extraction quality component – Month 12 January 2012 (CREAF)

• D7.2 Report on current GCI and identification of issues in integration with GEOSS – Month 12 January 2012 (UAB)

• D2.1 User requirements document – Month 13 February 2012 (FRAUN)

• D2.2 Technical requirements document – Month 14 March 2012 (FRAUN)

• D6.1 Data quality encoding as a best practice paper – Month 15 April 2012 (ASTON)– Connected to BPW tutorials.


2011

52

Actions

• WP 3.1 Quality extraction tools should be generated soon.– Deliverable D3.1 should be produced

• WP 7.1 Current status of the clearinghouse finish.– Deliverable D7.2 should be produced

• WP 6.1 Data Model should be finished soon – philosophical paper should be produced

– Deliverable D6.1 should be produced

• WP 2.1 WP 2.2 requirements process first phase should end– Deliverables D2.1 and D2.2 should be produced

• Scenarios storyboard should mature and components shall be clearly identified– Presented in the next PTB telecon

• I would like to nominate a volunteer to look on the Liferay 5.1.2 platform and Liferay database

– Results presented in the next PTB telecon

– Name:


2011

53

Sprint to project evaluation

• We will have a project evaluation on Month 18 (July 2011)

• We are more or less on track but we need to move forward.

• We have– Requirements, scenarios, components, data models

• It is time to start developing components for elicitation WP3, search WP4 and visualization WP5.

• They don't next to be finalized but ideally some prototypes could be used to illustrate the feasibility of our ideas in some controlled situations

• Plans to achieve this objective would be presented by the 3 WP leaders in the next PTB telecon (January)– Monthly telecons will follow


2011

54

Logistics

• Please, please upload your presentations to the twiki page– http://twiki.geoviqua.com/twiki/bin/view/GeoViQuaIntranet/

PTBMeeting03– Done: SC GEOPortal, CREAF introduction + split session + GCI

metadata study, WP4, 52N materials

• Next Face2face meeting will be in ~July– The place will be announced soon

• Thanks to University of Reading (Jon + Kevin) to organize this event.

®



Summary

®



Thank you!

The meeting is adjourneduntil next face to face meeting