Work plan – Evaluation - General Aspects - NEFIS – mid-term, WP2 and WP3 Meeting, Vienna 24-25 March 2004 University of Hamburg, Aljoscha Requardt WP 5:

Work plan – Evaluation- General Aspects -

NEFIS – mid-term, WP2 and WP3 Meeting, Vienna 24-25 March 2004 University of Hamburg, Aljoscha Requardt

WP 5: Evaluation

Completion date: month 26 – April 2005

Partners responsible:

• Partner 4 (Hamburg)

• Partner 1 (EFI)

• Partner 5 (Fraunhofer)

• Partner 10 (Greenwich)


Focus of evaluation on the “five aspects” which were stated at the NEFIS Kick-off-Meeting in Joensuu, March 2003:

• metadata and metadata guidelines

• definitions for the data/information types

• levels of access – data rights management in NEFIS

• information needs and user’s expectations

• electronic information system for periodical data reporting

The evaluation is going to focus more on the contextual aspects within NEFIS

rather than on the technical ones.


Aspects and Work plan of Evaluation should harmonize and correspond

with the “Wallingford Steps”.

keyword lists and thesaurus


Two ways of approach:

a) Standardized questionnaire

b) Validation by Experts

Questionnaires will collect responses regarding:

• metadata records and metadata guidelines• data preparation and data rights• performances, user friendliness of the system • capability to satisfy information needs - user expectations

Validation by Experts will give theoretical appraisements and statements regarding:

• metadata and metadata guidelines• levels of access – data rights management in NEFIS• keyword lists and thesaurus• electronic system for periodical reporting• UseCases and Storylines


All comments plus reviewed “expert validation” will be documented - summarized and evaluated - in the Final Evaluation Report (April 2005)by Partner 4 (Hamburg).

Work plan – Evaluation- Time Schedule -


05 06 07 08 09 10 11 12 01 02 03 04

EvaluationReport

05

Hamburg Meeting

Validation Report by Experts (P1, P3, P6, P10) “Metadata guidelines“

data linking in full swing

Vienna Meetinga) Keywords for

selected topics a) Metadata guidelines

data links completed

improved version of all metadata records

data available through VTK

questionnaire “Metadata guidelines” to all partners

and questionnaire “data rights”back to P1 – collected to P4

Validation Report by Experts (P1) – review by P5 “Levels of Access – data rights management“

Validation Report by Experts (P1, P10)“UseCases and Storylines”

Validation Report by Experts (P4)“Electronic system for periodical reporting”

questionnaire “Information needs” back to P1 – collected to P4

questionnaire “Information needs” to all relevant partners

delivery of final metadata records

Validation Report by Experts (P3)“Keyword lists and Thesaurus”

2004/2005

questionnaire “Data rights” to all relevant partners (WP4)

questionnaire „Metadata guidelines“ back to P1 – collected to P4

Work plan – Evaluation- Metadata and Metadata Guidelines -

Questionnaire: After Vienna Meeting development of questionnaire by Partner 4 (Hamburg) and Partner 1 (EFI) July 2004: sending questionnaire to all relevant partners – Coordination of Mail: Partner 1 (EFI) Delivery of answered questionnaire: September 2004 – Coordination of Mail: Partner 1 (EFI),

Collected answers back to Partner 4 (Hamburg) first results: November 2004 (Hamburg Meeting) - Presentation


Aspects: • Analysis and evaluation of use and structure of the NEFIS Metadata

• Analysis and evaluation according applicability of the Metadata Guidelines

Questions and comments like discussed at the Solsona Meeting!

• What problems do occur and for which fields/ records?

• Which fields should be changed, modified or improved?

• Are there any other metadata records which should be included?

• Are the guidelines easy to understand? Which fields should be explained more in detail?

Work plan – Evaluation- Metadata and Metadata Guidelines -

Validation by Experts: Theoretical appraisement by a specified and qualified NEFIS partner:

IUFRO/Greenwich/EFI/METLA Validation regarding use and structure of NEFIS metadata and guidelines (~2 pages) Start: after Vienna meeting Delivery: October 2004 to Partner 4(Hamburg) and Partner 1 (EFI) Presentation: Hamburg Meeting


Work plan – Evaluation- Keyword lists and Thesaurus -


Validation by Experts: Final appraisal regarding Keyword lists and NEFIS thesaurus – considering linkages to GFIS Evaluation Report by IUFRO Expert Team – executive summary and review by Partner 1 (EFI)

and Partner 4 (Hamburg) Delivery: January 2005 to Partner 4 (Hamburg)

Aspects:• use and structure of the NEFIS keyword lists and thesaurus

• Where are linkages to GFIS? Where are linkages to any other thesauri?

• see aspects and questions by Eric Landis (questions of discussion)

Work plan – Evaluation- Information needs -


Questionnaire: development of the questionnaire by Partner 4 (Hamburg) in contact with Partner 5 (Fraunhofer),

Mail coordination Partner 1 (EFI) – evaluation of answered questionnaire by Partner 4 (Hamburg) brief presentation of content and issue of the questionnaire at the Hamburg Meeting (November 2004)

by Partner 4 sending questionnaire right after Hamburg Meeting (November 2004) to all relevant partners,

collected answers back End of January 2005

Aspects:

• Evaluation of information needs and expectations – looking back to the EFIS-Final Report

and EFICS Study: Identification of User Groups

• Evaluation (technical) performances of VTK

• Description of gaps, problems using the VTK and the data linking

• Is the VTK sufficient for expected purposes?

• Are there any features which should be considered in the further development and

improvement of the VTK? – see comments and discussions from the Solsona Meeting

• How are temporal and spatial resolutions of the data dealt with? Do they fit user needs?

Work plan – Evaluation- Levels of Access -


Data provider must state what data are accessible and for whom, and which type of

data rights should be assigned to which kind of data

a) via metadata records (15. Rights)

b) via a specified questionnaire

In the frame of metadata records a questionnaire for more detailed analysis of levels of

access and data rights should be sent to all relevant partners.

Questionnaire “data rights”: development of the questionnaire by Partner 1 (EFI) and Partner 4 (Hamburg), sending questionnaire to all relevant partners (WP4), July 2004 – Coordination of Mail: Partner 1 (EFI),

collected answers back to Partner 4 (Hamburg), September 2004

Work plan – Evaluation- Levels of Access -


Validation by Experts: Theoretical validation according data rights management in the NEFIS – by Partner 1 (EFI),

Delivery: November 2004 - Presentation: Hamburg Meeting

Aspects:• What are the used levels of access? Which types of data rights are currently

used in the NEFIS? Which types would be appropriate for which kind of data?

• Which levels of access and forms of data rights are theoretically possible, and which are

technically possible? Which should be applied for which data and to which type of user?

• Where are technical, organisational or legal limits and problems?

• Rights management: Who sets the rights? Who sets passwords? Who has the organisational

and administrative overview? Who is coordinating the whole data management?

• General overview regarding data bases and forms of data access - theoretical background

announcing problems such as data redundancies

Work plan – Evaluation- Information needs -


Validation by Experts: Evaluation “UseCases and Storylines” Final appraisal by Partner 1 (EFI) and Partner 10 (Greenwich) - reviewed by Partner 4 (Hamburg)

Delivery: January/February 2005 - April 2005 (Final Evaluation Report)

Aspects:

• Summary of development and status as well as short description of functionality and

meaning of UML UseCases.

• How far could/did WP3 achieve to model the European Forest Information context using the

Unified Modelling Language (UML) - for a better understanding of who the stakeholders are, what

their information requirements are, what the driving forces are, what the policy context is and how

these components relate to each other.

Work plan – Evaluation- Electronic system for periodic data reporting -


Validation by Experts: Evaluation and statement by Partner 4 (Hamburg)

Delivery: January/February 2005 - April 2005 (Final Evaluation Report)

Aspects:• consider evaluation and discussion regarding UseCases and use of UML

• theoretical discussion of possibilities and developments of NEFIS and its functionality

as a electronic system for periodic reporting

• discussion outlined for three examples: Forest area, Volume of Timber, Carbon stock

• international comparability

• reliability of converted and harmonised data

• Background of: How can national data be reported on international level?

• Aspects: harmonisation/standardisation, data conversion/ double data assessments/

top down and bottoms up approach, costs and organisational charges

• What is within the limits of NEFIS?

• Where are demands for further projects? What progress seems to be relevant?

Work plan – Evaluation- Time Schedule -


05 06 07 08 09 10 11 12 01 02 03 04

EvaluationReport

05

Hamburg Meeting

Validation Report by Experts (P1, P3, P6, P10) “Metadata guidelines“

data linking in full swing

Vienna Meetinga) Keywords for

selected topics a) Metadata guidelines

data links completed

improved version of all metadata records

data available through VTK

questionnaire “Metadata guidelines” to all partners (WP4)

and questionnaire “data rights”back to P1 – collected to P4

Validation Report by Experts (P1) – review by P5 “Levels of Access – data rights management“

Validation Report by Experts (P1, P10)“UseCases and Storylines”

Validation Report by Experts (P4)“Electronic system for periodical reporting”

questionnaire “Information needs” back to P1 – collected to P4

questionnaire “Information needs” to all relevant partners (WP4)

delivery of final metadata records

Validation Report by Experts (P3)“Keyword lists and Thesaurus”

2004/2005

questionnaire “Data rights” to all relevant partners (WP4)

questionnaire „Metadata guidelines“ back to P1 – collected to P4

Work plan – Evaluation- WP5 Evaluation Meeting -

November 2004in Hamburg


Documents

Work plan – Evaluation - General Aspects - NEFIS – mid-term, WP2 and WP3 Meeting, Vienna 24-25 March 2004 University of Hamburg, Aljoscha Requardt WP 5: