View
214
Download
0
Category
Preview:
Citation preview
ESDSWG meeting – 10/21-23/2008
Metrics Planning and Reporting (MPAR) WGBreakout Summary
H. K. (Rama) RamapriyanNASA GSFCClyde BrownNASA LaRC
Co-Chairs, MPARWG
8th Earth Science Data Systems Working Group
Joint Working Group Meeting
Wilmington, DE
October 20-22, 2009
ESDSWG meeting – 10/21-23/2008
MPAR Working Group
Attendance – 23
Topics DiscussedDiscussions with Martha MaidenMetrics baseline reviewFY 09 ReportingCore and Community metrics collection
ESDSWG meeting – 10/21-23/2008
MPARWG Breakout Sessions
Item Who
Day 1 - Session 1: FY09 Activities and Progress
Introduction Rama
HQ / Program view of Metrics Maiden Martha Maiden's Four Questions and MPARWG responses Hunolt
E-Books vs Metrics - What is Reported Via Each Rama
Action / Open Items from Oct 2008 workshop RamaSummary of FY09 Metrics Reporting Hunolt
Day 2 - Session 2: Metrics and MEaSUREs
MEaSUREs Telecon Results RamaMetrics Baseline Review Hunolt / All
Day 2 - Session 3: Metrics Collection - Core and Community
Overview of Metrics Collection MCT & EMS Rama
Introduction for MEaSURES and Changes to MCT Randy Barth
Transition to "MCT2" and EMS for Metrics Reporting Kevin Murphy
Day 2 - Session 4: TBD, Action Items and Work Plan
Continued discussion of metrics baseline Hunolt/All
ESDSWG meeting – 10/21-23/2008
Discussions with Martha Maiden Martha emphasized importance of metrics for her reporting to her management and OMB Metrics from both Community systems and Core systems are important
User satisfaction metrics – American Consumer Satisfaction Index is an important metric watched by OMB
Aggregates and highlights from core and community metrics are useful
Impact metrics are very useful, but a better format is needed
Encouraged use of metrics for the benefit of the Projects themselves
ESDSWG meeting – 10/21-23/2008
Actions Planned in Response
Provide separate aggregates for community systems that include trends, so community systems are not lost in the totals.
Provide periodic reports in a form useful to MarthaProgram level summaries and trendsHighlights from individual community systems. These
could be based on the regular metrics, e.g. noting significant growth in users, distribution, addition of new projects or services, etc.
Projects could use supplemental comments to highlight / call attention to items.
The form in which this information is provided to Martha could be quad-chart or a short pdf ‘newsletter’.
ESDSWG meeting – 10/21-23/2008
Metrics Baseline Review
Products and Services Metrics -
Common Metrics:Metric 1 - Distinct Users – No changeMetric 2 – Distinct Users by Class – No changeMetric 3 – Products Distributed – Added option of breaking down by product types
Metric 4 – Product Types AvailableMetric 5 – Volume Distributed – after considerable discussion, left unchanged
Metric 6 – Volume Available – after considerable discussion, left unchanged
Metric 7 – no longer usedMetric 11 – Services ProvidedMetric 12 – Services Available – need explanatory examples for projects to use this better
ESDSWG meeting – 10/21-23/2008
Metrics Baseline Review
Project Defined (Project Specific) Metrics (up to 4, defined by each project, metrics 100, 101, 102, 103)
Programmatic Metrics:Metric 8 – Science Focus Areas SupportedMetric 9 – Applications Areas SupportedMetric 10 – Education categories Supported
Impact Metrics – decided to recommend use of quad chart format, but will develop strawman examples and validate with Martha
New Citation Metric Deborah Smith (RSS, Inc) presented an approach they have
been using to collect citations of their productsRecommend use of citation metric to be used optionally
ESDSWG meeting – 10/21-23/2008
FY 09 Reporting
22 REASoN; 15 ACCESS; 8 MEaSUREs [Many of the REASoN and ACCESS Projects completed during FY 09
Impact metrics – 3 from 2 REASoN and 1 MEaSUREs Projects
We expect 10 or more MEaSUREs projects to start reporting in FY 10Generally, projects report when they start distributing data to users
Last REASoN and ACCESS Projects are expected to finish in FY 10
Upcoming ACCESS reporting - TBD
ESDSWG meeting – 10/21-23/2008
Core and Community Metrics CollectionCore system metrics
EOSDIS Data Center metrics collection using EMS was demonstrated
Internal use of metrics in ESDIS for planning purposes
Network flows vs. requirements are analyzed using periodic “pinging” and measuring actual data flows
Community metrics Collected using MCTFY 09 improvements to MCT discussed (e.g., 508 compliance)
Transition to EMS-based collection of community metrics discussed – needs further examination
Recommended prototyping automated collection with two separate (diverse) projects using log files
Recommended