Food and Agriculture Organizationof the United Nations
Statistics Division - ESS
CGIAR Data Standard Summit, November 26-28, Rome, Italy
FAO Approach to Data Exchange and Dissemination
An introduction to a free, flexible and transferable platform
Josef SchmidhuberDeputy DirectorFAO STATISTICS DIVISION
Food and Agriculture Organizationof the United Nations
Statistics Division - ESS
CHALLENGESCLOSE THE GAP BETWEEN DATA AVAILABILITY AND DATA NEEDS
Food and Agriculture Organizationof the United Nations
Statistics Division - ESS
Lack of data and metadata description and
harmonization
Restrictive or unclear data policies
Dispersed and uncoordinated data life cycle management
Poor governance, ineffective or missing institutional frameworks (internal & external)
Limited interoperability between systemsLimited user orientation and focus on needsPoor data dissemination systems, limited communication and user awareness
1
2
3
4
5
6
7
Food and Agriculture Organizationof the United Nations
Statistics Division - ESS
ESS APPROACH TO DATAMANAGEMENTKey principles
Food and Agriculture Organizationof the United Nations
Statistics Division - ESS
Data and tools provided as a “public good”; no licensing constraints; full redistribution rights
Open data are converted into accessible data (standards, classifications, formats)
Technical in-country assistance provided. Data collected and disseminated are the result of the collaborative effort between FAO, countries and regions.
1
2
3
Food and Agriculture Organizationof the United Nations
Statistics Division - ESS
Ensure country ownership of data, methodologies and IT systems
Ensure that data produced and formats used meet user needs
Data sharing strengthened by an open-source IT platform
5
6
4
Food and Agriculture Organizationof the United Nations
Statistics Division - ESS
FAO APPROACH TO
DATAKEY COMPONENTS
Food and Agriculture Organizationof the United Nations
Statistics Division - ESS
Partnership & Institutional network(Committees of key stakeholders)
Data access
and sharing
(IT platform) Dat
a man
agem
ent
and
harm
oniza
tion
(stati
stica
l met
hods
)
Statistical governanceData p
olicy
& d
ata s
harin
g rul
es
Definition of standards and tools
Political dimension
Technical dimension
Data sharing Network
(SDMX, DDI)
Food and Agriculture Organizationof the United Nations
Statistics Division - ESS
Food and Agriculture Organizationof the United Nations
Statistics Division - ESS
Statistical Governance System: external and internal
CountrySTAT: Support to Multidisciplinary Technical Working Groups (TWG) already established in 23 countries and several regional organizations
Linking all data initiatives such as CountrySTAT, the Global Strategy, AMIS, Censuses, IHSN
Agency-to-agency collaboration: with international agencies (WB, IMF, OECD, CGIAR)
1
2
3
4
Partnership & Institutional network
Food and Agriculture Organizationof the United Nations
Statistics Division - ESS
Food and Agriculture Organizationof the United Nations
Statistics Division - ESS
Adoption of international standards
Universal code-lists and common metadata templates
Use of international statistical classification systems, mapping systems and DB
Data collection and processing methodologies to ensure data compatibility / comparability
1
2
3
4
Data management and harmonization
Food and Agriculture Organizationof the United Nations
Statistics Division - ESS
Food and Agriculture Organizationof the United Nations
Statistics Division - ESS
Service-oriented architecture (SOA) based on open-source technology
Integrated metadata and implementation of well-known data exchange protocols (e.g. SDMX, DDI)
Web services (APIs) to facilitate data sharing between countries, regions and international organizations
Widget approach to deliver functionalities and integrate tools with other systems/websites
1
2
3
4
Data access and sharing (FENIX platform)
Food and Agriculture Organizationof the United Nations
Statistics Division - ESS
Different options for data collection:• Synchronization of data via APIs• Online data entry forms• Uploads from csv/xls files• Automated load of email attachments• Collection of data through smartphones
Ability to handle databases, geospatial data (e.g. remote sensing, GIS layers, etc.) and text
Advanced analytical capacities with the embedded “R” statistical package
5
6
7
Data access and sharing (FENIX platform)
Food and Agriculture Organizationof the United Nations
Statistics Division - ESS
CGIARAccess and dissemination of the Agricultural Science & Technology Indicators (ASTI) through FAOSTAT
Technological Integration of other systems with FENIX
UNFCCC / IPCCDissemination of Greenhouse Gases data in FAOSTAT using the UNFCCC methodology
USDAIntegration of Production, Supply & Demand database in AMIS using web services and widgets
IMFDissemination of Government Expenditure data in FAOSTAT
IMFCollaboration for a joint implementation of the SDMX standard specifications
OECDIntegration of Official Development Assistance (ODA) data in from the Creditor Reporting System in ADAM
IFPRISharing of HarvestChoice GIS layers through Web Map Services (WMS)
National DataLots of national data from different institutions in CountrySTAT
FAO challenge (t-5) FAO solution CGIAR challenge
Distributed data dissemination systems and platforms
New FAOSTAT platform, integration of internal and external sources
Disparate data dissemination?
Limited adoption of international
standards (metadata, classifications, data exchange formats, code lists)
End of proprietary classifications, shift to CPC, HS, adoption of SDMX, DDI, MDM and universal code lists
International standards, classifications? DDI?
Limited institutional integration, weak
governance both internally and externally (CC and GPG)
SCWG and SPSC, IDWG, Chief StatisticianGlobal FAO Commission on Statistics, Integrated regional commissions
Cross-centre governance?Cooperation with IOs?
Limited user orientation User-oriented statistical products, e.g. Capital stock and investment data
User needs known?
Cooperation: Limited integration of statistical initiatives both in house and with external partners
Internal: Cross-cutting work on AMIS, ADAM, FAOSTAT, SDWExternal: UNSD, C-STAT, GS, AMIS
Well integrated?
Data life cycle management: no integration of the data production cycle, upstream and downstream
Harvesting – Dissemination: New SWS, new FAOSTAT, CS, country focal points
Common approach to data life cycle systems?
IT platforms: limited interoperability within FAO and countries/regions
FENIX platform for all data products (FAOSTAT, C-STAT, ADAM,
Common standards, platforms?
Data quality management Corporate QAF Any QAF?
Food and Agriculture Organizationof the United Nations
Statistics Division - ESS
Thanks
Food and Agriculture Organizationof the United Nations
Statistics Division - ESS
The largest global database on agriculture
Supports national/regional agencies on data preparation and publication. It feeds into
FAOSTAT
Early warning and monitoring tool for food market prices and
balances for the 4 main food commodities (wheat, maize,
rice and soybean)
Helps mobilize resources and tracks official development assistance (ODA) data flows
Technological Integration of FAO Systems with FENIX
Food and Agriculture Organizationof the United Nations
Statistics Division - ESS
Food and Agriculture Organizationof the United Nations
Statistics Division - ESS
Food and Agriculture Organizationof the United Nations
Statistics Division - ESS
Open source technologies used by FENIX