FAO Approach to Data Exchange and Dissemination

Food and Agriculture Organizationof the United Nations

Statistics Division - ESS

CGIAR Data Standard Summit, November 26-28, Rome, Italy

FAO Approach to Data Exchange and Dissemination

An introduction to a free, flexible and transferable platform

Josef SchmidhuberDeputy DirectorFAO STATISTICS DIVISION



CHALLENGESCLOSE THE GAP BETWEEN DATA AVAILABILITY AND DATA NEEDS



Lack of data and metadata description and

harmonization

Restrictive or unclear data policies

Dispersed and uncoordinated data life cycle management

Poor governance, ineffective or missing institutional frameworks (internal & external)

Limited interoperability between systemsLimited user orientation and focus on needsPoor data dissemination systems, limited communication and user awareness

1

2

3

4

5

6

7



ESS APPROACH TO DATAMANAGEMENTKey principles



Data and tools provided as a “public good”; no licensing constraints; full redistribution rights

Open data are converted into accessible data (standards, classifications, formats)

Technical in-country assistance provided. Data collected and disseminated are the result of the collaborative effort between FAO, countries and regions.

1

2

3



Ensure country ownership of data, methodologies and IT systems

Ensure that data produced and formats used meet user needs

Data sharing strengthened by an open-source IT platform

5

6

4



FAO APPROACH TO

DATAKEY COMPONENTS



Partnership & Institutional network(Committees of key stakeholders)

Data access

and sharing

(IT platform) Dat

a man

agem

ent

and

harm

oniza

tion

(stati

stica

l met

hods

)

Statistical governanceData p

olicy

& d

ata s

harin

g rul

es

Definition of standards and tools

Political dimension

Technical dimension

Data sharing Network

(SDMX, DDI)





Statistical Governance System: external and internal

CountrySTAT: Support to Multidisciplinary Technical Working Groups (TWG) already established in 23 countries and several regional organizations

Linking all data initiatives such as CountrySTAT, the Global Strategy, AMIS, Censuses, IHSN

Agency-to-agency collaboration: with international agencies (WB, IMF, OECD, CGIAR)

1

2

3

4

Partnership & Institutional network





Adoption of international standards

Universal code-lists and common metadata templates

Use of international statistical classification systems, mapping systems and DB

Data collection and processing methodologies to ensure data compatibility / comparability

1

2

3

4

Data management and harmonization





Service-oriented architecture (SOA) based on open-source technology

Integrated metadata and implementation of well-known data exchange protocols (e.g. SDMX, DDI)

Web services (APIs) to facilitate data sharing between countries, regions and international organizations

Widget approach to deliver functionalities and integrate tools with other systems/websites

1

2

3

4

Data access and sharing (FENIX platform)



Different options for data collection:• Synchronization of data via APIs• Online data entry forms• Uploads from csv/xls files• Automated load of email attachments• Collection of data through smartphones

Ability to handle databases, geospatial data (e.g. remote sensing, GIS layers, etc.) and text

Advanced analytical capacities with the embedded “R” statistical package

5

6

7

Data access and sharing (FENIX platform)



CGIARAccess and dissemination of the Agricultural Science & Technology Indicators (ASTI) through FAOSTAT

Technological Integration of other systems with FENIX

UNFCCC / IPCCDissemination of Greenhouse Gases data in FAOSTAT using the UNFCCC methodology

USDAIntegration of Production, Supply & Demand database in AMIS using web services and widgets

IMFDissemination of Government Expenditure data in FAOSTAT

IMFCollaboration for a joint implementation of the SDMX standard specifications

OECDIntegration of Official Development Assistance (ODA) data in from the Creditor Reporting System in ADAM

IFPRISharing of HarvestChoice GIS layers through Web Map Services (WMS)

National DataLots of national data from different institutions in CountrySTAT

FAO challenge (t-5) FAO solution CGIAR challenge

Distributed data dissemination systems and platforms

New FAOSTAT platform, integration of internal and external sources

Disparate data dissemination?

Limited adoption of international

standards (metadata, classifications, data exchange formats, code lists)

End of proprietary classifications, shift to CPC, HS, adoption of SDMX, DDI, MDM and universal code lists

International standards, classifications? DDI?

Limited institutional integration, weak

governance both internally and externally (CC and GPG)

SCWG and SPSC, IDWG, Chief StatisticianGlobal FAO Commission on Statistics, Integrated regional commissions

Cross-centre governance?Cooperation with IOs?

Limited user orientation User-oriented statistical products, e.g. Capital stock and investment data

User needs known?

Cooperation: Limited integration of statistical initiatives both in house and with external partners

Internal: Cross-cutting work on AMIS, ADAM, FAOSTAT, SDWExternal: UNSD, C-STAT, GS, AMIS

Well integrated?

Data life cycle management: no integration of the data production cycle, upstream and downstream

Harvesting – Dissemination: New SWS, new FAOSTAT, CS, country focal points

Common approach to data life cycle systems?

IT platforms: limited interoperability within FAO and countries/regions

FENIX platform for all data products (FAOSTAT, C-STAT, ADAM,

Common standards, platforms?

Data quality management Corporate QAF Any QAF?



Thanks



The largest global database on agriculture

Supports national/regional agencies on data preparation and publication. It feeds into

FAOSTAT

Early warning and monitoring tool for food market prices and

balances for the 4 main food commodities (wheat, maize,

rice and soybean)

Helps mobilize resources and tracks official development assistance (ODA) data flows

Technological Integration of FAO Systems with FENIX







Open source technologies used by FENIX

http://www.r-project.org/index.html