DATA RESCUE IN AFRICA: SYNERGY WITH NATIONAL AND REGIONAL

PROJECTS

Climate&Environment DepartmentACMAD

International Workshop on the Rescue and Digitization of Climate Records for countries in West Africa

19-23 November 2012, Accra, Ghana

- Historical Background

- ACMAD plans for African Data Rescue& Management

- Proposed support to other DARE projects in Africa

- Concluding Remarks

OUTLINE

HISTORICAL BACKGROUND

With funding from Belgium, WMO sent experts in 46 African countries to locate and image hydromet data

Microfiches of over 30 millions old observations from about 1000 sites were produced with copies to

countries

Microfiches were handed over to ACMAD around themid 90s

Discussions with IEDRO and recently with IRI at Columbia University for imaging and digitalization

Call for bid and selection of a firm to procure data rescue equipments and related training at ACMAD

( Aug-Oct 2012)

Training of ACMAD/country experts on scanning of

microfiches, creation of images library , data keying and

entry into the national and continental databases

Inventory , scan all microfiches ( about 100

thousands) and save as digital images

Key in the data and enter them into the ACMAD/Regional

Climate Centre’s African climate database

ACMAD DATA RESCUE PLANS

Provide on the job training on data rescue at ACMAD for countries

Countries data Rescue with seconded experts and visiting scientists programmes at ACMAD

Each country expert will bring back a copy of digitized

Data in a Climsoft managed database interfaced with application tools ( RCLIMDEX, CPT, Excell, GIS macros…) for climate monitoring and forecasting products

Organize workshops to produce climate indices, Atlases

and other added value products using rescued data

ACMAD DATA RESCUE PLANS ( support to country and sub-regional DARE projects)

DEVELOPMENT AND USE OF THE CONTINENTAL CLIMATE DATABASE AT ACMAD

Data base design and architecture

Ingestion of Daily Data

Ingestion of Synoptic data

Curent status of the Data base

DATA BASE DESIGN AND ARCHITECTURE DATA BASE DESIGN AND ARCHITECTURE Data model is the model for CLIMSOFT version 3, which was last

updated in 2010

Data model allows storage of different data formats: numeric, text, Boolean (Yes/No) and graphical images

Though originally designed in MS Access, the database design has been ported to MySQL and PostgreSQL. It can be ported to any other RDBMS.

For the ACMAD climatological database, the open source PostgreSQL was chosen for its efficiency, stability and robustness.

PostgreSQL data storage capacity is on the order of Terabytes (TB).

INGESTION OF DAILY SYNOP DATA

CLIMSOFT ingest GTS data via FTP.

The data are automatically imported into the CLIMSOFT database at a scheduled time.

This module for the ingestion of synoptic data became operational in June 2012.

Current Database StatusCurrent Database Status

Historical daily data from the NOAA/NCDC FTP site date back to 1972.

GTS SYNOP data imported from GTS data reception system at ACMAD.

Total database size is approximately 24 GB

Very Basic QC is carried out. We are planning to implement more rigorous Quality checks.

PROVISION OF FORMATED PROVISION OF FORMATED DATA FOR CLIMATE DATA FOR CLIMATE OPERATIONSOPERATIONS Table of 10-day summaries for selected stations for all Africa input to dekadal Bulletins

Data output in RClimDex, CPT and Instat inputs formats.

We have plans to include other outputs in formats required by any climate application tool

Long-term climatological means generated for all GTS stations in Africa.

Climatological means available for the period 1981-1990, 1991-2000, 2001-2010, 1981-2010.

Data products in XML format and can be viewed with any web browser

RCLIMDEX and INSTAT input data format from Chad RCLIMDEX and INSTAT input data format from Chad climsoft climsoft

Database at ACMADDatabase at ACMAD

Multiple key entry for countries data using on the job training, secondment and visits on data rescue to reduce possible errors

Continue development of CLIMSOFT for station data by building interfaces with the web, application tools, other environmental database management systems as well as GIS software

Expand CDMS training across Africa integrated into workshops for generating value added information for climate data