Stripe Review of Social Sciences in the CGIAR

Chris Barrett (panel chair)Arun AgrawalOliver Coomes

Jean-Philippe Platteau

September 2009

Review commissioned by CGIAR Science Council- Follows 2007 scoping paper and Phase 1 desk study

in 2008: background review and normative framework for CGIAR SS.

Phase 2 review: Panel mandate: provide a System-level assessment of SS in the CGIAR – quality, coherence, relevance, productivity, capacity – and to provide recommendations for improvement.- 1/2009 global e-consultation; site visits to Centers;

ARI and NARS surveys; extensive reading of CGIAR publications, project documents, impact and partnership statements, etc. Panel interviewed more than 260 people from 13 Centers/CPs and many stakeholders organizations.

- 1st draft reviewed by 9 Centers, 2 CPs and 15 experienced observers of CGIAR social science.

Background

Great opportunities:1) Demand for agricultural research is resurgent among donors and governments. 2) Renewed interest in empirical development research in the social sciences at the elite levels globally.

But, CGIAR has to be strategic:3) Developing country university/NARS SS is weakened.4) CGIAR is a modest-sized player: 310 IRS social scientists are a small fraction of the global research community.5) Increasingly competitive landscape: NGOs, ARIs, World Bank, private firms and others play many of the roles CGIAR social science does.6) Donors far more demanding, esp. of impact evidence.

External environment

CGIAR social science has comparative advantage in generating international public goods (IPGs) due to:(a)Low interdisciplinary boundaries and problem

focus(b)Locational advantages, honest broker reputation(c) Highly trained, IRS staff

Key topical areas of CA thus relate to:(i) sustainable agricultural productivity

increases by and for the poor (ii) natural resource conservation that benefits the

rural poor directly or indirectly via crucial environmental services, especially to help agricultural productivity

(iii)institutional, policy and technological innovations that enhance the quality of life for poor and marginalized agrarian populations.

Comparative advantage

Growing Heterogeneity, with Declining Average Quality- Much strong CGIAR social science, especially where

adequate core funding, critical mass, strong ties w/ARIs and NARS, and a local culture that prizes rigorous research.

- But much CGIAR social science is methodologically weak, w/ limited outputs of mediocre quality and little clear impact either on development outcomes or global research directions.

- Mismatch of new business model – restricted funding and short-term development impact oriented – but with old staffing model of PhD scientists. Donors and management need to focus on quality rather than growth before mediocrity is locked in by hiring patterns and cultural transformation.

Panel Assessment

Funding:Total real expenditures grew 69% 1985-2008, but unrestricted fell 39%..

By 2007, only 4 Centers had U/R > 0.6; most are 0.4-0.5 .

This implies staff only 30-40% core funded. Far too fragmented.

0

50

100

150

200

250

300

350

1985 1987 1989 1991 1993 1995 1997 1999 2001 2003 2005 2007

Real

exp

endi

ture

s (19

85 U

S$ m

illio

n)

Unrestricted Restricted

Panel Assessment

Staffing:- Total IRS grew 17% 1995-2008, but SS IRS nearly

doubled, up to 26.7% of all CGIAR IRS in 2008, up from 17% in 1995.

- SS very junior: more than 1/3 within 8 years of their Ph.D.

- Economics heavy (60%); 8% have no SS graduate degree!

- Concentrated at IFPRI (29%); most not at Centers’ HQs.

- Widespread morale problems, esp. in smaller Centers.

- Big recruitment/retention problems, esp. mid/senior level.

- Compensation globally uncompetitive at junior level, highly variable but generally competitive at more senior levels.

Panel Assessment

Pursuit and Measurement of Impact:Impact is difficult to measure but apparent in CGIAR SS.

Key impacts come from ex ante IA for research prioritization. But this has dwindled as demand for ex post IA has grown.

Most CGIAR EPIA is poorly done – quantity trumping quality – and has low credibility with scientists and very limited impact on donor or management decisions.

Centers’ perverse incentives for low quality EPIA have trumped SPIA and others’ efforts at upgrading CGIAR EPIA.

ILAC/IS concerns valid and important. But System lacks skills for it and limited potential as a research tool.

Panel Assessment

Organization and Partnerships:Some reorganization of CGIAR social science is needed. - Lack of critical mass at most Centers.

- Excessive fragmentation of effort. Management and culture see no boundaries (derives from growth orientation and restricted funding dependence).

- Matrix management essential … best of the bad options.

- Eroded quality of partnerships, especially with ARIs. Increasingly transactional. Limited inter-Center cooperation.

- Failure to seize available economies of scale (e.g., data mgmt, research methods, library support, etc.)

Panel Assessment

Research Quality:- Insufficient serious research prioritization; much chasing $.

- Highly varied. A few world-class social scientists. And offset by too frequent poor quality research design and outputs.

- Too much one-time, special purpose data collection; too little time spent by IRS on design and data collection/cleaning. Most data not publicly available/useful, missing IPGs.

- Systematic underinvestment in long-term, high-quality longitudinal data to track changes in rural systems.

- Only 2 Centers appear to use an Institutional Review Board to ensure research on human subjects meets ethics standards.

Panel Assessment

Research Quality (continued):- Big dispersion in output rates and quality: > 25% of

IRS SS had no publications 2005-7 while 15% had >4 articles/year.

- Little shared sense as to what constitutes high quality social science research and thus few clear incentives for investing in research quality, as opposed to quantity, and few penalties for generating poor quality outputs.

- Limited demand for CGIAR published research, as measured by downloads, paper citations and scientists’ lifetime citations (e.g., Hirsch’s h index).

- Quality of 216 “best publications” highly uneven.

Panel Assessment

Global + local (“Glocal”) Solutions

Recommendations

Recommendation 1:Undertake essential management reforms

1a) Resolve the mismatch between the business model and

staffing patterns1b) Realign management incentives1c) Improve leadership selection 1d) Tighten the focus on comparative advantage1e) Focus on impact but end the impact measurement

obsession1f) Mainstream gender equity as a basic axiom of CGIAR

research1g) Require full indirect cost recovery

Global + local (“Glocal”) Solutions

Recommendations

Recommendation 2:Re-organize and re-focus CGIAR Social Science

2a) Restore longer-term partnerships, especially upstream 2b) Focus training and capacity building on research

mentoring2c) Organize a Regional Systems Analysis Mega-Program2d) Organize a Mega-Program on Stimulating and

Evaluating Innovations 2e) Shrink unproductive social science units

Global + local (“Glocal”) Solutions

Recommendations

Recommendation 3:Update social science personnel management practices

3a) Introduce a CGIAR Young Scientists Program 3b) Increase entry-level compensation packages3c) Establish a clear research career track for social scientists3d) Restore competitive travel and sabbatical programs3e) Employ modern human resources management practices

Global + local (“Glocal”) Solutions

Recommendations

Recommendation 4:Foster a culture of rigorous social science research

4a) Restore social science research seminar programs 4b) Individual performance measurement4c) Stop wasting money on in-house publications other than

policy or research briefs4d) Establish a CGIAR Institutional Review Board

Global + local (“Glocal”) Solutions

Conclusions

The CGIAR needs high quality SS and the global SS community needs the CGIAR. The opportunities today are great.

But the model for high quality CGIAR SS is broken:- General loss of focus on areas of comparative

advantage- Excessive heterogeneity in output and research

quality- Dilution of partnerships and thus of direct and

indirect impact

The System must:- sharply increase unrestricted share of funding for IRS- more tightly focus on areas of comparative advantage- attract and retain the best social scientists

The time is ripe for change in CGIAR social science and the likely rewards to effecting such change are great.

Thank you for your time, interest and comments!

Thank you