14
CGIAR Consortium Board CB21-03 Twenty-First Meeting, Virtual, 29 July 2015 Page 1 of 14 CB21-03 Revision 1 IEA CRP Evaluations – Consortium Management Responses Purpose: This document annexes the Consortium Management responses to the following Independent Evaluation Arrangement (IEA) CGIAR Research Program (CRP) evaluations: PIM (Policies, Institutions and Markets) Maize, Revision 1 Wheat, Revision 1 AAS (Aquatic Agricultural Systems) The management responses have been prepared in line with the Consortium Office’s standing processes for review of IEA final evaluation reports. Noting that the 4 evaluations were commissioned under Section 23 of the CGIAR Consortium-Fund Council Joint Agreement 1 , the Consortium Board was requested to endorse the management responses pursuant to its CRP oversight role set out in Article 14(9). Record of decision taken by the Board at its Twenty-first meeting on 29 July 2015: 1. The Consortium Board supports the Consortium Office overall process for preparation of Consortium Management Responses as and when final evaluations become available. 2. The Consortium Board endorses the Consortium Management Responses to the IEA reports for PIM, Maize - Revision 1, Wheat – Revision 1, and AAS (as set out in Annexes 1, 2, 3 and 4 of this paper), for submission to the IEA and Fund Council, and requests that they be issued after consideration of the appropriate Board response to the questions arising on the IPFRI assessment. 1 The 15 April 2011 agreement entered into between Bioversity International (on behalf of CGIAR before establishment) and the Fund Council (as represented by the International Bank for Reconstruction and Development), and approved by the Fund Council on 5 April 2011 and the Consortium Board on 5 May 2011.

Consortium IEA management response › cgiarorg › 2015 › 11 › CB21-03_IEA... · Synthesis of all 15 CRP evaluations: April 2016 2 Final Meeting Summary, Fund Council November

  • Upload
    others

  • View
    6

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Consortium IEA management response › cgiarorg › 2015 › 11 › CB21-03_IEA... · Synthesis of all 15 CRP evaluations: April 2016 2 Final Meeting Summary, Fund Council November

CGIAR Consortium Board CB21-03 Twenty-First Meeting, Virtual, 29 July 2015 Page 1 of 14

CB21-03 Revision 1

IEA CRP Evaluations – Consortium Management Responses Purpose: This document annexes the Consortium Management responses to the following Independent Evaluation Arrangement (IEA) CGIAR Research Program (CRP) evaluations:

PIM (Policies, Institutions and Markets)

Maize, Revision 1

Wheat, Revision 1

AAS (Aquatic Agricultural Systems) The management responses have been prepared in line with the Consortium Office’s standing processes for review of IEA final evaluation reports. Noting that the 4 evaluations were commissioned under Section 23 of the CGIAR Consortium-Fund Council Joint Agreement1, the Consortium Board was requested to endorse the management responses pursuant to its CRP oversight role set out in Article 14(9). Record of decision taken by the Board at its Twenty-first meeting on 29 July 2015:

1. The Consortium Board supports the Consortium Office overall process for preparation

of Consortium Management Responses as and when final evaluations become available.

2. The Consortium Board endorses the Consortium Management Responses to the IEA reports for PIM, Maize - Revision 1, Wheat – Revision 1, and AAS (as set out in Annexes 1, 2, 3 and 4 of this paper), for submission to the IEA and Fund Council, and requests that they be issued after consideration of the appropriate Board response to the questions arising on the IPFRI assessment.

1 The 15 April 2011 agreement entered into between Bioversity International (on behalf of CGIAR before

establishment) and the Fund Council (as represented by the International Bank for Reconstruction and Development), and approved by the Fund Council on 5 April 2011 and the Consortium Board on 5 May 2011.

Page 2: Consortium IEA management response › cgiarorg › 2015 › 11 › CB21-03_IEA... · Synthesis of all 15 CRP evaluations: April 2016 2 Final Meeting Summary, Fund Council November

IEA CRP Evaluations – Consortium Management Responses

CGIAR Consortium Board CB21-03 Twenty-First Meeting, Virtual, 29 July 2015 Page 2 of 14

Part 1 - Background 1. In November 2013, the CGIAR Fund Council agreed2 that all current CGIAR Research

Programs (CRPs) should undergo some form of evaluation before full proposal development is initiated for the next generation of CRPs to provide information for decisions on program formulation and selection during the CRP 2nd call full proposal writing, review and approval processes in 2016.

2. In that context, it was agreed that IEA would commission 10 evaluations, and support the management of self-evaluations by the other 5 CRPs during the period 2014-2016.

3. The status of the various evaluations is set out in table 1 below. Table 1: IEA Summary – CRP Evaluation status July 2013

CRP Evaluations

Draft report for comments

Final Evaluation Report

Responses

CRP Mgmt Consortium

FTA Completed July 2014 September 2014 November 2014

PIM Completed April 2015 April 2015 Awaiting response

AAS Completed April 2015 June 2015 Awaiting response

Maize Completed April 2015 April 2015 Awaiting response

Wheat Completed April 2015 April 2015 Awaiting response

Synthesis of first 5 CRP evaluations: September 2015

CCAFS September 2015 November 2015 December 2015 scheduled for Jan 2016

RTB November 2015 December 2015 January 2016 scheduled for Feb 2016

WLE November 2015 December 2015 January 2016 scheduled for Feb 2016

GRiSP October 2015 October 2015 November 2015 scheduled for Dec 2015

Livestock and Fish

November 2015 December 2015 January 2016 scheduled for Feb 2016

CRP Commissioned Evaluations (with IEA QA support)

CRP Mgmt QA Review (IEA)

A4NH N/A for Cons July 2015 October 2015 Feb 2016

Dryland Systems

N/A for Cons October 2015 December 2015 February 2016

Dryland Cereals

N/A for Cons December 2015 February 2016 February 2016

Grain Legumes N/A for Cons December 2015 February 2016 February 2016

Humidtropics N/A for Cons November 2015 December 2015 February 2016

Synthesis of all 15 CRP evaluations: April 2016

2 Final Meeting Summary, Fund Council November 6 – 7, 2013, Agenda Item 10 – CRP Extension and

Synchronization Plan, page 28, Conclusions and Decisions items 1 and 2.

Page 3: Consortium IEA management response › cgiarorg › 2015 › 11 › CB21-03_IEA... · Synthesis of all 15 CRP evaluations: April 2016 2 Final Meeting Summary, Fund Council November

IEA CRP Evaluations – Consortium Management Responses

CGIAR Consortium Board CB21-03 Twenty-First Meeting, Virtual, 29 July 2015 Page 3 of 14

Part 2 - Discussion 4. As set out in table 1 above, the Consortium’s management response comprises a final

step in advance of the full evaluation package being presented to the Fund Council under the terms of the Joint Agreement.

5. Systematically, before the Consortium is formally requested by the IEA to provide its management response, the following steps have been undertaken by IEA:

a. First, Preparation of assignment specific Terms of Reference; b. Next, compilation of an Inception Report to:

i. Present the principal purposes and scope of the evaluation, including the major overarching questions of the evaluation

ii. Provide an overview of the major evaluation issues and questions under two major topics: research/programmatic performance and organizational performance; and

iii. Provide the evaluation approach and methodology to answering these questions, with a particular emphasis on the data collection and data analysis instruments. The

c. Next, completion of a draft Evaluation Report which is discussed in debriefings with management of the relevant CRP and the Evaluation Reference Group

d. Next, presentation of the final Evaluation Report e. Next, an invitation to the CRP Management to present a formal CRP

Management Response; and f. Finally, an invitation to the Consortium to present a formal Consortium

Management Response.

6. A communications protocol exists between the IEA and the CCGIAR Independent Audit Unit (IAU) to ensure ongoing sharing of information throughout the various IEA steps.

7. All four of the CRP evaluations that are the subject of this paper are now at stage 5.f – Consortium Management Response.

8. Recognizing the voluminous nature of materials, table 2 below sets out an index to the summarized material that accompanies this paper. Attachments are separate files that contain the IEA’s Evaluation Executive Summary and CRP Management Response, and Annexes set out the proposed Consortium’s Management Response, and these follow within this same document. Table 2: Index of attachments

Separate Attachment

CRP reference Annex Number

A Policies, Institutions and Markets 1

B Maize, Revision 1 2

C Wheat, Revision 1 3

D Aquatic Agricultural Systems 4

Page 4: Consortium IEA management response › cgiarorg › 2015 › 11 › CB21-03_IEA... · Synthesis of all 15 CRP evaluations: April 2016 2 Final Meeting Summary, Fund Council November

IEA CRP Evaluations – Consortium Management Responses

CGIAR Consortium Board CB21-03 Twenty-First Meeting, Virtual, 29 July 2015 Page 4 of 14

9. The Consortium Office’s suggested Management Response to the four evaluations set out in annexes to this paper has been developed according the following comprehensive steps:

a. The Consortium Chief Science Officer and CEO reviewed the full IEA Evaluation Report and CRP Management Response to identify the elements of the materials that are within the scope of the Consortium’s responsibility, either from an oversight perspective, or looking forward to preparation of the 2nd call full proposal guidance note;

b. Preparation of an initial draft Consortium Management Response, which is discussed across the Consortium Office’s leadership team members, as relevant to the areas of accountability;

c. Formal clearance by the Consortium CEO before submission for endorsement

by the Consortium Board; and

d. Sharing of the proposed Consortium Management Response with the relevant CRP leader in advance of presentation to the Consortium Board for endorsement. If circumstances arise where the Consortium Office identifies an important area of disagreement with either the IEA evaluation report and/or the CRP Management response, the matter discussed before the recommendation goes forward. This has not been necessary in the case of the 4 documents now presented to the Consortium Board.

10. It is noted that in September 2015, in advance of release of the 2nd call full proposal

guidance notice, the IEA plans to release a synthesized report arising from the first five IEA evaluations (including the Forests, Trees and Agroforestry CRP evaluation that was fully completed by November 2014). That timing facilitates common lessons being incorporated into the full proposal development phase, and their subsequent review, to supplement other lessons/information points coming from the pre-proposal review process that was not envisioned at the time of the November 2013 Fund Council decision to incorporate this review process.

11. Noting that an IEA synthesized report is already planned, and the Consortium Board will be asked to provide a formal response to that synthesized report, the Consortium Office has not sought to duplicate this work in its proposed Management Responses. Instead it has focused on two to three areas within each evaluation that are seen as most relevant to the Consortium’s facilitative and oversight role, with some areas of commonality, but also quite different areas being identified by the evaluation teams.

12. Subsequent to Consortium Board endorsement, the final Consortium Management Responses are systematically communicated to the CRP leadership, the IEA for publication on the IEA’s evaluation website (http://iea.cgiar.org/evaluations), and also presented as an information piece to the IAU.

Page 5: Consortium IEA management response › cgiarorg › 2015 › 11 › CB21-03_IEA... · Synthesis of all 15 CRP evaluations: April 2016 2 Final Meeting Summary, Fund Council November

IEA CRP Evaluations – Consortium Management Responses

CGIAR Consortium Board CB21-03 Twenty-First Meeting, Virtual, 29 July 2015 Page 5 of 14

Annex 1

Consortium Management Response to the External Review of PIM The Policies, Institutions and Markets CRP was approved in December 2011 and the PIM evaluation started in February 2014 and was finalized in April 2015. The review emphasizes that the evaluation is considered formative in the sense that it covered the first three years of a new program, but that it also has evaluated the legacy work on which PIM is based, primarily from IFPRI, but also the pre-existing inter-center collaborative work that preceded PIM such as the CAPRI program or the gender research work. The overall conclusion of the evaluation team was that:

“PIM has added sufficient value to the CGIAR’s research on policies, institutions, and markets to warrant the continuation of a CRP like PIM in the second round of CRPs, starting in 2017. Engaging most of the social scientists across the CGIAR System has created an intellectual critical mass to pursue cutting-edge science. Inter-Center collaboration is

(a) strengthening the impact pathways of PIM research by engaging Centers that are closer to the users of the research;

(b) facilitating a more integrated approach to development challenges through a combination of discovery and delivery-type research;

(c) deepening country-level partnerships that have greater potential for more immediate development outcomes;

(d) enabling researchers in the commodity-based Centers to work on important socio-economic issues that are broader than the commodities covered by their Centers; and

(e) helping to raise the scientific quality of social science and policy research in the participating Centers.

The evaluation also concludes that IFPRI should continue to host the PIM Management Unit. IFPRI is the only CGIAR Center that has a principal focus on social science research, the largest concentration of social scientists in the System, and the strongest research infrastructure and quality assurance systems to support high quality social science research. Many of the benefits of PIM arise from connecting the social scientists in the participating Centers to IFPRI through PIM. However, all is not well with respect to the governance and management of PIM. In what follows, the evaluation draws conclusions and makes recommendations in a number of areas to improve PIM’s operations to become a more effective program.

Hereafter this note highlights some of the key conclusions of the review that relate particularly to the CGIAR as a whole, or to the management of the CRP that affects the Consortium’s oversight role.

Page 6: Consortium IEA management response › cgiarorg › 2015 › 11 › CB21-03_IEA... · Synthesis of all 15 CRP evaluations: April 2016 2 Final Meeting Summary, Fund Council November

IEA CRP Evaluations – Consortium Management Responses

CGIAR Consortium Board CB21-03 Twenty-First Meeting, Virtual, 29 July 2015 Page 6 of 14

The CGIAR has a strong comparative advantage in conducting social science research at the intersection of food security, poverty, and sustainable agriculture. There are, for example, few organizations or institutions that have a similar combination, range, and quality of modeling systems at their disposal as IFPRI, and therefore PIM.”

The evaluation reviews at length how the short term (annual allocation) of the W1-2 funding, and the lack of stability in these allocations (as well as the much lower level than foreseen in the proposals against which the CRPs were approved resulted in the following situation):

PIM has allocated its W1-2 funds, for the large part, to support new or expanded components of larger projects already being supported by W3 or bilateral funding. Principal investigators have come to view W1-2 funds as largely short-term and flexible and bilateral funds as more long-term and reliable, due to the uncertainties associated with the amount of W1-2 funding available and the annual allocation of these funds. They now tend to regard bilateral funds as forming the foundation of their research programs, and the W1-2 funds as supporting research which builds on this foundation.

The evaluation discusses the governance and management structures of PIM at some length, with an overall conclusion that the governance and management does not sufficiently involve the other CG Centers, or external partners, nor is there a steering committee that provides oversight – resulting in the risk of conflict of interest between the lead center and the other stakeholders:

PIM does not have a CRP governing body that exercises strategic direction and oversight of the program. It does have a Science and Policy Advisory Committee that has exercised some strategic direction in an advisory capacity, but little or no oversight.

On the quality of science the evaluation team concludes that:

The quality of science in PIM is highly variable. The program is doing well on the relevance of scientific topics and quality assurance mechanisms. It is doing less well in relation to minimum standards of scientific productivity and impact.

On gender research the evaluation team acknowledges the pioneering role of IFPRI scientists in this area, and the high degree to which gender-related activities are prioritized (roughly 30% of the overall budget) and the high degree of gendered data collection (an estimated 50% of all datasets), but it concludes that:

… an effective system to monitor the progress of PIM’s own activities with respect to gender is still a work in progress, and the Consortium has not yet provided sufficient guidance on methodologies to capture the level of attention given to gender issues.

Page 7: Consortium IEA management response › cgiarorg › 2015 › 11 › CB21-03_IEA... · Synthesis of all 15 CRP evaluations: April 2016 2 Final Meeting Summary, Fund Council November

IEA CRP Evaluations – Consortium Management Responses

CGIAR Consortium Board CB21-03 Twenty-First Meeting, Virtual, 29 July 2015 Page 7 of 14

So while the overall conclusion of the evaluation is that PIM’s work is relevant and of high quality, it makes a number of recommendations, many of which are highly relevant to the development of a PIM proposal for the CRP 2nd Call. Of the 15 recommendations made by the evaluation team:

4 are governance recommendations directed to the IFPRI Board of Trustees, the Fund Council and the Consortium;

6 are strategic recommendations directed to the (proposed) PIM Independent Steering Committee; and

5 more are operational recommendations directed to PIM management. Of the four governance recommendations, the first two recommend continuation of PIM into the CRP second stage, led by IFPRI. As part of the second recommendation, the evaluation team recommends that the IFPRI Board of Trustees should put in place a conflict of interest policy. IFPRI’s management response is that it partially accepts this recommendation as it proposes that the Consortium ought to develop a system-wide conflict of interest policy which PIM would then adopt. The Consortium agrees. With the proposal made by IFPRI.

The third recommendation is that:

PIM should put in place an Independent Steering Committee in accordance with the recent (January 2015) agreement between the Fund Council and the Consortium regarding CRP governance structures for the next generation of CRPs. PIM should consider having representatives of the participating Centers as well as independent members on the Steering Committee that would have greater than their numerical say in the allocation of W1-2 resources. The IFPRI Board of Trustees should delegate programmatic responsibility to the Steering Committee while retaining the fiduciary responsibility for ensuring that the W1-2 funds are used for their intended purposes.

The Consortium fully agrees with this recommendation. The Lead Center management response is that it partially agrees with this recommendation; it notes:

The evaluators recommend changes in governance, and these recommendations are welcome. Changes in governance may be warranted, and should be consistent across CRPs. The IEA should provide an overview of recommendations on governance from the different evaluations. If the IEA agrees that each CRP should have its research program managed by a Steering Committee, that committee would need to be advisory to the Lead Center’s Board.

The wording of the recommendation with regard to the role of participating Centers in allocation of W1-2 resources is unclear. Management argues that participating Centers have conflict of interest in allocation of W1-2 resources. Hence Center representatives on the steering committee should be a minority, and should be recused from budget processes.

Page 8: Consortium IEA management response › cgiarorg › 2015 › 11 › CB21-03_IEA... · Synthesis of all 15 CRP evaluations: April 2016 2 Final Meeting Summary, Fund Council November

IEA CRP Evaluations – Consortium Management Responses

CGIAR Consortium Board CB21-03 Twenty-First Meeting, Virtual, 29 July 2015 Page 8 of 14

The Consortium refers to the IEA evaluation of CRP Governance and management, which led to a new model for the CRPs in the second phase, agreed between centers, Consortium and Fund Council, and summarized in the Guidance for CRP 2nd Call Pre-proposals. The new ISC, which does indeed report to the Lead center Board of Trustees, but with significant delegated authority; and it does have a majority of independent members and an independent Chair. It has representatives of both the Lead Center and Participating centers on the committee, in line with the recommendation of this review. The PIM management response does indicate that PIM will adhere to the new system wide policy on CRP governance structures.

The final governance recommendation, directed at the Fund Council and the Consortium, is that:

the Fund Council and the Consortium should jointly commission a study on the problems that the Centers are facing in sustaining their research infrastructures and other research support under the CGIAR Reform.

The Consortium agrees with the context described by the evaluation in which the uncertainty in financing, particularly the fluctuating annual allocation of W1-2 levels, as well as the approval of initial contracts with budgets that were much higher than the resources that were realistically available, led to problems faced by the Centers in sustaining their research infrastructures. The Consortium considers, however, that these issues were adequately brought to light by the Mid Term Review of the Reform, and are being addressed by the subsequent governance reform now being implemented, as well as the effort to develop a Resource Mobilization Strategy for the CRP second round with the explicit objective to provide stable multi-year financing. The Consortium sees no additional value, therefore, in commissioning the recommended study at the current time and under the current circumstances. The Evaluation team makes a number of highly relevant proposals related to PIM’s strategy and operations, for example recommending a greater focus on the interface between research and policy; implementation of a more programmatic oversight as opposed to the current financial management; and a more strategic approach to collaborating with other CRPs. PIM, in its management response, fully accepts all these recommendations and the Consortium endorses these recommendations as well, and looks forward to seeing these adopted in the PIM Pre-proposal for the CRP 2nd Call.

Page 9: Consortium IEA management response › cgiarorg › 2015 › 11 › CB21-03_IEA... · Synthesis of all 15 CRP evaluations: April 2016 2 Final Meeting Summary, Fund Council November

IEA CRP Evaluations – Consortium Management Responses

CGIAR Consortium Board CB21-03 Twenty-First Meeting, Virtual, 29 July 2015 Page 9 of 14

Annex 2, Revision 1

Consortium Management Response to Maize external review

The Consortium commends the review panel for the quality of the report, and the insights it provides into the MAIZE research program. MAIZE is led by CIMMYT and brings together the research activities of CIMMYT and IITA. Initially approved for a three-year period beginning in mid-2011, a 6-month extension was awarded until the end of 2014, to synchronize the funding cycle for all CRPs and a further extension was approved for 2015–2016. Overall, the Evaluation Team concluded that “MAIZE is progressing well and it represents a promising program that is already delivering results”. The review concludes that:

MAIZE adds value compared with the previous center-based approach to crop research, and warrants continuation beyond the 2015-2016 extension phase.

For the most part MAIZE is a coherent program with a strong comparative advantage in the global setting, largely due to the unique genetic resources held in trust by the two centers

The long established partnerships of CIMMYT and IITA and their presence in and agreements with countries hosting research in the developing regions targeted.

Recommendations from IEA The evaluation team makes 11 recommendations, 8 of which are fully accepted by the Maize CRP team and 3 of which are partially accepted. The three partially accepted recommendations are considered below: Recommendation 2 (Relevance): MAIZE should review its priorities in Flagship Projects 4 and 5 where it has less comparative advantage and where smallholders already have a certain access to appropriate technology. This needs to be considered in the light of the large proportion of W3 funding. In particular, MAIZE needs to consider reducing efforts in final product (hybrid) delivery where the private sector is strong. MAIZE should also consider reducing investments in the non-germplasm components of FP5 areas of aflatoxin and postharvest storage research where other agencies have greater comparative advantage. This recommendation underscores the growing importance of the private sector and the need to take a more strategic view of the role of the maize CRP to magnify areas of comparative advantage. A more strategic view of comparative advantage should emphasize the inclusion of smallholders, in particular women producers, and their organizations as private sector actors. Indeed the evaluation team correctly point out that comparative advantage is a “dynamic term” that needs to be constantly reviewed, reinforcing the need for the Maize CRP to be agile and dynamic in outlook and action. The Consortium will play close attention to this topic when reviewing pre-proposals and also mechanisms that support and stimulate Public Private Partnerships that are now major priorities for key donor agencies.

Page 10: Consortium IEA management response › cgiarorg › 2015 › 11 › CB21-03_IEA... · Synthesis of all 15 CRP evaluations: April 2016 2 Final Meeting Summary, Fund Council November

IEA CRP Evaluations – Consortium Management Responses

CGIAR Consortium Board CB21-03 Twenty-First Meeting, Virtual, 29 July 2015 Page 10 of 14

Recommendation 3 (Relevance): MAIZE should establish pro-active research and monitoring capability to provide foresight on emerging issues in diseases and to support environmental characterization. The need for robust foresight analysis is essential to target and access the appropriate resources and is not a short-term issue but needs to be built into long-term programmatic planning. The CB is fully supportive of this recommendation and would to seek to ensure that appropriate foresight activities were built into a future Maize Agri-food program. Recommendation 6 (Effectiveness): MAIZE should institute management measures to ensure efficiency and effectiveness in management of staff and research activities over the long term. These measures should include: 1) processes for engaging and motivating staff in delivery oriented research through mentoring, training, and cross disciplinary and cross-institutional lateral learning; 2) protocols for data collection and management; 3) streamlined processes for linking exploratory science and research outputs through multiple stages to intermediate products and final products delivered by MAIZE, and; 4) integration of project implementation to program objectives over medium- and long-term through innovation platforms and long-term field trials. The need to convert existing data into knowledge is of great potential, particularly with respect to CGIAR field data and is reflected in the Expressions of Interest issued in the second call guidance documentation. In addition the need to create a balanced portfolio was emphasized by the MTR where discovery science needs to be managed alongside translational product development. The CB supports this recommendation and would encourage the Maize CRP to fully embrace the opportunities reflected in this specific recommendation. In regard to those recommendations that are accepted in full, the Consortium believes it important to reiterate in particular the recommendation on leadership of the MAIZE CRP, as follows: Recommendation 11 (Impact and Sustainability): CIMMYT and IITA should agree on the establishment of a single global maize program in the CGIAR that integrates efforts of the two centers. This MAIZE program should be led by a director. The Consortium considers it imperative that the governance arrangements for the new maize agri-food program are aligned with the IEA governance review, and the levels of responsibility assigned to the CRP Director clearly defined. In conclusion the Consortium accepts and supports all the recommendations of the evaluation team and agrees that they should be taken into account during the development of the Maize Agri-Food System next generation CRP.

Page 11: Consortium IEA management response › cgiarorg › 2015 › 11 › CB21-03_IEA... · Synthesis of all 15 CRP evaluations: April 2016 2 Final Meeting Summary, Fund Council November

IEA CRP Evaluations – Consortium Management Responses

CGIAR Consortium Board CB21-03 Twenty-First Meeting, Virtual, 29 July 2015 Page 11 of 14

Annex 3, Revision 1

Consortium Management Response to Wheat external review WHEAT is led by CIMMYT. It brings together the previously separate mandates of CIMMYT and ICARDA building on their long-term wheat research, and as a CRP was designed for at least ten years duration. It encompasses wheat related breeding, resource management and social science research at CIMMYT, and ICARDA’s wheat breeding. WHEAT was formally launched in the beginning of 2012. It is currently in its fourth year of implementation, having had its 2015-2016 extension proposal approved by the CGIAR Fund Council in November 2014 IEA Recommendations The review panel makes 11 recommendations, of which 10 are fully accepted and one partially accepted by the wheat CRP management. The partially accepted recommendation refers to the strategies, alignment and management of projects to reflect Theories of Change with appropriate and validated critical assumptions. This is considered to be of particular importance by the Consortium given the enhanced need to demonstrate value for money and effective. Thus, the Consortium accepts and supports all recommendations of the evaluation team and agrees that they should be taken into account in the next generation of CRP proposals. Overall, the evaluation concludes that WHEAT is “contributing sufficient value from CGIAR’s research investments to generate results to warrant continuation during the extension-phase (2015-16) and beyond”. Over the past year the management and Boards of Trustees of CIMMYT and ICARDA have made great progress and committed to greater cooperation in program oversight and improved WHEAT management, which includes updating the roles of WHEAT’s Management Committee and Independent Stakeholder Committee and its advisory functions together with the appointment of a WHEAT CRP Director. The following points are of particular relevance to the future development of the Wheat Agri-Food System CRP:

Particular attention should be paid to the plausibility of the Theory of Change and alignment with the SRF sub-IDOs and IDOs in the pre-proposals and full proposals.

In addition the specific use of W1/W2 funds need to be fully clarified so that the strategic integration of bi-lateral support can be fully understood and the added-value demonstrated.

The area of Sustainable Intensification provides a focal point for integration of genetics, agronomy and social sciences and is an important component of a future WHEAT program. More systematic/ meta analyses needs to be undertaken to underpin this work.

Page 12: Consortium IEA management response › cgiarorg › 2015 › 11 › CB21-03_IEA... · Synthesis of all 15 CRP evaluations: April 2016 2 Final Meeting Summary, Fund Council November

IEA CRP Evaluations – Consortium Management Responses

CGIAR Consortium Board CB21-03 Twenty-First Meeting, Virtual, 29 July 2015 Page 12 of 14

To accelerate genetic gain modern breeding approaches need to be rapidly incorporated in the WHEAT program.

Attention to the development of a more refined identification and characterization of end-users of wheat technologies so that outputs effectively meet the needs of farmers on whose decisions to adopt WHEAT’s impact depends. Included here, is the importance of Recommendation 10 advising more energetic attention to gender in impact pathways and project implementation towards gender-equitable outcomes.

In the view of the Consortium, CIMMYT should ensure that all recommendations of the WHEAT evaluation are taken into account when developing the proposal for the Wheat Agri-Food System CRPs.

Page 13: Consortium IEA management response › cgiarorg › 2015 › 11 › CB21-03_IEA... · Synthesis of all 15 CRP evaluations: April 2016 2 Final Meeting Summary, Fund Council November

IEA CRP Evaluations – Consortium Management Responses

CGIAR Consortium Board CB21-03 Twenty-First Meeting, Virtual, 29 July 2015 Page 13 of 14

Annex 4

Consortium Management Response to the External Review of AAS The AAS CRP was approved in July 2011 and this evaluation started in February 2014 and was finalized in March 2015. The review emphasizes that there are many novel elements of the AAS program, but that there was also an important legacy component of WorldFish research. On the one hand, therefore, the evaluation was an early review of a young program, but on the other hand the review assessed the WorldFish research that was brought into the new program and that was based on, in some cases, many years of research. The overall conclusion of the evaluation team was that:

“Aquatic agricultural systems present issues of sufficient importance and relevance to justify further investment by the CGIAR;

AAS has a number of important achievements where the potential for progress can be demonstrated; and

The program faces a range of obstacles and challenges, which affect quality of science, relevance and effectiveness, and to be overcome will require substantial changes in key areas including program design, staffing and implementation.

The evaluation team concludes that to date AAS has been led and managed primarily from the perspective of using AAS as a way to establish and legitimize new skills and competences. Insufficient attention has been given to the historic competences of WorldFish and other CGIAR centers. Instead of minimizing the value of these competences and experiences, the evaluation team recommends using them as an invaluable springboard. It is therefore the primary recommendation of the evaluation team that the CGIAR should justify further investment in aquatic agricultural systems more on the grounds of comparative advantage.” The evaluation team presents ten recommendations that jointly have significant consequences, in the view of the Consortium, for the re-design of the work undertaken in the AAS CRP. These recommendations range from a stronger conceptualization of aquatic agricultural systems, moving away from participatory action research as the core research methodology, significantly increasing the number of PhD level researchers working at field level, significantly increasing the capacity for systems research to moving towards a truly collaborative, multi-center research program. It should be noted that around 95% of all AAS resources were spent by or through WorldFish, with only 5% through IWMI and Bioversity (the only 2 other centers participating in AAS) – and that only WorldFish mapped any bilateral work to AAS (roughly 50% of the total budget). WorldFish and AAS management accepted 9 out of the 10 recommendations fully, but rejected the governance recommendation to significantly strengthen the role of the Program Oversight Panel (POP). Interestingly, AAS does not disagree with the recommended approach for governance, but maintains that the POP already largely played the role as recommended by the evaluation team.

Page 14: Consortium IEA management response › cgiarorg › 2015 › 11 › CB21-03_IEA... · Synthesis of all 15 CRP evaluations: April 2016 2 Final Meeting Summary, Fund Council November

IEA CRP Evaluations – Consortium Management Responses

CGIAR Consortium Board CB21-03 Twenty-First Meeting, Virtual, 29 July 2015 Page 14 of 14

The evaluation team indicates, and the Consortium agrees, that going forward the governance and management should be arranged as agreed in response to the IEA review of CRP governance and management, and as laid out in the Guidance for the CRP 2nd Call pre-proposals. The evaluation team did not make any recommendations directly addressing the Consortium. The Consortium accepts and supports all recommendations of the evaluation team and agrees that they should be taken into account in the next generation of CRP proposals. The new portfolio of second generation CRPs does not include a CRP on Aquatic Agricultural Systems per se, but it does include a Fish Agri-Food System CRP, which is expected to have a “systems flagship” as a key component. In the view of the Consortium, WorldFish should ensure that all recommendations of the AAS evaluation are taken into account when developing the proposal for the Fish Agri-Food System CRP.