“Independence is crucial for credibility”

OPEC Fund Quarterly: When and why did IsDB start to assess, monitor, report and evaluate the development results of its projects? 

Farhan Rashid: IsDB started training its staff on the logical framework in 2004 as a part of the project preparation. Then we started piloting its use in 2006 and strengthened that in subsequent years. The adoption of the results framework was institutionalized in 2010 within the bank’s systems and structure. 

This paved the way for a systematic way of introducing other project monitoring and assessment tools during implementation and after completion. This included a Project Implementation Assessment and Support Report (PIASR), a Project Completion Report (PCR), and a Country Portfolio Performance Report (CPPR). 

• PIASR – a tool for assessing the progress and performance of projects during implementation. It provides a formal framework for effectively supervising projects including offering the bank’s implementation support to the beneficiary with the ultimate objective of realizing development effectiveness and project results. 
• PCR – a report designed to assess the project’s performance at the time of project completion. This provides an assessment of the objectives against the envisaged plans and results at the time of project preparation. 
• CPPR – a tool that assesses countries’ overall performance and highlights the systemic challenges faced by a country towards achieving development results. In addition, IsDB has also started to publish an annual development effectiveness report, summarizing the project results achieved in a year. 

OFQ: Can you summarize what you do, how you do it, how many people are engaged in the process, and how the organizational structure and governance is set up? 

Farhan Rashid: I am responsible for reporting the bank’s corporate performance and results based on an integrated Results-Based Management Framework (RBMF). We have a team of six professionals. This division is placed within our Strategy, Budget and Corporate Performance Department. 

OFQ: What tools do you use? 

Farhan Rashid: We use our RBMF, which gathers information and data from various sources including the logical frameworks from projects, grants and investments. It has four levels: The first is development progress in our member countries. The second is developmental results from IsDB’s operations. The third covers the operations’ efficiency. The fourth and last level informs on organizational performance. 

OFQ: How do you feed lessons learned back into operations to ensure continuous learning for impact? 

Ahmed Ag Aboubacrine: There are many platforms that our Independent Evaluation Department (IEvD) is using to inform the operations teams and member countries during the design and implementation of new operations, programs, policies and strategies. For example, LEARN (an Intranet-based database for all evaluation knowledge products for the internal audience) and STEER (Systematic Tracking of the Execution of the Evaluation Recommendations) are platforms that we use to present evaluation reports with recommendations and requests for the operations team to develop management action plans to address our findings. 

We also organize evaluation dissemination and learning events such as the annual IsDB Group Evaluation Symposium. These lessons are organized and published in a searchable online database accessible to all users. 

OFQ: How can evaluation functions engage with management to facilitate the use of learning in programmatic decisions, e.g. increased use of evidence in intervention design, results frameworks or adaptive models? 

Ahmed Ag Aboubacrine: First, evaluation independence does not mean isolation. At IsDB, this engagement is an ongoing process that includes constant interaction with the operations teams and management. We engage with them in the planning of the macro evaluation process to develop an evaluation product that is relevant to the needs of management and can inform current and future interventions. Special launch events and knowledge-sharing sessions are organized at various steps during the evaluation cycle. Also, directly after each evaluation, especially our midterm assessment, our team engages directly with the operations team on the key findings, which will facilitate the use of recommendations and follow-up actions. 

OFQ: How important is the independent evaluation process for your organization and how is this independence institutionally secured? 

Ahmed Ag Aboubacrine: Independence is crucial for the objectivity and credibility of the bank’s development results measurement. It is envisioned to enhance performance, boost quality and foster transparency, accountability and learning. This does not mean that the independent evaluation is done in isolation. It should be participatory in nature and comprehensively engage all stakeholders. The independence of the independent evaluation function is secured by ring-fencing it from management control and interference. 

The department reports directly to the bank’s Board of Executive Directors, which also approves the department’s annual work program and budget. The selection and termination of the head is governed by independent policy and implemented in consultation with the Board. 

OFQ: What formal or informal mechanisms are in place to address disagreements on evaluation findings? 

Ahmed Ag Aboubacrine: We follow participatory evaluation approaches to handle disagreements on evaluation findings. Our evaluators engage the operations team from the beginning to understand the context. This helps the evaluators to understand the context better and therefore come to valid conclusions and make useful recommendations. It also increases the level of ownership of the stakeholders. In addition, as part of the validation process, the draft evaluation report is submitted to the bank’s relevant department(s) for comments. This offers management the opportunity to raise concerns and objections, if any. The feedback received together with the evaluator’s response is presented in a final evaluation report. 

OFQ: How do you make sure reporting and monitoring lead to meaningful messages and lessons? 

Intizar Hussain: The bank places great importance on its Monitoring, Evaluation, and Learning (MEL) process which plays a crucial role in project and program management. The process begins with selecting specific, measurable, achievable, relevant, and time-bound (SMART) indicators, using ongoing monitoring to track progress and identify issues. Rigorous data collection methods gather quantitative and qualitative data, which is analysed in the context of external factors. 

Lessons learned inform recommendations and action plans to improve project or program effectiveness. The lessons learned are also embedded in the design of new projects. Transparent communication of MEL outcomes builds trust and continuous feedback refines project results. This systematic MEL process generates valuable insights and lessons that support enhancing the success of active projects and designing future projects and programs while promoting transparency and accountability. 

OFQ: How do you see the impact of artificial intelligence on MEL? Are you already working on pilots? What is your vision in this field? 

Ahmed Ag Aboubacrine: AI is a tool that can add a lot to the accuracy and efficiency of the implementation of evaluation. IsDB is exploring the use of AI in evaluation knowledge management and dissemination. Another area of interest is exploring the use of AI tools to synthesize evaluation findings. However, these efforts are in the initial stages of trial and conception. 

OFQ: What have you learned from recent emergencies such as the COVID-19 pandemic or the food security crisis? To what extent do MEL inputs have traction and inform new strategies and/or operational decisions? 

Intizar Hussain: The bank has responded to these challenges with designated initiatives that introduced fast-track processes for faster approvals and disbursements. Our lessons learned have emphasized greater flexibility in approval and implementation procedures, the setting of realistic targets, strengthening ownership of response initiatives through stakeholder engagement, strengthening staff capacity for designing and implementing emergency response initiatives and institutionalizing emergency response guidelines and processes for increased agility and responsiveness in future crises. To strengthen the emergency response capacity, IsDB has established specialized units and is developing tools to effectively implement emergency and fragility-related strategies and approaches. 

OFQ: What are lessons learned at IsDB that could benefit other development finance institutions? 

Ahmed Ag Aboubacrine: First, evaluation capacity development is critical to ensure the development effectiveness of international financiers such as the Arab Coordination Group members. As their evaluation capacity varies, collaboration is crucial to strengthen their capacity. We conducted joint evaluations of co-financed projects with the Saudi Fund for Development (SFD) in Togo and Bahrain and the Arab Bank for Economic Development in Africa (BADEA) in Mozambique, Togo and Sierra Leone to promote knowledge sharing and capacity enhancement. Such practices enable harmonization, synergy and learning while lessening the burden on client countries and executing agencies.