Development effectiveness: Because every dollar counts

In a world where resources are limited, development finance is no different. Despite the growing amount of multilateral development resources committed to developing countries every year, the latest available data showed that these commitments represented a mere 0.7 percent of the developing world’s GDP, which is considered insufficient to meet the needs of developing countries. 

Given the scarcity of resources, ensuring that these are well spent is all the more important. Particularly since the early 2000s, development finance providers and recipients have paid increasing attention to whether development finance is effectively tackling development challenges where support is most needed around the world. The need to better understand whether, where and how development impacts are in fact materializing, and why or why not, led to the formulation of the Paris Declaration on Aid Effectiveness in 2005. 

This agreement reflects an unprecedented consensus among donor and aid recipient countries about what needs to be done to substantially improve the impact of development cooperation. 

What is development effectiveness and why is it needed? 

The Development Assistance Committee of the Organisation for Economic Cooperation and Development (OECD-DAC) defines development effectiveness as “the extent to which a given development intervention’s objectives were achieved, or are expected to be achieved, taking into account their relative importance”. Measuring the extent to which the development objectives were achieved requires the measurement of whether the expected development outcomes (and impacts, when possible) of a transaction or project that focuses on addressing a certain development challenge have in fact materialized. In turn, assessing the achievement of the results and therefore objective is only possible if the expectations had been clearly defined before undertaking a development intervention. 

Measuring development effectiveness is meant to provide clear and objective data that allows finance providers and recipients to take informed decisions on where to best direct the limited resources, and how – in an effort to target the most effective interventions, and to design all interventions in a way that they can reach the largest impact possible. Clear data and information about the development effectiveness of their activities also allows development finance providers to be accountable to their funders and other stakeholders. Knowing, and being able to show, the effectiveness in the use of their funds can motivate funders – such as national governments who are accountable to their taxpayers – to continue supporting development causes and therefore to tackle the many remaining challenges. Accountability and transparency around development effectiveness have therefore become increasingly important. 

Guiding principles for development effectiveness 

The assessment of whether or not a development intervention has been effective is usually performed after project implementation, as development benefits usually take some time to materialize. This evaluation typically considers a number of highly interrelated aspects. Most development finance institutions (DFIs) apply several, or all, of the evaluation criteria as defined by the OECD-DAC Network on Development Evaluation (see box, right) when measuring whether or not a development project has been successful. 

Whereas these criteria can be applied to all development interventions, the use of donor money for private sector interventions also requires an assessment of whether and why this assistance is needed in the first place – as opposed to private funds – to avoid distorting potentially functioning markets to the detriment of final beneficiaries. Providers of development finance to the private sector therefore typically also assess to what extent their financing offers financial or non-financial value – also called additionality – which is needed for development results to materialize, but which, at the same time, is not available from purely commercial financiers.  

Relevance and coherence 

Development projects that are considered highly relevant by both the developing country and their development partner are more likely to be successful, and their benefits sustained over time, because the importance of their success is clear to, and accepted by, all parties involved in project planning and implementation. 

Relevance ensures that all parties pull indeed in the same direction and hard enough for there to be movement. Assessing the extent to which development projects respond to actual development needs at the country and global level, as well as align to country and financier priorities, is therefore important. A lack of relevance can imply a higher risk of project failure or unsustainability. 

The same is true for coherence – if an intervention is inconsistent with, or overlaps with other interventions by the government or other development partners, scarce development finance resources may be wasted on duplicated, ineffective or unsustainable efforts. 

Development needs and country ownership 

Ensuring the success of development intervention usually requires hard work from all involved. The chances that development interventions are relevant to, and therefore fully supported by, developing countries are higher when the recipient countries take ownership over setting their priorities and strategies for tackling the needs that they, themselves, view as most pressing. 

Country ownership also increases the likelihood that development interventions are coherent, i.e. effectively complement (and don’t overlap with, or counteract) other interventions and actions by the government or other development partners. In short, developing countries’ priorities should drive the need for development aid and not the other way around. Ownership also provides incentives for countries’ institutions and private players to build the necessary capacity to be able to receive and use development aid effectively and efficiently. 

Recognizing the importance of country ownership for development effectiveness, in 2011 the Busan Partnership for Effective Development Co-operation, convened by the OECD-DAC, established four principles: (i) countries should define their own development model; (ii) sustainable impact should be in the center of any policy-making effort; (iii) development cooperation must be transparent and accountable to all citizens, and (iv) development depends on the effective involvement of all actors. 

All forms of development aid, which includes both public and private development finance and technical cooperation, should be bound by these four principles and, together, they should address the most pressing development challenges. 

Many development challenges are common among developing countries. The Sustainable Development Goals (SDGs), defined by the United Nations in 2015 (see box, below left) provide a high-level list of some of the most pressing development needs that affect developing countries and vulnerable populations around the globe. 

Alignment to donor/ financier priorities 

The scarcity of development finance resources implies that development finance providers cannot provide support to any and all development projects. They therefore have to prioritize, with such priorities usually spelled out in their strategies and business plans. Factors influencing these decisions are: 

1. Effectiveness 

This is achieved when an intervention’s development objectives are achieved. Since the development is fundamentally about improving people’s lives, development objectives should express what specific improvements the project is expected to bring about for the targeted beneficiaries. To understand whether a project is effective, it is therefore not enough to know whether a financed activity or asset has been delivered. One must also understand whether the provision of these activities or assets has resulted in measurable improvements for the affected populations. 

Quantitative indicators are therefore established not just for the products, or outputs, of each project, but also for the improvements from the perspective of the beneficiaries. A project financing the paving of formerly dirt roads, for example, would need to not only provide evidence on the kilometers of road paved by the project (the project output), but also measure improvements from the viewpoints of the road’s beneficiaries and other stakeholders, such as the travel time savings of road users, the increased number of days the road is passable, the reduced number of accidents if the road is made safer, reduced road maintenance costs, or other benefits depending on the specifics of each project. 

Some improvements can be measured relatively directly (such as the lower incidence of a specific illness because of a vaccination campaign), whereas some have to be measured by proxy (such as the avoidance of greenhouse gases, which is expected to lead to improved lives around the globe) due to issues in methodology or of attribution. 

To be able to assess whether a project is successful or effective, clear expectations for the future values of the indicators have to be set when designing the project – and such values should of course show a clear improvement over the current (or baseline) value for the project to be worthwhile undertaking. 

During and after project implementation, the indicators need to be measured to assess the extent to which their target values have been achieved. This serves to not only assess effectiveness but also, in case of their non- or insufficient achievement, to ask questions and learn about why the expected results have not materialized. Feeding these lessons back into the design of operations, to avoid or mitigate known pitfalls, allows development institutions to continually improve their effectiveness. 

2. Efficiency 

The efficient use of development resources requires assessing to what extent development benefits are achieved at a reasonable cost or not. This assessment helps not only inform decisions about where to direct resources for maximum impact, but also provides information needed to avoid wasting resources on too-costly and therefore unsustainable projects. The tool used to assess efficiency is most commonly called “economic analysis” and includes quantitative and/ or qualitative analysis of the costs and benefits accruing to all relevant project stakeholders, such as beneficiaries, the government, private sector actors, the environment, and other affected parties. 

Ideally, this analysis is quantified to the extent possible in the form of a cost-benefit analysis which produces measures (such as the economic rate of return, the cost-benefit ratio and the net present value of the project’s net economic benefits) that can be compared across projects. Too-low values of these measures can flag potential threats to project effectiveness and sustainability, and an analysis that separates these cost-benefit flows by stakeholder groups can furthermore help identify winners and losers, and therefore project risks. 

As some benefits are difficult to credibly quantify, economic analysis can also take the shape of cost-effectiveness analysis (where only costs of different options are compared) or a qualitative discussion of all relevant costs and benefits and their likely magnitude. 

For projects with the private sector an assessment of efficiency also requires the assessment of the purely financial returns, as too-low financial returns can jeopardize private sector interest in, and continued support to the project. On the other hand, very high private sector returns can call into question why development finance is needed, as commercial sources could be expected to fund ventures of such clear profitability. 

3. Impact 

In the development world, impact is understood as those effects that development is ultimately about, but which are often difficult to attribute to single development interventions. 

Take household income for example, which is a measure that is considered to be one of the main determinants of people’s quality of life. A development project that helps farmers in a certain region of a developing country uses more effective farming techniques ultimately aims to have a positive impact on farmers’ household incomes – this is what the project is in large part about. 

However, farmers’ household incomes are not only determined by farming techniques, but also, for example, by weather events affecting the harvest, the prices that farmers can sell their harvest for (which are often influenced by swings in global commodity prices, the market structure, etc.), changes in government subsidies and taxes, the employment status of other household members, and many other factors. 

To assess the impact of a single project on a measure such as household incomes, the effects of the project have to be separated out from all other factors that have a bearing on this variable. 

Many impacts also take a very long time to materialize - such as increased incomes due to a project that improves access to education. Assessing project impacts is therefore difficult (and often methodologically impossible) and usually costly. 

Different impact evaluation methods of varying rigor exist, all of which strive to isolate the effect of a specific intervention, or group of interventions, from other influences on the target variable. While some consider impact evaluations the gold standard for truly assessing development effectiveness, their cost, time requirements and methodological limitations prevent their application to all projects. 

However, there are certain types of projects that are considered a good use of impact evaluation resources. These include pilots of government programs that are based on different design options and rolled out at a large scale. Impact evaluations of the small initial pilots can help governments decide not only which option to scale up, but also provide a sense of the magnitude of the effects that can be expected at a larger scale. Other good candidates for impact evaluations are highly innovative projects, the effects of which have to be well-understood to assess their suitability for replication. 

Even when projects do not undertake an impact evaluation, their assessments should nevertheless provide clarity on the assumed chain of effects which are expected to lead to impacts in those long-term measures that development ultimately intends to improve. 

4. Sustainability 

Projects can be very effective in delivering development results in the short term, but the true test for the good use of scarce development resources is the question of whether the development benefits can be sustained over time. Even effective projects can constitute a waste of resources if they are not sustainable. 

Take for example the aforementioned road project – if the road is paved but then not maintained over time, the gained benefits can be reversed as the road deteriorates. Addressing known or likely risks to sustainability through effective project design is therefore important, to mitigate the risk of misallocating development resources to unsustainable interventions. 

Sustainability also includes the social and environmental impacts of and risks to a project. Adverse environmental and/or social impacts can diminish or even outweigh the overall development benefits of projects and pose risks to the continued stakeholder support that is needed for the continued delivery of development results. 

How DFIs and other impact investors assess development effectiveness 

Viewing development projects through a development effectiveness lens helps project teams design and implement operations that address the recipient’s most pressing development needs in an effective, efficient and sustainable way. Development effectiveness considerations are embedded in decisions by development finance institutions throughout the project cycle, from the design to the evaluation of development projects. Development effectiveness specialists typically support project teams to ensure that all development effectiveness aspects are considered, results set and tracked, and lessons learned extracted and fed back into new operations. 

Once a project is properly designed and its objective and expected results are set, project monitoring should ensure that data on the extent of achievement of targets is collected and documented. This information collection during implementation can also serve as an early warning tool to help bring projects back on track, in cases where expected development results do not materialize, or experience delays. A systematic approach to assessing and tracking development effectiveness also allows for a more informed and efficient evaluation process once a project is completed. 

Several development finance institutions use specific assessment and tracking tools and systems that help standardize and facilitate the process, provide guidance to project teams and help track effectiveness throughout a project’s life. These tools are generally guided by the development effectiveness criteria outlined earlier. A large number of impact investors focused on investments in the private sector have agreed on another set of principles which reflect these same development effectiveness criteria and also include guidance on the processes of including these considerations in investment decisions and monitoring. 

The multitude of development challenges around the world needing to be addressed by limited development finance resources heighten the need to know what works, and why, so that resources can be used in the most effective and efficient way possible. Incorporating development effectiveness considerations in strategic and investment decisions, and employing a systematic approach to gathering and using information on development results achieved allows DFIs to make informed choices for maximum impact. 

OECD-DAC evaluation criteria 

• Relevance: The extent to which the intervention objectives and design respond to beneficiary, global, country, and partner/institution needs, policies, and priorities, and continue to do so if circumstances change. 
• Coherence: The compatibility of the intervention with other interventions in a country, sector or institution. 
• Effectiveness: The extent to which the intervention achieved, or is expected to achieve, its objectives and its results, including any differential results across groups. 
• Efficiency: The extent to which the intervention delivers, or is likely to deliver, results in an economic and timely way. 
• Impact: The extent to which the intervention has generated or is expected to generate significant positive or negative, intended or unintended, higher-level effects. 
• Sustainability: The extent to which the net benefits of the intervention continue, or are likely to continue. 

Operating Principles for Impact Management 

The Impact Principles are a framework for investors for the design and implementation of their impact management systems, ensuring that impact considerations are integrated throughout the investment lifecycle. In 2019, IFC (World Bank Group) launched the Operating Principles for Impact Management, and then in 2022 it was handed over to the Global Impact Investing Network (GIIN) to strengthen the cohesion of impact measurement and management across the industry. By the end of 2022, there were 169 signatories representing 39 countries and US$510 billion in assets under management. Close to 90 percent of its signatories had published a disclosure statement. 

The Operating Principles for Impact Management are: 

1. Define strategic impact objectives, consistent with investment strategy: “The Manager shall define strategic impact objectives for the portfolio or fund to achieve positive and measurable social or environmental effects, which are aligned with the Sustainable Development Goals (SDGs), or other widely accepted goals.”
2. Manage strategic impact on a portfolio basis: “The Manager shall have a process to manage impact achievement on a portfolio basis. The objective of the process is to establish and monitor impact performance for the whole portfolio, while recognizing that impact may vary across individual investments in the portfolio.” 
3. Establish the Manager’s contribution to the achievement of impact: “The Manager shall seek to establish and document a credible narrative on its contribution to the achievement of impact for each investment. Contributions can be made through one or more financial and/or non-financial channels.” 
4. Assess expected impact of each investment: “For each investment the Manager shall assess, in advance and, where possible, quantify the concrete, positive impact potential deriving from the investment. The assessment should use a suitable results measurement framework that aims to answer these fundamental questions: (1) What is the intended impact? (2) Who experiences the intended impact? (3) How significant is the intended impact?” 
5. Assess, address, monitor and manage potential negative impacts: “For each investment the Manager shall seek, as part of a systematic and documented process, to identify and avoid, and if avoidance is not possible, mitigate and manage Environmental, Social and Governance (ESG) risks.” 
6. Monitor the progress of each investment: “The Manager shall use the results framework to monitor progress toward the achievement of positive impacts in comparison to the expected impact for each investment. Progress shall be monitored using a predefined process for sharing performance data with the investee.” 
7. Conduct exits considering the effect on sustained impact: “When conducting an exit, the Manager shall, in good faith and consistent with its fiduciary concerns, consider the effect which the timing, structure, and process of its exit will have on the sustainability of the impact.” 
8. Review, document, and improve decisions and processes based on the achievement of impact and lessons learned: “The Manager shall review and document the impact performance of each investment, compare the expected and actual impact, and other positive and negative impacts, and use these findings to improve operational and strategic investment decisions, as well as management processes.” 
9. Publicly disclose alignment with the Impact Principles and provide independent verification of the alignment: “The Manager shall publicly disclose, on an annual basis, the alignment of its impact management systems with the Impact Principles and, at regular intervals, arrange for independent verification of this alignment.” 

1