Which were the most important findings from evaluations in 2019/2020, and what are the most important lessons learned? On this stroll, you will gain insight into the wide variety of Financial Cooperation (FC) projects evaluated and the unique features of individual projects. You can find detailed information on the evaluations on the world map.
Making up 36 of the 171 projects in total (21%), the financial sector was subject to the most ex post evaluations in 2019–2020. Ten of these 36 financial projects addressed micro-, small- and medium-sized enterprises (MSMEs) as their target group, while a further eight projects were designed to increase energy efficiency in businesses. The comparatively large number of evaluations is closely linked to the evaluation unit's pooling of evluations of multi-phase projects; as soon as one project phase is included in the random sample from the FC portfolio, preceeding phases are as well evaluated. This approach allows for a more comprehensive assessment of the project’s effectiveness. For example, six phases of the regional energy efficiency fund in south-east Europe and five phases of the MENA regional fund for MSME financing were evaluated within one ex post evaluation report (EPE).
When compared to other sectors, projects from the financial sector perform particularly well. On average, financial projects received an overall rating of 2.3, while the average rating across all sectors was 2.8.
There are a number of reasons why the financial sector scored so well. On the one hand, major hurdles often have to be surmounted to implement these projects. For instance, stable regulatory conditions are needed to establish and disburse a financial product. This is one of the reasons why there are relatively few financial projects in Sub-Saharan Africa included in the overall random sample, while more projects are implemented in more politically stable regions, such as central and south-east Europe, and, as a result, more projects from these regions are evaluated. On the other hand, when projects are pooled, evaluations often empirically detect an improvement to the rating: projects with a very good or good rating in the initial phases were also able to achieve at least the same rating in later phases.
With the increasing importance of policy-based lending (PBL) or reform financing in the FC portfolio, the relevance of evaluation findings from PBL projects is growing. Four such projects were evaluated in 2019 and 2020. In two of these projects, several phases were analysed together (Colombia and Peru). The central findings are that project ownership is the most important factor for success. Furthermore, the evaluations showed that policy dialogues – in which specific measures and aspects of reform financing are clarified and agreed upon – are extremely important but also take up a great deal of time and other resources. Sufficient capacities, including on-site presence, are therefore essential components for a credible, ongoing and trust-based dialogue with partners. The example of a residential water management project in Peru shows that the likelihood of achieving a project’s goals increases if they are developed in close liaison with the government. Instead of defining an ultimate objective, individual stages in a reform process are agreed upon, leading to funds being disbursed on a pro rata basis (trigger). The evaluation results reveal that, while the current flexibility in defining triggers should be maintained, criteria need to be formulated more precisely. In Peru, for example, due to their rather vague definition, the agreed targets tended to be the basis of an ongoing political dialogue as opposed to an instrument for releasing disbursements following a trigger event.
The reform programme to promote the peace process in Colombia is also one of the largest projects evaluated (in terms of funding volume). An analysis of the separate projects evaluated in 2019/2020 by funding volume shows that projects with a larger budget tended to be more successful in the overall rating than projects with a smaller volume.
A high number of global conflicts results in more projects being implemented in conflict countries. This trend is also increasingly reflected in the evaluations. Almost ten years ago (2011/2012 period), 38 FC projects in fragile contexts were evaluated, a figure which has now almost doubled to 68. These projects are often executed in several phases.
The education sector is taking on a particularly important role within the portfolio of fragile countries. One example of this is the multi-phase EQUIP II education project in Afghanistan. This project has been extended three times and aims to improve primary education and contributes to the alleviation of poverty through the increased utilisation of educational opportunities. Not least due to the fragile circumstances in Afghanistan, it became evident that several aspects of this project required adjustment. For example, in addition to measures to increase capacities in schools, contributions to improving the quality of education are also required. In the future, improving the adjustment of measures to changing local security situations is recommended. Apositive factor in this project was the involvement of communities in the administration of schools. The school committees set up by EQUIP II looked after the safety of school children and played an important role in the reopening of schools that had been closed due to the conflict. The importance of strong local partners for project implementation, particularly in fragile contexts was also demonstrated by the Social Fund for Development in Yemen. In this case, the strong partner was one of the factors that led to the project’s regular extension, thereby helping to improve living conditions for young people and children in Yemen.
Against the background of the coronavirus pandemic, findings from the healthcare sector are of particular interest. Programmes for fighting infectious diseases may provide valuable insights into dealing with COVID-19, for example. The assessment of past evaluations revealed that consideration of cross-border infection dynamics plays an important role. Due to the high mobility of people, including their movement across borders, regional approaches are needed to supplement national ones. This was demonstrated in a programme to tackle tuberculosis in the Caucasus area, a region where people have a high level of mobility. Due to the heavy focus on purely national aspects, the approach was not sustainable, which is why the project was rated as ”not successful” on the whole. Furthermore, too few qualified staff were willing to expose themselves to the risk of getting infected – an important finding for future projects of this type. In contrast, multi-national cooperation was an integral feature of a successful multi-phase HIV prevention project in the CEMAC countries (Central Africa) and helped to increase the project’s efficiency with simultaneous positive impacts on the population’s reproductive health.
Involving local populations in the declaration of ecological conservation areas increases their acceptance and is an important factor in the success of projects in the environmental sector. Thanks to its participative approaches, a project to promote sustainable agriculture in Burkina Faso was able to achieve a solid developmental impact (impact rating of 2). As well as involving the project’s beneficiaries in the implementation, management and maintenance of the measures, the process also incorporated a research institute, government agencies, and private stakeholders, such as banks. A 2020 cross-sectional evaluation on conservation areas also highlighted the effectiveness of projects that combine measures for improving socio-economic conditions with nature conservation measures. The combination of income-generating measures and training programmes to promote behavioural change was identified as a further factor for success.
Satellite data has proven to be a particularly effective element for enhancing evaluations in the agriculture and environmental sectors. High-resolution satellite images of relevant variables, such as total area of conserved forests, can help identify deforestation or other developments at an early stage. The use of publicly available satellite data on forest coverage, for example, enabled the analysis of forest loss and growth as part of a forestry project in Vietnam. Satellite images revealed that, although reforestation measures were successful in restoring forests in some project sites, other parts of the intervention areas suffered from high deforestation activities, especially after project end. Consequently, the project was rated as having achieved “only limited success”. During the evaluation of another project in Laos, it was possible to track the movement of so-called deforestation hotspots, which are posing an ever-increasing risk to the promoted conservation areas. For example, a new method for measuring forest fragmentation revealed the construction of a major road, an issue that had not been mentioned during the visit to the area. At the same time, however, it was also possible to trace the positive impacts that the community patrol models are having on the maintenance of conservation areas. Overall, projects from the agriculture and environmental sectors have achieved a positive impact – the sector takes the top spot when it comes to average rating for the DAC criterion of impact.
In addition to the most frequently awarded overall ratings of 2, 3 and 4, on rare occasions particularly successful projects are awarded a rating of 1, while projects that are overwhelmingly disappointing can be rated as 5. In the 2019–2020 evaluation period, the only sector to contain projects receiving the entire range of possible overall ratings was the energy sector. Two projects in Cambodia related to rural electrification were able to exceed expectations. The construction of a new power line, a substation and a power distribution system led, among others, to lower electricity prices, meaning that the projects significantly contributed to improving living conditions.
In the water sector, a rating of 5 was awarded to three projects. The “hazardous waste disposal sites” project in Tunisia confirmed an important finding: involving the local population – for instance, when selecting a location – also plays a decisive role in a project’s success in this sector, too.
Over the past two-year period, the success of projects in the transport and storage sector was average, but once again emphasised the relevance of the DAC criterion of sustainability: despite some remarkable effects on traffic volume achieved by new infrastructure built to rectify flood damage in Cambodia, the project’s sustainability is limited due to a lack of routine maintenance measures. And the importance of sustainability as a criterion is still growing. Securing maintenance is particularily important in projects that promote resilience to climate change among local populations.
This sector contains projects without a specific overarching focus, including decentralisation projects, residential building projects and reintegration and recovery projects. One example of this type of project is the “Reintegration and Recovery Programme” (RPP) in Liberia, which was implemented during the Ebola epidemic. In this project, the possibility to deploy funds quickly enabled not only the implementation of the originally planned measures, but also construction of healthcare centers (overall rating: 3). A very positive, sustainable example in this sector is the construction of a very different type of infrastructure: Social housing projects in Honduras led to the establishment of a charitable foundation that constructs homes in rural areas without any further FC involvement (overall rating: 2).
This reporting period moreover mirrors FC’s decreasing involvement in decentralisation projects – only the first two phases of the FADeC project in Benin were evaluated in this area with an overall rating of 3. The project applied an innovative, incentive-based approach for transferring disbursements to municipalities.
As in the previous reporting period, Sub-Saharan Africa dominates the evaluation portfolio in terms of the number of projects (56 projects). In general, the regional distribution of FC activities also exhibits continuity over time. The only regions to have more projects evaluated in the 2019–2020 period were North Africa and the Middle East (32 vs. 15 projects in the previous period). On average, the overall rating awarded to projects in Sub-Saharan Africa is slightly lower than the overall rating across all regions. The hypothesis proposed in the 15th Evaluation Report, namely that the increasing number of projects in North Africa and the Middle East could have a negative impact on the overall result, has yet to be proven. As in previous reporting periods, projects in Asia/Oceania receive the best ratings. Over 90% of the projects here are rated as “successful”.
