Final Evaluation of the Irish Aid Programme Grant in Niger (2017 – 2022) At Concern Worldwide

1. Background

IAPG II covers 16 programmes in 17 Concern countries, an Active Citizenship (Development Education), and a Public Engagement programme. The Active Citizenship and Public Engagement components are implemented in Ireland and engage the Irish public on global issues. Concern implements the IAPG II programme in the poorest and most vulnerable contexts, identified through a Poor Vulnerable Index (PVI). Countries of operation experience conflict, food insecurity, inequality, a lack of infrastructural and institutional capacity, climate change and cyclical disasters. The response is based on needs and contextual analyses, to define programme options and targeting processes to identify and work with people living in extreme poverty. The approach focusses on providing an enabling and facilitating role, providing technical support, partnering with government, bringing proven interventions to scale and delivering them to the most remote communities and households. The programme builds household and community resilience by identifying risk and reducing the impact of shocks and peoples vulnerability to these by exploring solutions through analysis and innovation.. IAPG II is a performance-based partnership arrangement funded by Irish Aid and provides flexible and predictable funding to implement ambitious interventions aimed at delivering results and making holistic changes for poor and marginalised people. Each programme has a Theory of Change and Results Framework through which the achievements of the programmes are tracked from 2017 to 2022.

a) Overview of the Concern Niger IAPG II

Concern is implementing the IAPG II programme in Tahoua Department in Niger. The programme started in January 2017 and ends in December 2022. A costed extension of one year was approved in 2021 by Irish Aid as a result of the COVID-19 pandemic. Concern Niger will use the extension to consolidate and strengthen existing dynamics for sustainable change and to implement an exit strategy.

The programme, entitled ‘*Wadata Al’umma*’: Enhancing the Wellbeing of the Extreme Poor in Tahoua, aims to address the interconnected nature of the underlying causes of chronic poverty and vulnerability in Niger and focuses focuses on:

· Reducing biases and inequalities faced by the extreme poor and vulnerable, particularly women and girls;

· Improving access to quality health, nutrition, education and WASH services; and

· Enhancing livelihood systems, natural resource management and environmental protection, all of which are necessary in building the resilience of the most vulnerable to not only combat longer-term poverty but also to cope with the cyclical annual hunger gap season.

The overall aim is that the most vulnerable households and individuals in Tahoua Department enjoy sustainable improvements in health and nutrition, strengthened and diversified livelihoods, greater access to quality education and improved social capital. Programme activities focus on the household and village level. Activities at the household level ensure that households and individuals that make up that community are able to cope with shocks and stresses. Support is provided to communities to ensure they have the capacity to identify and address inequalities or heightened risks and vulnerabilities faced by different subgroups, households or individuals within their communities. The expected outcomes aim to address the multidimensional nature of extreme poverty and tackle the specific barriers that confront the extreme poor in Tahoua Region. The programme is based on a context analysis conducted in 2016, and is implemented in 25 villages in Bambeye and Tebaram Communes of Tahoua Department. The target groups comprises 38,975 people, of which 12,800 are extremely poor people in 2,000 households, (average household size of 6.4). However, since November 2020, the increased insecurity in Tebaram Commune is affecting implementation of activities in six villages of intervention (out of 10 targeted villages in the Commune).

OUTCOMES

Assets

Inequality

Risks & Vulnerabilities

1. 2000 extremely poor and vulnerable households in 25 communities have increased food security and improved livelihoods
2. Learning outcomes for the most disadvantaged students, especially girls, are improved
3. Household wellbeing is enhanced through improved communication, respect, and mutual support between couples, family, and the wider community
4. Community and government structures in 25 villages and 2 communes have the capacity to elaborate and implement development, mitigation and response plans.
5. Vulnerable communities have improved access to clean drinking water, water for livelihoods and sanitation
6. Households practice key behaviours necessary for improved maternal and child nutrition and health.

Programme activities were progressively introduced over the five-year timeframe. Core livelihoods activities centred on the establishment of demonstration plots for crop production, livestock and agroforestry extension service delivery; natural resource management and establishment of a wide range of committees to deliver programme activities common to all villages across two communes. Selected additional production activities were implemented across a smaller number of villages. In addition to the livelihood component, primary education; Water, Sanitation and Hygiene (WASH); and nutrition components were introduced. The DG ECHO funded Enhanced Response to Nutrition Emergencies which commenced in mid-2020 supported the development, mitigation and response outcome through a Disaster Risk Reduction (DRR) component, which has a comprehensive Early Warning Early Action element (Outcome 4). The map below shows the implementation area and distribution of livelihoods activities that complemented the core activities in livelihoods, primary education, WASH and nutrition with DRR and Early Warning Systems.

2. Purpose of the evaluation

Concern is seeking the services of an evaluation consultant to conduct an external evaluation of the IAPG II programme in Niger. The purpose of the evaluation is to assess the overall impact of the programme, the extent to which it achieved its stated objectives, the nature of the current level of linkages between the programme and Irish Aid’s stated priorities, and provide recommendations for future programming. The evaluation documents will constitute an independent assessment of the programmes’ performance, as well as, capturing programme learning; and proposing recommended options for the next programme cycle. The primary audiences will be Concern management in Niger and at Regional level; selected staff at head office in Dublin; and Irish Aid. Concern will share findings within the Francophone region. Findings and recommendations of the evaluation will inform the design of the next programme of work in early 2022.

A number of grants were implemented in the target area during the programme period which contributed to and were linked with the IAPG II programme. These provided co-funding and funds for activities and complementary interventions, which contributed to achievement of the outputs and outcomes. The evaluation will explore the attribution of overall achievement to Irish Aid funding and what proportion of the overall cost of the programme did this represent.

3. Essential and Desirable Experience/Qualifications

a) Essential:

i. At least 5 years’ experience in evaluation of overseas development and humanitarian programmes.

ii. Experience of evaluating against OECD-DAC criteria.

iii. Experience of working in the Sahel and of conducting evaluations in complex and insecure environments.

iv. Fluency in written and spoken French and English.

v. Technical expertise in food security and livelihoods and disaster risk reduction; general expertise in health, nutrition, education, and emergency response.

1. Ability to analyse budgets, results frameworks, Monitoring and Evaluation reports, and experience of results chain analysis (key issues of economy, efficiency, effectiveness, impact, design and relevance).
2. Commitment to participatory approaches to evaluation, with strong analytical and facilitation skills, and proven ability to produce high quality reports.
3. Willingness to participate in vetting process, which includes criminal background checking and/or satisfactory references.
4. Provides the service at a reasonable cost.

b) Desirable:

1. Previous experience of working with Concern Worldwide or Irish Aid a particular advantage.
2. Ability to manage the consultancy independently.
3. Previous experience of working or conducting evaluations in Niger.
4. Has an established reputation.

4. Objectives and Specific tasks to be undertaken

The overall objective of this consultancy is to provide an independent assessment of Irish Aid’s funding to Concern Worldwide in Niger through the Irish Aid Programme Grant II.

The specific objectives of this evaluation are:

· To assess the programme against relevant OECD-DAC criteria and other commitments, including core Concern policies and approaches;

· To provide evidence-based recommendations on a core set of activities that will be refined and defined as operating procedures for a food systems for nutrition security programme being developed for the next cycle of funding, applying lessons learned and review of the concept note developed for the new programme[3]; and

· To review and assess the exit strategy for IAPG II for 2022 to provide recommendations on appropriate closure of activities and key planning activities that can be implemented in advance of the new programme cycle.

Evaluation questions will be aligned to the OECD-DAC criteria (relevance, efficiency, effectiveness, impact and sustainability). Concern will provide the selected evaluator(s) with relevant information, documentation, contacts and guidance during a briefing meeting in the inception phase of this evaluation. Through their application, the selected evaluator(s) will propose the exact methodology they will use. However, the evaluator will base the methodology on the description in this section. Any proposed methodology is expected to assess the quality of available data, and then apply standard evaluation methods to get additional information required to effectively evaluate the programme and to identify the core reasons for success or failure. Data collection and analysis should adhere to good practice in research methods. The methodology must be participatory and allow for the inclusion of viewpoints of representatives of key stakeholders and actors, including beneficiaries and non-beneficiaries, implementing partners, government stakeholders, and Concern staff.

Evaluation Questions for the Concern IAPG II Programme in Niger

Relevance

· To what extent did IAPG II respond to the needs of participants/beneficiaries in the evolving context? What was the ‘evolving context’? What were participants/beneficiaries needs? Were these needs met?

· Did the Theory of Change (ToC) hold true? What was the intended results chain (causal linkage)? Did actual results follow this intended causal path?

· Were the assumptions valid? What assumptions were made? Did these assumptions hold true? Were other factors important? What were these factors?

· Did Concern Niger adapt appropriately in response to changes in the context and lessons learnt during the implementation of the intervention?

· What was the context? What were the lessons learnt? How did Concern Niger adapt to changes in the context? How did IAPG II adapt to lessons learnt?

Effectiveness

· Is there evidence that anticipated results are being achieved at the current stage of IAPG II? What is changing as a result of IAPG II?

· To what extent can the emerging results be attributed to IAPG II?

Sustainability

· To what extent did Concern Niger build institutional capacity of local partners? Which local partners did Concern Niger (IAPG II) work with? Did these partners’ capacity change? If yes, how? Is this capacity sustainable without further external funding?

· Did IAPG II enhance local ownership and capacity to influence policy? How has IAPG II developed local ownership? Give examples. How has IAPG II built capacity to influence policy? Give examples.

· Where relevant, did IAPG II contribute to localisation of humanitarian aid?

· What has worked well in creating long-term change/not worked well and why (e.g., systems, policy, advocacy)? What has worked well in creating long-term change/not worked well and why (e.g., systems, policy, advocacy)?

Efficiency

· Did the intervention use the available resources in the most efficient manner to achieve its outputs? Did the intervention have sufficient and appropriate resources?

· Did the intervention pay competitive prices for its inputs?

· Were there any alternatives for achieving the same results with fewer inputs?

Impact

· Did the intervention achieve the intended impacts? Is there evidence of change at impact level (positive or negative)? If so, what contributed to this change? If not, why not?

· What are the wider effects of IAPG II on: individuals; groups (gender, age, disability); communities; institutions; or conflict dynamics? Were they positive or negative?

· Were there any unintended results? Were they positive or negative?

This evaluation will have two main areas of focus which centre on the structure and logic of the programme, and performance of the programme. For each of these areas, the following guidance is provided:

1. A review of programme documents, including all briefing papers and research undertaken. This includes an analysis of the budget and financial reports for the programme (including co-funding applied during the programme timeframe from a range of donors). This review will:

a. Identify and evaluate core programme activities implemented in all target villages over the programme timeframe.

b. Assess additional activities introduced through in a sub-set of communities that complemented the core set of activities; providing quantitative evidence of the extent to which outputs and outcomes were improved.in these selected communities compared with those that received the core set only.

c. Assess the extent to which specific activities and approaches are compatible as Concern works to align with a ‘triple nexus’ approach and provide recommendations on adopting this approach in future programming.

d. Provide recommendations on activities that produced results and achieved programme outcomes that would contribute to a food systems for nutrition security programme; including how particularly sensitive issues related to inequality, population growth and conflict can be treated in a sensitive, however urgent manner.

1. A review of performance against the OECD-DAC criteria on whether the programme achieved its stated objectives; the extent to which the programme applied and adhered to key Concern Worldwide policies, and the relevant Government of Ireland policy for international development.
2. The evaluation should specifically assess:

a. The application of a results-based management approach, including how systemic barriers and programme blockages are being addressed through evidence-based policy influencing and advocacy;

b. Achievements in efforts to scale-up resilience, including through replication of activities by non-targeted households or communities, sustainable systems strengthening; and disaster risk reduction, including how responses to shocks were dealt with during the programme timeframe;

c. The economy, efficiency and effectiveness of the results chain to identify the level of performance of activities against cost. Specifically, this refers to an opinion on the investment in each output compared with returns achieved in terms of the benefits accruing to the household and community levels. An additional opinion, with evidence (stated as being strong, medium or weak) should be provided on the performance of programme in terms of indirect benefits to other target groups. The issue of attribution of the results chain to the programme outcome should be analysed in terms of programme activities, however also in relation to other interventions conducted by organisations operating in Tahoua Department, specifically Tebaram and Bambeye Communes;

d. How the programme explicitly addressed cross-cutting themes of equality, protection, HIV and AIDS, partnership, Disaster Risk Reduction (DRR) and environmental impact, making recommendations for improvements in all areas and suggesting how the focus on environmental impact can be modified to address the environment and climate adaptation. The extent to which COVID-19 was addressed systematically during the latter years of the programme and how the response aligned with ongoing programme activities should be addressed as an additional cross-cutting issue.

5. Outputs

Concern Niger requires a report that is based on providing concise information, is evidence-based and provides recommendations based on the description provided in the section ‘Objectives and Specific Tasks to be Undertaken’. In addition, a detailed itinerary should be provided to list the programme participants met, their role and involvement; as well as key events undertaken during the evaluation process. It should be noted that a significant aspect of this consultancy is the review and analysis of secondary information pertaining to the programme. However, a participatory process is also envisaged to gather perceptions of programme participants as outlined. There is some flexibility in the structure of the report however it should contain the following sections as a minimum: Table of Contents, Executive Summary, Main body of the report divided according to the information provided above, Recommendations and Conclusions. The report should be no more than 30 pages in length and produced in the English language. Final payment is dependent on the submission of a good quality, well-written final report, following feedback on an initial draft.

6. Lines of Communication and Reporting

The evaluator will report to the Concern Niger Programme Director while in Niger. They will liaise with the Country Director, members of the Country Management Team and finance and programme staff. They will submit a first draft of the report to the Programme Director, who will coordinate an internal review. The evaluator will then submit the final report.

7. Timeframe

The selected evaluator will be expected to commence the this piece of work in May 2022 and submit the final report in July 2022 to allow for some recommendations to be planned and implemented in the final months of the current IAPG II funding cycle. The number of days should not exceed forty-five (45) days. As there is a significant amount of secondary research involved in this evaluation, the allocation should clearly outline the remote and in-country planning.