End term evaluation for North Central Transitional Aid in Nigeria At Christian Blind Mission

Terms of Reference for Evaluation

1. Evaluation Summary

Project Number and Program/Project Name P3982/North Central Transitional Aid in Nigeria (NoCTRAiN)

Name of Partner Organisation/s

1. CeGHaD – Centre for Gospel Health and Development
2. Evangelical Church of West Africa Hospital (ECWA)
3. Gede Foundation
4. Universal Reformed Christian Church (NKST)

Project start and end dates; phase of project if applicable: 1st October 2019 – 31st March 2023; ongoing

Evaluation Purpose

To establish the extent to which the project has successfully achieved (and potentially not achieved) the projected results in the target project locations in line with the project Log frame in an efficient and cost-effective manner. The evaluation will provide the opportunity to generate a reference document, detailing the achievements and challenges of the project by all partners and CBM (with special attention to a possible follow-up project), overall lessons learnt, and evaluation of value added.

Evaluation Type: Final evaluation

Contracting organisation/contact person: Christoffel-Blindenmission (CBM) International Nigeria Country Office Director: Mr Bright Ekweremadu

Evaluation Team members (if known): External evaluation team

Primary Methodology: Mixed Methods Design, incl. quantitative and qualitative methods*.*

Proposed Evaluation Start and End Dates: 20th March – 20th April 2023**:**

Anticipated Evaluation Report Release Date: 27th April 2023

Recipient of Final Evaluation Report: Christoffel-Blindenmission and partner organisations, BMZ

1. Project Description

CBM is an international development organisation, committed to improving the quality of life of persons with disabilities in the poorest countries of the world. Based on its Christian values and over 100 years of professional expertise, CBM addresses poverty as a cause, and a consequence of disability, and works in partnership to create a society for all. CBM humanitarian activities aim to equally reach and actively involve persons with disabilities, addressing specific needs while promoting and facilitating full inclusion in mainstream services. We work – together with local partners – on projects that are context-specific, from preparedness to immediate life-saving needs such as food, shelter and medical work, and longer-term rehabilitation interventions in fields such as education, water sanitation and hygiene; healthcare, livelihood and reconstruction.

The perennial violent attacks in the conflict between nomadic herders and local farmers in North Central Nigeria has resulted in untold socio-economic hardships and displacements in the region. January 2019 marked a six-year high in the attacks. From September 2017 to June 2018, an estimated 176,000 people in Benue, 100,000 in Nasarawa and 100,000 in Plateau were displaced by the conflict. The majority of internally displaced persons (IDPs) are women and children. Some of the displaced remain with relatives in safer parts of the states, others find shelter in temporary camps for Internally Displaced Persons (IDPs) (often in schools and churches). The Nigerian government can only partly cover the humanitarian needs of the internally displaced and the provisional IDP camps are often heavily overloaded and do not have sufficient clean drinking water and adequate Water, Sanitation and Hygiene (WASH) facilities. The health conditions of the IDPs are extremely poor. The influx of IDPs into various communities across the North-Central states have further put stress on the existing health and WASH infrastructures in the host communities. International aid and development organisations have so far paid only little attention to this conflict, as the current focus remains almost exclusively on those states affected by Boko Haram in the northeast. The Humanitarian Response Plan for UN-OCHA for 2018 does not include any activities/funds for Benue, Plateau, Nasarawa or Taraba States.

The economic burden of the conflict was increasing: the displacement and continuing insecurity have brought agriculture partly to a standstill due to farmers’ inability to access their farmlands. In the states of Benue, Nasarawa and Plateau where the people are predominantly farmers, food production is estimated to have fallen significantly due to the conflict. The conflict generates chronic health, social and economic burdens for host communities and internally displaced persons. These core problems prompted CBM to develop a project aimed at Strengthening the resilience of conflict-affected host communities and internally displaced persons with attention towards disability inclusion in Benue State, Nasarawa State and Plateau State by establishing inclusive health and WASH services and income generating activities.

The project by CBM is in collaboration with four local organizations implementing the project across 13 target Local Government Areas (LGAs) in Benue, Nasarawa and Plateau States with a 100% funding commitment of 2.8 Mio EUR by the German Federal Ministry of Economic Cooperation and Development (BMZ) between October 2019 and March 2023 (42 months). Being financed under the title BMZ Transitional Development Assistance, the project responds to the main areas of food security, reconstruction and (re-)integration of refugees or Internally Displaced Persons in the target LGAs. A comprehensive feasibility study was conducted before the commencement of the project and the targeted number of direct beneficiaries was 279,620 persons and that of indirect beneficiaries is 1,098,350 million persons across the three States.

The project aimed at achieving 3 specific results:

Result 1: The target group (host communities and IDPs) have access to qualitative and inclusive health (Eye, Reproductive and Mental Health) services. (Implementing partners: ECWA, Gede Foundation and NKST)

Result 2: Households in the communities (host communities and IDPs) have access to inclusive WASH services. (Implementing partners: CeGHaD and Gede Foundation)

Result 3: Vulnerable households in the communities (host communities and IDPs) have expanded and diversified their sources of income by forming savings groups- Village Savings and Loans Associations (VSLAs). (Implementing partners: CeGHaD and Gede Foundation)

1. Evaluation Objective and Intended Use

The end-line evaluation will be commissioned by CBM by the end of the project and it is aimed at analyzing the project’s contribution to strengthening community resilience through the provision of inclusive health and WASH services, creation of inclusive and sustainable livelihoods for the reintegrated households in the target areas.

Specific Objectives:

1. To assess the level of achievement of outputs and outcomes, including contributing and hindering factors.
2. To assess the potential longer-term impact in the communities (sustainability measures) with a particular view on persons with disabilities.
3. To generate a reference document outlining lessons learnt and success factors that can be used for an extension or similar disability inclusive projects.
4. Scope of the evaluation

The scope of the evaluation will cover activities implemented by all 4 partners (CeGHaD, Gede Foundation, ECWA and NKST) between October 2019 till March 2023. The evaluation shall be conducted in selected project communities, to give a representative sample. According to their technical focus, the partners’ project-related work is structured as follows:

State / LGA

Benue: Agatu, Guma, Gwer West, Logo, Kwande

**Nasarawa:**Awe, Keana, Obi

Plateau: Riyom, Bassa, Bokkos, Barkin-Ladi, Mangu

Eye Health: ECWA and NKST

Female Sexual and Reproductive Health: ECWA

Mental Health: GEDE

WASH: GEDE and CeGHaD

Livelihood: GEDE and CeGHaD

1. Limitations

Nigeria is in an electioneering period with the current federal government transmitting power to a new government to be elected on February 25th, 2023. Also the state election is scheduled to hold on March 11th, 2023. The outcome of the elections may generate a lot of political tension and this may coincide the timeframe for the activities. In the event of increased insecurity across the nation, there will be need for situational analysis and use of locally sourced enumerators for data collection that understand the socio-political early warning signs and safety measures based on their context.

CBM will carefully evaluate the situation once the evaluation contract is signed and determine whether the evaluation can be carried out as planned. In addition, CBM will provide a briefing on the current security situation and support as much as possible to ensure the safety of the evaluation team and other stakeholders who will be involved in the evaluation process, including project partners and beneficiaries.

1. Target audience and learning

The results of this end-term evaluation shall be used by the following stakeholders:

1. Government, implementing partners and beneficiaries will use the findings from this evaluation

• to understand the impact of the programme on different project stakeholders and beneficiaries, in particular on persons with disabilities
• To learn and support sustainability measures put in place by the programme
• lessons learned will be used to inform future planning and design of inclusive programmes

1. CBM will use the information from this evaluation

• to inform the way in which CBM can work with government and local organizations to establish inclusive and accessible, quality and sustainable services to the communities
• to guide provision of technical support to future inclusive projects
• to improve the overall quality and impact of CBM’s work

The evaluation team shall make an intentional effort towards ensuring that the project stakeholders (Government Agencies, sector working groups, implementing partners, community stakeholders including persons with disabilities through the Organisations of Persons with Disabilities, as well as Internally Displaced persons, beneficiaries/participants and CBM team) gain access to the results and recommendations in an appropriate format.

1. Evaluation Questions
2. RELEVANCE – Has the project been doing the right things?

Was the project designed in ways that respond to the needs and priorities of the various participants[1] (irrespective of gender, age, disability status)? To what extent did the project design reflect the rights of persons with disabilities and other marginalised groups and include feedback from a diverse range of local stakeholders? Were the strategies used during project intervention the most suitable within the political, economic and social context for achieving intended outcomes? How appropriate were the activities to local needs, increasing ownership, accountability and cost-effectiveness?

1. COHERENCE – How well did the project fit in the broader national and international context?

To what extent are the project’s design, delivery and results coherent with international laws and commitments to disability rights, gender equality and rights, including the CRPD[2], CEDAW[3], and the Agenda2030? To what extent does the intervention support national legislation and initiatives that aim to improve equality and human rights and the inclusion of persons with disabilities? Are there policies/norms that have hindered the smooth implementation of the project across Benue, Nasarawa and Plateau States? How has the project contributed towards the achievements of the Country’s strategic plans and policies on health, WASH and Livelihoods?

1. EFFECTIVENESS – Has the project achieved its objectives?

In how far did the project achieve its objectives and expected results in ways that contribute to inclusion and equality across groups, such as persons with and without disabilities or women/girls and men/boys? Were different approaches used to reach the different groups?

Was the theory of change and results framework informed by analysis of inclusion, gender equality, political economy analysis and human rights?

Was the intervention adjusted to address any concerns and maximise effectiveness?

1. EFFICIENCY – How well have resources been planned and used? The extent to which the intervention was able to deliver results in an economic and timely way.

Were different resources allocated in ways that considered inclusion and equality? Was

differential resource allocation appropriate?

1. IMPACT – What difference did the project make? The extent to which the intervention has generated or is expected to generate significant positive or negative, intended or unintended (‘spill-over’), higher-level effects.

Were there equal impacts (Outcomes) for different groups of people? If so, why did these differential impacts occur? How did norms relate to disability or gender and barriers within the wider political, economic, religious, legislative and socio-cultural environment impact outcomes? And has the project contributed to a positive change with regards to those norms?

To what extent have impacts contributed to equal power relations between different groups of people and to changing of social norms and systems? Or hindered those?

1. SUSTAINABILITY – Will the benefits last? The sustainability evaluation aims to measure the extent to which the net benefits of the intervention continue or are likely to continue. This includes an examination of the financial, economic, social, environmental and institutional capacities of the systems needed to sustain net benefits over time.

Did the intervention contribute to greater inclusion and equality within wider legal, political, economic and social systems? Did it result in enduring changes to social norms that are harmful to any specific group of people? Will the achievements in inclusion and equality persist after the conclusion of the intervention? Have processes contributed to sustaining these benefits? Have mechanisms been set up to support the achievement of inclusion and equality in the longer term?

1. Safeguarding of children and vulnerable adults

The extent to which safeguarding aspects have been considered and how the respective mechanisms been established, used and adhered to the target group with a special focus on persons with disabilities.

1. Equality and inclusion/gender equality

The extent to which women and men, persons with and without disabilities were actively involved during all steps in the project cycle and how the groups benefited from the intervention. Output and outcome data shall be disaggregated according to gender, age (adults/children) and disability status, IDPs and host communities. For the collection of data on disability, the use of the Washington Group Short Set of Questions is mandatory. Consultants will be guided on the Washington Group Questions and their application.

1. Methodology

The consultant will develop the detailed evaluation methodology based on this TOR and a briefing with CBM (and the partners). Mixed methods shall be applied and the methodology used shall fit the purpose of the evaluation and needs to be documented in detail as annex to the final report. The consultant is expected to submit a brief inception report detailing methodology for achieving the results of the study. The evaluation must meet the principles of being inclusive, participatory and interactive, involving both male and female members of the community, as well as persons with disabilities.

As a minimum, the evaluation process should include the following key steps:

• Desk review of relevant documents
• Initial briefing sessions with CBM Country Office and partners.
• Field Phase with application of specific data collection tools.
• Analysis of findings
• Reporting of findings and recommendations

As regards confidentiality or data protection, the consultant must take all reasonable steps to ensure that the respondents are not adversely affected by taking part in the evaluation in line with the – do-no-harm principle. S/he must keep their responses confidential, and must not do anything with their responses that they are not informed about at the time. Also, particular care must be taken with children and teenagers. Consent must be granted by a parent or caregiver for interviews with children aged under 18 and CBM’s child safeguarding policy should be applied in all circumstances. The consultant should create safe and accessible spaces for women and other vulnerable groups to participate in group discussions or interviews.

Following data collection and analysis, the consultant will share preliminary findings with CBM CO in Nigeria (and partners?). This shall be achieved through debrief sessions at CBM CO. The sharing of preliminary findings is an opportunity for the stakeholders to hear what the consultant has found and to be involved in thinking through recommendations. It should include constructive discussions around the key findings identified by the evaluation.

1. Management Responsibilities and Evaluation Team

CBM’s Responsibilities

• Contract the consultant;
• Oversee the implementation and quality of the evaluation;
• Provide project specific documents (Approved Project Narrative & Baseline study Report)
• Review and approve inception report
• Monitor and assess the quality of the evaluation and its process
• Provide guidance and institutional support to the external consultant
• Facilitate access to study areas;
• Facilitate and participate in briefing and de-briefing meetings
• Review draft evaluation report
• Pay the consultant based on the agreed terms and conditions specified in the expression of interest
• Approve the final report; all products related to the evaluation shall be submitted to CBM Nigeria at the end of the assignment and upon approval of the final report.

Responsibilities of the consultant

• The consultant will develop an inception report for review by the CO and Initiatives and approval by CBM CO.
• The draft evaluation report will be shared with CBM for review and feedback. The final evaluation draft will also be sent to CBM for approval and sign off. The final instalment of consultant’s fees will be disbursed following sign off.
• The consultant will be responsible for all logistics arrangements, flight, vehicles, accommodation and disability inclusive venues for interviews and group discussion, as well as for training of data collectors as needed.
• The consultant will be responsible for the arrangements of all data capturing tools/devices for the data collectors.
• The consultants will ensure that all data collectors have the basic skills to translate English into local languages used in areas of interventions, for easy understanding of the respondents.
• The consultant will be responsible for data security and anonymity.
• All consultants and enumerators and persons otherwise directly engaged with the target groups/project participants have to sign the Code of Conduct and Safeguarding Policy and must adhere to its stipulations.

Implementing Partner’s Responsibilities

• Facilitate the consultant’s access to key stakeholders and specific information or expertise needed to perform the assessment;
• Ensure that all stakeholders are kept informed;
• Support and participate in briefing and de-briefing meetings;

Requirements for the Evaluation Team

Consultant

The team leader will be an experienced senior consultant, who will be responsible for the overall evaluation process and the production of a high-quality final evaluation report.

Required Expertise

• Advanced degree in public health, humanitarian, development, social science or other related fields.
• Minimum of five years’ experience in conducting evaluations and two years’ experience in conducting evaluations in humanitarian settings with a mixed method approach.
• Understanding of emergency situations, monitoring processes, adaptation and evaluation.
• Analytical skills and ability to clearly synthesise and present findings in a concise and logical sequence.
• Ability to draw practical conclusions and to prepare well‐written reports in a timely manner.
• Awareness and sensitivity to the particular needs of persons with disabilities including psychosocial disability, women and children is desirable.
• Knowledge of the culture of the peoples in Benue, Nasarawa and Plateau States will be an added advantage.
• Consultants and data collectors must be able to speak and interpret Hausa and other local language to English language.
• An inclusive and gender balanced team is desirable.

Children and Adults at Risk Safeguarding Policy: As a condition of entering into a consultancy agreement the evaluators must sign the CBM Safeguarding Policy and abide by the terms and conditions thereof.

• The contracted consultant adheres to CBM Code of Conduct and commits to CBM’s Safeguarding Policy.
• Related Statements of the above have to be signed together with the contract.
• The consultant also commits to highest standards of data security.
• CBM is an equal opportunities employer, and particularly encourages qualified people living with disability to apply.

1. Deliverables

INCEPTION REPORT

The Consultant is expected to submit an inception report latest 2 weeks after receipt of all background documentation. The purpose of this report is to ensure that the consultant covers the most crucial elements of the exercise including the appropriateness and robust methodology to be employed as well as gender and diversity representation and participation of different social groups for inclusive analysis of needs.

The inception report provides the organization and the evaluation team with an opportunity to verify that they share the same understanding about the evaluation and clarify any expectations at the outset. The report should reflect the evaluation team’s review of literature and the gaps that the field work will fill. The report should also reflect opportunities for participation of persons with disabilities in the development of methodology, data collection and analysis

FINAL REPORT

The draft evaluation report (in easy to understand English and in accessible format) with a maximum 30 text pages (+ annexes) must be submitted to CBM Country Office within 3 weeks after completion of the field phase. CBM will provide feedback which has to be incorporated within 2 weeks.

The report shall include a 2-3 pages’ executive summary in English that can potentially be used for publication purposes (e.g. CBM website).

The evaluation report is an exclusive property of CBM and should not be released without prior authorization to any other party. The final report will be available through CBM.

DATA SETS

The Consultant will be expected to submit complete data sets (in Excel) of all the quantitative data as well as the original transcribed qualitative data gathered during the exercise. These data sets (filled consent forms) should be provided at the time of submission of the final report.

SUMMARY FINDINGS

On submission of the final report, the consultant is expected to submit a PowerPoint presentation (maximum 12 slides), summarizing the key findings and main recommendations. This should be submitted together with the final report.

Evaluation Schedule: Proposed Time Frame 20/03/2023 to 20/04/2023

Duration of Activities : The duration of the evaluation exercise shall be 30 days according to the schedule below:

Task: Desk Study/Literature review: Consultants’ desk

Location: Consultants’ desk

Number of Days: 4

Expected Dates;

Task: Submission of Inception Report and Briefing

Location: Online/CBM office

Number of Days: 2

Task: Field Phase, Project Visit or remote data collection phase

Location: Project locations

Number of days: 10

Task: De-briefing with initial findings

Location: Online/CBM office

Number of days: 1

Task: Analysis and draft report

Location: Consultants’ desk

Number of days: 8

Task: Review of draft report after feedback from CBM and partners (Finalising report)

Location: Anywhere

Number of days : 4

Task: Submission of final report

Location: CBM office

Number of days: 1

TOTAL: 30

1. Payment Mode

• Submission of inception report: First advance of 50% (based on invoice)
• Final payment following approval of all deliverables as stated above and receipt of invoice from consultant (50%)

Appendix 1: Stakeholder Analysis for Evaluation.

Stakeholders

Primary Stakeholders

e.g., Women and men with disabilities

What is their interest and contribution in the proposed project? They are also beneficiaries of the project and often time they play a major role as community volunteers and project influencers

What is their power and influence in the project (1-5 rating, 1=low, 5=high): 4

Will the project involve / these stakeholders in the evaluation? How? Yes

e.g., Girls and boys with disabilities

What is their interest and contribution in the proposed project? They are the main target of the project intervention, considering their vulnerability status, they contributed towards the achievements of the project

What is their power and influence in the project (1-5 rating, 1=low, 5=high): 2

Will the project involve / these stakeholders in the evaluation? How? Yes

Secondary stakeholders

e.g., Mothers, fathers, care givers

What is their interest and contribution in the proposed project? They play a vital role to the project, some of them are beneficiaries or community influencers.

What is their power and influence in the project (1-5 rating, 1=low, 5=high): 5

Will the project involve / these stakeholders in the evaluation? How? Yes

e.g., Volunteers

What is their interest and contribution in the proposed project? They also provide support to the project either directly or indirectly, their support is immeasurable.

What is their power and influence in the project (1-5 rating, 1=low, 5=high): 3

Will the project involve / these stakeholders in the evaluation? How? Yes

e.g., Local Government

What is their interest and contribution in the proposed project? They are good stakeholder and gate keepers of the project locations, they assist in integrating partners and the locals.

What is their power and influence in the project (1-5 rating, 1=low, 5=high): 4

Will the project involve / these stakeholders in the evaluation? How? No

e.g., Board of partner organisation

What is their interest and contribution in the proposed project? Administrative support are duly provided by them and sensitive discussions are mostly done with the board.

What is their power and influence in the project (1-5 rating, 1=low, 5=high): 4

Will the project involve / these stakeholders in the evaluation? How? No

e.g., Central Government

What is their interest and contribution in the proposed project? They regulate activities of organizations and provide the required support.

What is their power and influence in the project (1-5 rating, 1=low, 5=high): 3

Will the project involve / these stakeholders in the evaluation? How? No

e.g., Community

They took ownership of the project through the provision of support and acceptance of the project

What is their power and influence in the project (1-5 rating, 1=low, 5=high): 5

Will the project involve / these stakeholders in the evaluation? How? Yes

e.g., NGOs

What is their interest and contribution in the proposed project? They help in ensuring a good coordination of activities in project location and to mitigate duplications.

What is their power and influence in the project (1-5 rating, 1=low, 5=high): 2

Will the project involve / these stakeholders in the evaluation? How? No

e.g., Project staff

What is their interest and contribution in the proposed project? They are saddled with the day to day activities of the project.

What is their power and influence in the project (1-5 rating, 1=low, 5=high): 5

Will the project involve / these stakeholders in the evaluation? How? Yes

e.g., CBM staff

What is their interest and contribution in the proposed project? They provide technical support in project implementations

What is their power and influence in the project (1-5 rating, 1=low, 5=high): 5

Will the project involve / these stakeholders in the evaluation? How? Yes

[1] Participants are understood as those persons directly addressed and involved in the project/programme. The objective of a project/programmes is formulated to create positive changes in the lives of those participants.

[2] CRPD = Convention on the Rights of Persons with Disabilities.

[3] CEDAW = Convention on the Elimination of All Forms of Discrimination Against Women