Endline Evaluation Consultant, Improved Alert at Catholic Relief Services (CRS) At Catholic Relief Services

Catholic Relief Services (CRS) seeks an independent consultant to carry out the final evaluation of its IMPROVED ALERTS program in North East Nigeria.

Location and Timing of Task

The final evaluation is to be conducted in Maiduguri, Borno State in the month of AUGUST 2024

Background to task

In responding to the protracted crisis in the North-East of Nigeria, CRS is implementing since October 2023, the BHA-funded program in Borno – Improved Access to Lifesaving Emergency Response, Recovery and Timely Support Project (IMPROVED ALERTS) – in Gubio, Kaga and Magumeri LGAs. Improved ALERTS is funded by the United States Agency for International Development (USAID) Bureau of Humanitarian Assistance (BHA) and for its implementation, CRS has partnered with Justice Development and Peace Commission (JDPC) and Northeast Youth Initiative Forum (NEYIF).

As part of its deliverables to USAID/BHA, CRS intends to conduct a final evaluation for the Improved ALERTS project and submit a report to the donor.

Improved ALERTS project

Improved ALERTS project is built on over five years of successful humanitarian programming in NE Nigeria and is specifically layered on the Ceton Rayuka (Saving Lives) II (CRII) BHA-funded project. For the intervention, CRS has targeted the most vulnerable families for both IDPs and host families. The project

delivers an integrated package of lifesaving food assistance, Nutrition and WASH services to Crisis-affected IDPs, returnees, and host communities in Gubio, Kaga and Magumeri (the project’s logic of intervention (purpose, sub-purpose, outputs, etc.) is detailed in the SoW of the final evaluation). The project targeted to reach 63,457 participants with the Food assistance component, 15,000 participants with the agriculture component, 24,000 participants with the Nutrition component and 58,000 participants with the WASH component.

Purpose of the evaluation

This evaluation aims to assess the progress made and effects produced by the improved ALERTS project across its entire life (from the inception to closeout). This may include evaluating the change in food security status, reduction in hunger scores after receiving food assistance, different food security coping strategies adopted by communities, change in the nutritional status of malnourished children supported, knowledge, practices, and skills retention for all training, sensitizations, and counseling on optimal MIYCN-E practices conducted. Furthermore, the evaluation will consider improvements in the utilization of water and sanitation facilities and changes in hygiene behavior and practices. For quantifying the changes produced by the project, the evaluation will draw on evidence generated from the baseline survey and other related previous surveys, and will be guided by the following criteria: overall achievement, relevance, efficiency, effectiveness, coherence/ coordination, and indicative sustainability. The evaluation should provide evidence to gauge the extent of change compared to the baseline, the project’s effect and progress throughout its lifecycle, and the degree to which Crisis-affected IDPs, Returnees, and host communities in Gubio, Kaga, and Magumeri have been enabled to improve access to lifesaving Food Assistance, Nutrition and WASH services.

Methodology of the evaluation

The evaluation will use a mixed method design allowing for the collection and analysis of both quantitative and qualitative data to gain a more comprehensive understanding of the observed changes. The design will follow a sequential process, with the quantitative data being collected and analyzed first to identify any patterns or trends. The subsequent qualitative data collection and analysis will then be used to explain the observed changes and provide context to the quantitative results.

The quantitative component of the evaluation will, in addition to collecting primary HH-level, beneficiary-based cross-sectional Endline survey data for Food Assistance and WASH indicators, involve reviewing the baseline data and other available data sources for the Endline results of Nutrition indicators. The output of the quantitative arm of the study will be a comprehensive presentation of the baseline and Endline values for all indicators against the approved targets in the project indicator tracking table across the three sectors. The difference between baseline and end-line values will be tested for statistical significance and discussed where it is possible. The qualitative component will use participatory qualitative approaches to gain sufficient in-depth understanding of the contextual factors and the results of the multi-sectoral assistance, as well as the perceptions of the key stakeholders These two approaches will be complemented by content review and analysis of key project documents, deliverables, and other products.

Evaluation team composition and requirements

The evaluation is expected to be conducted by two evaluators: a lead evaluator and an assistant evaluator who will oversee the execution of the entire evaluation process in partnership with CRS staff. The Emergency MEAL Coordinator will lead the overall coordination of the evaluation, checking quality and compliance issues to the CRS and USAID evaluation guidelines. The evaluator will work with support from enumerators from CRS for data collection.

Required qualifications for the lead evaluator are as follows:

1. Postgraduate qualification in Development Studies, Monitoring and Evaluation, Statistics, or International Development, PhD will be an added advantage.
2. Extensive knowledge of evaluating multisectoral projects in emergencies and a good understanding of the emergency response in the Northeast.
3. At least 5 years’ experience in conducting evaluations in complex environments, experience evaluating USAID/BHA funded projects. Experience working in Nigeria’s Northeast states, preferably BAY states will be an added asset.
4. Extensive experience in evaluating multi-sectoral programs implemented through multiple partners.
5. Excellent oral and written communication skills in English and fluency in Hausa and Kanuri an advantage.
6. Extensive knowledge of qualitative and quantitative data collection methods and analytical methods and techniques.
7. Experience in generating strategic, informative, and action-oriented recommendations to CRS project staff and strategic management

Consultancy deliverables

The consultancy will provide a full report, written in English, not to exceed 25 pages, excluding title page, list of acronyms, table of contents and annexes comprised of among other things the updated Indicator Performance Tracking Table (IPTT), finalized data collection tools, raw datasets, transcribed notes from FGDs and KII and observation note, and files containing analyses of data, both end-line and primary data collection. The recommended structure of the report is available in the detailed evaluation SoW.