Final evaluation Consultant (Yemen ) At Save the Children

Terms of Reference for Evaluation

“Emergency Food Assistance for Conflict-Affected Families in Hajjah, Hodeida, Taiz, Aden, and Lahj Governorates, Yemen”

August 17, 2021

Type of evaluation

Final evaluation

Type of contract

International

Contract type

Consultancy

Tentative start date

October 5, 2021

Duration of contract

26 working days

Project duration

36-39 monthss

Project location

Hajjah, Hodeida, Taiz, Aden, Lahj – Yemen

Thematic area(s)

Food Assistance, Nutrition

Program objective

Improve the immediate food security of the most vulnerable, conflict-affected families in Hajjah, Hodeida, Taiz, Aden, and Lahj Governorates in Yemen.

Evaluation objective

Evaluate the effectiveness and relevance of food and nutrition assistance, service delivery mechanisms, implementation processes, and entitlements in achieving project goals, purposes, and targets

Program Background

Yemen is the worst humanitarian crisis in the world, characterized by large-scale internal displacement, food insecurity, lack of access to health services and water, disruption of livelihoods, and a high prevalence of malnutrition. SC responds to food insecurity and nutrition needs through a BHA-funded project which seeks to increase household access to and consumption of diverse, nutritious food and improve nutrition among pregnant and lactating women and children under five through training and infant and young child feeding (IYCF) activities.

Scope and Purpose of Evaluation

The evaluation will assess and measure the extent to which the project was able to improve the food and nutrition security of beneficiaries; the effectiveness of project implementation (beneficiary selection, food assistance delivery mechanisms, and feedback and response mechanism [FRM]); the effectiveness of using community nutrition voluteers (CNVs) in community-level training and IYCF service provision; and capture lessons learned, best practices, and recommendations. Project performance will be measured through the outcome indicators: Prevalence of households with moderate or severe hunger (Household Hunger Scale); Percentage of households with ‘acceptable’ FCS; and c) Average rCSI score.

Key Questions

Effectiveness

To what extent did the project enable beneficiaries to improve their food and nutrition security through project outcomes, output indicators defined within the project logframe?
To what extend were three food delivery mechanisms (Cash-in-Hand, Commodity Food Vouchers [CFVs], and Value Food Vouchers [VFVs]) effective in households’ food security improvement? Which among the three was most effective?
How effective was the use of CNVs in providing community-level nutrition training and IYCF services? What worked well and what did not?
How effective was the FRM in providing accountability and voice to beneficiaries? What worked well and what did not?

Relevance

To what extend did the beneficiary selection process (targeting and verification) reach the most vulnerable households?

Methodology

The consultant is encouraged to reference the below suggested methodology table in their technical proposal but may propose changes where deemed appropriate.

Evaluation Question

Key Data to be Collected

Suggested Methods

Existing Data Soruces

To what extent did the project enable beneficiaries to improve their food and nutrition security as defined by project outcome and output indicators established within the project logframe?

All endline values for outcome indicators established in the project logframe

Beneficiary perceptions regarding effectiveness of interventions on their household and community

Household survey

Key informant interviews (KIIs) with project stakeholders (CDCs, local leader, CNVs, etc)

KIIs with project technical and operations staff – and other NGO actors where possible

Desk Review

KIIs with key project staff, including programming and operations

KIIs with project beneficiaries

Focus group discussions (FGDs) with project beneficiaries

Project baseline report and data

Project IPTT

Distribution reports and data

Project proposal and quarterly reports

Impact of the Suspension Report

Detailed Implementation Plan

Beneficiary Feedback Database

FRM sensitization materials

Selection criteria documentation, including registration and verification forms

To what extent was the project able to implement interventions in a timely manner? What factors limited the timeliness of implementation and did others experience the same delays? What was the effect of any delays and how did the project adapt?

Key project staff perspectives on factors impacting intervention delays, impact on programming, and adaptations made

Which among the three modalities / delivery mechanisms (Cash-in-Hand, CFVs and VFVs) was the most effective? What challenges and successes did each modality face during the project? How could future projects improve the effectiveness and efficiency of each modality?

All endline values for outcome indicators established in the project logframe

Beneficiary and community leader perceptions regarding the relevance and efficiency of the food delivery mechanism

Project staff experiences with the design, implementation and delivery of cash and voucher assistance

How effective was the use of CNVs in providingcommunity-level nutrition training and IYCF services? What worked well and what did not?

Beneficiary perceptions, including knowledge gains, referrals, and/or reported behavior change due to CNV training and/or interventions

CNV and lead project staff perceptions of the project’s effectiveness and relevance

How effective was the FRM in providing accountability and voice to beneficiaries? What worked well and what did not?

Beneficiary awareness of, utilization of, and access to project FRM

Project staff perceptions of challenges and successes during sensitization, utilization and close-out of beneficiary feedback

To what extent did the beneficiary selection process (targeting and verification) reach the most vulnerable households?

Beneficiary perceptions regarding awareness of, understanding of, and agreement with eligibility criteria, selection process, and results

Community leaders’ experiences through beneficiary selection process

Programming and technical staff perceptions and experiences regarding the strengths and challenges of beneficiary selection process, including gaps and recommendations for future projects

Household Survey Sampling
The household survey will utilize one stage simple random sampling stratified by Northern and Southern Yemen. The simple random sample will be use the project’s list of beneficiaries. Sample size calculation is based on estimated proportion of beneficiaries who had ‘acceptable’ FCS (score FCS>42) and proportion of beneficiaries who had little or no hunger (score HHS<2). The household survey will include a minimum of 446 households (223 each in the North and South).

Evaluation Timeline

The evaluation is planned for 26 days between October 5 2021 and late November to mid-December 2021. The number of days, methodology, and elements will be finalized after discussion with the selected consultant. The timeline and Level of Effort may change depending on visas, travel to and within Yemen, local authority approvals, COVID-19, and other logistical issues.

Evaluation Management

The consultant will produce an evaluation plan and inception report to outline the proposed methodology, including data collection methods, sampling considerations, timeline, logistics, etc. This will include the final report outline, which must align with the the donor recommended outline: https://usaidlearninglab.org/library/evaluation-report-template. Before any evaluation work is performed, SC must review and approve the draft plan, after which the consultant will design context- and project-specific data collection tools for KIIs, household interviews, and FGDs based on agreed evaluation questions. SC must also approve data collection tools before data collection begins. SC Yemen will provide the consultant with trained enumerators to support data collection.

Application process

The following qualifications and skills are expected of the consultant:

Minimum education of master’s degree in Emergency response, International Development, Social Science, Economics, Evaluation, or relevant field from a recognized university.
At least 7 years’ experience in evaluation of programs, including cash, food security, and/or nutrition.
Thorough understanding of mixed-methods data collection, including design and collection of household surveys.
Strong skills in data analysis and familiarity with at least one data analysis software (SPSS, STATA).
Ability to create compelling data visualizations to convey key findings.
Excellent analytical, research methods and communication skills.
Fluency in English is required and Arabic preferre).
Experience working in the middle east and/or Yemen is a plus.

Roles and responsibilities:

Submit inception report and finalize based on SC feedback and approval.
Obtain SC approval of evaluation methodology and data collection tools.
Notify SC of delays during the final evaluation.
Supervise and ensure proper data collection and entry per methodology and protocol.
Clean and analyze qualitative and quantitative raw data.
Debrief preliminary findings to SC .
Submt draft and final evaluation report to SC.

Expected outputs

Inception report (max. 20 pages) detailing proposed methodology.
Soft copies of final quantitative and qualitative tools
Completed copies of survey questionnaires, consent forms, and qualitative notes.
Soft copies of raw and clean compiled data sets.
Final Report (in English) following the outline described above.
PowerPoint presentation of findings (max. 15 slides) and delivery of findings to key stakeholders.

Interested candidates should submit an expression of interest including:

Technical proposal: briefly and clearly describe your understanding of the task, technical aspect of the proposal, and proposed methodology (evaluation strategy, sampling design, data collection methods, data processing & analysis, data quality control measures, and timelines or operational plan).

Previous Experience: previous experience conducting final evaluations for BHA-funded projects (including FFP, OFDA, or other USAID-funded grant/project), working in Yemen (if any), and/or in non-permissive environments with a fragile security situation.

Detailed curriculum vitae (CV): detailed CV with contact details (including at least two traceable references). If the consultant is a firm, provide CVs of key staff that will work on the evaluation.

Signed financial proposal/budget: tasks should be broken down into modules: Consultancy fees, Questionnaire development, Data processing & analysis, Communication, Reporting, and Miscellaneous (stationeries, printing, etc.).

Example of Previous Work: SC strongly suggests that applicants submit one copy of an evaluation report completed by the consultant/firm. The copy may be a limited sample of the report to protect client confidentiality but should demonstrate past experience and ability.

Applications should be sent with the subject “Emergency Food Assistance for Conflict-Affected Families in Hajjah, Hodeida, Taiz, Aden, and Lahj Governorates, Yemen – FFP”.

Applicants should reference the full TOR found here when submitting their full application:

https://drive.google.com/file/d/1s7WorurdsF1SMY0KYtqmKD9lzJWcRKjv/view?usp=sharing