Consultant – McGovern-Dole Food for Education and Child Nutrition 2017 Final Evaluation – Remote At Mercy Corps

Background:

Mercy Corps is a leading global organization powered by the belief that a better world is possible. In disaster, in hardship, in more than 40 countries around the world, we partner to put bold solutions into action — helping people triumph over adversity and build stronger communities from within.

Mercy Corps has been working in Kyrgyzstan since 1994. For over fifteen years, Mercy Corps has been providing nutritional and technical support to education institutions in the Republic of Kyrgyzstan, including public schools, kindergartens, and vocational schools. Since 2003, Mercy Corps has served as the implementing partner of the United States Department of Agriculture (USDA) McGovern-Dole Food for Education and Child Nutrition program (MGD). This program is implemented in support of the National School Feeding Program developed by the Ministry of Education and Science (MoES) of the Kyrgyz Republic, which is a key stakeholder for this program.

In 2017-2021, Mercy Corps supported 440 public schools in rural areas across the country by providing over 2,000 metric tons of supplementary commodities. More than 90,000 primary grade students benefited from nutritious hot meals in schools prepared from these commodities. In addition to school feeding, the program includes several other components for a comprehensive approach, such as WASH infrastructure improvements, Social Behavior Change (SBC), literacy promotion, and capacity building for school staff.

Consultant Objectives / Project Description:

Initially, the program was scheduled to complete on September 30, 2021. The original Final Evaluation (FE) was commissioned by Mercy Corps Kyrgyzstan in March 2021, and the latest draft of the report was reviewed by USDA in October 2021. However, due to implementation delays caused by the ongoing COVID-19 pandemic, the program was extended until the end of December 2021, including completion of remaining activities and a final round of monitoring data collection in October-November. These final results will need to be incorporated into the FE analysis and report.

In addition, in November 2021 MC expects the results of a quasi-experimental study on several key behavioral variables to help attribute program impact at the household level. The first phase of this study was completed in early 2020 and showed promising results. Therefore, MCKG would like to have the full results of both phases reflected in the FE report as they are directly relevant to the program results framework.

The evaluation methodology was a non-experimental mixed-methods study and included the following:

• Analysis of existing monitoring data collected throughout the program against set indicator targets (secondary data);
• A quantitative endline reading assessment for second grade students^[1] (primary data);
• Semi-structured interviews with key program staff, USDA, WFP, USAID, Save the Children, and stakeholders from MoES at national and district levels (primary data);
• FGDs with adult program beneficiaries: parents and school staff (primary data);
• Structured observation carried out in a sample of program schools to assess state of provided equipment and infrastructure improvements^[2] (primary data);
• Document review.

Consultant Activities:

In order to finalize this FE, MCKG is looking for an individual consultant to review newly available program data and research results and update the FE report accordingly, specifically:

• Read the latest evaluation report draft and identify areas for strengthening and follow up; conduct a briefing with the senior MCKG staff to review the gaps and outline the proposed review.
• Conduct 2-3 KII with program staff to get the latest information on the status of program activities, COVID-19 adaptations in light of the recent wave in Kyrgyzstan that took place after the primary data collection for this evaluation was completed and discuss any other remaining follow up points.
• Analyze newly collected program monitoring data for key program performance indicators, such as number of school meals provided, number of students enrolled in program schools, number and value of infrastructure repair projects completed, and student attendance rate. Update LoP indicator results accordingly in the report narrative and relevant annexes.
• Review the results of the second phase of the household study to identify results relevant for the key evaluation questions and incorporate these results in corresponding sections of the evaluation report. If necessary, suggest data-based revisions to the preliminary evaluation findings and conclusions.
• Review and complete the section on COVID-19 program adaptations. MC will provide a record of all adaptations and program changes.
• Review the primary data collected for this evaluation to support any additional conclusions made during this review.
• Review and update evaluation recommendations, where relevant (based on the updated analyzed data).
• Review the latest feedback from USDA and MC and make sure that all relevant comments are addressed in full. Record responses to all USDA comments and explain how they were addressed in a separate document^[3].
• Ensure that all different sources of primary and secondary data and additional analysis are well integrated together for a comprehensive final evaluation narrative.

Consultant Deliverables:

The Consultant will produce the following deliverables:

• A revised Final Evaluation report.
• Cleaned datasets and original transcripts of all primary data collected.
• Feedback tracker with corrections and responses to USDA questions and MC feedback.

Timeframe / Schedule:

The estimated LoE for this Review is 28 workdays between November 1 to December 20, 2021 (dates are tentative and will be confirmed with the selected consultant).

The assignment will be carried out remotely, no travel is required. MC Kyrgyzstan will provide all available data and resources (see Annex I for indicative list).

The Consultant will report to:

MC Kyrgyzstan Country Director

The Consultant will work closely with:

• MC Kyrgyzstan: MEL Manager and Director of Programs;
• MC HQ: Senior MEL Director and Senior Program Officer

Required Experience & Skills:

• At least 10-15 years of evaluation experience, particularly of long-term development programs and complex and dynamic contexts; experience with evaluating food assistance and/or school feeding programs will be a significant advantage.
• Expertise in conducting mixed-methods evaluations and integrating data from different sources to support evaluation conclusions and recommendations.
• Proven experience in conducting advanced statistical and econometric analysis and practical knowledge of experimental and quasi-experimental research methodologies.
• Demonstrated knowledge of various qualitative and quantitative research/evaluation methods, including experimental and/or quasi-experimental research design.
• Previous experience with carrying out evaluation assignments for USDA and/or Mercy Corps will be a strong advantage.
• Knowledge of and prior field experience in the Kyrgyz Republic will be a strong advantage.

Diversity, Equity & Inclusion Achieving our mission begins with how we build our team and work together. Through our commitment to enriching our organization with people of different origins, beliefs, backgrounds, and ways of thinking, we are better able to leverage the collective power of our teams and solve the world’s most complex challenges. We strive for a culture of trust and respect, where everyone contributes their perspectives and authentic selves, reaches their potential as individuals and teams, and collaborates to do the best work of their lives.

We recognize that diversity and inclusion is a journey, and we are committed to learning, listening and evolving to become more diverse, equitable and inclusive than we are today.

Equal Employment Opportunity We are committed to providing an environment of respect and psychological safety where equal employment opportunities are available to all. We do not engage in or tolerate discrimination on the basis of race, color, gender identity, gender expression, religion, age, sexual orientation, national or ethnic origin, disability (including HIV/AIDS status), marital status, military veteran status or any other protected group in the locations where we work.

Safeguarding & Ethics Mercy Corps team members are expected to support all efforts toward accountability, specifically to our stakeholders and to international standards guiding international relief and development work, while actively engaging communities as equal partners in the design, monitoring and evaluation of our field projects. Team members are expected to conduct themselves in a professional manner and respect local laws, customs and MC’s policies, procedures, and values at all times and in all in-country venues.

Annex I: Existing Program Data Sources and Documents

Mercy Corps will provide the following documents and data to support carrying out agreed upon review activities:

• Final Evaluation report draft with the latest feedback from MC and USDA;
• MC-USDA McGovern-Dole Agreement and all subsequent modifications reflecting key adjustments made to the program design, including the COVID-19 pandemic;
• Reading assessment results and datasets, both baseline and endline;
• Infrastructure needs assessment conducted to select schools for infrastructure and WASH repair grants;
• Gender assessment report conducted by an independent researcher to support adaptation of the Social Behavior Change component to include more male participants;
• Biannual program reports submitted to USDA (seven reports at the time of evaluation);
• Midterm program evaluation report conducted by independent evaluators in 2019, available here;
• Baseline Study on Nutritional and Learning Variables in Kyrgyz Households report and the dataset, both Phase I and II;
• Yearly program workplans approved by USDA, including narrative description;
• A summary of program adaptations made during the program, including due to the COVID-19 pandemic, and rationale behind them;
• Monitoring, Evaluation and Learning Plan with a summary of adjustments made throughout the program;
• Performance Management Plan containing information on program indicators and results up to and including at the time of evaluation;
• List of participating program schools, local government bodies, and key stakeholders at MoES, MoH, etc.;
• List of program schools and stakeholders included in the final evaluation primary data collection;
• Indicator Performance Tracking Table with biannual and LoP indicator results and comments;
• Indicator-specific databases, such as number of school meals provided and student attendance;
• Routine monitoring reports from the field team on program implementation quality;
• Program success stories, media coverage archives, video and photo reports, infographics, and other media and communication materials.

^[1] A baseline reading assessment was conducted by the implementation partner, Save the Children, in 2018. A follow on endline assessment using the same methodology was planned for the final evaluation, but at the moment its feasibility is unclear due to the ongoing COVID-19 pandemic. This issue will be discussed in more detail with the selected evaluating agency and Dave the Children during the inception phase.

^[2]Appropriate sample size and sampling approach should be suggested by the bidding evaluation entity in the application; it should be taken into consideration that the program is implemented in all 7 provinces (oblasts) in the country.

[3] MC has a feedback tracking sheet for this evaluation; the consultant will be expected to work with the same document.