Project Evaluation Consultant – Cotabato (Filipino Nationals Only) At Acción contra el Hambre España

PROJECT Background

Since July 24, 2022, Action Against Hunger has been implementing a BHA-funded project an INCLUSIVE DISASTER PREPAREDNESS PROJECT or “IPrepared Project” that aims to build inclusive and gender-equal disaster preparedness capacities of local government units and communities in areas vulnerable to natural and human-induced hazards in the Bangsamoro Autonomous Region of Muslim Mindanao.

They contribute to the building of capacities of local government units and communities on DRR and resilient livelihoods within the BARMM provinces, many of which have limited capacities to develop the mechanisms and systems, and implement the modalities to ensure prepared and resilient communities and local government units. The interventions were supplemented in coordination with other donor-funded activities, thereby providing a holistic package of support to target communities. The project is being implemented in BARMM, 3 municipalities in Lanao del Sur, and 4 Maguindanao del Sur.

Rationale for the Project

On December 2021, the devastation brought by Typhoon Rai, locally named, Typhoon Odette, saw the imperatives of ensuring preparedness capacities are built at the Local Government Units (LGUs) and communities’ level, particularly, strengthening local policies, processes, and systems on early warning and preparedness, mainstreaming of resilient livelihood (RL) in the frame of disaster risk reduction and management (DRRM).

The experience and devastation of the current natural events provides the opportunity to look at risk reduction and resilience building in the core of programming and planning for provincial and municipalities in the BARMM region. It was also an avenue to shift activities from needs-based response and rehabilitation to more sustainable and strategic forecast based, climate-informed, shock responsive social protection (SRSP) measures to address the recurring risks in the areas.

Upon knowing to precarious situations in the locality, AAH realized the necessity of implementing an INCLUSIVE DISASTER PREPAREDNESS PROJECT or “IPrepared Project” that aims to build inclusive and gender-equal disaster preparedness capacities of local government units and communities in areas vulnerable to natural and human-induced hazards in the Bangsamoro Autonomous Region of Muslim Mindanao. AAH then partnered with the Bureau of Humanitarian Agency so that the envisioned iPrepared Project would finally be implemented in BARMM.

The Iprepared project was aimed to reach 120,773 individuals in 7 municipalities of the province of Lanao del Sur and Maguindanao del Sur and with 21 assisted barangays, through building of capacities of local government units and communities on DRR and resilient livelihoods; support for Municipal and Barangay level Local Government Units and communities at substantial risk of natural hazards to improve their forecast based early warning and DRR planning and responsive social protection including resilient livelihoods. This action will focus on building local capacities to develop the mechanisms and systems to ensure prepared and resilient communities and local government units.

While the action is focused on strengthening the capacities of the LGUs and the communities, it will also ensure that the expertise of the mandated national government agencies, science-based institutions, academe, local sectoral-focused non-government organizations, and development partners among others are maximized. The Action will ensure partnerships with mandated agencies, i.e., the Office of Civil Defense (OCD) and MILG for contingency planning, and to ensure the alignment of tools used and adherence to the required standards set by these agencies. Science-based institutions like PAG-ASA and the academe will provide the technical substance, and in-depth analysis to the assessments and action. The Iprepared project has been implemented since July 2022 and will end by June 2024. The evaluation period is set to be June 16 to July 30, 2024, to determine its impacts, relevance, and sustainability, among others.

Project Objectives

• Program Goal: To Build Inclusive and Gender Equal Disaster Preparedness Capacities of Local Government Units and Communities in Areas Vulnerable to Natural and Human-induced Hazards in the Bangsamoro Autonomous Region of Muslim Mindanao, Mindanao, Philippines.
• Objective 1: To improve local government and community processes, integrating climate and risk information in early warning, appropriate and mitigation interventions in preparedness planning, and responsive social protection to reduce impacts of natural hazards.
• Objective 2: To enhance the agricultural skills of farming communities by integrating preparedness frameworks and strategies.
• Objective 3: To improve community access to livelihood opportunities*.*
• Geographical Scope of the Project: xxx
• Lanao del Sur
  • Municipality of Balabagan
  • Municipality of Calanogas
  • Municipality of Kapatagan
• Maguindanao del Sur
  • Municipality of Datu Piang
  • Municipality of Datu Salibo
  • Municipality of Datu Saudi Ampatuan
  • Municipality of Radja Buayan

Project Current Status

Currently, the project operation team is in the concluding activities for some other indicators proposed and in discussion with communities and local partners regarding the sustainability and transition strategy of the project.

PURPOSE AND OBJECTIVES OF THE EVALUATION

The external evaluation of the project is planned to be carried out as per the planning in the proposal and in line with the agreement between BHA and Action Against Hunger (Award Number: 720BHA22GR00194). The evaluation results are expected to inform future Action Against Hunger, other stakeholders, and donors’ humanitarian assistance and to assess the overall performance and relevance of the project to the affected population.

Objectives of the Evaluation

This external evaluation is expected to be carried out following the Development Assistance Committee (DAC) evaluation criteria. The evaluation aims to assess the overall performance of the intervention, to determine if it has reached its intended objectives primarily focusing on the likelihood of impact on beneficiaries’ self-reliance and the project’s sustainability aspects, and to look at all the components of the result chain (inputs, activities, and results) as well as key contextual factors that might have enabled or hindered its delivery. The key objectives of the external evaluation are the following:

• To assess the relevance, effectiveness, efficiency, and sustainability of the project planning and implementation.
• To provide accountability to Action Against Hunger’s various stakeholders, to the project donor, BHA
• To identify lessons learned and generate knowledge and to inform the development of future projects that support disaster Preparedness in similar contexts. All the lessons and recommendations on project’s process shall be of great value for similar projects in the future.

Users of the Evaluation

Target users of the external evaluation will be the following:

• Direct Users: Action Against Hunger field teams, Action Against Hunger (HQ),
• Indirect Users: Action Against HungerNetwork, Partner Organizations, Others

Use of the Evaluation

The evaluation will be used to learn from experience in developing new strategies at the global level, collect lessons learned and good practices for future projects in the country, and collect evidence of Action Against Hunger’s success on the ground to inform beneficiaries of the developments of the project.

EVALUATION SCOPE

Evaluation Purpose

Action Against Hunger will conduct an external evaluation to appraise the program’s progress at a single point in time and to answer specific questions to guide decision-makers and to provide information on whether underlying theories and assumptions used in program development were valid, what worked and what did not work, and why.

Evaluation Type

Action Against Hunger will conduct an external evaluation to assess the impact of the activities implemented. In the case of engaging the external evaluator, the MEAL Coordinator of Action Against Hunger will lead the design and management of all evaluation plans. This will include:

• Developing terms of references and scopes of work.
• Designing and/or approving research methodologies, data collection instruments, and timelines.
• Overseeing data analysis plans and reporting requirements/deliverables; and
• Ensuring findings and reports are widely disseminated.

Evaluation Questions

As per the Organization’s Evaluation Policy and Guidelines, Action Against Hunger adheres to the Development Assistance Committee (DAC) criteria approach for evaluating its programs and projects. Specifically, Action Against Hunger uses the following DAC criteria:

• Relevance/Appropriateness
• Coherence
• Coverage
• Efficiency
• Effectiveness
• Sustainability
• Likelihood of Impact

To the latter list, Action Against Hunger adds an additional criterion which is Design. Action Against Hunger also promotes a systematic analysis of the monitoring system in place within the criteria.

Evaluation questions have been developed to help the evaluator/s assess the project against these seven criteria. The evaluator may adapt the evaluation criteria and the questions, but any fundamental changes should be agreed between the Mission focal persons and the evaluator/s that will be reflected in the inception report.

The evaluation should consider the following questions:

• The real needs and problems of the beneficiaries that the project aims to address: Was the assistance appropriate about the customs, practices, and social organization of the target population and were beneficiaries consulted about their needs and priorities?
• Evaluate how recommendations made during the implementation of the project were integrated or used for other identified opportunities and/or constraints that need to be accommodated in the implementation, to increase the impact and relevance of the project.
• Assess the validity of the project approach and strategy to address needs/gaps based on local context and beneficiary situation in different target areas; were the planned activities targeted to specific vulnerable groups and inclusive in nature?
• Were the expectations of the roles, capacity, and commitment of stakeholders realistic and likely to be achieved?

All independent external evaluations are expected to use the DAC criteria in data analysis and reporting. In particular, the evaluator/s must complete the DAC criteria rating table and include it as part of the final evaluation report.

Evaluation Methods

Prior to the evaluation taking place, the evaluator is expected to attend an evaluation technical briefing with the mission focal person. In case a face-to-face briefing is not possible due to budget or logistic constraints, briefings by telephone or online must be agreed in advance. The evaluator/s will then undertake a desk review of project materials, including the project documents (Annex IV) and proposals, progress reports, outputs of the project (such as Baseline Report, Post Distribution Monitoring (PDM) Report, Technical Assessment Report, Interim Report, publications, communication materials, videos, recordings, etc.), results of any internal planning process and relevant materials from secondary sources.

The evaluator should clearly state the sampling approach in terms of sites and beneficiaries. Due to the short period of time in the field, access to a representative sample through a probability sampling method is highly challenged. Convenience sampling is suggested for all project sites and beneficiaries. The selection of the Barangays should consider the ones with highest number of beneficiaries to have a greater representation of beneficiaries. This is because some of the areas are remote and hard to reach. Beneficiaries belonging to different types of vulnerability and those who are staying in hard-to-reach areas will be invited to a nearby and more accessible location for the evaluation. The criteria for the selection of beneficiaries should involve Action Against Hunger staff. The sampling approach should be adjusted and further detailed by the evaluator in the inception report.

Taking into consideration the current situation wherein some areas of intervention are armed conflict situation, the external evaluation will use remote type of data gathering using mobile devices (e.g. text message, call) to adapt to the travel restrictions being observed.

Evaluation Timeline

Considering the duration of the action and budget, the evaluation will be conducted at the end of the action for which this Terms of Reference and workplan will serve as a guide for the entire evaluation process.

Evaluator Profile

Action Against Hunger will conduct an external independent evaluation with the following profile for the evaluator/s:

• Local consultants will be given priority depending on qualifications and availability before the consideration of engaging international consultants.
• Extensive knowledge of multi-sectoral interventions in humanitarian settings.
• Significant field experience in the evaluation of emergency/early recovery interventions in conflict-affected areas.
• Relevant degree/equivalent experience related to the evaluation to be undertaken.
• Significant practical experience in coordination, design, implementation, monitoring, and evaluation of programs.
• Good communication skills and experience in workshop facilitation.
• Ability to write clear and useful reports (may be required to produce examples of previous work).
• Fluent in English, Tagalog, Bisaya, Maranao, and Maguindanaoan will be an advantage.
• Understanding of BHA requirements.
• Ability to manage the available time and resources and to work to tight deadlines; and
• Independence from the parties involved.
• Acceptable budget according to the format submitted (Annex VII).

Cross-Cutting Issues

Throughout the evaluation process, gender concerns will be addressed in line with the Action Against Hunger Gender Policy. All data will be disaggregated by sex and age and different needs of women, men, boys and girls will be considered throughout the evaluation process.

EVALUATION APPROACH AND QUESTIONS

As per the Organization’s Evaluation Policy and Guidelines, Action Against Hunger adheres to the Development Assistance Committee (DAC) criteria approach for evaluating its programs and projects. Specifically, Action Against Hunger uses the following DAC criteria:

• Relevance/Appropriateness
• Coherence
• Coverage
• Efficiency
• Effectiveness
• Sustainability
• Likelihood of Impact

Evaluation questions have been developed to help the evaluator/s assess the project against these seven criteria (refer to Annex II). The evaluator may adapt the evaluation criteria and the questions, but any fundamental changes should be coordinated with the Mission focal persons to be reflected in the inception report.

All independent external evaluations are expected to use the DAC criteria in data analysis and reporting. In particular, the evaluator/s must complete the DAC criteria rating table (refer to Annex III) and include it as part of the final evaluation report.

EVALUATION DESIGN AND METHODOLOGY

This is a one-group design evaluation, examining a single intervention, without including any comparison with units (people, communities, etc.) that did not participate in the intervention. This evaluation is also expected to look at units participating in the project “before and after” the intervention has been in operation.

Outlined below is the suggested methodological approach for the evaluator to collect quantitative and qualitative data and the chronological steps of the evaluation process. To the possible extent, the evaluator will develop data gathering instruments and methods which allow collecting sex-disaggregated data. The instruments need to make provision for the triangulation of data where possible.

Evaluation Briefing

Prior to the evaluation taking place, the evaluator is expected to attend an evaluation technical briefing to be facilitated by the Mission focal persons. In case a face-to-face briefing is not possible due to budget or logistic constraints, briefings by telephone or online must be agreed in advance.

Action Against Hunger HQ and Mission Briefing

As part of the evaluation, the evaluator/s will attend a briefing with HQ and Mission stakeholders to get preliminary information about the project being evaluated. In case a face-to-face briefing is not possible due to budget or logistic constraints, briefings by telephone or online must be agreed in advance. At the mission level, the evaluator/s may be requested to meet and brief the donor representative based in Manila.

Desk Review

The evaluator/s will undertake a desk review of project materials, including the project documents and proposals, progress reports, outputs of the project (such as Baseline Report, Post Distribution Monitoring (PDM) Report, Technical Assessment Report, Interim Report, publications, communication materials, videos, recordings, etc.), results of any internal planning process and relevant materials from secondary sources.

Sampling

The evaluator should clearly state the sampling approach in terms of sites and beneficiaries. Due to the short period of time in the field, access to a representative sample through a probability sampling method is highly challenged. Convenience sampling is suggested for both project sites and beneficiaries. The aim is for the evaluation to ideally cover 2 barangays per Municipality minimum. Beneficiaries belonging to different types of vulnerability and for those who are staying in hard-to-reach areas will be invited to a nearby and more accessible location for the evaluation. The criteria for the selection of beneficiaries should involve Action Against Hunger staff. The sampling approach should be adjusted and further detailed by the evaluator in the inception report.

Data gathering methods will adapt to the travel restrictions observed by the community at the time of the evaluation.

Inception Report

At the end of the desk review period and before the field mission, the evaluator/s will prepare a brief inception report (5 pages maximum). The report will be written in English and will include the following sections:

• Key elements of the TOR to demonstrate that the evaluator will adhere to the TOR.
• The methodological approach to the evaluation including an evaluation matrix in annex to specify how the evaluator will collect data to answer the evaluation questions, pointing out the limitations to the methodology, if any, and the choice of sites per field visit.
• The data collection tools (i.e. Focus Group Discussions (FGDs), Survey Questionnaires, Key Informant Interviews (KIIs));
• A detailed evaluation workplan.
• State adherence to Action Against Hunger policies such as Evaluation Policy, Code of Conduct while engaging with all stakeholders of the project; and
• Outline the evaluation report format.

The inception report will be discussed and approved by the country office focal person who will be the primary contact in managing and coordinating the evaluation.

Field Mission

Primary data collection techniques

As part of the evaluation, the evaluator will interview key project stakeholders (Annex V: expatriate/national project staff, local/national representatives, local authorities, civil society leaders, and donor representatives). The evaluator/s will use the most suitable format for these interviews as detailed in the inception report. The evaluator/s will collect information directly from beneficiaries. Towards enriching triangulation, the evaluator is expected to conduct FGDs (beneficiaries, non-beneficiaries, key informants – communities and LGU leaders) and surveys.

Field visits

The evaluator/s will visit the project sites and the facilities provided to the beneficiaries.

Secondary data collection techniques: Desk review

The evaluator/s will further review complementary documents and collect project monitoring data or any other relevant statistical data.

Debriefing and stakeholder’s workshop

The evaluator shall facilitate a learning workshop in country to present preliminary findings of the evaluation to the project and key stakeholders: to gather feedback on the findings and build consensus on recommendations; and to develop action-oriented workshop statements on lessons learned and proposed improvements for the future. In case a face-to-face workshop is not possible with stakeholders, debriefing will have to be done through Skype or messenger.

Presentation of Findings to AAH

The evaluator/s is expected to prepare a presentation of the evaluation report to AAH.

5.7 Evaluation Report

The evaluation report shall follow the following format and be written in English:

• Cover Page.
• Summary Table.
• Table of Contents.
• List of Acronyms.
• Executive Summary (must be a standalone summary, describing the project, main findings of the evaluation, and conclusions and recommendations. This will be no more than 2 pages in length);
• Background Information.
• Methodology (describe the methodology used, provide evidence of triangulation of data and presents limitations to the methodology);
• Findings (includes overall assessment of the project against the evaluation criteria, responds to the evaluation questions, all findings are backed up by evidence, cross-cutting issues are mainstreamed, and unintended and unexpected outcomes are also discussed).
• Conclusions (conclusions are formulated by synthesizing the main findings into statements of merit and worth, judgements are fair, impartial, and consistent with the findings).
• Lessons Learnt and Good Practices (presents lessons that can be applied elsewhere to improve project performance, outcome, or impact, and identify good practices: successful practices from those lessons which are worthy of replication; and further develop on one specific good practice to be showcased in the template provided in Annex VI).
• Recommendations (recommendations should be as realistic, operational and pragmatic as possible. That is, they should take careful account of the circumstances currently prevailing in the context of the action, and of the resources available to implement it locally. They should follow logically from conclusions, lessons learned, and good practices. The recommendations should include an analysis on enhancing the quality of the Monitoring and Evaluation (M&E) framework/tool and management process/response. The report must specify who needs to take what action and when. Recommendations need to be presented by order of priority);
• Annexes (These should be listed and numbered and must include the following: good practice template, Evaluation Criteria Rating Table, list of documents for the desk review, list of persons interviewed, data collection instrument, and evaluation TOR).

The whole report shall not be longer than 30 pages, 50 pages including annexes. The draft report should be submitted no later than 10 calendar days after departure from the field. The final report will be submitted no later than the end date of the consultancy contract. Annexes to the report will be accepted in the working language of the country and project subject to the evaluation.

Debriefing with Country Office Focal Person

The evaluator should provide a debriefing to Country office focal persons to discuss any issues related to the evaluation report. In case a face-to-face debriefing is not possible debriefing will have to be done through TEAMS or other online means.

Debriefing with Action Against Hunger HQ

The evaluator should provide a debriefing and a presentation with the relevant Action Against Hunger HQ on his/her draft report, and on the main findings, conclusions and recommendations of the evaluation. Relevant comments should be incorporated in the final report. In case a face-to-face debriefing is not possible with Action Against Hunger HQ due to budget or logistic constraints, debriefing will have to be done through TEAMS or other online means.

KEY DELIVERABLES

The following are the evaluation outputs the evaluator/s will submit to the Mission focal person:

Outputs

The quality of the inception report and the evaluation report will be assessed, and the consultant will be requested to make necessary quality improvements. The evaluator will follow the format, structure and length defined in the template provided. All outputs must be submitted in English and under Word Document format.

MANAGEMENT ARRANGEMENTS

The evaluation TOR was developed in a participatory manner, by the Philippine Mission based on inputs from relevant stakeholders.

The evaluation TOR will be advertised (particularly in Relief Web, Action Against Hunger website and Facebook) and will be managed by Action Against Hunger – Philippines.

The evaluator will directly report to the Mission focal person for this evaluation. The evaluator will submit all the evaluation outputs directly and only to the Mission focal persons to ensure quality of the evaluation and decide whether the report is ready for sharing. The Mission focal persons will forward a copy to key stakeholders for comments on factual issues and for clarifications. The Mission focal persons will consolidate the comments and send these to the evaluator/s by date agreed between the Mission focal persons and the evaluator/s or as soon as the comments are received from stakeholders. The evaluator will consider all comments to finalize the report and will submit it to the Mission focal persons who will then officially forward to relevant stakeholders.

Once the evaluation is completed the Mission focal persons will prepare the management response follow-up form to track implementation of the recommendations outlined in the evaluation report.

TOTAL CONSULTANT WORKING DAYS: 45 days

Profile of the Evaluator/s

The evaluation can be carried out by an international or national evaluation consultant or team of consultants with the following profile:

• Extensive knowledge in multi-sectoral interventions in humanitarian settings.
• Significant field experience in the evaluation of emergency/early recovery interventions in conflict-affected areas.
• Relevant degree/equivalent experience related to the evaluation to be undertaken.
• Significant practical experience in coordination, design, implementation, monitoring, and evaluation of programs.
• Good communication skills and experience in workshop facilitation.
• Ability to write clear and useful reports (may be required to produce examples of previous work);
• Fluent in English, Tagalog, and Bisaya will be an advantage.
• Understanding of BHA requirements.
• Ability to manage the available time and resources and to work to tight deadlines.
• Independence from the parties involved.

Note: In case a team of consultants is proposed for the evaluation, Action Against Hunger will sign the contract with only one lead consultant and he/she will be responsible for the evaluation and contract-related matters.

LEGAL AND ETHICAL MATTERS

The ownership of the draft and final documentation belongs to the agency and the funding donor exclusively. The document, or publication related to it, will not be shared with anybody except Action Against Hunger before the delivery by Action Against Hunger of the final document to the donor. Action Against Hunger is to be the main addressee of the evaluation and its results might impact on both operational and technical strategies. Action Against Hunger is likely to share the results of the evaluation with the following groups:

• Donor(s)
• Implementation partners
• Governmental partners
• Civil society partners
• Various coordination bodies

For independent evaluations, the consultant mustn’t have any links to project management, or any other conflict of interest that would interfere with the independence of the evaluation.

Intellectual Property Rights

All documentation related to the Assignment (whether in the course of your duties) shall remain the sole and exclusive property of the Charity.