External Final Evaluation DS.0018 as attached ToR At International Organization for Migration

TITLE: COMMUNITY REVITALIZATION PROGRAM (CRP) – Phase XII
External Final Evaluation
Commissioned by: IOM IRAQ COUNTRY OFFICE
The International Organization for Migration (IOM) is part of the United Nations System as the leading inter-governmental organization promoting since 1951 humane and orderly migration for the benefit of all, with 174 member states and presence in over 100 countries. IOM works on migration and development, facilitating migration, regulating migration and solutions for forced migration. IOM activities that cut across these areas include the promotion of international migration law, policy debate and guidance, protection of migrants’ rights, migration health and the gender dimension of migration.
1. EVALUATION CONTEXT
1.1. Political, Environmental, and Socio-economic Context:
Since the official end of the conflict between Iraq and the Islamic State and the Levant (ISIL) in 2017, programming in Iraq has gradually transitioned from humanitarian to recovery and stabilization efforts. As of 31 December 2023, more than 4.8 million formerly displaced individuals returned to their areas of origin mostly through their own means, however, around 1.1 million Iraqis remain displaced, with 161,635 still residing in camps.
As a result, the country continues to face complex challenges in addressing the needs of the Internally Displaced Persons (IDPs), returnees, host community members and the wider population. In addition to the IDP caseload, the Government of Iraq (GoI) is facing mounting pressure from the international community to expedite the repatriation of all Iraqis citizens from North-East Syria (NES) and notably those in Al-Hol camp in light of both the deterioration of the security situation in NES as well of the dire living conditions inside Al-Hol.
1.2. Summary of the IOM Iraq CRP XII Project:
Donor
Project Title
Start date
End date
Locations
Total Budget
PRM (USA – Bureau of Population Refugees and Migration)
Community Revitalization Program – Phase XII
1-Oct-23
29-Sept-24
Dohuk, Erbil, Ninewa, Diyala, Salah al-Din, Anbar
34,997,946 USD
Considering the complexities around advancing durable solutions in Iraq, IOM proposed a multi-layered approach that combines the provision of durable solutions and pathways to return and reintegration while simultaneously addressing the underlying barriers to returns as well as supporting receiving communities in efforts to strengthen the social fabric. The project focused on two main groups, IDPs in a state of protracted displacement, as well as Iraqi nationals that have returned from NES.
Under this grant, IOM contributed to improving return, local integration, and settlement in new locations by:
1. Supporting access to basic needs for persons in Jeddah 1 Rehabilitation Center (J1) and improving the conditions to meet their basic needs, improve their resilience, and prepare for their sustainable (re-)integration. Planned
EVALUATION TERMS OF REFERENCE
2
activities will aim to facilitate access to basic services in Jeddah 1 and improving the capacity of returnees to address blockages that impede sustainable reintegration into their areas of return following departure. 2. Supporting IDPs and returnees from NES and improve their access to pathways to durable solutions through facilitated, safe, and voluntary return, local integration, or settlement in new locations.
3. Supporting the implementation of community-based interventions and multi-sectoral service provision in target areas.
4. Supporting government, local civil society, and the international community to increase their capacity to support pathways towards durable solutions and the process of reintegration of returnees from NES, through tailored technical support, strengthened knowledge and evidence, and coordination of responses.
2. EVALUATION PURPOSE AND OBJECTIVES
The evaluation is an IOM final evaluation to be conducted through an external firm or consultant in all the targeted governorates and locations. Lessons learned, best practices, and recommendations will be used at a strategic level to improve implementation, and service delivery for future interventions.
The evaluation specific objectives are to:
o Assess the overall project’s performance from planning, implementation and knowledge management by identifying the key strengths and areas of gaps and make the necessary recommendations for improvement.
o Document vital lessons-learned/best practices for future strategies and interventions.
o Support the use of relevant and timely contributions to organisational learning, informed decision-making processes resulting from the analysis, conclusions or recommendations as well as and accountability for results.
o Endorse IOM’s obligation on transparency and Accountability to the Affected Populations (AAP), donors and Iraqi government authorities as well as assess the effectiveness of IOM’s CFM effectiveness and the level of beneficiaries’ usage.
3. EVALUATION SCOPE
The scope should focus on all the outcomes of the project as below:
1. Persons in need in J1 have increased support to meet their basic needs, improve their resilience, and achieve their sustainable (re-) integration
2. IDPs and returnees from NES have improved access to pathways to durable solutions through facilitated, safe, and voluntary return, local integration, or settlement in new locations
3. Progress towards durable solutions and sustainable reintegration for displacement-affected people is enhanced through community-based interventions and multi-sectoral service provision in target areas
4. Government, local civil society, and the international community have increased capacity to support pathways towards durable solutions and the process of reintegration of returnees from NES, through tailored technical support, strengthened knowledge and evidence, and coordination of responses
In addition, the evaluation will also focus on IOM’s implementing partners.
EVALUATION TERMS OF REFERENCE
3
4. EVALUATION CRITERIA
Project’s performance should be evaluated against the evaluation OECD/DAC criteria of relevance, effectiveness, coherence, efficiency, sustainability and impact. In total, the evaluation criteria form the basis and guidance for the evaluation suggested questions as presented below.
5. EVALUATION GUIDING QUESTIONS
Relevance
a) How appropriate are the project’s intended results for the context within which it operates?
b) Is the intervention well-designed to address needs and priorities?
c) Was the project aligned with and supportive of national strategies?
d) To what extent have gender considerations been integrated into the design, planning and implementation of the project and the results achieved?
Effectiveness
e) To what extent did the intervention achieve its objective, including the timely delivery of assistance?
f) Were the target beneficiaries reached as expected?
g) Did the project successfully translate the resources (inputs) into tangible and quality outputs and outcomes in accordance with the stated plans?
h) What external socio-economic, political, or other factors impacted the project’s performance and achievements?
i) Were gender-disaggregated targets set and were gender-disaggregated indicators used?
Coherence
j) Do synergies exist with other interventions carried out by IOM as well as intervention partners?
k) To what extent is the intervention consistent with other actors’ interventions in the same context?
l) Is the project well-alined with IOM’s national, regional and global strategies and the Migration Governance Framework?
Efficiency
m) Were the project activities undertaken and the outputs delivered on time? If not, what were the reasons?
n) Are there any other alternative approaches that could have been more cost-effective in achieving the project’s objectives?
Impact
o) What significant change(s) did the intervention bring or is expected to bring, whether positive or negative, intended or unintended?
p) To what extent has the inclusion of gender issues led to better quality results? (outcome and impact)?
Sustainability
q) To what extent are the results achieved sustainable?
r) To what extent were stakeholders involved/consulted during the project?
s) What are the assumptions about gender roles, norms and relations that supported or hindered the project? And how will these factors affect the sustainability of the results?
EVALUATION TERMS OF REFERENCE
4
6. EVALUATION METHODOLOGY
The firm/ consultant is ultimately responsible for the development of the overall methodological approach of the evaluation and is expected to propose methodologies that the firm/ consultant considers most appropriate to achieve the aims of this evaluation.
Efforts shall be exerted to safeguard the inclusivity and engagement of relevant stakeholders to bring out their voices on how they perceived the implementation of the project; notably, the returnees, IDPs, host communities, the most vulnerable conflict-affected populations, civil societies, government counterparts, community and local authorities’ leaders. Key Informants Interviews (KIIs) with representatives from the community, government authorities, and other relevant actors (if any) should be held. Similarly, interviews, FGDs, and surveys with a selected sample from the Returnees, IDPs and host-communities or beneficiaries should be conducted to assess how the project has responded according to their expectations, objectives and priorities.
The evaluation processes shall be in line with IOM Data Protection Principles1, IOM code of conduct, Do no harm principles, UNEG norms and standards for evaluations2. The results of this evaluation will contribute to improving learning for future interventions.
In addition, the evaluation would also have to cover cross-cutting issues such as gender, accountability ot affected populations and inclusion of Persons with Disabilities.
The evaluation is expected to take place in-person and to travel to Iraq. The firm/ consultant may engage national consultants based in Iraq to support their work.
Summary of the Evaluation Methodology and Data Collection Tools
Table 01: Evaluation Methodology and Data Collection Tools
Method
Tasks
Tools required
Desk study
Review of proposal, project reports, work plans and other documents.
Key Informant Interviews
Identify and secure an interview with relevant KIs and implementing patern: programme staff, community and stakeholder’s representatives (IDPs, host-communities, governmental authorities, civil societies, and other actors).
Structured questionnaire
On-site observation
Conduct field visits, observation of project activities and sites
Observation Guides
Focus group discussion (FGD)
• FGDs to explore stakeholder opinions and judgements towards the engagement level, processes, and project implementation.
• In-Depth information on the needs, motivations, intentions, and experiences of the group to assess how the project has responded according to their expectations
FGD Guides
1 IOM Data Protection Manual, https://publications.iom.int/books/iom-data-protection-manual.
2 UNEG norms and standards for evaluation: http://www.uneval.org/document/guidance-documents.
EVALUATION TERMS OF REFERENCE
5
Beneficiary Interviews and surveys
Identify and secure interviews with Beneficiaries (sample for each activity should be selected based on a statistically relevant sample per the survey system, that is, a 95% confidence level and a confidence interval of 5)
Structured questionnaire
7. EVALUATION DELIVERABLES
The evaluator should produce:
1- First draft of inception report as per the IOM template including the tools/materials. A data collection plan as well as an evaluation matrix should also be submitted with the inception report: 14th October 2024 COB Baghdad Time
2- A final version of inception report: 31st October 2024 COB Baghdad Time
3- Weekly report on activities starting from the data collection: every Thursday
4- Routine meetings and discussions with the M&E Officer, Programme Manager/ project team, Project focal points : based on needs
5- First draft of the final evaluation report (based on IOM template) with a summary of the evaluation (2-pages as per IOM template) and an updated Results Framework (indicators). All data collected from the different sources must also be shared to IOM with the 1st draft of the final report : 07th December 2024 COB Baghdad Time
6- Meeting with IOM team to present the main findings, recommendations and challenges: 08th December 2024
7- Final version of the evaluation report, including a) updated Results Framework and b) Management Response Matrix: 21st December 2024 COB Baghdad Time
8. EVALUATION PROPOSED WORKPLAN
Table 02: The Evaluation proposed work plan
Activity
Responsible
Location
Start
Data collection, and reporting
October
November
December
1. Kick off meeting
IOM
Home-based
X
2. First draft of the inception report, tools and data collection plan
Evaluation firm
Home-based
X
3. Final version of the inception report
Evaluation firm
Home-based
X
4. Fieldwork – data collection
Evaluation firm
Iraq – Field site
X
5. First draft of evaluation report including raw data and Results Framework
Evaluation firm
Home-based
X
EVALUATION TERMS OF REFERENCE
6
9. EVALUATION BUDGET AND DISBURSEMENT
The payment terms shall be issued per the terms and condition of the Purchase Order (PO) based on the disbursement schedule below;
• Satisfactory inception report submission – 30%
• Submission of first draft of the evaluation report together with raw data – 30%
• Satisfactory final report with relevant annexes – 40%
The final payment shall be issued not less than 30 days upon (1) the completion of the work, (2) receive of the final original invoice and (3) receive of the final evaluation report and summary of the evaluation brief following the incorporation of feedback from the IOM ERG.
10. REQUIREMENTS
An international consultancy firm with valid registrations, and it should have legal registration paperwork in Iraq, and formal access to the locations of the programme implementation.
The selected firm/ consultant should possess the following minimum qualifications as follows:
Table 03: Qualifications and Experience
Qualifications and experience
Academic skills
Master’s degree in advanced applied research/evaluation methods/ economics, business, or any related academic discipline or an affiliation with a research institution or a university, holding a PhD, or being in pursuit of a PhD in a relevant field is an advantage.
Previous Experience
o 5 of years evaluating humanitarian programmes
o At least two evaluation contracts of similar value, nature and complexity implemented over the last three years or more.
o Strong background in monitoring and evaluation techniques and ideal experience in conflict-affected countries.
o Conversant with the context in Iraq, other countries in the Middle East or MENA region.
o Experience in developing and implementing Evaluations with the UN, International NGOs or donors.
o Familiarity with the OECD/DAC and UNEG evaluation framework.
o Excellent knowledge and experience in survey design, implementation of surveys and statistical data analysis.
o Excellent analytical, communication, writing and presentation skills in English.
o Ability to analyse complex intervention.
Accountability
o Creates a respectful office environment free of harassment and retaliation and promotes the prevention of sexual exploitation and abuse (PSEA).
6. Meeting to present main findings, recommendations
Evaluation firm
Home-based
X
7. Final version of the evaluation report including Management and Action Plan Matrix
Evaluation firm
Home based
X
EVALUATION TERMS OF REFERENCE
7
o
Accepts and gives constructive criticism.
o Follows all relevant procedures, processes, and policies related to the organisational principles.
o Meets deadline, cost, and quality requirements for outputs.
o Monitors own work to correct errors or incorporate inputs.
o Takes responsibility for meeting commitments and for any shortcomings.
Orientation
o Identifies the immediate and peripheral programme staff of own work.
o Establishes and maintains productive working relationships with staff.
o Identifies and monitors changes in the needs of evaluation, including donors, governments and project beneficiaries.
o Keeps staff/managers informed of developments and setbacks related to the evaluation.
11. SUBMISSION OF APPLICATION/EXPRESSION OF INTEREST
The interested firm/ consultant should submit a technical proposal with a detailed evaluation methodology, indicative work plan, and the overall approach to the evaluation and an all-inclusive budget proposal no later than 23rd June 2024. The submission of proposals (technical and financial) and/or related questions should be directed via procurement email to: IRAQTENDERS@iom.int
The submission should include the followings:
o Company/ Consultant profile including a history of similar projects (if applicable);
o A cover letter;
o CV and biographies of independent consultant/consulting firm and key assessment team members (if any);
o References for each evaluation team member or the firm;
o An example of a recent evaluation report.
Important Note: When evaluating the competing applicants, IOM will consider the written qualifications/capability, financial offer, the information provided by the applicants, and any other information obtained by IOM through its research.
IOM reserves the right to change the calendar of events or revise any parts of the requirements of the evaluation at any time.