Consultancy service for Project Final Evaluation At International Organization for Migration

• Background

Afghanistan is going through a severe humanitarian crisis, which poses a high risk of collapse and catastrophe. The end of the 20-year armed conflict between the Taliban and the Afghan National Security and Defence Forces in August 2021, followed by the Taliban’s takeover of Afghanistan, has brought about a series of significant socio-economic challenges. The country is currently grappling with a rapid economic decline, widespread hunger, inflation driven by global commodity shocks, rising poverty rates, a fragile public health system, restricted media and civil society sectors, and the exclusion of women and girls from public life. Additionally, this transition has had a severe impact on Afghanistan’s economy, leading to instability and limited opportunities for its people. The economic downturn, combined with the disruption of livelihoods, has resulted in many households falling into poverty, widening existing inequalities and socio-economic gaps. Durable solutions remain elusive for the majority of those remaining in displacement, due to a complex intersection of conflict and climate drivers, often exacerbated by economic, social and livelihood factors. Those in informal settlements in particular face insecure land tenure, and in some areas, access to land, housing and resolution of secondary occupation is a key barrier to return or integration elsewhere. An estimated 56% of new displacement in 2021 and 2022, and 95% in the second half of 2022, was due not to conflict but to an inability to tolerate the effects of disaster. 92% of IDPs displaced due to disaster between 2012 and 2022 were displaced due to drought and the dire implications of drought on agriculture and livestock production, which 45% of those displaced in 2021 and 2022 relied upon. Data from 2022 indicates that only 6% of IDPs displaced in the six months prior to data collection intend to return to their areas of origin in the following six months, but 20% of these IDPs would prefer to settle in their areas of origin if conditions were adequate. Key enablers of return are linked to livelihood opportunities (39%), financial means for transport (27%), rehabilitation or reconstruction of the home (20%) and increased security (7%).

According to the 2023 Humanitarian Needs Overview (HNO)5, around 28.3 million people (about two-thirds of the population) require humanitarian assistance to survive. The restrictions placed on Afghan women are particularly severe, as they are completely denied access to secondary and tertiary education and are prohibited from participating in humanitarian aid efforts through NGOs and the UN. In 2022, more than 924,000 Afghan nationals returned from neighboring countries, mainly the Islamic Republic of Iran and Pakistan, through reception centers managed by IOM. Many migrants, often using unofficial border crossings, were detained by increasing numbers of border patrols and pushed back by Iranian officials. These trends are expected to continue in 2023, and it is estimated that over a million undocumented returnees will require humanitarian assistance. Additionally, around 6.6 million individuals who have been displaced since 2012 remain in displacement across Afghanistan.

To address these pressing issues, the International Organization for Migration (IOM) has launched a 12-month project (1 August 2023 – 31 July 2024) with the objective of offering comprehensive and data-driven assistance to vulnerable migrants in Afghanistan. The project will be implemented in ten (10) provinces, where IOM has Protection caseworkers (Badakhshan, Balkh, Faryab, Herat, Kabul, Kandahar, Nangarhar, Nimroz, Sar-e-Pul, and Takhar).

• Project Context and Scope***:***

The project aims to target the humanitarian, protection, and health needs of undocumented returnees at borders while identifying the long-term requirements of returnees and IDPs in Badakhshan, Balkh, Faryab, Herat, Kabul, Kandahar, Nangarhar, Nimroz, Sar-e-Pul, and Takhar provinces. IOM’s strategy includes operating at reception and transit centers in border regions and provinces with high return rates, screening returning migrants for immediate assistance and protection concerns, and providing health services through Basic Health Centers. The project also aims to enhance coordination of durable solutions, support Camp Coordination and Camp Management (CCCM) Mobile Teams and address the consequences of recent displacements to informal settlements. IOM’s Displacement Tracking Matrix (DTM) Program will provide data-driven insights into mobility dynamics, migrant populations, multi-sectoral needs, and migration drivers, enabling evidence-based interventions.

• Overall Objective:

Is to ensure the continuity of a comprehensive and data-driven response for vulnerable migrants in Afghanistan by addressing the humanitarian, protection and health needs of vulnerable undocumented returnees at the borders and the longer-term needs of returnees and IDPs in key provinces of high return.

• Outcome 1: Humanitarian needs of vulnerable migrants returning from Iran and Pakistan and protection concerns of these migrants and internally displaced populations (IDPs) are addressed through operations at the reception centers on the border with Iran in Herat and Nimroz and with Pakistan in Kandahar and Nangahar and their respective transit centers, and in ten provinces of high return.
• Outcome 2: Health needs of vulnerable Afghan returnees, displaced populations, and host communities in the border areas with Iran have been addressed and health education and awareness promoted through the operation of Basic Health Centers (BHCs) in Herat and Nimroz transit centers.
• Outcome 3: Durable Solutions action is enhanced through improved coordination at the regional and national levels, and informal settlements are supported through CCCM, with an IDP-centred approach and durable solutions plans.
• Outcome 4: Humanitarian and development partners and counterparts use DTM data and its analysis to inform and deliver targeted, timely and mobility-sensitive assistance to return, displaced, mobile and cross-border populations and host communities.
• Scope of Work and geographical coverage

The consultant will support the Afghanistan mission, specifically the “Ensuring a comprehensive and data-driven response for vulnerable migrants in Afghanistan through addressing the humanitarian, protection and health needs of undocumented returnees at the borders and longer-term needs of returnees and IDPs in key provinces of high return,” to conduct the final project evaluation at the end of the project from 1 August 2023 – 31 July 2024. The evaluation will be conducted in the 10 project provinces (Badakshan, Baikh, Faryab, Herat, Kabul, Kandahar, Nangarhar, Nimroz, Sar-e-Pul, and Takhar) in Afghanistan. The consultant will also support post distribution monitoring.

• Target Population

This evaluation will primarily focus on undocumented Afghan migrants residing in the Islamic Republic of Iran and Pakistan. This will include women, men (18-59), children (boys and girls under 5 and 5-17), and individuals with specific needs:

• Chronically ill
• Individuals with substance abuse concerns
• Families forcibly returned (pushed back)
• Individuals with medical conditions
• Individuals with mental health concerns
• Individuals with physical disabilities
• Individuals from low-income families
• Single women
• Single parents
• Individuals facing unique circumstances (special cases)
• Unaccompanied elderly individuals
• Unaccompanied and separated children

This is not an exhaustive list, other stakeholders will be mapped and included in the evaluations.

• Organizational Department:

Programme Support Unit (under the direct supervision of the Monitoring and Evaluation Officer and overall Supervision of the Head of Programme Support Unit).

Name of the project: Ensuring a comprehensive and data-driven response for vulnerable migrants in Afghanistan through addressing the humanitarian, protection and health needs of undocumented returnees at the borders and longer-term needs of returnees and IDPs in key provinces of high return.

Tasks to be performed under this contract

Purpose and Objective of the Consultancy

The consultant’s primary responsibility is to conduct a comprehensive evaluation of the project. This evaluation will critically analyze the project’s outcomes. By employing a systematic approach to data collection and analysis, the consultancy will ensure the evaluation is aligned with the project’s results framework, encompassing all relevant indicators as indicated in annex A. The Endline Evaluationwill assess the cumulative effects of the project’s interventions, analyzing both completed and ongoing activities. The evaluation will leverage the OECD DAC criteria to thoroughly examine the project’s effectiveness, efficiency, impact, relevance, and sustainability. It aims to provide a comprehensive review of the project’s achievements against its objectives, offering viable recommendations for future initiatives, and identifying lessons learned throughout the project lifecycle.

Objectives of the Endline Evaluation

The endline evaluation will be conducted at the end of the project from January-March 2025. This will assess the changes resulting from the project interventions.

• Measure changes and assess the extent of project achievement against initial benchmarks using the OECD Criteria
• Evaluate the project’s impact on the well-being and protection of returnees and IDPs, with a focus on vulnerable groups such as women, children, elderly individuals, and persons with disabilities.
• Determine whether the project has achieved its intended goals and objectives, including the provision of sustainable support to returnees and IDPs in line with the project’s design.
• Document the positive changes, stories of transformation, Impact of AAP and the project’s overall contribution to returnees and IDPs’ lives.
• Identify and assess key lessons learned, challenges and draw recommendations for future programming.
• Provide an overall analysis and recommendations for enhanced future intervention implementation while also assessing cross-cutting themes, particularly gender equality and the sustainability of interventions.

Methodology

The selected consultants will employ a mixed methods approach for the endline evaluations, integrating quantitative and qualitative research techniques. This approach is intended to establish a comprehensive condition within the target provinces at the project’s inception, detailing immediate needs, service availability, and initial benchmarks. This foundation will enable the effective tracking of project progress and the assessment of its impact over time.

The evaluation at the endline will rigorously examine the cumulative effects of the project’s interventions, analyzing both completed and ongoing activities. Utilizing the OECD DAC criteria, the evaluation will critically assess the project’s achievements in relation to its objectives, offer actionable recommendations for future efforts, and identify lessons learned throughout the project’s lifecycle.

Quantitative Methods

• Sample Size Determination: The consultants will determine sample sizes in collaboration with IOM to ensure gender-sensitive and statistically representative sampling of the project’s beneficiaries and stakeholders.
• Data Analysis: The consultant will develop a structured surveys and questionnaires that will be used to collect numerical data on project indicators, facilitating the quantitative assessment of changes and the measurement of progress against the project’s goals.

Qualitative Methods

• In-depth Insights: The consultant will develop a Structured interviews and focus group discussions that will be utilized to gather qualitative data, providing insights into the experiences, perceptions, and feedback of project beneficiaries and stakeholders. This will enrich the evaluation with nuanced understanding of the project’s impact.
• Vulnerable Groups: A special focus will be placed on including vulnerable populations, such as women, children, the elderly, and individuals with disabilities, ensuring their experiences and needs are thoroughly represented and addressed.

Data Sources and Collection

• Primary Data Collection: Direct engagement with beneficiaries and stakeholders through surveys, interviews, and focus groups will be crucial for capturing the immediate and broader impacts of the project.
• Secondary Data Review: A comprehensive desk review of project-related documents (e.g., project proposals, log frames, PDM reports, beneficiary surveys) and relevant literature will provide contextual background for the evaluation.

The evaluation will leverage existing data to conduct a comprehensive analysis that incorporates both historical and current perspectives on the project’s implementation and outcomes. This analysis will consider data from international, national, and local sources:

• International Databases: Information from organizations like UNHCR, WHO, and IOM will provide broader insights into migration patterns, health indicators, and protection issues.
• Community-Level Data: Data collected by local organizations and community structures will provide valuable insights into immediate needs, vulnerabilities, and coping mechanisms.

Credibility testing and quality assurance will be carried out on the international, local, and national datasets sourced from reputable government agencies, community organizations, and international datasets to further enhance credibility.

Data Analysis and Reporting

• Triangulation: The integration of quantitative and qualitative data through triangulation will ensure the reliability and validity of findings, allowing for a robust interpretation of the project’s outcomes.
• Disaggregation of Data: All collected data will be disaggregated by age and gender, offering detailed insights into the project’s differential impacts across demographic groups.
• Gender-Aware Analysis: The evaluation will specifically focus on gender, examining how the project has addressed gender-specific needs and contributed to gender equality among beneficiaries.

Collaboration and Participation

The methodology should emphasize collaboration with IOM and engagement with stakeholders at all levels, aiming to make the evaluation process inclusive, participatory, and reflective of a broad range of perspectives and experiences.

Technical Proposal Expectations

Applicants are required to detail their proposed methodology in their technical proposal, elucidating why the chosen approach is most suitable and how it aligns with the objectives and scope of the evaluation. The proposal should also outline the logistical aspects of the study, including the identification and recruitment of target respondents, the execution of data collection and assurance of data quality, and the approach to data analysis.

Endline Evaluation Guiding Questions

Endline Evaluation Guiding Questions

1

Relevance and Adaptation

• How have the evolving humanitarian, protection, and health needs of the target population been addressed by the project’s objectives and interventions? To what degree has the project adapted to demographic changes and emerging needs, particularly considering the shift from NFIs to cash-based support?
• Evaluate the project’s responsiveness to the dynamic context and needs of vulnerable Afghan migrants, returnees, and IDPs. How has the project’s approach to inclusion and diversity evolved to ensure relevance?

2

Effectiveness and Efficiency

• What tangible evidence indicates the project’s success in improving the well-being, safety, and protection of undocumented returnees and IDPs? Assess the efficiency of resource utilization, including the cost-effectiveness of cash-based interventions versus traditional support mechanisms.
• Analyze the contribution of health services, protection case management, and the joint screening processes towards achieving the project’s goals. How have these services impacted the lives of beneficiaries, and what improvements are suggested for future initiatives?

3

Impact and Transformative Change

• Identify and detail the sustained positive changes observed in the safety, dignity, and overall well-being of the target groups as a direct result of project interventions. How have these changes contributed to long-term improvements in the lives of beneficiaries?
• Evaluate the project’s impact on reducing vulnerabilities and protection concerns, particularly regarding violence, exploitation, and access to legal documentation. What evidence supports these outcomes, and how can they inform future projects?

4

Sustainability and Legacy

• Assess the degree to which the project’s capacity-building efforts have led to the institutionalization of services and mechanisms for continued support. How sustainable are the established protection mechanisms, health services, and coordination structures without external project support?
• Discuss the project’s success in fostering local ownership, community engagement, and the integration of interventions into local systems. What strategies have been most effective in ensuring the sustainability of benefits and services for returnees, IDPs, and host communities?

5

Learning and Recommendations for Future Programming

• What are the key lessons learned from the project regarding best practices, challenges, and areas for improvement, especially in the context of shifting from NFIs to cash-based interventions?
• Based on the endline findings, provide specific, actionable recommendations to improve the design, implementation, and outcomes of future interventions targeting similar populations. How can these recommendations address the distinct needs and contexts of vulnerable Afghan migrants, returnees, and IDPs?
• How can future projects better incorporate gender dynamics and the specific needs of women and girls, ensuring gender equality and empowerment are central to project design and implementation?

6

Cross-Cutting Themes and Innovation

• Examine how the project addressed cross-cutting themes such as gender equality, environmental sustainability, and social inclusion. What innovative approaches were adopted, and how effective were they in enhancing project outcomes?
• Provide insights into how future interventions can incorporate cross-cutting themes more effectively, leveraging lessons learned from this project’s implementation and outcomes.

Ethical Considerations

The consultant is expected to adhere to strict ethical guidelines, ensuring confidentiality, informed consent, and the protection of participants’ rights throughout the data collection and analysis processes. The consultant is required to respect the following Ethical Principles:

• Comprehensive and systematic inquiry: The consultant should make the most of the existing information and the full range of stakeholders available. The consultant should conduct systematic, data-based inquiries.
• He or she should communicate his or her methods and approaches accurately and in sufficient detail to allow others to understand, interpret, and critique his or her work. He or she should make clear the limitations of the evaluation and its results.
• Competence: The consultant should possess the abilities, skills, and experience appropriate to undertake the tasks proposed and should practice within the limits of his or her professional training and competence.
• Honesty and integrity: The consultant should be transparent with any conflict of interest, any change made in the negotiated project plan and the reasons why those changes were made, and any risk that certain procedures or activities produce misleading review information.
• Respect for people: The consultants should respect the security, dignity, and self-worth of respondents and program participants. The consultant has the responsibility to be sensitive to and respect differences among participants in culture, religion, gender, disability, age, and ethnicity.
• Adherence to IOM Ethical guidance: Intending bidders must prioritize adherence to IOM ‘s ethical guidance for research, evaluation and monitoring activities throughout the evaluation process.6The guidance, meticulously structured around the project cycle, mandates a comprehensive consideration of ethical issues at every stage, ensuring unwavering adherence to high standards and transparency. Furthermore, it is imperative for bidders to recognize the significance of incorporating digital tools in the evaluations in line with IOM standards.

Potential Risks and Challenges for Evaluation Implementation

Risk: Security and Access Constraints

• Contextual Description: The project areas in Afghanistan, especially border regions, may pose security challenges, limiting access to certain locations for data collection. Security risks could impact the safety of the evaluation team and the ability to gather accurate information.

Risk: Data Collection Limitations

• Contextual Description: In some regions, logistical issues, cultural sensitivities, or the reluctance of respondents might impede the effectiveness of data collection. The limited availability of reliable secondary data could also affect the completeness of the evaluation.

These identified risks and challenges, both contextual and methodological, should be thoroughly addressed by suppliers in their proposals. By providing detailed mitigation strategies, suppliers can demonstrate their understanding of the potential issues and their commitment to ensuring a robust and successful evaluation process.

• Deliverables and Timelines

The consultant is expected to produce the following deliverables within the specified timeline.

• Inception Report: Provide an inception report mapping stakeholders and outlining the evaluation approach, how quality will be assured, work plan, data collection methodologies, and tools.
• There might be a need to get formal approval from an Ethical or Institutional Review Board depending on the situation on ground.
• Draft Report: Submit a draft report including data, analysis, and recommendations.
• Final Report: Incorporate feedback from the validation workshop and submit the final report within the specified days after the validation workshop.
• Database and Tools: Develop a database and provide all data collection tools used during the evaluation.
• The consultant to produce a 2-page summary (“evaluation digest”) of the finalized evaluation report

Activity — Responsible Party — No of Days

Meetings: M&E team, project manager/staff/focal points, management. — IOM — 1 Day
Desk review of project documents — Consultant — 5 days
Inception process: Development/submission of inception report/work plan, tools for the FGD, survey, and KIIs. — Consultant — 10 days
Fieldwork – data collection — Consultant — 15 days
Data Analysis — Consultant — 10 days
Report writing and submission of the first draft (in IOM template — Consultant — 14 days
Incorporate feedback and submit a final report with a two-pager evaluation brief (as per IOM template) — Consultant/ IOM — 7 days
Final report presentation — Consultant — 2 days
Total No of Days — 64 days

Payment Installment

The consultancy Payment will be based on the below breakdown

• Submission & Approval of inception report (20%)
• Submission of the first draft report after data collection and conducting evaluation debriefing (40%)
• Final report submission and approval (40%)

  Core Functions / Responsibilities:

Expected Deliverables and Reporting Requirements:

• Support the development/modification of existing data collection tools to be fit for purpose.
• Ensure appropriate sampling methodology is applied.
• Lead the training of the field data collectors and closely monitor them during field data collection to ensure adherence to the agreed protocol and methodology.
• Coordinate field data exercise and ensure data collectors work within the stipulated time frame and all data collected using the mobile application.
• Assure data quality through supervision of teams and review of database.
• Submit draft report using an agreed reporting template.
• Present findings to key internal and external stakeholders.

Proposal Outlining

• Technical Proposal including the following:
• A proposed methodology
• Evidence of ability to deliver in Afghanistan in a flexible manner.
• CVs of Lead Consultant/firm and other Team members
• Proposed work plan
• Financial Proposal
• A sample of similar study conducted
• Performance indicators for the evaluation of results:

Timeline, accuracy, and reports as per the Term of Reference.

• Required Qualifications and Experience and/or skills required:

Education:

• Master’s degree in social science, Statistics, Demography or another relevant field.

or

• University degree in the above fields with seven years of relevant professional experience.

Experience:

• At least seven years of professional experience in monitoring and evaluation.
• Experience of carrying out mixed-methods evaluations and producing high quality analytical reports (examples of previous work will be requested);
• Statistical analysis skills and strong proficiency with data analysis packages
• Excellent communication and written skills in English and knowledge of local languages and the ability to present data concisely and clearly.

Skills:

• Excellent computer skills (MS Office, particularly Word and Excel, web-based database development tools, etc.).
• Good analytical, negotiating and communication skills.
• Content Management and attention to detail.
• Interpersonal and interview skills.

Languages:

• IOM’s official languages are English, French, and Spanish.
• For this position, fluency in English is required (oral and written). Working knowledge of other local languages is an advantage.

Travel required:

Yes.

Competencies:

The incumbent is expected to demonstrate the following values and competencies:

Values

• Inclusion and respect for diversity: respects and promotes individual and cultural differences; encourages diversity and inclusion wherever possible.
• Integrity and transparency: maintains high ethical standards and acts in a manner consistent with organizational principles/rules and standards of conduct.
• Professionalism: demonstrates ability to work in a composed, competent and committed manner and exercises careful judgment in meeting day-to-day challenges.

Core Competencies – behavioral indicators

• Teamwork: develops and promotes effective collaboration within and across units to achieve shared goals and optimize results.
• Delivering results: produces and delivers quality results in a service-oriented and timely manner; is action-oriented and committed to achieving agreed outcomes.
• Managing and sharing knowledge: continuously seeks to learn, share knowledge and innovate.
• Accountability: takes ownership of achieving the Organization’s priorities and assumes responsibility for own action and delegated work.
• Communication: encourages and contributes to clear and open communication; explains complex matters in an informative, inspiring, and motivational way.