Consultant (Final External Evaluator) At International Organization for Migration

Vacancy Notice Number: CON#/2023/014

Position Title: Consultant (Final External Evaluator)

Classification: Consultancy Contract

Duty Station: Türkiye

Deadline of Applications:10 December 2023

Number of People to be hired: 1

Eligibility: Internal & External Candidates

General Functions:

The International Organization for Migration (IOM), the United Nations (UN) Migration Agency, was established in 1951 and is the leading inter-governmental organization in the field of migration, working closely with governmental, intergovernmental, and non-governmental partners. With 175 Member States, eight states holding observer status, and over 18,000 staff working in over 150 countries worldwide. IOM is dedicated to promoting humane and orderly migration for the benefit of all.

The IOM established its operations in Türkiye in 1991. IOM’s partnership with the Government of Türkiye (GoT) was formalized in November 2004, when Türkiye became an IOM Member State. IOM Türkiye now has eight offices and nearly 1,359 staff, the largest UN agency in the country. IOM Türkiye closely works with the government of the Republic of Türkiye, regional authorities, the UN, donors, and civil society organizations to address migration challenges in Türkiye by implementing programmes through three pillars: Resilience, Mobility, and Governance. Across the country, IOM Türkiye provides a comprehensive response to the humanitarian needs of migrants, internally displaced persons, returnees, and host communities through direct humanitarian assistance, recreational activities, and various other efforts. Alongside IOM’s role in addressing the needs of migrants during crises, the mission works in close collaboration with the Government of Türkiye to address the longer-term impact of migration, including migrant assistance programmes, labour integration and migration management, immigration and border management and research and data collection on migrant movement.

In order to fulfill the tasks and obligations of the European Union (EU) Customs Administration, an important goal for Türkiye is to ensure proper protection of the external borders of the EU through a modernized Ministry of Trade (MoT). Being tasked by the government to lead the reforms associated with the alignment of customs enforcement policies and procedures with EU standards, the MoT has aimed to improve its capacity further by allocating adequate budget resources to investments related to customs control equipment. This has reinforced its bilateral and multi-lateral relations with the key countries advanced in customs control techniques to ensure information exchange and experience sharing while also managing EU-funded projects under IPA and participating actively in the Customs 2020 Programme since 2003.

Although significant progress has been achieved at sea and land Border Crossing Points (BCP) supported by national and EU projects implemented by the MoT, no comprehensive assessment of the customs enforcement capacity at airports has ever been conducted. The need for such an assessment is justified by the increasing number of international passengers transiting through İstanbul airports, arriving in Türkiye for leisure or business, as well as an increasing portion of Turkish citizens preferring air travel over other modes of transport in recent years. For instance, according to statistics available on the Directorate General of State Airports Authority (SAA) website, the number of international passengers in Turkish airports increased from 73 million in 2013 to 83 million in 2017, thus constituting a 14% increase. As for İstanbul Atatürk Airport alone, the number of international passengers reached 44.4 million in 2017, indicating the growing workload of MoT in the field of Immigration and Border Governance (IBG).

Considering the increased passenger flows and cargo volumes, as well as the increasing number of flights to and from the Atatürk, İstanbul Grand Airport, Antalya and Esenboğa (Ankara) airports – these sites were selected as a pilot for the activities foreseen under this project. The project will focus on the needs associated with strengthening and standardizing customs control procedures and those associated with the need to upgrade border controls and border surveillance.

The EU-funded Improvement of Customs Enforcement Capacity project aims to contribute to Türkiye’s capacity to improve border security and surveillance through the modernization of border surveillance systems and enhanced inter-agency collaboration in line with the European Union’s integrated border management policies and strategies. This is being achieved by strengthening the capacity of the MoT to enhance its customs control and surveillance operations in line with the best EU standards and practices as defined by the Union Customs Code (UCC), as well as the associated legal framework on facilitation of legitimate trade. The action primarily focuses on operations in three selected pilot airports (Istanbul, Antalya, and Ankara.)

Overall objective: To improve border security and surveillance in line with the EU’s integrated border management policies and strategies.

Specific objective: To support the strengthening of customs surveillance and control functions of the MoT in order to deal with both intended and committed offences at airports more efficiently.

Result 1: Passenger controls at the pilot airports are enhanced.

Result 2: Cargo controls at the pilot airports are improved.

Result 3: Airport crime at the pilot airports is reduced.

Result 4: Aircraft search at the pilot airports is strengthened.

Result 5: Capacity of MoT Dog Training Center is increased.

The initial project implementation period was from 01 February 2019 to 31 July 2021. The project received no-cost extensions twice, until 30 November 2022 and 30 November 2023. The evaluation will assess the entire implementation period of the project from 01 February 2019 to 30 November 2023.

Evaluation purpose and objective

The final evaluation aims to comprehensively assess the EU-funded project “Improvement of Customs Enforcement Capacity.” The objective is to assess the attained outcomes, identifying the strengths and weaknesses within the project design, management, and implementation. This evaluation encompasses an assessment of the project’s success in realizing its intended results and delves into the impact and sustainability of the project’s activities. This evaluation will utilize the OECD-DAC evaluation criteria -relevance, coherence, efficiency, effectiveness, impact, and sustainability. The evaluation serves the purpose of informing strategic decisions and future programming by identifying lessons learned, promoting the adoption of good practices, and identifying stakeholder satisfaction with project activities and results.

Evaluation scope

The final evaluation aims to comprehensively assess the project’s performance from the period of February 01, 2019, to November 30, 2023. The geographic coverage of the evaluation will align with the implementation of the project, encompassing the stakeholders from the headquarters of the project beneficiary institutions in Ankara, Istanbul, and Antalya. Through a participatory approach, the evaluator will actively engage relevant stakeholders, including IOM Türkiye staff, MoT focal points in Türkiye, and beneficiaries from different activities involving Ministry of Trade (MoT), Ministry of Interior (MoI), Ministry of Transport and Infrastructure (MoTI) and the private sector.

Furthermore, the evaluation will also provide recommendations, elucidate good practices, and lessons learned, contributing valuable insights for future or similar programs. An important aspect of the evaluation involves analyzing the integration of IOM cross-cutting themes, particularly gender considerations and human rights approach, within project activities and implementation.

The project is in line with the EU’s integrated border management policies and strategies, the EU Customs Blueprints document, Accession Partnership and National Programme for the Adoption of the Acquis’ priorities, Chapter 24 on Justice, Freedom and Security of the EU’s Acquis Communautaire. The evaluator will assess the project’s adherence with the objectives outlined in these documents, gauging its overall success.

Evaluation criteria

The evaluation will use the six OECD-DAC criteria encompassing relevance, coherence, efficiency, effectiveness, impact, and sustainability, as well as cross-cutting issues of gender and human rights.

Evaluation questions

In the inception phase, the evaluator will further develop and refine the evaluation questions, aiming for a detailed and specific information for each criterion. The evaluation matrix will be collaboratively reviewed with IOM.

The following key questions will guide the project evaluation:

Relevance

1. Do the project’s intended results align and support the Government of Türkiye’s strategies and priorities related to migration and border management?
2. How well was the project aligned with and supportive of IOM national, regional, and global strategies, including the IOM Immigration and Border Governance (IBG) Guidance, Community Engagement & Policing (CEP) Document and Migration Governance Framework?
3. Did the project respond to the needs and priorities of MoT, all stakeholders, and other key groups, as identified at the design stage?

Coherence

1. How did the project seek synergies and complement other related initiatives, whether implemented by IOM, the government or other national and international actors? What is the added value, if any, of this project compared to those other efforts in migration management?
2. To what extent were the project design, delivery and results coherent with improving border security and surveillance in line with the EU’s integrated border management policies and strategies?

Effectiveness

1. Did the project achieve the objectives and results at various levels of the results chain? If not, why not?
2. What were the major internal and external factors influencing the achievement of the project’s expected outcomes? Did existing practices effectively adapt to mitigate the impacts of the COVID-19 pandemic and the 2023 Türkiye -Syria earthquakes?
3. How did the intervention address the gaps in the capacity of the different stakeholders?
4. Were there differential results for different people? If so, how and why?

Efficiency

1. Were the activities implemented per the project’s work plan? Was the resource allocation appropriate considering the goal of the project?
2. What measures have been taken to ensure that resources are efficiently used?
3. What were the reasons for possible delays in project implementation? How was the project affected by delays caused by external and internal factors, and how did the project respond to and address delays? Did the project have the necessary coordination mechanisms and communication flow to efficiently convert the allocated resources into the expected outputs?

Impact

1. To what extent did the project contribute to improving border security and surveillance in line with the EU’s integrated border management policies and strategies?
2. To what extent did the project contribute to the positive, negative, intended, and unintended changes? Did the project take timely measures to mitigate unplanned negative impacts? What are the direct and indirect impacts of the project at the outcome level?

Sustainability

1. How well was the project embedded in institutional structures that were likely to survive beyond the life of the project? Were the government institutions well-integrated?
2. What were the significant challenges affecting sustainability? How should the project address these challenges to increase sustainability, considering design, implementation, and monitoring?
3. Are the benefits generated likely to continue once external support ceases? If so, how?

Cross-cutting themes:

1. To what extent does the project design reflect the rights and participation of all genders and include feedback from a diverse range of stakeholders?
2. Will the achievements in gender quality persist after the end of the project? Have processes contributed to sustaining these benefits?
3. To what extent has the project helped to ensure respect for the relevant human rights of migrants?

Evaluation methodology

The evaluation will employ a comprehensive approach, utilizing both qualitative and quantitative methods to collect and analyse data from various sources. This rigorous approach aims to establish robust and reliable evidence for assessing the project’s results, including outputs, outcomes, and objectives. The data will be cross-validated and triangulated using multiple methods to enhance the accuracy of the data. For effective evaluation, the findings and data will be disaggregated based on gender, age, geographic location, and type of activity.

The evaluation will involve the analysis of data from various sources, including desk reviews, surveys, and interviews with beneficiaries, as well as key stakeholders such as the Ministry of Trade (MoT), Ministry of Interior (MoI), Ministry of Transport and Infrastructure (MoTI) and relevant staff from IOM Türkiye.

The evaluation will use a mixed methods approach, combining both quantitative and qualitative data through the following methods:

• Desk review: The evaluator will conduct a systematic and detailed examination of various documents, including the project proposal, results matrix, work plans, donor/progress reports, monitoring reports, data, and budgetary documents.
• Key informant interviews and focus group discussions: Primary data will be gathered through interviews and focus group discussions with staff from IOM. The interaction will be based on semi-structured questions to gather their perspectives and deeper insight into the project.
• Surveys and in-depth semi-structured interviews: The structured surveys and semi-structured interviews will be conducted with the project stakeholders who participated in project activities.

The evaluator will prepare the tools, and IOM Türkiye will approve them before data collection. The evaluation approach and data collection and analysis methods will consider gender and rights-based approaches, ensuring a comprehensive and inclusive evaluation process.

Ethics, norms, and standards for evaluation

The evaluator must follow the IOM Data Protection Principles, UNEG norms and standards for evaluations, and relevant ethical conduct guidelines while carrying out the evaluation.

Evaluation deliverables

The external evaluator is expected to deliver the following:

• Inception report

The evaluator will prepare and share an Inception Report with the MEAL and project team. It will demonstrate the evaluator’s understanding of the assignment, methodology, and approach based on a review of ToR and initial discussions. The report will include an evaluation matrix, methodology, work plan, and data collection tools (focus group discussion guides, survey questionnaires, interview guides, and other data collection instruments for the evaluation). The inception report will include the data collection schedule and areas of the assignment that require IOM’s support. The evaluation matrix will outline data collection and analysis plans for each evaluation question posed in the ToR. The IOM team will review and approve the Inception Report and Evaluation Matrix before the data collection phase. IOM can provide templates for an Inception Report and an Evaluation Matrix.

• Presentation of the initial findings

After data collection, the evaluator will prepare a short MS PowerPoint presentation of the initial findings, conclusions, recommendations, and lessons learned. This will debrief the IOM team, address any misinterpretations or gaps, and provide feedback.

• A draft evaluation report

A draft evaluation report will be submitted to the IOM team, incorporating the debrief and initial feedback. The IOM team will review the report and provide feedback on the technical aspects. The evaluator will only make factual corrections to the report.

• Final evaluation report

Once finalized, the evaluator will submit a final evaluation report to the IOM team. IOM will provide a template for an evaluation report. The report should include an executive summary, a list of acronyms, an introduction, evaluation context and purpose, evaluation framework and methodology, findings, conclusions, and recommendations. Annexes should include the TOR, inception report, documents reviewed, persons interviewed or consulted, and data collection instruments.

• Evaluation brief (two-page summary according to the IOM template)

The evaluator will prepare a two-page evaluation brief using the IOM template to share the key findings, conclusions, and recommendations.

• Management response matrix (IOM will provide the template)

The evaluator will draft a management follow-up response matrix using the IOM template, including recommendations and an indicative timeframe or implementation. The IOM team will finalize this matrix.

• The final presentation of the evaluation report (online briefing with a PowerPoint for IOM staff, the donor, and key stakeholders to be identified and agreed upon)

Upon completion of the report, the evaluator will present the findings in an online meeting.

• Final evaluation report

The evaluator will submit the final evaluation report per the IOM rules. All deliverables are to be written in English and meet professional language standards. The reports should meet the standards in the UNEG Quality Checklist for Evaluation Reports and the IOM Quality Control Tool for evaluation reports (tools will be provided to be used by the evaluator). Also, the IOM House Style Manual and the IOM Publication Layout Manual should be applied to the report.

Once the deliverables receive final approval, the IOM Türkiye MEAL team will take the necessary steps to make the final evaluation report accessible to all relevant stakeholders.

Specifications of roles

The roles of the IOM project team, evaluator, and stakeholders are identified below:

The Evaluator:

• Carry out the evaluation per the Terms of Reference (ToR),

• Implement data collection, analysis, and reporting tasks for the final evaluation,

• Conduct bilateral meetings with IOM to discuss progress, challenges, and emerging issues,

• Provide regular updates to the IOM team about the progress of the evaluation,

• Prepare and deliver the inception report, final evaluation report, two-page evaluation brief, the visual presentation of the evaluation findings and recommendations and a draft management follow-up response matrix.

The MEAL Unit of IOM:

• Initiate a kick-off meeting with the evaluator and establish a work plan,

• Provide the evaluator with general information and documents related to the project monitoring,

• Review the deliverables submitted by the evaluator,

• Coordinate the review of the evaluation deliverables with relevant individuals, IOM staff, and key stakeholders,

• Review and ensure that factual corrections provided are incorporated and accepted in the final report,

• Facilitate the finalization of the management response matrix,

The Project Team of IOM:

• Provide the evaluator with general information, project documents, donor reports, and budget information relevant to the project,

• Review the deliverables submitted by the evaluator,

• Assist in scheduling in-person and virtual meetings and send meeting requests to identified key stakeholders,

• Provide support in logistical and travel arrangements as needed,

• Coordinate with relevant stakeholders, including government counterparts, throughout the evaluation process.

Schedule

The final evaluation will span 53 days, encompassing the review process of the deliverables by IOM. The assignment is scheduled to commence on 8 January 2024, with a final report expected to be submitted by 29 February 2024.

The final evaluation of the project is divided in three phases:

Phase 1 – Inception

Proposed timing:

Week 1 and 2

Outputs:

• Workplan and schedule of meetings
• Inception report and presentation (understanding of assignment, work plan, methodology and approach)
• Interview guidelines/questionnaire and evaluation matrix developed
• Prepare data collection plan

Areas to be covered

• Kick-off meeting
• Project document review

The inception report, including the evaluation matrix, should be submitted within two weeks of initiating the assignment. This report will showcase the evaluator’s understanding of the assignment, outlining the proposed methodology and a work plan aligned with the identified outputs of the evaluation. Prior to the inception phase, the IOM Türkiye IBG Unit and MEAL Unit will collaborate to compile all the necessary documentation for the desk review. Additionally, a stakeholder list with the contact information will be shared for the efficient organization of the meetings and interviews. The evaluator and IOM team will have a kick-off meeting in the first week. During the initial week of the evaluation, the evaluator will thoroughly review all project documents, reports provided by IOM, and monitoring reports from the MEAL team. The evaluator will prepare a detailed inception report during this phase, incorporating the evaluation matrix, the data collection tools and data collection plan.

Phase 2 –Analysis and Reporting

Proposed timing:

Weeks 3, 4, and 5

Outputs:

• Results of documentation review/consultations/interviews
• Draft Evaluation Report

Areas to be covered:

• Interviews, surveys, and focus group discussions
• Draft evaluation report

In this phase, the evaluator will conduct interviews, surveys, and focus group discussions with the project stakeholders and IOM staff. IOM will review and provide feedback on the technical aspects of the draft report. The evaluator will only make factual corrections to the report.

Phase 3 – Finalization and Sharing of Final Report and Other Deliverables

Proposed timing:

Weeks 6, 7 and 8

Outputs:

• Final Evaluation Report
• Evaluation Brief and Draft Management Response

Areas to be covered:

• Evaluation Report finalized in coordination with relevant colleagues.
• Evaluation findings and recommendations were presented to the IOM IBG Unit, senior management, and other key stakeholders.

The evaluator will finalise the evaluation report based on feedback received during Phase 2. These three phases will make the final evaluation process comprehensive and provide valuable insights into the project’s outcomes and effectiveness. The following table gives an overview of the estimated timeline and distribution of responsibilities for the evaluator:

Activity: Kick-off meeting and sharing relevant project documents.

Responsible party: IOM Team and Evaluator

Number of days: 1 day

Timing: Week 1

Activity: Review documents and prepare a detailed inception report, including the evaluation matrix, workplan, data collection tools and field visit plan

Responsible party: Evaluator

Number of days: 6 days

Timing: Week 1

Activity: IOM Team to review the inception report and provide inputs on the evaluation matrix, data collection tools and field visit plan

Responsible party: IOM Team*

Number of days: 4 days

Timing: Week 2

Activity: Finalize the inception report and the field visit plan

Responsible party: Evaluator

Number of days: 2 days

Timing: Week 2

Activity: Planning and facilitation of the data collection activities, logistical arrangements, and schedule

Responsible party: Evaluator, in coordination with the IOM Team.

Number of days: 5 days

Timing: Week 3

Activity: Data collection

Responsible party: Evaluator, in coordination with the IOM Team.

**Number of days:**12 days

Timing: Weeks 3, 4, and 5

Activity: Data analysis and draft evaluation report, and preparation of the presentation, (MS PowerPoint) to the IOM team

Responsible party: Evaluator

Number of days: 6 days

Timing: Week 5

Activity: Power point presentation to IOM team

Responsible party: Evaluator

Number of days: 1 day

Timing: Week 6

Activity: Prepare a draft evaluation report

Responsible party: Evaluator

Number of days: 5 days

Timing: Week 6

Activity: Review of the report and provide factual corrections

Responsible party: IOM Team*

Number of days: 4 days

Timing: Week 7

Activity: Prepare the final report, evaluation brief and management response matrix and share with IOM for review

Responsible party: Evaluator

Number of days: 5 days

Timing: Week 8

Activity: Review the evaluation brief and draft management response matrix

Responsible party: IOM Team*

Number of days: 1 day

Timing: Week 8

Activity: Share the final report, presentation, evaluation brief and management response matrix

Responsible party: Evaluator

Number of days: 1 day

Timing: Week 8

*The dates mentioned pertain to IOM are not factored into the total days allocated for the evaluator. However, it encompasses in the overall duration required for the evaluation to conclude.

Evaluation budget

The evaluator will be responsible for covering all expenses associated with the external evaluation, including travel expenses and accommodation. The evaluator will bear these expenses as part of their evaluation responsibilities.

Payment Plan

Deliverable: Submission of the inception report

Percentage of the Payment: 15%

Deliverable: Submission of the first draft of the report

Percentage of the Payment: 30%

Deliverable: Submission of the final report

Percentage of the Payment: 55%

**Total:**100%

Duration of the Contract

The overall evaluation process, incorporating the IOM review, will extend to a period of 53 days, with 44 days allocated for the evaluator. This timeline includes various tasks involved in the evaluation, such as preparation, data collection, analysis, and reporting. It is essential that the evaluator can efficiently manage and carry out these tasks concurrently within the planned timeframe, ensuring the expected quality of the evaluation. The assignment is scheduled between January 8 and February 29, 2024.

Required Qualifications:

IOM Türkiye is seeking an independent, multidisciplinary external evaluator. The evaluator will be based in any country but should be willing to travel to Türkiye to conduct the evaluation. The candidate should demonstrate prior experience in conducting similar types of evaluations.

Experience:

• The evaluator should have at least ten years of experience in conducting project evaluations.
• Sound experience with both quantitative and qualitative research methods and analytical tools.
• The candidate should have a master’s degree or equivalent in evaluation, law, social sciences, public policy, development studies, or related fields.
• Demonstrated sound understanding of thematic migration topics, for instance, migration governance, border management, labour migration, trafficking in persons, and crisis management.
• Experience in evaluating IOM Immigration and Border Governance (IBG) projects is an advantage.
• Excellent analytical, oral, and written communication skills in English.
• Experience collecting qualitative and quantitative data under challenging circumstances and using inclusive and participatory approaches.
• Ability to create graphic visuals on key findings.
• Experience in technical and analytical report writing.
• Experience in working in complex institutional environments.
• Displays cultural, gender, religion, race, nationality, and age sensitivity and adaptability.
• Ability to work with minimal supervision and to meet deadlines.

Languages:

• Fluency in English and Turkish is required.

Required Competencies

The incumbent is expected to demonstrate the following values and competencies:

Values

• Inclusion and respect for diversity: respects and promotes individual and cultural differences; encourages diversity and inclusion wherever possible.
• Integrity and transparency: maintains high ethical standards and acts in a manner consistent with organizational principles/rules and standards of conduct.
• Professionalism: demonstrates ability to work in a composed, competent and committed manner and exercises careful judgment in meeting day-to-day challenges.

Core Competencies – behavioural indicators level 2

• Teamwork: develops and promotes effective collaboration within and across units to achieve shared goals and optimize results.
• Delivering results: produces and delivers quality results in a service-oriented and timely manner; is action oriented and committed to achieving agreed outcomes.
• Managing and sharing knowledge: continuously seeks to learn, share knowledge and innovate.
• Accountability: takes ownership for achieving the Organization’s priorities and assumes responsibility for own action and delegated work.
• Communication: encourages and contributes to clear and open communication; explains complex matters in an informative, inspiring and motivational way.

Other:

The consultant must adhere to the International Organization for Migration (IOM) Data Protection Principles (IN/138) and maintain confidentiality.

The consultant will be responsible to follow IOM writing guidelines and latest glossaries in all given assignments for accurate translation.

Any offer made to the candidate in relation to this vacancy notice is subject to funding confirmation. Appointment will be subject to certification that the candidate is medically fit for appointment, accreditation, any residency or visa requirements, and security clearances.