Consultant (External Evaluator) At International Organization for Migration

*Position Title : Consultant (External Evaluator)

Duty Station : Home-based with travel to project locations (Istanbul)

Type of Appointment : Consultant, July 8 to August 26, 2021

Estimated Start Date : As soon as possible

Closing Date : 30 June 2021

Requisition: CON2021/04

Context:

Under the overall supervision of the Senior Programme Development Officer and direct supervision of Monitoring and Evaluation Officer, the incumbent will assess the relevance, efficiency, effectiveness, impact, coherence, and sustainability of the project activities to contribute to the strengthened response of protection needs of the beneficiaries and enhanced understanding of migration and mobility patterns of refugees and migrants in Turkey. S/he will provide findings and recommendations to the project management, donor, and key partners with an indication of how well the project has been implemented.

Core Functions / Responsibilities:

Evaluation context

Following IOM’s successful completion of its “Supporting Refugees and Migrants through a Consolidated Mediterranean and Flow Monitoring Response in Turkey” project’s Phase I and II, and in recognition of the continued need for support in relation to data collection and humanitarian assistance provision to migrants rescued/apprehended at sea/land (hereinafter beneficiaries), IOM implements a continuation (Phase III) of the project. The project ensures:

· The continuation of the Mediterranean Response Program to extend the provision of humanitarian assistance to refugees and migrants rescued at sea/land

· Coverage and implementation of IOM’s Migrant Presence Monitoring (MPM) program

The humanitarian assistance consists of the provision of food items, non-food items (NFIs), and psycho-social support (PSS), including counselling and referrals to protection services for refugees and migrants rescued at sea, specifically in the border zones between Turkey and Greece in Kucukkuyu (Canakkale), Ayvalık (Balikesir), Kusadası (Aydin) districts and Izmir province; and at land, in Edirne province.

Furthermore, MPM works to continue its operations to publish regular overview and updates on the presence, flows, locations, intentions and vulnerabilities of refugees and migrants in Turkey. In this regard, MPM also creates customized surveys.

Implemented jointly with the Turkish Government (GoT), these actions ensure that the GoT is supported in managing the pressure of irregular migration by integrating the activities of the Turkish Coast Guard (TCG), Turkish National Police (TNP), and Turkish Gendermerie (TG) in their refugee and migrant responses, as well as the interdiction efforts. IOM supports Directorate General of Migration Management (DGMM) by institutionalizing valuable data collection structures as part of this support. The data collection structure component specifically is addressed by MPM through various on-the-job training and regular skill- and information-sharing meetings with both DGMM and Provincial Directorate of Migration Management (PDMM) officials.

Throughout the project duration, IOM has received two Non-Cost Extensions, until May 2021 and August 2021, respectively. To reach the project’s targets, IOM has requested the Non-Cost Extensions since COVID-19, and its continuing challenges resulted in a decline in the number of migrants attempting an irregular crossing to the EU and a consequent reduction in the beneficiaries assisted.

The outcomes and outputs of the project:

Objective: To contribute to strengthened response to protection needs of refugees and migrants rescued/apprehended at sea/land and enhanced understanding of migration and mobility patterns of refugees and migrants in Turkey.

Outcome 1: Migrants and refugees rescued/apprehended at sea/land are provided with humanitarian assistance.

· Output 1.1: Migrants and refugees rescued/apprehended by the TCG/land forces benefitting from distribution of NFIs and food items.

· Output 1.2: Migrants and refugees rescued/apprehended by the TCG/land forces benefitting from PSS and provision of information, including referrals, which consider specific vulnerabilities (including gender-based concerns).

Outcome 2: DGMM/PDMM and other stakeholders have increased access to comprehensive information relating to migration trends, thus contributing to their capacity to better address the migrant and refugee needs across Turkey.

· Output 2.1: Updated and systematic information on migration flows and mobility patterns in Turkey, including journey experiences, vulnerabilities (including gender-specific concerns), movement intentions, trends and needs.

· Output 2.2: Thematic and technical support is provided to DGMM to sustainably enhance data collection and evidence-based reporting.

The project activities are:

· Distribute the non-food item packages in coordination with TCG, TNP, and TG.

· Distribute the food packages in coordination with TCG, TNP, and TG.

· Implement First-Aid PSS.

· Design and carry out individual and group counselling and ad-hoc information dissemination as well as recreational activities for children.

· Refer beneficiaries to specialized service providers.

· Deploy data collectors and carry out field data collection activities, including Baseline Assessment and FMS.

· Collect, process, and analyze the collected data.

· Prepare analytical reports.

· Initiate preparation activities for analytical reports planned for 2019.

· Regular dissemination of MPM products to relevant stakeholders.

· Organize monthly meetings with DGMM to coordinate data collection efforts and ensure the transfer of knowledge.

· Provide support to DGMM to strengthen their local capacities in displacement tracking and monitoring in the country and analyzing migratory trends.

Evaluation purpose and objective

The general purpose of the final external evaluation is to assess the relevance, efficiency, effectiveness, impact, coherence, and sustainability of the project activities to contribute to the strengthened response of protection needs of the beneficiaries and enhanced understanding of migration and mobility patterns of refugees and migrants in Turkey. The purpose of the exercise will be to provide findings and recommendations to the project management, donor, and key partners with an indication of how well the project has been implemented. Furthermore, the evaluation will assess the quality of the project implementation and identify any bottlenecks in the implementation to identify problems as well as which activities should be modified for the upcoming 4^th Phase of this project. The evaluation also aims to capture achievements, determine challenges and best practices/lessons learned to inform the project team and concerned stakeholders for future programming. IOM will share the evaluation report with the project’s donor and stakeholders to demonstrate the value of the project and its impact. Thus, the evaluation will also be an opportunity for the donor and stakeholders to assess the relevance and accountability of the project towards the intended objective and beneficiaries.

Evaluation scope

The evaluation will cover the timeframe from September 1, 2019, to August 31, 2021. The data collection will be conducted in Izmir, Edirne and Ankara.

Evaluation criteria

To remain objective, the IOM Turkey MEAL team will facilitate a final external evaluation mainly based on the Organization for Economic Co-operation and Development, Network of Development Evaluation (OECD DAC) evaluation criteria, with the coherence criteria according to the ALNAP definition. This evaluation will assess the relevance, effectiveness, efficiency, impact, coherence, and sustainability of the project and identify key performance gaps to improve programming. The results of the external evaluation will inform project lessons learned and identify best practices to inform the 4^th Phase of this project which is currently under development.

Evaluation questions

Evaluation Criteria / Evaluation Question

1. Relevance: assessing to what extent the project’s objective and intended results remain valid and pertinent either as originally planned or as subsequently modified.**

• To what extent does the project/ programme continue to make sense in the current context, and to what extent do project objective and tasks remain valid and pertinent?
• Do the project’s expected outcomes and outputs remain valid and pertinent either as originally planned or as subsequently modified?
• Does the project still respond to the needs of the project beneficiaries and stakeholders?
• Is the project aligned with and supportive of IOM national, regional, and/or global strategies and the Migration Governance Framework?

3. Effectiveness: assessing the extent to which the project achieves its intended results.

• Is the project/programme effective in reaching planned results, in bringing planned changes? Are the target beneficiaries satisfied with the services provided?
• What are the major factors influencing the achievement of the project’s expected outcomes?
• To what extent has the project adapted or can adapt to changing external conditions to ensure project outcomes?

4. Efficiency: assessing how well human, physical and financial resources are used to undertake activities, and how well these resources are converted into outputs.

• Is the project being implemented in the most efficient way compared to alternative means of implementation?
• To what degree are resources provided/available at planned cost (or lower than planned) from all parties involved?
• Is a workplan and resource schedule available and used by the project management and other relevant parties?

5. Impacts: assessing the positive and negative, primary and secondary long-term effects produced by a project, directly or indirectly, intentionally or unintentionally.**

• Which positive/negative and intended/unintended effects can be observed in the project?
• Does the impact come from the project activities, from external factors or from both?
• Did the project take timely measures for mitigating any unplanned negative impacts?

6. Coherence: assessing the extent to which all policies take into account humanitarian and human rights considerations and coordination with other actors. **

• Are the project activities coordinated with other actors?
• Are there other actors involved in the same type of activities, and are they complementary to IOM’s activities?

7. Sustainability: assessing to what extent the project’s results will be maintained for a certain period of time after the current project phased out.

• Are structures, resources, and processes in place to ensure that benefits generated by the project continue once external support ceases?
• Do the project partners have the financial capacity, and are they committed to maintaining the benefits of the project in the long run?
• How far is the project embedded in institutional structures that are likely to survive beyond the life of the project?
• Are the results obtained sufficient to draw any conclusion on sustainability before the actual end of the project/programme? Are some activities already ongoing without any more support from the project/programme?

Though the Evaluator should collect findings and draw conclusions on the above questions, particular emphasis should be placed on questions of effectiveness, efficiency and impact of the project and selected approaches. Recommendations should focus on how to deal with identified challenges and gaps and improve the upcoming 4^th Phase of this project.

Evaluation methodology

The data collection will include the use of quantitative and qualitative approaches to gain a deeper understanding of the results of the project (outputs, outcomes, objective) and to ensure the cross-validation of data. The evaluation report will compose of analyses of various evidence-based sources of information, including desk analysis/review, evaluation survey data, and interviews with beneficiaries, counterparts, project team.

The evaluation will provide quantitative and qualitative data through the following methods:

• Desk analysis and review of all relevant project documentation including project proposal, log frame, work plans, project quarterly donor/progress reports, project reports, monitoring reports,
• In-depth interviews will be used to gather primary data from key stakeholders using semi-structured face-to-face questionnaires (TBC pending travel restrictions).

Ethics, norms and standards for evaluation

The evaluator must follow the IOM Data Protection Principles, UNEG norms and standards, and UNEG ethical guidelines and UNEG codes of conduct while carrying out the evaluation.

Evaluation deliverables

An inception report will be prepared by the evaluator and shared with the IOM team. The report should include an evaluation matrix, evaluation methodology, work plan, interview guide and other data collection instruments to be used for the evaluation. IOM can provide templates for an Inception Report and an Evaluation Matrix, for reference. The evaluator may use their own format but should include all of the elements of the IOM templates.

Following the field visit, the evaluator will prepare a short PowerPoint presentation of the initial findings, conclusions and recommendations. This will be used to debrief the IOM team to identify and address any misinterpretations or gaps.

A draft report will be shared with the IOM team building on the debrief and initial feedback. The IOM team will review and provide feedback on the report. The feedback will be provided on technical aspects and not on the findings, conclusions or recommendations. The evaluator will make factual corrections to the report.

A final report will be prepared and shared with the IOM team. IOM can provide a template for an evaluation report. The evaluator may use their own format but should include all of the elements of the IOM template: an executive summary, list of acronyms, introduction, evaluation context and purpose, evaluation framework and methodology, findings, conclusions, and recommendations. Annexes should include the ToR, inception report, list of documents reviewed, list of persons interviewed or consulted, and data collection instruments.

A two-page evaluation brief will be prepared by the external evaluator to share the key findings, conclusions, and recommendations. IOM template will be provided to prepare the brief.

Finally, the evaluator will prepare draft management follow up response matrix (template to be provided by IOM) by inserting the recommendations as well as an indicative timeframe or deadline for implementation. This will later be finalized by the IOM team.

A project closure event a one-day event, will be organized by the IOM MEAL team within the last week of August 2021, with the participation of project focal points and stakeholders, where the external evaluator will share the evaluation findings. The external evaluator will prepare and deliver a visual presentation of the evaluation findings and recommendations.

All deliverables are to be written in English. The final report should meet the standards laid out in the UNEG Quality Checklist for Evaluation Reports.

Specifications of roles

Role of each party are as follows:

· External Evaluator

o Duly implement the data collection, analysis, and reporting activities of the final evaluation.

o Timely flag any challenges or emerging issues to the project team.

o Prepare and deliver the inception report, final report, two-page evaluation brief and the visual presentation of the evaluation the findings and recommendations.

· MEAL Unit

o Facilitate the communication between the external evaluator and the program team.

o Facilitate arranging associated meetings and logistics.

o Help address any challenges or emerging issues flagged by the evaluator.

o Provide feedback on the deliverables submitted by the external evaluator.

· Program Management

o Oversee the general process of the final evaluation.

Time schedule

The overall evaluation process is expected to take 8 weeks (37 working days) including preparation, data collection, and analysis and reporting. The Evaluator should be able to undertake some of the tasks concurrently to fit within the planned timeframe without compromising the quality expected. The assignment is expected to be conducted between 08/07/2021-26/08/2021, mentioned dates included. Besides the mentioned interval, one-day project closure event would be organized within the last week of August, the exact date to be determined and announced, in which the participation of the external evaluator is required.

Activity/ Working Days/ Responsible/Locations/ Timing by weeks

• Review project documents and relevant literature/ 4 days / Evaluator/ Home-based/ Week 1 and 2
• Prepare and finalize Inception report data collection instruments / 4 days / Evaluator/ Home-based/ Week 2 and 3
• Data collection process (interviews with respective actors) / 7 days / Evaluator/ Face to face, in project locations / Week 3 and 4
• Initial analysis of the data collected / 5 days / Evaluator/ Home-based/ Week 4 and 5
• De-brief IOM team on initial findings / 1 day / Evaluator/ Online meetings/ Week 5
• Draft Report and full analysis of data*/ 10 days / Evaluator/ Home-based/ Week 5, 6 and 7*
• Finalization of the evaluation report and submit two-page evaluation brief, and draft management response*/ 3 days / Evaluator/ Home-based/ Week 7 and 8*
• Finalization of the evaluation key findings presentation, sharing evaluation findings with the project team and stakeholders at Lessons Learned meeting / 3 days / Evaluator/ Meeting venue/ Week

Evaluation requirements

IOM Turkey is seeking an independent, multidisciplinary external evaluation consultant.