Consultation Service for the Final Evaluation of ECHO HIP (Oct 2021 to Nov 2022) At Danish Refugee Council

Who is the Danish Refugee Council?

Founded in 1956, the Danish Refugee Council (DRC) is a leading international NGO and one of the few with specific expertise in forced displacement. Active in 40 countries with 9,000 employees and supported by 7,500 volunteers, DRC protects, advocates, and builds sustainable futures for refugees and other displacement-affected people and communities. DRC works during displacement at all stages: In the acute crisis, in displacement, when settling and integrating into a new place, or upon return. DRC provides protection and life-saving humanitarian assistance; supports displaced persons in becoming self-reliant and included in hosting societies; and works with civil society and responsible authorities to promote the protection of rights and peaceful coexistence.

As the world’s largest refugee-hosting country, Turkey currently hosts more than 4 million refugees and asylum seekers. While the majority of refugees remain Syrian (over 3.6 million registered as of February 2019), the country also hosts a significant number of refugees (over 300,000) from Iraq, Afghanistan, Iran, Somalia, and 60 other nationalities. The Turkish government has acknowledged the protracted nature of the Syrian crisis and taken leadership in the response to the Syrian refugee crisis with an evolving institutional structure and approach. In response to gaps at the level of response and the enduring needs of displacement-affected populations, DRC is currently implementing protection programs supported by ECHO, SDC, DANIDA, and public donations. These programs are designed to together support the realization of DRC’s Protection Strategy objectives.

DRC’s strategic vision is that all displacement-affected populations – refugees, migrants, and vulnerable host communities – can enjoy their rights in accordance with national and international legal and normative frameworks and are supported towards achieving durable solutions. Central to DRC’s overall program objective for protection is the recognition that the humanitarian situation in Turkey has evolved into the protracted displacement/recovery phase. It further recognizes that the protection response in Turkey is government-led and that the majority of refugees reside among host communities (96%).

Purpose of the consultancy

The Danish Refugee Council based in Türkiye seeks proposals from a consultant/consultancy to conduct a final evaluation for an ECHO-funded Protection Project implemented by DRC and two local partners in Hatay, Kilis, Sanliurfa, Adana, and Istanbul.

Background

Working with implementing partners Refugee Rights Turkey (RRT) and Support to Life (STL), the Danish Refugee Council (DRC) is addressing the risks faced by refugees in Turkey, including by sharing key information, providing specialized services, conducting research into pressing protection-related topics, and building the capacities of key stakeholders to respond to protection risks of displacement affected population in Hatay, Sanliurfa, Kilis, Adana, Edirne and Istanbul through a protection Programme funded by ECHO. The program started on the 1st of October 2021 and ends on the 30th of November 2022, including a two-month No Cost Extension (NCE). DRC aims to undertake an external evaluation to assess the achievements of the program according to the DAC criteria. The evaluation will be shared with ECHO, DRC program units, and implementing partners.

The principal objective of the project is: **Displaced populations in Southeast Turkey to enjoy an improved protective environment and have the capacity and knowledge to pursue safe and dignified solutions.**To achieve this objective, DRC and local partners (RRT and STL) have provided direct protection services e.g. GBV case management, Legal case management, Individual Protection Assistants (IPA), Legal counseling, Awareness Raising Sessions, Psycho-Social Support (PSS), and referrals to external specialized service providers. In addition to this, DRC and partners conducted research on protection-related topics, collated protection data into advocacy products, led advocacy meetings and roundtable discussions, and provided capacity building to local actors to implement protection services. Thus, the project aims to improve access to rights and services for displaced populations in Turkey, through information dissemination, specialized protection services, and collaboration with local stakeholders.

The specific objective of the action is measured through three outcome indicators:

1. % of beneficiaries reporting that humanitarian assistance is delivered in a safe, accessible, accountable, and participatory manner
1. % of beneficiaries reporting mitigated protection risks following Programme interventions
1. % of targeted beneficiaries reported increased knowledge about rights, entitlements, and available services by the end of the Programme

In addition to the above outcome indicators which are in the Programme Log frame, DRC and partners are reporting the below indicator which is included in the Facility Report platform.

1. % of refugees reporting being satisfied with the protection services they received

The Programme is designed into four results, each focusing on a different aspect of the protection programming and each result is measured through a set of output indicators:

Result-1: Increased knowledge of and appropriate information on risks, rights, entitlements, and available services

- 1.1 # of persons with increased/relevant information on appropriate rights/entitlements – 1.2 # of people reached by the implementation of specific GBV measures (GBV Awareness) – 1.3 # of persons trained on Number of persons trained on general protection and safety information

Result-2: Improved well-being and safety, reduced risks and immediate threats, through specialized protection services

2.1 # of survivors who receive an appropriate response to GBV
2.2 # of individuals who achieve appropriate documentation/ legal status
2.3 # of persons who have received detention visits by the end of the Programme
2.4 # of persons reached through IPA
2.5 # of persons reached through legal case management
2.6 # of persons reached through general case management
2.7 # of persons reached through legal counseling and assistance (CM)
2.8 # of external referrals made
2.9 # of people reached through PSS sessions

Result-3: Evidence-based advocacy initiatives targeting relevant stakeholders and policymakers to mitigate the risks faced by displacement-affected people

3.1 # of Protection Information Management Products enabling evidence-informed advocacy for quality protection outcomes produced
3.2 # of protection advocacy products disseminated and/or the number of meetings/events held

Result-4: Strengthened technical capacity of relevant stakeholders to better respond to protection risks of the affected population

4.1 # of participants showing an increase in knowledge on the protection subject in focus
4.2 # of individuals participating in capacity building training in protection and organizational capacity building

The objective of the consultancy

The purpose of this consultancy is:

To assess the performance of the program against key evaluation criteria for all the Programme activities implemented by DRC and partners;
To assess the program in terms of coordination, complementarity, and coverage with the local capacities, the network of relevant actors and authorities;
To assess the programme implementation concerning the participation of beneficiary communities in decision-making through relevant mechanisms established by both DRC and implementing partners;
To identify lessons learned and best practices in programming implemented by DRC and partners;
To identify and assess key internal and external factors that have contributed to, affected, or impeded the achievements of the Programme, and how DRC and the partners have managed these factors.
To identify and validate unintended impacts of the program (if any) and suggests recommendations to address them.
To provide recommendations based on findings for DRC, its implementing partners, ECHO, other relevant stakeholders, etc.

The consultant will be required to conduct the evaluation considering the following six criteria: Relevance, Effectiveness, Impact, Efficiency, Sustainability, and Coherence.

Relevance/Appropriateness

Is the intervention doing the right things?

To what extent were the Programme objectives and interventions valid or in line with the needs of people, context-specific and ECHO mandate in Türkiye?
- - How needs-based, context-adapted, and capacity-conscious was the design of the Programme?
To what extent the activities and outputs of the program were consistent with the overall goal and the attainment of its objectives?
- - How appropriate were DRC’s program design and implementation structure?
To what extent the response modalities to the identified needs and the context were relevant?
- - How relevant was the capacity building to the identified gaps? – How responsive were program activities to the needs of different targeted groups, e.g. male and female, in targeted communities (if possible, please distinguish between gender, age, nationalities (Syrians and Non-Syrians) refugees, and migrants) given their circumstances and priorities?
To what extent have the key contextual changes, threats, and opportunities that arose during implementation influenced and informed program implementation?
How appropriate were DRC’s beneficiaries’ participation and complaints and feedback mechanisms to the context?
How and to what extent were monitoring and research findings used to inform decision-making and the improvement of program implementation by DRC and partners?

Coherence

How well does the intervention fit?

To what extent does the program complement existing interventions in the country as well as government policies and strategies?
To what extent was the program coordinated with other relevant actors?
To what extent were local capacities developed or strengthened by DRC through the program? What was the added value of DRC to its partners’ implementation?
To what extent were government stakeholders consulted in the design of the program, and how was it coordinated with them to improve complementarity and coverage?
What are the synergies and interlinkages between the Programme and other interventions carried out by DRC Turkey?

Effectiveness

Is the intervention achieving its objectives?

To what extent were the objectives achieved/are likely to be achieved?
- - What were the major factors influencing the achievement or non-achievement of the objectives?
How effective was DRC’s response within the protection program concerning intervention modalities (CM, IPA, PSS, GBV, Travel Safety Information, Awareness raising conducted for all program activities, and Legal support) as well as the referral mechanism and pathways and intended results?
How well did DRC’s partnership approach work and how well were the partners’ capacity gaps to deliver assigned protection interventions addressed through organizational and capacity development?
To what extent the protection staff structure and the overall management structure were appropriate to deliver an effective protection response?
- - What are the recommendations in terms of structure for similar programs?
How were beneficiaries selected and were they informed of the selection criteria or participation in the program?
- - How effective were the selection criteria in reaching the most vulnerable populations?
Were there any risks inherent to the duration of the program, either in the course of or towards the end of the intervention?

Efficiency

How well are resources being used?

To what extent the interventions were cost-efficient and the objectives achieved timely?
- - What are the indicators that show the program implemented most efficiently compared to alternatives?
How efficient were the management structures and the implementation modalities in terms of timeliness of delivery and cost-effectiveness of the interventions?
To what extent the M&E and Information Management tools used by DRC and partners were efficient for this program?
To what extent was the program implemented based on the best use of existing resources/capacity; e.g. the capacity of the partners and the internal capacity and expertise of DRC itself?
- - What key limitations existed on this front? – What could DRC’s future programs do to increasingly develop and invest in existing resources? – What cost-effective alternatives could have been used?

Impact

What differences does the intervention make?

What are the most significant changes/impacts of the program in the lives of targeted populations? (Analyse the contribution of the program to any observed impact (intended, unintended, positive, negative) and analyze what other actors and factors contributed to the impact.
What are the intended and unintended effects of the program, and how do they relate to the principal objective?
- - Did all intended beneficiaries benefit equally from interventions as per their needs?

Sustainability

Will the benefits last?

To what extent are the changes made by the program likely to continue after donor funding ceases?
- - How did the program plan and implement an adequate transition and exit strategy that ensures longer-term positive effects and reduces the risk of dependency?
What were the major factors which influenced the achievement or non-achievement of sustainability of the program?
To what extent are the local communities and stakeholders more resilient (in line with the program results) than before?

Scope of work and Methodology

DRC Turkey is seeking to conduct an external evaluation with an overall objective of generating evidence to inform DRC’s Protection Programming to ensure that the program interventions are relevant to the context and that overall program performance is satisfactory and has made a contribution to its objective.

During the evaluation, apart from evaluating the performance of the program, DRC will be capturing lessons learned and best practices as a continuous part of the learning process. The scope for evaluation is determined using OECD-DAC criteria, i.e. Relevance, Effectiveness, Impact, Efficiency, Sustainability, and Coherence. Details of each criterion are given in section H.

The Consultant will be required to prepare a detailed methodology and work plan indicating how the objectives of the project will be achieved, and the support required from DRC.

DRC is committed to carrying out an external evaluation for the ECHO-funded Protection Programme in line with the principles of ethics in evaluation, i.e. Integrity, Accountability, Respect, and Beneficence. DRC considers engaging the beneficiaries and stakeholders in line with the OECD DAC Quality Standards for Development Evaluation. DRC will use qualitative methods using both primary and secondary data collection techniques for this evaluation. The evaluation firm/individual consultant is expected to develop a detailed and robust methodology upon their selection (to be approved by DRC Turkey) to generate a representative sampling size and credible findings. The evaluation methodology should also make use of existing monitoring and research studies data already generated by DRC Turkey and its partners.

The evaluation is expected to be conducted in the field (Hatay, Kilis, Sanliurfa, Adana, and Istanbul) while it will also allow 20-30% of remote data collection as it is found necessary by the selected consultancy and approved by DRC. The samples will be taken based on the nature of the specific interventions within the program and the tools will be designed to reflect relevant target groups accordingly-.eg: FGDs for community groups, KIIs for key stakeholders, and individual interviews for those not able or willing to join an FGD. Other creative modalities and tools are expected to be suggested by the consultancy as needed for the purposes of the evaluation, outlined above

DRC expects the evaluation methodology to be highly participatory and inclusive, reflecting the program’s commitment to Age, Gender, and Diversity. As such, evaluation tools should seek to reflect the opinions of men, women, non-binary, and other gender identities, people with disability, different age groups, and various nationalities, locations, etc.

Thus, it requires the consultant(s) to be experienced in participatory approaches to learning and inquiry, and especially in seeking the views and perceptions of key stakeholders that include:

Targeted beneficiaries
Partners and actors directly involved in the program at a different level

The implementing partners
Community leaders (if applicable) and representatives of people of concerns
Local authorities
ECHO representatives/relevant program staff
Protection Technical Working Group Lead in the region

DRC staff involved in the implementation of the program

In addition, the following should be taken into consideration:

That the method(s) and the approach chosen are relevant to the objective of the evaluation
That the individuals to be consulted during the evaluation are relevant to the focus of the evaluation
That the method(s) and the approach chosen allow for source and method triangulation of findings
That the data which is not used in the evaluation report is not collected.
That the method and approach chosen are ethically sound and culturally sensible.
Data privacy and security are the sole responsibility of the contractor.

Note: Individual consultants or firms who are operational and aware of the local context in Turkey will be given priority.

6. Deliverables

The Consultant will submit the following deliverables as mentioned below:

The following are the expected deliverables of the evaluation:

Inception Report: The inception report (max 10 pages) will be submitted to DRC 15 days after signing the contract with a clear evaluation methodology, timeline, and data collection tools (as annexes). The inception report will need to be authorized by DRC before data collection starts.
First Draft of Evaluation Report: Consultant/Firm will prepare the first draft evaluation report in English and will share it with DRC (between 25 to 30 pages excluding annexes, including an executive summary not exceeding 2 pages and recommendations). Feedback from DRC will be provided within one (1) week after the submission of the draft report. Minimum guidelines on the evaluation report :
1. The report should systematically answer the key questions posed;
2. It should fairly and clearly represent the views of the different actors/stakeholders;
3. It should give the conclusions of the evaluator in a way that is clear and substantiated by the collected evidence.
Evaluation Management Response Plan: Evaluation team will include all key findings in the management response plan for the team to prepare a management response plan to be presented in the debrief meeting by the DRC team. DRC will share the management response plan template.
Evaluation Debrief Meeting: A remote or in-person debrief meeting/workshop will be conducted by the selected firm or consultant to present the draft findings of the evaluation, and to give feedback to extended SMT.
Final Draft Evaluation Report: A final draft of the evaluation report will be shared within 7 days after the debrief meeting with DRC and partner staff. It will include changes/modifications, agreed upon between DRC and the consultant/firm.
Lessons Learned Note and Quality of Evidence Checklist: Two separate annexes will be prepared along with the final evaluation report, i.e. DRC lessons learned note and BOND principles and checklist for assessing the quality of evidence. DRC will share the templates for both annexes.

The Consultant will provide the documentation by, email / Word, Excel, PowerPoint, and PDF formats.

Duration, timeline, and payment

The total expected duration to complete the assignment will be no more than 55 working days (from the 1st week of October until December 15, 2022).

The consultant shall be prepared to complete the assignment no later than 15/12/2022.

Payment will be tied to the submission of evaluation deliverables.

Payment will be done in two installments, 30% after the submission of the inception report and approval by DRC, and 70% upon DRC’s approval of the final evaluation report including all annexes.

Note: Upon signing of a contract, a final timeline of evaluation deliverables will be agreed upon and a final payment schedule will be specified by DRC.

Proposed Composition of Team

Lead Evaluator
Evaluation field team
Data Analyst
Quality control

Eligibility, qualification, and experience required

Essential:

The evaluation consultant or firm should have the following expertise and skills:

Extensive expertise in evaluations of complex programs in a humanitarian context and a minimum of seven years of experience in conducting complex evaluation studies that include protection programming consisting of case management, GBV response, and PSS activities;
Proven experience in conducting evaluation studies with international humanitarian organizations and knowledge of the protection sector;
Proven experience in the use of quantitative and participatory qualitative methods for data collection and analysis including field and remote data collection and field and remote management of evaluation;
Familiarity with the humanitarian response to the Syrian conflict and with Turkiye’s context
Knowledge of ICT tools for mobile data collection;
Excellent writing and communication skills in English;
Post-graduate university degree in social sciences or another relevant academic discipline
Understanding of data collection ethics including GDPR (Global Data Protection Regulations) and KVKK (Kişisel verileri koruma Kanunu)

Desirable:

Proficiency in Arabic and/or Turkish
Familiarity with DRC programs and operations in the middle east especially Turkey.
Eligibility:
- The consultant has the authorization to work in Türkiye
Qualification:
- At least a Post-graduate university degree in social sciences or another relevant academic discipline
Experience:
- A minimum of seven years of conducting complex evaluation studies that include protection programming consisting of case management, GBV response, and PSS activities; proven.
Skills and knowledge:
- As mentioned above (candidates selected for interviews will be invited to share examples)
- A solid understanding of Turkiye’s cultural/economiccontext
Language requirements:
- Written and spoken fluency in English
- Working knowledge of Arabic and Turkish is an advantage

Technical supervision

The selected consultant will work under the supervision of:

MEAL Manager, Fariduddin Barzgar, Fariduddin.barzgar@drc.ngo
Head of Programme, Nicolas Metri, Nicolas.metri@drc.ngo

Location and support

The evaluation will take place in Hatay, Kilis, Sanliurfa, Adana and Istanbul provinces of Türkiye.

The Consultant will provide her/his own computer, mobile telephone, and other required working devices.

DRC will provide working place in area offices to the evaluators.

Travel

The evaluation team is expected to travel to the program area to conduct the data collection in the field.

The Contractor shall organize and pay for the costs of all travel-related expenses, including hotel accommodation. The calculated costs for travel and hotel accommodation must be included in the Financial Bid. The Contractor shall complete the DRC Travel Clearance Request and obtain approval thereof prior to commencing the travel. The DRC Turkey Hatay office will be responsible for arranging and covering the costs of:

(a) Organising pick-up and drop-off as required.

(b) Assisting in hotel bookings, if required.

(d) Organising local transport, if required.

(e) Assisting in arranging interviews with stakeholders and facilitating access to people of concern.

DRC will provide local transportation to the evaluation team in Hatay, Kilis, Urfa, and Adana (basically vehicles to travel with the cities or one to another for the data collection purpose).

The consultant will therefore be expected to arrange flights, accommodation, insurance, and food and to make adequate provisions in the Financial Proposal.

Submission process

Interested teams or consultants should submit an expression of interest and updated CVs and other listed documents detailing:

A cover letter of no more than 2 pages introducing the evaluator/the consultancy firm and how the past experiences, skills, and competencies meet the expected qualifications, with concrete examples. Please also use this cover letter to indicate the consultants’/firm availability for the proposed period. Note: As stated above the evaluation should start in October 2022 and the final evaluation report should be submitted to DRC no later than December 15th, 2022.
An outline of no more than 10 pages of the proposed process including:
- Key considerations for this evaluation;
- Outline the proposed methodology for this evaluation;
- Indicative work plan with realistic deadlines
CVs of the proposed evaluation team
2 reference letters/feedback from previous clients
Financial Bid (Annex A.2), covering all major anticipated technical and non-technical costs (DRC will not be responsible for covering or reimbursing any cost related to boarding/ lodging, accommodation, travel, insurance, food, etc.)
Two sample reports from previous evaluations

(For the complete list of requested documents please check RFP Invitation Letter)

For the evaluation of the bids please check the evaluation criteria stated in the RFP Invitation Letter.