Consultant, Evaluation At International Federation of Red Cross And Red Crescent Societies

Final Evaluation – IFRC Population Movement Emergency Appeal Greece

Summary:

1. Purpose: The International Federation of Red Cross and Red Crescent Societies (IFRC) seeks to evaluate the relevance, effectiveness, efficiency and sustainability of the Greece: Population Movement (MDR65003) Emergency Appeal which had the operational timeframe of March 2020 to September 2022.
2. Audience: The findings will be the basis for the Hellenic Red Cross, IFRC and Partner National Societies (PNS) to inform future planning and response preparedness for similar operations in the future.
3. Commissioners: This evaluation is being commissioned by the IFRC Regional Office for Europe jointly with the IFRC Country Delegation in Greece in compliance with its Evaluation Policy for final evaluations.
4. Consultant or consulting organisation: A lead evaluator and optionally up to one additional evaluation team member or a consulting organisation not directly involved with this project. If two evaluators apply jointly, the name of the lead evaluator must be indicated in the application.
5. Reports to: The evaluator(s) will report to the Evaluation Management Team (EMT). Whilst in Greece (if COVID-19 related situation allows in-country presence), the evaluator(s) will be under the management of the IFRC Head of Delegation in Greece. If a single evaluator applies, a Research Assistant may be recruited locally through a separate process to support the evaluation.
6. Duration: 6 weeks (including briefings, desktop review, field work, report writing, presentation, and follow-up). Maximum 35 days covered by consultancy fees (can be divided between two evaluators if necessary).
7. Timeframe: 15 September 2022 – 31 October 2022, with a possibility of a 2 week no-cost extension for report finalization.
8. Methodology summary: Secondary data review, key informant interviews with stakeholders across different levels and, internal discussions, analysing the quantitative data available from the project interventions and other evaluations/PDMs over the lifecycle of the project, validation on a Lessons Learned Workshop.
9. Location: Remotely for desktop review and in Greece for the primary data collection.
10. Application requirements: The consultancy for this final evaluation requires understanding of the International Red Cross and Red Crescent Movement and experience of evaluating post-disaster relief and recovery programmes. The evaluator will be responsible for the outcome of the evaluation.

Background:

Please refer to this link for reports on the Emergency Appeal operation:

https://www.ifrc.org/appeals?date_from=&date_to=&appeal_code=MDR65003&text=

Purpose:

The final evaluation aims to assess the relevance, effectiveness (and any clear impact), efficiency and sustainability of the activities, systems, management, and coordination of the IFRC Emergency Appeal (MDR65003) responding to the Greece: Population Movement.

Scope:

Timeframe of the emergency appeal: March 2020 – September 2022.

Timeframe of the evaluation: September – October 2022.

Geography: Greece, all major operational points (primarily Attica region, Ritsona Site, and Thessaloniki)

Programmes: by key sectors that have been funded through the Emergency Appeal

Objectives:

1. To assess the overall achievements (responsiveness, impact and quality) of services implemented under the Emergency Plan of Action

• Do project objectives correspond with identified needs? Were the intervention choices appropriately prioritised to meet the most urgent needs first?
• What mechanisms were put in place to capture community’s complaints/feedback, and how accountable was the project to implement the feedback and follow up with the community?
• To what extent do the activities reach the target population and were there any factors that hindered the ability to reach the population most in need?
• What was the impact of COVID-19 and public health restrictions on the Emergency Appeal’s implementation? How did programs adapt in the face of this change in context?
• How effective were processes for planning, monitoring and quality management?
• How gender and diversity needs, and potential differentiated impacts analysed, accounted for, and integrated in programming, monitoring and reporting processes?

2. To assess HRC and IFRC collaboration, including the efficiency and relevance of IFRC support and identify lessons learned for other operations in the context of the Emergency Appeal

• How did IFRC add value through its operational and strategic coordination in this context? Did the IFRC ways of working with the National Society foster collaboration between sectors/divisions? How could this have been improved/adapted to produce better results?
• How did the funding mechanisms impact the effectiveness of the response? What was the level, type and timing of earmarking under the appeal and how did this impact the delivery of activities, flexibility and alignment with needs?
• How did the IFRC logistics requirements on specific items (especially on medicines) impacted the operation and the timely delivery of services to the beneficiaries?

3. To assess whether the EA reinforced the National Society’s capacity in the area of Migration

• Were there newly established capacities following the appeal that can be sustained?
• Did the interventions result in enhanced institutional capacity of the HRC across different levels?
• What are potential opportunities for continued collaboration between NS and authorities or other relevant stakeholders?

Evaluation Methodology:

The evaluator is expected to develop a detailed methodology for this evaluation in the inception report.

The methodology needs to include: review and analysis of key documents, key informant interviews with Host and Partner National Society stakeholders, IFRC personnel, government authorities and other relevant NGOs active in country (FGDs, and individual interviews). The evaluator can suggest other methodologies.

• Desk review: Conducting a desk review of documentation, including the Appeal, Plan of Action, proposals, operation updates, revisions, pledge-based reports, M&E data, final reports to back donors, and other reviews and lessons learned papers as well as analysing the appropriateness of PDM surveys and reports. Most of this work can be done from outside Greece.
• Key informant interviews: the evaluator need to interview a sufficient number of persons having been involved in the operation to have a solid overview of the different phases. This includes persons from the HRC, IFRC, Partner National Societies as well as authorities and other NGOs (if possible) engaged in the population movement operation. Online interviews can be arranged if it is a preferred channel and with the persons who have already left the operation. All PNSs who had/have in-country presence (programme under its responsibility) should be interviewed.
• Field visits: if COVID-19-related restrictions allow.
• Secondary analysis of quantitative data and data collection tools used in the operation: the data available and collected through various means, should also be analysed where pertinent.

Evaluation Criteria:

The following evaluation criteria should be used as applicable for every question and objective, based on their relevance for the given objective and the questions related to the objective.

1. Relevance and appropriateness of the operation in delivering assistance based on needs and context.

2. Coverage in reaching target population across different groups, ethnicity, gender, physical construct etc.

3. Efficiency of the interventions in delivering quality services with minimum resources, effectiveness of the management and systemic mechanisms and strategies; and accountability to the impact population in relation to service delivery, sustainability, inclusion and resilience building.

4. Impact of the interventions on the National Society’s capacities and the level of resilience of the communities and institutions in areas covered by the operation.

5. Connectedness and sustainability of the interventions in developing the organizational and institutional capacity of the HRC.

Deliverables / Outputs:

• Inception report: The inception report should include the proposed methodologies, a data collection and reporting plan with identified deliverables, draft data collection tools such as interview guides, questionnaire, sampling method, a timeframe with firm dates for deliverables and travel (if applicable) and logistical arrangements for the evaluation.
• Debriefing: A debriefing will be conducted with the IFRC Country and Regional Teams after data collection. The debriefing is to update the progress and initial findings with recommendations of the evaluation.
• Lessons Learned Workshop: LLW is part of the evaluation process, the evaluator provides input for the workshop, and then also uses it as a data source for triangulation, validating results, or for going in-depth on certain key topics identified previously.
• Draft report: The consultant will produce a draft report (identifying key findings based on facts, conclusions, recommendations and lessons for the current and future operations) which will be reviewed by the IFRC CCST operations and the Europe Regional Office teams. The consultant will be given the feedback after 10 working days to incorporate into the final report.
• Final report: A Final report highlighting key findings, conclusions and recommendations will be submitted within 10 days of receiving the feedback from the draft report. Final evaluation report of no more than 40 pages (excluding executive summary and annexes such as copy of the ToR, cited resources, a list of those interviewed and any other relevant materials)

The findings and all products arising from this evaluation will be jointly owned by the Hellenic Red Cross and IFRC. The evaluator will not be allowed, without prior authorization in writing, to present any of the analytical results as his / her own work or to make use of the review results for private publication purposes. All case studies, anecdotes, any rough tool used and copies of participatory tools to be submitted to the IFRC teams to ensure scientific accountability and data protection of the evaluation.

An Evaluation Management Team (EMT) will manage and oversee the evaluation and will ensure coordination between the IFRC Regional Office in Budapest, the IFRC delegation in country and representatives from the implementing National Society. The EMT will support the Evaluator in establishing contact with relevant stakeholders, and with developing modalities to ensure remote and, if feasible, on-site access for conducting the evaluation. The EMT is tasked with reviewing and approving the evaluation methodology and deliverables.

Position Requirements:

Education

Required:

• Relevant academic degree

Experience

The evaluator must have experience or significant knowledge of the humanitarian response mechanisms, specifically interventions in population movement, and have previous experience in conducting evaluations for medium-to-large scale programmes. In case it is deemed necessary, tasks can be divided between a lead evaluator and an evaluation team member.

Required:

• At least 7 years proven experience in leading evaluations in humanitarian programmes responding to emergency and recovery programs
• Previous experience in coordination, design, implementation and monitoring and evaluation of humanitarian programmes
• Experience in the evaluation of population movement programmes
• Experience in participatory approaches to evaluations

Knowledge, skills and languages

Required:

• Fluently spoken and written English
• Excellent writing and presentation skills in English, with relevant writing samples of similar evaluation reports.

Preferred:

• Very good understanding of the RC/RC Movement and types of humanitarian response
• Knowledge of Greek

Competencies and values

Values: Respect for diversity; Integrity; Professionalism; Accountability

Evaluation Quality and Ethical Standards: The evaluators should take all reasonable steps to ensure that the evaluation is designed and conducted to respect and protect the rights and welfare of people and the communities of which they are members, and to ensure that the evaluation is technically accurate, reliable, and legitimate, conducted in a transparent and impartial manner, and contributes to organizational learning and accountability. Therefore, the evaluation team should adhere to the evaluation standards and applicable process outlined in the IFRC Framework for Evaluation. The IFRC Evaluation Standards are:

1. Utility: Evaluations must be useful and used.
2. Feasibility: Evaluations must be realistic, diplomatic, and managed in a sensible, cost-effective manner.
3. Ethics & Legality: Evaluations must be conducted in an ethical and legal manner, with particular regard for the welfare of those involved in and affected by the evaluation.
4. Impartiality & Independence; Evaluations should be impartial, providing a comprehensive and unbiased assessment that takes into account the views of all stakeholders.
5. Transparency: Evaluation activities should reflect an attitude of openness and transparency.
6. Accuracy: Evaluations should be technical accurate, providing sufficient information about the data collection, analysis, and interpretation methods so that its worth or merit can be determined.
7. Participation: Stakeholders should be consulted and meaningfully involved in the evaluation process when feasible and appropriate.
8. Collaboration: Collaboration between key operating partners in the evaluation process improves the legitimacy and utility of the evaluation.

It is also expected that the evaluation will respect the seven Fundamental Principles of the Red Cross and Red Crescent: 1) humanity, 2) impartiality, 3) neutrality, 4) independence, 5) voluntary service, 6) unity, and 7) universality.