FINAL EVALUATION CONSULTANCY FOR DG SANTE PROJECT At International Federation of Red Cross And Red Crescent Societies

The International Federation of Red Cross and Red Crescent Societies (IFRC) is the world’s largest humanitarian organization, with a network of 192-member National Societies. The overall aim of the IFRC is “to inspire, encourage, facilitate, and promote at all times all forms of humanitarian activities by National Societies with a view to preventing and alleviating human suffering and thereby contributing to the maintenance and promotion of human dignity and peace in the world.” The IFRC works to meet the needs and improve the lives of vulnerable people before, during and after disasters, health emergencies and other crises.

The IFRC is part of the International Red Cross and Red Crescent Movement (Movement), together with its member National Societies and the International Committee of the Red Cross (ICRC). The work of the IFRC is guided by the following fundamental principles: humanity, impartiality, neutrality, independence, voluntary service, unity, and universality.

The IFRC is led by its Secretary General, and has its Headquarters in Geneva, Switzerland. The Headquarters are organized into three main Divisions: (i) National Society Development and Operations Coordination; (ii) Global Relations, Humanitarian Diplomacy and Digitalization; and (iii) Management Policy, Strategy and Corporate Services.

The IFRC has five regional offices in Africa, Asia Pacific, Middle East and North Africa, Europe, and the Americas. The IFRC also has country cluster delegation and country delegations throughout the world. Together, the Geneva Headquarters and the field structure (regional, cluster and country) comprise the IFRC Secretariat.

This end of project evaluation will investigate the performance of the DG SANTE Mobile Testing Initiative Project against its intended results. The findings and recommendations emerging from the evaluation will be used to draw lessons and inform the conceptualization, design and implementation of similar projects in the future for IFRC and National Societies.

The overall objective of the DG SANTE Mobile Testing Initiative is to contribute to the reduction of morbidity and mortality of the novel coronavirus (COVID-19) outbreak by interrupting transmission chains in selected EU Member States by scaling up COVID-19 testing capacity of Red Cross-National Societies in EU member states including Austria, Germany, Greece, Italy, Malta, Portugal and Spain.

During the implementation period, National Societies with overall coordination of IFRC Regional Office for Europe (ROE) aimed to build new and develop existing capacities of their testing teams. This included training National Societies’ non-healthcare staff and volunteers on taking samples for COVID-19 testing with a nasopharyngeal swab or other non-invasive testing methods in accordance with WHO/EU recommendations in order to increase the necessary resources to take samples. Red Cross Mobile Testing Teams were equipped with the necessary equipment, lab items and reagents to take samples and perform tests. These actions intended to contribute to supporting the national public health systems in Austria, Germany, Greece, Italy, Malta, Portugal and Spain to increase COVID-19 testing through the adequate collection of samples and performance of tests to ensure rapid and adequate response to outbreaks and second waves. These strengthened capacities are to be embedded into the organizational knowledge and therefore, available also beyond the project timeframe due to the permanent presence of the National Societies in-country.

• Primary objective

The overall purpose of the evaluation is to investigate the intended and unintended outcomes of the project, specifically focusing on the following areas:

• National Societies’ support to health authorities in their COVID-19 response, including coordination and any benefits derived from their collaboration.
• Evaluate the visibility and ease of access to testing services amongst vulnerable groups including migrants, refugees, etc.
• Evaluate the impact of the project on the capacities of National Societies
• Identify potential opportunities for continued collaboration between NS and health authorities or other relevant stakeholders resulting from this project.
• Evaluate the coordination between NS and IFRC.
• Secondary objectives

In addition, the evaluation will provide concise recommendations around quality and operational improvements that should be made in order improve implementation, procurement, coordination and management for any similar projects in the future, taking into consideration IFRC as well as National Societies’ capacities and the local context.

Finally, the evaluation will contribute to ongoing organisational and operational learning for IFRC and RCRC as well as others in the sector, by documenting and providing recommendations around best practices and lessons learnt from the project to date in relation to training in mobile testing, testing, case identification, and linkage to care, with a particular focus on linkages between health services provided by National Societies and coordination and collaboration with project countries’ health authorities.

Possible key questions for the evaluation are provided in Annex 1 at the end of these Terms of Reference.

• Evaluation methods and processes

The evaluation methods must adhere to the Organisation for Economic Co-operation and Development (OECD) evaluation criteria, as they apply to the primary and secondary objectives of this evaluation.

An IFRC evaluation management team (EMT) will manage and oversee the evaluation. The EMT will consist of three to four people: IFRC Project Manager, Regional PMER Manager, a member of the IFRC Regional Health Team, and another PMER practitioner. The EMT will support the Evaluation Team with establishing contact with relevant stakeholders in the implementing National Societies, and with developing modalities to ensure remote and, if feasible, on-site access for conducting the evaluation. The EMT is tasked with reviewing and approving the deliverables listed in Section 5 of these Terms of Reference.

The evaluation team will consist of a lead evaluator and up to two evaluation team members. The lead evaluator will have the ultimate responsibility to lead the evaluation process, design the methodology and deliver the outputs as described in these Terms of Reference.

• The consultant or consulting organisation should bring an independent, objective perspective as well as technical experience around evaluations, ideally within the public health and humanitarian sectors. The consultant/ consulting organisation will report on progress and challenges to the EMT and be the primary author of the evaluation report. S/he will not previously have been involved or have a vested interest in the project. Ideally, s/he should have knowledge of mass screening activities and formal training in public health with demonstrated experience in undertaking similar
• It is expected that the individual consultant or consulting organisation will be able to conduct a reliable and informed evaluation of the project that has legitimacy and credibility with
• Countries to be sampled for potential field visits will be selected by the EMT in agreement with National Societies that have the capacity to provide logistical support to the evaluators. The selected National Society teams in country will provide in-country logistical
• The specific evaluation methodology will be determined by the individual consultant or consulting organisation, in close consultation with the EMT and relevant key stakeholders, but will draw upon the following primary methods:
  1. Desk review of project background documents, relevant organisational backgrounds, including relationship and structural arrangements between National Societies involved in the project and their respective health authorities with attention provided to national policies and/or guidelines around mass screening for COVID-19 and any relevant sources of secondary data.
  2. Field visits/ observations to selected project sites as applicable
  3. Key informant interviews (including implementers, health authorities and other stakeholders)
  4. Focus group discussions (implementers and other stakeholders, as appropriate)

  Evaluation Deliverables

• Inception report: Following an inception meeting, the inception report will be a scoping exercise for the evaluation and will include the proposed methods, data collection and reporting plans with draft data collection tools such as interview guides, the allocation of roles and responsibilities within the team (if applicable), a timeframe with firm dates for deliverables, costs and the travel and logistical arrangement for the team. The inception report will also be presented to National Societies involved in the project.
• Debriefings and validation of preliminary findings: The evaluation team will share and validate their preliminary findings in consultation with IFRC and National Societies. This will also include a power point presentation summarizing the preliminary findings.
• Draft report: A draft report will identify key findings based on facts, and will also outline relevant opinions, recommendations, conclusions and lessons for future project, and partnership with DG SANTE. This draft report will be submitted within one week of the evaluation team’s return from the field. This draft report will be circulated among key IFRC and NS staff for feedback.
• Final report and final presentation of findings: The final report will contain a short executive summary (no more than 1,000 words) and a main body of the report (no more than 10,000 words) covering the background of the project, a description of the evaluation methods and limitations, findings, conclusions, lessons learned and clear The recommendations should be specific and feasible. These should also be outlined in a management response matrix as shown in Annex 2. The report should also contain appropriate appendices, including a copy of the ToR, cited references, a list of those interviewed and any other relevant materials. The final report will be submitted one week after receipt of the consolidated feedback from IFRC and NS. Additionally, a concise Power Point presentation summarizing the main findings will be required.

  The individual consultant or consulting organisation should take all reasonable steps to ensure that the evaluation is designed and conducted to respect and protect the rights and welfare of the people and communities involved and to ensure that the evaluation is technically accurate and reliable, conducted in a transparent and impartial manner, and contributes to organisational learning and accountability. Therefore, the consultant or consulting organisation should adhere to the evaluation standards and applicable practices outlined in the OECD/DAC guidelines. Relevant evaluation standards include:

• Utility: Evaluations must be useful
• Feasibility: Evaluations must be realistic, diplomatic and managed in a sensible, cost-effective manner
• Ethics and legality: Evaluations must be conducted in an ethical and legal manner, with regard for the welfare of those involved in and affected by the evaluation, with attention to safeguarding the rights and confidentiality of project beneficiaries and/ or service providers
• Impartiality and independence: Evaluations should be impartial, providing a comprehensive and unbiased assessment that considers the views of all stakeholders
• Transparency: Evaluation activities should reflect an attitude of openness and transparency
• Accuracy: Evaluations should be technically accurate, providing enough information about data collection, analysis and interpretation methods so that its worth or merit can be determined
• Participation: Stakeholders should be consulted and meaningfully involved in the evaluation process when feasible and appropriate
• Collaboration: Collaboration between key operating partners in the evaluation process improves the legitimacy and utility of the evaluation

Selection of the individual consultant or consulting organisation will be based on the qualifications and experience outlined below:

For the lead evaluator (team leader):

1. 7-10 years demonstrable experience in leading or conducting evaluations of public health projects preferably with a focus on mass screening and /or infectious diseases.
2. Understanding and knowledge of the Red Cross Red Crescent network and its mandate would be preferable.
3. Knowledge of programme management and proven ability to provide strategic recommendations to key stakeholders
4. Strong analytical skills and ability to clearly synthesise and present findings, draw conclusions, make pragmatic recommendations and prepare well-written reports in a timely manner
5. Experience in both quantitative and qualitative data collection and data analysis techniques, especially in public health sector
6. Demonstrated capacity to work independently
7. Excellent English written and presentation skills, and able to provide relevant writing samples of similar evaluation reports
8. Familiarity with DG SANTE and the operational context in Europe within the COVID-19 pandemic would be an advantage
9. Immediate availability for the period

For the evaluation team members:

1. Demonstrable experience in conducting key informant interviews
2. Strong analytical skills and ability to clearly synthesise and present findings, draw conclusions, make pragmatic recommendations and prepare well-written reports in a timely manner
3. Excellent English writing skills
4. Ability to work remotely or on-site
5. Competent on the use of platforms such as Microsoft Office, Microsoft Teams, Zoom, etc.
6. Language skills in any of the languages used in project countries would be desirable (German, Greek, Italian, Portuguese, Spanish).