Final Evaluation C4U3 & RISE At Caritas Switzerland

Leave a Comment / Ukraine / By

Request of Proposal: Final Project Evaluation

Caritas Switzerland (CACH) is seeking an evaluator to conduct the independent final evaluation of the “Cash Assistance and Resilience for Ukraine Phase III (CARE4U3)” and Resilience through Integrated Services for Social Empowerment (RISE) projects in line with the below Terms of Reference (ToR).

Application Deadline: Latest by 25 July 2025, 17:00 CET.

· Application Process: Please send an e-mail with your full application package in English with the subject header “Evaluation Offer” to the following email: ukrainecrisis@caritas.ch. Any queries may be submitted to the same address.

Evaluators intending to apply must provide the following information:

1. A short technical offer describing evaluation methodology to be used in conducting the evaluations to include:

  • a. Proposed activities, time frame and total level of effort.
  • b. Ethical considerations and gender sensitive evaluation approaches.
  • c. Evidence that the evaluator has the experience to conduct the evaluation (e.g., a list of previous evaluations that are relevant in relation to the context and subject-matter of this assignment that can be highlighted)

2. A financial offer accompanying the technical offer stating the daily professional fees expressed in CHF, exclusive of travel or field visit costs, translation, which should be outlined separately.

3. CV(s); team composition, expertise, and level of effort of each proposed team member and a description of how the evaluator(s) skills, qualifications and experience are relevant to the required qualifications of this assignment.

4. A statement confirming the availability of the candidate(s) to conduct this assignment within the intended timeframe.

5. If available or publicly shareable, two evaluation reports of related evaluations with explicit acknowledgement of the applicant(s) role.

Terms of Reference – Final evaluation CARE4UIII Project

  1. Purpose of the Evaluation

The main purpose of this final evaluation is to ascertain impact and assess the effectiveness, efficiency, relevance and sustainability of the projects. The evaluation should provide findings, conclusions and recommendations with respect to the project’s intervention logic in order to draw lessons for future design and implementation and learning and accountability to various stakeholders including Caritas Switzerland’s back donor Swiss Solidarity (SwS).

2. Summary of the Intervention

The projects are briefly summarised below. Both projects are quite similar in terms of overall design – with some relatively minor differences. The primary distinguishing factors between the two projects are the project partner and geographic split with Cartias Ukraine implementing the C4U3 project in three oblasts (Zaporizka, Kharkivska and Dnipropetrovska), and our other partner Cartias Spes Ukraine, delivering the RISE project in three separate Oblasts (Odeksa, Mykolaivska, Khersonska). Therefore, the same evaluation approach, methodology, tools etc (with minor adjustments) can be applied to each project.

Project title: Cash Assistance and Resilience for Ukraine Phase III (C4U3)

Brief Description: The CARE4U3 project is a multisectoral intervention supporting conflict- affected households, covering Shelter, MHPSS and protection sectors.

Objectives /Outcomes:

Overall objective: Enhance the resilience, safety, and well-being of conflict-affected vulnerable populations through integrated support services that include shelter, protection, MHPSS, and cash assistance, while promoting sustainable living conditions and access to rights and services.

Outcomes

1. Shelter: Targeted beneficiaries live in safe, dignified, and adequate living spaces. Including through: household level shelter repairs, collective site repairs.

2. Protection: Beneficiaries, especially from vulnerable and diverse groups, experience enhanced protection, improved capacity to cover their basic needs, increased access to rights and services, and demonstrate reduced reliance on negative coping mechanisms, contributing to their overall safety, dignity, and empowerment. Including through: case management, cash for protection, legal advice, improved inclusion for PWD to caritas services, emergency MPCA.

3. MHPSS: Improved personal/interpersonal well-being and mental health of targeted individuals and communities. Including through PSS, peer to peer support, individual counselling.

Project Partners: The project is delivered with and by our project partner Caritas Ukraine

Duration: 07.2024 – 08.2025 (14 month)

Location: Zaporizka, Kharkivska and Dnipropetrovska Oblasts

Target: Circa 7,500 conflict affected individuals

Project title: Resilience through Integrated Services for Social Empowerment (RISE)

Brief Description: The RISE project is a multisectoral intervention supporting conflict- affected households, covering Shelter, MHPSS and protection sectors.

Objectives /Outcomes:

Overall objective: Enhance the resilience, safety, and well-being of vulnerable populations affected by conflict in Southern Ukraine through integrated support services that include shelter, protection, and MHPSS, while promoting sustainable living conditions and access to rights and services

Outcomes:

1. Shelter: Targeted beneficiaries live in safe, dignified and adequate living space, through household level shelter repairs.

2. Protection: Beneficiaries from diverse groups can cover their basic needs, access services, empower themselves and reduce negative coping mechanisms, leading to better Protection outcomes. Including through: case management, cash for protection, legal advice, emergency MPCA.

3. MHPSS: Improved personal/interpersonal wellbeing and mental health of targeted individuals and communities. Including through PSS, peer to peer support, individual counselling

Project Partners: The project is delivered with and by our project partner Caritas Spes Ukraine.

Duration: 07.2024 – 09.2024 (15 months)

Location:Odeska, Mykolaivska, Khersonska (to a limited extent).

Target:Circa 7,500 conflict affected individuals

3. Key issues / Evaluation Questions

3.1 Scope of the Evaluation

  • The evaluation covers all components of the projects and the full project phase. It is expected to review and assess all the outcomes and key outputs that have been produced in this period.
  • The consultant and their team are required to design, plan, and carry out data collection, conduct data analysis and present the projects’ final evaluation in line with the OECD-DAC Standards for Development Evaluation of relevance, coherence, efficiency, effectiveness, impact and sustainability. These criteria are to be chosen subjectively based on relevance.
  • The study is to be conducted in close cooperation with our implementing partners, and consider key project stakeholders in the target oblasts, as well as other actors directly or indirectly involved in the projects
  • The consultant and their team are required to familiarize themselves with all documents relevant to the projects.

3.2 Focus and Evaluation Questions

In line with the results-based approach, the evaluation will focus on identifying and analysing results through addressing key questions related to the evaluation criteria and the achievement of the outcomes/objectives of the project using mainly, but not only, the indicators in the logical framework of the project. The evaluator may propose adjustments and related rationale in the inception report and submit it for approval to CACH. Throughout the evaluation, the following themes should be considered:

  • Social Cohesion
  • Inclusion of vulnerable groups (Persons with disabilities, Veterans and their families, Elderly people, Households headed by women, Children, Socioeconomically, Ethnic minorities (i.e. Roma/Roma communities) vulnerable households – precise vulnerable groups to be confirmed during inception phase)
  • Partnership and localisation

Examples of key questions are listed below, however, the precise key questions will be developed and agreed between the service provider and CACH.

RELEVANCE:

The extent to which the intervention’s objectives and design respond to beneficiaries’ global, country and partner/institution needs, policies and priorities, and continue to do so if circumstances change.

  • To what extent does the program respond to the identified needs, priorities, and preferences of affected populations in the current context?
  • Was the intervention designed (and delivered) in ways that respond to the needs and priorities of all genders, age groups, people with disabilities, displaced and other vulnerable groups?

COHERENCE:

The compatibility of the intervention with other interventions in Ukraine, the sector or institution.

Internal coherence:

  • To what extent are the Shelter, MHPSS and Protection components fully integrated into the Caritas humanitarian response? What improvements can be made in their implementation or modality of response?
  • What are the synergies – or lack thereof –between MHPSS and Protection sectors within the response?

External coherence:

  • How well is the program aligned with national strategies, technical or cluster standards and coordinated with other actors (at the national and sub-national level) e.g. through the OCHA coordination mechanisms, or otherwise.

EFFECTIVENESS:

The extent to which the intervention delivers, or is likely

  • To what extent has the planned impact been achieved? Which major factors have influenced the achievement or non-achievement of the expected impact?
  • During the implementation, did the project establish and apply effective systems and processes for
    • Accountability,
    • MEAL and
    • Risk management?
  • What strategies or approaches were adopted to promote conflict- sensitive programming during the implementation?
  • Which activities could be considered best practices and can be possibly replicated or scaled-up in other projects?
  • Did the project adapt effectively to changes in context or emerging needs?

EFFICIENCY:

The extent to which the intervention delivers, or is likely to deliver, results in an economic and timely way.

  • Have resources (financial, human, technical support, etc.) been allocated strategically and sufficiently by Caritas to achieve the project outputs/outcomes? How could similar results be achieved with lower costs or in less time?
  • Is there a clear understanding of roles and responsibilities by all parties involved (partner national and regional offices, Caritas Switzerland)?
    • If not, what could be improved?
  • Did the project receive sufficient administrative and technical support from Caritas Switzerland
    • What support was the most effective?
    • What was less useful?
    • How could this be improved?
  • Did the project teams receive sufficient technical support from internally within their organisations?
  • How well aligned are the internal processes compared with quality standards for Shelter, MHPSS and Protection services? Are relevant SOPs in place and fit for purpose?
  • To what extent are the project supported tools, methodologies, and new or improved policies institutionalized? What gaps should be addressed?

IMPACT:

The extent to which the intervention has generated or is expected to generate significant positive or negative, intended or unintended, higher- level effects.

  • What were the particular activities of the project that made a difference? What was less and the more impactful?
  • Did the project contribute to any unintended harmful or negative outcomes?
  • Were there equal impacts for different genders, age and other vulnerable groups?
  • If there were differences, why did these differential impacts occur?
  • Can the project’s approach or parts of it, and results be replicated or amplified by national partners or other actors
  • Has the project contributed to reducing or exacerbating tensions between relevant community groups?

SUSTAINABILITY:

The extent to which the net benefits of the intervention continue, or are likely to continue

  • To what extent are the results of the intervention likely to have a long-term, sustainable positive effects?
  • Has the project contributed to alignment between social protection systems Government of Ukraine and the wider humanitarian response?
  • Which positive, lasting effects and programmatic changes can be perceived as a result of the capacity building activities?

4. Evaluation Methods and Process:

The evaluator can be based inside Ukraine or outside, but primary data collection shall be expected on samples of beneficiaries in Ukraine from each outcome to inform the evaluation. This can be done through telephone interviews, Focus Group Discussions (FGDs), and other methods that the evaluator can propose, security situation allowing.

It is expected that the evaluations will employ mainly qualitative data collection methods, such as analysis of documents, structured key informant interviews, focus group discussions, and/or semi-structured interviews (face-to face or remote modality). All data collected needs to be disaggregated by gender, age, disability status (if feasible), and displacement status. The consultant(s) shall be sensitive to the age of beneficiaries, gender disparities, ethnicity and disability status, and be prepared to probe into unintended outcomes of the projects.

The methodology to appear in the proposal should include but not limited to:

  • Review of key project documents including: proposal, project reports, MEAL reports and other related documents.
  • Review of process and outcome monitoring data as well as other data collected in the course of the interventions (e.g. primary data for outcome and process monitoring, etc.).
  • Interview key staff (Caritas Ukraine, Caritas Spes and Caritas Switzerland) associated with the projects and other stakeholders with each meeting documented (face-to-face or remote modality).
  • Meet and interview project beneficiaries and document all discussions (if feasible).

While maintaining independence, the evaluations shall use a participatory approach and be based on the views of all key stakeholders (specifically: beneficiaries, CUA staff, CS staff, CACH staff, coordination bodies). The final methodology, tools, and samples to be used are to be determined by the evaluator in the form of the inception report and approved by CACH.

The evaluator will be provided with all relevant documents, in particular Annual reports, earlier Evaluations, Monitoring Data, Country Program, IC Strategy and other relevant documents.

5. Deliverables:

  • Inception report (3-5 pages): detailing work plan approach, methodology and timeframe for field work and logistical arrangements. The report should include description of the evaluation methodology and instruments to be used. One inception report may be provided, however, where relevant information should be disaggregated by project.
  • Tools to be used for data collection.
  • Two Field work debriefing meetings (one per project): Present and discuss key initial findings with project staff, clarify open questions.
  • Two Draft evaluation reports (one per project):
  • After the data collection phase, the evaluator(s) will develop a draft evaluation report per project.
  • The total length of each report should be a maximum of 20 pages for the main report, excluding annexes and executive summary. Each report should be sent as one complete document. Photos, if appropriate to be included, should be inserted using lower resolution to keep overall file size low. The draft report will be circulated to stakeholders for their review on factual errors and clarifications. Comments from stakeholders will be presented to the evaluator(s) for integration into the final report as appropriate or to document why a comment has not been included.
  • Two final reports (one per project):
  • In English, not exceeding 20 pages, including an executive summary of 1-2 pages. Presents the evaluation results as defined in the ToR. Findings (statement of facts), conclusions, lessons learned and recommendations are presented and structured in a logical line of arguments (recommendations deriving from conclusions; conclusions sourced in the facts and findings). Annex 1 demonstrates the components the evaluation report shall include as a minimum
  • The evaluator(s) will finalize and submit the final report to the assigned responsible staff of CACH who will solicit input and feedback from relevant stakeholders. The report should address all comments and/or provide explanations why comments were not considered.
  • Two Workshops: The consultant will provide two workshops (one per project) on the key findings, conclusions, and recommendations for future programming. The workshop will primarily be conducted in a discussion format with key staff.

All reports, including drafts, will be written in English. Ownership of data from the evaluation rests with CACH. The copyright of the evaluation report will rest exclusively with CACH. Use of the data for publication and other presentations can only be made with the written agreement of CACH.

Upon request, the results of this evaluation will be made available to any interested third parties.

6. Schedule:

Expected timing of the assignment: Given the timeline for delivery of some project components, data collection during the month of August or early September is preferred. Therefore, we would wish to commence the evaluation process as soon as possible following the conclusion of the procurement process. The final report should be completed at latest by 31 October. An indicative timeline is listed below – however the evaluator is free to suggest their own schedule.

Week 1-2 Inception phase:

  • Inceptionmeeting
  • Desk review of project-related documents
  • Short inception report
  • Preparationof the fieldphase

Week 3-5 Field phase:

  • Beneficiarysurvey by thefield team
  • FGDs with beneficiaries by the fieldteam
  • KIIs and/or FGDs with the projectstaff by the evaluator (translation needed)

Week 5-7 Reporting phase:

  • Data analysis
  • Debriefingworkshop, presentation of most important findings
  • Draft evaluation report and feedback round
  • Finalization of evaluation reportThe evaluation key dates will be agreed with CACH.

7. Evaluation Team / Qualifications

  • Consultant (individual or company) with relevant professional and academic background and proven experience in humanitarian evaluation.
  • Technical expertise in one (preferably more) of the following technical areas: Shelter, MHPSS and Protection.
  • Familiarity with the Ukraine context is preferred
  • Ability to set up or draw on a team of local enumerators to perform the primary data collection with some but minimal support from Caritas (e.g. organizing FGDs, conducting high-quality household surveys).
  • Experience working in humanitarian contexts with a volatile security situation.
  • Excellent analytical skills, presentation and report writing skills.
  • Excellent oral and written language skills in English.

Knowledge of Ukrainian language a distinct advantage.

8. Logistics:

  • The consultant will receive a security briefing from Caritas – and will be required to abide by Caritas security protocols when on site with Caritas, or interacting with Caritas projects/beneficiaries. Further details will be clarified at the time of contracting. Medical and evacuation insurance etc is the responsibility of the consultant, and is not provided by CACH.

9. Budget & payment schedule:

Proposed:

  • 40% of the agreed fee will be paid on approval of the inception report.
  • 60% of the agreed fee on approval of the final reports.

10. Mandatory:

The evaluation is to be carried out under the mandate of Conor Dolan, Caritas Switzerland with whom the evaluator should discuss any technical and methodological matters. The mandator must be timely informed about upcoming difficulties and any adaptation of the evaluation. All draft and final outputs, including supporting documents, analytical reports and raw data should be provided to Conor Dolan in electronic version compatible with MS Word.

How to apply

11. Application procedure

Application Deadline: Latest by 25 July 2025, 17:00 CET.

  • Application Process: Please send an e-mail with your full application package in English with the subject header “Evaluation Offer” to the following email: ukrainecrisis@caritas.ch. Any queries may be submitted to the same address.

Evaluators intending to apply must provide the following information:

  1. A short technical offer describing evaluation methodology to be used in conducting the evaluations to include:
  • Proposed activities, time frame and total level of effort.
  • Ethical considerations and gender sensitive evaluation approaches.
  • Evidence that the evaluator has the experience to conduct the evaluation (e.g., a list of previous evaluations that are relevant in relation to the context and subject-matter of this assignment that can be highlighted)2. A financial offer accompanying the technical offer stating the daily professional fees expressed in CHF, exclusive of travel or field visit costs, translation, which should be outlined separately.3. CV(s); team composition, expertise, and level of effort of each proposed team member and a description of how the evaluator(s) skills, qualifications and experience are relevant to the required qualifications of this assignment.4. A statement confirming the availability of the candidate(s) to conduct this assignment within the intended timeframe.5. If available or publicly shareable, two evaluation reports of related evaluations with explicit acknowledgement of the applicant(s) role

12. Annex

1. Annex 1: Standard Format for Evaluation Report