External evaluation of the EuroPana Program. At Caritas Switzerland

Caritas Switzerland (CACH) is a non-profit organization founded in 1901, and is a member of the Caritas Internationalis network. CACH is committed to a world guided by solidarity, justice and peace, in which the inviolable dignity and rights of every human being are respected. CACH works to ensure that every person not only has access to food, water, shelter, health, work and education, but is also able to expand their capabilities and opportunities to lead the lives they value. CACH provides professional, effective and efficient help to people in need and poverty, both in Switzerland and abroad, regardless of their political or religious beliefs, gender or ethnicity. CACH is committed to improving the livelihoods and guaranteeing all human rights of migrants, especially vulnerable Venezuelan migrants in South America.

Caritas Germany (DCV) was founded in 1897 and gave rise to all other Caritas organizations in the world. It is an international humanitarian organization that provides humanitarian aid and rehabilitation during and after disasters, supports social programs for children, people with disabilities and the elderly, advocates for women’s empowerment and promotes peace-building activities in Germany and in 165 other countries around the world, working in collaboration with other Caritas organizations in countries in Eastern Europe, Latin America, Africa and Asia.

Cáritas Germany (DCV) and Cáritas Switzerland (CACH) were among the first international humanitarian organizations in the region to respond to the impact of the migration crisis from Venezuela through the EuroPana program and previous projects since 2016.

1. Program Summary

Structure and implementation

The EuroPana Program is implemented through the consortium of Caritas Switzerland (CACH) and Caritas Germany (DCV), organizations that have worked together to respond to the migration crisis that the region is experiencing within the framework of comprehensive human development and the construction of a sustainable peace. This consortium is developed with the Humanitarian Aid Office of the European Commission (ECHO) to address the migration situation that occurs in several countries in the region. In this, assistance and protection actions are carried out in Colombia, Ecuador, Peru, Brazil and Venezuela, as follows:

• In Colombia, National Secretariat of Social Pastoral/Cáritas Colombiana (SNPS/CC), Social Pastoral of Riohacha (SEPAS)
• In Ecuador: Caritas Ecuador
• In Peru: Cáritas Perú and CEDEH (Center for Human Development)
• In Brazil: Cáritas Brasileira and Servicio Pastoral Dos Migrantes (SPM)
• In Venezuela: Cáritas Venezuela

Action locations:

• Venezuela: Mara, Insular Padilla, Guajira, Páez, Ciudad Guayana
• Colombia: Apartadó, Turbo, Necoclí, Caucasia, Riohacha, Maicao, El Bagre, Ipiales, Ocaña and Ábrego.
• Ecuador: Tulcán, Lago Agrio, Ibarra, Quito, Cuenca and Machala
• Peru: Juliaca, Puno, Desaguadero, Zepita, Arequipa, Lima, Tacna, Tumbes
• Brazil: Boavista, Pacaraima, Porto Velho, Rio Branco, Brasileia, Manaus, Cuiabá

Duration

The project thus develops from December 1, 2022 to May 31, 2024 (implementation began on June 1, 2023 in Venezuela and on November 30, 2023 it ended in Colombia).

General objective

Contribute to access to food and fill critical gaps in humanitarian assistance through multipurpose cash transfers to vulnerable Venezuelan citizens and vulnerable host communities in Colombia, Ecuador, Peru and Brazil, particularly for women, children, indigenous populations, people with disabilities and seniors

Specific goal

Provide humanitarian assistance to 59,252 migrants, asylum seekers, returnees and local people.

Results

• Result 1 Multipurpose monetary transfers: Multipurpose monetary transfers for migrants in highly vulnerable situations.
• Outcome 2 Food assistance and livelihoods: Improved access to food and food security of the target population.
• Result 3 Protection: Improve the protection conditions of the migrant population through safe and informed transit.
• Result 4 Shelter and settlements: Improvement of shelter and migrant conditions for beneficiaries in coordination with institutions.
• Outcome 5 Water and sanitation: Access to water and hygiene items to make migration safer.
• Outcome 6 Crisis Modifier: Disaster Risk Reduction/Disaster Preparedness.

2. Objective of the contract to be signed

Hire professional services for the design and implementation of an external evaluation of the program “Promote protection, assistance and food security to migrants, asylum seekers, victims and vulnerable local populations in Colombia, Ecuador, Peru, Brazil and Venezuela. ECHO/-AM/BUD/2022/91036”, so that:

• Analyze the structures, processes, methodologies and approaches, identifying the best practices, weaknesses and strengths of the program in the current context.
• Evaluate the relevance, effectiveness, efficiency and impact of the program.
• Make recommendations that can serve to strengthen local partners and the consortium. This both at the level of joint action and at the operational level, taking into account the best practices and lessons learned at the local level by the partners.

This call is open to all natural and legal persons who consider they have the appropriate profile described here. There is no discrimination based on confession, race, age, gender or any social or economic condition.

3. Requirements and commitments on the part of people interested in participating in the call.

Both individuals and work teams are requested to comply with the professional training, experience, skills, competencies, general and specific obligations of the position, as indicated below:

a) Professional training and experience

• Professional in Social, Human, Economic, Administrative Sciences or related to the development of the position.
• It is desirable to have a postgraduate degree or extensive demonstrable experience in evaluation and/or project management.
• Proven technical capacity and demonstrated experience of at least 5 years in carrying out evaluations of results or impacts of similar humanitarian aid projects.
• Experience in evaluating humanitarian and ECHO-funded projects is desirable.
• Experience in evaluating regional projects and projects in which multiple organizations participate is desirable.
• Experience in survey design, conducting surveys and analyzing statistical data.

b) Specific knowledge

• Extensive knowledge of quantitative and qualitative research methods and techniques and evaluation of social projects.
• Analysis capacity to identify the strengths and areas of improvement, as well as the critical aspects of the analyzed reality and the intervention of the project based on a human rights approach, differential approach (gender, ethnic, population).
• Analysis capacity to identify the impact of the project actions on the participating communities.
• Extensive knowledge in human rights, international humanitarian law, humanitarian aid and comprehensive protection.
• Respect for cultural diversity and religious beliefs.
• Not have family ties with the staff of the main or local partners.
• Not having a direct relationship with personnel hired in the project and not having been employed for this project.
• Knowledge of Portuguese and English (Desirable)
• Office package management.

c) General requirements:

• Carry out activities under the coordination of the EuroPana program coordinating committee.
• Punctuality in compliance with the plans and deadlines stipulated by the contractor for the fulfillment of the tasks in charge, as well as in the conditions and criteria for the delivery of results for the entrusted activities.
• Immediate and permanent availability to execute the tasks in charge.
• Protection, reservation and effective confidentiality of files, programs and other information obtained and produced by virtue of the execution of your contract, which Caritas Switzerland and Germany have not expressly and formally authorized to use for personal benefit, and/or deliver and disseminate to third parties .
• Strict observance of the technical procedures and criteria agreed upon with the financing sources.

d) General Competencies

• Team organization and leadership.
• Methods and methodologies of working with communities and social organizations.
• Ability to work with interdisciplinary teams.
• Ability to relate to different types of population.
• Aptitude and critical, conciliatory, proactive and purposeful attitude.
• Comply with the processes and procedures established in the institution.
• Punctuality in compliance with the deadlines and criteria stipulated by the contractor for the delivery of products and activities established in the contract.

e) Specific tasks

• Design the evaluation work plan and schedule and fully comply with it, in accordance with the times agreed with the contractor.
• Identify and analyze the available documentation on the context of the regions targeted by the project to be evaluated.
• Design, apply and analyze the methodological instruments (qualitative and quantitative) that allow for objective and reliable information on the objectives of the final evaluation of the project.
• Carry out a review of documents and sources of information: narrative request, logical framework, intermediate technical reports, characterization of beneficiaries and aid and care delivery supports – including the monitoring system.
• Participate in meetings to monitor progress with the regional project team for content validation and socialization of methodologies (in person or virtually).
• Travel to the field to visit project implementation municipalities in three countries (Colombia, Venezuela and Ecuador) agreed upon with the coordinating committee. The visit must be carried out to the municipalities targeted by the project, in accordance with the methodology described below.
• Drafting intermediate and final project evaluation reports, with criteria of effectiveness, efficiency, relevance and impact, where good practices, lessons learned and recommendations for future actions are identified.
• Formally present the results of the intermediate and final evaluations in at least one institutional space convened by the contractor.
• Comply with the legal requirements in force in the country for the hiring of personnel assigned to carry out the consultancy.
• Comply with the ethical and behavioral standards established by the Caritas network and the funding agency.

f) Deliverables

Deliverable 1: Evaluation work plan specifying:

1. Methodologies to develop
2. Instruments to use for collecting information
3. Work schedule: detailed schedule of tasks to be performed (work plan).
4. Detailed budget
5. The role and responsibilities of each member of the evaluation team.

For the production of the evaluation plan, the proponent will have access to key program documents such as the technical design, project performance reports and donor guidelines, among others.

Activities to develop:

• Documentary review of key program documents and meeting with supervision team to define and understand the scope of the evaluation.
• Attend coordination and monitoring meetings virtually or in person.
• Review all key program documents, namely, technical design, logical framework, reports presented to the donor, post-distribution monitoring reports, among others.
• Design and agree with the Coordinating Committee of the EuroPana Consortium, a detailed implementation plan that responds to the objectives and expected results of the contract.
• Design qualitative and quantitative data collection tools with their respective methodologies.
  1. Synthesis report of the results, identifying lessons learned, good practices and recommendations for future actions.

Deliverable 2: Interim evaluation report

1. Synthesis report of the impacts.

Activities to develop:

• Field trips to visit project implementation areas in three countries (Colombia, Venezuela and Ecuador), in agreement with the Coordinating Committee. The contractor will provide the corresponding contacts.
• Virtual meetings with the project implementing teams at the national and local levels of the five countries, in agreement with the Coordinating Committee.
• In-person and/or virtual meetings with key informants, including the donor and external actors.
• Collect data and analyze the set of quantitative and qualitative data for the study, based on duly refined information sources.
• Present preliminary versions of all products to obtain feedback from the Coordinating Committee. Collect feedback on all delivered products and make requested adjustments and improvements.
• Compile and analyze secondary sources of information compatible with the objective of this evaluation.

Deliverable 3: Final evaluation report

1. Final evaluation report with its corresponding annexes (information collection and systematization supports).
2. Synthesis report of the results, identifying lessons learned, good practices and recommendations for future actions.
3. Synthesis report of impacts

Activities to develop:

• The final document must have complied with the adjustments after the comments of the EuroPana Consortium Coordinating Committee.
• The final report should not exceed 50 pages (not counting annexes). These will be accompanied by an executive summary, of a maximum of 3-5 pages. Both the final report and the summary must be presented in Spanish and English. The summary should incorporate the essence of the information contained in the report with particular emphasis on the main results, conclusions, recommendations and lessons learned.

The reports must contain the following structure:

1. table of Contents
2. Executive Summary
3. Acronyms
4. Introduction:
   1. Background and objective of the evaluation.
   2. Initial questions and value criteria: definition.
      1. Summary description: of the intervention evaluated, summary of the background, organization and management, actors involved and context in which the intervention is developed.
      2. Methodology used in the evaluation.
         1. Methodology and applied techniques.
         2. Conditions and limits of the study carried out.
         3. Analysis of the information collected, and evidence regarding the issues previously established. Interpretation of the evidence in relation to the evaluation questions stated in title 4 of this document.
         4. Conclusions of the evaluation, in relation to the key evaluation criteria established in title 4 of this document.
         5. Lessons learned that arise from the general conclusions that indicate good practices and that can be extrapolated and provide feedback to the actions of the intervention in execution or for future interventions.
         6. Recommendations derived from the evaluation classified according to the criteria chosen by the evaluation team. Whenever possible, mention the actor (or actors) to whom the recommendation is specifically addressed.

g) Other considerations to take into account

As basic premises of ethical and professional behavior on the part of the hired work team, the following are required:

• Anonymity and confidentiality: The evaluation must respect the right of people to provide information, ensuring their anonymity and confidentiality.
• Responsibility: Any disagreement or difference of opinion that may arise between the team members or between them and those responsible for the intervention, in relation to the conclusions and/or recommendations, must be mentioned in the report. Any statement must be supported by the team or record disagreement about it.
• Integrity: The evaluation team will have the responsibility of highlighting issues not specifically mentioned in the TOR, if this is necessary to obtain a more complete analysis of the intervention.
• Independence: The team must guarantee its independence from the evaluated intervention, not being linked to its management or any element that composes it.
• Data protection: The consulting firm or natural person is obliged to maintain the strictest professional secrecy and confidentiality with respect to the personal data to which it has access due to the evaluation carried out, as well as to diligently comply with the duty of safeguarding. and custody imposed on them by the Organic Law on Protection of Personal Data. These duties will be required of the evaluation company or natural person during the validity of the service provision contract, and even after it has ended for any reason. Likewise, the evaluation company or natural person expressly undertakes to adopt the necessary technical and organizational measures that guarantee the security of the personal data to which it has access and to avoid its alteration, loss, treatment and unauthorized access, taking into account of the state of the technology, the nature of the stored data and the risks to which they are exposed, whether they come from human action or the physical or natural environment, and must comply at all times with the provisions on this point in Organic Law 15/ 1999, of December 13, on Protection of Personal Data.
• Copyright and disclosure: It is clarified that all copyright resulting from the services that are the subject of this call belongs to the entity contracting the evaluation.
• Penalty regime: In the event of delay in the delivery of reports or in the event that the quality of the reports delivered is manifestly inferior to what was agreed upon, the penalties and arbitrations established by the contracting entity in the Administrative Clauses of the Contract will be applicable. contract to sign with the evaluating entity.

4. About the evaluation to be carried out

The external evaluation of the project is proposed according to the following evaluation matrix in which the key issues will be analyzed according to the following criteria, as well as any other aspect that is considered of special relevance during the process of obtaining data and its analysis.

This proposed matrix will be discussed with the evaluation team for its possible expansion and improvement, possibly resulting in another matrix without reducing the selected criteria.

Key evaluation questions and issues.

• Adaptation of the project to the context in which it is executed ( RELEVANCE )
  • Have the beneficiaries participated in the identification of the project? How?
  • Do the identified problems correspond to the proposed objectives?
  • Have the actions developed in the intervention reached the target communities?
  • Does the intervention correspond with the priorities and needs of the beneficiary population? Does the implementation of the project respond to the humanitarian challenges and current situation of the country?
• Has the project achieved its objectives? ( EFFICACY )
  • Was the specific objective well formulated and has it been achieved?
  • Do the planned indicators allow adequate measurement of the achievement of the specific objective?
  • Are the verification sources reliable, timely and reasonably priced?
  • Have the interventions been appropriate to achieve the specific objective?
  • To what extent has the project adapted to external conditions (assumptions) to continue benefiting the project’s target groups?
  • Have corrective measures been taken for unforeseen negative effects suffered by beneficiaries?
• Have you used the right resources to achieve the results? ( EFFICIENCY )
  • Has the project had the necessary/appropriate material and human resources for its correct execution?
  • Have the budgets initially established in the document been respected?
  • Have the planned schedules and times been respected?
  • Have the objectives been achieved efficiently with the available resources?
  • Were the expected results and their indicators well formulated and have they been achieved?
  • Has there been adequate monitoring of project activities and have corrective measures been taken when necessary? (carrying out new activities when new needs arise or after changing an activity).
• Has the project been aimed at causing positive impacts? ( IMPACT)
  • Is there a real positive impact on the participants or is it limited to the technicians and promoters of the local partners and their associations?
  • Are there any expected or unforeseen negative impacts on direct beneficiaries? To what extent has project management taken appropriate corrective measures?
  • Have the most vulnerable groups benefited from the intervention?
  • Are community perceptions more positive since the arrival of the project?
  • Level of knowledge appropriation by the communities about the strategic axes of the project (Food Security, PTM, Water, Shelter and Protection) of which they were beneficiaries.
  • What has been the impact of the project on the situation of women (participation, decision-making, their position in the community, etc.)?

5. Duration and period of execution

The provision of services will begin on February 26, 2024 and will conclude on May 26, 2024.

6. Payment method

Payment will be made like this:

• Deliverable 1 – 30%
• Deliverable 2 – 30%
• Deliverable 3 _ 40%

7. Budget

The maximum budget for payment for this service is EUR 10,000. Due to the nature of the service, all expenses incurred to carry out the results of the contract must be included within this value (including travel). This value must be broken down in the economic proposal

8. Responsibility, safety and security

The Consultant is primarily responsible for his or her own safety. He has the obligation to learn and understand the security situation, political, social and cultural characteristics of the assigned project area. Inappropriate or offensive behavior can put Caritas in a difficult position, disrupt operations and jeopardize the safety of staff. The Consultant must comply with all Caritas policies on safety and security.

Caritas accepts no responsibility for costs arising from illness, injury, disability or death. There is no form of insurance for this assignment.

Caritas is not responsible for unapproved or unaccounted expenses, nor for unapproved activities beyond the approved amounts attached.