MID TERM EVALUATION OF AN URBAN DRR / READINESS PROJECT (Madagascar) (2020-2022) at German Red Cross

1. Background

The project “Increasing readiness of CRM and humanitarian disaster risk reduction to address humanitarian needs of vulnerable people exposed to natural hazards and disasters in informal urban settlements in Antananarivo, Madagascar” is implemented by the Malagasy Red Cross (CRM) with support of the German Red Cross (GRC) and funded by the German Federal Foreign Office (GFFO). The project started in April 2020 and will end 31.12.2022. the project focuses on 2 main pillars:

1 (Urban) Readiness of CRM aiming to support CRM in strengthening its role as a humanitarian actor in Antananarivo. Main objectives of this pillar are:

• The HNS is enabled to actively find and to manage its roles and responsibilities within the humanitarian system on various levels in an integrated and coordinated manner.
• The host national society has the operational capacities in terms of mechanisms, procedures and protocols to effectively respond to crisis and disaster situations within their mandate.
• The host national society is provided with the necessary means to activate plans and mechanisms in a timely and effective manner

2 Urban humanitarian Disaster Risk Reduction aiming to support communities/settlements in Antananarivo which are regularly exposed to especially floods and landslides. Main objectives of this pillar are:

• Target communities have increased knowledge and awareness about hazards and climate change and know the developed [or existing] community action plan.
• The communities have carried out small scale mitigation measures and know how to maintain them.
• The targeted schools have awareness, knowledge, functioning community-based structures and procedures for effective Disaster Risk Reduction.

2. Objectives of the Mid-term Evaluation

2.1 Purpose

The mid-term evaluation should analyse and comment on the achieved status of the project and evaluate how effective and efficient the project has so far achieved its specific objectives. The mid-term evaluation should be carried out based on the approved project proposal, in particular the logical framework, outlining the objectives of the projects and the operational work plans. Based on its results, the evaluation will provide CRM and GRC with recommendations for the further project implementation and/or recommend modifications.

2.2. Users of the evaluation

The primary audience of the findings of the evaluation are the i) CRM ii) GRC in Madagascar and at HQ in Berlin, Germany, iii) relevant stakeholders, iv) German Federal Foreign Office (GFFO).

2.1. Evaluation scope

The purpose of this mid-term evaluation is to support the GRC and the CRM to assess the status of the overall implementation of the project, and to provide recommendations, suggestions for modifications as well as to drawing lessons learned for the further implementation of the project.

2.2. Evaluation criteria: including specific evaluation questions related to the project

The evaluation is intended to assess the following questions:

a) Appropriateness and relevance of the project activities

• To what extent are the objectives of the project met?
• Are the activities and outputs of the project consistent with the intended impacts and effects?
• Are the outputs/outcomes consistent with the beneficiaries’ actual needs?
• How relevant are activities which have not been conducted, yet? Are there (modified/other) activities which would suit the achievement of the expected results and outcomes better?

b) Effectiveness

• Effectiveness of the project in achieving its objectives / To what extent are the objectives likely to be achieved?
• What are/were the major factors influencing the achievement or non-achievement of the objectives?
• To what extent could the intended target group be reached so far?

c) Efficiency

• Are the objectives achieved on time?
• Are/Were appropriate human resources (skills, experience and seniority) available to the project in key areas of management, coordination, support services, technical project design and implementation?

d) Impact

• What is the impact of the project so far, at all urban levels as well as within the CRM?
• What is the impact of the project on the different target sectors: Readiness, WASH, DRR?
• What real difference has the activity made to the reached people on neighborhood level and on partners level and to the CRM overall and specifically on governance, staff, and volunteers?

e) Sustainability and Connectedness

• Potential sustainability within the CRM and Antananarivo, including capacity development?
• Which measures were so far implemented to achieve sustainability?
• How well is the project integrated into the wider National Society strategies and priorities?

f) Operations Continuity during the Covid-19 Pandemic

• How have activities of the work plan been adapted to the context of the Covid-19 pandemic?
• What has been the impact of the Covid-19 pandemic on project execution to date?

g) Coherence and Coordination

• In how far activities are coordinated with other (humanitarian) actors and stakeholders in the region and the same area of intervention?**3. Evaluation Design and Methodology**

It is essential that the consultant uses diverse sources of information in the process, at various levels. The evaluation methodology will include the following (but is not restricted to):

(i) Briefing with the project team of GRC and CRM

(ii) Desk review of project documentation (project proposal, reports etc.)

(iii) Interviews with relevant GRC, CRM and IFRC country office and HQ staff

(iv) Interviews with CRM personnel (staff and volunteers) in selected project areas

(v) Interviews with other relevant stakeholders involved, such as the BNGRC the Antananarivo Municipality (CUA) and the Municipal-level Education authorities (CISCO).

(vi) Interviews of beneficiaries.

4. Evaluation Process:

4.1. Reporting: Consultant Deliverables

Different deliverables are expected, as outlined below:

4.1.1. Inception report (max. 5 pages), including

• tentative list of persons/stakeholders to be interviewed
• feedback/amendment of the ToR – suggestions for ToR amendment if and where necessary
• status of the evaluation preparation (time table, team, distribution of tasks, reporting)
• evaluation design (chosen methods, approach, steps for their implementation)
• tools for the implementation (questionnaires, data processing and analysis etc.)
• draft implementation plan for the evaluation

The inception report will be discussed with GRC and the evaluators. Any changes of the ToR need an agreement of both parties because they might change the conditions and thereby the contract between GRC and the evaluators.

4.1.2. Validation Workshop: Representatives of stakeholders and the evaluators will come together in the (virtual) validation workshop. The workshop will be organized to discuss and validate findings, lessons learnt, and recommendations proposed by the evaluator/s. Stakeholders might formulate additional recommendations if necessary. Possible content of an evaluation workshop:

• Presentation and discussion of the preliminary evaluation report
• Validation of lessons learnt and recommendations by all stakeholders
• Collection of additional observations or recommendations

It is expected that the evaluators present a structure for the workshop as part of their preliminary report. GRC/CRM and partners are responsible for the workshop preparation and all related logistics.

4.1.3. Final report: The final evaluation report should consider the validation of the stakeholders during the final workshop and has to be delivered latest 2 weeks after the workshop.

All consultant works, inception-, preliminary- and final reports have to be delivered in English or French.

The report will have to be approved by German Red Cross (GRC Head of Office in Antananarivo and Country Manager at GRC HQ Berlin).

The final report should, as a minimum, include the following elements:

• Key data of the evaluation (from the inception report)
• Executive summary – a tightly drafted, to-the-point, free standing document (around 5 pages max) with the following, fixed structure:
• Short project description
• Key questions of the evaluation
• Key findings (Structured (if applicable) along the OECD DAC criteria: Relevance / Effectiveness / Efficiency / Sustainability / Impact)
• Lessons learned
• Major project-specific recommendations
• Introduction – incl. purpose of the evaluation, scope, key questions, short description of the project to be evaluated and relevant framework conditions.
• Evaluation design and methodology
• Key findings with regards to the questions pointed out in the ToR
• Conclusions based on evidence and analysis
• Recommendations as expected in the ToR, which are relevant, feasible and targeted to the respective audience
• Lessons learnt for further project implementation
• Annexes (ToR, list of consulted persons/organisations, consultes documentation, literature, etc.)

Key findings, conclusions and recommendations should be presented in a clear and transparent way, possibly put next to each other in a table to demonstrate the logic.

The report can be extended by the evaluator/s by additional points if necessary.

4.2. Timeline and contract duration

Overall, 21 consultant working days (excluding travel days) are considered for this evaluation, tentatively starting end of January 2022

Day – Task – Responsible Person – Day/person

1 – Introductory meeting with evaluation team – GRC and consultants – 1

4 – Analysis of relevant documents – Consultants – 3

6 – Delivery of inception repor t – Consultants – 2

7 – Kick-off meetings with project team – GRC and consultants – 1

13 – Qualitative and quantitative data collection – Consultants – 6

15 – Preliminary report delivery – Consultants – 2

16 – Workshop report validation – Consultants – 1

20 – Final report preparation – Consultants – 4

21 – Report reception, feedback and final discussion – Consultants and GRC – 1

24 – Travel days (international) – if possible/needed -* Consultants – 3

Total 21-24 days

*in case of international travel, a quarantine of at least 2 days (up to 5 days) needs to be taken.

5. Responsibilities and tasks

5.1.GRC

• Will be responsible for organising and covering transportation within Antananarivo and accommodation during the evaluation.
• Will support the consultant with the necessary working material for workshops.
• Will facilitate the organization of the workshop or other activities depending on the proposed methodology.
• Will give a security briefing to the consultant.
• Will sign the contract with the consultant and cover the consultancy fees as per the contract.
• Will maintain coordination with the CRM.

5.2. Evaluator(s)

• Will conduct the evaluation as outlined in the TORs / inception report.
• Will specify arrangements required to organize the lessons learned workshop or any other activity in close collaboration with GRC and CRM.
• Will follow the timeframe agreed and shall communicate any unforeseeable change as soon as possible.
• Will submit all deliverables (inception report, preliminary and final evaluation) to GRC as per the agreed timeline.
• Will revise the draft, based on the comments from CRM and GRC.

6. The evaluation team / consultant

Minimum Qualifications

Obligatory

• At least 5 years of experience as consultant/evaluator
• In-depths knowledge of DRR (in the urban context is an advantage)
• Knowledge of Readiness/Preparedness projects is an advantage
• High analytical, writing and workshop facilitation skills
• very good command of written and spoken English and French

Preferable

• Knowledge of/experience with the Red Cross Red Crescent Movement highly desirable

7. Application

Interested consultants should submit their dossier by 12^th of December 2021.

The application should include:

• Curriculum Vitae
• Brief letter of motivation summarising relevant experiences and qualifications for the consultancy
• Legal status to conduct this activity as a consultant in Madagascar.
• Technical proposal: the technical proposal should contain the following elements:

a) A chronogram for how the consultant proposes to complete all tasks

b) A description of the proposed methodology

• Financial proposal:

a) The financial proposal is inclusive of daily fees and applicable taxes

Please indicate in your financial proposal: daily fees + % of taxes = overall fees

b) the validity of your offer (at least 45 days) and payment terms

c) a tax/VAT/NUIT no.

In case a consultant is not located in Madagascar, but needs to travel there, the flight tickets to/from Madagascar should be included in the financial proposal. Insurance and per diem/DSA are not reimbursed by GRC, thus these should be factored into the daily fee. The financial proposal needs to be in EUR.

(GRC will cover the consultant´s accommodation and travel costs within Antananarivo).

GRC will not consider incomplete dossiers.

8. Evaluation Criteria of the Application

The decision for the award of contract will be determined via method credit points divided by total price; ratio 50/50%. Credit points will be added up through the following criteria: A =Technical Proposal, B = Experience as a consultant and knowledge about DRR and urban contexts, C = Comprehension of the ToRs.

Weight:

A =Technical Proposal = 60%,

B = Experience as a consultant and knowledge about DRR/Urban contexts = 35%,

C = Comprehension of the ToRs = 5%).

Applied Scoring system:

5 points: Excellent

4 points: Very good

3 points: Good

2 points: Sufficient

1 point: Bad

0 points: Insufficient