MID TERM EVALUATION OF THE FBF PROJECT IN SOUTHERN AFRICA (Mozambique, Namibia, Lesotho) (2020-2022)

TERMS OF REFERENCE

CONSULTANCY: MID TERM EVALUATION OF THE FBF PROJECT

IN SOUTHERN AFRICA (Mozambique, Namibia, Lesotho) (2020-2022)

Background

Forecast-based Financing (FbF) has emerged with the objective of using increasingly accurate forecasts to take action before disasters occur thereby reducing their impact on the vulnerable population. FbF has established mechanisms to automatically allocate funding for early action once forecasts indicate a high probability of an extreme weather event. Based on forecast information and risk analysis, FbF releases humanitarian funding for pre-agreed activities, or early actions, to be taken in the window of time between a forecast and a potential disaster. Early actions in FbF aim to prevent or reduce the humanitarian impact of extreme weather events, enhance preparedness activities and prepare for response. The Climate Centre and German Red Cross have established 13FbF pilot projects in various countries around the world (see here). The regional FbF-project to be evaluated is implemented in Mozambique (3^rd phase), Namibia and Lesotho. The project focuses on the development of Early Action Protocols for droughts in Mozambique, Namibia and Lesotho as well as on the maintenance of Early Action Protocols for floods and cyclones in Mozambique. The project is running between 2019 and 2022.

Objectives of the Mid-term Evaluation **

2.1 Purpose

The mid-term evaluation should analyse and comment on the achieved status of the project and evaluate how effective and efficient the project has so far achieved its specific objectives. The mid-term evaluation should be carried out based on the approved project proposal, in particular the logical framework, outlining the objectives of the projects and the operational work plans, as well as on the feasibility study. Based on its results, the evaluation will provide CVM/NRCS/LRCS and GRC with recommendations for the further project implementation or recommend modifications.

2.2. Users of the evaluation

The primary audience of the evaluation findings are the staff of i) CVM/NRCS/LRCS as well as in the headquarters, ii) GRC in Maputo and at HQ in Berlin, iii) relevant institutions iv) German Federal Foreign Office (GFFO).

2.1. Evaluation scope

The purpose of this mid-term evaluation is to support the GRC and the CVM/NRCS/LRCS to assess the status of the overall implementation of the FbF project, and to provide recommendations, suggestions for modifications and drawing lessons learned for the further implementation of the project.

2.2. Evaluation criteria: including specific evaluation questions related to the project

More specifically, the evaluation is intended to assess the following questions:

a) Appropriateness and relevance of the FbF activities in the southern African context

To what extent are the objectives of the project met?
Are the activities and outputs of the project consistent with the intended impacts and effects?
Are the outputs/outcomes consistent with the beneficiaries’ actual needs, including appropriateness of the EAPs developed for CVM/NRCS/LRCS?
Is the degree of confidence of the forecasts and thresholds (triggers) that are considered in the EAPs good enough?
Quality and appropriateness of the Early Actions proposed in the EAPs and tested in drills or activations
How relevant are activities which have not been conducted, yet? Are there (modified/other) activities which would suit the achievement of the expected results and outcomes better?

b) Effectiveness

Effectiveness of the project in achieving its objectives / To what extent are the objectives likely to be achieved?
What were the major factors influencing the achievement or non-achievement of the objectives?
To what extent could the intended target group be reached so far?

c) Efficiency

Efficiency of the coordination mechanisms within the Red Cross movement and with other stakeholders, at regional, national and provincial levels
Are the objectives achieved on time?
Are/Were appropriate human resources (skills, experience and seniority) available to the programme in key areas of management, coordination, technical programme design and implementation?

d) Impact

Impact of the project so far, at all levels (municipal, provincial, national and regional) as well as within the CVM/NRCS/LRCS.
What real difference has the activity made to the beneficiaries?

e) Sustainability and Connectedness

Potential Sustainability within the CVM/NRCS/LRCS, including capacity development, as well as within the broader DRRM framework in the countries:

To what extent has FbF been integrated in CVM/NRCS/LRCS strategies and operating procedures?

To what extent has FbF been integrated in governmental strategies and operating procedures?

Which measures were so far implemented to achieve sustainability?
How well is the project integrated into the wider National Society?

f) Operations Continuity during the Covid-19 Pandemic

How have activities of the work plan been adapted to the context of the Covid-19 pandemic?
What has been the impact of the Covid-19 pandemic on project execution to date?

g) Coherence and Coordination

In how far activities are coordinated with other (humanitarian) actors and stakeholders in the region and the same area of intervention?
Evaluation Design and Methodology

It is essential that the consultant use diverse sources of information in the process, at various levels. The evaluation methodology will include the following (but is not restricted to):

(i) Briefing with the project team at GRC and CVM/NRCS/LRCS Headquarters

(ii) Desk review of project documentation (project proposal, reports, EAPs, test activation reports, activation evaluation reports etc.)

(iii) Interviews with relevant GRC, RCCC and IFRC country office and HQ staff

(iv) Interviews with CVM/NRCS/LRCS personnel (staff and volunteers) at Headquarters and in selected project areas (at least two regions, with chapters involved in both Cyclone and Flood EAP)

(v) Interviews with the 510 Initiative of the Netherlands RC for the triggers development

(vi) Interviews with other relevant stakeholders involved, such as the World Food Programme, the FAO or members of the Technical Working Group

(vii) Interviews with partners in/representatives of DRM authorities and Hydromet Services, at national and provincial levels

(viii) Interviews of beneficiaries from the test activation, if possible

Evaluation Process:

4.1. Reporting: Consultant Deliverables

Different deliverables are expected, as outlined below:

4.1.1. Inception report (max. 5 pages), including

tentative list of persons/ organisations to be interviewed
feedback /amendment of the ToR – suggestions for ToR amendment if and where necessary
status of the evaluation preparation (time table, team, distribution of tasks, reporting)
evaluation design (chosen methods, approach, steps for their implementation)
tools for the implementation (questionnaires, data processing and analysis, etc)
draft implementation plan for the evaluation

The inception report will be discussed with GRC and the evaluators. Any changes of the ToR need an agreement of both parties because they might change the conditions and thereby the contract between GRC and the evaluators.

4.1.2. Validation Workshop: Representatives of stakeholders and the evaluators will come together in the (virtual) validation workshop. The workshop will be organized to discuss and validate findings, lessons learnt, and recommendations proposed by the evaluator/s. Stakeholders might formulate additional recommendations if necessary. Possible content of an evaluation workshop:

Presentation and discussion of the preliminary evaluation report
Validation of lessons learnt and recommendations by all stakeholders
Collection of additional observations or recommendations

It is expected that the evaluators present a structure for the workshop as part of their preliminary report. GRC/CVM and partners are responsible for the workshop preparation and all related logistics.

4.1.3. Final report: The final evaluation report should consider the validation of the stakeholders during the final workshop and has to be delivered at latest 2 weeks after that workshop.

All consultant works, inception-, preliminary- and final reports have to be delivered in English.

The report will have to be approved by German Red Cross (GRC Head of Office in Maputo, GRC project delegate and Country Manager at GRC HQ Berlin):

The final report should, as a minimum, include the following elements:

§ Key data of the evaluation (from the inception report)

§ Executive summary – a tightly drafted, to-the-point, free standing document (about 5 pages max) with the following, fixed structure:

Short project description
Key questions of the evaluation
Key findings

(Structured (if applicable) along the OECD DAC criteria: Relevance / Effectiveness / Efficiency / Sustainability / Impact)

Lessons learned
Major project-specific recommendations

§ Introduction – incl. purpose of the evaluation, scope, key questions, short description of the project to be evaluated and relevant framework conditions.

§ Evaluation design and methodology

§ Key findings with regards to the questions pointed out in the ToR

§ Conclusions based on evidence and analysis

§ Recommendations as expected in the ToR, which are relevant and feasible and targeted to the respective audience

§ Lessons learnt for further project implementation

§ Annexes (ToR, list of consulted persons/organisations, consultes documentation, literature, etc.)

Key findings, conclusions and recommendations should be presented in a clear and transparent way, possibly put next to each other in a table to demonstrate the logic.

The report can be extended by the evaluator/s by additional points if necessary.

4.2. Timeline and contract duration

Overall 27 consultant working days (excluding travel days) are considered for this evaluation, tentatively starting in October/November 2021

Day, Task, Responsible person, Days/person
1, Introductory meeting with evaluation team, GRC and Consultants, 1
4, Analysis of relevant documents, Consultants, 3
6, Delivery of inception report, Consultants, 2
7, Kick-off meetings with FbF teams in Mozambique, Namibia and Lesotho, GRC and consultants, 1
19, Qualitative and quantitative data collection, Consultants, 12
21, Preliminary report delivery Consultants, 2
22, Workshop report validation Consultants, 1
26, Final report preparation Consultants, 4
27, Report reception and final discussion, Consultants and GRC, 1
31, Travel days (national and international) – if possible, Consultants, 4
Total 31 days

The consultancy can be conducted in Mozambique and remotely in Lesotho and Namibia. In case travel restrictions allow, the consultancy can be complemented with travels to Lesotho and/or Namibia. Depending on the development of the Covid-19 pandemic in the three countries, the consultancy can also be conducted in Maseru (Lesotho) and remotely in Mozambique and Namibia, with travels to Mozambique and/or Namibia if the evolving context allows.

5. Responsibilities and tasks

5.1.GRC

Will be responsible for organising and covering transportation within the country/countries in which the consultancy is implemented, transportation between the countries (Namibia, Lesotho and Mozambique) and accommodation during the evaluation.
Will support the consultant with the necessary working material for workshops.
Will facilitate the organization of the workshop or other activities depending on the proposed methodology.
Will give a security briefing to the consultant.
Will sign the contract with the consultant and cover the consultancy fees as per the contract
Maintain coordination with the CVM/NRCS/LRCS

5.2. Evaluator

Will conduct the evaluation as outlined in the TORs / inception report;
Will specify arrangements required to organize the lessons learned workshop or any other activity in close collaboration with GRC and CVM/NCRS/LRCS;
Will follow the timeframe agreed, and shall communicate any unforeseeable change as soon as possible;
Will submit all deliverables (inception report, preliminary and final evaluation) to GRC as per the agreed timeline;
Will revise the draft, based on the comments from CVM and GRC.

6. The evaluation team / consultant

Minimum Qualifications

Obligatory

At least 5 years of experience as consultant/evaluator
In-depths knowledge of DRR (in the context of Southern Africa is an advantage)
Knowledge of Forecast-based financing / Early Warning Early Action concepts
High analytical, writing and workshop facilitation skills
very good command of written and spoken English and Portuguese

Preferable

Knowledge of/experience with the Red Cross Red Crescent Movement highly desirable

7. Application

Interested consultants should submit their dossier by 12^thof September 2021.

The application should include:

Curriculum Vitae
Brief letter of motivation summarising relevant experiences and qualifications for the consultancy
Legal status to conduct this activity in Mozambique, Namibia, Lesotho.
Copy of ID, BI, DIRE or Work Visa
Technical proposal: the technical proposal should contain the following elements:

a) A chronogram for how the consultant proposes to complete all tasks

b) A description of the proposed methodology

Financial proposal:

a) The financial proposal is inclusive of daily fees and applicable taxes

Please indicate in your financial proposal: daily fees + % of taxes = overall fees

b) the validity of your offer (at least 45 days) and payment terms

c) a tax/VAT/NUIT no.

In case a consultant is not located in Mozambique or Lesotho, but needs to travel there, the flight tickets to/from Mozambique/Lesotho to and from the country of origin should be included in the financial proposal. Insurance and per diem/DSA are not reimbursed by GRC, thus these should be factored into the daily fee. The financial proposal needs to be in EUR.

(GRC will cover the consultant´s accommodation and travel costs within Mozambique (Namibia and Lesotho as well as between the countries)).

GRC will not consider incomplete dossiers.

Deadline

The deadline of submission of proposals will be on 12^th of September 2021. Please submit your proposals to f.neubauer@drk.de

8. Evaluation Criteria of the Application

The decision for the award of contract will be determined via method credit points divided by total price; ratio 50/50%. Credit points will be added up through the following criteria: A =Technical Proposal, B = Experience as a consultant* and knowledge about Forecast-based financing /Early Warning Early Action, C = Comprehension of the ToRs.

Weight:

A =Technical Proposal = 60%,

B = Experience as a consultant and knowledge about FbF/EWEA = 35%,

C = Comprehension of the ToRs = 5%).

Applied Scoring system:

5 points: Excellent

4 points: Very good

3 points: Good

2 points: Sufficient

1 point: Bad

0 points: Insufficient