Consultancy for Evaluation of ISoLT – Supporting Social Inclusion and Local Governance in the areas most impacted by the Lake Chad Crisis At Norwegian Refugee Council

The Norwegian Refugee Council (NRC) invites qualified firms/ Consultants to tender for the following consultancy service:

Purpose of the evaluation and intended use

The evaluation seeks to assess the project performance by examining its delivery/outcomes a well as the quality of its implementation. As a change-oriented evaluation approach, it is especially attuned to assessing any discrepancies between the logic model and outcomes of the project and what is happening in reality, to analysing strengths and weaknesses, to uncovering obstacles, barriers or unexpected opportunities, and to generating understandings about how the project could be better implemented in the future or in another similar context.

Assessing to what degree the project has been successful in promoting a cross border approach and increasing experience sharing and learning will be part of the evaluation.

The intended direct and indirect users of the evaluation are: advocacy, Implementing Partners, Project teams, Core Competencies Specialists, Heads of programmes, Regional Advisors, Monitoring and evaluation units, key national (ministries, etc) and regional stakeholders in Nigeria and Cameroon, and the donor.

Scope and lines of inquiry

The evaluation will cover project intervention areas and the whole duration of the project since March 2019, in Cameroon and Nigeria. It will consider the strategic and operational aspects that contributed to achievements. Specifically, it will focus on the following elements:

• Analysis of the relevance of the objectives, indicators, activities, implementation strategy and approaches used;

• Measuring and analysis of the level of achievement of the results (outcomes) including processes; focus will be put on the analysis of the deviations between project initial forecasts and progress achieved;

• Identification of unintended positive or negative effects resulting from the implementation of the project;

• Identification and analysis of external factors (social, economic, political, geographical, cultural) that participated in the project implementation and their effect/impact on results and processes.

The evaluation is intended to assess the project’s performance against the following criteria: (i) relevance, (ii) efficiency, (iii) effectiveness, (iv) impact, (v) sustainability, (vi) Non-discrimination and gender (vii) replicability and scalability.

However, we are conscious that the timing of the evaluation does not lend itself to a consideration of (iv) and (v), given that some outcomes will take time to manifest. As such, the evaluation is expected to focus on assessing the likely future impacts and sustainability of the project, as well as ideally proposing a reflective practice approach and a ‘resource light’ approach that can be used for capturing future impacts.

Relevance

To what extent has the project been aligned to NRC priorities, policies and needs of beneficiaries?

To what extent was it designed according to local needs/priorities of the countries involved?

To what extent did it take into account different needs and priorities of boys/men and girls/women? How was the project adapted to meet these various needs?

Effectiveness

How did the project contribute to strengthening the capacity of beneficiaries (business skills, etc)?

How did the knowledge and skills acquired through the project have been used?

How successful was the programme in improving the quality of identification-civil documentation/legal identity, access and learning outcomes, and income of targeted households?

Are there underlying factors beyond the project’s control that influenced its performance?

To what extent has the project contributed to an increase in confidence of communities towards traditional/formal authorities. To what extent is the change similar for women/girls and men/boys?

Impacts

To what extent is the programme contributing to improving the resilience of the most vulnerable displacement-affected populations, especially women, girls and youth? What are the major contributing factors?

Sustainability

To what extent did the project create conditions enabling the changes on children, women, men, and community actors to continue?

To what extent does the project theory (logic model) enable it to be sustainable?

To what extent women/girls and men/boys took ownership of actions and works?

To what extent is the project compatible with local perceptions?

How has local knowledge been valued in the implementation of the project?

Efficiency

To what extent the resources deployed (time, human, financial, material) justify the results?

How well are resources being used in the project?

Non-discrimination-gender equity

To what extent and how are we delivering appropriate and effective interventions for people with disabilities?

To what extent did NRC minimise the negative consequences of its interventions and ensure safety, dignity and wellbeing (and equal access) for women/girls and men/boys?

Replicability and scalability

To what extent will the project work in a different context?

What would happen if we scale up in one context rather than another?

Methodology

The ongoing COVID-19 crisis needs to be taken into consideration in defining the methodology.

This evaluation will follow a quasi-experimental design to generate credible results and evidence about the performance of the project in both countries. It will use a mixed methods approach with a focus on integration. With the assumption that unobserved community and individual characteristics in this project are time invariant and uncorrelated with the outcomes/results, a difference in differences approach can be used to estimate results/outcomes accruing from the project. Control communities for the evaluation will be selected based on their individual and community level similarities with areas of intervention in both countries. Similarity could be based on the following: socio-*economic status, type of education and health facilities, household characteristics, food security indicators and Water sanitation and hygiene*.

The suggested sampling design will be a three-stage cluster sampling using a Probability Proportional to Size (PPS) method. The first stage will include community/village segments, the second stage will target schools, households, etc while the third one will focus on individual units (youth, teachers, farmers, etc). Power calculations^[1] will be prioritised as follows: Power =minimum 0.8, Minimum Detectable Effect = 0.10, ICC=Integrated clustering coefficient. = 0.001, Loss to Follow Up = 10% (especially not controlling but reducing statistical bias resulting from it using requisite analysis).

The evaluation will use a combination of quantitative and qualitative methods and the final methodology will be determined by the evaluator, taking into account the suggestions from the steering committee. The detailed methodology will be elaborated by the evaluator on the basis of this ToRs and documented in an Inception Report, which is subject for review of the Steering Committee. The methods can include but not limited to: desk review of relevant documents (Baseline report, project indicators monitoring data, After Action Review report, etc), interviews with stakeholders, household survey questionnaire, photovoice, Q methodology, most significant changes, online consultations and workshops. Triangulation shall be done in the desk review and analysis phases.

Interview questionnaires will be prepared by the evaluator in consultation with the steering committee. While the evaluator can propose changes in the methodology, any such changes should be discussed and agreed upon with the committee.

All relevant data should be sex-disaggregated and different needs of women and men and vulnerable groups should be considered throughout the evaluation process. Data analysis will be gender responsive and ensure requisite statistics (median, mean, Pcorr, T-tests, etc) are used to highlight trends and correlation. T-statistics deriving from each estimation results will determine significance and robustness. A content analysis will be used for qualitative data.

It is expected that the evaluator will work to the highest evaluation standards and codes of conduct. Transparency and objectivity will be observed at all times.

Evaluation follow up and learning

• The findings will be used to inform future programming.

• A management response will be developed within one month of the evaluation report being finalised. This will be followed up and tracked by the steering committee.

• A dissemination plan will be developed to ensure that important learning is shared with internal and external stakeholders. The report can be disseminated through the following: INGOs Forums in Nigeria and Cameroon; policy briefs; publications in peer reviewed journals; blogs (eg IDMC, NRC), conferences & seminars, restitution workshops with beneficiaries, posters.

1. Deliverables

Deliverable 1: Draft Inception Report – The draft inception report should set out any changes proposed to the methodology or any other issues of importance in the further conduct of the evaluation. The inception report will: i. include a desk review;

ii. describe the conceptual framework that will be used to undertake the evaluation;

iii. present an evaluation matrix: sets out in some detail the approach for data collection, the evaluation methodology, i.e. how evaluation questions will be answered by way of data collection methods, data sources, sampling and selection criteria, and indicators;

iv. provide a detailed work plan for the evaluation, which indicates the phases in the evaluation and key deliverables;

v. set out a plan for data collection, interviews or discussions;

vi. present an analysis plan;

vii. set out the list of key stakeholders to be interviewed.

Deliverable 2: Inception Report – The inception report will be finalised upon consultation with the Steering Committee Members (max 2 weeks review).

Deliverable 3: Draft Evaluation Report (30 Pages max) – including an Executive Summary of key findings, conclusions and recommendations, and database (Nigeria and Cameroon).

The draft evaluation report will be produced upon completion of the desk review and after the conduct of field work and informal feedback meetings with the steering committee. The draft report will be validated through:

(i) an evaluation stakeholders’ workshop (possibly online due to the COVID-19 crisis) to be organised by the steering committee. The evaluator is expected to facilitate and deliver a presentation of key findings.

Deliverable 4: A final^[2] evaluation report (maximum 30 pages, French and English) structure as follows: a. Cover page

b. A list of acronyms and abbreviations

c. A table of contents

d. An executive summary

e. Introduction and background

f. Scope and objectives of the evaluation

g. Methodology

h. Results

i. Conclusions

j. Lessons learned and recommendations

Appendices of the report, including:

k. Terms of reference

l. Evaluation matrix

m. List of documents consulted

n. Evaluation tools

o. List of persons and organisations/institutions consulted

A management response plan will be prepared by the project teams in both countries after the finalisation of the evaluation. It will specify key actions to be undertaken, key partners to be involved in the execution of these actions and the implementation schedule.

Deliverable 5: Participatory video as an additional mean to convey messages coming out of the evaluation.

[1] Purely based on mathematical formulae and indicate the minimum sample size required for the evaluation.

Tell us how large the samples need to be and assess whether existing data sets are large enough for the purpose of conducting an evaluation. Avoid collecting too much information, which can be very costly and also prevent from collecting too few data (Lack of power).

[2] The quality criteria of the report will be specified to the consultants during scoping meetings.

Evaluation consultant team

NRC seeks expressions of interest from both individual and firms for this assignment.

Consultants must have:

• Advanced university degree or equivalent in Humanitarian/Development Studies, Social Sciences, statistics or other fields related to the provision of humanitarian assistance in Education, Livelihoods and food security, Information Counselling and Legal Assistance, community development and humanitarian coordination;

• Good knowledge of data collection methods using tablets and CAPI systems, data analysis and presentation of results;

• Good knowledge of outcomes evaluation approaches especially quasi-experimental as well as participatory mixed methods;

• Experience in planning, implementation, monitoring and especially in evaluation of programs in humanitarian settings;

• Good knowledge of community development mechanisms; gender and equity;

• Good French and English communication, writing and synthesis skills;

• Knowledge of local cultures and languages and social, economic and political background of Cameroon and Nigeria –the Lake Chad region is an asset. The composition of the team must be balanced to enable an exhaustive coverage of the various aspects of the evaluation set out in these terms of reference, including cross-cutting issues.

• Have the capacity to deliver result with COVID 19 context and restrictions.
**
Ethical Considerations**The evaluation must respect the rights of the people interviewed by guaranteeing confidentiality and security. Respondents should not be exposed to any risks and discomfort. As such data collection tools will be pre-tested and evaluation activities will provide a safe, creative space where respondents feel that their thoughts and ideas are important. All staff involved, evaluator(s) and enumerators are expected to be briefed on NRC’s Safeguarding Policy and Code of Conduct with which they must comply. The data collection plan must state how NRC and the evaluator(s) will file the data and keep it secure. Enumerators (male and female) will be trained on ethical data collection techniques.

Application process and requirements

The technical and financial proposals of up to 6 pages maximum (excluding annexes) should include the following:

• Proposed methodology, including a tentative workplan;

• Composition of the evaluation team;

• CV of the member(s) of the evaluation team including references;

• Detailed budget of the offer (consultant(s) will cover the costs for local travel, food, accommodation, etc.);

• An extract (3-5 pages) from reports written for similar missions;

• A cover letter with the following information: a description of how their skills, qualifications and experience are relevant to the requirements of the assignment, a list of previous evaluations that are relevant to the context and subject of this assignment, a statement confirming their availability to conduct the assignment and the budget (please indicate currency of amount), a statement confirming that the candidate has no previous involvement in the delivery of the project or any personal relationship with anyone engaged in it.

• Proof of registration (business registration documents and VAT or company tax registration (If the consultant is a sole-trade (self-employed) a confirmation of the status from a certified accountant or the tax authorities is mandatory.

Applications are accepted in English at the following address sn.procurement@nrc.no no later than 30 November 2021.