Consultancy – Final Evaluation of “Durable solutions for vulnerable, displacement-affected People (INT10396) At Oxfam GB

Project title

Durable solutions for vulnerable, displacement-affected women, men, girls, and boys in Diyala and Kirkuk, Iraq

Geographical coverage

Diyala (Muqdadiya and Khanaqin Districts) and Kirkuk (Hawija and Daquq Districts),

Project duration

June 15, 2022- June 14, 2024

Project budget

GBP 1,729,419.32

Evaluation Commissioning Manager

Deputy Country Director, Oxfam in Iraq

Evaluation Manager

Program Quality and MEAL Lead, Oxfam in Iraq

1. BACKGROUND:

Oxfam is a global movement of people working together to end the injustice of poverty. Together we save, protect and rebuild lives. And we won’t stop until every person on the planet can defeat poverty or any type of inequality or discrimination. We are an international confederation of 20 organisations (affiliates) working together with partners and local communities in the areas of humanitarian, development and campaigning, in more than 90 countries. All our work is led by three core values: Empowerment, Accountability, Inclusiveness.

The Iraq 2021 Humanitarian Needs Overview (HNO) identified 4.1 million individuals in need of humanitarian assistance[1]. Meanwhile, COVID-19 increased the severity of humanitarian needs by an estimated 35% by reducing livelihood opportunities and access to basic services, which were already inadequate prior to the pandemic[2]. The situation for displaced communities worsened early 2021 after the Government of Iraq (GoI) launched its camp closure and consolidation campaign, that forced tens of thousands of men, women, and children into secondary, and in some instance tertiary, displacement. This protracted displacement remains critical, with Oxfam’s 2021 Multi-Sector Needs Assessments (MSNA) evidencing highly complex needs while local water, sanitation, and hygiene (WASH) infrastructure and livelihood opportunities remain severely inadequate in both areas of re-displacement and return.

Against this backdrop, Oxfam Iraq proposes an area-based response in areas of (re)displacement and return. The intervention seeks to bridge humanitarian and development responses by meeting urgent basic needs of 84,000 individuals (51% female), while strengthening community resiliency and socio-economic development. This will be achieved through a One Programme approach that consists of the delivery of WASH and livelihood activities in Hawija and Daquq Districts, Kirkuk and Muqdadiya and Khanaqin Districts, Diyala, paired with capacity building, advocacy, and policy work to address the systemic causes of vulnerability. Social cohesion and protection will crosscut all elements to minimize conflict dynamics and capitalize on opportunities to build safety and cohesion. Activities will be implemented in partnership with the Foundation of United for Relief and Sustainable Development (FUAD), Rehabilitation, Education, and Community Health (REACH), and Iraqi Al Amal (IAA).

The project has been delivered through two phases: Phase one will improve the conditions in areas of origin and integration to contribute to the sustainability of return and relocation, and therefore to overall reintegration in target areas. This includes enhancing the ability of community and government to seek and maintain basic services at HH, community, and district level. Phase two will build on these investments to enhance the capacities of people and institutions to address the causes of risk, fragility, and vulnerability, while staying agile to respond to new shocks. Both phases will be co-designed with the Swiss Agency for Development and Cooperation (SDC) to ensure alignment with the Swiss Cooperation Programme for the Middle East (2019-2022) and the forthcoming strategy.

3. PROJECT INTERVENTION AND TARGET GROUPS:

The project has the following goal, outcomes, outputs and key activities.

Goal: Supporting displacement-affected communities to move towards Durable Solutions through improved, inclusive, and sustained access to WASH services and livelihood opportunities in Diyala and Kirkuk

Water Sanitation and Hygiene (WASH)

Outcome 1: Increased and equitable use of potable water and sanitation facilities for 64,000 conflict-affected people including IDPs, returnees, and host community members.

Output 1.1: 64,000 conflict affected people have sustainable access to safe drinking water through rehabilitation/upgrading of existing water supply systems.

Output 1.2: 3,800 conflict affected people have access to improved and dignified sanitation facilities through rehabilitation and upgrading of inclusive WASH facilities in public institutions (health centers, schools).

Output 1.3: 64,000 conflict affected people have increased access to dignified hygiene items and public health promotion through community-based public health/hygiene promotion and distribution of various hygiene kits.

Emergency Food Security and Vulnerable Livelihoods (EFSVL)

Outcome 2: 485 marginalised and conflict-affected women and men have improved access to income and contribute to Durable Solutions

Output 2.1: 6 cash-for-work (CFW) schemes improve communities’ resiliency against climate change and provide urgent income to 291 conflict-affected individuals.

Output 2.2: 120 conflict affected people benefit from asset replacement, agriculture/livestock, and small business grants.

Output 2.3: 74 conflict affected people diversify their employable skills through vocational training.

Output 2.4: 1 menstrual hygiene management (MHM) innovation launched and assessed for its potential for inclusive income generation.

Outcome 3: Governorate-level community, state, and non-state actors have increased thematic and organisational capacity to ensure provision of quality services, in line with humanitarian principles and the Durable Solutions Framework

Output 3.1: A supportive environment for Durable Solution is promoted through a community-based approach identifying barriers and opportunities for return, resettlement or relocation of conflict-affected population.

Output 3.2: local government institutions have improved capacity to plan, implement and sustainably manage WASH services, through provision of tools, equipment and various training events.

Output 3.3: Community structures are equipped with knowledge and resources to ensure sustainable service delivery and respond to local risks, including climate change, social cohesion and conflict sensitivity.

4. OBJECTIVES OF THE FINAL EVALUATION

This SDC-funded project will end on June 14, 2024, and an external final evaluation is planned to take place on June and July 2024.

The overall objective of the SDC final evaluation is to systematically review the overall progress of the project towards the project’s outcomes, assess the learnings from project design and project implementation from our work with partners and communities, and provide recommendations on overall design of the project, modifications, and specific actions that might be taken into consideration in the future during the design of future projects of a similar nature. The scope for evaluation is determined using OECD-DAC criteria for evaluating humanitarian action. Relevant criteria are associated with a number of key questions (under section 5) that are to be addressed and explored.

Specific Objectives of the Evaluation:

• To assess the design, planning, delivery and management of the project by Oxfam and its partners in accordance with Oxfam Programme Standards and donor requirements.
• To identify and assess key internal and external factors that have contributed, affected, or impeded the achievements of the project, and how Oxfam and the partners have managed these factors.
• Review the project outcome indicators and update the final/endline values in the log frame.
• To assess the beneficiary targeting, verification and selection processes/approaches of the project
• To draw key lessons and learning from the project and make recommendations that will help inform Oxfam’s formulation and design of future projects that will benefit the Iraqi displaced, host and returnee communities affected by the armed conflict.
• To assess the relevance, effectiveness, efficiency, impact and sustainability of the project on IDPs, returnees and remainees on Oxfam and its partners.
• Describe and assess efforts of stakeholders in support of the implementation of the project.
• Assess the likelihood of continuation and sustainability of project outcomes and benefits after completion of the project.
• Examine the extent to which the impact of the project has reached the intended beneficiaries.

5. EVALUATION CRITERIA AND KEY QUESTIONS TO BE ADDRESSED IN THE EVALUATION:

The project evaluation should be organized taking into consideration of Oxfam’s key evaluation criteria, and program quality standards. The evaluation will also consider key sector and sub-sector indicators for assessing the performance. The following provides a guide to the questions to be addressed by this evaluation, under each of the criteria below:

Relevance and Appropriateness:

• Have Oxfam and its partners selected relevant operational areas for their work?
• Have Oxfam and its partners targeted the most vulnerable people, including vulnerable men, women, boys and girls?
• To what degree is the intervention addressing the WASH and livelihoods needs of the targeted women, men, boys and girls and contributing to reduced vulnerability?
• Was project design consistent with the Do No Harm principle? and with a Safe Programming approach?
• What was the level and quality of participation of the beneficiaries and the partners in project design?
• How responsive were project activities to the needs of targeted women, men, boys and girls in targeted communities (if possible, please distinguish between displaced/returnee and host communities) given their circumstances and priorities?
• To what extent have the key contextual changes, threats and opportunities that arose during implementation been influencing and informing project implementation?
• How and to what extent were monitoring and evaluation findings used to inform decision-making and the improvement of project implementation by Oxfam and partners?
• Was the project adapted appropriately to changing context and needs in a timely manner? What changes took place in the implementation of the project differed from the original design?

Efficiency:

• Was the project implemented based on the best use of existing resources/capacity, e.g. the capacity of the partners and the internal capacity and expertise of Oxfam itself? What key limitations existed on this front? What could Oxfam’s future projects do to increasingly develop and invest in existing resources?
• How cost-effective was the intervention? What cost-effective alternatives could have been used?
• Were the resources (financial, human, etc) for running all the activities available, adequate and was this the best use of resources to achieve the results?

Effectiveness:

• Has the project proceeded well towards the achievement of its outcomes?
• Is there anything about this project that has had any impact on the beneficiaries (e.g., predictability of aid, etc.)? (positive or negative)
• Has this project had any impact on the partners? (positive or negative)
• What were the main challenges of the project and how well have they been addressed? Is there anything in this project that made these challenges more/less manageable?
• How were beneficiaries selected and were they informed of the selection criteria?
• How effective were the selection criteria in reaching out to the most vulnerable populations?
• How inclusive and culturally sensitive were the activities carried out in terms of the approach, quality of participation, information and its dissemination?
• To what extent has gender been mainstreamed into the project? How could gender be better mainstreamed in a future project?
• What unintended consequences (if any), whether positive or negative, did the intervention have on women, men, boys and girls so far (IDPs/returnees/host communities)?
• What system and mechanism are in place (at Oxfam and partners) to ensure accountability to the beneficiaries and how well have they been working throughout the project?
• Have the ways of working with the partners been contributing to building the capacity of the partners?
• Were there any risks inherent to the duration of the project, either in the course of or towards the end/after the end of the intervention?
• What factors have contributed to achieving or not achieving intended outcomes and set objectives of the project? Identify any exceptional experiences/achievements of the project substantiated with evidence, such, but not limited to case studies, best practices, etc

Impact:

• What the positive and negative, intended and unintended, changes produced by the project?
• Have these changes contributed to any identified changes according to aspirations of project?
• Analyse the contribution of the project to any observed impact (intended, unintended, positive, negative) and analyse what other actors and factors contributed to the impact. What real difference has the intervention made to the beneficiaries?
• How many people have been affected? – Are the local communities and stakeholders more resilient than before?
• How likely is it that any positive changes may be sustained in the short- and medium-term?

Sustainability:

• To which extent did the planning and implementation of the interventions take longer-term and integrated problems into account?
• How did the project consider adequately to the context and the nexus of between humanitarian action, recovery and development?
• Did the project plan and implement an adequate transition and exit strategy that ensures longer-term positive effects and reduces risk of dependency?
• How likely will critical services and effects be sustained beyond the duration of the project?
• How well are the project’s outputs linked to more long-term focused objectives?
• Assess capacity of key actors to contribute to sustaining the positive changes according to the developed ToC.
• What were/are the major factors which influenced the achievement or non-achievement of sustainability of the project?
• To what extent did the benefits of the project continue after donor funding ceased? What were the major factors which influenced the achievement or non-achievement of sustainability of the project?

6. METHODOLOGY

Oxfam recommends mixed methods (a combination of both quantitative and qualitative methods) that will improve the evaluation findings by ensuring that the limitations of one type of data are balanced by the strength of another. Therefore, the data collection for this evaluation will contain various tools and methodologies (both quantitative and qualitative approaches). The consultant will be responsible for designing of tools, data collection, data analysis and report writing using both quantitative and qualitative methods.

For the quantitative approach, the consultant will conduct the data collection activities of the endline HH survey in the selected intervention areas to find the endline values of outcome level indicators, which will be compared later against the baseline survey values which was already done at the beginning of the project. Quantitative data using structured questionnaire can be used to see how the project’s intervention bring change to the life of conflict affected communities. The endline survey findings will be also integrated with the evaluation findings according to the key evaluation criteria (relevance, effectiveness, efficiency, sustainability, and impact).

Regarding the qualitative approach, the consultant is expected to develop a through methodology (to be endorsed by Oxfam) for this evaluation based on the expected deliverables & OECD-DAC criteria. The consultant will use qualitative data collection methods, such as, focus group discussions/FGD/ with different groups (men, women), key informant interview/KII/, with local government authorities, Oxfam staff, community leaders, partners and other stakeholders, case studies, etc).

The consultant(s) will also do desk study/ literature review, which includes review of different documents, such as baseline and other surveys, project proposals, progress reports, etc and analysing secondary data. The evaluation methodology should make use of baseline and other data already generated by Oxfam’s monitoring and evaluation team. The information and findings of the desk review should be integrated with the data and findings from primary data collection and analysis, and not placed in a separate section.

We anticipate that this evaluation will be a participatory review and learning exercise. Thus, it requires the consultant(s)/firms to be experienced in participatory approaches to learning and inquiry, and especially in seeking the views and perceptions of key stakeholders that include:

• Targeted beneficiaries
• Partners and actors directly involved in the project at different levels:
• The local implementing partners.
• Community leaders (if applicable) and representatives of the affected population.
• Local authorities.
• Oxfam staff involved in the implementation of the project (e.g. AMs, Technical Leads, and field program and MEAL team.)

The consultant should develop all the necessary data collection tools and sampling methods and share them with the Oxfam MEAL & PQ team for review and approval. And the data collection tools proposed by the consultant should be able to address the key evaluation questions mentioned above.

The evaluators/external consultants should consider the following methodological topics at a minimum:

• The evaluator’s understanding of the evaluation questions.
• Description of the phases related to the suggested evaluation methodology.
• Sources of information for gathering primary and secondary data.
• Sampling techniques, such as sample size, sampling methodology, the area and population group the sample represents.
• Instruments to be used for data collection.
• The type of software or ICT tool used for data analysis.
• Reference indicators and benchmarks for each evaluation question (if relevant).

REQURIED EXPERTISE FOR THE EVALUATION CONSULTANT(S):

This final evaluation should be led by a person (or persons) or firm have following:

Mandatory qualification:

• A minimum of 10 years of experience in humanitarian and development interventions in conducting evaluations for international humanitarian organisations and donor agencies
• Demonstrated experience in the monitoring and evaluation of WASH and Livelihoods, with preferably some experience in gender and protection in emergencies programming or programmatic knowledge of these two sectors.
• Experience in the use of quantitative and participatory qualitative methods of data collection
• Strong facilitation and English writing skills are also required.
• Knowledge of ICT tools for mobile data collection (Mainly Survey CTO)
• Commitment to safeguarding policy.
• Commitment to feminist principles in Evaluation.
• Knowledge of the context in Iraq or in middle east and familiarity or direct experience working in Iraq or in the Middle East broadly.

Following are not mandatory but desirable:

• Familiarity or fluency in Arabic is an advantage.
• Experience of working modalities of Oxfam, is preferred.

8- TIMEFRAME, KEY ACTIVITIES AND EXPECTED LEVEL OF EFFORT

Timeframe***:*** The evaluation is to be carried out in June and July 2024, with the final report submitted to Oxfam as per the timeline below. The exact dates of the evaluation are to be confirmed with the selected consultant(s) or firm.

Total expected level of effort: 25 working days

Note: Following are the suggested number of days. Actual days will be agreed with the selected consultant(s)/firm as per the workplan to be delivered during the time frame.

• Review essential documents of the project, including but not limited to the original project proposal, interim or on-going internal reports, and evaluations and lessons learned exercises undertaken thus far and review the key questions suggested and if necessary, propose adjustment (3 to 4 days) – work to be done from consultant’s/firm’s home location, all documents will be shared by Oxfam via email);
• Develop and submit the draft inception report with a detailed evaluation plan (to be endorsed by Oxfam) (2 to 3 day);
• Developing and translation of evaluation tools (Tools must be developed in English and translated in Arabic to administer in the field. (2 to 3 days)
• Primary data collection (quantitative and qualitative) (8 to 10 days in Iraq- Diyala and Kirkuk Governorate);
• Data analysis and preparation of draft evaluation report (7 to 10 days);
• Meeting (online) to share and validate the findings from the evaluation (1 day). Selected Oxfam and partner staff will participate in this meeting.
• Finalize the final evaluation report and send it to Oxfam (2 to 3 days). Oxfam will then prepare a management response to be annexed to the evaluation report.

Working closely:

The consultant(s) /firm will report directly to the Program Quality and MEAL Lead and will work closely with Area Managers and field teams. The Program Quality and MEAL Lead will coordinate and facilitate the evaluation process with the selected firm/consultant(s) in collaboration with the Senior MEAL & PQ Officer, Area Managers and field MEAL team.

9. EXPECTED OUTPUTS:

1. Develop a detailed evaluation plan (to be submitted after the document review but before the data collection – to be endorsed by Oxfam), outlining the proposed methodology.
2. Draft evaluation report in English to be presented to Oxfam (no more than 40 pages excluding annexes, including executive summary not exceeding 3 pages);
3. Meeting to present the draft findings of the evaluation, and to give feedback to Oxfam staff and the partners.
4. A draft copy of the evaluation report within one week (7 days) of the meeting with Oxfam and partner staff. Feedback from Oxfam will be provided within one (1) week after the submission of the draft report. The consultant/firm will review, then submit a second draft, which will also be subject to Oxfam’s feedback within 2 days. The final report will be produced in 3 working days of submission of the comments. It will include changes/modifications, agreed between Oxfam in Iraq and the consultant/firm. Some minimum guidelines on the evaluation report:
   • The report should systematically answer the key questions posed.
   • It should fairly and clearly represent the views of the different actors/stakeholders.
   • It should give the conclusions of the evaluator, in a way that is clear and substantiated by the available evidence.

Please note that the consultant should submit the final evaluation report with Oxfam before/on 31, July 2021

10. Payment and instructions for interested consultants/firms

Overall expression of interest/EOIs/ will be evaluated based on technical and financial proposals demonstrating value of money and strong technical description.

Payment will be done in two instalments, 30% upon contract signature, 70% upon Oxfam’s approval of the final evaluation report.

What costs to include in the offer: Consultants/firms should include the following costs in their offer’s budget: daily rate, cost of international travel (home location of consultant/firm to Erbil and back, applies to international consultants only; cost for enumerators/data collectors for the endline survey, ).

What costs not to include in the offer: Oxfam will pay for and procure the following for the evaluation and therefore the following costs should NOT be included in the offer: pick-up/drop-off of consultant(s) from airport, travel inside Iraq for data collection and meetings, interpretation services during meetings with beneficiaries/the partner, printing/photocopying costs.

Note that payment will be made based on the budget in the offer (not based on actual expenses incurred by the consultant). No receipts will be requested from the consultant towards the end of the evaluation.

11. CODES OF BEHAVIOUR:

The evaluation process will be directed by Oxfam’s guidelines for the ethical conduct of evaluations and research, guiding the evaluation team through careful consideration of the key ethical implications at every stage of the evaluation. These guidelines are available at this link: http://policy-practice.oxfam.org.uk/publications/undertaking-research-with-ethics-253032

Selected consultant or firm will be asked to sign and adhere with Oxfam’s Code of Conduct.

12. SHARING AND USING FINDINGS:

The Oxfam International’s Policy on Program Evaluation requires Confederation members to act on the commitment to transparency by making public the Executive Summary and a Management Response to all final evaluations. The Policy is available at this link: http://www.oxfam.org/sites/www.oxfam.org/files/oxfam-program-evaluation-policy-dec10.pdf

13. DISCLOSURE:

Although free to discuss with the authorities on anything relevant to the assignment, under the terms of reference, the consultant is not authorized to make any commitments on behalf of Oxfam. All data collected as part of this consultancy belongs to Oxfam and public dissemination of the data and evaluation products can only be done with the written consent of the Oxfam.

14.Bid Evaluation Criteria

Bid Evaluation Criteria

Part

Criteria

Award criteria’s

Score

A

Consultant’s Qualifications and experience in the field covered by the TOR

Academic Qualifications

10%

Specialization and proven experience in the field related to the assignment.

15%

Qualifications, skills, and experience of the Team to be engaged for the assignment

10%

B

Quality / Understanding of the TOR

Meeting technical requirements (quality of the proposal) OR Level of understanding the assignment as per TOR. It considers how deep the consultant understood the work including the scope**. (Not just copy and paste from the TOR content)**

10%

Adequacy of the proposed approach and methodology for carrying out the task in accordance with the ToR

20%

Proposed workplan (realistic of workplan)

Detailed Work Plan to carry out the work highlighting key milestones & deadlines.

5%

C

Financial Proposal

Clarity of the cost breakdown

20%

Fair and reasonable budget

10%