Consultant for the final evaluation of Action against Hunger project At Action Contre la Faim France

A final evaluation of Action Against Hunger’s “Improving the resilience of Syrian refugees and vulnerable Jordanians through integrated solid waste management practices in Zarqa governorate” project.

1. PROJECT BACKGROUND

Eleven years into the Syrian conflict, more than 1.3 million Syrians have sought refuge in Jordan; 676,164 are registered refugees and approximately 80% live in non-camp settings. This influx of refugees has strained Jordan’s economy and resources, with Syrians often accepting worse working conditions and lower compensation than Jordanians and taking on low-skill, informal jobs in sectors such as in manufacturing, construction and agriculture. In 2022, Jordan had a 23% unemployment rate, with 32% of women and 52% of youth unemployed, showing clear gender and age gaps. For Syrian refugees, unemployment rates are significantly worse with 80% of Syrian refugees unemployed in 2021, up from 50% and 65% in 2019 and 2020.

The growing industrialisation and population growth rate in Jordan has led to rapid increase in solid waste generation with over 2.7 million tons of waste produced annually, of which 7% is recycled and 93% landfilled or openly dumped. This has put increasing pressure on waste management infrastructure and municipalities (who are responsible for collecting solid waste), particularly in the northern and central governorates where refugee populations are most concentrated. Municipal waste in the country has steadily increased from 1.5 million tons in 2000 to about 2.5 million tons in 2016. While municipalities are responsible for waste collection, developing SWM plans, incentive schemes, proper enforcement of regulations (such as illegal dumping) improving public awareness regarding waste disposal, they face challenges in meeting these responsibilities through existing resources and capacities.

Action Against Hunger (ACF) is a leading humanitarian agency working to combat hunger, malnutrition and its determinants in over 45 countries worldwide. Working in Jordan since 2013, ACF has focused its response on WASH, Food Security and Livelihoods, and MHPSS programming for refugees and vulnerable Jordanians, both in camps and host communities. ACF’s 2020-25 country strategy aims to strengthen the resilience of vulnerable people to socio-economic and environmental challenges in Jordan with an emphasis on rural and peri-urban women and youth. As of 2022, ACF has operations across Amman, Irbid, Mafraq, Zarqa and Madaba with over 70 staff.

Since 2021, ACF has implemented in partnership with GIZ and the Azraq Municipality, a livelihoods program to improve employment opportunities for vulnerable refugees and host communities in the solid waste management sector. This has included cash for work opportunities through the establishment of a compost unit to support the sustainable management of organic waste, cleaning campaigns in line with national strategies and a sorting at source pilot.

Effective and sustainable waste management services simultaneously aim to improve municipal service delivery and clean cities, while it is also a vector for environmental protection, revenue, and employment for a wider range of stakeholders. By approaching different levels and actors within the waste value chain, facilitating their interaction, and generating income for the multiple actors engaged, the intervention aimed to strengthen solid and organic waste management and service delivery at municipal level and guarantee sustainable results.

Project Areas and Timeframe

The project was implemented from December 2020 to January 2023 in Azraq city in Zaraq governorate.

Project Objectives

The specific objective of the project:

This project intervention aimed to improve municipal solid and organic waste management services and provide short-term job opportunities for vulnerable Jordanian and refugee households in Zarqa Governorate.

Action against Hunger built on its experience under GIZ’s Waste to (positive) Energy project to continue its support for Azraq Municipality and the targeted communities to enhance improved delivery and management of solid waste management services, integrating key stakeholders among the value chain. Interventions focused on capacity strengthening and support for the municipality on the implementation of the local solid waste management plan, as well as the operation of the compost unit and pursuance of a sustainable marketing model for the compost. The interventions also engaged private sector actors and community participants to increase awareness and improve access to short-term livelihood opportunities in the waste value chain.

Overall objective of the project:

To improve living conditions of vulnerable refugees and host community populations in Zarqa Governorate through integrated and sustainable solid and organic waste management practices.

The following results are contributing to achieve this objective:

• Result 1: Key recommendations on enhanced SWM practices and cost-efficient service delivery are implemented at the municipal level.
• Result 2: Separation, sorting, and processing of recyclables at source level is promoted in the targeted municipality
• Result 3: Production and marketing of certified compost under the responsibility of the municipality has increased in quantity and quality
• Result 4: Cooperatives provide formal livelihoods opportunities in the field of SWM and improve their capacities to become a lead actor on environmental awareness in the community

PURPOSE OF THE EVALUATION

This evaluation aims to look at the level of achievement of verifiable indicators of the action in line with project objectives. It will also leverage the OECD-DAC evaluation criteria around relevance, effectiveness, efficiency, and sustainability of the project.

The focus of the evaluation is to review the project interventions against its objectives and to assess the impact on beneficiaries and key stakeholders from the intervention. The evaluation will particularly look into the following:

• Relevance: defined as the extent to which the initiative responds to the contextual priorities, policies in place and real needs of the target groups, recipients and donor;
• Coherence: defined as the level of compatibility of the intervention with other interventions in the country, area, or sector
• Effectiveness: defined as the extent to which activities have led to achieve intended results;
• Efficiency: defined as the extent to which the intervention delivers, or is likely to deliver, results in an economic and timely way. This includes operational efficiency, such as management and coordination
• Sustainability and impact: defined as the extent to which the benefits of the intervention can continue after the end of the project.

Questions

Relevance: What is the relevance of the project in the wider SWM sector?

• To what extent does project align with the priorities of the SWM sector as per the municipal plans in Azraq and the national strategies?
• How well does the project help the municipality of Azraq meet the current needs of SWM?
• Was the SWM plan developed in a participatory way, ensuring relevance to the municipality and its inhabitants? Did the priorities identified reflect those of the local population?
• Was the support provided to the municipality to implement the SWM plan relevant and useful?
• Was it relevant to build a compost unit in this location based on the local context?
• Were the CFW activities selected in a participatory way and did they reflect local priorities?
• Did the CFW activities support the other project results?
• Were the capacity building topics for the cooperative (RFS) selected in a participatory manner?

Coherence: How much is the intervention responsive to the strategies and needs in the SWM sector and Azraq municipality?

• To what extent does the intervention draw upon local and national SWM strategies and plans?
• How much does the intervention complement other priorities in the sector or in the area?
• How well was the project designed vis a vis other contextual factors?

Effectiveness: To what extent has the program met its targets?

• To what extent did the program meet its objectives as stated in the project documents?
• What adaptations were made based on a changing context?
• To what extent have beneficiaries’ capacity and skills been strengthened and their access to employment improved in the short term?
• To what extent were SWM services provided during the project?
• How satisfied were the persons reached by the program?
• Has this project resulted in improvement in municipal solid and organic waste management? What project strategies worked well in effecting this change and what worked less well?
• How effective was sorting at source as compared to the original project plan? What were the barriers and enabling factors?
• Were 3W awareness activities effective in achieving behavioural change around recycling? What strategies were more/ less effective in resulting in behavior change?
• What could have been done to increase uptake of recycling at source?
• To what extent has the local recyclable waste value chain (sorting, collection, storage, marketing as per A2.2) been strengthened? What could have been done to strengthen this further?
• To what extent has the compost value chain been strengthened through this project? Has local demand for compost been generated/ increased? What could have been done to strengthen this further?
• Was the CFW beneficiary selection process and opportunities provided inclusive of women, youth & PWD?
• Were effective capacity building strategies for RFS identified and implemented?

Efficiency: To what extent has the project setup contributed to the success of the project?

• How has the engagement of various stakeholders facilitated the project’s results?
• Were the right stakeholders involved, and the most efficient strategies of engagement selected? What could have been improved (particularly with government stakeholders)?
• How has the different project modalities (ie cash for work) facilitated the project’s results?
• To what extent were partnerships utilized to improve implementation?
• How efficient was the process of CfW beneficiary selection and what could be improved?
• How efficient were project management and implementation structures (project team + cooperative in Y1; project team only in Y2) in delivering programme objectives across the four results? Were the right resources in place within the project? What alternatives could have been more efficient?
• Was the compost unit run in the most efficient way available to ACF? What factors contributed to efficiency and what could be improved?

Sustainability: How has the program ensured the long-term contributions to the SWM strategy in Jordan?

• To what extent are the project activities able to sustain beyond the end of the project?
• Was the requisite operational and technical capacity built within the municipality to manage the compost unit? Why/ why not?
• How have the changes in the compost unit handover plan impacted the potential sustainability of this activity? What other factors impacted the sustainability of these activities?
• What are the perceptions of the municipality of the support of the project toward their implementation of the SWM strategy for Azraq?
• What could have been done differently to sustain the sorting at source implemented by the project?
• Is the compost unit a viable venture in the long term?

EVALUATION METHODOLOGY

The evaluation methodology will be proposed by the evaluator after thorough review of the relevant documents and existing secondary data. The evaluator is expected to use a mixed methodology to collect primary data from targeted household, communities and stakeholders, ensuring the coverage and representation of different groups.

The evaluator is expected to propose a methodology of implementation for the evaluation in the form of an Inception Report, which is also a key deliverable for this work. This inception report should include a detailed overview of the methodology for the evaluation including a list of key informants, focus groups, targets for quantitative surveys (if relevant) and an inclusive list of questions for all of the above data collection. In addition, the methodology should outline the analysis framework they will employ, including the qualitative data analysis approach.

Desk review

The evaluator will undertake a desk review of project materials, including the project documents and proposals, progress reports results of any internal planning process and relevant materials from secondary sources. (Please refer Annex II)

Inception Report

At the end of the desk review period and before initiating the evaluation activities, the evaluator will prepare a brief inception report. The report will be written in English and will include the following sections:

• The methodological approach to the evaluation include an evaluation matrix in annex to specify how the evaluator will collect data to answer the evaluation questions, pointing out the limitations to the methodology if any and the choice of sites per field visit;
• A list of targeted key informants and planned focus groups, in line with the methodology
• A list of questions for KIIs and FGD
• A detailed evaluation workplan

Primary Data collection

As part of the evaluation, the evaluator will interview key project stakeholders (i.e. project staff, local/national representatives, local authorities). The evaluator will use the most suitable format for these interviews as detailed in the inception report and agreed with the Action Against Hunger team. The evaluator is also expected to collect information directly from beneficiaries, ensuring gender balance and equal representation towards enriching triangulation, the evaluator could also conduct Focus Group Discussions (beneficiaries, non-beneficiaries, key informants) and surveys.

Should the evaluator seek support from Action Against Hunger staff for the data collection exercises, either from enumerators, translators, facilitators, etc., this should be indicated in the proposal and detailed in the inception report.

Evaluation Report and final deliverables

The evaluation report is the main deliverable for this evaluation, shall follow the following format, and be written in English:

• Cover Page
• Table of Contents
• Executive Summary must be a standalone summary, describing the intervention, mainfindings of the evaluation, and conclusions and recommendations. This will be no more than 2 pages in length
• Background Information
• Methodology describe the methodology used, provide evidence of triangulation of data andpresents limitations to the methodology
• Findings includes overall assessment of the project against the evaluation criteria, respondsto the evaluation questions, all findings are backed up by evidence, cross-cutting issues are mainstreamed and; unintended and unexpected outcomes are also discussed
• Conclusions are formulated by synthesizing the main findings into statements ofmerit and worth, judgements are fair, impartial, and consistent with the findings
• Lessons learnt and Recommendations should be as realistic, operational and pragmatic aspossible; that is, they should take careful account of the circumstances currently prevailing in the context of the action, and of the resources available to implement it both locally. Recommendations should be presented by order of priority and with specificity on who the recommendation is for

The whole report shall not be longer than 30 pages, without annexes. The draft report should be submitted within 45 calendar days of signing the contract. The final report will be submitted no later than the end date of the consultancy contract. Annexes to the report will be accepted in the working language of the country and intervention/country office subject to the evaluation.

In addition to the report, a final summary presentation of findings in PowerPoint should be delivered by the consultant(s) as the secondary deliverable.

MANAGEMENT ARRANGEMENTS AND WORKPLAN

The evaluator will directly report to Action Against Hunger’s Food Security and Livelihoods Head of Department (based in Amman) and will coordinate directly with this focal point throughout the evaluation. All deliverables should first receive validation from this focal point prior to being considered final.

The focal point from ACF will be responsible to engage any other stakeholders (either internal ACF staff or external project stakeholders as required) in the review of the deliverables, as needed. The focal point will consolidate any comments and send these to the evaluator by date agreed or as soon as the comments are received from stakeholders.

Tentative Work plan

Activities & Anticipated Level of Effort

Briefing with key ACF Jordan staff : 1 day

Desk review, preparation of methodology, field work plan, and Inception Report : 5 days

Review of inception report by Action Against Hunger : 4 days (not included in the total days of consultancy)

Finalization of Methodology and Inception Report with Action Against Hunger : 1 day

Field work and/or remote data collection, including interviews with field team + field interviews/FGDs with stakeholders, cooperatives, and beneficiaries for the evaluation : 5 working days

Analysis of Primary and secondary data : 5 days

Draft Evaluation Report : 4 days

Draft Evaluation Presentation : 1 day

Review of report and presentation by Action Against Hunger and any key stakeholders, comments for finalization : 5 days not included in the total days of consultancy

Final evaluation reportand presentation submission : 3 days

Total days of the evaluation assignment : 25 days

Profile of the evaluator

The evaluation will be carried out by national/international evaluation consultant with the following profile:

• Extensive knowledge in environmental management and/or solid waste management.
• Relevant experience in evaluation of donor funded projects in humanitarian and development contexts
• Experience in coordination, design, implementation, monitoring and evaluation of programmes;
• Good communications skills and experience of workshop facilitation;
• Ability to write clear and useful reports
• Good knowledge of Middle East context.
• Fluent in English is required. Arabic is considered an added value. If a translator will be required, this should be indicated in the proposal.
• Understanding of donor requirements, familiarity with GIZ standards an advantage;

If you are interested you can ask for the full package (with the annexes) at : supplymng@jo-actionagainsthunger.org