End of Project Evaluation for Emergency Urban Safety Net Project At Norwegian People’s Aid

INTRODUCTION:

Norwegian People’s Aid (NPA) South Sudan is an International Non-Governmental Organization involved in humanitarian, relief and long-term development cooperation in South Sudan. NPA has worked in South Sudan since 1986 and currently runs three programmes: Civil Society Development, Rural Development, and Emergency Response. The Urban Safety Nets (USN) Programme funded by the United Nations World Food Programme (WFP) commenced on 1st May 2021 and is scheduled to end on 30th April 20222. The USN programme which is under NPA’s Emergency Response Programme (ERP) is being implemented in Rumbek Center County in Lakes State. The target locations include Nyot tik Angui, Nyangakot, PDU & Abubu Centre of Rumbek Town. The project seeks to address immediate food security needs through Cash Based Transfers (CBT) and support resilience building by transferring knowledge and skills to improve the livelihoods of vulnerable urban residents in Rumbek Centre County of Lakes State.

DESCRIPTION OF THE PROJECT:

Background: The USN project seeks to address immediate food security needs with cash-based transfers and support resilience building by transferring knowledge and skills that will improve the livelihoods of vulnerable urban residents. The CBT component aimed to place vulnerable households on accelerated path to early recovery and to address the long-term food insecurity challenges in a sustainable manner. The project also aimed to enhance the capacity of youth including vulnerable girls and young women through vocational skills training and Income Generating Activities (IGA). To enhance sustainability of the vocational skills training and IGAs, graduates who completed the various market-led courses were provided with start-up kits support to enable them start various businesses in Rumbek Town or surrounding areas. Further, the project provided trainings to targeted vulnerable smallholder farmers to improve their knowledge, attitude and practices on agricultural production, nutrition, post-harvest handling and processing, value addition, and marketing. The project supported vulnerable households to construct and maintain community access roads and in return, the target households received cash transfers. By focusing on improved food security through cash transfers, vocational skills training, and awareness raising on hygiene, nutrition, conflict sensitivity and post-harvest handling management under vegetable crop production and improving market access through community road maintenance, the USN programme sought to address the most critical challenges faced by target population. The livelihoods of the targeted beneficiary communities remain fragile and vulnerable due to a combination of man-made and natural shocks that continue to undermine household food security and erode gains that were made through previous food security interventions. Thus, through the USN Programme, NPA with support from WFP, sought to contribute towards enhanced community resilience to shocks and stressors by improving food security and increasing access to livelihood opportunities in Rumbek town of Lakes State.

Overall Objective: The primary goal of the USN Programme was to address immediate food security needs with cash-based transfers and support resilience building by transferring knowledge and skills that will improve the livelihoods of vulnerable urban residents.

Specific Objective: The specific objective of this intervention was to enhance community resilience to shocks and stressors by improving food security and increasing access to livelihood opportunities for vulnerable populations in Nyot tik Angui, Nyangakot, PDU & Abubu of Rumbek Town, Lakes State.

PURPOSE OF THE ASSIGNMENT:

The main purpose of this evaluation is to assess the extent to which the USN programme delivered on the planned outputs and outcomes as well as progress towards achievements of the respective objectives and indicators, while highlighting enabling factors and challenges encountered. The evaluation also seeks to render accountability to beneficiaries, stakeholders, NPA and WFP.

The specific objectives of this evaluation include:

1. To assess the project’s performance and achievements vis-à-vis the project’s overall objectives and ascertain the impact of the project amongst the target beneficiaries.
2. To generate lessons learned from the implementation of the project activities that will be useful during implementation of similar projects.
3. To develop specific recommendations for major stakeholder groups (Line ministries, local leaders, beneficiaries and NPA/ Partners/ Donors). An action plan for major stakeholder groups shall be developed to promote sustainability and long-term impact to the beneficiary communities.

The evaluation is expected to provide data on the performance of project interventions as per the DAC Criteria (Relevance, Efficiency, Effectiveness, Impact and Sustainability). The findings and recommendations will contribute to a learning process which will enable the NPA, the donor- WFP and other project stakeholders (line ministries and its directorates) and other partners (NGO/ UN Agencies) to draw lessons in order to improve the quality of service provision to target communities. The findings of the evaluation will also inform NPA on the effectiveness of beneficiary accountability system and mechanisms used during the project implementation.

METHODOLOGY:

The desired methodology for this survey is participatory whereby NPA, Stakeholders and the donor (WFP) will all be part of the evaluation process. The consultant will however provide leadership and bear responsibility for the process, the findings, the recommendations and the content of the final report. The evaluation methods will mainly focus on generating quantitative and qualitative data through literature review of relevant reports from reliable and trusted sources/ projects documents, household interviews, key informant interviews (KII) and focus group discussion (FGD). All the data collected through this evaluation assessment should be disaggregated by Age, Gender and Diversity (AGD). The design and implementation of this evaluation must ensure that, principles of gender equality, inclusion and non-discrimination are considered and acted upon throughout, and that the meaningful participation of the most vulnerable groups and other key stakeholders is promoted in the design and implementation processes.

LINE OF INQUIRY:

During this evaluation, the consultant is expected to consider the following evaluation criteria as laid out in the DAC principles for evaluation of development assistance. Upon review of project documents, the consultant is expected to develop its questions as guided by the following evaluation criteria as outline below; – Relevance, Effectiveness, Efficiency, Impact and Sustainability.

a) Relevance: This examined the extent to which the project objectives and design responds to beneficiaries and partner needs, interests, policies, and priorities and how it has continued to do so if circumstances change.

b) Effectiveness: This criterion measures the extent to which the USN project has attains its objectives. This includes analysis of the attainment of outcomes and impacts, project objectives, and delivery and completion of project outputs and activities as outlined in project log frame.

c) Efficiency and Cost Effectiveness: This criterion measures the outputs – qualitative and quantitative – in relation to the inputs, costs and implementing time. It is an economic term which signifies USN project assistance provided used the least costly resources possible in order to achieve the desired results.

d) Impact: This criterion measures the Positive and negative changes produced by this intervention, directly or indirectly, intended or unintended. This involves the main impacts and effects resulting from the activity on the social, economic, environmental and other development aspects within the target communities.

e) Sustainability: This criterion measures the benefits of an activity that are likely to continue after the project has been completed and no more donor funding is available. It also seeks to identify and outline relevant factors that will help to improve sustainability of project.

Other Criteria to be considered for this evaluation include:

Project Achievements: Evaluate the achievements of the project against the expected project outcomes taking into consideration the various factors that contributed to the successful implementation of the project. Refer to the immediate objectives, outputs, indicators and activities specified in the project document (Log frame).

Implementation Approach: Analyze the project’s approach vis-à-vis the development problems being addressed, adaptation to changing conditions, partnerships/ collaboration in implementation arrangements, changes in project design and overall project management.

Replication and Scaling-Up Approach: Based on the lessons and experiences in the project, this is the process of duplicating the design and implementation in adjacent communities and other sectors to create a multiplier effect that will expand the coverage of the project.

SCOPE OF WORK AND EXPECTED DELIVERABLES:

The scope of evaluation is limited to collecting data and generating information based on the project document and logical framework that will be used to assess the performance of the USN project. Particularly for impact level indicators, this may also include reference to very credible secondary data sources within time and relevance to the project overall goal. To accomplish the scope of work, the consultant shall.

1. Inception report:

Upon signing of Contract, prepare and submit inception report, hold inception debrief meeting with NPA and WFP staffs (Virtual/ or in NPA office Juba)

Holding consultative meeting with staff and management of NPA in Juba at field level and other key stakeholders including relevant government departments of Agriculture and rural development.

Work with NPA staffs to develop questionnaires, prepare the sample size, identify and train the survey enumerators on use of Mobile Data Collection tools (Tablets/ smartphones), survey questionnaires and methodology

2. Enumerators Training/ and Questionaires pre-testing at field level

Conduct Data collection (Household interviews through use of household questionnaires with project direct beneficiaries, Focus group discussions (FGD), Key informant interviews (KII) with key stakeholders (Direct line ministries staffs, local groups, community leaders and other service providers).

3. Preliminary Report/ Validation workshop

Validation workshop with NPA staffs and key stakeholders from relevant government departments at field office in Rumbek and at Juba office upon completion of data collection processes.

4. Final Report (including Annexes)

Preparation of a detailed Final Evaluation report of not more than 30 pages of the main findings. The report should contain very clearly detailed values for all project indicators at Impact, Outcomes and Outputs level with a table detailing these values presented in a matrix form

RESPONSIBILITIES OF THE CONSULTANT:

The consultant will be required to report on regular basis to the Emergency Program Manager on the progress of the survey, which will have the overall responsibility of the survey.

i) Prepare/ develop questionnaires and upload to Kobo server

ii) Prepare the sampling strategy and sample plan for the survey

iii) Conduct survey enumerators trainings

iv) Coordinate and supervised data collection in target location

v) Checks for data quality throughout the data collection period, data entry and final data cleaning

vi) Conduct data analysis and draft report write up

vii) Collation of inputs from relevant programme staff, finalization of recommendations

viii) Conduct validation workshop for presentation of preliminary findings

ix) Produce the finalized End of Project Evaluation (EoP) Report.

RESPONSIBILITIES OF THE NORWEGIAN’S PEOPLE AIDS- NPA

The Norwegian’s Peoples Aid will be responsible to;

i) Provide survey enumerators training materials and incentive

ii) Provide relevant security briefings and organize relevant travel permits if required for international consultant.

iii) Provide payment and cover other necessary cost with the evaluation survey enumerators.

iv) Provide SMART phones for data collection

v) Logistical/ Fleet Support

vi) Provide vehicles to facilitates the data collection processes

vii) Provide accommodation for the consultant while in the field and in Juba throughout the evaluation period

viii) Provide flights/ transport to and from field location

CONDITIONS OF WORK:

• During the period of this job, the consultant will be based in NPA Field office in Rumbek town with frequent travel to field locations for data collection, supervision and monitoring. He/she will be asked to overnight in the far locations of the target project sites under NPA cost if in hotel and with no cost in NPA guest house.

• The consultant will be required to abide by NPA security procedures provided by the NPA safety and security coordinator and other relevant policies, such as Code of Conduct, Conflicts of Interest from the human resources, which are outlined in the contract for this Job.

• The consultant will be required to sign NPA Conflict of interest form before commencement of tasks.

• All raw data collected and reports generated will remain the property of the Norwegian Peoples Aid- NPA and the consultant will have no any rights of ownership after the end of the survey.

TIME FRAME:

This assignment will last approximately for 30 days starting 15th May 2022 – 14th June 2022. The consultant is expected to accomplish the task in the period provided. The timeframe assumes data collection with electronic handsets, and as such does not include time for data entry. For applicants intending to propose data collection using paper questionnaires please indicate this clearly in the proposal document. The overall work should be completed in not more than 30 days including the final approved report for the End of Project (EoP) Evaluation.

QUALIFICATIONS AND EXPERIENCE:

a) Required

The successful candidate should:

i) Have an advanced university degree/ PHD desired or the equivalent in Agriculture, public health, development studies, economics, social sciences qualitative and quantitative research studies, M&E with at least five years of experience in conducting similar research for food security and livelihood (FSL) and nutrition, with a specific competency in humanitarian emergencies setting.

ii) Experience in Conducting Cash Base Transfer (CBT), IGA’s and CfW project evaluation

iii) Experience in conducting Evaluation for WFP funded project in South Sudan or elsewhere

iv) Have significant experience in undertaking nutrition surveys using quantitative and qualitative methods (Design and Methodologies, staff recruitment and training, field supervision and data analysis/write up).

v) Be familiar with the use of mobile data collection tools (MDC) and able to upload and manage cloud data as required by the organization.

vi) Familiar with statistical data analysis tools such as SPSS, STATA

vii) Able to work in difficult and hostile environment

viii) Have experience in negotiating tangible ideas with opposing parties

ix) Able to manage work stress while keeping high performance

x) Able to travel long distant in rough roads with minimal nutritional support

xi) Be fluent in English with excellent writing and presentation skills

xii) Experience in conducting trainings

xiii) Experience/exposure to similar contexts

b) Desired

i) Previous experience in South Sudan of similar job is desired

ii) PHD holders in any relevant field desired

ALL APPLICANTS SHOULD INCLUDE THE FOLLOWING:

Technical proposal: –

• Brief explanation about the lead and associate consultants with particular emphasis on previous experience in this kind of work

• Understanding of TOR and the task to be accomplished

• Proposed methodology

• Draft work/implementation plan

The technical proposal should include;

• Copies of reports of previous work conducted.

• A written submission on understanding of TOR, methodology / approach the consultant will use; time and time-bound activity schedule, financial issues (budget, number of people he/she will hire, costs per activity line – people, logistics etc.)

• Organizational (if it is a company applying) or personal capacity statement (if it is an individual that will hire data collectors)

• Resume and 3 references

THE EVALUATION CRITERIA ARE BASED ON TECHNICAL AND FINANCIAL CRITERIA

• Education background: minimal- Advance Degree for both lead and co-associates

• 6 Years of experience in conducting similar tasks especially for WFP funded project

• Known reliability in delivery of timely and quality services to WFP funded projects

• Relevant field/ country experience especially fragile country like South Sudan

• Relevant sector specific technical experience & qualifications especially in agriculture and livelihood programming

• Experience in conducting and designing research methodology (Both Qualitative and Quantitative)

• Relevant experience in project monitoring and evaluation (M&E)

• Demonstrate excellence skills in Mobile Data Collection (MDC) tools using XLS form

• Realistic and Cost-effective budget proposal

• Realistic and clear work plan with define deliverables within the allocated time period.

• Clear referencing (3 referees’ maximum) and latest sample of previous work done (one sample minimum)