End of Project Evaluation Consultant At Centre for Agriculture and Biosciences International

1. Background

Pest Risk Information SErvice

An estimated 40% of the world’s crops are lost to pests, consequently affecting smallholder farmers’ ability to feed their families, international trade and food supply chains, thus hampering the pursuit of Sustainable Development Goals (SDGs) 1 and 2 of ending extreme poverty and ensuring zero hunger. Pest outbreaks can be devastating; they respect no geopolitical boundaries and are becoming increasingly unpredictable due to climate change, increased international trade and mobility of humans.

Pest Risk Information SErvice (PRISE) aimed to contribute to targets under SDGs 1 on alleviating extreme poverty and reducing vulnerability to climate related extreme events; SDG 2 on ensuring access by all people to safe, nutritious and sufficient food, doubling agricultural productivity and income of small-scale food producers, implement resilient agricultural practices that increase agricultural productivity in the face of climate change and other disasters, and increased investment in extension services and technology; and SDG 12 on achieving environmentally sound management of chemicals to minimize their adverse impacts on human health and the environment.

The implementation of PRISE Phase I started in January 2017 in three countries; Ghana, Kenya and Zambia. PRISE Phase II was implemented in Malawi, beginning January 2019.

1.1 PRISE Implementation Approach

PRISE is an innovative project that brings together novel Earth observation (EO) technology, satellite positioning, plant health modelling, and on-the-ground real-time observations to deliver technology-based solutions to users in sub-Saharan Africa. It uses cutting-edge space infrastructure and state-of-the-art technology to deliver solutions to pest problems in real-time. Appropriate communication methods are employed to deliver pest risk information and mitigation measures to country agricultural extension and plant health information systems and collect feedback from users.

The project was implemented through nine work packages (WPs) with the following WPs as key:

1. WP1 EO Data Validation: Use of standardized satellite earth observation datasets that supply environmental data (rainfall, temperature) to drive the pest lifecycle models
2. WP2 Pest and Disease Model Research and Development (R&D): Desk-based and field research on the creation and, or validation of selected pest and disease models used in PRISE
3. WP5 Crowdsourcing R&D
4. WP6 Knowledge Sharing, Adoption and Outreach: Dissemination of pest alerts to agricultural extension and plant health information providers.

1.2 Aims and Objectives of the PRISE Project

Aim: To contribute to improved food security by enabling male and female farmers in Africa to lose less and feed more. This was to be achieved through a pest risk forecasting system based on Earth observation (EO) and Plantwise data.

Objective: The overall PRISE project objective is to increase the resilience of smallholder farmers to pest outbreaks by drawing on expertise in Earth Observation and meteorological sciences, together with a state-of-the-art understanding of crop pest growth, to provide farmers in developing countries with timely warning of potentially hazardous pests, and advice on pest management practices.

PRISE is an innovative approach for CABI and partners to improve rural livelihoods by providing an early warning system to farmers on possible occurrences and outbreaks of pests and diseases affecting their crops, and for them to employ mitigation measures in good time to reduce crop losses of these pests and diseases.

1.3 PRISE project impact and outcomes

The project outcomes and impact as envisaged are contained in the project logical framework (LF), and are as follows:

Project impact (UN SDG2: Zero Hunger): Reduce hunger and increase food security by enabling male and female farmers to reduce crop losses from pest outbreaks by 10% and increase their household incomes by 5% in up to 5 project countries over 5 years.

Outcomes:

The project aimed to deliver 5 outcomes at the end of the project as listed below:

· Outcome 1: Increased resilience of 630,000 male and 360,000 female farmers to pest and pathogen outbreaks by use of results from space forecasting expertise by the end of year 5

· Outcome 2: Six (6) stakeholders using PRISE decision support tools in their work for ecosystem monitoring by the end of year 5

· Outcome 3: Increased use of PRISE outputs and space expertise in enabling 5 governments to changes policy to address crop pest and diseases challenges in the target countries by year 5

· Outcome 4: Increased and strengthened partnerships between UK and 11 (public and private sector) partners across 5 project countries for sustainability of outcomes

· Outcome 5: £2 million in associated income secured by PRISE consortium to make additional use of space expertise by the year 5

PRISE Implementation approach in Zambia

In Zambia, PRISE was implemented via two main approaches, 1) Bulletins were sent to plant doctors and non-plant doctors. The bulletins contained information on estimated time to action based on planting date and related management recommendations 2) SMS messages were sent to over 75,000 farmers in representative districts of all provinces of Zambia, through mass messaging using the government platform, Zambia Integrated Agriculture Management Information System (ZIAMIS). Subscribers received pest alerts as part of agricultural advisory services for either maize, beans or tomato, along with recommended time to action windows and management advice for major pests. Messages were sent out in the six languages of Nyanja, Bemba, Tonga, Silozi, Kiikaonde, and Luvale.

The proposed end-line activity is aimed at assessing the extent to which the project impact has been delivered/achieved after 4 years of implementation of interventions. The project key performance indicators (KPIs) at the impact level are; average annual farming income and the average amount of crop losses from pest damage in project areas, both disaggregated by male and female farmers. In addition, the assessment should provide information to project outcomes, through their respective logframe KPIs.

2. Objectives and Evaluation Questions

The main objective of the proposed end-line assessment is to provide end-line information that would show changes in conditions and behaviours of project participants, compared to the baseline situation on the project outcomes and impact indicators, as depicted in the project logframe. During project implementation, various stakeholders were mobilized to work with the PRISE partners and therefore changes at the partnership level will be assessed in terms of the use of PRISE outputs.

2.1 Evaluation Scope and Learning questions

To support this learning process, the end of project evaluation will examine performance against the stated objectives to identify progress, effects, lessons learnt, guided by the Organisation for Economic Co-operation and Development (OECD) Development Assistance Committee (DAC) evaluation criteria, around the following evaluation questions:

1. Relevance – Determine the extent to which the PRISE project activities are suited to the priorities and needs of smallholder farmers in Zambia. Examine the extent to which the project has accommodated changes in the operating environment to make it relevant to the needs /priorities of the target participants;

· Is the project successful in reaching low-income smallholder farmers, youth and women?

· Has the contextual relevance changed since the project’s inception?

2. Coherence– How well do the PRISE interventions fit, and what are the synergies and interlinkages that exist between the PRISE intervention and other interventions carried out by the implementing partners?

· Are the PRISE interventions consistent with the relevant international norms, standards and the intervention of other actors in the same context?

3. Efficiency– Assess the relationship between quantity and quality to
outputs, outcomes and impact.

· Did the planned project interventions deliver the expected outcomes in terms of achievement of the immediate objectives, in accordance with the schedule and as cost-effective as initially planned?

· How did the project overcome unforeseen difficulties and deliver project outputs on time and within budget?

4. Effectiveness – Determine the outcomes generated by the project in relation to the stated goals, objectives and desired results.

· To what extent has the project interventions affected the knowledge, skills, attitudes and behaviours of agricultural extension staff and smallholder farmers?

· What are the external factors that may have influenced the outputs, outcomes and impact of the project?

· What are the notable achievements, best practices and lessons learnt from the project?

· How far were the planned benefits delivered and received by all key stakeholders, and how unplanned events may have affected the intended project benefits?

5. Impact – Assess, to the extent possible, the long-term impact of the project.

· Are there early indications of lasting project impact? To what extent can project impacts be attributed to the project interventions?

· To what extent have the project interventions generated or are expected to generate significant positive or negative, intended or unintended, higher-level effects to the project stakeholders

6. Sustainability –To what extent are the outcomes and anticipated impacts of the project likely to continue beyond active project delivery and into the near future?

· What, if anything, is further needed to ensure the long-term viability of the PRISE project?

· What Country policies and regulatory frameworks, if any, will further support the PRISE model for continuity in sharing crop pest /disease?

· To what extent has the institutional capacity (systems, structures, staff expertise, etc.) contributed to the overall implementation of the project?

7. Replicability– Assess the extent to which the project’s management, implementation, and results can be replicated in other contexts.

· What elements of the PRISE project can easily be replicated, and why?

· What are the possibilities of using the project-trained individuals, institutions to replicate the project’s outcomes in other regions?

2.2 Scope

The end-line assessment will be conducted in the sites in Zambia where the PRISE interventions were implemented, and where the baseline assessment was conducted. These sites are in the seven provinces; Central, Eastern, Luapula, Lusaka, Muchinga, Northern and Western. The assessment will target smallholder farmers reached by the PRISE pest alerts and county agriculture extension officers, or ‘plant doctors’[1] who were the intermediaries between the project and the smallholder farmers in disseminating the pest alerts information. A comparison group of farmers, who did not receive pest alert information will be selected from the same locations and should have similar characteristics- agroecological zones, farming systems, socioeconomic conditions, etc., like those in the treatment sites.

3. Approach and Methodology

The end-line assessment will utilize a similar methodology as used in the baseline study and should be robust enough to estimate and demonstrate any changes that happened from the baseline status as a result of project interventions. The Consultant will examine the project’s approach vis-à-vis the development challenges that were being addressed. The Consultant is expected to employ a variety of data collection and analysis techniques for both quantitative and qualitative data to ensure a comprehensive analysis of the processes. This will likely include, at the least:

· Document review: Review of existing documentation, including; the project document, theory of change (ToC), project logframe, quarterly reports, case study reports and Zambia country baseline survey report

· Survey: Application of structured survey questionnaires with a representative, random sample of the target sample population, to gather data for end-line indicator values.

· Focus Group Discussions (FGDs): With target groups and other stakeholders to understand access to pest alerts on plant health advisory information, using the alerts information and effects of the pest alerts information among smallholder farmers and agriculture extensions workers

· Key Informant Interviews (KIIs): Consultations with key project stakeholders and partners to obtain their expert opinion about plant health response systems.

4. Expected Deliverables and Timeline

All written documentation will be submitted in English using Microsoft Word in electronic copy. The main body of all reports should be written in simple, non-technical language, with any technical material to be presented in annexes. All primary data collected and analysis conducted for the assessment shall remain the property of CABI, and must be submitted electronically, and in a clear and comprehensible format in Excel, and any other statistical software.

The assessment is expected to begin no later than July 2021, with the consultant(s) expected to take a total of 60 working days from the day of contracting to complete the assignment.

The Consultant will provide the following deliverables to the CABI project team within the timeframe stated:

a. Inception Report: Within [7] working days of end-line assessment launch, the Consultant shall submit to CABI for approval a detailed inception report of the proposed approach for the assessment. The report should provide an understanding of the ToR, a detailed description of the survey design, methodology and tools, research questions, expected outputs, budget with a breakdown of costs and a detailed work plan for the entire assignment. This, with any draft questionnaires or interview checklists, shall also be submitted for review at this stage, before the main assessment/review exercise begins.

b. Preliminary Report: Within [40] working days of assessment launch, the consultant shall present a draft report for review and discussion with the CABI project team. This should include a draft set of project end-line indicator values and lessons learnt. After the review and discussion, a report incorporating comments and, where necessary, responses will be submitted to the project team.

c. Final Report: Within [60] working days of assessment launch, a detailed report of the overall findings of the end-line assessment shall be submitted to CABI for approval. This report should provide project end-line findings and indicator values.

The main body of the report shall contain:

• An Executive Summary of no more than 3 pages
• An outline and rationale for the methodology
• The main findings
• Conclusions and recommendations
• A schedule/matrix of end-line and baseline values against the outcome and impact indicators
• Data collection tools and checklists used in the assessment, in the annexes.

5. Management and Implementation Responsibilities

The consultant will report directly to the CABI M&E team and Project Manager (PM). In addition, the Consultant will also be expected to liaise closely with the PRISE national partners for further information on how the project was implemented, the effects of the project on target participants, and the working of various partnerships in the project. Any proposed changes to the team composition listed in the application must be approved by CABI.

The Consultant shall be responsible for:

· Proposing a survey design and implementation plan, including developing a detailed survey methodology, sampling procedures and determining sufficient sample size that can bring out clear differences in outcomes between the target and control groups

· Developing survey tools targeting the various respondents (questionnaire and checklists)

· Conducting all desk reviews and field data collection. The use of digital data collection tools is highly recommended

· Data cleaning and data quality assurance, analysis and reporting, in a clear and accessible format

· Cater for all his /her field expenses and those of the team of enumerators

· Regular progress reporting to the CABI M&E team and Project manager, including responding to any comments or technical inputs within agreed timelines.

· Producing deliverables within agreed timelines, and in accordance with quality requirements from CABI

· Seeking comments and feedback from CABI, through the M&E team, in sufficient time to discuss and incorporate these into the final report

· Producing the final end-line report containing data against all indicators in the project logframe, lessons learnt and any recommendations

· Submitting to CABI clean datasets of the end-line assessment.

CABI shall provide:

· Guidance and technical support as required throughout the end-line assessment

· Copies of all key background documents, baseline documents, mini study/case study reports and mid-line reports, and other available information resources

· Introductory meetings with key partner staff, and other stakeholders

· Comments and feedback on, and approval of all deliverables, within agreed timelines.

6. Requirements

Essential

· Experience of 5-10 years’ implementing project evaluations and impact assessments, with a focus on agricultural, extension advisory and information interventions

· Demonstrable academic and practical experience in quantitative and qualitative research methodology, baseline, end-line and evaluation design, and implementation

· Possess strong analytical, data interpretation, facilitation and communication skills

· Excellent reporting and presentation skills

· All team members should be fluent in spoken and written English and the most used languages, Nyanja, Bemba, and Tonga.

· The lead evaluator should possess a minimum of postgraduate qualifications in agriculture, environmental or agricultural economics, agricultural extension, development studies, or related subjects.

· Experience working with rural communities in conducting project baselines, evaluations and impact assessments.

Desirable

· In-country knowledge of public agricultural extension and information services

· A good understanding of the measurement of agricultural extension-based intervention outcomes

· At least 3 publications in reputable journals within the past 3 years.

7. Proposal for Assignment

Interested candidates or firms are requested to submit:

1. A full proposal detailing their interpretation of the TOR, proposed methodology including, sampling, sample size, work schedule and detailed plan of activities. The proposal should ideally not exceed 10 pages
2. The proposed budget, in US Dollars (Propose a realistic budget for this assignment), including a breakdown of the budget and a justification of expenses. The budget should include only those costs that can be directly attributed to the activities proposed, with an explanation of budget line items
3. Copies of all relevant certificates, curriculum vitae and company profile
4. A sample of an end-line /evaluation report for similar interventions, completed within the last 18 months (this will be treated as confidential, and only used for assessing suitability for assignment)
5. Three referees (including, one from a most recent assignment).
6. Upon a review of the submission, CABI may require additional documentation to fulfil due diligence requirements. Requests for this will be made on a case by case basis, and a timeframe for turnaround agreed.

[1] Plant doctors are Camp Extension and Block Extension Officers who are specially trained persons with skills in plant protection and good agricultural practices (GAPs), involving crop pests and disease diagnosis, and providing advice to manage the pests and diseases.