Terms of Reference for Project Mid-term Review Consultancy At CBM Global Disability Inclusion

Description of consultancy

To assess the performance of the Comprehensive and Inclusive Eye Health (CIEH) project at mid-term, gauge the extent to which the project is on track to achieving its objectives, and document lessons learnt that will be used to improve the project over its remaining life span.

Background and rationale

The World Health Organization (WHO) estimates that at least 2.2 billion people have vision impairment or blindness globally with over 1 billion cases that could have been prevented or have yet to be addressed. This 1 billion people include those with moderate and severe distance vision impairment or blindness due to unaddressed refractive error (88.4 million), cataract (94 million), glaucoma (7.7 million), corneal opacities (4.2 million), diabetic retinopathy (3.9 million), and trachoma (2 million), as well as near vision impairment caused by unaddressed presbyopia (826 million).1

In Nigeria, there is paucity of data to ascertain the prevalence and causes of blindness, which impacts service organization and accessibility, especially for people who have the most need. Most data used for planning eye care services are generated either from urban areas where the large eye hospitals are situated or from small, focal surveys.2 However, available statistics shows that cataract is the most common cause of visual impairment in Nigeria being responsible for approximately 44% of cases, followed by glaucoma with 16.7% of cases3. Moreover, due to sub-standard post-surgery management practices, among other reasons, the visual outcomes after surgery are often quite poor. 80% of blindness can be prevented if the right steps and treatments are applied at the right times. Baseline assessment findings reveal that access to services are exacerbated by huge information gaps influenced by cultural and religious beliefs and the dearth of trained eye health professionals. There is a nexus of socio-economic factors that impede access to quality eye care services in Nigeria, especially for persons with disabilities.

ECWA Eye Health with funding support of CBM Global Disability Inclusion (CBM Global for short), is implementing the Comprehensive and Inclusive Eye Health (CIEH) project in Kano and Jigawa States aimed at contributing to an equitable access to eye care services among the target population in the North-West region of Nigeria. The specific objective of the CIEH project is improved quality of life of people with visual impairment through provision of comprehensive, inclusive, affordable, and accessible eye health services in Kano and Jigawa States through increased awareness on eye health care amongst communities in Kano and Jigawa States, improved skills of health personnel in relation to the provision of comprehensive Eye Health services in relevant facilities, improved access of the target population to comprehensive and inclusive eye health services (such as surgeries, management, spectacles) and livelihood support in target states, and improved disability inclusion and accountability practises at ECWA Eye Hospital and ECWA Community Health Initiative Clinics.

In line with standard project management practice, midway into the project, a mid-term review (MTR) is required to assess the implementation of the project against its set objectives and intended outcomes so that observable deviations from the original course of the project from achieving its desired results through this exercise will be corrected. Lessons learned and recommendations arising from the MTR will be used to improve the project through the second half of implementation. Of particular importance, the MTR offers an opportunity to review the project’s SBCC intervention approaches towards improving on them.

Purpose and Deliverables

The purpose of the mid-term review is to assess the performance of the CIEH project now halfway into the project implementation to gauge the extent to which the project is on track to achieving its objectives, and document lessons that will be used to improve the remaining life span of the project.

More specifically, the mid-term review objectives include:

1. To ascertain the extent to which the project objectives, planned indicators, and results at the project mid-term have been achieved.
2. To identify the changes (if any) to be made to set the project on track to achieve its intended results and outcomes.
3. To find out if the strategies used contributed effectively and efficiently to the achievement of the desired results.
4. To identify barriers to achievement of objectives and make recommendations based on findings that will guide the CIEH project staff through the last half of the project.

The deliverables of the project MTR will be used to track the progress of this exercise. The following documents (expected to be produced in English Language) are expected to be developed by timelines as final products of the MTR:

1. An inception report (including methodology, information collection plan, and review/evaluation matrix before the field data collection); produced by 8th August 2024.
2. A stakeholder workshop to validate the findings of the MTR and discuss the recommendations with the selected stakeholders on 26th August 2024. ECWA Eye Hospital will administer the funds for this workshop.
3. A draft mid-term review report produced no later than 30th August 2024.
4. A final mid-term report produced by 10th September 2024. This needs to be written up according to the Evaluation report template that CBM Global will share.
5. Data sets for all collected data, transcribed for CBM Global’s and ECWA Eye Hospital’s future use submitted together with the final mid-term review report by 10th September 2024.
6. PowerPoint presentation summarizing the key findings from the mid-term review submitted together with the final mid-term review report by 10th September 2024.

Both the Inception report and the Mid-term review reports should be presented in the standard evaluation report format to be shared by CBM Global.

Scope

The mid-term review will be conducted in selected communities in Kano and Jigawa States. CBM Global is engaging an external consultant to lead the mid-term review with a scope of covering the period of project implementation from 1st September 2022 – 30th June 2024. The MTR will assess the project’s progress across the result areas and indicators, document the findings, and provide recommendations to improve the project implementation as required.

The consultant will assess the following categories of project progress.

Project design

The consultant is expected to review the following:

1. The challenges that the CIEH project addresses, the underlying assumptions, and whether the project’s result areas are still relevant in addressing the problem.
2. The likely effect of any incorrect assumptions or anticipated changes to the achievement of the project results as outlined in the log frame.
3. Whether (or not) stakeholders were taken into consideration during the project design and implementation phase.
4. The extent to which gender and disability were considered in the project design.

Progress towards results

The consultant is expected to do the following:

1. Conduct a progress-towards-outcome analysis using appropriate tools. This should indicate progress made for each objective and result level indicator on the log frame.
2. Identify barriers to achieving the project’s objectives in the remaining project timeframe.
3. As may be required, make recommendations for achieving the project targets.

Project management and Project efficiency

The consultant is expected to conduct the following:

1. Critical analysis of the project’s log frame, indicators, and targets, and assess the practicability of the project activities towards achieving the project objectives and results within the project timeframe and whether the end-of-project targets are still achievable considering the project’s prevailing context.
2. Clarity of roles and responsibilities of the project partners and other stakeholders involved in the project and the effectiveness of the performance of those roles and responsibilities.
3. Analyse the cost-effectiveness of the current approach in mobilizing for the project services and proffer suggestions for improvement where applicable.
4. Determine whether (or not) the current project work plan is result-based and suggest ways of improving the process.
5. Determine the extent to which the project uses the log frame as a management tool.
6. Critical analysis of the quality of support that CBM Global provides to ECWA Eye Hospital.
7. Determine whether (or not) there have been changes to fund allocations because of budget revisions, the factors responsible, the extent of implementation of the changes, and the effects of those changes on the project.

Project Monitoring, Evaluation, Accountability, and Learning system

The consultant is expected to review the following:

1. Review the monitoring tools in use and comment on the quality of the data collected, the data collection methods, and whether they are sufficiently robust and are aligned with national guidelines and tools and meet CBM Global reporting needs.
2. Assess the level to which stakeholders are provided with needed project data for informed decision-making.
3. Assess whether sufficient resources are allocated to monitoring and evaluation.
4. Assess whether the reporting and decision-making timelines were in line with donor requirements.
5. Assess whether progress reporting meets CBM Global standards.
6. Document management processes, lessons learned, and at least two (2) human interest stories.

Stakeholder involvement

The consultant is expected to review the following:

1. Participation and contribution of education and health authorities across the three tiers of government and their level of involvement in the governance of the project.
2. Level of involvement of various care and support providers (e.g., eye health professionals, mental health professionals, pediatricians, anesthesiologists, hospital administrators, frontline healthcare providers, social workers, community leaders, school teachers, beneficiaries and their carers’, etc.) in the delivery of integrated services.
3. Confirm the benefits of the various eye care services on the lives of the project beneficiaries and highlight any challenges and areas for improvement. Also, identify the role of project beneficiaries in the decision making of current referral process and make recommendations as applicable.

Sustainability

The consultant is expected to do the following:

1. Identify and document likely financial risks to sustainability.
2. Identify and document likely socio-economic risks to sustainability.
3. Analyse the sustainability of the current referral pathway taking cognizance of the project context.
4. Analyse the level of ownership of the project by the government of Kano and Jigawa State.

Gender and Disability Inclusion

The consultant is expected to do the following:

1. Identify the level of involvement and participation of men, women, boys, and girls in the project.
2. Identify the extent to which organizations of persons with disabilities themselves participate in project activities.
3. Assign active participation roles to persons with disabilities in the MTR process.
4. Mid-term Review Questions

Area of Enquiry

Suggested guiding questions

Access to comprehensive eye health services and treatment

How accessible in Kano and Jigawa States are eye health services becauseof the CIEH project? Howmany people have accessed services since the beginning of the project?

What measures arein place to ensure sustainability of the eye health services provided throughthe CIEH projectin Kano and Jigawa State? To what extent are these measures effective?

How has the training of health workforce for eye carebeen effective?

To what extentis the current referral pathway in ensuring access to quality eye care in Kano and Jigawa States sustainable?

How has the project ensuredquality of carein delivering comprehensive and person-centred eye care services?

Stakeholder empowerment and engagement – collective advocacy for sustained quality eye health services

How effective is the inclusive Eye Health Action Teams (iEyeHATs) in Kano andJigawa States? Whatareas of the iEyeHATs need further support?

How has the project contributed to improving the eye healthof children in schools because of the Primary Eye Care intervention?

To what extenthas the government at the federal, state, and local government levels been involved in the CIEH project?

To what extenthas OPDs have been involved in the implementation of the CIEH project?

Access to sustainable livelihoods for persons with disabilities

What is the status and impact of the vocational skills training element of the project? How can theidentified gaps (if any) be addressed?

To what extenthave both men and womenwith disabilities been engaged in the livelihood interventions?

Gender anddisability inclusive practice

To what extentis the projectmainstreaming gender and disability into its activities?

Safeguarding of children and adults- at-risk

What mechanisms are in place within the project to ensure that children and adults-at-risk who come in contact withthe project are kept safe? How can the identified gaps (if any) be addressed?

Project monitoring, reflection, and learning

To whatextent has the project teamengaged in ongoing project monitoring and reflection?

To what extent has the project been able to build on lessons learned from otherin-country projects, e.g.,the CBM Seeingis Believing (SiB) programme?

What are the significant learnings from the project implementation thus far? Howcan these learnings be applied to this project in its second half?

Project relevance

To what extent are the expected outcomes of the programme on track?

What factors have led to the achievement or hindered the achievement of expected outcomes?

To what extent has CBM Global or ECWAEye Hospital supported contributions to the achievements of the project?

Project effectiveness

Is there an approved budget, log frames to support implementation?

How effective are the strategies used for the implementation of the CIEH project? To what extent are the strategies meeting the needs of the beneficiaries?

Project efficiency

What alternative strategies for the project implementation would have produced better results

What are the strengths, weaknesses, opportunities, and threats to the implementation of the CIEH project?

Project sustainability

To what extent are the advantages of the CIEH project likely to continue after its lifespan?

What is the probability of sustainability of the expected project outcomes?

What factors will be required in order to improve the possibility of sustaining the project?

What are the major lessons learnt?

What are the recommendations for better programme implementation in the future?

Methodology

The consultant is expected to develop the mid-term review methodology in line with the framework of the available budget for this task in collaboration with ECWA Eye Hospital and CBM Global. S/he is expected to submit a brief inception report where a mid-term review methodology should be proposed. The mid-term review must meet the principles of being inclusive, participatory, and interactive, involving both male and female beneficiaries. As a minimum, the mid-term review process should include the following key steps:

• Review and analysis of relevant project documents:
  • Project proposal, budget, log-frame, activity schedule, risk register, M&E plan, and reporting plan.
  • Financial and narrative reports from September 2022 to June 2024.
  • Memorandum of Understanding and Agreements with partners.
  • Programme data.
• Initial briefing sessions with CBM Global and ECWA Eye Hospital CIEH project teams.
• Application of appropriate data collection tools.

The data that are being collected and stored by the project in various formats are stored in Excel, and others collected as descriptive and so, are stored as narrative reports. The data collected are disaggregated according to gender, age and disability or impairments.

The qualitative methods to be used could include open-ended questions on questionnaires, personal interviews, observation, logs, journals, records, etc. Quantitative methods to be used could include surveys, close-ended questions on questionnaires, observation checklists, etc.

Regarding confidentiality or data protection, the consultant must take all reasonable steps to ensure that the respondents are not adversely affected by taking part in the mid-term review. The consultant must keep the responses from respondents confidential, unless their permission is granted, and must not do anything with their responses that they are not informed about at the time. Also, particular care must be taken with children and teenagers by ensuring that permission is granted by a parent or responsible adult, such as a teacher, for interviews with children aged under 18. The CBM Global Child and Adults-at-risk Safeguarding and Protection from Sexual Exploitation, Abuse and Harassment (PSEAH) policies should be applied in all circumstances.

During the mid-term review, the stakeholders consulted by the consultant should include, project staff (personal interviews) and management; consultants and field staff; beneficiaries (workshop trainees, Eye Health program leaders in Kano and Jigawa State, patients – children and adults); key community leaders where applicable (traditional rulers, pastors, and other sectional leaders in the community); representatives of the iEyeHATs, State Ministry of Women Affairs, State Ministry of Education, and related departments and agencies of government; frontline healthcare providers; CBM Global Country Team; and CBM UK.

Following data collection and analysis, the consultant will share preliminary findings with CBM Global and ECWA Eye Hospital. This shall be achieved through debrief sessions at CBM Global and project stakeholders at the evaluation sites. The sharing of preliminary findings is an opportunity for the stakeholders to hear and validate what the mid-term review has found and to be involved in thinking through the recommendations. It should include constructive discussions around the key issues identified by the mid-term review.

Timeframe and duration

This mid-term review is expected to be conducted over four (4) weeks, anticipated to start on 5th August 2024. The deliverables and their expected time frame are highlighted as follows:

• Inception phase: The consultant will review relevant documentation as itemized in the preceding section above. Based on the review, s/he will produce a mid-term review inception report within the first week of commissioning the consultancy – by 8th August 2024. The report should include methodology, data collection tools, evaluation matrix to be used to guide the field data collection and analysis, and the consultancy work plan.
• Field data collection phase: This phase of the mid-term review will seek to collect primary data on the key review questions explained under the Mid-term Review Questions. The deliverables expected are the raw data collected from the field using appropriate tools, approaches, and techniques. A validation workshop to disseminate the initial findings of the research is expected during this phase.
• Data analysis and reporting phase: The deliverables include the MTR report, PowerPoint presentation, data sets, transcripts, and signed consents used in the research.

Place/ location of service delivered

The consultant’s regular place of work for this contract is in Kano State, with travels to project communities in Kano and Jigawa States.

Required Expert Profile

The Mid-Term Reviewer will be a consultant. S/he will be an experienced evaluator, who will be responsible for the overall MTR process and the production of the MTR report and related products.

The consultant must sign the CBM Global Safeguarding and PSEAH policies before they embark on field work. S/he should be familiar with disability-inclusive approaches in evaluations in line with international standards.

The consultant will be supported by a staff of ECWA Eye Hospital during the field visits. The consultant will be selected based on the following criteria:

• Seven (7) to 10 years of proven experience in programme design, implementation, and evaluation with three (3) to five (5) of those years in evaluating public health programmes.
• Advanced degree/MSc degree in Public Health, Social sciences, International Development, or other relevant fields.
• A strong background in eye health programming and in disability inclusive development is desirable.
• Knowledge of disability-inclusive practices is an added advantage.
• Ability to draw practical conclusions and to prepare well‐written reports promptly and available during the proposed period.
• Ability to provide strategic recommendations to key stakeholders.
• Excellent interpersonal and communication skills including ability to facilitate and work in a multidisciplinary team.
• Analytical skills, proven through submission of a past project MTR and evaluation reports.
• Submission of references to include clients and other team members.

Coordination and Logistics

CBM Global has responsibility for*:*

• Overall coordination of the mid-term review process.
• Recruitment of evaluator.
• Gathering documents and data for the consultant in collaboration with ECWA Eye Hospital.
• Liaising with ECWA Eye Hospital.
• Organizing post-evaluation debriefing with the consultant, including ECWA Eye Hospital in the process.
• Signing off on the deliverables for this mid-term review.

ECWA Eye Hospital has responsibility for***:***

• Working with CBM Global Country Office to organize meeting schedule for the evaluation team.
• Identifying safe, secure, “neutral” and disability accessible locations for interviews/ meetings to take place (where people will feel free to speak as openly as possible).
• Organizing for interviews with project beneficiaries and community leaders according to the consultant’s requests/methodology.
• Organizing for interviews with the stakeholders in eye health in Kano and Jigawa States, according to the consultant’s requests/methodology.
• Organizing for interpreters for the evaluation exercise as required.
• Provision of guidance on security and safety at the field sites.

Payment schedule

The consultant will be paid upon presentation of invoices after each milestone under this consultancy has been completed. At the point of contracting, CBM Global will provide the Consultancy Invoice Information form, which gives the requirements for the formal content of such invoices. CBM Global will only settle invoices in the agreed currency.

The schedule of payment for this consultancy includes:

• Signing Contract: First advance – 25%.
• Submission and acceptance of inception report and data tools: Second advance – 25%.
• Submission and acceptance of final report, datasets, transcripts, and PowerPoint Summary by CBM Global and ECWA Eye Hospital: Final payment – 50%.