Consultancy to conduct an End line Evaluation for the In Their Hands (ITH) Project At Aga Khan Foundation

Introduction

The Aga Khan Foundation (AKF) is an agency of the Aga Khan Development Network (AKDN), a group of private not-for profit, non-denominational international development agencies, institutions and programmes that work primarily in the poorest parts of Africa and Asia to improve the quality of life of communities in remote and resource poor areas. In East Africa, AKF works with partners to improve the quality of life by promoting and developing innovative solutions to challenges of development in Kenya, Tanzania, and Uganda. Reflecting the complex and multi-faceted nature of development, AKF programs in the region encompass Education, Health and Nutrition, Early Childhood Development, Agriculture and Food Security, Climate Resilience, Economic Inclusion and Civil Society.

Programme Background.

The In Their Hands (ITH) Uganda is a one-year pilot programme that aims to empower Ugandan adolescent girls to choose when they get pregnant, so that they might reach their full potential. The ITH programme is funded by Children Investment Fund Foundation (CIFF) and Aga Khan Foundation (AKF) and implemented by the Aga Khan Foundation (AKF) and Triggerise. The ITH is an expansion programme and adapted from ITH Kenya.

The programme is based on a mobile application platform (Tiko/ITH Platform) that offers SRH messaging, links teenage girls with local clinical and pharmacy service providers and promotes service uptake through a rewards-based system in which girls earn points (Tiko Miles) that can be redeemed for products (such as soap, cream, sanitary towels, etc.) at participating retailers.

The key innovation and adaptation of the ITH Uganda program was the inclusion of a comprehensive community-based component, comprised of several strategies to work with adolescent girls and key influencers to support their broader health and development and address determinants of health seeking services, specifically those related to SRH. Specifically, the community-based demand generation strategy is comprised of community-based girls’ and boys’ groups, community-based parents’ groups and the establishment of local communities of practice. One of the key operational priorities is to demonstrate proof of concept for the innovative strategies and materials.

The ITH pilot programme objective was to ensure that 20,000 adolescent girls 15-19 years access SRH services through the ITH platform in Arua (7000) and Kampala (13000). This was to be achieved through, (1) Improving agency, knowledge, and desire of adolescent girls to use SRH services, (2) Increasing and sustaining access to quality, relevant SRH products and services among adolescent girls and (3) Increasing community and government support of adolescent girls’ SRH uptake.

AKF would like to evaluate the ITH programme and is seeking a qualified consultant(s) to lead design, data collection, study coordination, data analysis, and reporting for a mixed methods endline study.

Purpose the evaluation

The overall purpose of the end-line study is to assess the performance of the project and capture project achievements against the project objectives, challenges, and best practices to inform future similar programming, including evaluating the added-value of the innovative community-based strategies. It will also generate recommendations for improving design and implementation of the ITH project scale up as well as provide end line indicator scores against ITH expected outcomes.

Specific objectives

The end line evaluation will adapt the OECD framework for evaluating programmes which will focus on; impact/goals and objectives of the programme, effectiveness, efficiency, sustainability, relevance complemented with best practices and lessons learnt from the programme. This has been detailed to support the programme in finding answers to the following specific objectives.

1. To assess the relevance and coherence of the In Their Hands programme strategies (digital app and community-based safe space groups) with the Ministry of Health, Local governments, CSO implementing partners and the target beneficiaries in Arua and Kampala.
2. To ascertain the extent to which the ITH programme achieved its intended objectives during the implementation of activities in Kampala.
3. To critically assess the project design and delivery, identifying key challenges, successes and best practices, lessons learned and draw recommendations for future programming around Adolescent health and well-being. This will present a chance for accountability to ITH program donors and the project’s beneficiaries.
4. To ascertain the most efficient approaches and strategies used in achieving the ITH programme goals, specifically evaluating the added-value of the community-based safe-space groups and proof of concept.
5. To assess the extent to which the program achievements and systems can be sustained and how.

Scope

The study will be conducted in Arua city and Vurra Subcounty, Makindye, Central and Kawempe divisions in Kampala.

The study population will include adolescents, parents group members, women, men, boys, peer group members, district officials, Community of Practice Members, Partner clinics, and selected Ministry of health officials. The list of indicators is presented in the table below:

Performance measure or indicator Methods of data collection

• % Reduction in pregnancies among young girls (15-19) -KIIs, Questionnaire and document review
• % Reduction in adolescents receiving Post Abortion Care services – KIIs, document review
• Contraceptive Prevalence Rate among adolescent girls -Review of facility documents, survey data, questionnaires, FGDs and KIIs
• % Increase in the Couple Years of Protection (CYPs) for adolescents 15-19 years -Review of facility records, interviews
• % Of adolescent girls reporting satisfaction with SRH solutions on the ITH platform – Interviews
• % Adolescents on the ITH platform taking up SRH Services – Document review
• % Adolescent girls group members showing improved self-efficacy in relation to SRH – Interviews
• Proportion of adolescents that rate received SRH services – Interviews
• %Adolescent girls who accessed contraceptive services through the ITH platform – Document review
• % Adolescent girls who accessed HIV services through the ITH platform – Document review
• % Adolescent girls who have sustained usage on the ITH platform for more than 4 months – Interviews
• % Of adolescent girls enrolled onto the ITH platform to uptake of services through mobilizer pathway – Document Review
• % Conversion rate from enrolment onto the ITH platform to uptake services through peer-to-peer pathway
• % Of adolescent girls enrolled onto the ITH platform to uptake services through Girls group pathway – Interviews
• Adolescent girls rating services at clinics on the ITH platform – Interviews
• Adolescent girls rating services at pharmacies on the ITH platform – Interviews
• % Parents group members reporting improved knowledge and support for adolescent SRH – KIIs and FGDs
• % Partners (boy’s group members) reporting improved knowledge and support for adolescent SRH – KIIs and FGDs
• % Community of Practice members that report improved knowledge and attitudes towards ASRH – Significant Stories of change tool, interviews & KIIs

Evaluation methodology

• Research designs

The Consultant is expected to adopt a mixed-methods approach in conducting the evaluation. The methods used should be a combination of participatory qualitative and quantitative data collection techniques to allow a comprehensive examination of all aspects of the programme, including those specifically tailored to adolescents. The consultant will also be expected to review secondary data to inform the assessment of the evaluation objectives. Comparisons are to be made against the capacity assessment, existing literature, and practice of similar interventions. To accomplish the goals of this study, the consultant will prepare the study methodology, construct the sample frame, adapt the baseline data collection tools, and prepare additional tools as needed, data collection; enumerators recruitment and training; quality assurance; data analysis and reporting. Before the study, the approach and pertinent tools will be refined with input from Aga Khan Foundation and Triggerise, pilot tested, and finalized.

• Data

Data will be collected from the Arua and Kampala project locations. All primary data collected during the evaluation must enable disaggregation by important socio-demographic factors. Data triangulation is expected for this evaluation; the secondary data available is expected to provide additional insight towards the triangulation of the collected data. A range of project documentation that provides information about the project design, implementation, and progress, including the baseline datasets for comparative analysis will be made available. The consultant is required to adhere to the Aga Khan Foundation and Triggerise’s data protection and privacy policy throughout all project evaluation activities, minimizing the collection of any non-essential personally identifiable information and ensuring secure storage and transfer of data. In addition, the consultant is expected to explore any personal and professional influence or potential bias among those collecting and/or analyzing data and mitigating them ethically.

Ethical Standards and Safeguarding

The baseline study must conscientiously abide by AKF’s Safeguarding Manual, and all members of the Consultant team must sign AKF’s Safeguarding Statement of Commitment upon contracting. The design and implementation of the studies must be in line with the research ethical standards including protection of participants from potential research-related harms, confidentiality, informed consent, beneficence, and safeguarding principles to protect respondents from potential harms such as sexual exploitation and abuse, sexual harassment, and bullying. The consultant will seek approvals at district level to gain entry permission to the districts.

Deliverables

The main deliverables and activities for this project are as follows:

• Inception report: It will include:
  • Evaluation objectives and evaluation questions
  • Description of the methodology, data sources, and sampling considerations
  • Risk and issue management plan
  • Work plan
  • Report structure
  • Draft data collection tools
  • Ethical considerations and evaluation team (List of enumerators plus their qualifications and experiences). – 25th September
• Finalized data collection tools
  • Survey instrument (review and digitalize for Kobo Collect)
  • FGD guides
  • KII guides
• Training enumerators and Pilot study
• Data collection
• A draft evaluation report, including the following elements:
  • Executive summary
  • Background description of the project and context relevant to the evaluation
  • Scope and focus of the evaluation
  • Evaluation methodology and limitations
  • Findings aligned to each of the key evaluation questions
  • Conclusions outlining implications of the findings
  • Recommendations
  • Annexes (Project workbook, evaluation TOR, inception report, study schedule, list of people involved) – 20th November.
• Validation workshop: Participatory workshop(s) with staff, ministry and district officials, ITH partners, adolescents, parents and other stakeholders to present, review and validate preliminary evaluation findings.
• Final endline evaluation report: Completed endline study report (the report should be disaggregated at district level for understanding district relevant situations) including quantitative and qualitative endline study findings, completed dummy tables, data visualizations, and analysis of findings. Incorporating feedback from consultation on the Draft. – 10th December.
• Data and analyses, including raw data, MIS database, analytic code, variable codebook as well as analysis outputs
  • The Consultant will be required to deliver a fully “cleaned-up” dataset in an agreed-upon file format (such as XLS or SPSS)
  • Transcribed transcripts: Complete, edited transcriptions and translations of all survey responses, focus group discussions and interviews conducted in English.
  • Analytic code and a variable codebook.

Key competencies

• Team lead should possess at least a master’s degree in public health, demography/ population studies, Statistics and with essential training in Monitoring and Evaluation. Any training in public policy is an added advantage.
• Other team members should ideally possess multidisciplinary qualifications across gender, social science, social work, clinical/ human medicine, and related qualifications.
• Strong experience carrying out surveys and associated qualitative data collection, particularly focusing on adolescent SRH or related thematic areas.
• A sample of three (3) evaluation reports on SRHR and related studies during the last 4 years and with studies done in the areas of programme implementation as an added advantage.
• The lead should have experience of conducting evaluations with and for adolescents for at least 8 years.
• Demonstrated experience in designing and implementing endline and endline studies including proven experience in sound sampling, mixed methods approach (quantitative and qualitative), tool development, enumerator training, etc.
• Demonstrated experience in employing Open Data Kit (ODK) or other computer assisted personal interviewing software.
• Demonstrated experience in data management, including designing and managing SRHR projects and capable of handling large datasets for study purposes.
• Country knowledge and experience in conducting research and impact evaluation studies Uganda.
• Demonstrated experience safeguarding and working with vulnerable populations.

Any queries and clarity regarding this consultancy can be directed to: akf.uganda@akdn.org and only within the first 5 days after publication of this call for proposals.