Request for Proposal for Foundations for Learning Endline study At Aga Khan Foundation

TERMS OF REFERENCE

Introduction

The Aga Khan Foundation (AKF) is an agency of the Aga Khan Development Network (AKDN), a group of private not-for profit, non-denominational international development agencies, institutions and programmes that work primarily in the poorest parts of Africa and Asia to improve the quality of life of communities in remote and resource poor areas. In East Africa, AKF works with partners to improve the quality of life by promoting and developing innovative solutions to challenges of development in Kenya, Tanzania, and Uganda. Reflecting the complex and multi-faceted nature of development, AKF programmes in the region encompasses Education, Health and Nutrition, Early Childhood Development, Agriculture and Food Security, Climate Resilience, Economic Inclusion and Civil Society. AKF is concluding an ambitious five-year regional project is seeking a qualified consultant(s) to lead design, data collection, coordination, analysis and reporting for a robust mixed-methods endline study.

Background

The Foundation for Education and Empowerment (F4EE) is a five-year, CAD $57.6 million (GAC: $49 million, AKF Canada: $8.6 million) project funded by Global Affairs Canada (GAC) and AKF Canada that improves the educational systems at the pre-primary and primary level, in addition to strengthening women’s empowerment, and gender equality in five countries: Kenya, Madagascar, Mozambique, Tanzania, and Uganda. The project is executed by Aga Khan Foundation Canada (AKFC) and implemented by Aga Khan Foundation (AKF) units in each country, in partnership with the Aga Khan Schools (AKS), Aga Khan University (AKU), and the Madrasa Early Childhood Programme (MECP) in Kenya, Tanzania, and Uganda, and local Civil Society Organizations (CSOs).

F4EE comprises three components: Foundations for Learning (F4L), Advancing Gender Equality through Civil Society (AGECS), and Advancing Canadian Champions for Development (ACCD). Through these interrelated components, F4EE will: (1) strengthen the delivery of quality, gender-responsive, and inclusive pre-primary and primary education, and other social development services, (2) reduce gender and social barriers to the utilization and uptake of education and other sustainable development services, and (3) enhance the engagement of international and Canadian stakeholders in gender-sensitive and evidence-based development programming.

Scope

The study will cover only the F4L component of the broader F4EE project. The endline study data collection will be conducted across various counties in Kenya (Mombasa, Kilifi and Kwale counties), Tanzania (Lindi Rural, Lindi Urban, Ruangwa, and Zanzibar districts/geographies), and Uganda (Arua and Yumbe districts) from approximately April – June 2025, based on academic calendars and school re-openings in each country. The entire endline study process will take place from approximately January to October 2025.

Table 1 – F4L Study Populations.pdf

The endline study will collect and/or analyze endline data for the following indicators from the project performance measurement framework:

Table 2 – Endline Study Indicators.docx

For the indicators above, the Consultant will validate and contextualize data collection tools used at baseline as necessary, adapt (as necessary) the methodology and sampling design used at baseline, train enumerators and oversee data collection, provide data visualization and analysis in the form of tables and/or charts, and provide narrative descriptions and analysis of these tables and/or charts.

* For these indicators, AKF will provide data obtained from country or partner-specific tools to the Consultant, who will be required to support data quality review, cleaning, and analysis. The Consultant will not be required to collect this data.

** For these indicators, AKF will provide cleaned, analyzed, and aggregated data tables to the Consultant directly for inclusion in the endline report. The consultant will not be required to collect, clean, or analyze this data.

Objectives of the endline study

The overall purpose of the endline study is to obtain data and establish endline values for all target indicators aligned with F4L’s expected outcomes. This study will enable AKF and its implementing partners to evaluate and assess the effectiveness of the programme’s interventions upon completion. The following are the specific objectives of the endline study:

1. To assess F4L’s effectiveness by comparing endline data with targets and baseline data across required disaggregation categories, rooted in F4L’s Theory of Change and quantitative and qualitative findings, to demonstrate where progress has and has not been made.
2. To provide recommendations rooted in evidence and backed by global research, evidence, and literature to inform the design and implementation of future programs, identifying elements of the project that could be scaled or adapted.

The endline study will consider gender and social inclusion needs in the design of instruments, protocols for administration (to the extent possible) and analysis framework.

Main Duties and Responsibilities

The evaluator will be responsible for study design, planning and methodology development; ethical approvals; data collection; enumerators recruitment and training; quality assurance; data analysis; and reporting. The areas of inquiry and methodology for the study are further detailed below.

Overall study approach: The endline study will adopt a mixed methods approach, incorporating a quantitative survey and multiple qualitative research elements in order to provide nuanced, triangulated results and to ensure a clear picture of the endline status for each indicator assessed. Further, the study will allow an assessment of the impact of each intervention when compared with baseline data and targets. The Consultant is expected to select and train enumerators (quantitative) and facilitators (qualitative) who will conduct the data collection in target geographies, although AKF will provide a recommended list of enumerators in each geography for the Consultant’s consideration.

Data collection tools and methods The successful bidder will utilize the tools used at baseline by AKF, making minor adaptations as required. It is preferable for bidders to use Kobotoolbox or another ODK-based platform, but not required. All data collection tools are gender-sensitive and adapted to the local context and language. Instruments include:

• Learning assessments: curriculum- and grade-aligned learning assessments on literacy, numeracy and a third non-academic competency for primary students and school readiness standards for pre-primary students will be conducted.
• Household survey: includes questions related to gendered participation in and barriers to education decision-making, gendered access to education, and knowledge, attitudes and practices regarding gender equality and education;
• School environment and classroom observations: includes gender-responsiveness and secure and quality learning environment standards, knowledge and skills of teachers’ and school heads’ gender responsive pedagogical practices and gender sensitive and inclusion policies, etc.;
• Teacher and school heads surveys: includes topics related to knowledge and skills of teachers’ quality teaching practices, promotion of gender and inclusion, etc.
• Education officers survey: includes topics related to knowledge and skills of education officers’ related to supporting quality education, promotion of gender and inclusion, etc.
• Focus group discussions and key informant interviews: includes questions related to key project indicators for triangulation of quantitative findings and additional insights.
• Document review: includes school transition and attendance rates, where data may be collected by AKF country units.

The tools will mirror the approach used at baseline, which will be shared with the consultant. There will be greater flexibility for adaptations to qualitative tools and methodologies, if recommended by AKF or the Consultant.

Endline sample: The Consultant will use the sampling strategy implemented at baseline for this endline study. For quantitative tools, the sample includes a representative sample of learners, parents/caregivers, teachers and student teachers, schools, and school heads. It will also include a census or near-census of engaged education officers and tutors (see Annex 4 for details). Focus group discussions, key informant interviews and any other qualitative data collection methods should be conducted with a non-representative, purposeful sample of various stakeholders, such as teachers, students, government education system leaders, local civil society and community members. The Consultant is expected to validate and implement the sampling strategy used at baseline.

Ethical Standards and Safeguarding: The endline study must conscientiously abide by AKF’s Global Safeguarding Manual and its adaptations in the study countries, and all members of the Consultant team must sign AKF’s Safeguarding Statement of Commitment upon contracting. The Consultant must also be sure to obtain the relevant research permits, as required, from all relevant national authorities (e.g. Mildmay Uganda research and Ethics Committee (MUREC)). Codes of conduct must be specifically referenced, and commitment confirmed in writing by each researcher involved in this study. The design of the study must clearly specify how data collection and use will be undertaken with integrity and honesty, respecting human rights and differences in culture, customs, religious beliefs and practices of all stakeholders. The Consultant must explain how its researchers will be mindful of gender-related needs and roles, ethnicity, children living with disability, age, language and other differences when designing and carrying out the study. The design and implementation of the study must strike an appropriate balance between recognition of the potential benefits of the research, and the protection of participants from potential research-related harms. Safeguarding principles to protect key informants from sexual exploitation and abuse, sexual harassment and bullying, including child protection, must be explicit.

1. Deliverables

The main deliverables for this project are as follows:

• Inception report: A detailed inception report per the requirements agreed with AKF. This is expected to include the following elements:
  • Study methodology, design and sampling strategy, following provided sample sizes, for quantitative components. Study methodology, design, sampling strategy, and sample sizes for qualitative components
  • Detailed work plan that includes all tasks by the Consultant and team members and incorporating the overall study timelines[2], including contingency planning.
  • Level of effort of each team member
  • Description of enumerator and facilitator training and piloting exercises
  • Procedures for data collection, management, and analysis
  • Safeguarding and child protection, ethics and risk management
  • A finalized data analysis plan which contains all definitions and computations for each indicator, along with updated planned tables and figures (dummy tables)
• Data collection: Documentation of the following: 1) data collection tools translated (where translation is required), 2) ethical and other national government research approvals obtained, 3) data collectors recruited and trained, 4) data output tables; 5) data collection, entry (where applicable), and quality assurance completed.
• Endline study report: Completed endline study report including quantitative and qualitative endline study findings, completed dummy tables, data visualizations, and analysis of findings. The reporting structure should follow the requirements established with AKF, and will include the following data as annexes:
  • Endline datasets: Raw data and complete, cleaned quantitative datasets in English. The Consultant will also be expected to provide a fully ‘cleaned-up’ dataset in an agreed file format (i.e., XLSX) accompanied by the code used to carry out analysis and a variable codebook. Transcribed and translated transcripts: Complete, edited transcriptions and translations of all qualitative data and a qualitative codebook or equivalent in English.
  • Presentations of results: A summary of the report results in PowerPoint format, highlighting key findings and recommendations for adaptive management.

Report requirements: All reports should be submitted in electronic form and should be submitted in English.

Detailed work plan: Bidders are required to provide a detailed work plan incorporating all relevant tasks and milestones from start to finish of the endline study.

Project milestones: Bidders are required to include in their detailed work plans the milestones set out below.

Table 3.Project Milestone .docx

1. Qualifications Required

   Bidders are required to clearly identify and provide CVs for all those proposed in the Study Team, clearly stating their roles and responsibilities for this evaluation. Please note that the Consultant will be ultimately responsible for the enumerators and facilitators they are subcontracting to.

The proposed Consultant team should include the technical expertise and practical experience required to deliver the scope of work and evaluation outputs, in particular, with regards to:

• Minimum of 10 years of experience in administering studies, collecting data and producing quality mixed-methods baseline/endline education focused study reports, preferably for international non-profit organizations or multilateral agencies and multi-country studies.
• Demonstrated experience in designing baseline and endline studies including proven experience in sound sampling, mixed methods approaches (quantitative and qualitative), tool development, enumerator training, etc.
• Demonstrated experience recruiting, training and managing enumerators and facilitators for both quantitative and qualitative studies (either in person or remotely)
• Demonstrated experience in quantitative data and statistical analysis, using software such as SPSS or Stata.
• Demonstrated experience in qualitative data analysis techniques, including the use of software e.g., ATLAS.ti, NVivo or equivalent.
• Demonstrated experience in employing Kobotoolbox, Open Data Kit (ODK) or other computer assisted personal interviewing software.
• Specialization in primary and pre-primary education, conducting studies with children, and gender equality issues and analysis.
• Demonstrated experience in data management, including designing and managing gender- and age-disaggregated data and information systems capable of handling large datasets for study purposes.
• Country knowledge and experience in conducting research and impact evaluation studies in Kenya, Tanzania, and Uganda.
• Demonstrated experience safeguarding and respecting vulnerable populations.

Fluency in English is mandatory. Additional relevant languages (e.g., Swahili), an added advantage

1. Evaluation Criteria

Proposals will be evaluated based on the following set of criteria:

CRITERIA MAXIMUM SCORE

Technical Component

Demonstrated experience in conducting and coordinating similar studies 25

Education and Technical Experience of Team 15

Proposed Methodology and Approach 30

Gender Equality experience and proposed approach 10

Total Technical Component 80

Financial Component

Realistic and appropriate costs for proposed work 10

All relevant costs included to carry out proposed work 10

Total Financial Component 20

TOTAL 100

Annex 2: Data Analysis Plan

Annex 2 – Endline Data Analysis Plan.docx

Annex 3: Preliminary Dummy Tables

Annex 3 – Preliminary Dummy Tables.xlsx

Annex 4: Indicator Grouping and Sampling Matrix

Annex 4 – Indicator Grouping and Sampling Matrix.xlsx

Word versions of all tools are available, excluding document review tools. Kobo/ODK XLSform versions are available for most quantitative tools, excluding most learning assessment tools, which would require adaptation to Kobotoolbox or another digital data collection platform.

Final timelines will need to be coordinated with AKF country offices and the regional project management team. The AKF regional project management team will facilitate this with the consultant.