Learning through Education and Access to Skills for Employment (LEAP) Endline Evaluation At World University Service of Canada

World University Service of Canada (WUSC) is a Canadian non-profit organization with a mission to enhance education, economic opportunities, and empowerment for youth. We work with a diverse network of students, volunteers, institutions, governments, and businesses. Together, we foster youth-centered solutions to overcome inequality and exclusion.

WUSC has been awarded a grant of $12.4 million from Global Affairs Canada to implement the “Learning through Education and Access to Skills for Employment” Project (the LEAP Project). The ultimate outcome of this project is to increase the empowerment of adolescent girls and young women in Kalobeyei Settlement, Kakuma Refugee Camp, and the surrounding host communities in Turkana County, Kenya. The project will enhance education opportunities for adolescent girls, and will also provide access to gender responsive and market-based skills training in Kalobeyei and Kakuma Refugee Camp for young women.

Background of the LEAP Project

The project’s expected outputs, and immediate, intermediate, and ultimate outcomes are presented in a Logic Model which is annexed to this Terms of Reference. The LEAP Project is designed to address gender, social, cultural and economic barriers that its targeted beneficiaries face in accessing education, and making life decisions. The ultimate outcome of the project is enhanced empowerment of adolescent girls and female youth in the targeted regions of Kalobeyei Settlement, Kakuma Refugee Camp, and surrounding host communities. The project specifically targets two of the most critical moments in a girls’ educational journey: when they are most at-risk of dropping out of the education system (late primary and the transition to secondary); and the transition from basic education to post-secondary education or employment.

The project seeks to improve learning outcomes for adolescent girls and female youth, as well as to enhance equitable participation in the formal and informal workforce for young women. The LEAP Project responds to specific barriers and educational needs identified by adolescent girls, female youth, community members, and education stakeholders through gender analyses, and aims to address the root causes of inequality by challenging and helping to transform structural barriers and entrenched social norms, attitudes, and power relations that impact girls’ ability to access and benefit from education and skills training opportunities. The LEAP Project will implement a range of interventions including teacher training, remedial education classes, cash transfers, educational technologies, media campaigns, community outreach, Guidance Teacher training, mentorship programs, market-based skills training, and training in digital skills.

Purpose and Objectives of the Endline Study

WUSC is seeking qualified consultant(s) or a consulting firm to lead a mixed method endline evaluation. This endline study is a contractual obligation included in the project’s Contribution Agreement (CA) with Global Affairs Canada. The purpose of the endline study is to provide key project stakeholders with evidence on the project’s achievements related to a set of expected outcomes and to empower them to collaboratively draw learnings and identify recommendations to foster sustainability after the closure of the activities. Aligned with this purpose, and based on the methodology designed for the baseline study as much as possible, the endline study is expected to achieve four specific objectives:

• To establish endline values and analysis for key performance indicators of the LEAP Performance Management Framework (PMF), as well as measures of the level of achievement of each target;
• To assess the validity of the assumptions identified in the project’s Theory of Change (ToC);
• To assess the effectiveness of the Power to Girl (PtG) methodology, in relation to the project’s expected outcomes, as well the emergence of unexpected outcomes;
• To provide an overall assessment of LEAP, based on a set of the OECD-DAC evaluation criteria as well as evaluative questions identified by the team;
• To identify lessons learned and recommendations for partners and stakeholders, to foster the sustainability of the achieved results and promote the replicability of successful interventions.

The Mandate of the Consultancy

The Consultant (or consultancy firm) will serve as the technical lead for this endline evaluation and will be responsible for revising and updating the methodology and tools developed for the baseline study, coordinating with the LEAP team throughout the exercise to facilitate data collection and interpretation, analyzing the collected data, preparing a final consolidated report, delivering a validation workshop with they stakeholders and inserting the validated data into the Performance Measurement Framework. The Consultant will design and facilitate a plan in close collaboration with the LEAP team. Specifically, the mandate of the consultancy is to:

• Finalize the design of the endline methodology and tools: Work with designated LEAP team members during an inception phase to revise the study design developed at baseline (including data collection methodologies, tools, data analysis plans, etc.) for the purposes of the endline study. This includes a revision of all data collection tools deployed by the LEAP team for the annual outcome assessment, as well as the tools already in use for the assessment of the PtG component. The final design will also contain a detailed timeline and necessary contingency plans;
• Conduct the data collection: Be responsible for all aspects related to the deployment of all enumerators needed for this exercise (selection, contracting, training and supervising). The consultant (or consultancy firm) will coordinate with the LEAP team for logistical support;
• Conduct data quality control processes: Ensure quality throughout the data collection, data entry (as needed) and data analysis processes, addressing any data quality issues that emerge;
• Conduct data analysis and compile consolidated findings: Analyze the all collected data in line with the endline data analysis plan, integrate the findings to create a final consolidated endline study report that meets all the objectives outlined above in section 3.

The endline study will be managed by the LEAP team, in close collaboration with the WUSC’s MERL Advisor. A more detailed consultative and decision-making management structure with clear roles and responsibilities will be determined during the inception phase of the study.

Scope and Methodology of the Endline Study

The following elements should guide the revision of the design and implementation of this endline study:

• The study should consciously apply qualitative and quantitative research methods that are gender and youth-sensitive, and engage as much as possible in participatory and consultative approaches;
• The geographic scope of the study will be aligned with the targeted areas, schools, households and individuals targeted by the project as laid out in the baseline report and all other relevant documents;
• The data collection approaches and specific tools used to establish indicator endline values should be adapted to the local contexts, to assure the appropriateness and reliability of the data collected.

The endline study is expected to collect data for all outcome indicators in the Project Measurement Framework (PMF). Outcome statements, indicators, sources and data collection methods for each indicator are included in the project PMF and summarized in the table below. While the Consultant, informed by discussions with the LEAP team is encouraged to make suggestions regarding methodologies, the PMF outlines the following data collection methods or evidence streams that were used during the baseline and should ideally be replicated at endline:

1. Literature and document review, including reference to rapid gender equality analyses conducted, baseline reports and annual reports, outcome monitoring reports as available, and extraction of data points from education management information systems, whenever available and appropriate;
2. Key informant interviews;
3. Facilitated small group or focus group discussions;
4. Surveys, at school and/or household level.

The Consultant will be expected to review the methodology/tools used at baseline for collecting data on all LEAP’s Project Outcome Indicators and propose detailed refinements as appropriate, using a combination of the above mentioned qualitative and quantitative methods.

A preliminary list of indicators and research questions can be found below for initial reference. The selected consultant will be provided with the final list of indicators. The final list of research questions will be discussed and finalized during the inception phase.

Quantitative Outcome Indicators

1. Percentage of adolescent girls and young women who successfully progress and complete learning levels (by age, learning level, institution type, location, refugee status)
2. Percentage of learners showing improved academic performance in national and school-based examinations (by sex, age, refugee status, grade level)
3. Completion rate to market-based skills training opportunities of young women (by age, type of training, refugee status)
4. Enrollment rate at upper primary and secondary for targeted adolescent girls and young women (by age, learning level, refugee status)
5. Retention rate at upper primary and secondary for target girls and young women (by age, learning level, refugee status)

Qualitative Outcome Indicators

1. Degree to which adolescent girls and young women make decisions about their education, employment pathway and personal domains (disaggregated by age, location, refugee status)
2. Perceptions of girl/boy learners and teachers on changes, if any, in school safety, quality of learning in the classroom.
3. Percentage of girls, boys, men, women and community leaders with improved knowledge about concrete measures to reduce barriers for adolescent girls and young women accessing education (disaggregated by sex, age, location, refugee status, role)

4. Level of awareness of girls about themselves and their rights (by age, refugee status)

5. Percentage of adolescent girls and young women with improved self-reported levels of self-esteem (by age, refugee status)

6. Percentage of vocational program trainees at partner institutions who are employed post-training (formal or informal) (by sex, age, type of training, refugee status)

7. Perception of young women that their male partners or male family members are taking on additional household responsibilities in support of young women’s economic activities

8. Perception of young girls and adolescent women on the types of concrete support received by school communities, families and leaders (disaggregated by age, refugee status).

9. Perception of community members about the causes of violence against girls and women, and how to address them (disaggregated by sex, age, refugee status)

10. Percentage of girls, boys, men, women and community leaders reporting a positive attitude towards decision-making by adolescent girls and young women on their employment pathways (disaggregated by sex, age, refugee status, role)

11. perception of girls, boys, men, women on attitude towards decision-making by adolescent girls and young women on their employment pathways (disaggregated by sex, age, refugee status).

12. Perceptions of young women on changes in the nature of barriers they face in accessing and remaining in skills training programs (disaggregated by sex, age, refugee/host, type of training program).

Research questions

1. As the project review at midterm led to a few adjustments in the approach adopted for the implementation of some components, did they prove to be better aligned to the girls’ needs in the targeted areas?
2. Has the improvement of non-academic outcomes contributed to the improvement of academic outcomes? Has focusing on improving non-academic outcomes (over academic) had a greater or lesser impact on girls’ overall empowerment? (This was among the key assumptions that led the midterm review of the project, which resulted in a broadened definition of “learning” beyond academic performance measured through national examinations).

Key Activities and Deliverables of the Consultancy

The endline evaluation will include four key phases:

1. Contracting
2. Inception phase
3. Data collection
4. Data analysis, report writing and dissemination.

After successfully completing the contracting process, the Consultant will immediately organize an initial round of consultations with key stakeholders, as well as complete an initial document review to better understand the project, its contexts and its informational needs. The Consultant will use the inception phase to assure a consensus on the endline study’s design, outline the details of the revised methodology for the study, and update the data collection tools.

Following approval of the inception report by the LEAP team, the Consultant will recruit, train and deploy the enumerators needed for the data collection.

Once data collection is completed, the Consultant will focus on the analysis of collected data and drafting of the final report, which will include an updated PMF with endline values, as well as insights on measurability and limitations to the adopted methodology whenever appropriate. During this phase, the Consultant will engage with the LEAP team to refine the final report and organize the validation workshop.

Based on the implementation timeline, the initial draft of the final report must be shared with the LEAP team and key stakeholders for validation no later than the 5th of December 2024. The final endline evaluation report must be submitted to the LEAP team by the 15th of December 2024.

An overall timeline for the key deliverables is laid out in the table below and will be confirmed during the inception phase.

Key Activities and Milestones

• Inception meeting to confirm understanding and scope of mandate;
• Literature and document review
• Consultations with LEAP team on the refinement of the methodology of the study as well as all the data collection tools
• Draft and submission of inception report
• Submission of the ethical protocols, as required by the relevant national authorities
• Identification, recruitment and training of enumerators
• Training of data collectors supervision by the consultant on data quality as needed
• Collection of all quantitative and qualitative data
• Data analysis (including but not limited to the calculation of the endline value for each of the outcome indicators in the PMF)
• Draft and submission of endline report
• Incorporation of feedback from LEAP team and submission of final report for approval

Phase-Specific Deliverables

• Inception report, including: detailed revised methodology and its limitations, all revised data collection tools and consent forms, detailed workplan, protocol and material for enumerators training; (A detailed table of contents for the inception report will be shared at the beginning of the contract)
• Enumerators’ training;
• Facilitation of a validation workshop, with all project teams, partners and key stakeholders;
• Final endline report, including: updated PMF with baseline and endline values for all outcome indicators, limitations to the methodology used for the study, lessons learned and recommendations on the sustainability of achieved results, evaluation insights for future replicability; (A detailed table of contents for the final report will be shared during the inception phase)
• All raw data.

Tentative deadline for deliverables

• Inception report: 20th September 2024
• Data Collection: October 2024
• First draft of the endline report: 15th November 2024
• Presentation for validation workshop with key stakeholders: 5th December 2024
• Final Consolidated Report: 15th December 2024

Ethical Standards

The research for this baseline study must abide with WUSC’s Code of Conduct and Child Protection and Safeguarding Policies, as well as by relevant recognized professional and ethical guidelines and codes of conduct for individual researchers. The Consultant must also be sure to obtain the relevant research permits, as required, from Kenyan authorities (i.e. NACOSTI). Codes of conduct must be specifically referenced and commitment confirmed in writing by each researcher involved in this study. The design of the study must clearly specify how data collection and use will be undertaken with integrity and honesty, respecting human rights and differences in culture, customs, religious beliefs and practices of all stakeholders. The Consultant must explain how its researchers will be mindful of gender roles, ethnicity, ability, age, language and other differences when designing and carrying out the study. The design and implementation of the study must strike an appropriate balance between recognition of the potential benefits of the research, and the protection of participants from potential research-related harms. Safeguarding principles to protect key informants from sexual exploitation and abuse, sexual harassment and bullying, including child protection, must be explicit.

Qualifications of Consultant

• Lead researcher/firm must demonstrate a minimum of 10 years of experience in administering mixed-method studies, collecting data and producing quality baseline/end line study reports, preferably in education and for international non-profit organizations or multilateral agencies;
• Demonstrated experience in designing baseline and end line studies including proven experience in participatory research methods and feminist evaluation approaches;
• Demonstrated experience of delivering research in complex settings, with refugee/IDP populations, and in similar contexts to the LEAP target geographies;
• Excellent facilitation skills and ability to recruit and manage enumerators for both the quantitative and qualitative components, in the three countries of the study (either in person or remotely);
• Demonstrated experience in quantitative and qualitative data analysis;
• Knowledge and experience with education policies in Kenya will be considered an asset;
• Knowledge and experience with livelihoods or skills training programs;
• Knowledge and experience in gender equality issues in the education sector is mandatory; specific experience in data collection on gender norms is desirable;
• Fluency in English is mandatory and in Kiswahili an asset;
• Demonstrated capacity to respect and safeguard vulnerable populations; and
• Ability to produce high quality work under tight timeframes.

Application Process

The LEAP project team will be in charge of the selection, recruitment and supervision (as appropriate and needed) of the enumerators and all logistical aspects and budget related to the field work. Additional details on this aspect of the process will be discussed during inception.

Qualified and interested parties are asked to submit the following:

1. Detailed technical proposal (8 to 9 pages maximum) clearly demonstrating a thorough understanding of this ToR and including the following:

• Description of the capacity and qualifications of the consulting firm, including previous relevant experience (1.5 to 2 pages)
• Description of the proposed approach and methodology, including sampling strategy, selection of enumerators, data analysis, integration of gender considerations and ethical standards, quality assurance throughout the process (3.5 to 4 pages)
• A proposed schedule/work plan (i.e. a Gantt chart), including contingency planning as appropriate (1 page)
• Team composition and level of effort of each proposed team member (1 to 1.5 pages pages)

2. A financial proposal (in CAD) with a detailed breakdown of costs for the study:

• Itemized consultancy fees/costs
• Itemized administrative expenses
• Validity period of quotations
• Expected payment plan and method

3. Curriculum Vitae of all proposed team members outlining relevant experience (max 3 pages per CV)

4. Names and contact information of three references who can be contacted regarding relevant experience

5. A copy of a previous report of similar work undertaken