Education Evaluation At Norwegian Refugee Council

ABOUT NRC

With the eleventh year of the Syrian crisis, Lebanon remains the country hosting the largest number of refugees per capita, with a government estimate of 1.5 million Syrian refugees, and a large population of Palestine refugees. The country is facing an unprecedented socio-economic crisis, further exacerbated by the COVID-19 pandemic and the Beirut port explosion. This dire situation is severely affecting refugees, host communities and other vulnerable groups, such as migrant workers. Poverty levels are rising, along with difficulties in accessing livelihoods and meeting basic needs. The protection space in the country has drastically deteriorated and protection needs, including legal protection and specialised services, are increasingly being deprioritized by refugees and vulnerable groups, exposing them to additional risks.

OVERVIEW

NRC established its programme in Lebanon in 2006 in response to the July war. The country office is in Beirut, with field offices in South (Tyre), North (Tripoli and Akkar), Bekaa (Zahle) and Beirut and Mount Lebanon (Beirut). NRC implements programmes in four core competencies: Shelter, Water, Sanitation and Hygiene (WASH), Information, Counselling & Legal Assistance (ICLA) and Education and has an Emergency Response Unit (ERU).

NRC’s shelter response includes a settlement-based approach, transitional shelter through the Occupancy Free of Charge programme, and shelter renovations and upgrades for Palestinian refugees and infrastructure support to Lebanese host communities. The WASH programme works to improve access to water, sanitation and hygiene services for refugee and host communities in urban areas, Informal Tented Settlements (ITSs) and public schools including schools’ infrastructure and environments. The Education programme seeks to support refugee, displaced, and host community children enjoy their right to education and achieve their full potential.

In June 2016, NRC was selected as one of few recognized partners of Ministry of Education and Higher Education (MEHE). The education programme includes psychol-social support activities for children; non-formal education support to in- and out of-school children and youth; parental and community engagement, and teacher training. The education program runs through 17 community centres in Bekaa, South, and North Lebanon, assisting over 70,000 children since 2007.

The ICLA programme assists refugees through more than 40 legal clinics and conducts mobile legal clinics that travel to refugee communities throughout Lebanon. Clinics provide group information sessions as well as individual counselling, followed by direct legal assistance, as needed. The ICLA programme covers civil documentation (birth, marriage, divorce and death registration), legal residency, and Housing, Land and Property (HLP) rights.

The youth programme supports refugees and host community youth in achieving their full potential and actively contribute to their community. NRC provides a variety of educational packages that cater to diverse needs of youth and are in alignment with the Government of Lebanon (GoL) policies. Youth between 15-30 years old from different educational backgrounds can participate in one or more of the educational services offered by the programme, based on their interest and willingness to learn. NRC’s Youth Education Pack (YEP) objective is to provide further education opportunities, increase community engagement and access to livelihoods enabling youth to be resilient and active members of their communities.

In December 2017, NRC Lebanon initiated a ERU team in the Bekaa Valley and later in the North. The ERU responds to any emergency cases or situations in our area of interventions. As such, the team assists refugees in need, both in IITSs and/or

sub-standard shelters. The support varies from provision of basic assistance (mattresses and blankets, kitchen sets, hygiene kits, etc.) to shelter kits distribution, site improvements and implementation of WASH activities, and emergency cash assistance.

EVALUATION

3.1 Scope:

The evaluation will cover the NFE programme in support of Syrian refugee children from February 2020 to September 2021.

The evaluation will cover areas in Akkar governorate, North governorate, Bekaa and Baalbeck-Hermel governorate and Nabatieh and South governorates.

3.2 Lines of Inquiry

NRC aims to evaluate its level of responsiveness to the emerging need of distance learning and the extent to which the RL model adopted by NRC was able to meet the programme quantitative and qualitative objectives. The evaluation will ultimately look at learning from the adoption of this new modality, and study outcome achievement, through main milestones:

1. Assess the efficiency of responding to the emerging RL need on the level of time responsiveness, capacity building, team formulation, and families’ preparedness (Efficiency).
2. Evaluate the technology (platform), mode of content delivery (individual-led vs group-led), and dosage of intervention in relation to learners’ academic and social preparedness/achievement.
3. Explore to what extent the stakeholders and working groups’ experience in this domain was considered when selecting the RL approach/model, and how it affected the effectiveness of the programme delivery and achievement (Efficiency/ Effectiveness)
4. Assess the extent to which Monitoring & Evaluation (M&E) tools allowed measurement of indicators and the generation of evidence for prompt programme adaptation, and suggest possible future adaptations to the tools (Effectiveness).
5. Study the effectiveness of NRC’s adaptive management approach followed throughout cycle 1 to cycle 4, taking into account the Lebanon-specific barriers (Effectiveness).
6. Evaluate programme impact on beneficiaries through comparison of outcome monitoring and programme data between: (a) cycle 1 and cycle 4 (reflecting on NRC’s self-learning), and (b) cycle 4 and previous in-person learning model (reflecting on the appropriateness and effectiveness of the RL in specific) (Impact).
7. Check if the existing Theory of Change (ToC) still applies for a remote and blended learning approach (Impact).

Efficiency

• How responsive was NRC in reacting to the emerging need of RL in terms of forming and training teams, and preparing parents for the new model?

• Was NRC able to consult with and build on stakeholder and working group experiences in the domain of RL when structuring its approach/model?

• What were the barriers to timely and quality implementation? How could internal process be improved to address those barriers?

• How did the approach of relying on programme staff (full time) compared to educators during RL to deliver the sessions affect NRC’s efficiency to the response?

• To what extent were families equipped to provide a home learning environment vs learning center, and how did it affect the achievement of learning outcomes?

• How efficiently did NRC respond to adapting the approach and tools at M&E and programme levels to monitor the programme during RL?

• How efficient was NRC in adapting its RL model throughout the programme evolvement from cycle 1 to cycle 4?

Effectiveness

• To what extent were the objectives of the programme achieved?

• What were the major factors influencing the achievement or non-achievement of learning outcomes in remote NFE programming?

• In what ways have the selected RL modalities effectively addressed the needs of all intended population groups, both Lebanese and non-Lebanese, including boys, girls and children with disabilities?

• How effective was the monitoring system (including tools) adopted during RL modality in collecting evidence-based data to answer ToC inquiries?

• What impact did the approach of relying on programme staff (full time) compared to educators during RL to deliver the sessions had on the learning outcomes achieved?

• How did the technology (platform), mode of content delivery (individual-led vs group-led), and dosage of intervention shape the effectiveness of learners’ academic and social achievement?

• How do programme outputs and achievements during RL compare to previous in-person learning modalities?

• Were the elements of Safe and Inclusive Programming mainstreamed in the design and implementation of the RL model? How can safety and the RL modality be improved to ensure increased inclusion?

Impact

• To what extent has the RL modality generated or is expected to generate, significant positive or negative, intended or unintended, high-level effects?

• What is the impact of RL on the engagement of parents in the education of their children and on parental perceptions towards education?

• How could all of the above inform the revision of the current RL modality (e.g. technology used, means of verifications, etc.) to improve programme impact?

• What could be strengthened to achieve greater impact in areas identified as important by children and their parents when using RL?

• How did impact on beneficiaries vary between the RL modality and previous in-person learning modality?

• Is the existing ToC still applicable/ relevant for a remote and blended learning approach? If not, what changes are recommended?*For more details, please refer to the Terms of Reference – TOR in Section 4 of the Document.*