CONSULTANT TO CONDUCT END OF PROJECT EVALUATION At Adventist Development and Relief Agency International

REQUEST FOR CONSULTANTS

The Adventist Development and Relief Agency (ADRA) Somalia is an International Non-Governmental Organization registered in Kenya and operating in Somalia. ADRA Somalia is seeking a consultant to carry out End of Project Evaluation.

SETS Project Background

The overall objective of SETS was ‘expanded education and training opportunities, contributing to poverty reduction within a peaceful, secure and democratic Somalia’; while the specific objective was ‘quality education and vocational training services, efficiently delivered to the target populations in Somalia.’

SETS specifically sought to support the local government both at national and state levels to address its education priorities and inclusive education needs in Banadir, Jubbaland, South West, Galmudug and Hir-Shabelle states as well as Banadir Regional Administration. The project was expected to contribute to a conducive environment for returnees, IDPs, pastoralists and other groups to be re-integrated in the education system. Minority groups, children, women, persons with disability, girls and youth were targeted for inclusive quality education and vocational training services, in order for them to participate in activities leading to poverty reduction, and a peaceful, secure and democratic Somalia.

SETS was planned to strengthen the roles and responsibilities of the local communities, institutions and the government to plan, implement, relevant policy changes that contribute to expanded education and training opportunities for all Somalis, which in turn would contribute to poverty reduction, peace, security and democratic advancement. The action targeted groups and institutions at both the macro and micro-levels of the education economy for sensitization and capacity strengthening so that they play a lead role in influencing policy decisions that contribute to reconstruction, inclusivity and growth of education and training services.

Notably, SETS project sought to consolidate and improve the existing education and training opportunities and provide additional good-quality basic education and vocational training services to the target populations in Somalia.

Targeted Primary Beneficiaries

Target beneficiaries included primary/secondary school pupils/students and teachers, parents, Community Education Committee (CEC) members and the Community Skills Development Centres (CSDC).

Additionally, there were interventions aimed at benefiting youths in TVET, those attending Non-Formal Education (NFE), Alternative Basic Education (ABE) and pupils with special needs. Overall, there was also a keenness to target children who are returnees, Internally Displaced Persons (IDPs) and/or pastoralists.

Based on lessons learned from previous projects and forging strong synergies and collaboration with other on-going interventions within the project regions, SETS aimed to ensure that equal education opportunities are provided to all.

2.1 Implementing Partners

The project was implemented by ADRA as the lead agency, in partnership with the Somalia Federal Ministry of Education, Culture and Higher Education (MoECHE), State Ministries of Education of: Galmudug, South West, Hir-Shabelle, Jubbaland and Banadir Regional Administration; as well as CISP, World Vision, Alight and Relief International who were implementing agencies.

2.2 Purpose and Objectives of the Evaluation

These Terms of Reference are for the End of Project Evaluation for the Strengthening Education and Training in Somalia (SETS) Project. The overall purpose of the evaluation is to assess the level of attainment of targets, results and objectives. The evaluation will contribute to shared learning and provide accountability to partners, beneficiaries and donors. The evaluation will come up with findings, lessons learnt and recommendations which will be shared with key stakeholders of the project and used by the implementing agencies to guide and inform future similar projects and programs.

The evaluation will assess the performance of the project against key parameters including the project’s relevance, effectiveness, efficiency, sustainability, timelines of activity implementation, and its strengths and weaknesses. In addition, the strengths, weaknesses and challenges during implementation will be analyzed with a view to extract lessons and best practices for the future.

2.3 Objectives of the Evaluation

The final evaluation will provide the decision-makers in the Government, the European Union and the wider public with sufficient information to:

a. Make an overall independent assessment about the past performance of the project/programme, paying particularly attention to the impact of the project actions against its objectives.

b. Identify key lessons and to propose practical recommendations for follow-up actions.

The evaluation will be based on the five evaluation criteria endorsed by the OECD-DAC (relevance, effectiveness, efficiency, sustainability and impact), and to the EC-specific evaluation criteria (EC added value and coherence). The evaluation will also assess the integration and impact of cross cutting issues in the project. The consultants are required to use their professional judgment and experience to review all relevant factors and to bring these to the attention of the Government and European Commission.

The key parameters to be assessed by the evaluation are:

a. Relevance: describes how well a project addresses a real problem of the beneficiaries and how well it matches the development policies and strategic objectives of the country/region.

b. Efficiency: stands for how well the inputs are transformed into output and outcomes.

c. Effectiveness: measures the degree to which the project’s outputs have provided benefits and contributed to the project purpose.

d. Impact: describes how and to which degree the project has contributed to the solution of the problem and to the achievement of the overall objective. While actual impact can only be measured ex post, the evaluation should nevertheless scrutinize the impact prospects, i.e. the project’s likely contribution to the project’s overall objective.

e. Sustainability: introduces a time dimension into the monitoring. It measures to the likelihood of a continuation in the stream of benefits produced by the project after the period of external support has ended.

f. Inclusion: Assess the extent to which marginalized groups have been reached by the project, in particular girls, IDPs, returnees, refugees, people living with disabilities and those from minority clans among other marginalised groups.

g. Role of federal member states: The roles played by the states of Galmudug, Jubbaland, Hir-Shabelle, South West State and Banadir Regional Administration during the project implementation period, including successes, lessons learnt and gaps identified per State. The support provided to each state also needs to be evaluated.

h. Mutual reinforcement (coherence): assesses the extent to which activities undertaken allow the European Union to achieve its development policy objectives without internal contradiction or without contradiction with other Community policies, i.e. the extent to which they complement partner country’s policies and other donors’ interventions.

i. EU value added: reviews the connection to the interventions of other programs supported by the European Union, and the extent to which the intervention is creating actual synergy (or duplication) with the intervention of other EU supported programmes/projects.

2.4 Scope of the evaluation

The evaluation will cover the accomplishment of all the expected results as outlined in the project document and detailed in the annual work plans during the period of implementation All the project result areas, outcomes and activities stated in the logical framework matrix of the project will be assessed.

The evaluation team will select sample districts, schools, and communities from the project regions in consultation with relevant stakeholders and using statistically acceptable parameters, and ensure that the sample selected is representative of the geographical coverage of the project as well as the target beneficiaries.

Specific Tasks: The specific tasks of the evaluator will be to:**

(i) Develop evaluation framework and methodology and refine these with the project team.

(ii) Develop a sampling frame that will be discussed and adopted by the evaluation team

(iii) Develop the evaluation plan

(iv) Undertake extensive document review to familiarize with the project

(v) Assess the capacity development interventions the project has undertaken with the MoECHE, State level MoEs and targeted schools

(vi) Critically assess and evaluate the 5 model schools and 3 TVET centres constructed by the project, looking at the key parameters of: achievement of targets, outcomes and lessons learned.

(vii) Review the progress in enrolment of learners in primary school, non-formal education and TVET as a result of the project in the target regions.

(viii) Assess the contribution to pedagogical methods for young learners and adult learners as a result of the project.

(ix) Conduct field visits, focus group discussions and interviews with relevant beneficiaries, partners and project team members to collect information on various parameters of the evaluation.

(x) Prepare the evaluation draft report and present the findings in workshops in Mogadishu, Somalia, for key national and international agencies operating in these locations;

(xi) Prepare and submit final report to the Lead Agency – ADRA in three hard copies (bounded) and CD-ROM.

(xii) Present the findings of the evaluation to the Education Sector Committee Members in Mogadishu, Somalia.

3 Evaluation Approach and Methodology

The evaluation approach should be developed and implemented in three main phases: An Inception Phase, a Field Investigation Phase and a Synthesis and Feedback Phase.

3.1 Inception Phase

This phase will involve a thorough and systematic review of the relevant programming documents related to the SETS project, as well as documents shaping the wider strategy/policy framework. This will provide the development context of the project. The consultant will then analyze the logical framework in order to come up with issues or evaluation questions relevant to the project. This stage will also involve the preparation of the workplan for the evaluation complete with a time schedule. Other tasks to be undertaken during the inception phase will include:

• Present the indicative methodology to the overall assessment of the project/programme.
• Interview the project management and European Union Program Managers (if relevant)
• Present each evaluation question stating the information already gathered and provide a first partial answer to the question, identify the issues still to be covered and the assumptions still to be tested, and describe a full method to answer the question.
• Identify and present the list of tools to be applied in the Field Phase;
• Prepare and submit a detailed work plan with an indicative list of people to be interviewed, surveys to be undertaken, dates of visit, itinerary, and name of team members in charge.

· Define issues and gaps requiring further analysis;

· Prepare and share inception report of not more than 10 pages; and

• List all preparatory steps already taken for the Field Phase.

3.2 Field Investigation Phase

The Field Phase should start upon approval of the inception Report by the evaluation managers (Program Director, Project Manager and M&E focal point). The consultant will:

• Undertake field mission applying the plan developed during the Inception Phase. This plan has to be applied in a way that is flexible enough to accommodate for any last-minute difficulties in the field. If any significant deviation from the agreed work plan or schedule is perceived as creating a risk for the quality of the evaluation, these should be immediately discussed with the evaluation managers.
• Hold a briefing meeting with project management team in the first days of the field phase.
• Undertake school visits and stakeholder interviews including relevant education authorities.
• Ensure adequate contact and consultation with, and involvement of, the different stakeholders; working closely with the relevant government authorities and agencies during their entire assignment. Use the most reliable and appropriate sources of information and will harmonize data from different sources to allow ready interpretation.
• Summarize its field works at the end of the field phase, discuss the reliability and coverage of data collection, and present its preliminary findings in a meeting with the project management team and relevant MoE administrations.**

3.3 Synthesis and Feedback Phase

This phase is mainly devoted to the preparation of the draft final report. The consultant will make sure that:

• Their assessment is objective and balanced, affirmations accurate and verifiable, and recommendations realistic.
• When drafting the report, they will acknowledge clearly where changes in the desired direction are known to be already taking place, in order to avoid misleading readers and causing unnecessary irritation or offence.

The consultant will submit the draft evaluation report to ADRA for review. On the basis of comments expressed by ADRA and consortium members & relevant education authorities, the consultant will amend and revise the draft report. The 2^nd draft report will be presented to ADRA management, while a summary of the key findings of the evaluation will be presented to a group of stakeholders in Mogadishu under the auspices of the Education Sector Committee for their synthesis and discussion. On the basis of comments made by participants, the consultant will prepare the final version of the report.

· Required qualifications/skills

The following are the qualification requirements for the evaluation consultant:

· Advanced University Degree in Education or related field.

· A minimum ten (10) years of relevant professional experience in the areas of basic/primary education, teacher development, capacity building of education administrations, education economics and in the definition and implementation of sectoral policies in the economic sector;

· Experience in the evaluation of technical assistance project, preferably, particularly those under the education portfolio;

· Fully conversant with the principles and working methods of project cycle management, EC aid delivery methods. knowledge of the activities of multilateral development donors will be an added advantage;

• Solid knowledge of, and practical experience with gender analysis and planning;

· Full working knowledge of English and excellent report writing skills

· Competence and adequate experience in the use of qualitative and/or quantitative methods of data collection and analysis including: sampling, desegregation of data, structured and semi-structured interviewing, focus group discussions, and observation and triangulation research methods.

· Ability to interpret and analyse complex qualitative and quantitative data, and to present findings and recommendations in a clear and concise way.

· Excellent inter-personal communication skills including experience of facilitation and presentation.

· Ability to work equally well with communities and international organisations.

· Knowledge and sensitivity to political and social contexts of Somalia