External Final Evaluation at People in Need

Country Office: People in Need (PIN) Armenia

Project Name: Public-Private Partnerships for Successful Transition from Education to Employment in Shirak Region

Project Length: 13 August 2019- 15 December 2021

Project Budget: 250.000 EUR

Duration of Final Evaluation: 25 days

Estimated Start Date: January 2022

1. Introduction

1.1. PIN Background

People in Need (PIN) is an international, non-profit organization established in early 1990s in the Czech Republic. Since then, PIN has grown to be the biggest relief and development organization of its kind in post-communist Europe and currently provides humanitarian and development assistance in 24 countries worldwide.

People in Need established a country program in Armenia in 2003. PIN’s goal in Armenia is supporting a strong civil society that takes an active part in poverty reduction and sustainable development. We work in four interlinked sectoral directions: sustainable livelihoods and environment; education and skills development; good governance at the local level; humanitarian support to the displaced people in Armenia. More information on PIN Armenia and its’ projects can be found here: Facebook and PIN Armenia Website.

1.2. Project Summary

‘Public-Private Partnerships for Successful Transition from Education to Employment in Shirak Region’ or shortly called ‘Transition from Education to Employment’ project funded by the European Union is designed to support socio-economic development in Shirak region by improving the synergy between vocational educational institutions (VET) and the private sector. For this purpose, the capacities of VET institutions have been enhanced, new mechanisms for VET-private sector coordination and work-based learning (WBL) have been established and apprenticeships schemes in short, non-formal courses have been piloted in the agriculture and textile sectors. As a final aim of the project the quality and relevance of the target VET institutions’ services and gaps between the education and private sector should have improved, thus the employability of VET students should have increased.

Main activities within the project are:

• Enhancing the capacities of 4 VET institutions: Amasia Craftsman State School (also the co-applicant), Maralik Craftsman State School, Artik State College, Shirak State Agricultural College;
• Developing or reviving of long and short-term courses, developing and/ or modernizing curricula in agriculture and textile sector and promoting work-based learning;
• Developing VET-private sector coordination platforms on Shirak-level agriculture and textile sectors in order to facilitate VET-private sector partnership.
• Organization of work-based learning awareness raising campaign and orientation for VET students.

1.3. Project Status

As of mid-October, 2021, most of the main project activities have been completed. In line with the objectives of the project: the capacity building activities for VETs have been completed, the short-term courses have been developed/ revived and piloted, the coordination platform has been established, the small round of grants to VETs have been awarded. The grants were aimed at implementing short-term courses by VETs and businesses. The project underwent an internal mid-term evaluation in January-February 2021 and identified its strengths and weaknesses and provided recommendation for the remaining implementation period. The project end date is mid-December. The selected Evaluator will be provided with the mid-term evaluation, all official donor reports and other key documents to inform the successful completion of this assignment.

Main Assignment: External Final Evaluation

2. Purpose of the Evaluation

The purpose of the final evaluation is to provide an overall assessment of the project’s performance in relation to achievement of the intended results and in terms of how they have been delivered. The evaluation will identify key lessons learned and propose practical recommendations that will enable PIN, its partners and other stakeholders to enhance the impact of future, similarly focused projects.

2.1. Scope

The evaluation will look over the whole life of the project’s delivery period; 13 August 2019 – 15 December 2021, and assess the intended direct results and also consider some of the indirect benefits brought about through the projects work. Specifically, it will focus its scope around the following workstreams:

• Capacity development of VET institutions;
• Study visits to GIZ work-based learning projects;
• Labour market assessment in agriculture and textile sector in Shirak region;
• Occupational standards and curricula modernization;
• Establishment and involvement of a VET-private sector coordination platform;
• Trainings for private sector;
• Piloting of the courses and work-based learning apprenticeship;
• Organization of a national level workshop;
• Awareness raising campaign on work-based learning;

The Evaluator will conduct their work primarily in Shirak region, with the focus in the main activity areas of Gyumri, Artik, Maralik and Amasia communities. Some aspects of the evaluation will be required to be undertaken in Yerevan considering that some of the stakeholders are based here.

2.2. Audience for the Evaluation Results

The evaluation results will be shared with the EU, project partners, RA Ministry of Education, Science, Culture and Sports, Shirak regional and local authorities and any other relevant stakeholders, as well as with PIN HQ to consider the key findings and recommendations for future programming.

2.3. Methodology

The evaluator will be responsible for developing the evaluation design and methodology in order to meet the above stated goal and objectives. The methodology should be informed by the Development Assistance Criteria (DAC) of Relevance, Coherence, Effectiveness, Efficiency, Impact and Sustainability and use a mix of qualitative and quantitative research methods to answer the questions articulated in this TOR. Both qualitative and quantitative methods should be used: document review, content analysis, field research, etc. It is expected that the evaluator uses participatory methods including key informant interviews and focus group discussions as well first-hand direct observations in the VETs and areas of business implementation. Desk-based review of key project documents including the project proposal, Logframe, reports including the mid-term project evaluation and procurement documents should be conducted. A list of key stakeholders that the evaluator will be expected to consult with is provided under Annex A.

Applicants to this assignment are requested to outline the methodology and approach they will use to undertake this evaluation, in their submitted proposal.

2.3.1. Evaluation Questions to be Answered

The following suggested questions are expected to be answered through this evaluation. Answers should be specific, analytical and evidence driven. Lessons and recommendations should be drawn out against each question and backed up by evidence. Additional questions that the evaluator deems appropriate must be agreed with PIN in advance and as part of the inception report approval process.

Proposed Questions

Relevance

1. How appropriate was the design of the project with regards to achieving the intended overall outcome to support socio-economic development in Shirak region of Armenia?
2. Has the project been responsive to the priorities, needs and future aspirations of the beneficiaries: VETs, students and the businesses (considering also the course development)?
3. What follow-on work could build on the results of this project to further enhance their relevance to the beneficiaries and key stakeholders? Why?

Coherence

1. Did the project undermine or supplement the effects of any existing programs or policies?

Effectiveness

1. Did the project achieve all of the intended outputs and outcomes, as per the final Logframe? If yes or no why? What was the reason for mis-achievement?
2. What internal and external factors were influencing the effectiveness of the project in achieving its results?
3. How effective was the project partnership in relation to effective delivery of the achieved results?
4. How effective was the establishment of agriculture and textile sector coordination platform in relation to establishing VETs-private sector cooperation?

Efficiency

1. How appropriate were the project’s governance and management structures to support efficient and effective addressing of risks and issues?

Impact

1. To what extent has the project contributed to its overall objective?
2. What difference has the project made to the target beneficiaries’ lives, and to VETs and businesses in the region?
3. Have there been any adverse impacts of the projects approach or achieved results?

Sustainability

1. To what extent are the project’s output and outcome results likely to be sustained beyond the life of the project?
2. To what extent have beneficiaries and key stakeholders taken ownership over the project outcomes and demonstrate an intention to build on them going forward?
3. Has the projects achievements created a platform for wider and continuous cooperation among VETs and local businesses?

If an answer cannot be provided on the foundation of date or evidence, this must be communicated to the nominated PIN Focal Person as soon as possible.Failure to do so can result in decreasing the consultant’s remuneration.

2.4. Specific Tasks to be Undertaken by the Evaluator

The evaluator will be required to undertake the following tasks:

1. Design and finalise the evaluation methodology, tools and workplan, detailing the resources and support needed to complete the evaluation to time and quality, and agree with PIN.
2. Undertake the evaluation activities in the field.
3. Analyse the data and present findings to PIN and project partners.
4. Prepare first draft of the Final Evaluation Report for review by PIN.
5. Finalise the Final Evaluation Report considering the feedback provided by PIN.

The report must provide clear, evidence-based and analytical answers to all evaluation questions. It should contain all the necessary raw data information used in the process of data collection and analysis, and any other necessary annexes and references used. Final Evaluation Report template is provided in Annex B.

2.5. Assignment Deliverables and Proposed Timeframes

Deliverable -Content -Delivery Point -Submission

Inception report -Information listed in Point 1. under sec.2.4-7 working days after contract signature -Electronic submission to PIN Focal Person

Initial findings presentation -Summary of findings against final set of agreed findings -3 weeks after approval of inception report -In person presentation

Draft Final Evaluation Report-As per specification outlined in Annex B-7 working days after Initial Findings Presentation -Electronic submission to PIN Focal Person

Final Report -As per specification outlined in Annex B-5 working days after receiving comments from PIN Focal Person -Electronic submission + 5 hard copies to PIN Focal Person

3. Assignment Terms

3.1. Budget

The maximum budget of the project final evaluation is 5,000 EUR including taxes. Per-diem, accommodation, transportation including air tickets if applicable, translation and other costs should be included in the evaluation fee.

3.2. Management Arrangement

The Evaluator will report to and liaise with the PIN Focal Person on all matters and decisions relating to this assignment. PIN will be responsible for arranging meetings with all relevant stakeholders. To enable this the Evaluator must provide a list and schedule of meetings in good time to ensure all meetings can be arranged in a respectful and timely way. If required, PIN can support the Evaluator with logistics arrangements, but the Evaluator will be responsible for making the respective payments.

3.3. Payment Modality

40%-Following the signing of the Contract on Evaluation Services.

60%-After the final version of the final evaluation report has been submitted and approved by PIN.

PIN reserves the right to deduct up to 0.5% of the total contract price for each day’s delay in meeting the deliverables specified. This deduction shall be applied to the last payment of 60% of the contract price.

4. Application Process

4.1. Evaluator Experience

Essential

• At least 6 years of professional experience in conducting development project evaluations;
• Proven experience and knowledge of conducting quantitative and qualitative research;
• Excellent analytical skills;
• Significant practical experience in report writing and formulation of recommendations;
• Excellent spoken and written English.

Preferred

• Experience of evaluating VET related projects and grant schemes;
• Experience evaluating EC-funded projects;
• Understanding of project management principles;
• Previous experience with results‐based monitoring and evaluation methodologies.

Please note that most targeted beneficiaries and stakeholders do not speak English, so relevant translation services into Armenian need to be considered in the proposed price of the work and organised by the Evaluator in case of foreign evaluators or organizations.

4.2. Proposal Submission and review of the applications

Interested candidates/organizations whose profile matches the qualification requirements stated above are invited to send their:

• Curriculum vitae or organizational profile highlighting relevant experience (max.4 pages);
• Cover letter stating how your experience and competencies can best deliver the assignment (max.1 page);
• 2 samples of previous similar work;
• Clear and concise proposal outlining the methodology and sub-questions to be asked (max. 4 pages)
• Assignment Budget (in EUR) broken down by:

a. Personnel Service fees;

b. Direct evaluation activity costs;

c. Anticipated logistics, travel and subsistence costs.