Final Evaluation for Regional Social Entrepreneurship Support Programme in the Western Balkans At GROUPE SOS Pulse

I. Description of the current project and background

a) History of the project

The project has been developed jointly by RYCO, SEEYN and GROUPE SOS Pulse since mid-2017 in cooperation with representatives of the MEAE, the French Institute, the MAA and the OFAJ. The participation of each partner in the process of identifying needs and collective reflection is explained by the history of the partnership relations. The pooling of information, skills and experience gained from other projects has gradually led to the emergence of the idea of a programme to support youth social entrepreneurship in the Western Balkans, from the idea to the creation of the business.

This current project is therefore part of the mission of the Regional Youth Cooperation Office (RYCO), an organisation founded in July 2016 with the support of the OFAJ (Franco-German Youth Office), to enable young people in the region to get to know each other better, to exchange and to create new synergies to promote reconciliation in the region. This project proposes to share French expertise in favour of the emergence of young social entrepreneurs so that they can be drivers of change in their society in key areas for the future: environment and climate, social inclusion, education, migration, youth employment, etc. Two transversal themes are also targeted: intercultural dialogue and regional cooperation.

The present project (AFD Convention n°: CZZ 2629 01 G) started on 01/10/2019 for a duration of 3 years.

b) Project objectives

Overall objective: To open new spaces of reconciliation for the youth of the Western Balkans through social entrepreneurship

Specific objectives :

• SO 1 – Strengthen the capacity of the social entrepreneurship ecosystem in the Western Balkans region
• SO 2 – Raise awareness of social entrepreneurship in the Western Balkans among people aged 15 and over and key actors in the economy.

c) Target audience

Young people aged 15-35, in particular :

• At least 216 young entrepreneurs aged 18-35 through the social entrepreneurship coaching programme
• At least 1,010 people, aged 15 and over, are made aware of social entrepreneurship during design thinking workshops (at least 810 people) and a regional forum on social entrepreneurship (at least 200 people)

d) Role and participation of the different actors

The steering, preparation of the operational action plan and monitoring of the project are carried out by GROUPE SOS Pulse, in partnership with all the partners.

Ø Presentation of the project management system in place

GROUPE SOS Pulse, with a project coordinator, is responsible for the supervision and coordination of the entire project, and has delegated the operational implementation of the majority of the project activities to local structures (see above).

Until now, the operational partners of the consortium have been monitoring the activities of the local structures; this work is carried out by 3 project managers, one per partner, with a geographical distribution as follows

PULSE: Serbia (Smart Kolektiv) and Kosovo (Balkan Green Foundation) RYCO: Albania (Yunus SB) and Northern Macedonia (ARNO)

SEEYN: Montenegro (IEC Tehnopolis) and Bosnia-Herzegovina (Nesto Vise)

• Monthly strategic meetings are held between the 5 consortium partners.
• Monthly coordination meetings are held between the project coordinator, the 3 project leaders of the consortium and the 6 project leaders of the local structures.
• Weekly meetings are held between the project coordinator and the 3 project leaders of the consortium
• Weekly individual meetings are held between the consortium project leaders and the project leaders of the local structures, according to the above geographical distribution.

These meetings provide feedback on operations and regular opportunities for strategic reflection.

The GROUPE SOS Pulse teams also carry out several missions in the field in order to meet the local teams, provide support and plan the next stages of the project.

In addition to its management role, GROUPE SOS Pulse provides its expertise to local partners via numerous training courses to ensure that they become more competent in supporting social entrepreneurs.

c) Activities and main expected results

Objective 1: To strengthen the capacity of the social entrepreneurship ecosystem in the Western Balkans region

• Outcome 1: The capacities of 6 local support structures for youth entrepreneurship initiatives are developed to respond effectively and efficiently to the needs of entrepreneurs
• Outcome 2: The ideation phases allow young project leaders from the Western Balkans to develop innovative solutions to the problems faced by their communities
• Outcome 3: : Winners of the ideation phases benefit from a regional incubation phase

Objective 2: To contribute to the awareness of social entrepreneurship in Western Balkan societies among young people aged 15 and over and key actors in social entrepreneurship

• Outcome 1: An identity is created around the project and promoted at regional level
• Outcome 2: Social entrepreneurship is promoted among youth in the Western Balkans
• Outcome 3: Key political and economic actors in the Western Balkans region are made aware of the value of social entrepreneurship for the development of the region

For a more detailed description, please click here.

II – Objectives and methodology of the evaluation

The final evaluation of the project will cover the initial application as well as the amendment requests submitted and validated since the start of the project.

The external final evaluation of the project will be carried out as follows:

• An initial evaluation will start in November 2021, with first recommendations expected for January 2022;
• A short update of the results of the external evaluation in the summer of 2022.

It should be based on the following methodology and objectives:

1. Rationale / Objectives of the evaluation

a) Relevance

The evaluation first looks at the relevance of the project:

• In relation to the changing needs of local support structures in the region ;
• In relation to the changing needs of project leaders in each intervention area ;
• In relation to the SSE and social entrepreneurship ecosystem in each of the project intervention areas;
• Compared to the current context, marked by the Covid-19 epidemic.

b) Effectiveness: achievement of project objectives

Have the objectives and expected results been achieved, in quantitative and qualitative terms? What are the existing gaps? How can they be explained?

For the evaluation of the two specific objectives of the project, particular attention should be paid to the outcome indicators reported. Other indicators may be used during the evaluation to assess the effectiveness of the project.

c) Efficiency

It is a question of measuring and analysing the human, technical and financial resources mobilised by each project partner in relation to the results achieved.

It is also about measuring the role, involvement and impact of each partner in relation to their initial role and respective expertise. How effective was cooperation among all the partners in the consortium?

The causes and consequences of deficits in the mobilisation of internal and external resources will be examined in relation to the quantitative and qualitative results of the project.

Other questions could be considered: To what extent did the project achieve the results in its proposed timeline? How the covid-19 restrictions and protective measures affected the efficiency of the implementation? How well did the project collect and use data to monitor results?

d) Sustainability/ Viability

The aim is to analyse the effects of the project and its chances of continuing once the external support has ended. Did the operational partners appropriate the methods and contents of the training courses during the incubation period? How have they adapted them? Will the entrepreneurs have sufficient tools to start their business? Will regional interactions continue after the support? To what extent the achieved results are likely to sustain over time? What are the factors that enable or impede the sustainability of achieved results?

It is also necessary to assess the financial viability of the project, taking into account the current demand for recurrent funding from AFD.

e) Measuring the impact

The aim is to measure the impact of the actions carried out during the WB6LAB project, in particular on local structures and the entrepreneurs they support.

Particular attention will be paid to the conclusions of the work in progress within PULSE on measuring the impact of the association’s incubator activities.

f) Gender

The evaluator will be asked to take into account the changes made between Tranche 1 and Tranche 2 of the project, and the ongoing work on gender mainstreaming within the project.

To what extent were gender considerations mainstreamed throughout the project? To what extent did the project support the involvement of young women and girls in social entrepreneurship initiatives? How has the pandemic affected or hindered the participation and contribution of young women and girls in this project in general?

g) Recommendations

The recommendations are expected to address the following issues

• The mechanisms put in place to support local structures within their respective ecosystems;
• The tools and mechanisms put in place to support project leaders and enable them to acquire new skills;
• The relevance and possible evolution of awareness-raising and advocacy activities ;
• The relevance and impact of the creation of a regional ecosystem and the mobility of project leaders.
• The impact of cross-cutting capacity building activities carried out by GROUPE SOS Pulse for the benefit of the Project partners.

As a request for the project to be repeated was submitted to AFD in July 2021, particular attention will be paid to the precision of the recommendations resulting from this final evaluation mission. These recommendations will be used, if necessary, to feed the NGO Initiative Note (NIONG) which will be submitted to AFD in February 2022.

2. Methodology

a) For the final evaluation mission

The number of days is given as an indication.

Preliminary work: 2 days x 1 consultant

• Document review: Review of the main existing documents on this project at PULSE headquarters or online (multi-country interim implementation report, etc.).
• Discussions, exchanges with the project managers in Paris (PULSE);
• Production of a preliminary note: the documentary review and all the interviews in Paris will be the subject of a progress note which will set out the questions and hypotheses that the evaluators will address in the field.
• First meeting in Paris (or remotely): to discuss the preliminary note.

Fieldwork: 8 days x 2 consultants

• Interviews (remote or face-to-face) with the operational partners of the project (names and structures are given for information only):
• In Albania :
  • RYCO: Ardita Bonatti PC, Bojana Bulatovic PM, Matilda Karcanaj M&E C
  • Yunus SB: Sonila Necaj, Shkelzen Marku
• In Bosnia and Herzegovina :
  • SEEYN: Marinela Sumanjski, Mirela Rajkovic
  • Nesto Vise: Adem Lisicic, Aleksandar Bundalo
• In Kosovo: Balkan Green Foundation: Shpend Mustafa, Vizar Azemi
• In Northern Macedonia: ARNO: Irina Janevska, Olga Rajcic
• In Montenegro: IEC Tehnopolis: Anja Grabovica, Aleksandar Janicic
• In Serbia :
  • GROUPE SOS Pulse Serbia: Adrien DELABY, Maja MARKOVIC ;
  • Smart Kolektiv: Ivana Stančić, Neven Marinović
  • OFAJ: Frank Morawietz
  • French Institute: Marion Picard
• Literature review (consultation of available documentation)
• Exchanges with beneficiaries – depending on the methodology proposed (working groups, surveys, etc.).
• Analysis of the data and drafting of the interim report.

Restitution: 1 day x 2 consultants

• 0.5 days for the restitution (oral presentation) of the draft report to PULSE and its partners;
• 0.5 days for the submission of the interim report to AFD (to be confirmed with the donor);
• 0.5 days for the writing of the final report after possible integration of comments.

b) For the update of the conclusions

Fieldwork: 4 days x 2 consultants

• Interviews (at a distance) with the project’s operational partners and some of the beneficiaries.
• Literature review.
• Data analysis and updating of the final evaluation report

Restitution: 1 day x 2 consultants:

• 0.5 days for the restitution (oral presentation) of the final report to PULSE and its partners;
• 0.5 days for the finalisation of the final report after possible integration of partners’ comments.

It should be noted that the results of the final evaluation will be transmitted AFD, RYCO as well as well as to the others partners of the Consortium.

Please note that a Steering Committee will be implemented to ensure the follow-up of this mission with regular meetings with the consultants.

III – Assessment procedures

1. Means

a) Human Ressources

The team will be composed of two (2) experts (or a team considering the scope of the evaluation), covering between them the following competences:

• English – knowledge of the French language (language used with AFD) and one of the languages of the region would be a plus;
• Expertise in evaluation techniques;
• The pair must have worked together several times before;
• Knowledge of the issues and main actors of social entrepreneurship in the Western Balkan countries.

b) Financials

The financial envelope for the realisation of this evaluation is estimated at a maximum of €15,000 including VAT (fifteen thousand euros). This amount covers all costs related to the evaluation, local and international transport, per diem, etc.

The methodological proposals must be presented (according to the outline in the Annex) and include a detailed provisional budget (in accordance with the model presented in the Annex) included in this envelope.

2. Timetable for the evaluation

The final evaluation mission will take place between November 2021 and 31 January 2022, so that the results can feed into the full application form requested by AFD for Phase 2 of the project³.

The update of the conclusions of the final report will take place before the end of the project (31/09/2022).

3. Expected products

The evaluation team will be asked to provide:

• A preliminary note (in electronic form in Word format), in French and English⁴, presented before the start of the evaluation on the ground. It will detail key elements from the technical offer initially transmitted (cf. Appendix 2): methodology, data collection tools and mechanisms, proposed work plan and timeline, proposed templates to carry out the interviews, etc.
• A mid-term final report, in French and English, in January (see Timeline);
• A draft final report in French and English, concluding the evaluation, sent in electronic form in Word format within a reasonable period of time to allow stakeholders to become familiar with all the documents.
• A final report, in French, and English integrating the remarks made during the evaluation feedback meeting will be produced and sent in electronic version. This report will be updated during the second part of the mission.

The reports must include, but not necessarily be limited to, the following:

• Title and opening pages;
• Table of contents;
• List of acronyms and abbreviations;
• Executive summary;
• Introduction;
• Description of the intervention;
• Evaluation scope and objectives;
• Evaluation methodology;
• Data analysis;
• Findings and conclusions;
• Concrete and constructive recommendations to be addressed to one or more stakeholders.
• Report annexes.

4. Criteria for the selection of the evaluation team:

The team of consultants/evaluators is invited to submit technical and financial proposals (in accordance with the attached templates) for the evaluation.

The selection will be based on the following criteria:

• Statement of the problem and understanding of the subject ;
• Proposed methodological approach;
• Qualifications, experience and skills of the experts ;
• Experiences of the area and the project issue to be evaluated ;
• Complementarity of the proposed team;
• Details of the prices and costs of the various services (to be submitted in accordance with the attached model) ;
• Timetable for the completion of all services.

Please review the complete mission statement here before submitting a proposal.