Consultancy – Project Evaluation – Forward Together: Empowering youth with disabilities in Asia At Handicap International – Humanity & Inclusion

General information

1.1 About Humanity & Inclusion

Humanity & Inclusion (HI), formerly known as Handicap International, works alongside people with disabilities and vulnerable populations, taking action and bearing witness in order to respond to their essential needs, improve their living conditions and promote respect for their dignity and fundamental rights. Founded in 1982 in refugee camps on the Thai-Cambodian border, our work quickly expanded across borders and continents. At present, Humanity & Inclusion works in over 60 countries globally, supporting a network of 8 national associations (Germany, Belgium, Canada, United-States, Luxembourg, United Kingdom, and Switzerland).

1.2 About Humanity & Inclusion in the country/region

HI has been operating in the Philippines since 1985, implementing emergency and development actions aimed at improving protection, quality of life, and the promotion of rights of vulnerable individuals, their families, and communities in different locations throughout the country. HI is working from a central office in Makati, and previously in Davao City and Roxas City, in collaboration with government, international, and local partners. In the last three decades, HI has been carrying out a range of projects in different thematic areas such as socio-economic inclusion, support to civil society organizations, access to rehabilitation, access to health and prevention, inclusive disaster risk reduction, and emergency response, integrating disability, age, gender, and other vulnerability factors in all its actions. HI’s projects in the Philippines have been supported by the Big Lottery Fund, World Diabetes Foundation, UK Government Department for International Development (DFID), the European Union (EU), Australian Red Cross, UNHCR, and the START Fund, among others. Based on this extensive experience working in the Philippines, HI has not only developed the technical and area-specific knowledge necessary for successful program implementation, but HI has also built positive working relationships that make it a trusted organization to implement inclusive programs, through specific approaches that have been developed, tested and used with good outcomes within the Philippines and in other countries.

Meanwhile, since 2005 in Indonesia, HI has a wide range of complementary projects that assist in promoting the inclusion of persons with disabilities and other vulnerable people and their issues in development policies and actions, build capacities of key local stakeholders, increase their participation in social and public life, and reduce the impact of natural disasters. HI in Indonesia is committed to enhancing persons with disabilities’ and other vulnerable people’s access to services, promote their active participation and social Inclusion, developing partnerships at all levels in the frame of local, national and international policies.

HI Philippines and Indonesia (HI PIN) is regional program which has been implementing the project Forward Together: Empowering youth with disabilities in Asia, funded by FWD Group Insurance, since 2018.

Context of the evaluation

2.1 Presentation of the project to be evaluated

Project title

Forward Together: Empowering youth with disabilities in Asia

Implementation dates

March 2018 – July 2021

Location/areas of intervention

1. Manila, Philippines: Quezon City, Valenzuela City, and Taguig City
2. Jakarta, Indonesia: South Jakarta, Bekasi City, Bekasi District

Operating partners

1. Manila, Philippines: Community Physiotherapy and Rehabilitation Facilitators (CPREF)
2. Jakarta, Indonesia: Yayasan Thisable

Target groups

· Youth with disabilities

· FWD offices in the Philippines and Indonesia

· Companies, businesses, business and trade associations

· TVET and academic institutions

· Local government offices

· Organizations of persons with disabilities (OPDs)

· Civil society organizations

Project budget

USD 1,060,000

Objectives of the project

At the end of the project:

1. Youth with disabilities have improved skills to enter the workplace or start/scale up their own entrepreneurial activities;
2. Micro-enterprises run by youth with disabilities are successfully created or strengthened;
3. Companies have improved capacities to employ persons with disabilities; and
4. A disability inclusive employment working group / think tank is active at National Capital Region (NCR), Philippines level

Expected results and indicators

1. 380 (100%) of beneficiaries (disaggregated by age and sex) in the Philippines and Indonesia report improved skills to pursue waged or self-employment and report positive outcomes, within 9 months of finishing their personalized support plan;
2. 130 (34%) of beneficiaries (disaggregated by age and sex) in the Philippines are engaged in/have scaled up their formal businesses / entrepreneurial activities; and
3. 10% of FWD country head office staff permanently committed to promoting disability inclusion in the workplace (e.g. becoming disability focal points, forming or participating in employee resource groups, attending trainings on disability inclusion, actively work to recruit employees with disabilities, make structural changes in the workplace etc.)
4. 250 decent jobs created for youth with disabilities in the Philippines and Indonesia due to this project

Main activities implemented

1. For youth with disabilities: job coaching, referral to services, training sessions on: life skills and employee preparedness, technical skills, micro-enterprise development, simple cash books, and financial literacy, provision of: training allowances, assistive devices, and workplace adaptations, access to rehab and medical services.
2. For FWD offices in Philippines and Indonesia: full technical package including the five dimensions of workplace inclusion (Policy, HR, Culture, Accessibility, Partnerships)
3. For company and local government partners: Training sessions on: disability awareness, inclusive service delivery, and MHPSS, coordination, referral of potential candidates for employment and/or TVET/company- or community-based training.

2.2 Justification of the evaluation

As the Forward Together project draws to a close effectively in July 2021, HI looks to conduct a final evaluation to assess the extent to which the project objectives were met and identify the circumstances that led to both high and low levels of success of the project implementation.**

The evaluation results are intended to be shared with the project donor, FWD Group, the entire HI network, and the project stakeholders, including and especially the project participants and their families. This final evaluation will be conducted to ensure and promote accountability and learning, especially within HI’s efforts in optimizing for success and discovering the stories and actions behind the project’s results. Additionally, this evaluation will be used as a tool to mobilize the various project stakeholders to take action based on the results and documentation.

Objectives of the evaluation

3.1 Overall objectives and expectations of the evaluation

This general objective of this final evaluation is to assess the overall results and impact of the Forward Together project, from inception to its final phase of implementation, and provide an understanding of the strengths and weaknesses of how the two approaches (twin-track approach and personalized social support) were used in the Philippines and Indonesia.

Overall expectations of HI

This project evaluation aims to provide teams within HI with evidence and information to push for context-appropriate, realistic, and localized adaptations and enhance future initiatives in increasing the economic participation of youth with disabilities, including the Forward Together project’s Phase 2.

Partners’ expectations concerning the evaluation

Through this final evaluation, the project aims to bring together the perspectives of all project partners – youth with disabilities, their families, local government units involved in the implementation in Manila and Jakarta, national government offices, TVET and academic institutions, FWD offices, companies, businesses, and ILO’s National Business and Disability Networks in the Philippines and Indonesia. Feedback and insights from the partners will aim to highlights areas of traction and areas that warrant further attention and specifics of how they can continue to provide inclusive services especially for persons with disabilities even after the project formally closes.

3.2 Specific objectives

The scope of this final evaluation will cover the Forward Together’s project implementation from March 2018 to July 2021, in Manila, Philippines and Jakarta, Indonesia.

Specifically, the objectives of the final evaluation is to collect data and understand:

a. the ability of the project to produce positive short-, medium- and/or long term changes for the youth with disabilities onboarded to the project, and its project stakeholders;

b. the ability of the strategies used in the project to sustain positive outcomes even after the project formally closes;

c. the project’s administrative capacity;

d. the project’s efficiency and ability to optimize its resources; and

e. the project’s effectiveness and ability to successfully meet its objectives.

3.3 Evaluation criteria and evaluative questions

Evaluation criteria

As set out in 3.2, the final evaluation will focus on four criteria, as set out within’s HI’s project quality framework: https://hi.org/sn_uploads/document/QualityFramework_EN.pdf

1. Changes
2. Sustainability
3. Administration
4. Efficiency
5. Effectiveness

Evaluative questions

1. Changes

a. Did the Forward Together project produce significant positive changes in the lives of youth with disabilities in a sustainable manner? (Needs)

b. Has the project sufficiently taken into account and/or avoided the risks of negative effects (i.e., occupational hazards, natural disasters, COVID-19, economic recession)? (Context)

2. Sustainability

a. Were the actions implemented during the Forward Together project in Manila and Jakarta really sustainable? Has the project met the identified needs of its beneficiaries and/or has provided a transfer to other actors in Manila and Jakarta who can continue the support/interventions? (Continuity)

b. Has the project significantly contributed to reducing the vulnerability of all beneficiaries? (Resilience)

3. Administration

a. Did the project team (both technical and operational) and supporting teams (HI-PIN Support Services and SMT/COMEX, and implementing partners) know their roles and have provided their contributions in an appropriate time, cost, and quality? (Organization)

b. Was a system of control taken to ensure monitoring activities, including corrective actions when needed? (Control)

4. Efficiency

a. Was the project flexible and adapted to the evolving needs and risks, especially around COVID-19? (Flexibility)

b. Were the necessary resources mobilized and optimized throughout the project implementation? (Optimization)

Evaluation methodology and organization of the mission

4.1 Collection methodology

HI encourages the use of methods that are participatory and inclusive (with considerations on disability, gender, age, and others) and must be aligned with HI’s global standards in data collection, and security and protection (see section 5.2 Ethical measures, highlighting safety and informed consent) especially in light of the ongoing COVID-19 community quarantine measures in Manila and Jakarta.

Evaluators are also encouraged to use quantitative and qualitative data for the project’s final evaluation in line with the criteria and evaluative questions set out in 3.3. For both types of methods, the evaluation can use different approaches as necessary that range in complexity from simple desk reviews of Quality of Surveys (QOLs), KAP surveys, and lessons learned activities conducted by the project, or collecting external primary data sources if/when necessary, such as surveys, interviews, and focus group discussions.

4.2 Actors involved in the evaluation

The Forward Together Project Coordinator will be responsible for:

· Identifying and organizing the documents, retroplan, budget, and other resources needed for the final evaluation;

· Organizing a project evaluation Steering Committee;

· Briefing the final evaluator and Steering Committee on the scope, timelines, and objectives of the final evaluation;

· Monitoring, and implementation of corrective actions as needed, of final evaluation activities and quality;

· Reviewing and validating evaluation reports;

· Action planning based on evaluation results and outputs;

· Report sharing and dissemination (internal and external); and

· Integration of evaluation results to Forward Together Phase 2 logframe and Theory of Change

An external evaluator will be onboarded to formally carry out and coordinate on the evaluation activities, with support from the Forward Together Project Coordinator.

A project evaluation Steering Committee will be consisting of internal and external focal persons organized to be responsible for:

· Supporting in the finalization of the criteria and evaluative questions to be included in the evaluation;

· Supporting in the recruitment and selection of the evaluator;

· Coordinating with the evaluator in the development and completion of evaluation questionnaires and aligning on activities related to the project’s closure activities, including Lessons Learned workshops, KAP surveys, and case studies

· Supporting the final evaluation action plan and ensuring the quality of evaluation during the implementation of final evaluation activities and report dissemination;

· Attending the kick-off meeting and other meetings leading up to the final evaluation report;

· Providing insights as needed and appropriate in the evaluative questions presented in 3.3 and 4.3;

· Recommending appropriate methods and tools to be used by the external evaluator; and

· Supporting in the review and validation of the final evaluation report.

Principles and values

5.1 Protection and anti-corruption policies

Code of Conduct. https://hi.org/sn_uploads/document/ID_CodeOfConduct.pdf

Protection of beneficiaries from sexual exploitation, abuse and harassment https://hi.org/sn_uploads/document/PI03_HI_Protection-Beneficiaries_EN.pdf

Child Protection Policy https://hi.org/sn_uploads/document/PI02_HI-Child-Protection_EN_1.pdf

Anti-fraud and anti-corruption policy https://hi.org/sn_uploads/document/PI04_IP_antiFraud-bribery-corruption-policy_1.pdf

5.2 Ethical measures*

As part of each evaluation, HI is committed to upholding certain ethical measures. It is imperative that these measures are taken into account in the technical offer:

· Guarantee the safety of participants, partners and teams: the technical offer must specify the risk mitigation measures.

· Ensuring a person/community-centred approach: the technical offer must propose methods adapted to the needs of the target population (e.g. tools adapted for illiterate audiences / sign language / child-friendly materials, etc.).

· Obtain the free and informed consent of the participants: the technical proposal must explain how the evaluator will obtain the free and informed consent and/or assent of the participants.

· Ensure the security of personal and sensitive data throughout the activity: the technical offer must propose measures for the protection of personal data.

* These measures may be adapted during the completion of the inception report.

5.3 Participation of stakeholders and beneficiaries

In line with the different data collection methods (see 4.1) encouraged to be used for this evaluation, it is crucial that project participants, their families, and project partners (including FWD offices, company and government partners, OPDs and CSOs) are active participants in data collection activities and analysis. The involvement of stakeholders and project participants in the evaluation include:

· Participation in surveys and group discussions;

· Consultation with project participants on the construction of the tools;

· Consultation in ensuring that data collection modalities are inclusive; and

· Participation in data aggregation and analysis.

5.4 Others

· In line with HI’s data privacy standards, unless otherwise explicitly expressed through informed consent, all information from project participants and stakeholders will be reported anonymous;

· In line with the ongoing national government guidelines in the Philippines and Indonesia in preventing the transmission of COVID-19, the evaluation activities must ensure the safety and well-being of all participants involved, and explore as many remote options as possible.

Expected deliverables and proposed schedule

6.1 Deliverables

The evaluator is expected to deliver following deliverables:

a) Inception report, including

i) Detailed evaluation methodology/ies;

ii) Tools: data collection methods and detailed tools actor by actor, sampling considerations if any;

iii) Timeline and logistics

iv) Data collection plan

b) Validation workshops: This will include all actors/stakeholders of the project, as well as the final evaluation Steering Committee. Different workshops can be planned with different audiences to share the information and validate the findings in the most appropriate manner.

c) Evaluation Final Report, no more than 25-30 pages, written in simple English. Any additional information should be provided in Annexes.

i) Executive Summary (1-2 page)

ii) Methodology, including limitations (2-3 pages)

iii) Main Findings (15-20 pages)

(1) Data must be disaggregated by disability, gender, and age.

(2) Must reflect on key criterias of the evaluation as mentioned in 3.3. The evaluation main findings must have separate and consolidated results for Philippines and Indonesia operations.

iv) Conclusions and recommendations (5 pages)

v) Evaluation response and dissemination plan (to be developed by HI after Validation workshops)

6.2 End-of-evaluation questionnaire

An end-of-evaluation questionnaire will be given to the evaluator and must be completed by him/her, a member of the Steering Committee and the person in charge of the evaluation.

6.3 Evaluation dates and schedule

Total mission duration

June-July 2021 (8 weeks)**

Evaluation timeline

Steering Committee kickoff – June Week 1

Review and finalization of TOR + briefing with HQ – June Week 1

TOR publication – June Week 1-2

Selection of evaluator – June Week 3 – 4

Inception: Documentation review, desk research & initial consultations – June Week 4

Submission of Inception report, including detailed methodologies, tools – June Week 4

Data collection – July Week 1-2

Submission of first draft of evaluation report – July Week 3

Review and validation of first draft report, including validation workshops with Steering Committee and stakeholders –

July Week 3

Submission of final evaluation report – July Week 3

Logistical and operational arrangements for data collection in Manila and Jakarta will be partly arranged by HI, and partly by the consultant, depending on the context and accessibility. The group discussions and meetings with project participants and project partners will be arranged together with HI and partner organizations. These arrangements will be pre-agreed prior to the data collection.

Means

7.1 Expertise sought from the consultant(s)

HI encourages both companies and individual evaluators to apply for this evaluation service. The main requirements are:

· Based in the Philippines, with experience and network in the Philippines and Indonesia;

· Educational background in: public policy, statistics, social development, or similar areas;

· At least five years of experience in designing and conducting project evaluations on projects implemented in the Philippines and Indonesia;

· With technical expertise on inclusive employment and entrepreneurship;

· Experience in using innovative and mixed methods approaches, including the considerations for remote modality due to COVID-19;

· The evaluator/s (or teams engaged in the data collection, in case) are expected to speak Filipino and English, and if possible, Bahasa – and must demonstrate flexibility especially in intercultural communications. HI can also provide support in finding a data collection team in Jakarta, in case the lead consultant is not based or has network in the region;

· Demonstrated experience in objectively assessing results and the relationships of results with processes, and conducting and analyzing outcomes;

· Sound writing and reporting skills; and

· (Desired) Experience working with persons with disabilities.

7.2 Budget allocated to the evaluation

Detail : External evaluator fee including GST and other costs – 1 Unit

Unit Price Php : 250,000

Total Cost : 250,000

The total cost indicated above will include costs incurred during the evaluation including:

· Evaluator fee per day for 22 business days

· Ancillary costs: direct out of pocket expenses such as documentation and copying, mobile phone, internet, and communication costs

· Overall cost of the mission including transport costs (local and international, if fitting current context)

· Logistical costs

· Translation costs, if needed

The payment will be distributed as follows:

1 – Five (5) days after signing of contract – 125,000 Php (60%)

2 – One (1) day after submission of first draft of evaluation report – 75,000 Php (30%)

3 – One (1) day after submission of final evaluation report – 50,000 Php (20%)

7.3 Available resources

· Data collection software: SurveyCTO

· Teleconferencing system: Zoom

· Online whiteboard collaborative platforms: Miro, Jamboard. https://miro.com/

· Documents archiving: Google Drive (to be shared with all participants)