Evaluation of Partner-led Response by NRC Poland and NRC Moldova – NRC-PL-RFQ-24-012 At Norwegian Refugee Council

The Norwegian Refugee Council (NRC) is an independent humanitarian organisation helping people forced to flee. We protect displaced people and support them as they build a new future.

NRC has been present in Ukraine since 2014 and after the escalation of the war in 2022 it significantly scaled up operations in Ukraine as well as setting up operations in Moldova, Poland and Romania, including a regional office based out of Warsaw. NRC is one of the largest international NGOs working in the region. NRC works across emergency response and early recovery specialising in cash assistance, protection, shelter, education, legal assistance, and livelihoods to reach internally displaced persons (IDPs), refugees and returnees as well as others affected by the conflict. Partnerships with local organisations and other actors are at the forefront of our response.

On behalf of the regional office, the Norwegian Refugee Council in Poland seeks a qualified consultants/firms to provide services for the following services:

Evaluation of Partner-led Response by NRC Poland and NRC Moldova

The purpose of this assignment is to conduct an evaluation of NRC’s partnership-based response and modalities in Poland and Moldova implemented from 2022 to 2024 in response to the Ukraine crisis.

Based on relevance for each Country Office and interest for NRC globally, the programmatic focus of the evaluation will be on two Core Competency’s in each country implemented through NRC’s Local Partners.

The evaluation aims to evaluate the Quality, Impact and Cost-Effectiveness of NRC’s response though local partners.

Background information
Background on the context

The escalation of the war in Ukraine in February 2022 led to one of the largest refugee movements seen since the Second World War in Europe, particularly in the countries of Central and Eastern Europe such as Poland and Moldova. The demand and support for the humanitarian response was unprecedented and much of the international aid system turned its attention towards meeting this demand. The numbers of internally displaced people inside Ukraine and refugees fleeing the country continue to be high.

More than 20 million people have entered Poland from Ukraine since the escalation of the war in February 2022. The Polish-Ukrainian border is still being crossed by around 30,000 refugees a day. The traffic, however, is in both directions. According to recent data from the Border Guards, almost 12 million exits to Ukraine have been counted. The number of refugees from Ukraine staying in Poland is estimated at 1 million.

With over one million border crossings from Ukraine to Moldova since February 2022, UNHCR estimates that more than 120,000 Ukrainians remain in country as of May 2024, alongside almost 10,000 Third Country Nationals (TCNs).1 The war in Ukraine continues to have profound economic and social repercussions on the Republic of Moldova.

NRC’s activities and presence

The Norwegian Refugee Council (NRC) is an independent humanitarian organisation helping people forced to flee. We protect displaced people and support them as they build a new future.

NRC’s intervention specific to the evaluation

Poland

NRC in Poland works entirely through partners and has the highest proportion of funding (as well as absolute financial amount) going to local partners in NRC. The only direct implementation used to be the running of the East Warsaw transit site, which NRC phased out in Spring 2024. Currently, with 9 partners, NRC provides standalone and integrated programming with Shelter and Settlements, Education, ICLA, Livelihoods and Food Security, Protection from Violence, and Multi-Purpose Cash Assistance (MPCA).

Moldova

In Moldova, partnerships began with the Ukraine crisis in 2022 where NRC surge teams initiated partnership engagement with local and national actors in Romania and Moldova,

to address displacement related needs. While we closed operations in Romania by the end of 2023, in Moldova we continue implementing projects both directly and through our partners. Our approach prioritises a partner-led response that builds on the capacity of local civil society actors to deliver programmes and work to support national authorities. Our multi-sectoral portfolio encompasses livelihoods and food security, information, counselling, and legal assistance (ICLA), Shelter & Settlements, protection from violence, and education programming. As the displacement context in Moldova becomes protracted, we are committed to ensuring that the basic needs of displacement-affected populations are met, prioritising those with specific needs, and to support individuals on their pathway to self-reliance and durable solutions.

While the scale of NRC’s partner-led response in Central and Eastern Europe is unprecedented for NRC as an organization, globally there are a few examples of other established setups and platforms that we use for working with and through local actors. These have been used to address legal protection gaps for people on the move, support local civil society partner projects that aim to build or rehabilitate physical or social infrastructure, strengthen governance, improve access to essential services, support or create livelihood opportunities and reinforce social ties in local communities.

Purpose of the evaluation and intended use

Overarching purpose

The purpose of this assignment is to conduct an evaluation of NRC’s partnership-based response and modalities in Poland and Moldova implemented from 2022 to 2024 in response to the Ukraine crisis.

Education (in both Poland and Moldova)
Information, Counselling and Legal Assistance (ICLA) – in Poland
Livelihoods and Food Security (LFS) – in Moldova
Advocacy – As part of NRC’s One Response framework, joint advocacy initiatives with partners will also be a focus of interest

The evaluation aims to:

Evaluate Quality, Impact and Cost-Effectiveness of NRC’s response though local partners focusing on:

If, and how these partnerships have enhanced the quality and impact of NRC’s humanitarian response to the Ukraine crisis in neighbouring countries.
Local partners’ response alignment with NRC’s response policy, strategic objectives and priorities, and vice versa – alignment of intended response funded by NRC with partners’ strategies and priorities.
Relevance of services provided by partners to project participants (PPs) and their satisfaction and feedback with services received through partner implementation.
Timeliness and efficiency of response implementation by local partners.
Sustainability of the response capacity by local partners built through NRC’s 2022-2024 funding, given planned exit of NRC from Poland and transition in Moldova, as well as considering the importance of preparedness for any future scale-up.

Examine in more detail:

Effectiveness of capacity sharing efforts provided to local partners, and improvements in partners’ technical and organisational capacities especially the support for more sustainable response and promote durable solutions for the displaced populations.
Coordination mechanisms between NRC and partners, and of communication flow and feedback mechanisms between NRC and partners.
Assess the efficiency, advantages and disadvantages of the organigrammes and capacity of each country office for a partner-led (Poland) and hybrid response (direct implementation by NRC combined with partner implementation – Moldova).
Assess the impact of both modalities of delivery (partner-led and hybrid response) on local communities, the partners themselves, and the long-term sustainability of outcomes achieved through them
- Assess or capture changes over time on partners in quality, compliance, procedures, etc.
Appropriateness and efficiency of NRC resource allocation to partners, of financial management and transparency in the use of funds by partners
- Assess costs incurred relative to benefits achieved (cost per project participant, cost per major activity types, compare across the partners and NRC, identification of high impact low-cost activities)

Examine the Quality, Relevance and Efficiency of NRC’s internal and Country Offices’ procedures pertaining to local partners:

Analyse NRC’s internal procedures and their application by Country Offices related to partnership management, particularly the processes such as designing, negotiating and signing of Partnership Implementation Agreements (PIAs), Organisational Assessment Tools (OAT), as well as processes related to finance management, with a specific focus on the experiences and perspectives of partners in Poland and Moldova.

The analysis will cover, but is not limited to, the following aspects, as well as its positive and negative impacts on fostering a productive and healthy partnership:

The impact NRC’s internal compliance measures have on country offices and their ability to work with partners, and on partners due to NRC’s programme compliance processes expected of them.
NRC’s financial and other compliance standards, and NRC’s internal control procedures, flagging any deviation between donor requirements funding NRC and NRC’s internal procedures.
The extent to which NRC has created the space for partners to co-design projects, plan related budgets, including admin, operations and capacity sharing costs.
Identification of potential financial, operational and compliance deficiencies (e.g. adequacy and clarity of due diligence criteria, effectiveness of risk mitigation strategies, transparency of shared financial information, consistency across funding sources).

Identify both partners’ and NRC added value in partnerships:

Through the analysis and gathering feedback (anonymous if needed) from partners regarding NRC’s partnership approach, capturing both successes and areas of improvement in Poland and Moldova COs, identify the added value NRC brings to its partners, and vice versa especially compared to peer international NGOs and UN agencies.

Identify Success Models for partnership management:

Analyse and identify successful models, replicable processes and practices of working with local partners, particularly but not limited to the areas of partner identification, assessment, capacity development and contracting, based on NRC’s experience of partner-led response in Poland and Moldova.

Identification and Assessment of Partners: Analyse the methods and criteria used for identifying and assessing local partners and evaluate their effectiveness.
Capacity Development: Examine the strategies and activities employed to build the capacity of local partners and determine which approaches have been most successful.
Contracting Processes: Review the contracting processes with local partners, assessing their efficiency and impact on the partnership’s success.
Replicable Practices: Identify practices and processes that can be replicated or adapted in other contexts, ensuring they contribute to a successful partnership-based response.

Provide Recommendations for future NRC country operations:
Develop practical and actionable recommendations for future NRC country operations in Moldova and Poland but also globally that intend to deliver humanitarian response through partnerships. These recommendations should address the identified gaps by the evaluation, the do’s and don’ts, aiming to improve the response impact, efficiency, effectiveness, and mutual benefits of NRC and its partners.
Identify recommendations that leverage the partnership approaches undertaken elsewhere in the organisation (e.g. Human Mobility Hub, Building for Peace [B4P], NORCAP) to further improve future partner approaches in Country Offices and NRC.
How will the evaluation be used?

NRC values evaluations for their ability to:

Enhance accountability towards NRC’s target group and donors by regularly examining performance and adapting NRC’s response based on findings.
Use findings to immediately bring about changes and improvements in practice.
Strengthen agency wide learning by drawing on evaluations to improve organisational approaches, strategy and policy.

Particularly for this evaluation, the findings and recommendations will be used to:

Inform NRC’s policy and practice change needed to advance partnership approaches for more effective and efficient local partnership-based responses.
Improve NRC’s strategies for partner identification, assessment, capacity sharing and contracting, ensuring these processes are robust, efficient, and effective.
Provide evidence-based insights to inform NRC’s policies and practices related to partnership management from operational perspective, fostering a more systematic and consistent approach.
Strengthen the capacity sharing initiatives for local partners, ensuring they are better equipped to meet quality standards and contribute meaningfully to program objectives.
Streamline contracting processes to reduce delays and enhance operational efficiency, ensuring timely and effective partnerships.
Disseminate success models and lessons learned within NRC and to external stakeholders, promoting continuous learning and improvement in partnership management.
Serve as a guide for future implementation of successful partnership models in the CEE region and possibly other regions, ensuring that NRC’s approach to working with local actors is both effective and impactful.
1. Who will it be used by?

The primary users of the evaluation will be the NRC Leadership and Management:

Regional Director (RD) and Regional Programme Director (RPD): To inform strategic decisions on partnership in CEE region and inform internal advocacy for policy changes that support more effective and efficient partnership models within NRC.
Country Directors (CDs): To enhance the implementation and management of partnerships within their respective countries.
Heads of Programme (HoPs): To ensure that programmatic strategies and activities are effectively aligned with the findings and recommendations when working with and through local actors.
Programme and Partnership Teams: To contribute to improving daily operations, facilitation of partnership management processes, and capacity sharing initiatives.
Regional Partnership Oversight Board: To guide the overall partnership support provided by the Board.
Monitoring, Evaluation and Learning (MEL) Teams: To refine monitoring and evaluation frameworks and ensure the integration of partnership evaluation insights into ongoing and future MEL work with local actors.
NRC’s Support Services: To adjust and streamline administrative, financial, and compliance processes to better support partnerships and reduce administrative burdens.

External Stakeholders:

NRC Local Actors: To provide feedback on NRC’s partnership approach and identify areas for improvement, ensuring that their perspectives and needs are considered in future collaborations.
Donors: To demonstrate the effectiveness and impact of NRC’s partnership modalities, potentially influencing future funding decisions and strategic partnership, and ensure transparency and accountability in the use of donor funds.
Broader Humanitarian Community: To share best practices and lessons learned, contributing to the broader humanitarian community’s knowledge and improving partnership approaches across the sector.

The evaluation report will be published on NRC website and NRC’s internal intranet.

Lessons learned and recommendations will be shared outside of the CEE region as well as outside of NRC.

While the evaluation results will be specific to countries with partnership-based responses, full or hybrid, in CEE region, they will also contribute to NRC’s partnership model globally.

Scope and lines of inquiry
Scope

Dates of the evaluation: September 2024 – February 2025
Location: Poland and Moldova
Partnership Based response and Projects between 2022 -2024
Main Donors: NORAD, GFFO, ECHO, SDC, NMFA
Partners Programmatic focus:
- Education

Poland: Core Competency (CC):

Information, Counselling and Legal Assistance (ICLA)

Moldova: Core Competency (CC):

Education
Livelihoods and Food Security (LFS)

Lines of Enquiry

The evaluation will seek to answer the following lines of Enquiry:

What successful models, processes, and practices in working with local partners can be identified and replicated from NRC’s experience in Poland and Moldova?
What successful models, processes, and practices in working with local partners can be identified and replicated from other NRC’s experiences and setups of working with and through local actors globally?
Have NRC policies, approaches, or systems hindered or promoted collaboration with partners?
To what extent and in what ways has partnership with local actors contributed to changes (positive or negative) in the effectiveness of our humanitarian response in Moldova and Poland?
How timely and efficient has the response implementation by local partners been, and how well does it align with NRC’s strategic objectives and priorities?
How relevant are the services provided by local partners to PPs, and how satisfied are participants with these services? How does this compare with level of satisfaction when undertaken directly by NRC in the same country?
How effective have NRC’s capacity-sharing efforts been in improving local partners’ technical and organisational capacities (programme, advocacy, and support)?
What are the strengths and weaknesses of the coordination mechanisms and communication flow between NRC and its partners?
How appropriate and efficient are NRC’s resource allocation, financial management, and transparency processes with its partners?
What is the cost-effectiveness of NRC’s partnership-based response, and which activities have the highest impact at the lowest cost? Equally, what is the cost-effectiveness of NRC’s direct activities in the same country. Can one be seen to be more cost-efficient than the other?
How effective are NRC’s internal procedures related to partnership management, including the design, negotiation, and signing of agreements, and how do partners perceive these processes? How does it compare to peer organisations?
What added value do both NRC and its partners bring to the partnership, and how can this be optimised compared to peer international NGOs and UN agencies?

NRC evaluation strategic questions:

To what degree has NRC enabled an effective locally led response?
To what extent are we enabling the participation of the people and communities we work with in support of effective, safe, and quality programming? (organisational learning question for 2022 – 2024)

Methodology

The evaluation methodology will be developed by the consultant team and presented in the report, including a detailed evaluation matrix. The evaluation should be based on combined quantitative and qualitative research methods. All key evaluation questions should be addressed.

The data collection strategy should include the use of appropriate tools to gain a deeper understanding of the outcomes of partnership-based responses and projects with and through local actors, including:

Desk review of background documents of NRC country-based Programmes, such as the Regional and Countries strategies, response plans, policies and principles on partnerships.
Desk review of background documents of partnerships, such as organisational assessment tool (OAT), Capacity Sharing Plans, the Project Implementation Agreement (PIA), CO RACIs and NRC Local Actor Partnership Handbook
Desk review of background documents on Localisation in Poland and Moldova, Internal NRC reports on Partnerships, and others.
Desk review of other NRC platforms and setups globally documentations, missions and mandates.
NRC partners reports (i.e. Projects Closures reports, evaluation reports, Health checks report etc.).
Review partnership reports and assessments in the region and COs made by NRC looking at learning outcomes of partnership.
Key informant interviews with key NRC Country Office Management Groups, programme and support teams.
Key Informant Interviews as the key approach for data collection with NRC ‘s local partners, complemented by the secondary data analysis, including an anonymous overview of contractual setups and approaches between NRC and partners throughout the Project Cycle Management (PCM), in terms of good and bad practice.
Key Informant Interviews with NRC’s Management groups from other Global NRC setups that work with and through local actor partners.

IMPORTANT NOTE: The evaluation must integrate the gender perspective at all stages in order to not miss a key dimension to understanding of what works, and does not work, for NRC’s targets group. Therefore, “Gender-blind” evaluations are not accepted.

Evaluation follow up and learning

The findings will be used to inform NRC’s policy and practice change needed to advance and progress partnership approaches for more effective and efficient local partnership-based responses.

A management response will be developed within two months of the evaluation report being finalised. This will be followed up and tracked by regional office and country management teams.
A dissemination plan will be developed to ensure that important learning is shared with internal and external stakeholders.

Management of the Evaluation

An evaluation manager has been appointed to internally coordinate the process and will be the evaluation team’s main focal point.

Evaluation Manager: Wesam Zaibak, Regional Partnerships Adviser CEE

The Steering Committee has the overall responsibility for ensuring that this evaluation/ review takes place.

Steering Committee Chair: Carrie Bodley-Bond, Regional Head of Risk and Compliance CEE
Steering Committee Member: Hasmik Baghramyan, Regional Head of Finance CEE
Steering Committee Member: Oleksii Grynov, Regional MEL Manager CEE
Steering Committee Member: Julie Chinnery, Regional Head of CC/Thematic Unit/Education Adviser

The Steering Committee will oversee administration and overall coordination, including monitoring progress. The main functions of the Steering Committee will be:

Establish the Terms of Reference of the evaluation
Select consultant
Coordinate review and comment on the inception report and approve the proposed evaluation strategy
Coordinate review and comments of the draft evaluation report
Establish a dissemination and utilisation strategy

Deliverables

The consultant will deliver, based on an agreed upon work plan:

Data collection instruments for review and approval by NRC prior to commencing work in Poland and Moldova.
An evaluation inception report (using NRC’s standard template), with proposed methodology for the evaluation and a detailed work plan.
Presentation of preliminary evaluation findings and validation workshop with the programme team, Country Management Groups.
A draft evaluation report (using NRC’s standard template).
A final evaluation report including executive summary, key recommendations and supporting analysis.

Timeframe

The evaluation will start in September 2024 (depending on consultant’s availability) with an initial briefing and document preparation. The consultant will then be required to conduct a review of the provided documents and submit an inception report. Following approval of the inception report, primary data collection will be conducted in Poland and Moldova with key staff.

Key dates for the evaluation will be as follows:

Submission of draft inception report: Mid-October 2024
Briefing and primary data collection in Poland and Moldova: October 2024
Presentation of preliminary findings and validation workshop with programme team, country management group and programme implementation focal points: 2nd week of January 2024
Submission of draft evaluation Report: End of January 2025
Submission of final evaluation Report: Mid-February 2025

Evaluation consultant team

NRC seeks expressions of interest from people (external to NRC) with the following skills/ qualifications:

Senior humanitarian professional – minimum 10 years’ experience with proven practical experience in project/programme evaluation, particularly Local Partnership-Based Programming
Minimum 5 years of experience with quantitative and qualitative research and experience or evaluations, experience in Local Partnership-Based Programming is an asset
Experience of the NGOs management in general and familiarity with organisational development
Ability to analyse, synthesise, and to write clear reports
Proven experience in NGOs management in general and conducting organisational reviews
Experience working with civil society organisations in conflict and post-conflict countries
Language proficiency: English (Polish and or Romanian being an asset)
Desirable: Experience working alongside humanitarian and/or local civil society organisations in Eastern Europe

All interested companies can access the full RFQ document and submission instructions by contacting pl.procurement@nrc.no. before 30th August 2024, 16:00 CET. E-mails need to include number “NRC-PL-RFQ-24-012” in their title.

Complete bids shall be electronically emailed strictly to pl.tender@nrc.no no later than 30th August 2024, 16:00 CET