Endline Evaluation “Civil Society Influence for Reduced Inequality in Iraq” Programme At Norwegian People’s Aid

Background:

The Norwegian People’s Aid (NPA) is a non-governmental organisation with roots in trade unions and labour movements. NPA supports processes towards democracy and equitable distribution of power through mobilisation, popular participation and collective organisation. The international strategy affirms civil society as a key pillar for nation building; democracy and development, and views human rights as building blocks for development and redistribution.

NPA’s goal is to support human worth and equal rights for all, irrespective of sex, disability, ethnicity, religion, age, identity, or social status. The NPA vision is solidarity in practice. The vision demands commitment and action to protect life and health, build democratic societies and strengthen people’s ability to master their own lives. NPA is politically independent but not a politically neutral organisation.

NPA and its partners are implementing a five-year programme (2020 – 2024) with funds from the Norwegian Agency for Development Cooperation (Norad) with an overall objective to reduce inequality in Iraq politically, socially, and economically.

The programme focuses particularly on reducing the inequalities faced by marginalised groups and local communities. The vision is that marginalised groups and local communities in Iraq enjoy equal access to decent public services, participate in decision-making and experience reduced social, political and economic inequalities. The programme will support partners’ capacity to mobilise their constituencies for effective and meaningful participation in decision-making forums on the local, provincial and national levels.

The programme’s target groups are marginalised people and communities, in particular ethnic minorities, workers and people deprived of liberty in all Iraqi provinces. The programme aims to promote the rights of these groups in Iraq by advocating for the authorities to reform unfitted legislation and practices, bringing an end to exclusion and discrimination based on inequality between men and women, class, and ethnicity.

Targeted stakeholders are local support committees, composed of various community actors: CSOs, activists, academics, religious leaders, and members of local councils. These actors are key to influencing proposed changes. NPA and partners aim to facilitate spaces for interaction between these targeted groups and local authorities to strengthen people’s participation and influence to increase access to decision-making. Organisations engaged through networks in the defence of the rights of minorities, and ethnic and religious communities are targeted actors.

Purpose of the evaluation:

The purpose of this evaluation is to help NPA assess how well the outcomes of this programme have been achieved, and what worked well and less well. This evaluation should look closely at lessons learned and recommendations, in line with NPA guidelines for end-of-programme evaluations and to inform programme development for the next four-year period.

The evaluation should provide a framework to collect data on immediate, basic changes that lead to longer, more transformative change, and allow for the plausible assessment of the initiative’s contribution to results.

To aid in this purpose, the evaluation will therefore respond to the following objectives:

• To critically review the achievement of the programme in terms of attaining its objectives using the evaluation criteria: relevance, effectiveness, efficiency, impact and sustainability.
• To reflect on the effectiveness of the NPA model of working in local partnerships with civil society organisations to achieve programme objectives with focus on women and youth mobilisation and how to engage with them is a meaningful and purposeful way.
• Suggest key practical targeted recommendations and learning to be considered by NPA for the proposed new programme (2025-2029), to inform with practical recommendations relevant for further programming in 2025 and beyond.
• To contribute to NPA’s overall institutional learning and referring to NPA Development and Humanitarian Cooperation Result Evaluation Plan 2020-2024.

The evaluation results are envisaged to identify and describe the lessons learned, through measurements of the changes in the set milestones, summarize the experiences gained, technically and managerially, and recommend ways in which the activities of the programme and future programmes can strengthen the democratisation and human rights process in Iraq.

Users of the evaluation:

The evaluation shall provide the donor, NPA and its partners with a review of the programme relevance, effectiveness, efficiency, impact, sustainability and partnership to make suggestions for potential adjustments if required.

The primary intended users of the evaluation are:

– NPA Iraq Country Office

– NPA Head Office (MENA Team and technical advisors)

– Norad (donor)

– Partner organisations

The Theory of Change of what will be evaluated:

The scope of this evaluation will focus on the programme global Theory of Change that was developed for this programme in 2019 (19 countries in total of which only Iraq is being evaluated in this evaluation), the role it has played in guiding NPA and its partners’ delivery of results against strategy, to learn how these results enable change for people and understand CSOs’ role, contribution and added value in this.

NPA’s vision for this programme is “a society with just distribution of power and resources”. In such a society, economic, political and social inequalities are low, the majority of its citizens consider the distribution of resources to be fairly just, and people do not experience discrimination based on gender, class or ethnicity. People have the right to organise, participate and express their opinions in a climate of respect and safety, and the possibility to hold decision-makers accountable and influence the development of their societies.

To achieve this, we must gradually reduce economic, political and social inequality, and to reduce inequality we must understand the power relations and causes of power imbalances.

As NPA believes civil society organisations play an important role in influencing decision-makers, the programme will cooperate with organisations that are, or have the potential to be, drivers of change for reduced political, economic and social inequality. Hence, NPA supports partners’ work to influence decision-makers and mobilise people.

Civil society organisations influence decision-makers policies and practices (outcome)

Partners identify and prioritize the policies, practices and decision-makers they have the potential to influence. These may vary from the implementation of already adopted policies to getting an issue on the decision-makers’ agenda. Some selected policy/practice goals may be easier to achieve, while others may be hard. Partners use a variety of strategies to achieve change. Although the contexts vary significantly, one strategy used by many partners is mobilising people through campaigning and demonstrations.

Civil society organisations mobilise people to act and participate for rights (outcome)

Many partners engage with communities to strengthen individuals and/or groups that experience political, economic and social inequality. They may support people, through different activities, to enable them to claim their rights and/or participate in local decision-making forums. In some cases, these community members are not organised, and lack tools to stand up against decision-makers. In other cases, the community members are members of the partner organisation. For instance, in several countries, partners organise entire communities, for instance, indigenous communities. Formal organising may sometimes be a goal in itself, other times the aim is that people are able to act independently and/or in alliances.

To strengthen people/community members, partners will enhance peoples’ awareness about their rights, and how they can challenge and influence decision-makers. NPA will support partners to systematise and learn from experiences and adjust their strategies to improve their support to the target group.

Civil society actors have strengthened their capacity to influence (outcome)

As there are strong actors that gain from unequal distribution of power and resources, and therefore resist changes, civil society actors need to strengthen their capacity to influence and represent marginalised groups. NPA believes that civil society actors are more effective and sustainable in their efforts to influence if they are organised. Many spontaneous mobilisations have an impact, and some can be game-changing, but many become isolated events and do not lead to lasting changes. Organisations are fundamental building blocks in civil society and organising has shown to be an effective tool for marginalised people to assert power.

Strengthening civil society organisations is in itself a contribution to reducing political inequality because, through collective organising, marginalised or discriminated groups can assert power. As inequalities affect people who are politically, socially and/or economically marginalised most, it is especially important that they organise, voice their opinions, assert power and assume representation, to influence the public discourse and the policy agenda.

Key evaluation questions:

It is expected that the consultant will give recommendations for further improvements of the programme based on Iraqi context, trends, and other key questions. The evaluation is expected to consider evaluation criteria: relevance, effectiveness, efficiency, sustainability and impact. In addition, the evaluation should focus on partnership and lessons learned. Lessons learned should be documented to inform NPA’s and partners’ decision-making processes. The evaluation should identify strengths, challenges and weaknesses in the programme design, implementation and monitoring process and recommend possible changes for future learning. Also, document the lessons learned in terms of the design, implementation, and monitoring of the programme, that should be applied for future programmes, with consideration to the NPA partnership model. Details of lessons learned about how change comes about, what should be done differently in the future and what needs to happen next.

Some of the key questions/issues to be evaluated are:

Evaluation Criteria

Key Review Questions

Relevance and quality of design

• To what extent was the programme relevant to the political situation in Iraq? Among the questions to be answered are but not limited to:
• Has forming the new government and Parliaments in both the Kurdistan Region of Iraq (KRI) and Federal Iraq significantly affected any of the premises for the programme?
• Has the programme strategy/methodology been adapted to the political and legal contexts in Iraq?
• What has been the actual relevance on the ground of the key target groups in the programme?
• What are the main activities/methods applied by both governments and Parliaments of Iraq and KRI, partners, Provincial Councils and civil society to mobilize, influence and create change, and how was the programme connected with those activities and methods? Among questions to be answered are:
• What are the actual political agendas of programme stakeholders on the ground in Iraq?
• What role does the programme play in terms of interaction among stakeholders on the ground?
• How has the programme contributed to the development of the activities/methods of the Iraqi and Kurdistan Parliament, partners, Provincial Councils and civil society (political dialogue, networking, funding, etc), the main strengths and weaknesses of this support?
• To which extent did the community and stakeholders participate in the planning and implementation of programme interventions?
• To what extent are the objectives of the programme still relevant?

Effectiveness

• To what extent did the programme achieve its objectives/ outcomes? What and how much progress has been made towards achieving the overall outputs, and outcomes of the programme (including contributing factors and constraints)?
• Were the inputs and strategies used effective, realistic, appropriate and adequate to achieve the programme results?
• To what extent did the programme’s NPA mechanism contribute to meeting programme results?
• How effective were the strategies and approaches used in the implementation of the programme?
• What are the recommendations in terms of effectiveness for future similar interventions?
• How satisfied are the communities with the programme interventions and their results?
• What equality between men and women-specific issues have been observed and addressed through the programme?
• The extent to which the outputs and results were achieved in relation to targets set in the logical/ results framework, taking into account any change in context.
• The effectiveness of the adjustments made to the programme to continuously respond to the (changing) capacity of local partners and changing context.

Efficiency

• Were the resources used effectively and in a timely manner to produce the outputs and results appropriate compared to the planned budget and activities?
• Were the resources effectively utilized?
• What factors contributed to implementation efficiency?
• Did the programme activities overlap and duplicate other similar interventions (funded nationally and/ or by other donors)?
• How efficient were the management and accountability structures of the programme?
• To what extent are the objectives of the programme still valid?

Impact

• What were the positive and negative, intended and unintended, changes produced by the programme?
• Analyse the contribution of the programme to any observed impact (intended, unintended, positive, negative) and analyse what other actors and factors contributed to the impact.
• What real impact has the intervention made to the beneficiaries?
• How likely is it that any positive changes may be sustained in the short- and medium-term?
• How has the programme influenced the interconnectedness of the Parliaments, provincial councils, partners and civil society?

Sustainability

• What are the prospects for the benefits of the programme being sustained after the funding stops particularly with reference to the local capacity (partner civil society organizations)?
• What is the partners’ capacity for management and further implementation of such programmes?
• Recommendations for improving sustainability when working in partnership with local civil society actors.
• To which extent did the planning and implementation of the interventions take longer-term and interconnected problems into account?
• Did the programme plan and implement an adequate transition and exit strategy that ensures longer-term positive effects and reduces the risk of dependency?
• How likely will the services and effects be sustained beyond the duration of the programme?
• What were/are the major factors which influenced the achievement or non-achievement of sustainability of the programme?

NPA capacity and partnership with local CSO

• How is the NPA’s organizational capacity set up in order to achieve the intended programme results? This includes implementation performance, work planning, NPA Head Office involvement, and NPA’s external office management model.
• How are the NPA capacity and systems for risk management?
• How is monitoring and documentation of outputs, outcomes and longer effects organized and what monitoring systems are in place?
• Assess the partnership process and the partnership relationship between NPA and its local partners, identify what worked well and what did not work well and recommend possible changes.

Lessons Learned

• Identify strengths, challenges and weaknesses in the programme implementation process and recommend possible changes/ potentials for future 2025-2029 programme.
• Document the lessons learned in terms of the design, implementation, and monitoring of the programme, that should be applied for future programmes, with consideration to NPA partnership model.

Evaluation implementation:

Timeframe:

The preliminary draft report should be submitted to NPA by 26th March 2024 at the latest and the final evaluation report should be submitted to NPA within two weeks of receiving NPA’s feedback to the draft report, but not later than 10th April 2024. The work is planned to commence no later than 10th February 2024.

The Consultant/Company should present a detailed timeframe for each activity in the work plan including working days for each activity.

The Consultant/Company should have the following competencies and/or experience.

• Proven expertise in the design and review of programmes focused on human rights, democratization and experience in the MENA context, preference will be given to candidates that have past experience with programme design, implementation and/or evaluation of development programmes in the MENA.
• Data analysis and writing up of evaluation reports experience is also key for this exercise. The Consultant should possess extensive work experience and in-depth knowledge of conducting programme evaluations.
• The Consultant should have a postgraduate degree in development studies and human rights, or equivalent knowledge in the areas mentioned in the above two points.
• Preferably, the Consultant should deploy a multidisciplinary team with a background in political and statistical data analysis qualifications.
• It is desirable to have a postgraduate in political studies. Preference will be given to those who possess past experience working with INGOs and conducting evaluation surveys.
• The Consultant should have proficiency in English. Working knowledge of Arabic is a plus.
• The field team must have proficiency in Arabic and Kurdish.

Organisation of evaluation management:

• The Consultant will report to the NPA steering committee, consisting of the Country Office team (Country Director, Programme Manager, Projects Manager, and MEAL Manager) and Iraq Advisor at the NPA Head Office.
• The MEAL Manager shall be the focal and first point of contact between the Consultant and the Steering Committee
• The steering committee will oversee and support the day-to-day delivery of the evaluation.
• The donor will provide feedback on draft findings and recommendations and inform NPA’s management response.
• NPA will provide administrative/logistical support towards the organisation of documentation, interviews, meetings, field visits and workshops. However, any support which requires cost shall be factored in the consultant’s financial proposal.

Methodology:

The Consultant will design the details in the methodology in cooperation with NPA. The evaluation should follow a collaborative and participatory mixed-methods approach that draws on both existing and new quantitative and qualitative data to respond to the evaluation questions mentioned earlier in this document. The Consultant should follow a theory-based approach based on the programme theory of change. Furthermore, it should combine evaluation tools based on international standards and guidelines, notably OECD DAC Quality Standards.

It is expected that the Consultant will assess the quality of the programme’s theory of change and, if necessary, to develop a realistic impact logic based upon the conducted interventions. The methodology design should be developed by the Consultant in consideration of the information outlined in this ToR to ensure accuracy and rigor. A detailed methodology and data collection methods should be included in the technical proposal, which will be further improved in consultation with the NPA team during the inception phase of the evaluation. The choice of methods must also consider the needs and capacities of the different target groups and stakeholders.

Desk review:

Desk Review should be conducted by the consultant to inform the methodology and development of the tools. In addition, the conducted desk review should cover the following documents:

• Programme proposal
• Results Framework
• Annual reports
• Existing documents available at NPA and among partners
• External reports and documents to better inform the methodology and execution of the evaluation plan
  1. Present to NPA Evaluation Inception Report: This document will operationalise and direct the evaluation. It will describe how the evaluation will be executed, thus refining and elaborating upon the terms of reference; it will be approved by NPA Senior Management and act as an agreement between parties for how the evaluation will be conducted. The report will include the following elements:

The desk review process should serve as guidance for the Consultant to continue gathering resources that would enable him/her to carry out the development of tools.

The document review will also include NPA policies, proposals, reports submitted to the donor, documentation from partners, and various forms of programme documentation.

It will also include field studies in the locations the programme covers in Iraq. This will include field visits and interviews with NPA staff, representatives of partners, local authorities and other stakeholders.

The evaluation should have a strong learning aspect and should therefore apply participatory methods that will include various stakeholders.

The main findings should be presented at the NPA Country Office before the final version of the report.

Deliverables:

• Expectations of evaluation
• Roles and responsibilities
• Evaluation methodology, questions
• Evaluation framework
• Information collection and methods of analysis
• Reporting formats
• Day-by-day work plan and timeframe for activities and deliverables
  1. Develop the tools and methodology for conducting the evaluation, using both quantitative and qualitative approaches to address the key evaluation questions listed above. The Consultant will then compare the findings with baseline values, using appropriate surveys, and statistical tests (relevant methodology), to draw the findings and conclusions and take into account the qualitative information to write the report.
  2. Review all of the programme documents (proposal, baseline report, mid-term evaluation and partners’ reports) and relevant NPA policy documents.
  3. Conduct qualitative/quantitative data collection through appropriate methods and carry out data analysis (including appropriate statistical tests). The data collected should be as much disaggregated as possible (sex, age, ability, and ethnicity). Data should be disaggregated by sex, age, ethnicity and ability, as relevant.
  4. Prepare a draft evaluation report which includes the following elements:

1. A stand-alone Executive Summary
2. Introduction
3. Methodology
4. Responses to questions/ Findings
5. Problems and needs
6. Achievement of purpose
7. Overall assessment
8. Conclusions, Targeted Recommendations and Management Responses
9. Annexes to the report, including data analysis report.
   1. Present findings to NPA and partners’ staff in a briefing session through a presentation (ideally PowerPoint) that should be also submitted to NPA.
   2. Consultant receives feedback on the first draft from NPA and partners’ staff.
   3. Prepare the evaluation report incorporating all the feedback from both the document review and briefing session and submit the final report to NPA.
   4. All developed tools, questionnaires, reports, guidelines should be delivered to NPA in soft and hard copy.

Qualities of the Consultants:

The Consultant should be skilled in conducting evaluations and reviews and must have a substantive understanding of issues related to human rights, youth and women issues. Knowledge of programme management and implementation is critical for this assignment. The Consultant should demonstrate ethical consideration including a respectful attitude towards NPA partners and their constituency, as well as other stakeholders.

The Consultant should have proven experiences of previous assignments with development analysis, evaluations and reviews of a similar nature, as well as engaging communities using participatory methodologies.

The composition of the evaluation team is up to the Consultant who can choose, depending on their internal system, ideas and logic but the team members must fulfil competency criteria.

NPA recommends that the team should be comprised of a Team Leader/Focal point who takes overall responsibility for this evaluation and coordinates/liaises between NPA and the evaluation team on the ground. The evaluation team should include other personnel such as a coordinator, data analyst, enumerators, etc.

Supervision of the fieldwork and quality (reliability and validity) of the data/information collected at the field level is the primary responsibility of the Team Leader. Similarly, the Team Leader will work closely with NPA’s Programmes Team. In each step and process, consultation with the NPA Programme Manager is vital.

Confidentiality and Data Protection:

Confidential information, the “Confidential Information”, refers to any data or information relating to the operations of NPA which would reasonably be considered to be proprietary to NPA and its partners including, but not limited to, accounting records, business processes, and other records and that is not generally known to the public and where the release of that Confidential Information could reasonably be expected to cause harm to NPA and its partners.

The consultant should not disclose, divulge, reveal, report or use, for any purpose, any Confidential Information which the consultant has obtained, except as authorised by the NPA or as required by law. The obligations of confidentiality will apply during the consultancy and will survive indefinitely upon termination of this consultancy.

All written and oral information and material disclosed or provided by NPA and partners to the consultant under an agreement is Confidential Information regardless of whether it is provided before or after the date of this consultancy or how it will be provided to the consultant.

Equality between men and women and equal rights for women:

NPA adheres to an Equality between men and women sensitive approach in partnerships, organizational development, and programming such as supporting equal opportunities for women in partner organizations and in rights holder groups.

NPA and its partners ensure that both women and men have equal access to the different components of the programme. Both NPA and the partners employ women and men within their organisational structures. This programme is also contributing to enhancing the role of marginalised people in the decision-making process and promoting their representation in local structures and involvement in local dialogues. It also aims at raising the awareness of the concerned targeted groups and locations.