Mid-Term Evaluation (Civil Society Influence for Reduced Inequality in Iraq Programme) At Norwegian People’s Aid

Terms of Reference for Mid-Term Evaluation

“Civil Society Influence for Reduced Inequality in Iraq*”* Programme

Background

The Norwegian People’s Aid (NPA) is a non-governmental organisation with roots in trade unions and labour movements. NPA supports processes towards democracy and equitable distribution of power through mobilisation, popular participation and collective organisation. The international strategy affirms civil society as a key pillar for nation building; democracy and development, and views human rights as building blocks for development and redistribution.

NPA’s goal is to support human worth and equal rights for all, irrespective of sex, disability, ethnicity, religion, age, sexual preference or social status. The NPA vision is solidarity in practice. The vision demands commitment and action to protect life and health, build democratic societies and strengthen people’s ability to master their own lives. NPA is politically independent but not a politically neutral organisation.

NPA and its partners are implementing a five-year programme (2020 – 2024) with fund from the Norwegian Agency for Development Cooperation (Norad) with an overall objective to reduce inequality in Iraq politically, socially and economic.

The programme focuses particularly on reducing the inequalities faced by marginalised groups and local communities. The vision is that marginalised groups and local communities in Iraq enjoy equal access to decent public services, participate in decision-making and experience reduced social, political and economic inequalities. The programme supports partners’ capacity to mobilise their constituencies for effective and meaningful participation in decision-making forums on the local, provincial and national levels.

The programme’s target groups are marginalised people and communities, in particular ethnic minorities, workers and people deprived from liberty. The programme aims to promote the rights of these groups in Iraq by advocating for the authorities to reform unfitted legislations and practices, bringing an end to exclusion and discrimination on the basis of gender, class and ethnicity. The programme has employed top-down and bottom-up approaches in influencing the existing legal and social structures, and bridging the gaps between decision makers and constituencies. During this intervention, public participation was greatly promoted and the programme encouraged citizen engagement in the 2021 October election, emphasising the promotion of women participation in decision-making.

Targeted stakeholders are local support committees, comprised of various community actors: CSOs, activists, academics, religious leaders, and members of local councils. These actors are key to influencing proposed changes. NPA and partners aim to facilitate spaces for interaction between these targeted groups and local authorities to strengthen people’s participation and influence to increase access to decision making. Organisations engaged through networks in the defence of the rights of minorities, ethnic and religious communities are targeted actors.

Gender and equal rights for women

NPA adheres to a gender sensitive approach in partnerships, organizational development, and programming such as supporting equal opportunities for women in partner organizations and in rights holder groups.

NPA and its partners ensure that both women and men have equal access to the different components of the programme. Both NPA and the partners equally employ women and men within their organisational structures. This programme is also contributing to enhance the role of women in the decision-making process and promote the women representation in local structures and involvement in local dialogues. It also aims at raising the awareness of the concerned targeted groups and locations.

Purpose of the evaluation

The overall purpose of this mid-term evaluation is to assess to what extent NPA is on its way in terms of achieving the defined outcomes or whether adjustments are needed.

The five main objectives of this midterm programme-level evaluation are:

• To reflect on programme progress, achievements and change achieved so far;
• To assess relevance of the programme;
• To reflect on the NPA approach of working in local partnerships with civil society organizations to achieve programme objectives;
• To assess the effectiveness of the methodologies applied by NPA and its partners; and
• To suggest key practical targeted recommendations and learning to be considered by NPA in particular inform with practical recommendations the continuation and further implementation of the current programme and provide a basis for decision making on necessary amendments and improvements.

The mid-term evaluation will assess early signs of programme success or failure; both intended and unintended positive and negative trends will be assessed. The mid-term evaluation report will further provide the basis for NPA decision-making with regard to necessary amendments of the programme.

The Consultant/Company will assist to identify potential programme design problems and identify actions that may be taken to improve programme progress in the remaining period.

Users of the evaluation

The evaluation shall provide the donor and NPA with a review of the programme relevance, effectiveness, efficiency and partnership to make suggestions for potential adjustments if required.

The primary intended users of the evaluation are:

• NPA Iraq Country Office
• NPA Head Office (MENA team and technical advisors)
• Norad (donor)
• Partner organisations

The Theory of Change of what will be evaluated

The scope of this evaluation will focus on the programme Theory of Change, the role it has played in guiding NPA and its partners’ delivery of results against strategy, to learn how these results enable change for people and understand CSOs’ role, contribution and added value in this.

NPA’s vision for this programme is “a society with just distribution of power and resources”. In such a society, economic, political and social inequalities are low, the majority of its citizens consider the distribution of resources to be fairly just, and people do not experience discrimination based on gender, class or ethnicity. People have the right to organise, participate and express their opinions in a climate of respect and safety, and possibilities to hold decision makers accountable and influence the development of their societies.

To achieve this, we must gradually reduce economic, political and social inequality, and to reduce inequality we must understand the power relations and causes of power imbalances.

As NPA believes civil society organisations play an important role influencing decision-makers, the programme will cooperate with organisations that are, or have the potential to be, drivers of change for reduced political, economic and social inequality. Hence, NPA supports partners’ work to influence decision-makers and mobilise people.

Civil society organisations influence decision-makers’ policies and practices (outcome) Partners identify and prioritize the policies, practices and decision-makers they have the potential to influence. These may vary from implementation of already adopted policies, to getting an issue on the decision-makers’ agenda. Some selected policy/practice goals may be easier to achieve, while others may be hard. Partners use a variety of strategies to achieve change. Although the contexts vary significantly, one strategy used by many partners is mobilising people through campaigning and demonstrations.

Civil society organisations mobilise people to act and participate for rights (outcome) Many partners engage with communities to strengthen individuals and/or groups that experience political, economic and social inequality. Partners may support people, through different activities, to enable them to claim their rights and/or participate in local decision- making forums. In some cases, these community members are not organised, and lack tools to stand up against decision-makers. In other cases, the community members are members of the partner organisation. For instance, in several countries, partners organise entire communities. Formal organising may sometimes be a goal in itself, other times the aim is that people are able to act independently and/or in alliances.

To strengthen people/community members, partners will enhance peoples’ awareness about their rights, and how they can challenge and influence decision-makers. NPA will support partners to systematise and learn from experiences and adjust their strategies to improve their support to the target group.

Civil society actors have strengthened their capacity to influence (outcome)

As there are strong actors that gain from unequal distribution of power and resources, and therefore resist changes, civil society actors need to strengthen their capacity to influence and represent marginalised groups. NPA believes that civil society actors are more effective and sustainable in their efforts to influence if they are organised. Many spontaneous mobilisations have impact, and some can be game changing, but many become isolated events and do not lead to lasting changes. Organisations are fundamental building blocks in civil society and organising has shown to be an effective tool for marginalised people to assert power.

Strengthening civil society organisations is in itself a contribution to reducing political inequality, because through collective organising, marginalised or discriminated groups can assert power. As inequalities affect people who are politically, socially and/or economically marginalised most, it is especially important that they organise, voice their opinions, assert power and assume representation, to influence the public discourse and the policy agenda.

Key evaluation questions

The areas of results to be explored are the programme outputs and outcomes at immediate level (progress towards achieving results in the main programme proposal and approved deviations). The evaluation should describe, analyse and assess the main activities that have been carried out. The following represents a non-exhaustive list of key questions that should be answered in terms of the relevance, effectiveness and capacity, of the programme.

Some of the key questions/issues to be evaluated are:

Evaluation Criteria

Key Review Questions

Relevance and quality of design

• To what extent is the programme Theory of Change relevant to the needs, rights and priorities of programme beneficiaries? Among questions to be answered are:
  • Have any changes in the political context significantly affected any of the premises for the programme?
  • Has the programme strategy/methodology been adapted to the Iraqi context?
  • What is the actual relevance on the ground of the key target groups in the programme?
• Has the programme design presented an in-depth understanding of the issues of targeted communities in different regions of Iraq? Among questions to be answered are:
  • What are the actual political agendas of programme stakeholders on the ground in Iraq?
  • What role does the programme play in terms of interaction on the ground?
  • How has the programme so far contributed to the development of the activities/methods of the partners, stakeholders and civil society (political dialogue, networking, funding, etc), the main strengths and weaknesses of this support?
• To what extent are the objectives of the programme still valid?
• Is the programme utilizing and supporting the organisational capacities of CSOs, both within the existing organizations and through supporting new initiatives and organizational forms?
• Assess the NPA partnership model for working in partnership with local CSOs for programme implementation (refer to NPA Partnership policy document).
• Does the programme have any positive or negative effects on the environment? Could environment considerations be improved in planning, implementation or follow up?
• Does the programme have any positive or negative effects on gender equality? Could gender mainstreaming be improved in planning, implementation or follow up?
• Has the programme been designed and being implemented in a conflict sensitive manner?

Effectiveness of implementation

• What are the main strengths and weaknesses with regard to performance of the stakeholders, partners and civil society, and how are these addressed by the programme.
• How effective are the methods for strengthening the influence on community processes and mobilisation/participation of constituencies? Among questions to be answered are:
  • Do NPA’s partners mobilize better to influence processes and decision-making?
  • Do partners and NPA have a clear common understanding of the process towards the desired changes?
• What activities/methods have been agreed with the programme partners and what are strengths and weaknesses in NPAs support to improve internal capacities the partners as well as the civil society?
• How has the programme been contributing to the development of partner and activities/methods (e.g. dialogue, networking, funding, etc), the main strengths and weaknesses of this support? Among questions to be answered are:
  • Do NPA’s direct beneficiaries contribute to organizational development and improve management in their institutions/organizations and networks?
  • Do NPA’s partners ally with other organizations, sharing experiences, developing policy proposals and joint action, and can this be related to the programme?
• Have NPA partners improved their internal capacities such as information and debate, communication between leaders and constituencies, gender equality, strategies, human rights, minority rights and political education?
• Have NPA partners maintained their work to influence despite shrinking of civil space and restrictions?
• Have the target groups of the project acted to influence for their rights?

NPA capacity

• How is the NPA’s organizational capacity set up in order to achieve the intended programme results? This includes implementation performance, work planning, NPA head office involvement and NPA external office management model.
• How is the NPA capacity and systems for risk management?
• How is monitoring and documentation of outputs, outcomes and longer effects organized and what monitoring systems are in place?
• Assess the partnership process and the partnership relationship between NPA its local partners, identify what worked well and what did not work well and recommend possible changes.

Evaluation implementation

Timeframe

The preliminary draft report should be submitted to NPA by 31st August 2022 the latest and the final evaluation report should be submitted to NPA after two weeks of receiving NPA’s feedback to the draft report, latest 15th September 2022. The work is planned to commence not later than 1st July 2022.

The Consultant/Company should present detailed timeframe for each activity in the workplan including working days for each activity.

The Consultant/Company should have the following competencies and/ or experience.

• • Proven expertise in design and review of programmes focused on human rights, democratization and experience in the MENA context, preference will be given to candidates that have past experience with programme design, implementation and/or evaluation of development programmes in the MENA. – Data analysis and writing up of evaluation reports experience is also key for this exercise. The Consultant/Company should possess extensive work experience and in-depth knowledge on conducting programme evaluations. – The Consultant/Company should have post graduate degree in development studies and human rights, or equivalent knowledge in the areas mentioned in above two points. – Preferably, the Consultant/Company should deploy a multidisciplinary team with a background of political and statistical data analysis qualifications. – It is desirable to have a postgraduate in political studies. Preference will be given to those who possess past experience working with INGOs and conducting evaluation surveys. – The Consultant/Company should have proficiency in English. Working knowledge of Arabic is plus. – The field team much have proficiency in Arabic and Kurdish.

Organisation of evaluation management

• • The Consultant/Company will report to the NPA steering committee, consisting of the Country Office team (Country Directors, Programme Manager and MEAL Manager) and Iraq Advisor at NPA Head Office. – The steering group will oversee and support the day-to-day delivery of the evaluation. – The donor will provide feedback on draft findings and recommendations and inform NPA’s management response.
• • NPA will provide administrative/logistical support towards organization of documentation, interviews, meetings, field visits and workshops.

Methodology

The Consultant/Company will design the details in the methodology in cooperation with NPA. The evaluation should follow a collaborative and participatory mixed-methods approach that draws on both existing and new quantitative and qualitative data to respond to the evaluation questions mentioned earlier in this document. The Consultant/Company should follow a theory- based approach based on the programme theory of change. Furthermore, it should combine evaluation tools based on international standards and guidelines, notably OECD DAC Quality Standards. It is expected that the Consultant/Company will assess the quality of the programme’s theory of change and, if necessary, to develop a realistic impact logic based upon on the conducted interventions. The methodology design should be developed by the Consultant/Company in consideration of the information outlined in this ToR to ensure accuracy and rigor. A detailed methodology and data collection methods should be included in the technical proposal, which will be further improved in consultation with NPA team during the inception phase of the evaluation. The choice of methods must also consider the needs and capacities of the different target groups and stakeholders.

Desk review:

Desk Review should be conducted by the valuator to inform the methodology and development of the tools. In addition, the conducted desk review should cover the following documents:

• • Programme proposal – Results framework – Annual reports – Existing documents available at NPA and among partners

The desk review process should serve as guiding for the Consultant/Company to continue gathering resources that would enable him/her to carry out development of tools.

The document review will also include NPA policies, proposal, reports to donor, documentation from partners, and various forms of programme documentation.

It will also include field studies to the selected locations in Iraq. This will include field visits and interviews with NPA staff, donor staff, representatives of partners, local authorities and other stakeholders.

The evaluation should have a strong learning aspect and should therefore apply participatory methods that will include various stakeholders.

The main findings should be presented at NPA country office before the final version of the report.

Deliverables

1. Present to NPA Evaluation Inception Report: This document will operationalise and direct the evaluation. It will describe how the evaluation will be executed, thus refining and elaborating upon the terms of reference; it will be approved by NPA senior management and act as an agreement between parties for how the evaluation will be conducted. The work plan will include the following elements:
   • Expectations of evaluation
   • Roles and responsibilities
   • Evaluation methodology, questions
   • Evaluation framework
   • Information collection and methods of analysis
   • Reporting formats
   • Day by day work plan and time-frame for activities and deliverables
2. Develop the tools and methodology for conducting the evaluation, using both quantitative and qualitative approaches to address the key evaluation questions listed above. The Consultant/Company will then compare the findings with baseline values, using appropriate surveys, statistical tests (relevant methodology), to draw the findings and conclusions and taking into consideration the qualitative information to write the report.
3. Review all of programme documents (proposal, baseline report, mid-term evaluation and

partners’ reports) and relevant NPA policy documents.

1. Conduct qualitative/quantitative data collection through appropriate methods and carry out data analysis (including appropriate statistical tests).
2. Prepare draft evaluation report which includes the following elements:
   • A stand-alone Executive Summary
   • Introduction
   • Methodology
     • Responses to questions/ Findings
     • Problems and needs
     • Achievement of purpose
     • Overall assessment
     • Conclusions, Targeted Recommendations and Management Responses
     • Annexes to the report, including data analysis report.
3. Present findings to NPA and partners staff in a briefing session.
4. Consultant/Company receives feedback on the first draft from NPA and partners’ staff.
5. Prepare the evaluation report incorporating all the feedback from both the document review and briefing session and submit final report to NPA.
6. All developed tools, questionnaires, reports, guidelines should be delivered to NPA in soft and hard copy.

Qualities of the Consultant/Company

The Consultant/Company should be skilled in conducting evaluations and reviews and must have substantive understanding of issues related to human rights, democracy, and the context of Iraq, including the status of minorities and people deprived from liberty. Knowledge of programme management and implementation is critical for this assignment. The Consultant/Company should demonstrate ethical consideration including respectful attitude towards NPA partners and their constituency, as well as other stakeholders.

The Consultant/Company should have proven experiences of previous assignments with development analysis, evaluations and reviews of similar nature as well as engaging communities using participatory methodologies.

The composition of the evaluation team is up to the Consultant/Company who can choose, depending on their internal system, ideas and logic but the team members must fulfil competency criteria.

NPA recommends that the team should be comprised of Team Leader/Focal point who takes overall responsibility for this evaluation and coordinate/liaise between NPA and the evaluation team. The evaluation team should include other personnel such as coordinator, data analyst, enumerators, etc.

Supervision of the field work and quality (reliability and validity) of the data/information collected at field level is the primary responsibility of team leader. Similarly, the Team Leader will work closely with NPA’s programme management team. In each step and process, consultation with the NPA Programme Manager is vital.

Resource needs

Submission of technical and financial proposition:

The potential and interested Consultant/Company must submit a technical and financial proposals expressing their interest to conduct an evaluation for the programme “Civil Society Influence for Reduced Inequality in Iraq” as the subject line by 25th June 2022 to:

Procurement Department, NPA Iraq: iraq_tender_dhc@npaid.org

Short listed Consultant/Companies will be contacted by NPA for an interview prior to a final revision of the technical proposal budget and finalization.

Budget:

The total budget allocated for this work is 20K USD. The total submitted budget should include the cost to implement the above listed activities and deliverables as per the below table.

Budget line

Unit

Max. total # of days

Unit cost (USD)

Total cost (USD)

1. Inception phase

Personnel (give details of each person’s title and his/her

role in this evaluation).

For example, 1.1 Team Leader (overall responsibility on design implementation and writing the evaluation report)/ day/ xx days/ yy USD per day/ zzz total USD

Sub-total 1

2. Data collection phase

Personnel (give details of each person’s title and his/her

role in this evaluation).

For example, 2.1 Team Leader/ day/ xx days/ yy USD per day/ zzz total USD

Mention role of the person if he was not involved in phase 1.

Sub-total 2

3. Analysis, validation and write up phase

Personnel (give details of each person’s title and his/her

role in this evaluation).

For example, 3.1 Team Leader/ day/ xx days/ yy USD per day/ zzz total USD

Mention role of the person if he was not involved in phase 1 or 2.

Sub-total 3

4. Field work expenses and logistics

4.1 logistics for the field work including transportation and overhead costs

Lump sum

4.2 Flight tickets (two way)

Lump sum

4.3 accommodation and peridium

Lump sum

Sub-total 4

5. Other expenses

Sub-total 5

Total cost

Evaluation Criteria:

The application will be evaluated based on following criteria: The key factors stated below will be taken in consideration during the evaluation process:

Criteria

Total obtained score

Technical

70%

1. Demonstrated understanding, objective and completeness of the evaluation

10%

1. Methodology and implementation plan
   1. Details and quality (adequacy) of methodology proposed for the evaluation
   2. Detailed implementation plan indicating time frame

30%

(20%)

(10%)

3. Proposed team

Detailed description of the proposed team, position with the company with CVs (not more than three pages for each CV) of the proposed team leader and other key team members.

15%

1. Organization
   1. Experience in similar work in last 5-year, client list, management control system.
   2. Exposure in working with international donor, international organizations and other development agencies.
   3. Additional resources/ logistics which can be made available to conduct the evaluation.

15%

Financial

30%

All technically qualified proposals will be scored out of 30% based on the below formula. The maximum score (30%) will be assigned to the lowest financial proposal. All other proposals receive points according to this formula:

P= y (m/z) Where:

P= points for the financial proposal being evaluated

Y= maximum number of points for the financial proposal m= price of the lowest priced proposal

z*= price of the proposal being evaluated*

Evaluation Ethics:

Due to the dynamic situation in which NPA and its partners operate, as well as the potentially life-threatening nature of the issues involved, it is essential that the Consultant/Company adheres to strict ethical and security guidelines. The purpose of this evaluation should never be to hurt anyone or find out information at the expense of other people. The Consultant/Company will only be considered where the ethics and appropriate protocols are embedded into their organizational or private consultancy profile. The Consultant/Company has to demonstrate they are keenly aware of particular ethical and security issues that arise from working in this specific environment, including protection of local communities and the need for vigilant data handling and management.

Literature reads

• Previous relevant evaluations
• Programme documents
• Programme reports and available results information
• NPA current strategy
• NPA country programme strategies
• NPA Partnership Policy
• NPA Humanitarian Policy
• NPA Gender Policy
• NPA MEAL Policy