Experienced Project Evaluator At The Hague Institute for Innovation of Law

General Information

These are the Terms of Reference for the project evaluation of the Justice Innovation in a Pandemic programme in Uganda.

Programme Background

HiiL has been active in Uganda since 2015. In addition to our team of 10 international staff at our headquarters in The Hague, we have a local office in Kampala with four local contractors (in November 2021) who facilitate the implementation of the Uganda programme. Between March and August 2021 we also had a fifth local contractor in Kampala.

In partnership with the Swedish Embassy in Uganda, HiiL has successfully implemented a justice innovation programme in Uganda between 2017 and 2020. In 2021, HiiL and SIDA partnered once again to deliver the Justice Innovation in a Pandemic programme: a one year programme aimed at improving access to justice for people in Uganda during the Covid pandemic.

Programme Objectives

The overall goal of the programme is to provide increased access to justice for people in Uganda, in particular the most vulnerable, to prevent or resolve their most pressing justice problems during Covid times (more people-centered justice).

In line with this goal, HiiL has identified three objectives/outcomes to be achieved after the programme:

• Enhanced access to and use of information and data for/by various justice stakeholders in Uganda on justice problems during Covid times;
• Increased access to and use of justice solutions for people in Uganda in a pandemic; and
• Increased dialogue and agreement amongst JLOS and Ugandan justice leaders on the enabling environment for new or improved solutions that increase people centred justice in a pandemic.

The corresponding outputs for each expected outcome, and the complete list of associated indicators can be found in the logframe document attached with this ToR.

Programme Rationale

To achieve the goal, we have developed a comprehensive programme covering 5 elements: data from Justice Needs and Satisfaction Survey, evidence-based working guidelines and outcome monitoring tool, supporting the development and growth of justice innovations and startups, an enabling environment from transformation processes, and a national, regional, and international network on people-centred justice. As element 5 is a continuous process of building effective relationships with the stakeholders and donors, the programme focuses on the first 4 elements. Lessons have been learnt along the way and some adjustment took place, which will need to be incorporated into the evaluation.

By gathering more (disaggregated) data on what the justice problems are in the selected countries, for which groups of people, their incidence rate, how they affect the lives of those people, how the problems are resolved, and then turning this data into digestible, shareable products, we are able to improve access to information for decision makers.

Gathering data and sharing it is, in itself, not enough to achieve the change we want. It is necessary to take evidence-based working a step further by working on the basis of best practices of processes and practices that generally seem to work when it comes to preventing or resolving a justice problem in the area of family, land, employment, neighbours or crime. With local stakeholders, we collect such best practices and make them shareable.

We also improve the ways in which justice solutions are delivered to people. Our research and knowledge from projects highlight seven categories of justice delivery model that appear to be able to scale on the basis of a sustainable funding model. We call them gamechangers. By using our data and the best practices such gamechangers are developed by our Justice Accelerator and through Innovation Labs.

By bringing justice stakeholders together in stakeholder dialogues an enabling environment can be created and maintained to allow HiiL to operate, that will support the continued collection of data, the systematic development of best practice guidelines and the development of gamechangers. Those justice stakeholders also co-create a vision of and pathways towards an enabling environment for implementing people-centred justice, including gamechangers, how to get there, and a prioritisation of what to tackle first.

The incentive structure for ensuring that justice sector institutions are responsive to people’s needs is not optimal. Therefore, our work is supported by a network: external forces – national and international – that support and continue to connect the change-making people and organisations which make people-centred programming happen. Such a network can be a shared voice that speaks if change is stalled or is going in the wrong direction. It can be a repository of good practices which are shared. Lastly, it can ensure that more funders are brought into the arena: states, philanthropic organisations, and the private sector.

The majority of the programme has taken place online due to the Covid pandemic. Where in person activities were possible, they mainly took place in Kampala, though we work with social entrepreneurs from different districts as well. The original logframe is attached.

Local Partners of the Programme

HiiL works together with a range of local partners in Uganda to implement the components. An overview of some partnerships is provided here:

• Justice Law and Order Secretariat (JLOS). The JLOS Secretariat is one of our key partners
• Barefoot Law: component A1 (E JNS ). Barefootlaw is our data collection partner: they reach out to people in rural areas to ask them about their justice needs and experiences.
• Evidence and Methods Lab: component A2 (outcome monitoring tool) and A3 (Family Justice Catalogue). An alumni from the accelerator programme and currently participating in the Business Resilience Programme. EML is a design bureau who are working on creating user-friendly versions of the Family Justice Catalogue and updating the outcome monitoring tool.
• House of DJs: component B1 (justice innovation lab) and D2 (local office). House of DJs is an events and marketing company. They provide event management services for the Justice Innovation Lab and run our local communications channels. They are also responsible for the communication campaigns for the E JNS and the programme in general.
• LASPNET: component A2 (outcome monitoring tool) and A3 (Family Justice Catalogue). We have signed a MoU with LASPNET in October 2021 to increase adoption of evidence-based working among justice workers in Uganda and develop learning tools for justice workers.
• Donald Byamugisha: component B2 (Business Resilience Programme). Donald is a business development consultant who is supporting the innovators in developing their growth strategy, financial management and developing their business model.

Evaluation requirement

Objective

The primary aim is to assess the relevance and overall project performance towards achieving its objectives, and to provide lessons learnt from experiences of the project. Specifically, the evaluation aims to:

• establish the extent to which the programme outcomes outlined in the logframe and project documents have been and can be achieved;
• assess the extent to which the programme objective, activities and design are responding to the needs of the different target groups;
• To identify lessons learned and good practices, and make recommendations to improve the development and execution of future programmes;
• To improve accountability within HiiL through transparency of the level of achievement of the programme objectives.
• Identify if there is any measurable/demonstrable change that can be attributed to the project outputs /interventions? To what degree with which the project outputs/outcomes have addressed access to justice challenges during the COVID-19 pandemic.

The primary users will be the HiiL team members at the central office and local office space to make adjustments and improvements to the design and implementation of future programmes. HiiL will also use the evaluation to understand how the programme activities fit within the local context of Uganda, share the lessons and recommendations with the Justice Law and Order Sector (recently changed to the Governance and Security) Secretariat and Leadership Committee and the Dutch Embassy in Kampala, and align with the national action plans of the Ugandan justice sector.

The evaluation will be shared with the Swedish Embassy in Kampala as the funding agency, to contribute to knowledge building in justice innovation programmes as well as provide outcomes and where possible results of the Swedish support to access to Justice.

Scope

The evaluation shall be based on the project logframe and proposal. The evaluation covers the year 2021 only and focuses on the processes, results at various levels, as well as connections and relations between results. It will concern activities conducted during the SIDA programme in Uganda only.

Criteria

The evaluation shall conform to OECD DAC criteria for evaluations and focus on the following criteria:

Relevance

• To what extent has the project responded to access to justice needs at the person level during Covid pandemic?
• To what extent were the activities aligned with the strategic plans and objectives of the Uganda Justice Law and Order Sector?
• How effective was the coordination and interaction with internal and external stakeholders?
• Internal: between the various departments and teams, between central and local office
• External: Ugandan justice sector, in particular JLOS Secretariat and JLOS Leadership Committee, innovators, local civil society organisations working on justice innovation
• To what extent were local stakeholders participating in project implementation and follow up?
• To what extent was the project able to adapt and provide appropriate responses and implementation strategies to context changes, changes in needs, and the priorities of the various target groups?
• To what extent were the project’s objectives consistent with the priorities and the needs of vulnerable persons and how appropriate were the implementation strategies adopted?

Coherence

• How well did the activities fit in the Ugandan context?
• To what extent were the activities compatible with other justice innovation programmes/activities in Uganda focusing on addressing justice challenges during Covid times?

Effectiveness

• To what extent were programme activities implemented?
• To what extent did the programme achieve the objectives (outcomes) for the targeted groups?
• To what extent did the programme deliverables (outputs) contribute to the overall objective of the programme?
• How effective did we work with local partners? How effectively did we communicate with local partners?
• Which factors hindered or contributed to the achievement of the programme objectives?
• To what extent is the frequency and depth of monitoring data gathered sufficient to make informed assessments on the achievement of results?

Efficiency

• To what extent were project results efficiently delivered in regard to the implementation strategies that were used, the use of resources – funds, equipment, time etc.

Sustainability

• How sustainable have the outcomes been over the project period?
• To what extent were sustainability mechanisms established by the project including strengthening of justice delivery mechanisms in Uganda?
• To what extent do the sustainable mechanisms link with existing access to justice programmes, government strategies and priorities in Uganda?

Impact

• To what extent has the project contributed to wider changes such as contribution to the justice innovation ecosystem, contribution to policy or justice sector priorities or contribution to increased use of justice data and, in that way, has the programme contributed to making justice more accessible in Uganda?
• Were there positive or negative unintended outcomes as a result of the programme?
• To what extent did women benefit from the current programme?

Lessons and best practices

• What has worked and not worked among the strategies used to deliver the project?
• What do we learn from the interventions implemented?
• To what extent did the current programme incorporate the lessons learned from the previous evaluation?

Methodology

The proposed evaluation shall build on existing theories and reports conducted previously. Due to the Covid 19 pandemic, travel to the region might not be possible or very limited. Meeting and travel restrictions may also apply in Uganda which will make in person interviews not possible. The evaluation will therefore be done by an international consultant at distance, who is expected to gather data virtually and/or liaise with a local data collection company in Uganda for primary data collection. The evaluation is expected to use a gender sensitive approach/methodology, methods, tools and data analysis techniques.

The design of the evaluation should include the following elements but HiiL remains open to suggestions:

1. Desk research. A desk review will be conducted to assess the activities, processes and results of the programme. It shall be based on collected impact data, outcome monitoring, existing reports, training materials, innovators reports, feedback from local partners.
2. Online interviews with key stakeholders. These informative interviews with internal and external stakeholders feed into the evaluation questions, including the level of coordination and value add of the project to the already existing justice innovation initiatives in Uganda, as well as assessing the sustainability of the programme outcomes. Examples of external stakeholders include: representatives of the JLOS Secretariat and the JLOS Leadership Committee, Uganda Association of Women Lawyers (FIDA), Innovation Village, LASPNET, IDLO, Uganda Law Society, Law Development Centre innovators, mentors from the Accelerator programme, those who have used the services from the entrepreneurs, Office of the Chief Justice and office of the Permanent Secretary to the Judiciary. Examples of internal stakeholders: local team in Uganda, Directors of Product Groups (central office), Project Management (central office), MEL Specialist (central office), Project Group members (central office).

Implementation

Deliverables

• An inception report with a maximum of 5 pages, with an evaluation matrix and data collection tools in the annexes. These shall be approved by HiiL before the data collection. The report will be submitted in English.
• Monthly briefings with the Steering Committee at HiiL via Google Meet, including a debrief of the initial findings.
• A draft evaluation report for review by HiiL of maximum 20 pages excluding annexes in English.
• A Final Report of maximum 15 pages excluding annexes in English
• A presentation in PowerPoint of maximum 15 slides representing the report including graphs and tables
• Presentation of the final report with recommendations (remote, maximum 2.5 hours) to HiiL
• Databases with the relevant quantitative (Excel) and qualitative (transcriptions) data and analyses

Timeline

Start date: 8 December 2021

Inception report: 15 December 2021

Evaluation start: 1 January 2022

Draft report: 1 March 2022

Final report: 31 March 2022

Logistical support

A Steering Committee composed of Project Management, Programme Director, MEL Specialist and 3 local team members will coordinate the process of the evaluation.

HiiL will provide the following support to the evaluation team:

• Transmission of background materials and information (project proposal, donor reports, logframe, training materials, innovators impact data, meeting notes etc.)
• Introduction to key stakeholders
• Meeting arrangements with internal and external stakeholders if requested

The focal point of contact is the Project Manager at HiiL.

Budget

The proposal shall not exceed a value of 15,000 EUR.

Expected profile of consultant/s

HiiL is looking for a consultant, or team of consultants, with the following experience and capacity:

• Senior justice sector consultant or development professional- minimum of 10 years experience;
• Demonstrated experience in conducting research and evaluations with quantitative and qualitative components;
• Advanced university degree in law, development, international relations, social sciences, or related field of study;
• Expertise in access to justice programmes in Africa required: ideally with experience in Uganda
• Comfortable working with a broad range of people from different backgrounds and cultures;
• Strong analytical, interpersonal and reporting skills;
• Demonstrated ability to produce high quality data; and
• Fluency in English.

The criteria for awarding contracts resulting from this Request for Proposals are based on “best value for money.” For the purpose of all tenders, HiiL defines “best value for money” as:

A complete assessment of technical, organisational and pricing factors (quality of the training, content, trainers, previous experience, the format in which it is presented, ability to execute the training online (if needed) and the delivery time).

Technical evaluation (total points =100) (counts for 60 %)

To be technically acceptable, a proposal must meet or exceed the requirements and specifications in the Request for Proposals as listed below.

Technical criteria

Documents required

-Proposal addressing all technical requirements in experience and capacity, demonstrating an understanding of the ToR, methodology, tools to be used, and format

-Proposal which outlines the work plan, methodology, team and risk mitigation due to Covid 19 situation

-Proof of expertise and knowledge needed to carry out the evaluation

-CV and 2 references from organisations that can verify the quality of the consultant’s work and /or publicly available example of a previous evaluation.

Only proposals that score a minimum of 50 points for the technical evaluation will go to the commercial evaluation.

Commercial evaluation (total points=100) (counts for 40%)

Commercial criteria

Documents required:

-Proposal clearly indicating the prices

-Financial offer outlining the budget (in EUR)