Call for Tender – Thematic Evaluation on External Oversight of the Security Sector

1. Background

About DCAF

The Geneva Centre for Security Sector Governance (DCAF) is an organization dedicated to improving the security of states and their people within a framework of democratic governance, the rule of law, and respect for human rights. DCAF contributes to making peace and development more sustainable by assisting partner states, and international actors supporting these states, to improve the governance of their security sector through inclusive and participatory reforms. It creates innovative knowledge products, promotes norms and good practices, provides legal and policy advice and supports capacity‐building of both state and non‐state security sector stakeholders.

DCAF’s Foundation Council comprises 60 member states. Active in over 70 countries, DCAF is internationally recognized as one of the world’s leading centers of excellence for security sector governance (SSG) and security sector reform (SSR). DCAF is guided by the principles of neutrality, impartiality, local ownership, inclusive participation, and gender equality. For more information, please visit www.dcaf.ch

External Oversight

Democratic oversight, as both an established international norm and an essential trait of a democratic state, is a crucial component of good security sector governance. Oversight ensures that state resources are managed efficiently and effectively, the security sector personnel behave with honor and integrity, misconduct is detected and corrected, and those who commit abuses are held accountable. Within such a framework, accountability is provided through internal and external supervision of security providers.

Non-State governance and oversight mechanisms like independent bodies, civil society, and the media, help ensure that the security and justice services provided by relevant authorities are delivered in accordance with the rule of law, and that they advance the well-being of the society as a whole and its vulnerable groups. They can act as a platform to voice the concerns of the people affected by poor security and justice delivery and help ensure that SSG/R programmes are representative, inclusive and people centered.

DCAF Partnerships

DCAF works with a range of oversight actors, including parliaments, the judiciary, independent human rights commissions, Ombud’s institutions, civil society, and the media. Projects are implemented across the five geographic regions in which DCAF is active: Europe and Central Asia, the Middle East and North Africa, Sub-Saharan Africa, Latin America and the Caribbean and Asia Pacific. Of these, a number are taking place in fragile and conflict-affected contexts. Gender is a key focus area for DCAF, and is mainstreamed across many projects, including those involving external oversight. Projects may involve direct support to national reform processes, engagement with the bilateral and multilateral stakeholders supporting these processes, informing, and shaping international policy, or a combination of two or more of these approaches.

2. Purpose of the evaluation

Thematic evaluations are strategically important and provide opportunity for organizations to assess their performance within a specific thematic area. This thematic evaluation on external oversight of the Security Sector is part of a wider lesson learning process that focuses on DCAF-wide efforts to implement key aspects of its mandate.

DCAF is a knowledge organization, not only learning from its operational experience, but also applying its broad thematic knowledge to the operational challenges it faces. The purpose of the thematic evaluation is to support corporate learning and, at the same time, to strengthen DCAF’s accountability to partners and donors, including its Foundation Council.

By carrying out this evaluation, DCAF provides timely knowledge and lessons from experience that can feed into programming at all levels. By sharing findings and recommendations the evaluation report also presents a general learning output for the wider SSG/R community.

3. Evaluation criteria and questions

Thematic evaluations are usually not based on a common results framework and do not assess projects in detail and individually. Thematic evaluations are exploratory, and attention needs to be paid to keep their scope realistic. It is thus neither necessary nor recommendable to use the complete standard set of OECD DAC evaluation criteria. The evaluation questions for this thematic evaluation therefore cover the central dimensions of effectiveness, coherence, sustainability, and impact and were identified through demand analysis and stakeholder consultations.

The evaluation must be guided by the following questions, grouped per evaluation criteria:

1. Effectiveness

· Question 1: In what ways has DCAF been successful in strengthening external oversight through its interventions? Are there any unintended outcomes (both positive and negative) and what are their implications?

· Question 1.1: Have the underlying key assumptions relevant to the achievement of improved oversight been explicitly articulated, and tested prior to, during and after implementation?

· Question 1.2: Which project approaches/ pathways have been more successful than others in encouraging and facilitating reforms? And why?

· Question 1.3: To what extent have approaches which involved working with multiple oversight actors resulted in different outcomes than those with single stakeholders?

· Question 1.4: Which approaches have proved to be more effective / less effective in fragile and conflict affected contexts?

· Question 1.5: Has DCAF’s work on improving oversight contributed to the protection of human rights, including gender equality and women’s empowerment? If so, how? And if not, what gaps exist in DCAF’s approach?

· Question 1.6: What lessons can be drawn from projects which have included regional exchange and dialogue?

2. Coherence

· Question 2: How have DCAF projects contributed to and strengthened international and multilateral approaches to reform of external oversight?

· Question 2.1: How effective has DCAF been in coordinating and partnering with other actors in security sector reform?

· Question 2.2: How has DCAF used its diverse organizational expertise and comparative advantages to achieve better outcomes?

3. Sustainability

· Question 3: Are there indications that successful DCAF interventions with external oversight actors are being sustained? If so, which factors have been most influential?

· Question 3.1: What opportunities exist to improve sustainability through a focus on local ownership and engagement with political actors/dynamics?

4. Impact

· Question 4: What indications exist that DCAF’s interventions around external oversight contributed to more effective and accountable provision of security as a public good?

4. Scope

The evaluation will cover approximately 26 projects, funded by different donors, that have been or are being implemented in the Middle East, Africa, Asia, Europe, and Latin America (some of those in fragile contexts) over the past ten years with external oversight actors of the security sector, i.e., political/ independent bodies, media and civil society. About half of the projects are ongoing but a number of these are close to their end dates. In average, projects have/ had a duration of 2-3 years. In some cases, ‘external oversight’ is an outcome within a larger project.

The selected projects are different in nature – while many projects focus on directly supporting security sector reform processes with national or multilateral partners, some projects focus on the development of knowledge products and others on providing policy advice. DCAF’s aim is to ensure that its operational experience influences its work on policy and research, and vice versa. Together these projects reflect the multi-dimensional approach that DCAF uses in supporting SSG, and the chosen selection aims to derive lessons from an all-of-DCAF experience working with external oversight actors.

5. Methodology

The TE report must provide evidence-based information that is credible, reliable, and useful. The evaluators are expected to follow a participatory and consultative approach ensuring close engagement with the Project Teams, government counterparts, Implementing Partners, Field Offices, direct beneficiaries, donors, and other stakeholders.

In thematic evaluations that cover multiple interventions, there will not be a logical framework (logframe) available to draw upon. Therefore, the evaluation will need to explore what general causal pathways and assumptions are implied in projects, and whether there are commonalities between the intervention logics and the underlying assumptions on how change takes place. This will require review of project design documentation, and discussion with staff and partners involved in design and implementation.

The evaluation needs to combine various data collection and analysis methods that allow gathering quantitative and qualitative evidence to inform the findings, conclusions, recommendations, and lessons learned drawn from the exercise. Methods include document reviews, meta-analysis of previous (mid-term and terminal) evaluations or reviews, thematic, regional and/ or country case studies, and interviews.

The desk review of secondary project documentation includes but is not limited to project documents, concept papers, logframes, ToCs, Terms of References (ToR), narrative internal and external reporting, MoUs with partners, media reports, press statements, and project outputs e.g. Knowledge Products.

To ensure triangulation of information, primary data needs to be collected directly by the evaluators through written correspondence and interviews. Interviews should be conducted virtually with key stakeholder groups for each selected project, including DCAF staff in Headquarter and Field Offices, implementing partners, beneficiaries, relevant international SSG/R actors, and donor representatives. Travel is not foreseen.

The specific design and methodology for the thematic evaluation should emerge from consultations between the evaluators and the above-mentioned parties regarding what is appropriate and feasible for meeting the purpose and answering the evaluation questions, given limitations of budget, time and data. The final methodological approach must be clearly outlined in the Inception Report and be fully discussed and agreed between DCAF and the evaluators.

The evaluation will assess results according to the criteria and evaluation questions (see point 3) in a ‘findings’ section. The section on “conclusions” will be written considering the findings. Conclusions should be comprehensive and balanced statements that are well substantiated by evidence and logically connected to the evaluation’s findings. They should highlight the strengths, weaknesses, and results of DCAF’s work with external oversight actors.

Recommendations need to clearly address a previous stated issue, be precise and actionable. Lessons should succinctly specify the context from which they are derived, establish their relevance beyond that context (where it will be applied and by whom) and suggest some prescription or action. Although lessons are derived from a specific situation, they are intended to have wider relevance.

The final report must describe the full approach taken, repeat the rationale for the chosen methods, and lay out any limitations that came therewith. In any case, the evaluators must use gender-responsive methodologies and tools.

6. Management of the evaluation

The principal responsibility for managing this thematic evaluation resides with the commissioning entity. The commissioning entity for this evaluation is the Operations Department, with support from the Director’s Office. The commissioning entity will contract the evaluators and ensure the timely provision of guidance and fees. Together with all the involved project teams, it will be responsible for liaising with the evaluators to provide all relevant documents and set up stakeholder interviews. Finally, the commissioning entity will oversee circulation of the evaluation report and managing the follow-up process.

7. Deliverables

Expected deliverables include:

1. An inception report which defines the methodology and timeline of the evaluation. It needs to include a preliminary list of documents and interviewees and an evaluation matrix that pairs evaluation questions with indicators and information collection tools and sources.
2. A presentation of initial findings shortly prior to the dissemination of the draft report.
3. A draft evaluation report of no more than 30 pages (without annexes), including an executive summary of no more than 2-3 pages.Geneva Centre for Security Sector Governance
4. The final evaluation report must be proofread and presented in print-ready format.

The structure of the final report shall be based on the following approximate items:

· Executive Summary.

· Concise Background Chapter, including an overview of the types of projects included in the evaluation

· Purpose, scope and methodology used for the evaluation, including evaluation questions and assessment criteria, methods, and limits to the evaluation.

· Analysis of the collected information: Findings in response to evaluation questions and criteria.

· Conclusions, lessons learned and recommendations.

· Annexes (ToR, list of documents reviewed, list of interviewees, logic model, audit trail on how all received comments have (and have not) been addressed in the final report.

1. A final presentation of the findings and recommendations to DCAF staff and invited partners.

All deliverables must be produced in English.

8. Process and timeline

The evaluation is expected to take place during Q3 and Q4 of 2021. Any delays that may result from events beyond DCAF’s and /or the selected evaluator’s control, shall be mutually communicated and agreed upon.

The tentative TE timeframe is as follows:

· August 20th: Application closes

· 4th week of August: Selection and contracting of evaluators

· 1st week of September: Preparation of the evaluator(s), provision of project documents

· 2nd week of September: Document review and preparing Inception Report

· 3rd week of September: Finalization of Inception Report

· From 3rd of September: Interviews and written correspondence with stakeholders

· End of October: Presentation of initial findings

· Mid-November: Preparation of draft report

· End of November: Circulation of draft TE report for comments

· 1st week of December: Incorporation of comments on draft report and finalization of report

· 2nd week of December: Final presentation

9. Criteria for bids (selection of evaluators)

Applicants can be a firm or a group of professional evaluators (minimum 2 evaluators), who stand in no direct or indirect relationship to DCAF’s current or planned external oversight projects. The evaluators also cannot have participated in any of the projects’ formulation and/or implementation, must not have conducted projects’ midterm or terminal evaluations or reviews, and should not have a conflict of interest with the projects’ related activities.

The proposed evaluators shall have demonstrable experience in conducting thematic evaluations, including evaluating projects or programmes related to external oversight actors in the security sector. The proposed evaluators should be able to work in English. Ability to also work in French and Spanish would be a distinct advantage.