CALL FOR TENDER – EXTERNAL EVALUATION At Geneva Centre for Security Sector Governance

DCAF program: Strengthening the role of journalists and social leaders in oversight of the security sector in Colombia

1. Brief description of DCAF

DCAF – Geneva Centre for Security Sector Governance is dedicated to improving the security of states and their people within a framework of democratic governance, the rule of law, respect for human rights, and gender equality. Since its founding in 2000, DCAF has contributed to making peace and development more sustainable by assisting partner states, and international actors supporting these states, to improve the governance of their security sector through inclusive and participatory reforms. It creates innovative knowledge products, promotes norms and good practices, provides legal and policy advice and supports capacity‐building of both state and non‐state security sector stakeholders.

DCAF’s Foundation Council members represent over 50 countries and the Canton of Geneva. Active in over 70 countries, DCAF is internationally recognized as one of the world’s leading centres of excellence for security sector governance (SSG) and security sector reform (SSR). DCAF is guided by the principles of neutrality, impartiality, local ownership, inclusive participation, and gender equality. For more information visit www.dcaf.ch and follow us on Twitter @DCAF_Geneva.

Since 2017 DCAF has been working in Latin America, more specifically in Colombia (various thematic areas: Media and SSR, Gender and SSR, Use of Force, Climate Change and SSR and Hybrid Security), Honduras (Police Reform) and Chile (Police Reform) with the support of the German Ministry of Foreign Affairs (MFA), the Norwegian MFA, the Dutch MFA, the COSUDE and the EU.

2. Programme’s Context

Journalists and the media play a crucial role in security sector governance, as they monitor the work of security institutions and have an impact on the public’s perception of the security forces. Similarly, security forces play a key role in protection, ensuring an enabling environment for freedom of expression, access to information and other fundamental freedoms. Therefore, a relationship of mutual respect and understanding of each other’s rights, duties and limitations is essential in a democratic society to guarantee the right to freedom of expression and freedom of the press.

In this sense, at least 58% of Colombia’s municipalities correspond to zones of silence, places where there are no media outlets producing local news. The lack of access to information and the articulation of a relationship of mutual understanding between journalists and members of the National Police during social protests and other tense scenarios has affected guarantees for freedom of expression, the prevention of human rights violations and the promotion of proper journalistic practice.

Likewise, Colombia is considered one of the most dangerous countries in the region for social leaders. According to the UN Special Rapporteur on the situation of human rights defenders, violence against human rights defenders has been on the rise, and the UN Verification Mission in Colombia has considered the murders of human rights defenders, social leaders and ex-combatants as the most serious threat to peace.

3. Programme’s Justification and Objective

Freedom of the press is a cornerstone of democratic systems, which are based on the values and principles of equality, justice, protection of freedoms, guarantee of human rights and promotion of the rule of law. One of the essential characteristics of the right to freedom of expression is that it has a dual dimension: individual and collective. On the one hand, it protects the individual right to express oneself and to use any appropriate means to disseminate through, and on the other, the collective right of every person to receive any information and to know the expression of the thought of others.

DCAF recognizes that a proper understanding between the role of public security officers, who are empowered to enforce the law and ensure public order, and the role of journalists, who are interested in covering events and news of public interest, is key to good governance of the security sector within a framework of respect for human rights, and the values of a democratic society.

In this regard, DCAF is about to conclude (15 October 2022) a program on Media and SSR in Colombia, with the support of the Federal Foreign Office of the Federal Republic of Germany, entrusting the Foundation for Press Freedom (FLIP), the Colombian National Police and the Digital School (Escuela Digital) with the implementation of the project.

The program supported the National Police and journalists in establishing dialogues to address challenging or tense scenarios in their communications and interactions, through training, focus groups and the development of a guide with common agreements.

Under the same program, DCAF also supported the production of local journalism by training local journalists and producing communication pieces in “silenced zones”, these are, areas where there is a lack of media outlets and locally produced information and conflict affected areas.

DCAF also supported the “Digital School” initiative to train social leaders in the use of digital tools to amplify their voices and make their work known to a wider audience, to promote and raise awareness of the social justice work they do. Those three components of the project are reflected in the objective of the program and the expected outcomes, focused on strengthening the role of journalists, local media and social leaders in monitoring the security sector in Colombia and fostering a better working relationship between media professionals and the Colombian National Police.

• Outcome 1. Improve the working relationship between police and journalists by fostering a clear understanding of the roles, functions, duties and rights of the parties, especially in tense scenarios.
• Outcome 2. Strengthen the monitoring role of social leaders and grassroots journalists in remote regions of Colombia (“silenced zones”), through capacity building and promotion of local journalism.
• Outcome 3. Promote the use of social networks and digital channels by recognized social leaders and human rights activists as a tool for awareness-raising, democratic participation, and oversight of the security sector.

4. Objective, scope, and approach of the evaluation

4.1. Purpose and objective The purpose of the evaluation is to analyze the degree to which the program has met and achieved its objectives at the outcome and impact level according to the results framework in qualitative and quantitative terms, including planned and unplanned outcomes and impact.

Likewise, possible divergences or unintended effects should also be made visible. The evaluation should clearly show the progress of the program with a specific focus on the relevance, coherence, efficiency, effectiveness, impact and sustainability of the project.

4.2. Scope The evaluation will cover all program components and outcomes and will be guided by the indicative questions detailed in the following section. The timeframe is from the project starting date 15 October 2020 to its final date 15 October 2022.

The geographical focus is national for the first outcome, so the evaluation can be conducted mainly in the city of Bogota and the implementing partners have been the FLIP and the National Police; for the second and third outcomes the geographical scope is regional considering the usual place of residence of the direct beneficiaries of the project activities; for the second outcome the implementing partner was the FLIP and for the third one the Escuela Digital.

4.3. Indicative evaluation questions/ main areas of interest The evaluation should be guided, among others, by the following questions, grouped by evaluation criteria:

Relevance • What is the perception of direct beneficiaries, the National Police, FLIP and the Digital School (stakeholders) of the project? What kind of added value/difference does the program make from their perspective? • What were the emerging challenges and opportunities that the project should account for? Coherence • To what extend this programme coordinated actions with other actors to achieve the expected results? Efficiency • Was the intervention implemented on the basis of a results-oriented approach? • Is the monitoring system in place adequate to stipulate the impact of the project in terms of its objective at the outcome/impact level? Effectiveness & Impact • How did the program activities contribute to the achievement of the intended outcomes and impact? • To what extent were the mitigation measures effective to face risks and undesirable side-effects that could hinder the attainment of the purpose and aim, as well as the long-term impact of the project? • What type of behavioral and/or institutional functioning change is the project contributing to? Sustainability • What assurances are there that the achievements at the outcome/impact level will be sustained after the project ends? • Is there ownership of project activities and impacts by the National Police, journalists and social leaders? Cross-cutting aspects • How was gender considered during the implementation of the project? • How the programme contributed to have a better understanding of the right of freedom of expression & press and for the protection of journalists and social leaders?

5. Evaluation process and methodology

5.1. Evaluation Methodology The evaluation must conform to the principles and quality standards accepted by the DeGEval – German Evaluation Society (Deutsche Evaluation Society or Deutsche Gesellschaft für Evaluation e.V.) and the OECD-DAC principles and standards of evaluation quality.

A participatory evaluation approach should be applied. The evaluation should be conducted using a combination of methodologies that incorporate quantitative and qualitative data collection and analytical methods used in empirical social research. Secondary data sources should be considered, as primary data sources should be taken into account. The methodology is expected to include the following components: • Desk review of program documents (proposal, logical framework, results planned final narrative and financial reports, internal reporting documents) and other relevant documentation (memoranda of understanding with local partners, media reports, strategies and policies, press statements from partners, etc.). • Desk review of program outputs (reports, publications, online databases and other web resources, project deliverables, etc.). • Field visits to the regions, including: – Interviews with DCAF staff – Interviews and focus groups with key security sector stakeholders and partners – Interviews/focus groups with beneficiaries – Interviews with stakeholders’ groups (including local media, civil society and experts, etc.) – Interviews with donor representatives. • The evaluation process should be designed to identify lessons learned and key findings to improve the knowledge base, actions and relationship with partners for an eventual continuation of the programme. The process of the evaluation should therefore include appropriate reflection and feedback phases.

5.2 Principles and ethics criteria The following principles and ethics criteria should be observed in the development of the evaluation: • Transparency of values: The values of those involved and affected, which are manifested in their perspectives and assumptions and have an influence on the evaluation and interpretation of its results, should be documented transparently in order to better understand evaluation results. • Completeness and clarity of reporting evaluation: reports should provide all essential information and be precise, understandable and comprehensible. • Diplomatic approach: Evaluations should be planned and carried out in such a way that the highest possible level of acceptance of the affected parties with regard to the procedure and results of the evaluation can be achieved. The interests of all participants and affected parties should be taken into account in a balanced way to ensure their acceptance, consent and cooperation. • Fairness: The fairness standards are intended to ensure that in an evaluation, respectful and fair treatment is given to all involved and affected persons and groups are treated respectfully. • Protection of individual rights: Anonymity and confidentiality of individual informants and all interviews must be protected and guaranteed. Data must be protected from unauthorized access by third parties. • Comprehensive and fair examination: The evaluation should examine and present the strengths and weaknesses of the object of evaluation in fairness and as comprehensive as possible. When identifying the strengths and weaknesses of an evaluation object, it is important to always take unintended effects into account. This also requires that differing views, interpretations, and conclusions be considered and documented. • Data Economy: The principle of data economy must be considered in that no additional information is collected. • Impartial implementation and reporting: The evaluation should consider the different views of stakeholders on the subject and results of the evaluation. The entire evaluation process as well as the evaluation reports should show the impartial position of the evaluators. • Disclosure of results and reports: Evaluation results and reports should be made available to all stakeholders and affected parties as far as possible. • Accuracy: The accuracy standards are intended to ensure that an evaluation produces and conveys valid and comprehensible information and results on the respective evaluation object and the evaluation questions are produced and communicated. An accurate and complete description of the object of evaluation helps to understand what exactly the evaluation and its results refer to. • Due to the sensitivities around primary data collection, it is possible that beneficiaries, staff, and stakeholders to be interviewed for this evaluation may choose not to answer certain questions. Participants must not be quoted in the reports without their consent. In principle, interviews are to be anonymized.

6. Duration of the evaluation mandate

The duration of the evaluation is a maximum of 40 working days from the signing of the contract with the evaluator(s). This will include a maximum of 15 days in the field (Bogotá and selected regions). To be noted that DCAF staff may need up to 5 working days to provide feedback on the draft methodological report before going to the field and another extra 5 days for the draft of the final evaluation report.

The time frame is structured as follows (to be reviewed and specified by the evaluator(s)):

• Call for external evaluation / Sept.-Oct. 2022
• Evaluation of proposals and selection of the evaluator(s) (DCAF) / Oct. 2022
• DCAF – Evaluator(s) interview / Oct.2022
• Sending of relevant documents to the evaluator by DCAF / Nov. 2022
• First review of key project documents and first interviews (valuator(s)) / Nov. 2022
• Preparation of the initial methodological report: defining the objectives and evaluation questions, evaluation design, methodology, etc. (evaluator(s)) / Nov. 2022
• Submission of draft initial methodological report (evaluator(s)) / Nov. 2022
• Review of the initial methodological report (DCAF) / Nov. 2022
• Logistical and administrative preparation for data collection (e.g., interviews, focus group interviews, etc.) (evaluator(s)) / Nov. 2022
• Execution of the evaluation with detailed review of written documents, collection of information (conducting interviews, etc.) (evaluator(s)) / Dec. 2022
• Meeting DCAF – evaluator(s) at the end of the evaluation / Dec. 2022
• Analysis of information and preparation of the draft evaluation report (evaluator(s)) / Dec. 2022
• Submission of draft evaluation report and review by DCAF / Dec. 2022
• Preparation and delivery to DCAF of the final evaluation report in accordance with the observations made by DCAF / Dec. 2022

7. List of expected deliverables

7.1. Initial methodology report: Report defining the evaluation objectives and evaluation questions, evaluation design, methodology, etc. containing at least the following information: • The description of the development project/intervention; • The objectives and final evaluation questions; • A summary of the documents reviewed; • Characteristics of the evaluation methodology (incl. strengths and limitations) and evaluation issues; • An analytical framework for answering the evaluation questions with evaluation scales to be used to assess information, data sources and collection, sampling and key indicators; • Conceptual framework(s) to be used in the evaluation; • Proposed timeline; • First list of interviews; • Initial findings, if any.

7.2. Draft evaluation report: The draft evaluation report should include the evaluation findings, conclusions and recommendations.

7.3. Preparation of a meeting for the presentation of the draft evaluation report: After the execution of the evaluation the evaluator(s) should organize an accountability workshop in order to share initial findings and discuss with DCAF and project stakeholders, as well as to receive feedback from them. This will assist in the drafting of the final report.

DCAF will comment on the draft report, focusing on completeness, language, structure, intelligibility and any factual inaccuracies. The evaluator should finalize the report taking these comments into account.

7.4. Final evaluation report: The report should be written in English, be logically structured, contain evidence-based findings, conclusions, lessons, and recommendations, as well as their correlations. All information that is not relevant to the overall analysis should be moved to the annex. The report should answer in detail the evaluation questions and key priority areas. It should include 5 to 10 specific recommendations made for the project, and identify the necessary actions required indicating the responsible party, and possible timelines (if applicable).

The evaluation report should be between 25-40 pages maximum including an executive summary (2-3 pages) without counting annexes. The report should contain clear references to important information and data available in the annexes.

The evaluation report describes the sources of information used (documents, respondents, administrative data, literature, etc.) in sufficient detail so that the adequacy of the information can be assessed, and the conclusions should be explicit and justified based on the data collected and analyzed so that they can be understood and judged.

8. Reference Documents

Upon contract signature, DCAF will share key documents such as: • Documents regarding the program, e.g., proposal, logical framework, results planned, final narrative and financial reports, internal reporting documents • Tentative list of persons and organizations for interviews. • DCAF’s strategy in Latin America and the Caribbean 2021 – 2024. • DCAF may also share security reports of the areas where the evaluator(s) will be conducting interviews and activities related to the evaluation.

9. Profile and qualifications of the evaluator(s)

Given the magnitude of the evaluation, DCAF is looking for a team of up to 2 evaluators or a single evaluator, national, international or mixed, with demonstrated ability to implement an evaluation project of the scope described above in a professional and successful manner.

Profile and qualifications: • Professional(s) with higher education in areas related to social sciences, history, law, development cooperation; preferably with continuing education or postgraduate training in issues related to security sector reform & public institutions and/or to civil society. • Extensive experience in evaluation of development programs and projects supported by international cooperation, (minimum of five years of specific experience); • Broad knowledge of the Colombian political and social context, preferably with knowledge of some or all of the following: the institutional context, the security sector and/or social movements, and the journalism sector as well as sectorial strategies and priorities, as well as its main actors (in particular the Colombian National Police); • Extensive experience in facilitating evaluation processes (internal and external) with a participatory and multi-stakeholder approach; • Mastery of evaluation methodologies and tools from an internal review perspective by the actors themselves and from the external perspective; • Knowledge of gender equity principles and conflict sensitive program management; • Good management in facilitation – systematization of workshops and management of evaluation tools; • Excellent analytical and synthesis skills; • Excellent command (oral and written) of Spanish and English; • At least one person should have extensive experience in leading evaluation teams in case the evaluation is conducted by more than one person;