Consultancy: Impact Evaluation of JES Projects in Strengthening Criminal Justice in Central America’s Northern Triangle At Justice Education Society

Type: consultancy

Location: remotely

Application Deadline: Monday, June 21^st, 2021

Expected Start Date: ASAP

End Date: Thursday, August 15^th, 2021

I. Justice Education Society

The Justice Education Society of British Columbia (JES) is a non-profit organization with more than 30 years of experience providing public legal education and justice system capacity building in Canada and overseas. In Canada, JES helps British Columbians learn about the justice system and address their legal issues. Internationally, JES has programs in Guatemala, El Salvador, Honduras, Panama and Guyana, working with national institutions to strengthen justice systems, improve citizen security and increase access to justice. JES has offices in Canada, Guatemala, Honduras and Guyana and diverse sources of funding including donors and provincial, federal and overseas governments.

JES’ work contributes primarily to Goals 4 (Quality Education for All), 5 (Gender Equality) and 16 (Peace, Justice & Strong Institutions) of the UN Sustainable Development Goals (SDGs). More information about JES is available at: www.JusticeEducation.ca.

II. Background Information

Since 2009, JES has been actively working in Central America’s Northern Triangle countries, providing technical support and expertise to justice institutions so that they strengthen their capacity to prosecute criminals and improve services for victims. A leader in the justice sector in the region, JES has to date completed seven successful criminal justice projects in Guatemala, Honduras and El Salvador and is currently implementing four projects.

JES’ work mainly focuses on providing technical training and expertise for justice professionals (e.g. law enforcement, security, investigators, analysts, prosecutors, and judges) to strengthen institutional capacity (e.g. developing policies, institutionalizing procedures, incorporating technology, case management systems, and introducing quality assurance) and to provide better access to justice for victims and vulnerable groups.

JES works in close collaboration with local and national institutions by incorporating needs-based program planning, delivering technical assistance with leading subject matter experts, and building capacity through a proven train-the-trainer methodology. Ongoing monitoring, endorsed by all stakeholders, ensure that activities and outputs are captured, and gaps can be addressed on an ongoing basis, with each project progressively building capacity for partner institutions.

JES’ projects support justice system institutions to deliver effective and accessible justice systems. JES’ technical approach and expertise has led to sustainable change in the justice sector in Central America and therefore seeks to conduct an impact evaluation to capture and highlight its key impacts for the past 15 years, while identifying areas of future opportunities.

III. Current and Past Projects

Since 2009, JES has completed 7 projects in the Northern Triangle countries, the majority of which have been funded by the Government of Canada (GAC), through the Anti-Crime Capacity Building Program (ACCBP):

1. DFATD- ACCBP: Special Investigative Methods Project (SMI) (2008-2010) in Guatemala – Project # 2008-274
2. DFATD- ACCBP: Diagnostic: Criminal Investigation Guatemala (2010-2011) Project#2010-115) & Building an Effective Model for Criminal Investigation in Guatemala (2011-2013)- Project #2011-146
3. DFATD- ACCBP: Special Methods Of Investigation (SMI) Program for Central America Project (2011-2013) #2011-145
4. CIDA: Guaranteeing Human Rights by Building Effective Justice Systems in Central America and Africa (2009-2013) – Project #A-034016
5. CIDA: Mayan Traditional Justice Knowledge Exchange (2008-2010) (Canada-Guatemala) Project # A-031825-069-PR1
6. DFATD – ACCBP: Strengthening Anti-Crime Capacity in the Northern Triangle Phase 1 and Phase II (2013-2015) #2013-291
7. DFATD- ACCBP: Combating Gangs and Criminality in Central America (2015-2017) – Project #2014-353

JES is also currently implementing four projects in the region, including the Strengthening Criminal Investigations and Evidence Sharing in Central America’s Northern Triangle Project (2017-2021), funded by Global Affairs Canada – ACCBP and built on JES’ previous projects. The Project focuses on building and improving the capacity of justice and law enforcement institutions and agencies (Police Forces, Public Prosecutors’ Offices, Judiciary and Forensic Science Services) in Guatemala, Honduras and El Salvador.

Other active JES projects in the Northern Triangle region include:

• Swedish International Development Cooperation Agency (Sida): Guatemalan Institutional Justice Strengthening Project (2018-2022)
• GAC: Building a Technological Platform to Support Victim Service in Guatemala (2017-2021)
• GAC bilateral: Justice for Victims of Sexual and Gender-Based Violence in Honduras (2018-2022)

Examples of results

Notable outcomes of JES projects to date include (not exclusively):

• Increased skills and knowledge in the use of tactical and strategic criminal investigation techniques and intelligence analysis (CIA)
• Improved major case management and investigations (MCMI)
• Increased skills, knowledge and use of wire taps in criminal investigations
• Increased skills, knowledge and use forensic video analysis (FVA)
• Improved use of surveillance equipment and techniques
• Increased skills, knowledge and use of DNA kits
• Strengthened capacity in oral trial preparation for prosecutors and judges
• Increased access and capacity to use sophisticated equipment and software to support CIA and specialist investigative techniques
• Improved practices in the protection of justice system personnel
• Increased skills, knowledge, and awareness in sharing of forensic and investigative information, intelligence and evidence between prosecuting and police authorities
• Increase in the region’s capacity to share evidence and intelligence across borders to combat organized crime
• Improved capacity in anti-corruption forensic financial investigations
• Reduced case backlog at the Public Prosecutor’s office and implementation of a digital case management system
• Improved skills and knowledge in the criminal investigations of sexual and gender-based violence and femicides
• Improved access to justice institutions for vulnerable victims of crime
• Greater understanding and application of a gender-responsive and trauma-informed approach within the justice system
• Improved capacity of individual justice operators to fulfill their mandates

IV. Purpose

The purpose of this impact evaluation is to identify, evaluate and present the key results and longer-term impacts that can be attributed to JES’s current and past initiatives in Guatemala, Honduras and El Salvador in strengthening criminal justice over the past 13 years. The evaluation will identify and analyze relevant quantitative and qualitative indicators and data and highlight evidence of key achievements and impacts of JES’ work in the region, while identifying areas and opportunities for greater impact for future programming.

For the purposes of the assignment, impact will be understood as the effects of JES projects and trainings on direct and indirect beneficiaries, including trainees (disaggregated by gender, country, and profession), institutions (disaggregated by country and unit), final beneficiaries and other relevant stakeholders. The impact evaluation will also look at the wider context of the region, including social, economic, technical, and institutional externalities that may have complemented or diminished the impact of programs. Impacts can be direct or indirect, intended or unintended, positive or negative, macro (within the sector) or micro (units, individuals), but should always be medium to long-term in duration.

The results are expected to inform JES about its key impact and achievements in the region and support JES in planning and future programming in the Northern Triangle. Within this context, the impact evaluation should also identify and address the relationship between JES’ impact in the region and broader international indicators related to peace, justice and strong institutions (SDG 16), Gender Equality (SDG 5) and other internationally recognized measures.

Findings and conclusions from this study may be partially or fully shared with the public on JES’ website and social media platforms, and/or with project funders and partners, such as the Global Affairs Canada, Public Prosecutor’s Offices and other stakeholders.

V. Expected Outputs

1. Provide a Preliminary Evaluation Plan detailing the methodology, tasks, time distribution and initial observations, as well as a Desk Research of relevant studies and literature.
2. Compile and analyze quantitative and qualitative data and results from JES project reports.
3. Work with JES staff to collect data through quantitative sources (statistical research, prosecution rates, etc.) as well as qualitative sources (Key Informant Interviews, impact stories and/or case studies).
4. Identify evidence of key achievements and impacts, identify gaps, areas of opportunities and recommendations that can be used for future programming.
5. Provide a Draft and Final Impact Evaluation report relating regional findings to international measures of impact in the sector.

VI. Scope of Work and Methodology

The impact evaluation will focus on JES’ seven past projects and four current projects from 2009 to 2021, carried out in Guatemala, Honduras and El Salvador.

The Consultant(s) will report to the International Project Director and collaborate with the International Program Assistant Director and the Northern Triangle Project Director. Field research will be facilitated by local JES project officers and translators, who will participate in interviews (remotely) of intermediaries and beneficiaries, for which JES will assist in identifying relevant contacts.

The work will be conducted remotely through emails, virtual meetings using Zoom, Google surveys, OpenEdx and any other relevant platform.

The results of the impact evaluation may be shared and discussed with partners and stakeholders to ensure their validity, and the Consultant(s) may be asked to make changes to their final report to ensure that recommended strategies are inclusive and sustainable.

The Consultant(s) will be conduct following:

1. Methodology and Desk review: the Consultant(s) will conduct a desk review and present an evaluation methodology, that includes tasks, time distribution and initial observations within the wider context of social impact and justice. The Consultant(s) will look at similar studies and initiatives that support strengthened and improved justice delivery and identify key indicators, best practices, norms, international standards.
2. Internal project data analysis: the Consultant(s) will examine JES project documents, such as logic models, performance management frameworks, data, final reports, budget, subject matter reports, contribution agreements, project implementation plans, interviews and any other relevant documents, to identify, compile and analyze the relevant indicators, quantitative and qualitative data and results.
3. Primary data collection: the Consultant(s) should engage with JES and its partners to collect and compile additional data, including staff, government agencies and subject matter experts. The main purpose for employing this methodology is to understand the impact of JES first-hand by those involved and contextually from a local perspective. JES will participate in these activities, and can also facilitate access to relevant empirical data.
4. Case studies/impact stories/testimonies: the Consultant(s) may gather testimonies (video or written) and write impact stories or case studies of beneficiaries who successfully completed the training. This will provide further flexibility and depth in presenting qualitative results and impact of JES on beneficiaries and government staff, and the transformation of individual capacity as a result of JES training. Case studies on specific departments or units may also be conducted to assess the impact of JES on institutional and organizational capacities.
5. Submit and present the following reports:
6. Preliminary Evaluation Plan detailing the methodology, tasks, time distribution and initial observations from desk research
7. Draft Impact Evaluation Report detailing observations about JES project impact in the region
8. Final Impact Evaluation Report incorporating JES feedback, and providing recommendations for future programming in the region, and how JES projects can better capture project impact relative to internationally recognized justice and security indicators.

VII. Confidential Information

The Consultant(s) will treat as confidential any information about JES, or the partners and business associates of JES, which may be gathered in the course of this assignment.

The Consultant(s) will not at any time during or after this assignment disclose any Confidential Information to any third party or person whatsoever, unless otherwise directed by JES. The Consultant(s) will not publish anything connected with the results of the work performed without the express written permission of JES. The Consultant(s) will take all reasonable precautions for the protection of Confidential Information.

The Consultant(s) will not discuss the Project with the media or external interest groups or make any reference to the Project on any social media site, unless approved by JES. Any requests by the media or external interest groups for information about the project implementing the Project must be referred to JES’ International Program Director.

VIII. Intellectual Property

All materials developed and any intellectual property rights that are created by the Consultant(s) with respect to this contract will be the property of JES. Any materials provided for work on this Agreement will be returned to JES or deleted, unless otherwise agreed to in writing.

IX. Qualifications

The selected Consultant(s) (maximum 2) must have at minimum the following knowledge, skills, and experience:

• Post-graduate degree in criminology, sociology, economics, or other relevant fields.
• A minimum of 7 years of professional work experience in the areas of program and/or project evaluation, ideally in international development, in the justice sector, and/or in institutional capacity building and strengthening.
• Extensive knowledge of and experience in applying qualitative and quantitative evaluation/research methods. Knowledge of best practices in the field.
• Excellent English communication skills necessary for developing methodology and survey, building rapport with stakeholders, facilitating participation and present results.
• Experience in conducting impact evaluations or assessments in Central America is an asset.
• Experience in conducting impact evaluations or assessments within the justice sector is an asset.
• Ability to communicate in Spanish is an asset.

X. Terms of Payment

The payment shall be paid upon submission of the deliverables in the following phases:

Deliverable

Target Date

Payment milestone

1. Preliminary Evaluation Plan including a des research and detailing the methodology, tasks, time distribution and initial observations. Target Date: 7 July 2021 Payment milestone: 25%
2. Draft final report describing the general observations and preliminary recommendations. Target Date: 31 July 2021 Payment milestone: 25%
3. Final Impact Evaluation report. Target Date: 15 August 2021 Payment milestone: 50%

All payments are subject to review and acceptance by JES.