Consultant: Development of a Toolkit for Measuring the Effectiveness of Financial Education Programs

1. Background

The Alliance for Financial Inclusion

The Alliance for Financial Inclusion (AFI) is an innovative, member-driven organization that enables policymakers in developing countries to share their knowledge of financial inclusion policies that have delivered positive tangible results. The network currently comprises over 110 central banks, government ministries and other financial policymaking institutions from about 95 emerging markets and developing countries.

By providing members with the opportunity for peer learning and supporting technical level knowledge exchange on key aspects of financial inclusion policy, AFI works with its members to influence policy reforms, which result in expanding the reach of quality financial services to low-income populations and disadvantaged groups.

AFI Consumer Empowerment and Market Conduct Working Group

Since its launch in April 2011, the Consumer Empowerment and Market Conduct Working Group (CEMCWG) examines how consumer empowerment and protection can help to secure access to financial services and improve the quality of these services. Approaches taken by the CEMC Working Group to achieve its objectives include promoting transparency and disclosure, encouraging effective sales and marketing practices, promoting the harmonization of international initiatives, creating avenues for help and redress and financial education.

Existing CEMCWG publications on Financial Education include:

• 2013: Guideline Note 8 – Consumer Empowerment and Market Conduct – National Financial Education Strategies (Link)
• 2017: Financial Capability Barometer (Link)
• 2018: The Effectiveness of Short-Term Financial Education Workshops in Rural Areas: The Case of Armenia (Link)
• 2021: Digital financial Literacy Guideline Note (Link)
• Soon to be published: NFES Guideline Note and Toolkit, DFL Toolkit

64 Country Policy Profiles of CEMC are available on the AFI Data Portal, and 77% of these member countries, the regulator is involved in consumer education or information dissemination in relation to financial consumer protection (2019 update). Several examples of Financial Literacy and Financial Education Policy Changes reported to AFI by AFI Members, which could be associated with higher account ownership, included i) an impact assessment of financial education strategies (Peru), ii) the development of a natural framework for financial literacy and a financial education and capacity building pilot programs (Kyrgyzstan), and iii) a program on childhood financial literacy and education launched by Banco Central Timor Leste (the Central Bank of Timor Leste).**Project Background**

Financial education and financial inclusion

Financial education is defined in AFI as the process of providing people with the knowledge, skills, attitude, and exposure through access to relevant objective information, and training to enable them to make informed financial decisions and take actions appropriate to their circumstances. Financial literacy is the awareness & knowledge of financial concepts and products needed for managing personal finances, per one’s economic and social circumstances. Financial capability is the individual’s confidence and competencies in financial knowledge, skills and behaviours needed to undertake informed and confident decisions and actions about personal and household finances to improve one’s financial well-being. The field of financial education covers budgetary education (personal or family budget), learning financial tools (daily banking, savings, insurance), as well as understanding economic concepts (operation and financing of the economy) and public policies. Effective and well-guided financial education should contribute to the development of more stable, more competitive, stronger, more shock-resistant, more transparent and more competitive financial markets, as well as a more resilient and prosperous society where vulnerable groups are better served and more financially included.

Drivers for financial education:

• Consumers are bearing more risk for their own financial wellbeing;

• The amount and complexity of financial products is increasing;

• Financial resilience is low, problem debt is increasing;

• Levels of financial literacy and motivation are low;

• Widespread failure to plan for the long term;

• Particular groups are at risk: women, youth, elderly, those living with a disability, those living is rural or remote areas and forcibly displaced persons.

Focus on vulnerable groups
It is important to refine and prioritize the key target groups when developing financial literacy/education programs. These vulnerable market segments may include women, youth, forcibly displaced persons, those living with a disability, older persons, and the rural/remote population. For example, research shows that access to and usage of financial services is highly unequal between women and men, especially women living in rural areas, being the least served by existing financial institutions. Financial education programs for women are a critical investment in women’s financial inclusion. Despite the efforts being taken by many AFI members to improve financial literacy and financial inclusion through development of policies and other financial inclusion frameworks, there is a significant opportunity for more members to specifically focus on increasing women’s financial inclusion through their NFES. Additionally, young populations should be provided with the financial education from an early age in order to be prepared for managing their finances independently and making informed and responsible decisions upon graduating from school1. Financial education can also help forcibly displaced persons (FDPs) “to become familiar with the new financial landscape that they find themselves in, understand the relevant terminology and technology and gain confidence when interacting with financial service providers.”2

Financial Education Theory of Change

A theory of change can be used to evaluate the results of literacy/education programs. As such, it is important to integrate key performance indicators (KPIs) for financial education in the intermediate and long-term outcomes (e.g. change in behaviours and overall financial well-being) of a TOC to thus capture the effectiveness and impact of the programme on key target groups. KPIs should also align with the key learning objectives in the curriculum and should be SMART:

• Specific to programmatic goals – measures changes in knowledge, skills, attitudes or behaviors
• Measurable – aligned with outputs and outcomes
• Attainable – given the constraints of existing infrastructure and stakeholder capacity
• Relevant – to key target groups, trainers and implementing partners
• Time-bound – based on the action plan and goals

Measuring the effectiveness of financial literacy/education programs

Gender sensitive monitoring and evaluation is essential for ensuring financial literacy/education programs are effective across all target groups and for identifying areas of improvement. Evaluation results can be used to identify the most efficient programs and influence future decisions. Financial education programs must set gender-sensitive targets and measure gender relevant results in order to make improvements and support an ongoing case for financial education.

Financial literacy/education programs and initiatives have some unique characteristics that should be considered when designing and implementing an evaluation:

• Translating increased financial knowledge to actual behaviour changes
• Obtaining resources to demonstrate program impacts may be challenging
• Understanding the financial realities of program participants
• Recording emphasis on remedial efforts
• Understanding the time sensitivity of program impacts

Monitoring and evaluation should be grounded in a logic model or theory of change (TOC) that links implementation, with results that incorporates a gender lens. Monitoring measures the inputs, activities, outputs and intermediate outcomes of TOC (cf. above table).

AFI member institutions seek various objectives, such as 1) the definition of specific gender-sensitive indicators and results framework to guide the monitoring of the financial education at a strategic level; 2) the implementation of specific monitoring process at a programme / initiative level, based on their expected outcomes in attitude and behavioural changes. Therefore, an evaluation process definition, including the institutional setup and skills required, as well as the scope of the evaluation depending on the programmes is another issue to be considered by the Toolkit.

2. Objective

The Consumer Empowerment and Market Conduct Working Group (CEMCWG) identified the need to initiate a comprehensive Toolkit to support AFI member institutions to analyze the main components, requirements, structure and practical tools to develop and implement a complete monitoring and evaluation framework of financial education4 programs. These include the identification of gender sensitive measurement tools to track the progress, effectiveness, efficiency, and eventually the impact of different financial education programs and resources. This is to review, guide and support the implementation of these initiatives. National financial education programs have the potential to promote economic empowerment of the population, particularly for the most vulnerable groups, but there is an investment in time and resources that requires to be assessed to ensure that the programmes are creating a positive impact on the attitudes, behaviour, knowledge and skills of the intended beneficiaries.

Specifically, the Toolkit will:

1. Tackle the importance of measuring financial literacy/education/capability programs, understanding of the customers’ levels and needs, and, hence, the impact of financial education efforts towards financial inclusion.
2. Showcase AFI member countries’ experiences and good practices in financial education programmes and gender sensitive measurement.
3. Analyze how measurement can support efficient financial literacy/education programs help un(under)served populations, such as women, youth, forcibly displaced, people, those living with a disability and those living is rural or remote areas, transition into the formal financial sector.
4. Examine innovative measurement tools and models, their benefits and expected results.
5. Provide guidance on how to develop or structure monitoring and evaluation for financial education programme.

The Toolkit will also form the basis for a number of capacity building activities, further development of related gender sensitive knowledge products such as the Financial Education Toolkit, and in country implementation support aimed at supporting members to meet their Denarau Action Plan commitments and reduce/eliminate their gender gap.

3. Scope Of Work

This assignment is expected to involve extensive desk research and key informant interviews. The consultant will be supported and advised by the AFI Management Unit throughout the assignment and will have access to AFI’s extensive knowledge database related to and for the purposes of this assignment. AFI will facilitate and arrange any interviews with members and partners. There is no travel associated with this assignment.

Key Activities

The appointed consultant is required to:

• Submit a methodology proposal.
• Inception meeting: preliminary consultation on the scope, expected deliverable and related timeline.
• Conduct a comprehensive review of all AFI products and services related to financial literacy/education/capability, as well as monitoring and evaluation, thus complementing the existing knowledge products and link/map these with member-reported evidence/case study.
• Submit the final draft version of the deliverable by 27 August 2021;
• Including the reviews and comments from the CEMCWG members, submit the final version of the deliverable by 20 September 2021.

Additional information:

Branding and Communications

The Toolkit will be branded with the logos of AFI and Consumer Empowerment and Market Conduct Working Group (CEMCWG) on the front page of the document.

4. Consultant Expertise

The consultant undertaking this assignment should have the following qualifications:

• Advanced degree in a field related to Economics, Public Policy, International Development or another related discipline.
• Knowledge and experience in financial literacy / education / capability programmes; innovative measurement tools and models, their benefits and expected results; especially in the area of women, youth, forcibly displaced, people, those living with a disability and those living is rural or remote areas, transition into the formal financial sector.
• Fluency in English (oral and writing) is compulsory.
• Writing experience; submit related working papers or research around the topic.

5. Timeline

• Metholody submission proposal (July 2021)
• Start the project (July 2021)
• Submit the 1^st Draft for review (by 30 July)
• AFI MU review (By 16 August)
• Submit the final draft (by 27 August)
• Document commented by CEMCWG members (by 15 Sept)
• Finalizing document (by 20 Sept)

6. Reporting

The consultant will work closely with the Policy Programmes and Regional Initiatives (PPRI) Unit and with the AFI’s Consumer Empowerment and Market Conduct Unit, and other key staff. The consultant will report directly to Head of the PPRI Unit / CEMC Policy Manager.

7. Travel

This consultancy work is home-based work. No travelling is required.

8. Criteria Of Evaluation

The proposals submitted will be evaluated based on the following criteria:

1. Academic Qualification; 10%
2. Experience and technical competence of the key staff for the assignment; 50%
3. Adequacy for the assignment (financial literacy/education/capability programme) ; 20%
4. Measurement Toolkits and models and analysis experience; 30%
5. Adequacy of the proposed work plan and methodology in responding to the Terms of Reference; 20%
6. Technical approach and methodology; 10**%**
7. Workplan; 10**%**
   1. Sample work – Relevance to Assignments and demonstrated experience in writing ; 20%

Total : 100%