Consultancy – Technical support for the restructuring of the Asmae Monitoring-Evaluation System At Asmae

I. Presentation of the organization

Asmae – Association Sœur Emmanuelle is an international solidarity organization founded in 1980 working to defend and realize the rights of children and young people. Its action is structured around four main thematic axes:

• Access to quality education;
• Training and professional integration of young people;
• The protection of children from all forms of violence and exploitation and the promotion of their well-being;
• Living together, to enable children and their communities to flourish in a sustainable and peaceful world.

Present in eight countries (Egypt, Lebanon, Burkina Faso, Ivory Coast, Madagascar, Philippines, India and France), Asmae works alongside local partners to guarantee more than 39,000 vulnerable girls and boys opportunities and spaces to learn, train and develop. Asmae’s actions take into account, in a transversal manner, issues related to gender, participation and inclusion. The rights-based approach is also a central axis.

II. Presentation of the Asmae monitoring and evaluation system

In a context of development and strengthening the quality of its actions, Asmae pursues a learning organization approach, promoting the strengthening of team capacities and the structuring of these processes and practices. The strengthening of monitoring and evaluation mechanisms for international projects meets the objectives set by Asmae in its 2025 strategic plan and its Ambition 2030. This involves formalizing procedures and tools to monitor and analyze the proper implementation and performance of the organization’s project and program activities.

As part of external support, supported by the F3E, Asmae requested the support of a consultancy to create a monitoring and evaluation system (DSE), also called an “organizational monitoring system”, a tool capable of measuring the results of projects in a consolidated and systematic manner. Developed between 2020 and 2021, the system makes it possible to collect key information, using predefined indicators (activities, results/changes) linked to Asmae’s intervention themes.

This tool is expected to provide a detailed analysis of the organization’s intervention methods, its major developments and progress, and ultimately the impact that Asmae’s actions have on the groups targeted by the actions. This tool also aims to examine the relevance of activities with Asmae’s strategic objectives and to provide evidence and angles of analysis to better adapt practices. Finally, this tool provides systematic and reliable data that can be easily shared internally and externally.

III. Device format

Asmae’s monitoring and evaluation system is, in practice, a data entry, consolidation and visualization mechanism designed on Microsoft Excel and hosted on SharePoint. It currently consists of 8 Excel files:

• 5 files corresponding to the 5 countries of intervention of Asmae on the date of creation of the tool: Burkina Faso, Egypt, Lebanon, Madagascar and the Philippines;
• 2 files that can be used for data entry for two other countries (it is planned to configure these files for India and Ivory Coast);
• 1 “international” file allowing the aggregation and visualization of all data entered in the countries of intervention.

The 7 Excel “country” files are composed of 5 project tabs. The number of active tabs (inactive tabs are hidden) depends on the number of projects set up in the file. The project tabs contain a list of indicators selected with the country office implementing the project.

These files all follow the same format, with pre-created tabs and already defined rows and columns (with formulas included to perform calculations, color-code certain cells, perform checks, etc.). As with tabs, rows and columns not relevant to a specific project are hidden.

These files also have a summary tab, which links to a series of pivot tables, and which consists of several graphs facilitating the visualization of the data.

Finally, these files are linked to the “international” Excel file. This latter file only has a summary tab, in the same format as the summary tabs of the country files (and connected to a series of pivot tables). In order for the data from the “country” files to be able to go back to the “international” file, a series of computer manipulations is necessary to update the data in all the data entry and summary tabs of all the files.

The system is accompanied by a user manual (“Guide to the Asmae organizational monitoring system”) and several annexes providing details on data entry, configuration of country tabs, addition/deletion of indicators, organization of data audits and use of the international file and part of data updating.

In order to review and develop the tool, to reflect certain changes (thematic framework) as well as improve certain modalities of the system, Asmae would like to benefit from new support.

IV. Findings: limits of the system and projects

The device has several limitations. These limitations, and their related projects, have been identified as follows:

1/ Fragility of file security as well as links between country files and international files:

Passwords protect the data entry tabs. One of these passwords, protecting a project tab on the Philippines country file, was inadvertently changed. It is now impossible to reconfigure the tab in question and to proceed with the refresh of the indicator selection.

The data update in the “country” files and the “international” file is carried out annually. On several occasions in recent years, synchronization could not be done due to non-identification of the data sources.

In order to be able to synchronize the data in the “international” file, it is necessary to open all Excel files at the same time so that the connections are active. It has already happened that the Excel software closes unexpectedly due to an overload of the computer.

Finally, hosting Excel files on SharePoint allows all Asmae teams to easily access the country file they are responsible for. Some difficulties were encountered when several people logged into a file at the same time. A copy of the file is then automatically created on SharePoint. There is a risk of data loss when this happens, because the wrong file, not containing the right updates, can be deleted, or it is necessary to integrate the data from the copy into the original file.

Expectations : Passwords for protecting input tabs are reconfigured. File links are strengthened/secured. Synchronization no longer requires opening all source files. Excel files can only be opened by one person at a time.

2/ Modification of the indicators and the structuring of these indicators:

There are currently 32 indicators (excluding disaggregations and composite indicators). Work has begun internally to update this list of available indicators, in light of several findings: 1/ need to reformulate the titles of certain indicators; 2/ need to delete certain indicators; 3/ need to add additional indicators.

In the same vein, the current DSE indicators are structured on the basis of the old thematic intervention framework (modified since 2022, reduced from 5 to 4 themes).

Expectations : These changes (indicators and thematic structuring) are reflected in the system.

3/ Customization of files 6 and 7:

Files 6 and 7 are called “Pays6” and “Pays7” respectively and will eventually concern Côte d’Ivoire and India. A modification to update their titles would be necessary as well as a check of the status of their connection to the “international” file. The tutorials initially produced recommend not to modify the names of these files; no other tutorial explains, for the moment, the procedure to follow to make these modifications internally.

Expectations : Files 6 and 7 are renamed, links with the international file are ensured and a tutorial explaining the steps and actions to follow is produced.

4/ System extension:

The maximum number of country files is 7. However, the 2025 strategic plan aims to open a mission in an 8th international country.

Furthermore, the maximum number of project tabs that can be contained in a country file is 5. However, this limit has already been reached or is in the process of being reached.

The maximum duration of a project is 5 years. However, many projects have several phases – often 3 years. The current structure of the DSE will not allow data to be entered for the last year of this second phase, nor the third phase.

Finally, to allow better visualization of the data, it is envisaged to have: a/ the addition of a tab including a table for consolidating quantitative data; b/ to be able to have, for certain specific indicators, a space for entering narrative information.

Expectations : The creation of an 8th country file and the addition of project tabs by country are carried out. In addition, the production of a tutorial will ensure the internal configuration. The duration of the projects is extended. A table listing the main data as well as a space for entering narrative data for certain indicators are available.

V. Support requested

This request aims to support Asmae in improving its monitoring and evaluation system, through the following support: –

• Based on the list of modifications proposed above and the flaws identified during the consultancy, if any:
  • Make the necessary changes to all DSE files
  • Review/update the DSE usage annexes so that they are consistent with the changes made
  • Writing new tutorials
• Deliverables: Updated device; DSE usage documents updated on this basis and new documents produced (tutorials mentioned above); summary consultancy report including the list of modifications made to the DSE and the modifications to the usage documents.

VI. Expertise sought

This support is expected to be conducted by a consultant meeting the following prerequisites:

• Solid knowledge in the installation, management and maintenance of computer systems;
• Solid knowledge of Windows operating systems and Microsoft Office applications (Excel);
• Knowledge of databases, particularly those that can be used in the international solidarity sector;
• Proven experience in the development and international solidarity sector