Assessing the performance of Social Protection Systems: Experiences using the Core Diagnostics Instrument (CODI) in Lebanon, the Philippines and Guatemala

The “Assessing the performance of Social Protection Systems: Experiences using the Core Diagnostics Instrument (CODI) in Lebanon, the Philippines and Guatemala” webinar took place on 13 June 2019. The event was organized by the Inter Agency Social Protection Assessment (ISPA) and it explored the process of using CODI in different countries. Panelists shared their unique experience using and adapting the tool to fit their country context, steps, methodology, challenges, and impacts.

The session was moderated by Ana Veronica Lopez (Social Protection Consultant, World Bank), who was joined by speakers Ana Cíntron (Independent Consultant), Francesca Bastagli (Head of Programme - Social Protection and Social Policy, ODI), Jorge Avalos (Social Protection and Jobs Consultant, World Bank) and Veronika Wodsak (Social Protection Policy Specialist, ILO).

 

You can watch the webinar recording here and access the webinar presentation here.

 

What is the Core Diagnostic Instrument (CODI)?

Veronika Wodsak opened the webinar introducing the tool and its key features. The Core Diagnostic Instrument (CODI) is an ISPA tool, which provides a framework for a comprehensive assessment of the entire social protection system. Its main objectives are:

• To map the key elements of a social protection system in a given country, including national objectives, strategies, policies, programs, and schemes;
• To analyze social protection system performance against national social protection objectives and track progress against a standardized set of performance criteria over time;
• To serve as an evidence base for country dialogue and identify a set of entry-level policy reform options; and
• To promote exchange and coordination between national and international partners.

 

Countries can use CODI to systematically take stock of their social protection provisions, assess their social protection systems, and identify ways to improve system performance.

In terms of the analytical framework, CODI indicates three levels of analysis: policy, program design, and program implementation. Each level presents key aspects that need to be taken into consideration when looking at a system’s performance.

 

 

These key areas are then assessed against ten performance criteria: 1. Inclusiveness, 2. Adequacy, 3. Appropriateness, 4. Respect for Rights and Dignity, 5. Governance and Institutional Capacity, 6. Financial and Fiscal Sustainability, 7. Coherence and Integration, 8. Responsiveness, 9. Cost-Effectiveness, and 10. Incentive Compatibility.

 

Experiences from Lebanon, the Philippines, and Guatemala

Following the introduction to CODI and its main aspects, the panelists were invited to respond to questions related to their experience implementing CODI in three different countries. Francesca Bastagli provided information about Lebanon’s use of CODI, Ana Cintrón, talked about the implementation of the tool in Guatemala, and Jorge Avalos provided key insights about the Philippines’s experience.

 

Why was CODI used? How was the tool adapted to fit the country context?

Due to the ongoing crisis in Syria, Lebanon has seen a big influx of Syrian refugees, while simultaneously dealing with a critical scenario of economic downturn. Currently, the country is characterized by a low provision of social assistance and insurance for workers. A study with UNICEF and the Ministry of Social Affairs for mapping social assistance in the country was commissioned. The study required the usage of the CODI tool, which was then adapted, to become better aligned with the Lebanese socioeconomic context.

In Guatemala, the tool was first requested by the Minister of Social Development to UNDP to harvest data on the social protection programs in the country. CODI had to be adapted for the country context to make it possible to come up with a functioning framework.

CODI was piloted for the first time in the Philippines, at a time when the country was suffering the devastating consequences of Typhoon Haiyan. The tool was employed to build a framework of the situation and then develop an action plan.

 

Who were the key stakeholders and what was the composition of the assessment team? What was the process of identifying it?

In Lebanon, the composition of the assessment team, UNICEF and the Ministry of Social Affairs were the main stakeholders. The team was mainly composed of two social policy specialists from ODI. In terms of the process, it was critical to get as many institutions and partners involved in the implementation of the CODI. When it came to implement and reach the completion of the study, the review process stimulated and triggered important discussions.

The main stakeholders involved in the Guatemalan experience were the Ministry of Social Development, the Planning Secretary, and the Minister of Finance, which were responsible for providing information. Additionally, the process involved United Nations agencies and other International Organizations, alongside the local communities. The team was comprised of specialists from several institutions and agencies.

In the Philippines, key stakeholders employing the tool were the World Bank, ILO, European Commission and other NGOs.

What went into the desk review, overall methodology, selection of programs, and data collection? How were the data and information collected used to assess performance?

In Guatemala, the methodology was adapted into six frameworks, consisting of: historical background of the country, legal framework, programmatical framework, functional outline, conclusions and recommendations, and sectioned programs for evaluation. To collect and assess performance, module one was especially useful, which refers to the policy aspects of the analysis. The main source of information was provided by documents produced by the government. There were three programs selected: a cash transfer programme, a school feeding programme and a social security programme concerned with sickness, maternity and accidents.

The overall process in Lebanon was conducted in constant consultation with the government and the other organizations involved. The process resulted in the creation of a poverty and inequality report, presenting a historical overview of social protection in the country, featuring a section linking humanitarian and social protection. From this report, it was possible to identify three key components of the country’s social protection system: Social services, Social care for children and the National Poverty Targeting Programme.

The process of desk review was crucial in developing a literature review and updating data for the questionnaire in the Philippines. The data collection framework was adapted to improve surveys and make it more comprehensible. The programs selected for analysis were agreed upon between the mission team and high-level management and chosen in terms of size and importance. Two social assistance programs, two active-labor market programs, one social insurance and one community-driven development program were evaluated. The questionnaires were used as a guiding tool for the performance assessment. Despite having used reports as a source of information, there was still a need for complementary data.

What were some of the key findings, challenges, and lessons from your experience?
 

To understand the current social protection system of Guatemala, one must consider that the country lacked any social legislation during a period of thirty years.  Furthermore, the process of analysis emphasized that there was no intercommunication amongst existing information systems, which entails that the synergy between these systems was unsatisfactory – a cross-comparison between data present in different systems had to be conducted manually. Furthermore, there was a lack of coordination of information between different levels of government.

 CODI was the starting point of the assessment process in Lebanon. The tool was populated with the available information and with the support of other organizations, government officials and academic experts. Concerning the process, it was absolutely critical to get as many institutions and partners involved in the implementation of the CODI. In terms of implementing and reaching the completion of the study, the review process stimulated and triggered important discussions.

During the process of applying the tool in the Philippines, it was emphasized that a Government’s commitment and ownership is very important. According to this experience, the questionnaires should also be made user-friendly. In terms of lessons and looking into the future, creating an inventory of social protection programs is vital, given that it may be helpful for future implementations of CODI.

 

What was the dissemination process? Was there take-up by governments? What were the results/impact?

In terms of the dissemination process, it was suggested that workshops should be held to make CODI more approachable. Although the study is not yet concluded, all the results presented in the report will be available at the UNDP Guatemala’s website. The report functioned as an initial social protection evaluation of the country.

The CODI tool is currently a part of a social protection study in Lebanon. The outcomes exposed by this analysis should hopefully be incorporated into a broader framework. The results of the endeavor have not yet been disseminated, however, it is expected to be made public soon.

As a result of the process, the Philippines and Social Protection Review Assessment report was released, without the assessment matrix, as there was no time to get feedback from the government on this matter. It has not been disseminated as a CODI report, but rather as a part of a broader study. The outcomes have been good, including some citations in the media, government, and the institutionalization of some of the suggested programs.

 

This blog post is part of the ISPA Tool Series, which brings together the summaries of webinars organised by ISPA on the topic. Please join the Social Protection Payment Delivery Online Community if you are interested in following the most recent discussions on the topic. If you have any thoughts on this webinar summary, we would love to hear from you. Please add your comments below!