The ‘Monitoring and Evaluation of Brazilian South-South and Trilateral Cooperation (SSTC) on capacity development and social protection: learnings from the field’ webinar took place on 24 October 2019. The event was organised by the Articulação Sul (ASUL). The session was dedicated to the monitoring and evaluation systems in the context of Brazilian South-South and Triangular Cooperation with an emphasis on capacity development in social protection.

The session was moderated by Laura T. Waisbich (Articulação Sul, ASUL), who was joined by speakers Guilherme Dantas Nogueira (Brazilian Cooperation Agency, ABC), Niklas Stephan (United Nations Children’s Fund, UNICEF Brazil), and Mariana Rocha (Centre of Excellence Against Hunger, World Food Programme, WFP).

You can watch the webinar recording here and access the webinar presentation here.

 

The importance of Monitoring and Evaluation to South-South and Trilateral Cooperation (SSTC)

The concept of South-South Cooperation (SSC) first appeared on the Buenos Aires Plan of Action Conference (BAPA) in 1978, which defined the main characteristics of horizontal cooperation between countries of the global south. This strategy was based on the sharing of good policy practices in the developing world and capacity building, aiming to contribute to the establishment of development agendas across countries.

The Brazilian Cooperation Agency (ABC) primarily acts on SSC initiatives, having established the concept of trilateral cooperation in 2008, expanding cooperation practices to international organisations in addition to governments. Trilateral cooperation bases itself on the equal participation of all parties involved, working together to tackle specific issues on countries that compose the global south. Although the countries remain the most central actors in sharing their experiences, international organizations provide technical expertise in programme design and implementation.

The process of knowledge-sharing implies in describing each step of the process and their subsequent evaluation. There are two different aspects of conducting evaluations, which occur in two different moments:

  1. Evaluation of the work itself, taking into account the process of establishing the initiative;
  2. Evaluation of the results.

In order to thoroughly analyze the evaluations, the  ABC refers to external consultants to ensure that the evaluation was conducted correctly in an unbiased manner, outsourcing this process to other institutions, such as the Articulação Sul.

Such partnerships allowed for the use of innovative methodologies and new evaluation tools.  The ABC is currently working on the development of a new methodological system, consisting in the:

  1. Establishment of a methodology of work;
  2. Creation of a computer-based data management system;
  3. Conceptualizing a precise and time-efficient monitoring and evaluation process.

Despite being an essential process to ensure project success, conducting an appropriate evaluation is an arduous task, given that it is an ongoing process. It allows for precise sharing of results, and consequently, for efficient capacity development.

 

South-South Cooperation and Monitoring & Evaluation – Lessons from UNICEF Brazil

UNICEF defines South-South Cooperation as a cross-cutting implementation strategy and a means of pursuing human rights and development results for children and women, through a results-focused exchange of knowledge, skills, resources, including technology and information between and among stakeholders and partners in developing countries. Thus, as per the UNICEF Strategic Plan (2018-21) SSC is not defined as a programme with a means to its end, but rather as a strategy by which UNICEF can fulfil its mandate, bringing about results for children, especially the most vulnerable.

The definitions of capacity development evolve from three levels of capacity. One related to an enabling environment which comprises policies, legislation, budgets etc. An organizational level which contains the procedures and frameworks for action and, finally, an individual level in which skills, knowledge and experiences are present. All these levels co-exist and interact throughout a cooperation process.

 

Monitoring and Evaluation of Brazilian South-South and Trilateral Cooperation (SSTC) on capacity development and social protection: learnings from the field

The ‘Monitoring and Evaluation of Brazilian South-South and Trilateral Cooperation (SSTC) on capacity development and social protection: learnings from the field’ webinar took place on 24 October 2019. The event was organised by the Articulação Sul (ASUL). The session was dedicated to the monitoring and evaluation systems in the context of Brazilian South-South and Triangular Cooperation with an emphasis on capacity development in social protection.

The session was moderated by Laura T. Waisbich (Articulação Sul, ASUL), who was joined by speakers Guilherme Dantas Nogueira (Brazilian Cooperation Agency, ABC), Niklas Stephan (United Nations Children’s Fund, UNICEF Brazil), and Mariana Rocha (Centre of Excellence Against Hunger, World Food Programme, WFP).

You can watch the webinar recording here and access the webinar presentation here.

 

The importance of Monitoring and Evaluation to South-South and Trilateral Cooperation (SSTC)

The concept of South-South Cooperation (SSC) first appeared on the Buenos Aires Plan of Action Conference (BAPA) in 1978, which defined the main characteristics of horizontal cooperation between countries of the global south. This strategy was based on the sharing of good policy practices in the developing world and capacity building, aiming to contribute to the establishment of development agendas across countries.

The Brazilian Cooperation Agency (ABC) primarily acts on SSC initiatives, having established the concept of trilateral cooperation in 2008, expanding cooperation practices to international organisations in addition to governments. Trilateral cooperation bases itself on the equal participation of all parties involved, working together to tackle specific issues on countries that compose the global south. Although the countries remain the most central actors in sharing their experiences, international organizations provide technical expertise in programme design and implementation.

The process of knowledge-sharing implies in describing each step of the process and their subsequent evaluation. There are two different aspects of conducting evaluations, which occur in two different moments:

  1. Evaluation of the work itself, taking into account the process of establishing the initiative;
  2. Evaluation of the results.

In order to thoroughly analyze the evaluations, the  ABC refers to external consultants to ensure that the evaluation was conducted correctly in an unbiased manner, outsourcing this process to other institutions, such as the Articulação Sul.

Such partnerships allowed for the use of innovative methodologies and new evaluation tools. The ABC is currently working on the development of a new methodological system, consisting in the:

  1. Establishment of a methodology of work;
  2. Creation of a computer-based data management system;
  3. Conceptualizing precise and time-efficient monitoring and evaluation process.

Despite being an essential process to ensure project success, conducting an appropriate evaluation is an arduous task, given that it is an ongoing process. It allows for precise sharing of results, and consequently, for efficient capacity development.

 

South-South Cooperation and Monitoring & Evaluation – Lessons from UNICEF Brazil

UNICEF defines South-South Cooperation as a cross-cutting implementation strategy and a means of pursuing human rights and development results for children and women, through a results-focused exchange of knowledge, skills, resources, including technology and information between and among stakeholders and partners in developing countries. Thus, as per the UNICEF Strategic Plan (2018-21) SSC is not defined as a programme with a means to its end, but rather as a strategy by which UNICEF can fulfil its mandate, bringing about results for children, especially the most vulnerable.

The definitions of capacity development evolve from three levels of capacity. One related to an enabling environment which comprises policies, legislation, budgets etc. An organizational level which contains the procedures and frameworks for action and, finally, an individual level in which skills, knowledge and experiences are present. All these levels co-exist and interact throughout a cooperation process.

UNICEF follows the UNEG (United Nations Evaluation Group)’s definition of evaluation as ‘assessment, conducted as systematically and impartially as possible, of an activity, project, programme, strategy, policy, topic, theme, sector, operational area or institutional performance’. Evaluation is in the core of UNICEF’s activities having a specific office designed for this purpose. At a programme level, monitoring and evaluation activities are addressed by individuals exclusively assigned to this task.

M&E is a key mean to understanding expected and unexpected achievements. It works by promoting learning and decision-making, accountability and supporting the development of national evaluation capacity.

In the UNICEF Brazil programme set up and office structure, SSC forms part of the Social Policy and Monitoring & Evaluation division. The SSC programme outline entails policy dialogues, technical cooperation and knowledge management.

 

In 2012, UNICEF agreed with the government of Brazil to work together for attending the requests from other countries through arrangements of Trilateral South-South Cooperation (TSSC). Many of these requests were social protection related. A MoU and a programme set-up defined the lines of action, rules and responsibilities for each actor involved.

In 2015, programmatic and operational guidelines were developed jointly by ABC and UNICEF Brazil as to better inform and structure TSSC demands as well as the design of TSSC initiatives. These guidelines were revised and updated in 2017, in line with the Sustainable Development Goals.

To better comprehend the changes that SSC brings for children UNICEF Brazil and ABC are currently developing a MEL (Monitoring, Evaluation and Learning) system. So far, a conceptual framework has been developed, which includes the theory of change and results from the matrix of the overall Brazil-UNICEF TSSC programme. Building on already existing instruments, such as the TSSC Guidelines, expectations are that this system - once fully developed - will help gather and systematize information throughout programme implementation and thus provide inputs for improved monitoring and evaluation of the ABC-UNICEF TSSC initiatives as well as institutional learning.

In parallel, an external evaluation os ongoing, covering the scope of 8 years of implementation of the Brazil-UNICEF TSSC programme across 16 countries. UNICEF Brazil is the first country office to have a programme together with a government in SSC. Regarding the next steps, UNICEF and ABC will continue the development of a functional MEL system and will also be finalizing the external evaluation of the programme (due to be finalized by the end of 2019), which in turn will inform the future TSSC programme design.

 

M&E of South-South Cooperation: the experience of the WFP Centre of Excellence Against Hunger (CEAH) in Brazil.

The Centre of Excellence Against Hunger was created in 2011. It was an initiative of the Brazilian government who decided to establish a partnership with a United Nations agency to accelerate its trilateral cooperation efforts.

The centre supports countries in developing their national food and nutrition security policies by focusing on capacity development. This is done through south-south exchanges which primarily between Brazil and some African and Asian countries. The CEAH operations are guided trough the SSC principles of horizontal partnership, national ownership, recognition and mutual benefit.

So far 30 governments engaged to advance its national policy framework and 18 countries developed new national policies where the school feeding programmes were led by national- ministries rather than an International Organization.

Low and Middle-income countries are increasingly interested in establishing similar centres, which led WFP’s to open two more CEAH’s offices around the world - one in China and another in Cote D`Ivoire. The centres contribute to national school feeding programmes and its M&E system has had a great influence in this process.

M&E systems building were largely influenced by the different structure the centre assumed in relation to other WFP offices, by the learning process which was greatly based on practical experiences and by its operations been strongly influenced by the Brazilian government’s approach to SSC.

In 2016 the first impact evaluation from the first eight years of activity revealed the importance of strengthening the MEL systems by developing stronger management systems and a clearer theory of change that could go beyond the logical framework previously established.

The impact evaluation was an important process to learn about information gaps in the centre work. The results captured by the evaluation contributed to inspiring ownership and supporting domestic coalitions for change, strengthening capacity, fostering an enabling environment and contributing to an international development landscape, by collaborating to the Brazilian SSC approach and national programme.

In terms of use, M&E tools have been allocated for programme management purposes such as key outputs indicators, for accountability and learning purposes.

Within the challenges faced by WFP CEAH some were:

  • The fast-paced environment in which the cooperation took place had many requests for cooperation, consuming time allocation;
  • When the centre was created SSC and TSSC was in a relatively early stage, so were the M&E framework for this process;
  • Not rarely projects do not have significant resources to allocate for M&E activities;
  • A change in focus from study visits to in-country activities with objectives for data collection and monitoring;
  • Data collection was challenging since it is performed by partners and, most frequently, they do not have enough time to precisely collect the data the centre needs;
  • Capturing impact: developing policy frameworks and improving national programmes systems are time demanding activities, sometimes lasting longer than the duration of a specific project;
  • Attribution vs. Contribution: SSC establishes a more horizontal relation, hence, prone to work towards contribution, justification and analyses contribution instead of focusing efforts on developing attribution;
  • Ownership by strategic partners of evaluation results;

A strong theory of change, development of corporate indicators, proper allocation of resources to create solid monitoring systems with human resourced dedicated to data collection and management and adequate dissemination of evaluations results are some of the main lessons learned from WFP CEAH’s experiences from South-South cooperation.

Today, the centre looks to enhance its information management systems, conduct focused research with emphasis on qualitative indicators, expand its indicator base aiming at better integration with the Brazilian Government SSC indicators and work on continuous engagement in exchanges on SSC M&E with other UN agencies.

Q&A session

The webinar was closed with a highly interactive Q&A session, which can be accessed here.

Social Protection Programmes: 
  • All programmes
Social Protection Topics: 
  • Monitoring and evaluation
Countries: 
  • Global
Regions: 
  • Global
The views presented here are the author's and not socialprotection.org's