Targeting in Crises Online Discussion, Tues 7 – Fri 10 June

Welcome to Day One of a week-long online discussion about targeting social assistance in situations of crises, co-facilitated by Rachel Sabates-Wheeler (Institute of Development Studies) and Emily Wylde (Independent Economist).  

This online discussion is the first in a series of four, which will run throughout June, as part of the Better Assistance in Crises (BASIC) Research programme and kindly co-hosted by socialprotection.org. This first e-discussion on targeting will be structured and moderated around four topic areas (one per day), with scope to follow other debates that emerge. For more information see attached e-discussion guide. On Wednesday, we will also have a one-hour informal hangout at 2pm (GMT +1), where we can reflect on the key questions and emergent debates from the discussions.  

The purpose of the discussion is to bring practitioners, researchers, policymakers, and anyone interested to engage and promote a realistic and forward-looking research and policy agenda for targeting social assistance in acute and protracted crises – through BASIC Research and beyond. 

To participate in this e-discussion, please make sure you have registered on the socialprotection.org platform (see instructions in attached guide). You will then be able to add comments/replies to the thread below. If you have any difficulties please reach out to the moderator of the discussion, Tina Nelis, via email at: [email protected]. We kindly ask all participants to be respectful of the views of others.  

 

Tomorrow (Tues 7 June) we will kick off with Topic One – Targeting and Objectives … watch this space!

 

 

 

Comments

Day One/Topic 1: How does targeting enable programme objectives?

Morning all!  Our topic for today is targeting and objectives in crisis settings. There are many issues we could discuss, so please feel free to share and raise any of these including your experience and suggestions.

To kick things off,  here are some of the challenges we have been considering, followed by a question or two to get our conversation started:

In the immediate aftermath of a sudden onset shock, such as an earthquake, objectives are about life-saving and basic needs provision, whereas, in a protracted crisis objectives can be more focused on livelihood promotion.

  • If programme objectives vary according to crisis context, then how should we adapt the way we target, including where on-going social protection programmes operate simultaneously with crisis response?
  • And….does anyone have any examples of where differing objectives/needs have usefully led to different targeting strategies, or where there have been challenges?

Hi all,

Looking forward to a great discussion this week.

Just wanted to come in on these questions and raise the challenge of fluidity (and need for flexibility) in crises. This is the case when people vulnerable to food insecurity or poverty vary with time. For instance, traditionally, the PSNP payments in Ethiopia served chronically food-insecure households. However, those that received payments from the humanitarian response in the 2017/18 crisis were those most affected by the acute shock (not necessarily the most chronic food insecure). The PSNP has also dropped work requirements in response to shocks.

I cant think of many protracted crises where livelihoods promotion is a very realistic prospect. Either because livelihoods are constrained due to status (refugees not allowed to work except illegally at casual labour), IDPs stuck in camps (NE NIgeria) or because of ongoing conflict (DRC, South Sudan, CAR etc). So there are occasiohnally programmes with objectives around resilience (BRiCs in Somalia? ) - but not much evidence of them working. Lewis Sida was working on an evaluation (https://assets.publishing.service.gov.uk/media/57a089e9e5274a27b200030b/... ) which I think found very limited impacts. I can only find the inception report - is there a final version somewhere? 

Is it not also the case that in some crisis contexts (like Yemen) 'routine' social protection systems may have broken down or become severely constrained, so humanitarian or crisis social protection objectives may need to encompass provision of support for groups that would not ordinarily be the focus of livelihood promotion because they are not of active age or do not have labour capacity (older persons, children, people living with disabilities...)?  These groups have different levels and types of need, so one question is whether bespoke emergency programmes are required to support them? Or, if they are to be wrapped up in a more general programme of emergency support, how to ensure such groups are effectively targeted along with everyone else?   

In Ethiopia under PSNP phase 5, the target population (including both the 'direct support' beneficiaries and the public works programme beneficiaries) actually changed from the food insecure to the chronic poor, with then an emergency scale-up facility designed to respond to food insecurity. Getting this new system up and running has been (and is still ongoing) a tremendously complex endeavour for a whole array of reasons - from politics to finances to organisational capacity... - but on paper at least it does provide one example of where, as Rachel put it, differing objectives/needs have led to different targeting strategies.

There is a similar  approach to targeting the 'dual' components - routine support plus emergency support - of a social assistance package in Kenya under the HSNP, which also operates in conditions of a protracted (climate-related) crisis that flares up periodically, hense the need for a system of both routine and emergency support that works in tandem. A new dimension of the challenge today in that context is how to integrate livelihood support packages into that mix, and what the targeting of that should look like. 

Interesting observations and examples, Fred.  On the PSNP and Humanitarian response, i did some work with colleagues (https://www.ifpri.org/publication/expanding-social-protection-coverage-h... ) showing that the charateristics of beneficiaries from the PSNP and HFA are different....and so they should be....in addition to the most poor, an acute crisis it likely to hit those who otherwise would not be poor (pre-shock).  and if the poorest pre-shock are likely to already receiving transfers and support from a safety net programme, then the hope is that they will manage the sudden shock (as is the purpose of a regular and predictable transfer).  Anyway,  if the target populations are diferent for different crises....we should surely expect that differnt targeting mechanisms should be on offer....Speed of response in a sudden and devastaing crisis must be an over riding factor......or not?  - absent up to date registration systems and data, blanket geographic targeting might be the best strategy

Link to Lewis Sida's work on livelihood promotion and resilience can be found: http://www.validevaluations.org/publications and https://odi.org/en/publications/multi-year-humanitarian-funding-a-themat...

 

I'd really like to see some analysis of the comparative efficiency and effectiveness of categorical vs vulnerability or poverty targeting in protracted crises. From dim memory the Somalia Cash Consortium is doing some work comparing categorical lifecycle approaches to vulnerbality approaches to targeting in Somalia.

Some description here - but I'm failing to find the thing I was half remembering. 

 https://devex.shorthandstories.com/building-up-a-social-protection-syste... 

@Paul Harvey, thanks for the link on Somalia, will be interesting to see what comes from that.  Would definitely be great to have some actual evidence on categorical vs. poverty-targeting in these settings, ideally in a like-for-like comparison of, say, X number (or %) of households reached.

One of the things I wonder, which follows a bit from the universal vs poverty-targeted comparison, is even going back a step in terms of first of all have the objectives of targeting been clearly enough articulated to be able to even then assess across a range of performance measures?

While it's not strictly a targeting issue alone, there's obviously a number of househods vs size/frequenncy of transfers inherent in programme budgets in these contexts (particularly ones that are single-year, and which might even be revised downards within-year).  And targeting, as opposed to blanket distribution, often emerges from the reality of limited resources, and needing to balance size of transfer versus coverage. But this also needs to be balanced against other concerns like ease/cost of targeting, social cohesian and perceptions of fairness, etc.

Are there any examples where those kinds of trade-offs across different performance measures are actually articulated?  What are the biggest equity concerns  - is everyone really equally poor/food insecure and equally in need, or are some able to provide better for themselves and therefore less in need?  Can we try to quantify these equity trade-offs, which could help articulate the objectives of targeting (do we need to go narrower/larger transfers, wider/smaller)?  That could then help point to targeting approach (How narrow do categories need to be for categorical? How small a share targeted by poverty approaches? These have huge implications for standard targeting efficiency measures.)

And then how do these equity trade-offs and efficiency performance considerations compare to risks to social cohesion or perceptions?  How are organisations assessing/measuring these other risks?

I don't see discussions like these coming through much (targeting objectives and criteria often seems from the outside to be based on very large assumptions if even articulated at all), but perhaps it's being done within organisations/working groups?  If so, what kind of evidence is used to balance assess these trade-offs, if any? And then are they measuring success, however defined?

 

 

Emily - to add to your list of trade-offs (ease/cost of targeting, social cohesian and perceptions of fairness) - I'd add corruption - the more complex and opaque the targeting system the greater the risks of corruption. And sharing - particularly where local perceptions are that everyone is in need there is often quite widespread sharing of assistance - e.g in South Sudan https://www.ennonline.net/fex/34/community . And taxation.

But there's rarely any good monitoring of the extent of sharing, diversion or post-distribution taxation. If there was I suspect it would strengthen the case for giving more people less and not trying to target too narrowly. 

Hi everyone, great to be participating in this discussion. As this thread specifically mentioned Somalia and we took part in a study of a significant programme in Somalia recently, I thought we should mention a few considerations about targeting in this country. (Because the study is yet to be published, I’m unable to share any links, I’m afraid.) The programme used (sequentially) both geographical and community-based selection mechanisms. While the challenges we observed in this study are specific to this programme in Somalia, they can perhaps be generalised to other contexts in fragile states. Geographical targeting was challenging because of the reliance on outdated population data and inexplicit data on food insecurity and nutrition (main factors considered to select locations). Furthermore, like in many setups, the selection of locations also had to consider other factors such as security, accessibility, and local political economy considerations, which are likely to arbitrarily exclude communities. On the other hand, a key challenge of community-based targeting in such contexts is ensuring recommended processes are adhered. Inevitably, there will be variation across communities in their level of preparedness of the selection process and the way in which the selection is eventually carried out. Understanding of the programme among community leaders, effectiveness of communication channels, and inclusivity of the selection process across communities are key factors in this process.

A few random thoughts based on the threads above from colleagues (aknowledging I do not of course answer your Day 1 questions ;) )!

1. The extent of sharing ex-post (due to the often discussed 'we are all in need') I feel pops up in almost every monitoring report, ex-post evaluation (especially qualitative) that I read and discusison with country practitioners.. but those sometimes never make it online (e.g. most recently monitoring reports I saw in Niger) or are never systematically looked at ACROSS programmes and countries..  The most recent place I have been reading this is a paper based on qualitative research in Chad by seveal World Bank colleagues where the ultimate verdict was to do universal blanket distribution within carefully targeted geographical locations - worth a read!

2. On making trade-offs explicit I am completely with you Emily, and that is partly why the Informal Hangout on SP in Crisis Contexts was pushing for a framing (see the Mural and iterated version below it with inputs from many colleagues) that would help make every single one of those factors explicit.. so choices could be argued and traded off againts all key factors, not just the 'accuracy' dimension (and even if just accuracy, evaluated against what)... and aknowledging any choice will have some negative implications/risks making sure we mitigate those ex-ante to the extent possible. 

3. On targeting design to maximise outcomes of interest (i.e. programme objectives).. there has been  a recent literature wave that has had me worried as it feels like it is coming from a purely academic perspective that does NOT engage with all teh dimensions and tradeoffs just discussed above... see for example this paper on TARGETING IMPACT VERSUS DEPRIVATION ("We ask a different question, namely, whether targeting the most deprived has the greatest social welfare benefit: in particular, are the most deprived those with the largest treatment effects or do the “poorest of the poor” sometimes lack the circumstances and complementary inputs or skills to take full advantage of assistance?") and this one on (MACHINE) LEARNING WHAT GOVERNMENTS VALUE ("This paper develops a method to better understand and align the objectives of a program with the targeting criteria used to implement that program"). Is a utlitaristic and un-transparent approach to targeting a direction we truly want to pursue? Happy to hear other opinions.. 

 

 

Fascinating World Development article - thanks Valentina. Isn't a concern with universal provision within tighter geographies that you swap tensions within communities with tensions between districts? And that could be particularly dangerous in conflicts that have regional and ethnic dimensions? 

Hi all, 

Today was the first day of the Targeting e-discussion in which we considered targeting and programme objectives in crisis contexts. In different acute and protracted crisis situations, programme objectives vary. This means that eligibility and targeting criteria will also vary. Where budgets are constrained, limited resources require a trade-off between the number of recipients reached, and how much they receive. But this is not the only trade-off. Some of the key points and questions that were raised include:  

  • Programme objectives can vary according to the context: whether it is the immediate aftermath of a shock, or a protracted crises situation, different objectives can entail, among others: 
    • Saving lives and providing basic needs 
    • Livelihoods promotion: although some flags that livelihood promotion is not a common objective due to barriers and conflict 
    • Resilience (see work on resilience shared) 
  • Acute crises are likely to hit both pre-shock poor and non-poor. If the poorest pre-shock were already benefiting from safety nets, to what extent should emergency response cover them? In other situations, some routine SP systems in crisis may have broken down or be constrained (no safety nets): to what extent do objectives in emergency situations need to fill in this gap? 
  • Different targeting criteria may entail different targeting strategies. Is it about speed of response through blanket geographic targeting in a sudden devastating crisis with no updated data? Flagged examples included (not without challenges):  
    • PSNP from Ethiopia (changing targeting criteria with a scale-up mode – see work on PSNP link shared) 
    • HSNP from Kenya (targeting dual components of SP + Emergency in tandem) 
  • Before a focus on a comparative analysis of targeting approaches performance, it is important to assess and balance dimensions and trade-offs that go beyond accuracy and include equity aspects relevant to the crisis. These dimensions will form part of the objective and serve to define and measure success and include: social cohesion, perceptions of fairness, corruption, sharing, taxation, etc. A need for more emphasis on making explicit  (see link to Hangout Mural, and link to work on Sharing assistance in South Sudan) and quantifying these trade-offs to support the definition of targeting objectives and approaches was highlighted. This can also help in risk mitigation. For instance, in a context with great concerns of transfer sharing (although not systematically looked at yet), the case for giving more people less and avoiding narrow targeting could be made (Chad example shared). Engaging with these dimensions and trade-offs when aiming to maximise outcomes of interest (i.e. programme objectives) is key. 

Some documents shared: 

We will meet here again tomorrow for the next e-discussion on the criteria for targeting success. Also, don’t forget we will be having the Hangout tomorrow Wednesday at 2 pm UK time to reflect on the key questions and emergent debates from the discussion. Join the hangout tomorrow by following this link: https://socialprotectionorg.zoom.us/j/89060758355  

 

Hello everyone, welcome to Day 2!

Today we’ll be kicking off with our second topic, which follows on from much of the discussion yesterday: What are the criteria for measuring targeting success?

How do we define success - is it purely in terms of reaching those ‘most in need’, and how do we define ’need’ if that’s the case, or do we define it in other ways as well (social cohesion, community acceptability, etc)?  Should criteria be different in different crisis contexts, and what do we know about the costs of inclusion and exclusion?  

  • Where do you see more concern with inclusion versus exclusion errors, and in which contexts (humanitarian vs social protection)?  Do you think the focus is correct? 
  • What are the implications of working with outdated household data when targeting a social protection response to a crisis? All countries (whether crisis-affected or not) face problems with having up to date data, but are there specific implications in crisis situations? Or, does this not matter in practice, since PMT proxies are fairly invariant anyway?

*And do fee free to continue to hit reply on any of yesterday’s discussions above as well.

Let me offer here a few reflections on Lebanon and how the devaluation crisis has rapidly transformed poverty and precarity. 

The most crucial background information is that Lebanon used to operate on a dollar-to-Lebanese pound 'peg,' at 1500 LBP to $1. This peg collapsed as dollar supplies ran out, confidence in the state eroded, and currency speculation was unleashed. Blackmarket rates have significantly varied, sitting now at 27,000 to 1 USD. 

If you have 1) relatives abroad sending remittance, or 2) you are paid in dollars, you are, in some cases doing better than before. But if you are paid in LBP and working in the public sector, you are doing far worse than before, and such situations will, of course, demand targeting that can keep up with rapid changes. What technical mechanisms exist that can respond to situations like this? 

Another common complaint from ordinary Lebanese people regarding targeting 'most in need' is that, right now, "everyone is poor" (82% of the population are living in multi-dimensions poverty, so these sentiments are not unsurprising). I understand, of course, that restricted budgets mean that blanket targeting is impossible, but it feels important to be aware of these sentiments and their risks to cohesion. 

In the Lebanese context, a final impact of exclusion, when combined with state fragmentation, is that several political actors are stepping up to fill the gaps in the absence of a universal rights-based social protection strategy. When thinking of how these groups are targetting, the easy answer would be "giving money to loyalists," and sometimes, it is the case. Still, a crucial nuanced piece of academic work on this subject by Melani Cammett argues that some parties do, in fact, target outside the sect/party for various reasons, like building intraconfessional support and legitimacy. But in such a context – when attempting to launch a non-sectarian universal SP system – what strategies (or best practices) exist to ensure targetting is not captured (or minimally captured) by entrenched interests?  

 

from  Rasmus Schjødt @rasmusjs via Twitter - 

I am looking forward to the first humanitarian cash transfer to e.g. all children or older ppl. Not sure why this is so difficult for humanitarians to imagine.

@Rasmus : i imagine that a cash transfer to ALL children in many humanitarian scenarios would be a transfer to pretty much ALL households - ie, blanket targeting.  I guess one reason it can be hard to imagine is to do with limited budgets!

I am trying to finalise my PhD dissertation and don't actually have time to join the discussion, but couldn't help scrolling through because this is such an interesting and important topic - and noticed that I was actually part of the discussion after all!

A few thoughts:

- The fact that pretty much all households would be reached could also be seen as an advantage, after all blanket targeting is often the starting point/ideal situation. At the same time there are many options for lowering coverage and expenditure if needed, most obviously by lowering age of eligibility (for children - or increasing it, in the case of older people).

- Household targeting can be problematic, because the concept of a 'household' is often problematic - just like the concept of a 'community' for community based targeting. It's often surprisingly difficult to define in practice what a 'household' is and who should belong to it, and 'households' can change (sometimes in response to targeting approaches). So perhaps there could be some advantages to individual targeting, although in the case of children of course it still has to be somehow decided which adult is to receive the money on behalf of the children.

So, I don't think there are any reasons in principle why individual and universal life cycle transfers couldn't be an alternative to targeting 'vulnerable households'. But my twitter comment came out of the experience of suggesting this to a room full of humanitarians (from WFP), who were simply not willing to consider that there could be other options than focusing on vulnerable households and using PMT for targeting, even in a context where it had been demonstrated that it would not make much sense to attempt to identify the most vulnerable households.

On out of date data - Yemen represents a pretty extreme example where social assitance targeting has as it's starting point a pre-war, before 2015 database of people targeting through a PMT exercise. For more detail - https://openknowledge.worldbank.org/handle/10986/35421 . Given widespread displacement and violence it's very unclear how 'invariant' the PMT proxies have remainded over the last 7 years. 

This very much links to some of the issues raised yesterday, especially around targeting performance dimensions that are beyond accuracy and trade-offs at play. Some of these dimensions include legitimacy for example. This has implications for targeting strategies, such as the choice of methods – broader where these risks/trade-offs are higher, and potentially narrower where they may be not so relevant. This also links to the (often) stronger focus on exclusion errors in settings of crises (as opposed to inclusion errors), where the priority may be to ensure that everyone requiring help receives it quickly. This focus is also a concern in communities with mass displacement, where income distribution is flat, or even where there is a lack of confidence in the targeting process (e.g. corruption). This was the case in Turkey with the ESSN targeting criteria, where beneficiaries expressed a preference for a larger number of refugee beneficiaries, even if that meant smaller benefits. Exclusion errors can arise during design and also implementation (e.g. barriers in registration).  Interesting to flag some new efforts around using lotteries in social assistance and the relative success in the different trade-offs that they are having…

On the second question, is this focused on PMT? If crises led to changes in poverty/vulnerability, then out-of-date data will not correctly identify those ‘in need’. In conflict situations, especially those of protracted crisis, it is usual that recent national poverty data do not exist and data are either out of date or not accessible, so using them to create a PMT indicator of eligibility or a poverty line may be problematic if there have been changes since. For instance, the current proposal for use of the PMT in Lebanon’s poverty programme will rely on data from 2011, from before multiple macro-economic and political crises. Outdated data cannot predict newly poor, which ideally requires frequent updates in fluid contexts (where exclusion is more likely). Especially in politically charged situations, using out-of-date data sources can create distrust among different population groups, given risks of not capturing the newly poor. Other types of methods may have lesser data and capacity requirements such as categorical targeting.

Hi all,

Responding to the question: "Where do you see more concern with inclusion versus exclusion errors, and in which contexts (humanitarian vs social protection)?  Do you think the focus is correct?"

My experience (based particularly in sub-Saharan Africa but also parts of South and South East Asia) is that in social protection the focus is mainly on inclusion errors. Resource limitations and sustainability challenges for long term programmes, and the assumption that we are dealing with chronic rather than short-term / acute needs in social protection, mean that inclusion (often also called 'leakage') becomes the predominant concern.  In contrast, humanitarian actors are deeply concerned with exclusion errors - so avoiding situations in which people in (urgent) need for support get left out. This is especially important in conflict-affected settings where exclusion can result in grievances that can trigger violence and in other ways worsen an already unstable situation. As someone who comes from the social protection side, I can understand these different preoccupations, even though the simple 'humanitarian = lifesaving, social protection = not urgent / not lifesaving' is never so clear on the ground.  But I do wonder what I can learn from humanitarian actors about how to successfully get just as much attention paid to exclusion as we do to inclusion in the social protection sector.  Any suggestions out there about how we rebalance the focus in the social protection sector?

Thanks for the discussion!

 

On the 'what are the implications of working with outdated household data when targeting a social protection response to a crisis' question... a few thoughts that pick up on colleagues' too:

  • data needs across different approaches to targeting vary (dha!), the lower these are the better when thinking about shock response.. there are reasons humanitarians keep it simple and actually put a lot more precision-effort intodoing geographic targeting of vulnerability than HH/individual targeting of 'vulnerability'.
    • many categorical 'markers' of vulnerability that are de facto used by humanitarians when they do data-informed 'score-card' approaches to targeting Hhs/individuals rather than community based (drawing on vulnerability needs analysis) are the types of variables that do not change significantly over time so could be held ex-ante and used e.g. disability, chronic illness, gender,... also of course children & household size that do change more (but that can be kept more up to date if drawing data from CRVS etc, thinking about medium-term sustainability of government information systems beyond new data collection each time). 
      • Where post-shock data is necessary on e.g. household damage etc this could be collected via a reduced form that is linked to ex-ante data, as is the case in Chile and several high income countries
    • NOTE: Simple forms of affluence testing (just removing those who are sinificantly better off) could also quite easily be drawing on some ex-ante data and information systems (tax data, social security data, land cadastre data etc), as was often the case in the COVID response (see e.g. this paper)
    • of course PMTs are also designed via proxies of vulnerability .. but often beyond these 'categorical' variables they also include others (housing materials and assets, acces sto services etc, depending on country tehy vary a lot). One of teh key criteria for their choice is relative invariance over time... (though we could argue this is often not the case especially if .g. looking at livestock!). --> However, what makes PMTs most 'risky form an 'up to dateness' perspective  in my opinion is not this... but teh fact they rely on TWO data-sets  not just one: the administrative registration data (often out of date unless linked to on-demand registration of some type of data interoperability with other government databases) + teh sample survey data the econometric analyisis is drawing on (oftn several years old). 'Best practice' e.g. by WB new targeting guidance would be to recalibrate the PMT via new data collection after any given shock, but we know that is almost never the case - for many good reasons including timeliness, cost, feasibility etc. Guidance also stresses PMTs can be designed to be more reliable especially in terms of 'predicting vulnerability' where the sample dataset they are built against includes panel data and/or data collected in different moments of the year to reflect seasonality + sample size enablies calibrating different PMT models for different geographic areas or population groups etc.. which is also rarely the case.... which is why I really struggle to feel that their advantages in terms of 'ex-ante accuracy' are important and meaningful to affected populations visavis all the other factors to be traded off as discussed further up the thread.....
  • Obviously teh type of shock also matters to how up to or out of date (or entirely useless) ex-ante 'prepositioned' data may be.... too long to go into that now but key implications quite obvious as others have stressed

Sorry if that got convoluted...!

I forgot one thing that I have been stressing at teh informal hangout which is broadly related :)

....social protection people sometimes think that humanitarians do better targeting of 'vulnerability' (most often defined as HHs with low coping capacity, high food insecurity etc) because they collect data ex-post and target based e.g on Coping Capacity Index, Food Insecurity Index and so forth. I have even seen efforts to include these questions into social registry questionnaires to make them more 'risk informed'. THIS IS MISLEADING, however...

While these are collected within humanitarian Vulnerabiltty Assessments and similar sample survey exercises that inform GEOGRAPHIC targeting of vulnerability I have not yet found ANY examples* where this type of data is collected at registration stage to inform HH and Individual level targeting. Why? a) Because collecting this data at registration would be too costly and lengthy, b) because answers to those types of questions are easy to 'force' (less observable), c) because answers to these questions change significantly over brief periods of time... etc. In other words, that is why community based processes or categorical markers of vulnerability drawn from recent data (the VNAs) are preferred..

see e.g. 2021 WFP targeting guidance which stresses the following: “Socioeconomic and food security outcome indicators are used to assess vulnerability and food insecurity and to understand the number of people in need and their key characteristics (+ location i.e. geographic targeting), which will inform and validate eligibility criteria. However, these indicators (e.g. food consumption score or those related to coping strategies, income and expenditure) should not be used as the actual eligibility criteria for three reasons: 
o These indicators are used in assessments and surveys on a sample population for a given point in time and are rarely available and up to date for a full population; 
o Even when available, outcome indicators are too dynamic – they fluctuate over time (due to seasonality or household-specific events) as well as with the provision of assistance, which makes verification of beneficiary selection impossible; and 
o They are critical to monitoring and validating the outcomes of targeting decisions. If used as direct eligibility criteria, they could not be used for this important function. Especially if eligibility criteria are communicated to affected populations (in line with good AAP practice), this would likely make households more inclined to underreport on these indicators.”

* I have been told they do exist for small projects run by small NGOs but have not seen any documentation as yet :)

Hi all,

How to measure targeting success? This is the issue that we examined on day 2 of our e-discussion. ‘Success’ is often discussed as aiming to minimise errors, but it encompasses multiple criteria in different crises contexts. Some key thoughts put forward include:

  • A stronger focus on exclusion errors in humanitarian settings and on inclusion errors in stable settings. In conflict contexts, exclusion can result in grievances that may lead to violence and increase instability (unsuccessful beyond accuracy). In stable settings, the budget and sustainability challenges for long-term programmes and assumptions of chronic need dominate. It would be interesting to do some cross-sector learning. However, there are certain risks when considering using coping capacity and food insecurity data for beneficiary selection to social protection programmes (as humanitarians do), potentially due to:
    • Cost and time
    • Likelihood of misreporting
    • Fluidity of this data
  • In acute crisis situations where the focus is on reducing exclusion errors, a case can be made for increasing coverage. This might entail a trade-off between the size of the transfer and coverage, ie, go for higher coverage but at a lower transfer value.
    • In contexts where there is a weak or fragmented state, exclusion may lead to/enable alternative providers offering assistance to fill gaps left by the state. This can be provided to their followers, but also beyond to improve their legitimacy
    • In humanitarian settings, some push for blanket targeting (in a specific demographic category) although this has obvious budget considerations
  • Some comments on variability of (e.g. poverty, displacement, violence) data during crisis, even if protracted, which implies data is irrelevant if not updated to predict new poor. This also has potential for increasing tensions and reducing social cohesion. Importantly highlighted that the type of shock is meaningful to how relevant data is (and any need for updating).
  • Important to consider data and capacity requirements of the different targeting methods. For example, categorical targeting has overall fewer requirements than PMT (which depends on 2 datasets), as many of its markers do not significantly change over time. Lower requirements is better for shock response and for sustainability.

Flagged examples included:

  • Transformation of poverty and precarity in Lebanon stemming from its devaluation crisis – can targeting strategies keep up with these changes?
  • Yemen and Lebanon as cases with targeting approaches based on pre-crises data
  • Issues emerging from ESSN targeting criteria in Turkey, where beneficiaries expressed a preference for broader coverage
  • Chile’s case of using a reduced form to collect household data after a shock linked to ex-ante data
  • Some COVID responses using simple forms of affluence testing data to target (excluding the better off) drawing on ex-ante data

Some documents shared:

Furthermore, we had a very interesting hangout where lots of targeting-related questions were posed, ideas were discussed, and examples were shared. For example, around terminology (harmonising vs homogenising), issues of political economy of targeting, use of multi-targeting approaches with different priorities, politicisation of some targeting methods, pragmatism for funding, “inclusive” social protection, pros and cons of universal targeting (e.g. vs cost of targeting, or sharing of transfers), targeting strategies that empower communities and are understood, and targeting through machine learning.

We will meet here again tomorrow for the next e-discussion on targeting and institutional orthodoxies. 

Welcome to day 3 of this e-discussion! Today we will be discussing: How do institutional orthodoxies impact targeting?

Debates about targeting are often presented as purely technical – about which approaches work best in terms of inclusion and exclusion errors and affordability. However, targeting is also a deeply contested and political exercise. In contexts of crises, beyond who is eligible, and how to identify and register these people, a mix of humanitarian and development actors with different principles, objectives, and preferred mechanisms often co-exist with democratic and institutional deficits.

  • To what extent (and why) do institutional orthodoxies and path dependency drive decisions about targeting in crises settings?
  • What should we be most concerned about with respect to these institutional/political positions? Do crisis settings create space for agencies to do things differently, or reinforce these positions?

What do you think?

Feel free to reply to any of the discussions from previous days as well.

One of the things that we found in our recent review of inclusiveness of social assistance in response to Covid-19 was that there wasn’t necessarily a great deal of consideration about the best or optimal targeting mechanism. With the focus on scaling up or rolling out transfers as quickly as possible, mechanisms and registries already in place were the ‘go-to’ option in many cases. While this certainly facilitated rapid action and coverage of relatively large swathes of vulnerable populations (notably in urban areas and among informal workers), it also led to considerable exclusion. There are some exceptions where new systems were set up, such as Nigeria where they used satellite imagery and an SMS-based registration system to underpin new Rapid Response Register and urban Covid-19 cash transfer. And in other countries, eligibility criteria and identification/ targeting mechanisms were adapted after first onset of the pandemic (e.g. Peru, Pakistan).

So in response to the question ‘Do crisis settings create space for agencies to do things differently, or reinforce these positions?’, I would say there is certainly scope for doing things differently (cash transfers having been rolled out to urban informal workers being case in point) but – especially in case of a large-scale shock like Covid-19 – paving the way BEFORE any potential crisis plays out might make this more successful.  

I'd like to see much more focus on governments in this discussion. The politics of targeting are fundamentally about how government's view citizenship and who should get benefits. Humanitarian actors should engage with those politics in line with commitments on the 'primary responsibility of the state to assist and protect' and be encouraging states to fulfil those responsibilities not skipping by default to substitution and their own targeting approaches. 

You raise an important point Paul. One could go further and distinguish between national and sub-national/local government. In the hangout yesterday, Emily echoed a point that Valentina raised earlier in the week around the distance between high-level (and often technical) discussions on targeting and what happens at the coalface of delivery. The very fact that agencies themselves sometimes struggle to explain targeting decisions (another point raised in the hangout) speaks to the problem with more sophisticated approaches that are used. Ultimately, targeting decisions on the ground will reflect notions of deservingness. This speaks to the state's view of citizeship but also to local social constructs. Frontline workers/local-level implementers invariably must triangulate programme-specified criteria and processes with local-level expectations of who should be targeted and who should be involved in making that decision.

One example is from Somali Region in Ethiopia, where various clan officials and other locally-trusted leaders are involved in targeting for the PSNP. In effect, the PSNP was only able to expand into areas where state capacity is historically weak through involving clan officials/elders (some of whom also have official roles in political administration but not always). Whereas quantitative evidence indicated that the better-off are targeted, qualitative evidence suggests wide approval of targeting decisions. For more on this example, see this paper I authored with Rachel SW and others: 

https://onlinelibrary.wiley.com/doi/abs/10.1111/dech.12694

And i am happy to share a PDF version for those without access!

This is an interesting point Paul. Yes it is about politics and the role of the state, and maybe also about access. What about in areas with high level of conflict and low access by the state or even where state is a party to local conflict... there are cases of non-state actors filling in some of these gaps, and targeting in certain ways. And yes, Jeremy, CBT is key in gaining legitimacy - but also as said above who forms parts of the community consultation process could also be contested in these contexts.

1. To what extent (and why) do institutional orthodoxies and path dependency drive decisions about targeting in crises settings?
2. What should we be most concerned about with respect to these institutional/political positions? Do crisis settings create space for agencies to do things differently, or reinforce these positions?

1. A crisis setting typically has a narrow window for response. The principle to Do no harm will often compete with the principle of No Regrets. Targeting, as important and sensitive as it is, is only one aspect of the delivery chain and whilst there are political or political economy implications, there are also immediate operational implications for then actually registering, informing, paying, monitoring etc. Many organisations have 24-72 hour playbooks that seek to answer not just questions around targeting but everything that needs to come after. Humanitarian donors may push for immediate answers to the question of how many people will benefit in order to award massive funding allocations. It's not so much a matter of institutional path-dependency, but of institutions understanding that certain approaches to targeting tend to have a higher and faster return on investment even if there are challenging trade-offs. They will over time develop certain approaches that become paradigms, that is not to say that within those institutions there are not many voices and vibrant debates on the degree to which the approach is relevant and should be applied in each new context. Actors might push a certain general playbook, but the practical implementation of that will often differ from context to context depending on available structures, resources, actors etc.

2. Yes, there is a lot of space to do things differently, maybe not ex ante, but over time, depending on the setting and the people involved. In contexts where there are strong systems or information in place this might be easier - e.g. Ukraine many humanitarian actors deployed their normal playbooks, but also deployed SP scoping and adapted over time to tap into national SP systems and processes. In contexts where there may be less systems and information available but the crisis moves beyond a months to becoming protracted (multiple years), the dynamics of the necessary assistance change, more granular data becomes available that can inform more detailed profiles of vulnerability.

We should be most concerned, I think, about the fact that there might be unclear mandates and visions in many settings between different international actors (agencies and donors), particularly when the topic of nexus is raised and a balance is sought between purely international humanitarian aid, and national response capacity and system building – often dubbed “development” even though they might directly overlap with humanitarian priorities. Can we realistically do both at the same time? I think it's hard in an initial phase but possible, perhaps even imperative over time. At the same time, whilst the assumption is that the state or authorities have the overriding duty to protect its people, this cannot be assumed in complex emergencies that have dimensions of violence and political instability.

I haven’t seen much in this discussion yet about the specific implications of targeting in conflict settings. To borrow a bit from WFP guidelines for conflict sensitivity analysis and risk assessment, some key questions need to be asked ideally before deploying aid such as “Could targeting result in disproportionate benefit to any ethnic, religious, tribal, gender, political etc group to the exclusion of others?, Could targeting coincide with key divisions in society/existing conflict(s)? Is there a risk that targeting exercises could be manipulated and potentially lead to breaches of impartiality? Has the targeting been done in a participatory and transparent way? Has the process created tensions with local governance structures (e.g. local authorities, traditional leaders, service providers, etc). Has targeting previously led to grievances, tensions, or conflict?"

I'd love to see the WFP conflict sensitivity guidelines Vincent, are they publicly available? A quick google search on 'conflict sensitive targeting' shows very thin resources. There are some interesting papers but many are now a decade or more old. Maybe the vocabulary around this has evolved? Or maybe there is a need for fresh thinking around this!

Great questions Vincent, and ones we should certainly struggle with moving forward.  Perhaps another 'hang out' to really try to understand the socially negative impacts of targeting in crises.   DO we have good evidence to draw on?

I am also keen to see whether there is any space for agencies - those who often cling to a specific mechanims such as HEA or PMT - to consider the advantages and disadvantages of the differnet approaches in different settings and for different groups of people.  Ie,  some critical self- reflection would be helpful.

Hi all,

During the third day of this e-discussion, we looked at different principles and objectives for targeting both from agencies and government. Some key points raised:

  • It may not be a matter of institutional path-dependency, but of “return on investment” for certain institutions despite known trade-offs. These may become preferred approaches and paradigms, which, following internal debates on relevance, can differ from context to context depending on local structures, resources, actors, etc.
  • Concern around unclear mandates of different international actors (i.e. nexus – for agencies and donors – is it realistic?)
  • Because in shock situations (e.g. COVID) the focus is on speed and scale-up, there may not be much consideration of optimal targeting mechanisms, instead using the ones already in place. This has exclusion implications.
    • Paving the way before crisis can potentially be more successful but also challenging to do ex-ante.
    • If there is a strong system and data in place, it may be easier to do things differently/adapt over time, especially if there is capacity for updated data collection (if/when poverty/vulnerability changes)
  • Views of citizenship: Need to focus more on governments’ role and the politics of targeting: who should get benefits? Who deserves it? To assist and protect WHO?
    • Government has responsibility to protect people – but this is complex where there is political instability and violence
    • Access is also a relevant point, especially in high conflict areas with low state access / state as a party to a conflict, and non-state actors filling gaps.
  • Local social constructs: consultation and triangulation with communities is key in making targeting decisions. But in community-based targeting processes, who the community representatives are can also be contested
  • Conflict sensitivity and risk assessment: who is more likely to disproportionately benefit from a targeting strategy? Could it reinforce society divisions or tensions? Could it lead to manipulation/impartiality? Is it participatory and transparent? Are local governance structures on board?

Examples and documents shared:

We will meet here again tomorrow for the final e-discussion on community perceptions in targeting. 

Day 4/Topic 4:  What is the role of community perceptions and social divisiveness in targeting?

Dear all!!  Welcome to our last day of the e-discussion on Targeting in Crises. In a number of posts during the week, and in our informal hangout on Wednesday, the issue of community/client perceptions of targeting has been raised as a critical issue.  For instance, when targeting in crisis contexts, there are likely to be existing social dynamics and beliefs at play which may influence the effectiveness and perception of targeting.

So here are a few questions to kick off the discussion today:

  • What community perceptions of targeting have you encountered in your work? Is this exacerbated in crisis settings, and why do you think it is?
  • What factors ensure targeting approaches are perceived as fair and transparent by communities? To what extent do communities ‘adjust’ for perceived unfairness, for example by informally sharing amongst themselves?

There's a great paper just published in World Development that explores issues of fairness and transparency in Chad:

https://www.sciencedirect.com/science/article/pii/S0305750X22001127

Della Guardia, Lake and Schnitzer find, for Chad:

' ... targeted programs can also reconfigure social relations, carrying a social stigma that bifurcates communities'.

'Drawing from rich qualitative data from a cash transfer program in Chad, we explore both the economic
and social implications of targeting in cash transfer programs in contexts with widespread poverty. We
find significant positive economic effects on non-beneficiaries. At the same time, not only does participa-
tion engender considerable social costs, but several punitive and economic costs arise for recipients as a
result of their inclusion in the program, with repercussions for the transfer’s productivity. We conclude
that in contexts where everyone is poor, targeting can create new fissures within a community, stemming
from a combination of jealousy and skepticism with regard to the perceived deservingness of transfer
recipients vis-à-vis other village inhabitants. When budgets are insufficient to cover all poor, the positive
effects of cash transfer programs may be enhanced by reducing the geographic focus of social safety net
programs to ensure all inhabitants can access benefits.'

This is similar to work I did some time ago in Solomon Islands - where you have strong, longstanding forms of mutual reciprocity systems in existence, there's a risk that targeting that cuts across existing social and community relationships in ways that can undermine reciprocity and create cleavages at a community level. That doesn't mean we shouldn't target (because the existing systems might themselves of course be unequal, unequitable, exclusionary, regressive, etc) but we do need to think about how our targeting might have wider relational effects, including exacerbating tensions.

Thanks for the discussion Emily, Rachel and Carolina.

Thanks for that post Rach.  I look forward to reading the article.

On another aspects of targeting and community perceptions - i was recently in Lebanon with my colleague Philip Proudfoot, and a very quick field visit to one location in the south, unearthed that there is much community discontent with the way that the PMT is currently being verified.  It reminded me of when I was working in Rwanda, helping to supported the development of a score card mechanism to traingulate community based targeting.  the score card was essentially a PMT with 13 variables.  Overall,  it was easy to administer and the statistical algorithm was relatively inutitive for those who understood regressions.  yet,  it was still considered a 'black box' to the community and a number of officials - and was viewed suspiciously in some quarters.  

 

So how can we simpify the PMT approach and explanation?   and to what extent is it important for the beneficiaries of a SP programme to understand why they have been included?  I would say that recipient understanding of the targeting mechanisms is vital to the ease and equitable provision of transfers. 

 

 

That's interesting Rachel.  It leaves me thinking about two things: 

First, what exacty it is that isn't trusted - the PMT itself or the actors that implement it?  One of my hunches is that it's probably both - that reactions to specific programmatic procedures and mechanisms are a combination of a reaction to that specific procedure (whether that's PMT or another programme or targeting feature) but also to the governing authorities.

Second, what are the alternatives? Lotteries have come up a few times times in resource-constrained situations where it's difficult to find a hair's width of difference between those deemed eligible and ineligible for support. LONDO in CAR is a good example (https://socialprotection.org/discover/publications/can-public-works-enha...) of how to create a sense of fairness when expanding programmes out of urban areas into rural contexts where needs are high. 

I guess that solves only one part of the trust and transparency problem - the lottery mechanism itself might be fair and well set up but that doesn't mean people trust those that are implementing them.  We learned from the securelivelihoods.org research project that the most important thing is usually information and communication.

 

Targeting methods that are conducted away from recipients can be viewed with distrust. Research from Development Pathways highlights how refugees do not understand beneficiary selection through PMT (https://www.developmentpathways.co.uk/blog/targeting-humanitarian-aid-so...). This issue may apply to any context but is intensified in contexts of protracted crisis, where up-to-date data is less likely to exist and the propensity for social unrest is higher. Perceptions of lack of transparency or unfairness are not specific to one method and may occur with different targeting mechanisms (even though more likely with some than others). Also, in some contexts, such as where income distribution is flat, the act of targeting is already considered unfair (see case of Turkey ESSN: https://docs.wfp.org/api/documents/WFP-0000100401/download/?_ga=2.657045...). 

I find the role of technology in community perceptions interesting, more specifically, to what extent does technology improve these perceptions or damage them even further?

Hi Rachel, thanks for facilitating such an interesting discussion!! I would love to hear more about your work in Rowanda and the scorecard you mentioned. I have been reading/listening to the debate about which targeting modality is better and whether PMT, CBT or categorical targeting are more suited to certain contexts, given the strengths each one of them can provide, but I was curious about the value of combining more than one targeting modality as means to triangulate/verify eligibility, and to balance the trade-off between community acceptance and applying a seemingly objective scientific way of selecting beneficiaries to ensure no specific community groups are left out, especially in contexts you want to respect existing community elders/tribal structures 

1. What community perceptions of targeting have you encountered in your work? Is this exacerbated in crisis settings, and why do you think it is?
2. What factors ensure targeting approaches are perceived as fair and transparent by communities? To what extent do communities ‘adjust’ for perceived unfairness, for example by informally sharing amongst themselves?

on 1) The Della Guardia, Lake and Schnitzer on Chad I think captures a lot of debate about community perceptions, unintended consequences for economic activity and community perception, sharing of transfers etc. They also pointed out that the dynamics of negative perceptions and associated conflicts differed vastly (85% reported a conflict in the region they studied vs 40% in another region). In my work in different countries the experience has generally been that its a matter of local perceptions as well as local politics. We can't say that PMT across the board will be considered a black box that potentially causes grievances, or even that a categorial approach with accepted vulnerabilities will not. Local politics matter, local implementation partners and the training and sensitivity of their staff matter, cultural norms matter and vary from region to region, even from community to community. Moreoer, the challenge isn't just the targeting method, its whether that data is regularly updated in a way that is accurate and perceived as fair, and whether there are then enough resources to do the data justice in terms of actual transfers. Applying a method that highlights people that are only slightly poorer today, but maybe not tomorrow, and setting a cut-off quota because of budget restrictions and then not revisiting the data for years because it is too costly, is not helpful. Particularly if local leaders are then asked to justify who is in and who is out and leaving them to convince their own constituents that their choices were fair, will likely create resentment. I think the Chad paper makes a good proposal in saying that if resources are limited in contexts where there is widespread poverty and hungry, it may just be better to apply a universal approach for a more limited geographical region (of course then regional and ethnic politics can come into play, which is particularly problematic in conflict environments.) On a side-note, I think that sometimes not enough resources are invested in sub-national political economy analysis, mainstreaming it into appropriately differentiated targeting approach that are sensitive of socio-political dynamics, and that moreover the monitoring of the implementation of targeting and enrolment strategies, often by third actors and parties that may potentially have limited training, unclear incentives and not be perceived as neutral by the communities.

2. On factors that might make perceptions of targeting more fair, firstly a well-designed and implemented communication strategy that is informed by effective local knowledge attituted and practices studies and adequately funded, and that ideally has buy-in from local leaders. Secondly, the use of effective complaints and feedback mechansims. This latter part in particular is crucial in fragile and conflict settings or areas where one may suspect that power is captured by elites and people are being marginalised - creating a safe space for both beneficiaries and non-beneficiaries to voice and channel concerns shouldn't be a "nice to have" but a critical component of social assistance. 

Summary Day 4:  What is the role of community perceptions and social divisiveness in targeting?           

Hi all,

Today, in the last day for the e-discussion, we explored some questions about community perceptions of targeting transparency and fairness in situations of crises, which can ultimately affect effectiveness:

  • Targeting methods conducted at arm’s length can be viewed suspiciously. Recipients understanding targeting mechanisms is key for effectiveness (can be challenging specifically for PMT)
  • Wider relational effects of targeting: effects can go beyond programme implementation and even affect social relations in a community (it can do so negatively in contexts where ‘everyone is poor’, or where mutual reciprocity systems exists)
  • Lack of trust / transparency problem: more of an issue in crises where likelihood of tensions and social unrest is higher
    • Lack of trust about the mechanism
    • Lack of trust concerning the implementers
  • Local politics matters in relation to:
    • Local implementation partners
    • Training and sensitivity of staff
    • Whether data is updated and perceived as transparent
  • Ideas for improving fairness perceptions of targeting:
    • Sub-national political economy analysis: need for targeting to be sensitive to socio-political dynamics
    • Well-designed and implemented comms strategy informed by local knowledge with local buy-in
    • Use of effective complaints and feedback mechanisms: critical in places with power imbalances and marginalisation
    • Monitoring and implementation of targeting and enrolment by third parties that are sufficiently trained and neutral
    • Triangulation of eligibility by combining multiple methods: potential to reduce some of the trade-offs

 

Examples and documents shared:

- On issues of fairness and transparency in Chad, along with unintended consequences: https://www-sciencedirect-com.ezproxy.sussex.ac.uk/science/article/pii/S...
- Case of Solomon Islands where targeting had wider relational effects: risk of social tensions in the community and undermining mutual reciprocity
- Community discontent in the south of Lebanon around verification of PMT
- Case of score card mechanisms in Rwanda to triangulate CBT – considered a black box to community and even suspicious
- LONDO used lotteries in CAR as a ‘fairer perceived’ alternative to targeting https://socialprotection.org/discover/publications/can-public-works-enha...
- Case of refugees in Turkey not understanding PMT method and attributing selection to luck https://www.developmentpathways.co.uk/blog/targeting-humanitarian-aid-so...

Hi All!

Thanks to all of you who have participated in this e-discussion and were able to get online for the wednesday hangout.  W.hile we will not be keeping a watching brief on this chat,  it is, nonetheless, a live chat so feel free to keep posting.  We will log on from time to time to catch up.  

We are hoping to have more to discuss later in the year/early next year on topics of targeting in crises.  We will keep everyone posted.

All the best and do log on to the new e-discussion taht will go live on Monday

Rachel, Emily and team!

The best way to win your lottery games is through the help of Dr Amber that uses his spiritual powers to make anyone a winner on any lottery game. I am a living testifier of his good works and today I'm a proud winner of $7,000,000 million dollars. Anyone out here that needs help can connect with Dr Amber today via email: [email protected] gmail . com