Category: SDG

“To accomplish great things, we must not only act, but also dream; not only plan, but also believe.”

“It is well for the heart to be naïve and for the mind not to be.“

Anatole France

THE GOOD INTENTION

Leaving no-one behind is the “transformative promise” of the SDGs. It has become a mantra in development planning. It is everywhere, invading discussions, plans and interventions with a raft of good intentions. It has even developed into an acronym, a sure sign of having arrived in a world awash with short forms of speech. It is a persuasive argument that unless those worst off see their lives dramatically improved, we cannot claim that the SDGs have been met:

“As we embark on this great collective journey, we pledge that no-one will be left behind. Recognising that the dignity of the human person is fundamental, we wish to see the Goals and targets met for all nations and peoples and for all segments of society. And we will endeavour to reach the furthest behind first.” (my emphasis)

And:

“By 2030, empower and promote the social, economic and political inclusion of all, irrespective of age, sex, disability, race, ethnicity, origin, religion, or economic or other status.”

Governments and those engaged in development everywhere feel compelled to attend to it, and over the past five years evaluators, evaluation thinkers and evaluation commissioners have had this notion firmly in their line of sight. Speakers in conferences and webinars continue to ponder on how best to help ensure NOLB through evaluative practice. Discussion documents, guidelines and evaluations with a NOLB focus have proliferated, giving hope and showing strong commitment to the intent in spite of the challenge of moving from rhetoric to action.

THE PROBLEM

In their thoughtful 2019 book, Leave no-one behind: Time for specifics on the SDGs, Homi Kharas and his co-authors bring together many perspectives and experiences applied to different NOLB foci – gender, poverty, farmers, refugees and migrants, education, health care and more. Like so many articles and reports, they offer insights and recommendations on what can be done. They also eloquently point out the scale of the challenge, and the countries most deeply affected.

The key point? The countries where NOLB is of greatest concern, where most of the interventions are concentrated, are all in the Global South, that part of the world in shades of red or some other vibrant “this is bad” colour in just about every map that visualises data on the state of the world, where everything to do with development is more challenging, more resource-intensive, and a greater burden on governments and others who want to make a difference. 

This means that the burden of supporting and sustaining a majority of ‘leaving no-one behind’ efforts fall inevitably on many of the poorest (low-income) countries in the Global South.

The problem is that they cannot afford it, nor can they sustain it.

It will therefore be unfair to hold such countries accountable for ‘leaving no-one behind’ strategies. 

THE ETHICS OF ‘LEAVING NO-ONE BEHIND’

There are two sides to the ethical imperative for ‘leaving no-one behind’.

It is unethical to keep on neglecting the most vulnerable and those on the margins of society. It is equally unethical to intervene with time-bound interventions to ‘leave no-one behind’ when they have very little chance of being sustained once the source of financing leaves. This causes too much destruction of societal systems, too much hope that is created and then shattered, too much behaviour that is changed temporarily without thinking of the long-term, often negative consequences.

There is the need to dream, to have spectacular ambitions. But there is also the need for pragmatism among those engaged with development as well as those who evaluate for it.

As Oumoul Ba Tal from Mauritania, a former President of the African Evaluation Association, once said in a heated exchange with a senior evaluation specialist from an international agency:

“It is not about your project; it is about my country”. 

THE NEED FOR PRAGMATISM  

The pressure to achieve the SDGs means that aid agencies inevitably have a focus on no-one left behind. Authorities in the Global South also feel compelled to attend it as a priority. It is a fact that this type of focus resonates well with all of us. Development and evaluation specialists are at heart most often idealists who want to help to make the world a better place. Aiming to ensure that no-one gets left behind appears to be a noble part of what we can do at this time in the world. It has become soft-on-the-ear-and-mind rhetoric, something to be done because it sounds and feels right.

But from a pragmatic perspective, if we really want to see development that is effective and that sustains, and that eventually succeeds in leaving no-one behind, we have to consider the following:

One, the level of cost and effort matter. In some countries ‘no-one left behind’ relates to the urban-rural divide; in others to the invisibility everywhere of the poorest of the poor, the most marginalised, the most neglected; in others to those displaced or victims of war. Those left behind typically present a ‘last mile’ problem that is always much more costly in terms of effort, resources and time – and even much more so in low-income or ‘least developed’ countries where whole systems need to co-evolve to ensure a positive development trajectory over a long period. It becomes even more problematic if the intent is to address first those who would qualify as most marginalised or vulnerable, as the 2030 Agenda for Sustainable Development and even the recently released guidance for the revised DAC criteria both suggest. This is a highly undesirable approach, also for the reasons that follow.

Two, sustainability matters. Development projects and programmes tend to be terminated after three or five or perhaps ten years, often without having achieved much. In low-income countries the chance that even well-crafted and implemented interventions aimed at ensuring no-one is left behind will sustain is highly unlikely without a massive, intensive effort over a significant period. Once aid agencies leave, local actors have to pick up the pieces – something that is incredibly difficult to do in the often fragile, resource-constrained contexts in much of the Global South.

Three, long-term development trajectories matter. Countries that are already well advanced on a positive development trajectory – where many vectors or indicators of national progress move in desired directions, where many institutional and other systems have already advanced and reached some form of reasonable stability – will find it much easier to make the ‘last mile’ no-one left behind effort. This is why China, despite its extraordinarily rapid economic development, waited until recently before focusing on eradicating extreme poverty in rural areas.

Four, dealing with choices and trade-offs matter. If a low income country starts to move along a positive development trajectory, trade-offs will be required. One of them is that some will have to be left behind, until infrastructure and institutional systems, societal relationships and capacities have evolved to a stage where there is enough to support and sustain the development of those who are most marginalised and vulnerable. What is important is how carefully and systematically inequalities are uncovered through evaluative practices, and managed and slowly woven into strategies as they evolve throughout the different stages of development. In other words, mindful that there will be trade-offs, the notion that no-one should be left behind can be implemented systematically and pragmatically rather than too quickly based on the idealism reflected in so many unsustainable development interventions.

Five, complexity concepts matter, in particular ‘co-evolution’. All of the above reflect the need to view development through a complex adaptive systems lens – considering the path-dependence of societies’ development, a long-term development trajectory instead of snapshot thinking, dealing pragmatically and systematically with choices and trade-offs, and ensuring the type of sequencing of action that will help ensure that positive results sustain.

Most importantly for successfully dealing with no-one left behind strategies is recognising that system co-evolve, especially in countries where much starts from a low base. Development normally requires societal values and beliefs, capacities, institutions, markets and more to evolve together. This takes time and ongoing effort. In low-income countries interventions can try to nudge systems in desired directions, but if they try to shift part of a system or a nested system too quickly, it may not work; it will strive to return to its former state.

THE RESPONSIBILITY OF EVALUATION SPECIALISTS

Aid funding is increasingly concentrated in low(est) income countries in the Global South, and even within these contexts there is often a focus on the most vulnerable and fragile. It is crucial to think during evaluations about the systems on which they will depend when donor funding ends. What will be expected from local and national authorities when the financiers withdraw?

If the potential for sustainability of any positive achievements is lacking when financing comes to an end, any assessment against criteria such as ‘relevance’, ‘effectiveness’, ‘efficiency’ and ‘impact will mislead unless others take on or complement the support. This places a major responsibility on evaluation professionals to be very careful in the assessments that we make, and to consider the implications for evaluative practice of issues raised in this post.

As Oumoul Ba Tal inferred so long ago, we need a country and systems lens, not a project lens, if we truly want to ensure development that sustains and bears fruit for all once the financiers have left.

The Aspiration and the Responsibility

The Sustainable Development Goals (SDGs) provide a spectacular vision for

“eradicating poverty in all its forms and dimensions, combatting inequality within and among countries, preserving the planet, creating sustained, inclusive and sustainable economic growth and fostering social inclusion are linked to each other and are interdependent.” 

Further, the SDGs have a sharp focus on “leaving no one behind”:

“As we embark on this great collective journey, we pledge that no one will be left behind. Recognizing that the dignity of the human person is fundamental, we wish to see the Goals and targets met for all nations and peoples and for all segments of society. And we will endeavour to reach the furthest behind first.“

Such grand visioning and rhetoric need to be tempered by harsh realities how COVID, Climate Change, debts and discriminatory practices and structures have had cumulative and intersectional impacts on poor countries and poor people.  

Change often happens through ambitions and aspirations to achieve the near impossible. While the cold gaze of measurement and design should not dampen the enthusiasm inherent in long-term ambitions, the evaluation community could ethically and responsibly provoke questions around the types of systems changes, interventions, partnerships, governance structures and learning systems that might need to be in place to help achieve the goals.

Good evaluations often start with understanding pathways of impacts. For example, the whole essence of theory-of-change approaches to evaluation question is the humble question: How?  

I work in development/global health, with a majority of my work in the past three years in India. While there is a clear focus on indicators both in the SDG documents and planned interventions/systems change efforts, there continue to be ambiguities on the roles that evaluation and evaluators can play in helping achieve such spectacular ambitions.   

In this blog I raise questions related to the role evaluations need to play in helping achieve such grand goals. How do we move from a position of being passive responders to a more active stance that demonstrates leadership for enhancing the likelihood of achieving such grand visions?  

Getting Real

Understanding the cumulative, dynamic pressures of COVID, debts, discrimination and climate change become especially important as we seek to locate our role. Consider this recent article from April 2021:

“Without climate finance, poor countries face a bleak future of extreme weather, water and food shortages, and climate-driven migration, which all threaten to reverse decades of progress in lifting people out of poverty. Many governments are also being wooed by fossil-fuel developers eager to exploit coal, oil or gasfields in exchange for cash. The problem is compounded by new waves of the Covid-19 pandemic, and poor countries are seeing debts soar and the cost of borrowing rise.“

It is important that responding to such challenges are not seen as problems confined to the developed world. Many of the syndemic problems of racism, inequities and fractured civic ecologies are very much a reality in the ‘developed North.  Consider this Guardian article from February 2021:  

“For Black Americans, Covid-19 is another brutal reminder of the racist legacy of the American healthcare system. A disproportionate number of the 500,000 Americans who have died of coronavirus are Black. Yet African Americans and other people of color have struggled to access vaccines.  Racism corrupts every facet of the US healthcare system. Discriminatory practices barred the entry of Black candidates into medical, nursing, dental, pharmacy and other professional healthcare programs. Segregated medical facilities and unjust treatment within medical facilities continue to plague Black patients.“

As the evaluation community seeks to find and assert its voice in contributing to SDGs, it is important to take stock of naming the processes that contribute to amplifying the problems. In my experience, our focus tends to be the “intervention” or the “project”: we can get caught up in the project trap. We tend to focus less on the systems and syndemic processes that generate the conditions of poverty and inequities.  It is within the purview of evaluation and evaluative thinking to promote realism about the context and support structures and bring a clearer focus on systems and dynamic processes that will help achieve the vision of “no one left behind.”

As a field, we need to both recognize the formidable nature of the tasks and move toward action. As Ibram Kendi has brilliantly shown in How to be an Antiracist addressing the impacts of racism requires active commitment and action to reversing inequities. Lazily claiming to be a “non-racist” or “neutral” and “above the fray” or dismissing opportunities to push an agenda that takes inequities seriously with ‘this intervention has little to do with inequities’ is hardly enough. How do we move as a field beyond our passive focus on projects and interventions?  Should taking a more disruptive stance towards provoking clarity around the pathways, seriousness of addressing inequities, and realism about the complexity of the change process be part of how we approach our role and responsibility in helping achieve the SDGs?

Questions Galore

I briefly expand on these above issues by raising questions that I think are important as we locate our role in addressing the SDGs. My questions that seek to clarify roles are in four domains:

1. Building capacities and sustainable impacts
2. Multi-dimensional contexts and trajectories of outcomes
3. Understanding the coherence and the disruptive powers of interventions
4. What does it take to “leave no one behind”?

1) Building system-level capacities and sustainable impacts

It’s important to recognize that much of evaluators’ focus on impacts has not been matched with ensuring that these impacts are sustained in the longer run, after the funding of interventions has ended.  As example, Jindra Cekan has been a leader in calling out how few evaluations focus on issues of emerging and sustained impacts. Along with colleagues, I have argued elsewhere that we need to integrate planning for sustainability into our thinking of theories of change as we understand  how interventions/systems can bring about long-term change. The interface between interventions and whether these innovations are absorbed by systems capable of implementing the interventions in a sustainable way still needs deeper exploration. In a similar vein, there’s a need to pay attention to system level capacities including ‘adaptive’ and ‘absorptive’ capacities of systems.  As we focus on the interface of interventions and systems, some questions arise: What is a theory of change that focusses on sustainable impacts? How do we operationalize systems’/institutions’ abilities to absorb interventions? How can understanding mechanisms of sustainability occur at multiple levels?  How can concepts and methods from complexity sciences help inform our theorizing and measurement of sustainable impacts (see Zenda Ofir’s blog on viewing development as a complex adaptive system)?  The challenges of understanding development from a complexity lens goes beyond measurement: How can ideas of complexity sciences inform our view of what constitutes evidence?

2) Multi-dimensional contexts and trajectories of indicators

As an evaluation community we could be more focussed in understanding what are the multi-dimensional contexts that make changes (and reaching SDG targets) harder in some contexts than others. Much of the results frameworks that I see still have anticipations of success that are based on aspirations of ‘mechanical,’ ‘aspirations’ of results often based on linear extrapolations of past patterns. How can we incorporate contexts more deeply into our planning for what changes are realistic in different settings? We need a broader set of dialogues on what contexts matter and also how changes in context and planning for interventions in especially resource-deprived contexts need to inform our expectations of success. Further, we need better examples of how progressive evaluation approaches such as developmental evaluation, realist evaluation, and principles-focused evaluation help build a better understanding/intuition of what are realistic changes especially in very “sticky,” difficult to change conditions.

A focus on multidimensional contexts can also shed light on the anticipated kinetics of systems change.  I have found our approaches and tools remarkably limited for understanding anticipated timelines, trajectories and rates of change (see Michael Woolcock’s work for a refreshing introduction to the importance of these concepts).

3) Understanding the coherence and the disruptive powers of interventions

One of the significant aspects of the newly developed Development Assistance Criteria (DAC) is its focus on coherence as one of the evaluation criteria. How can evaluators help in mapping out the alignment between different levels of interventions to promote their coherence and disruptive powers? The mechanisms by which coherence matters and the contexts in which coherence matters need to be more clearly understood. Under what conditions should interventions be coherent/aligned with the rest of its local ecology/context? Alternatively, under what conditions can a lack of coherence add to the disruptive powers of an intervention in leading to favourable changes? Under what conditions should interventions not try to fit in with existing interventions, especially in contexts in which the contexts itself needs changes? Thinking about the coherence of multiple interventions forces us to explore design considerations.

4) What does it take to “leave no one behind”?

What could the evaluation community be doing to help operationalize strategies to population segments that correspond to intersectional categories of multiple disadvantages? Working in the international development/global health space, I continue to be surprised at the shallow nature of most theories of change focussed on inequities. The same focus that the evaluation community has spent on deciding what is a good, rigorous evaluation design we have not spent in answering what is a good enough theory of change. This problem is especially acute when we consider interventions that have the aspirations of addressing problems of inequities. What is a good enough theory — and by this, I simply mean a theory that can seriously aid translation/implementation — to address inequities?  Addressing these questions will help position our field to be more relevant in any dialogue that focuses on making a difference in the lives of individuals in need.

Looking Ahead

The SDGs provide fundamental challenges and amazing opportunities for the evaluation community to go beyond standard, stale ways of valuing. Addressing some of the above questions will have implications for how we think about evaluation competencies and how we approach building evaluation capacities. There is a need for the evaluation community to engage more deeply with both conceptual and evaluative issues involved with sustainability, inequity, contexts, disruption and coherence.

Addressing some of the above questions will require new partnerships, not just between evaluators of different stripes but between the evaluation and a number of other communities/sectors including ecologists, public health and nutrition, financial services for the poor, epidemiologists, etc. We might also need to consider how diverse evaluation approaches can be combined with other measurement opportunities (such as routine surveillance, etc.) to help build useful ecologies of evidence.

This is an exciting time to be an evaluator and an exciting time to rethink evaluation competencies and capacities.

Naïvely I used to think that sustainability was just about a program finding another funding source when the previous funding was coming to an end. The only other examples that I recall about sustainability were stories about international development projects that, for example, built a water pump in a village in a far-away country and then after the project implementers left, the water pump eventually broke but no one in the community knew how to fix or maintain the pump — so the benefits from the project were not sustained. Such examples hinted at the notion that gains had been made, but they could not be sustained over time, partially due to poor planning.

I finally got a clearer, fuller picture of what sustainability is about while writing a paper in 2019,

‘Till Time (and Poor Planning) Do Us Part: Programs as Dynamic Systems Incorporating Planning of Sustainability into Theories of Change’

Canadian Journal of Evaluation, 2019

This blog is about some of the lessons we learned in thinking about sustainability and the roles for evaluation in helping better ensure that gains made by projects/initiatives can be sustained.

What basically is sustainability?

Sustainability can be thought of in three ways: 1) sustaining components of the intervention, 2) sustaining impacts after the program ends, and 3) mainstreaming or incorporating active ingredients of the intervention into other programs.

Program vs Client Point-of-Views

In my observation, evaluators, planners, program managers and staff tend to conceive of the impacts of a program very much from the viewpoint of the program – in terms of its activities and outputs that presumably lead to the intended outcomes. But if we consider a program’s impact from the frame of a client/ intended program recipient, most clients intersect only at points with a program while the rest of the time they are outside from the program living their lives. The “impact journey” typically encountered by a client rarely takes the form of a linear trajectory through one program but is one of a “rugged landscape” where many hills need to be climbed, circuitous pathways navigated, rivers crossed to reach the promised land of impacts. The program may only partially help progress the client along their impact journey. In planning programs, one needs to consider what else may be needed for clients to eventually reach the promised land, or at least what else is needed to sustain the gains achieved.

In evaluation terms, this translates to a change in focus from more direct and immediate program impacts to more sustained impacts that take into account the complex and heterogeneous needs of clients/program recipients.  

Risk Landscapes of Clients  

The metaphors of heterogeneous landscapes and multiple hills that are involved in clients’ journeys toward achieving goals/outcomes are useful reminders of the risk landscapes that remain for clients to contend with even after the intervention ends. In our experience, programs typically end without sufficient attention paid in advance to the provision of resources or continuity of care to clients, or to implementing strategies to better ensure that the outcomes achieved during the intervention can be maintained after the program ends. Knowledge of clients’ heterogeneous impact journeys and what context of supports, capacities, opportunities, and motivational incentives they might need after the program ends to at least retain the gains achieved and hopefully stay on the journey can inform strategies that envision what needs to be in place. For example, what needs to be in place might include partnerships with other agencies that have the capacity to carry on, or equipping participants with needed capabilities and resources to carry on themselves. In some scenarios, it might be preferable for a program to divert funds from serving a larger number of clients and taking them halfway across the river, to reallocating resources in order to take a fewer number of clients all the way across the river.

What changes when planning for sustainability?

When you plan for sustainability, your intervention’s theory of change and program activities will be different than if you plan only for outcomes within the intervention’s sphere of control and influence. For example, there have been multitudes of exercise programs studied in the literature that show good results of attendance, compliance, improved health outcomes, etc., but after the program ends the participants stop exercising, so maintenance of the healthy gains dissipates. Some of the reasons given by participants include: no gyms or green spaces in their neighbourhood, lack of transportation to get to gyms, other exercise class times are not convenient, not feeling safe to exercise alone, and loss of motivation. If we know all of this, then a typical exercise program should be changed from simply leading participants through the best set of exercises to also teaching participants how to exercise on their own, helping them to understand what affects motivation, structuring exercise sessions so that endorphins (natural feel-good chemicals in the brain) kick in, reinforcing how good exercise feels so that participants start associating exercising with feeling good and will be more driven to continue. The intervention changes based on considerations of what clients will typically encounter in their journey beyond the intervention and what it would take to sustain them in that journey.

Planning for sustainability should be an integral part of what we consider to be a useful theory of change. Incorporating planning for sustainability can change the nature of the program itself (Sridharan & Gillespie, 2004).

Finally, here are tips and considerations for developing a theory of change to incorporate sustainability planning:

• Make explicit the uncertainties in the theory of change
• Map the connections between the intervention and the rest of the organizational system
• Explore the heterogeneities of client-level needs
• Explore key barriers that individuals will face in the intended impact journey
• Explore the potential timelines of impact
• Explore the dynamic context of supports needed both during and post-funding stages to assist with capabilities, opportunities, and motivational incentives
• Explore the role of boundary partners