Fred Carden

Coherence: A Principles-Focused Approach

  • What is Coherence?
    • OECD 1991 – initial focus on policy coherence
    • European Commission 1992 – put into law
    • Focus has been on global issues
    • Enshrined in SDGs
    • No accepted definition
    • OECD defines coherence as: an approach and policy tool to systematically integrate the economic, social and environmental dimensions of development at all stages of domestic and international policy analysis.
  • What is Coherence in a Country Program?
    • Development, Trade, Diplomacy – how are they coherent?
    • There are different levels of coherence: 1) Internal Coherence; 2) Intra-Organizational Coherence; 3) Inter-Organizational Coherence; 4) International Coherence
    • Focus of Global Affairs Canada in Columbia case was on first two levels of coherence
    • We first looked at the notion that coherence is not only about policy – what does coherence mean in the context of programming of global affairs in Columbia?
  •  Levels of Coherence
    • Policy Coherence
    • Organizational Coherence
    • Administrative Coherence
    • Knowledge Coherence
    • Informal Coherence
  • Measuring Coherence
    • Developed four-level rubric – no coherence to coherence
    • Breaking it up into different elements very helpful in discerning what was/was not coherent – Example: coherence across international assistance; coherence across international engagement
    • Found that there was pretty good policy coherence, but administratively weak

Lori Bell

  • In the context of UNHCR commissioned evaluations, coherence is always looked at through the lens of relevance – are we doing the right thing?
  • Presentation provides concrete examples of evaluations where coherence was looked at from an internal and external point of view

System-wide evaluation of cyclone Idai Mozambique 2019

External Coherence

  • Complexity, crowdedness of the humanitarian space
  • The divide and conquer of different agencies
  • How did we live up to our responsibilities and roles compared to what other actors did?
  • If you want to look at External Coherence, you can’t do it on your own – we need to understand the comparative advantage of UNICEF, etc.
  • We need to know a lot about other agencies operations and what the government is doing
  • Inter-agency joint evaluation, rather than a single agency evaluation

The Importance of Personalities, Perceptions, and Culture

  • Coherence boils down to individual behaviours and decision-making on the ground
  • We need to look at culture and perceptions

Elliot Stern

Aspects of Policy Coherence

  • ‘Policy Coherence’ in EU and OECD
    • Coherence now a “new” (2020) OECD criteria, but entered European policy vocabulary in 1992 Maastricht Treaty
    • Distinction between internal and external coherence – i.e., consistency of elements making up a policy action vs consistency between different policy actions (this distinction may be less clear-cut than 10 or 20 years ago)
  • 4 Aspects of Contemporary Policy Coherence
    • Unit of analysis
      • Growing understanding of complex and interdependent policy priorities (e.g., innovation, climate change mitigation, equity and justice)
      • Extending the Unit of Analysis: i) decontextualized policy interventions (coherent with policy inputs); ii) contextualized policy interventions (coherent with policy/non-policy inputs/contexts); iii) interdependent policies interventions (coherent between multiple policies in multiple contexts); iv) policy systems (many coherence possibilities)
      • Systems Mapping for complex policy evaluation
    • Coordination and governance – Coherence doesn’t happen on its own!
      • Multi-part and separate but interdependent policy interventions must be aligned – importance of coordination and governance arrangements
      • Success often depends on pre-existing community, sectoral/territorial networks; credibility of system leaders; capacities to use information and adapt; participatory engagement of stakeholders
      • Coordination and governance itself must be evaluated – part of ToCs, etc.
      • Encouraging those in governance and coordination roles to reflexively learn and generate and use M&E data a way of strengthening coherence
    • Timing: Re-setting the evaluation clock
      • Move away from short-run, ex-post evaluation at end of program/policy cycle to integrate evaluation into program planning and “up-valuing” mid-term/ongoing evaluations
      • Ex-post evaluations are moving long-term – look back over longer time scales on rolling basis
      • Mid-term evaluations increasingly inform reprogramming and “steering” and adaptive management
    • Importance of substantive theory
      • When we move from narrow focus on programs to recognize importance of complexity, context and interdependencies, it changes where we look for theory
      • Program theory is not enough
      • Becoming standard in UK and EU structural funds to look to academic research to identify building blocks of Theories of Change (e.g., Innovations Systems Theory)
      • Centrality of literature reviews and partnerships with domain experts

Vijayalakshmi Vadivelu

UNDP’s programming:

  • In past 3 decades, programs have looked at coherence in different program areas – support to livelihoods, peace building, etc. require synergies – intersecting elements to enhance coherence at the country level
  • UNDP’s consistent emphasis on coherence and partnerships – implemented with a range of partnerships – integrated role within UN system
  • Programming at national and subnational levels enables synergies/linkages

Coherence a key dimension of program performance

  • Assessed at multiple levels: Internal, External, Intra-organizational
  • Mapping of areas where maximum synergies are possible 
  • Also look at intersecting areas in response to crises

External Coherence      

  • UNDP collaborates with other actors to contribute to humanitarian response
  • Determine pathways that link, map key actors working in the area, identify intersecting process indicators and strength of partnerships
  • In bringing together different actors, coherence assessed using sector data
    • Increasing household income, increased investment = indicators for policy coherence

Different types of Coherence – horizontal and vertical pathways

  • We try to determine the level of coherence
  • Questions regarding linkages to sustainability
  • Use causal pathways through the Theory of Change – coherence is one we try to examine
  • Multiple methods – determining coherence is more challenging, especially subjectivity in identifying some of the indicators, even where synergies are happening
  • Can be helpful to disaggregate and then develop a composite score
  • For each level, we use a different rubric for mapping
  • Determining national sector policy coherence is often much simpler, but determining the contribution of an agency/program can be extremely challenging