Category: Webinar 3

Fred Carden

Coherence: A Principles-Focused Approach

• What is Coherence?
  • OECD 1991 – initial focus on policy coherence
  • European Commission 1992 – put into law
  • Focus has been on global issues
  • Enshrined in SDGs
  • No accepted definition
  • OECD defines coherence as: an approach and policy tool to systematically integrate the economic, social and environmental dimensions of development at all stages of domestic and international policy analysis.
• What is Coherence in a Country Program?
  • Development, Trade, Diplomacy – how are they coherent?
  • There are different levels of coherence: 1) Internal Coherence; 2) Intra-Organizational Coherence; 3) Inter-Organizational Coherence; 4) International Coherence
  • Focus of Global Affairs Canada in Columbia case was on first two levels of coherence
  • We first looked at the notion that coherence is not only about policy – what does coherence mean in the context of programming of global affairs in Columbia?
•  Levels of Coherence
  • Policy Coherence
  • Organizational Coherence
  • Administrative Coherence
  • Knowledge Coherence
  • Informal Coherence
• Measuring Coherence
  • Developed four-level rubric – no coherence to coherence
  • Breaking it up into different elements very helpful in discerning what was/was not coherent – Example: coherence across international assistance; coherence across international engagement
  • Found that there was pretty good policy coherence, but administratively weak

Lori Bell

• In the context of UNHCR commissioned evaluations, coherence is always looked at through the lens of relevance – are we doing the right thing?
• Presentation provides concrete examples of evaluations where coherence was looked at from an internal and external point of view

System-wide evaluation of cyclone Idai Mozambique 2019

External Coherence

• Complexity, crowdedness of the humanitarian space
• The divide and conquer of different agencies
• How did we live up to our responsibilities and roles compared to what other actors did?
• If you want to look at External Coherence, you can’t do it on your own – we need to understand the comparative advantage of UNICEF, etc.
• We need to know a lot about other agencies operations and what the government is doing
• Inter-agency joint evaluation, rather than a single agency evaluation

The Importance of Personalities, Perceptions, and Culture

• Coherence boils down to individual behaviours and decision-making on the ground
• We need to look at culture and perceptions

Elliot Stern

Aspects of Policy Coherence

• ‘Policy Coherence’ in EU and OECD
  • Coherence now a “new” (2020) OECD criteria, but entered European policy vocabulary in 1992 Maastricht Treaty
  • Distinction between internal and external coherence – i.e., consistency of elements making up a policy action vs consistency between different policy actions (this distinction may be less clear-cut than 10 or 20 years ago)
• 4 Aspects of Contemporary Policy Coherence
  • Unit of analysis
    • Growing understanding of complex and interdependent policy priorities (e.g., innovation, climate change mitigation, equity and justice)
    • Extending the Unit of Analysis: i) decontextualized policy interventions (coherent with policy inputs); ii) contextualized policy interventions (coherent with policy/non-policy inputs/contexts); iii) interdependent policies interventions (coherent between multiple policies in multiple contexts); iv) policy systems (many coherence possibilities)
    • Systems Mapping for complex policy evaluation
  • Coordination and governance – Coherence doesn’t happen on its own!
    • Multi-part and separate but interdependent policy interventions must be aligned – importance of coordination and governance arrangements
    • Success often depends on pre-existing community, sectoral/territorial networks; credibility of system leaders; capacities to use information and adapt; participatory engagement of stakeholders
    • Coordination and governance itself must be evaluated – part of ToCs, etc.
    • Encouraging those in governance and coordination roles to reflexively learn and generate and use M&E data a way of strengthening coherence
  • Timing: Re-setting the evaluation clock
    • Move away from short-run, ex-post evaluation at end of program/policy cycle to integrate evaluation into program planning and “up-valuing” mid-term/ongoing evaluations
    • Ex-post evaluations are moving long-term – look back over longer time scales on rolling basis
    • Mid-term evaluations increasingly inform reprogramming and “steering” and adaptive management
  • Importance of substantive theory
    • When we move from narrow focus on programs to recognize importance of complexity, context and interdependencies, it changes where we look for theory
    • Program theory is not enough
    • Becoming standard in UK and EU structural funds to look to academic research to identify building blocks of Theories of Change (e.g., Innovations Systems Theory)
    • Centrality of literature reviews and partnerships with domain experts

Vijayalakshmi Vadivelu

UNDP’s programming:

• In past 3 decades, programs have looked at coherence in different program areas – support to livelihoods, peace building, etc. require synergies – intersecting elements to enhance coherence at the country level
• UNDP’s consistent emphasis on coherence and partnerships – implemented with a range of partnerships – integrated role within UN system
• Programming at national and subnational levels enables synergies/linkages

Coherence a key dimension of program performance

• Assessed at multiple levels: Internal, External, Intra-organizational
• Mapping of areas where maximum synergies are possible 
• Also look at intersecting areas in response to crises

External Coherence      

• UNDP collaborates with other actors to contribute to humanitarian response
• Determine pathways that link, map key actors working in the area, identify intersecting process indicators and strength of partnerships
• In bringing together different actors, coherence assessed using sector data
  • Increasing household income, increased investment = indicators for policy coherence

Different types of Coherence – horizontal and vertical pathways

• We try to determine the level of coherence
• Questions regarding linkages to sustainability
• Use causal pathways through the Theory of Change – coherence is one we try to examine
• Multiple methods – determining coherence is more challenging, especially subjectivity in identifying some of the indicators, even where synergies are happening
• Can be helpful to disaggregate and then develop a composite score
• For each level, we use a different rubric for mapping
• Determining national sector policy coherence is often much simpler, but determining the contribution of an agency/program can be extremely challenging