What is a Complex Intervention?
The physicist Seth Lloyd has raised several questions as the critical dimensions in thinking about complexity:
- How hard is it to describe?
- How hard is it to create?
- What is its degree of organization?
We hope to dialogue about these dimensions of complex interventions with a number of varied programs in different settings including arts, citizen engagement, substance use treatment, international development, public health and health care.
Many complex interventions have the following features:
Multiple components: These components may or may not interact over time.
Heterogeneous mechanisms: Within the same intervention, different individuals (or communities) might receive different services. The services are tailored to local contexts and driven by an understanding that different mechanisms might be needed to activate the intervention under differing contexts and settings
Dynamics: The components and configurations of the complex intervention might change over time.
Nonlinearity: small actions can stimulate large reactions
Emergence: effects can spontaneously appear in a system
These properties explain why it is difficult (or even impossible) to initially describe a clear theory of how a complex program would work over time.
The two theoretical approaches that come closest to understanding the characteristics of complex interventions are developmental evaluation (developed by Michael Quinn Patton), and realist evaluation (developed by Pawson & Tilley).
Developmental Evaluation
Key areas of difference between developmental evaluation and ‘traditional program evaluation tendencies’, according to Patton, include purpose and situation, focus and target of evaluation, modeling and methods, roles and relationships, evaluation results and impacts, approaches to complexity, and professional qualities of the evaluator. Much of developmental evaluation focuses on the challenges of evaluation when there is no blueprint of how programs are likely to work at the start of an evaluation.
Developmental evaluation does not treat bringing clarity to an initial lack of program theory purely as methodological problem. Instead, it is careful to describe methods as only one aspect of developing clarity of programs over time (developmental evaluation encourages a reflective and non-technical view of methods, and is non-prescriptive). Key to the developmental evaluation approach is to build relationships between the program and the evaluation teams. Another key point is that evaluation design needs to respond to the complexity of the program. As example, uncertain knowledge of causal pathways of complex programs increases the likelihood that initial theories of the program are incomplete. This, of course, implies that the evaluation needs to be sensitive to “learnings” about the theories of change of the complex program over the course of the evaluation.
Realist Evaluation
The realist evaluator’s focus is on identifying the context-mechanisms-outcomes configurations that might be responsible for a program to have its impacts. Programs change over time and depend critically on the context in which they are implemented.
As in developmental evaluation, an understanding of principles of complexity is key, and the focus is less on technical methodological ideas and more on relationship building and understanding of the program contexts. Within a realist perspective, key evaluation tasks include: involving stakeholders and understanding their reasoning about how a program is likely to work; paying attention to how the program changes as stakeholders come to understand the program over time; better understanding the context in which programs are embedded; understanding how the program evolved as a result of negotiation among stakeholders; and developing impact evaluation designs that pay attention to the context-mechanism-outcomes configurations.