 |
|
A starting step in the evaluation is having clarity on what constitutes the intervention. Questions to raise include: What are the key components of the intervention? How stable are these components? Is the intervention likely to change over time? A complex intervention with very many components that change over time may need a very different evaluation design than a simple intervention that is stable over time. |
|
|
 |
 |
|
Fundamental to the evaluation of complex interventions is developing some initial idea of how an intervention is expected to work. Specifically, how will it impact outcomes? The framework that links program activities to short and long term outcomes is referred to as a theory of change. What is the relationship between the processes that constitute the intervention and the sequence of short term and long term outcomes? Under what context is the complex intervention likely to work? What mechanisms are needed for the intervention to prosper? Is the intervention likely to have very heterogeneous impacts for different groups under various contexts? Questions abound on developing the initial program theory. Given both the complexity of the intervention and the uncertainty of knowledge that initially exists, part of the focus of the evaluation needs to be sensitive to development of an emergent theory of change for the intervention over the course of the evaluation. |
|
|
|
 |
|
Even though many social innovations can be (by definition) new, the reality is often that there exists an evidence base of how “similar” interventions have done in other fields and in different contexts. The evaluation’s purpose can also be to review the literature to learn more about the pathways of change as well as contexts and mechanisms that support the change processes: to provide an evidence base for the program theory. A recent method called realist synthesis focuses on a review of the evidence base for underlying mechanisms of change and if a mechanism operates differently in differing contexts to help explicate the theory of change for complex interventions, rather than focus on “average” level effects of interventions. |
|
|
|
 |
|
It’s important that the evaluation be informed by and also develop knowledge of the anticipated timeline of impact of interventions. The state of knowledge of most social science theory is such that information on anticipated timeline of impact for interventions is often missing. One approach is to engage stakeholders who have been involved in prior interventions to help explicate such an anticipated timeline of impact. Understanding what outcomes are likely to be impacted and when such impacts are likely is important information that the evaluation can help generate. |
|
|
|
