TECCHI’s webinar, “Contexts, Trajectories, and Assessing Progress Towards Goals”, gave rise to a lot of discussion on the topic of timelines and trajectories for understanding programs and their impacts. Below is a table of some insightful comments from different webinar participants as they engaged with the ideas being presented.
| “ex-post we find things that ‘fail’ at final, sometimes succeed later, tho’ most of the time those that succeed (and we ASSUME sustained impacts) ‘fail’ a couple of years later.” |
| “People seem to think that funders and program designers are foolish to look for simple short term outcomes. I think a case can be made that such behavior is rational and the most effective tactic for addressing important issues. |
| “the short-term is, in my view, irrelevant to our participants and partners who need truly sustainable development that’s locally driven/ sustainable (listen to Time to listen by CDA/Mary Anderson’s crew) |
| “I agree the short-term view is problematic. I’m only saying that there are good reasons for it. If we don’t appreciate those reasons, progress will never happen. |
| “Surely the problem Jon Doe, is the preoccupation by traditional evaluators on projects/programmes because that’s who pays us. If you focus on policy and social change then the whole logic if short term impacts becomes absurd! |
| “Frankly I think we need Theories of Sustainability designed by those tasked with sustaining the outcomes, not just donors ToC during fixed lifetimes. |
| “I have been thinking about this. It is possible to know some relevant aspects of context in advance, but impossible to know how context affects outcomes except retrospectively. |
| “measuring multi-dimensional facets of contexts and trajectories should not force to add and burden the questionnaire/respondents… |
| “context, people and system matter! great said! How do we ensure the linkages to craft our evaluation more practical in day to day application? The human centered evaluation could lead us further towards reality of contexts? |
| “I always ask the evaluators in training to apply the concepts that we are learning to a program that the learners are working on or passionate about. Situational and Interpersonal Competencies Domains are so important to evaluators. There is more than the technical, reflective and management competencies domains that evaluators need. |
Some of these comments shed light on the tensions that are present when thinking about timelines for impact. What are outcomes that we can expect to see in the short term? What is the nature of these outcomes, and how are they different from the long-term impacts that we hope to see in the future? How do short-term outcomes and long-term sustained impacts cohabitate in the same space of program planning?
When establishing timelines for impact, it is important to think about the assumptions that we are making for an impact to be achieved within a given timeline. It’s also perhaps important to reflect on how we go about creating these timelines in which we hope to observe positive change. Sandra Albert, one of the presenters in the webinar, hinted at how interventions are often planned around naive assumptions about the cultural practices in a given context. Given that contextual realities are often like murky water and there are multiple intersecting factors that play a role in the success or failure of interventions, how do we establish “reasonable” timelines for programs? Michael Woolcock, one of the presenters, spoke about the non-linear nature of trajectories and how evaluating an intervention at different points in time could lead to very different understandings on whether a program is successful or not. He also spoke about doing activities in obscurity before change can be observed. These actions often go unnoticed. Some of the webinar participants also mentioned how funding cycles are oftentimes built around achieving short-term wins. What is the value of these short-term gains? Perhaps, achieving these short-term wins that are less complex in nature could help create momentum and generate excitement about a project, and also bring a sense of hope that change is possible. However, it is important to balance this view with a lens of sustainability and helping communities develop the capacity to sustain impacts even once the official funding runs out. Given that observing change takes a lot of time, how do we go about creating theories of sustainability from the onset of a project?
Moving from sustainable impacts and outcome trajectories, to learning systems, I thought that the comments from the webinar participants on “human centered evaluation” and not “burdening” participants with completing questionnaires, quite intriguing. How do we make the process of giving feedback something that participants want to engage with? In order to feel motivated and empowered to provide feedback, and engage in the process of learning, program participants should feel a sense of ownership of the program, and should feel like the input that they are providing will get fed back into the project, ultimately making their experience interacting with the program more pleasurable. As we engage with these questions, it might be beneficial for program implementers and evaluators to collaborate with individuals across other fields such as design, urban planning, psychology etc. to understand how they engage with processes of continuous learning around the interactions of humans with their environment.
