Adaptive Solution Team
How should an evaluator respond to unexpected shocks? We have devised a framework to guide you through the process
⛶ Full screen
1. The problem
Unexpected crises like the COVID-19 pandemic, wars and natural disasters pose a challenge to everyone, evaluators included. When facing shocks like these, they must ask themselves questions like the following:
- How does this crisis affect the assumptions of my Theory of Change?
- Do data access limitations force me to modify the scope of my evaluation?
- Does the crisis force me to change my research question or hypothesis? (related to the previous question )
- Do I have to change my data collection strategy and the logistics of the implementation of my pilot?
The challenge of revising an evaluation and finding the answers to questions like these can be tackled based on intuition alone, but this risks missing crucial steps due to lack of a systematic approach. That is what our solution has to offer.
Eval Hack Members – Nqabutho Nyathi, Norman Rodriguez, Paul Maluful, Edward Williams, Michael Stephens
2. The framework
We have called our framework 'the four Rs' (4R.) These four Rs are:
- Rethink – Rethink the evaluation system/process
- Remote – Since it will be difficult to physically obtain data, remote data collection must be considered (that does not mean only online forms and surveys)
- Review – Evaluators must be able to review and synthesise existing data and knowledge (includes conventional hand-collected data and big data)
- Report – Evaluators need to prepare informative reports to their stakeholders (what has changed as a response to the crisis, why, and what to expect next)
The approach can be visualised like this (the second circle corresponds to BeetterEvaluation's rainbow framework, which is ideal to use in tandem with the 4Rs):
3. Data gathering alternatives
As part of your adaptive process, you might want to reach those populations that are far apart or aren't very interconnected. Some good options are the following (click on the notepad at the top of each box to see a description):