Reaching women in Afghanistan to ensure their voices are heard in evaluations
⛶ Full screen
Team Members are: Audrey Wai Siong See Tho, Bob-Uchenna, Luce Simeon- Bulosan, Paola Fabbri, Sophie Gullivan
Our Team Hacked the Challenge:
How can we design effective and innovative evaluation processes, methods and tools for women who are hard to reach in Afghanistan in order to solve the problem of evaluations not adequately including and addressing women’s voices and concerns in evaluations.
Our design team sessions are virtually interactive and fun with each member bringing in his/her own expertise and contributing to the discussion. We finally found a name for our team and on days 2, 3, and 4 had very enlightening virtual workshops where we came up with lots of ideas for our solution,
Following ideation, prototyping and testing Team ReachMe! came up with an innovative solution to address our challenge. We call this the ReachMe! model:
Here's a video of our team explaining how the ReachMe! model effectively addresses the challenges of including hard-to-reach women in evaluations, using Afghanistan as a case study.
The “Reach Me!” Model
OUR UNDERSTANDING OF THE CHALLENGE
The essence of our challenge was how to ensure that evaluations adapt in a way that ensures equity, inclusiveness and “do no harm”. This requires adaptation and innovation in methodological approaches and offers an incredible opportunity for innovation in the sector.
HOW WE NARROWED DOWN THE SCOPE OF THE CHALLENGE AND WHY WE CHOOSE AFGHANISTAN WOMEN
When we speak of hard to reach populations, this can be applicable in all countries either economically, geographically or socially. However, we chose the women in Afghanistan to narrow down our focus and as a case study since they represent a major number in terms of population (50%) and yet they are vulnerable, marginalized, and systematically excluded by the conservative and conflict-ridden Afghan society.
Their voices have to be heard and amplified but one must also be cognizant of the cultural barriers that Afghan women confront to make sure that the conduct of evaluation does not exacerbate their already fragile and complex situation.
As we decided to focus only on one of many marginalised groups, we decided not to choose a specific sector of intervention (such as education, water, health etc...) or a particular project. We wanted to create a model that is particularly helpful to reach women and can be applied to many different types of projects.
Over the past 10 years, significant technological advancements have occurred in Afghanistan. In a study made by USAID in 2012, they have found that 80% of women have access to mobile technology, either directly by having a phone themselves or through a member of their family. This changes the stereotypical perception that it is difficult to have access to women. The use of mobile technology by a significantly large population of women, whether in urban or rural areas can be harnessed so that evaluators can reach out to more women and include them not only as respondents but as part of the entire process of evaluation.
As we did not choose a particular project, we listed all stakeholders that would be normally involved in projects addressing women.
We see women as both beneficiaries and stakeholders. We broke down women in different categories: Living in urban or rural areas, level of education, employed, students, age groups, different ethnic backgrounds and languages.
Men. They are essential in the process of reaching women. Dialogue with men is essential in order to be able to reach women.
Health workers. They continue to have access to women even during lockdown.
Implementing agencies. They continue to implement projects during lockdown with extreme challenges, adaptation, and risks.
Donors. They are commissioning and reviewing evaluations during COVID 19 period.
Evaluators. They had to face unprecedented challenges in the design and implementation of evaluations.
- Although there are differences (based on geographical areas (central-periphery), conflict-affected provinces, income level, education level etc...) women tend to be a vulnerable group in Afghanistan because there are a systemic marginalisation and exclusion embedded in the societal structure
- COVID and lockdown has worsened GBV situation and it is more difficult for victims to seek and receive help
- Women have access to technology. This was discovered after researching, interviewing, and testing. In fact, our first assumption was that women have no access to technology. We found that assumption to be wrong and made adjustments in our model accordingly.
- We also found that internet access in Afghanistan is reliable enough to allow the use of mobile technology for data collection. This is also an assumption that was tested through researching, interviewing and piloting. However, this needs further testing specifically for rural areas
- The current lockdown makes it impossible for evaluators to collect data in person
- Language and in depth understanding of cultural norms and context are crucial
- Gender dynamics in evaluations make it more complex to reach women, especially by male evaluators
- There is limited local evaluation capacity in the country. Evaluation often makes use of external capacity because the general assumption is that local capacity is not up to international standards
KEY ELEMENTS OF THE “REACH ME!” Model
FLIP THE SCRIPT OF EVALUATION
The COVID 19 crisis presents us with an incredible opportunity not only to overcome the many challenges that evaluators are presented with, but also to really transform and improve the way we conduct evaluations. We want to FLIP THE SCRIPT OF EVALUATION and transform it into an inclusive, empowering, and engaging process that develops local capacities.
Right now evaluations tend to be donor-driven, led by international experts, fast, data-oriented, "extractive". The lack of access to beneficiaries has forced evaluators all over the world to think outside the box and find new methods to conduct an evaluation. Let's not just stop there! Let's take this opportunity to design evaluations that put the beneficiaries at the centre of the process. Let's flip the script from the inside out!
Our Theory of Change is that
If we FLIP THE SCRIPT of evaluations from the inside out, we can meet COVID 19 challenges and at the same time create innovative, inclusive, evaluations that contribute to building local capacities, because: 1) We shift the focus on women putting them at the centre of the evaluation; 2) We develop local capacities through a learning by doing and mentoring approach, 3) We have more reliable, quality, and ethical data.
This is the essence of our approach:
- Inclusive and focused on beneficiaries: Evaluations can only be effective if they adapt the methods to make sure that women are at the centre of the evaluation process.
- Engaging: Women should not only be consulted, they should also play a role in shaping the evaluation, collecting the data, and analyse the data.
- Developing local capacities: Limited availability of local expertise is a major challenge in Afghanistan. We know that local expertise is crucial for effective and quality evaluations. We can conduct evaluations that contribute to develop local capacities, build on existing expertise and create opportunities, especially for women to develop skills and experience.
- Empowering: We want women to feel seen, heard and empowered through the evaluation process.
HOW WE DO IT:
- We ask women how they want to be involved in the evaluation. We take the “Do No Harm” principle very seriously. We want to ensure that: 1) We don't create extra burden on them (one of our findings is that women are currently overburdened), 2) we don't expose them to additional risks (inside or outside the household). We want to find a way to engage them that is both effective and empowering for them. We want women to be heard.
- We build on local capacities and we invest in women’s capacity for evaluation: We build on local capacities adopting a training-by doing method. We train evaluators and enumerators on the chosen data collection methods. We work with them using a mentoring approach with an iteration of training, piloting and testing data collection, looking at the data collected, providing feedback and guidance for further data collection and/or make necessary adjustments. We do the analysis together. We share the analysis with local enumerators and evaluators. We get their inputs to deepen and validate the analysis. Finally, we share the evaluation with women (beneficiaries) and we ask them for their feedback and inputs. We invest in women’s capacity for evaluation (especially those that have lost their income during COVID 19) and we train them as enumerators or evaluators, so that they can get new skills and earn some income.
- We use existing structures: Even during lockdown women are accessing some essential structures such as health clinics and midwives. We make use of these structures that women are currently accessing to gather data.
This way we are confident that we can transform evaluations into truly inclusive learning experiences.
METHODS/TOOLS USING THIS APPROACH
Eighty percent (80%) of women in Afghanistan have access to mobile technology. The “Reach Me!” model will make use of the existing technology and local capacity and structures to the maximum extent possible and adapt three methods to gather information. To the extent possible, we use more than one method of data collection, to triangulate data and ensure validity and accurateness of information.
- Focus group discussions. Instead of conducting focus groups face-to-face, we use social media platforms like WhatsApp and Facebook group chats to customize online group discussion that can be moderated in real-time. Moderators will be required to record and document sessions for transcription using the conventional sample size of 12 for each group.
- Interviews and Surveys. This will rely on existing structures that women access: i.e. Midwives and female social and health workers who can be trained as enumerators to collect information via phones and SMS. These existing structures should have the advantage of gathering data when only essential services are open. Enumerators can also reach respondents from the comfort of their homes creating the possibility to reach larger samples in less time. Selection of respondents will be based on network coverage and only respondents with phones will be reached.
- Closing Feedback method/loop. This will create a platform to interact with women and address major concerns through traditional radio shows and Facebook live shows, enabling them to call in and give feedback.
HOW DOES IT FIT INTO THE CRITERIA?
Level of Innovation: The “Reach Me!” Model is developed to initiate and stimulate a deeply transformative approach to evaluation. We are not just adapting evaluation we want to FLIP THE SCRIPT of evaluation!
Scope of Impact: The ”Reach Me!” Model hit the most crucial aspects of the challenge: Inclusion, reliable data, innovation and has an extremely high potential to address the challenge of including women in evaluation in Afghanistan. The model makes use of women capacities and networks while using technologies they are already familiar and comfortable with. The number of women that can potentially be reached with this method can grow exponentially. After further piloting and testing, this model can be applied in different regions of Afghanistan, and eventually (with appropriate adjustments) in other countries with similar network coverage.
Viability: The “Reach Me!” Model addresses several limitations: challenges in reaching women, and lack of local capacity for evaluations. It makes use of available and easy to use technology, but it also identifies other possible mechanisms (radios and health workers) to complement the collection of data.
Social value: The “Reach Me!” model has an extremely high social value. By flipping the script of evaluations, we put women at the centre of the evaluation. They are not only consulted, but they are also engaged, trained and empowered.
The “Reach Me!” Model is for you if:
- You want an effective method to conduct evaluations during COVID 19
- You are serious about participation and inclusion
- You take Do No Harm extremely seriously
- You want to build local capacities
- You are serious about learning from evaluation
- You believe in innovation
Does that sound like you? Then you should use the “Reach Me Approach”
The “Reach Me!” Model requires from the commissioner:
- Senior evaluators with learning and mentoring experience, who believe in the importance of building local capacities and know-how to work with enumerators and local consultants remotely.
- Adequate amount of financial resources needs to be invested in the learning and mentoring element.
- Technology. Some tools may have to be distributed to enumerators (for instance smartphones)
What does the donor get in return?
- Reliable data that are inclusive and ethical
- High return on investment because the evaluation will not only produce reliable data, it will also build in-country expertise