Team FIX IT
Creating an inclusive, easy to use tool for evaluators and local organisations to connect and build fair partnerships.
⛶ Full screen
COVID19 has disrupted traditional methods of data collection such as face-to-face surveys. Evaluators are resorting to various technologies such as mobility data, online survey tools ,etc. But in a country like Ecuador with low rates of smartphone penetration, it is difficult to solve this issue. Our team got together to find an innovative idea to help evaluators develop baseline data collection tools to evaluate economy reactivation projects in Ecuador.
Team FIX IT is here to fix into place new ways of conducting evaluation to address the challenges imposed by COVID-19.
The problem:COVID-19 has caused disruptions - but only temporary. New projects are emerging to work around the challenge that the virus has posed. For example, new organizations have sprung up to collect, buy and sell food and health supplies from local small producers to the people that need them. These new projects are instrumental to bring the economy back on track. The Evaluation Hackathon has challenged us to develop a tool to collect baseline data to evaluate such projects.
Our Goal: We got together to figure out how to help evaluators in Ecuador design baseline data collection tools to evaluate economy reactivation projects. We framed our challenge as:
How can we build a data collection plan to measure baseline indicators
So that we can work around restrictions/challenges imposed by COVID-19
To solve the evaluation needs of economic reactivation projects in Ecuador
Our process: We followed the design thinking process to tackle this challenge. Here is a snippet of the process which will also unravel the story we have created.
Who are the stakeholders?
While we identified several stakeholders, we decided to focus on two main ones and develop our personas.
Our primary persona is Sofia, an evaluator in Ecuador who wants to evaluate economy reactivation programs in Ecuador. She is facing several challenges due to COVID-19 that are making her data collection difficult including existing lockdowns, curfews across the country, risk of infections and participant recruitment in this environment. While many of her colleagues around the world are using digital platforms to get around these challenges, her projects involve stakeholders in rural areas of Ecuador where many people don’t have smartphones. Landlines in Ecuador are unstable which would further difficult the use of phones to collect data. SMS surveys are an option but cell-phone services are expensive in Ecuador and many participants might not be able to participate if they don’t have enough credits. Coming up with a solution for her baseline data collection needs will require additional funds that were not accounted for in her project.
Marco is a secondary stakeholder. He is a 50-year old farmer in Ecuador who is producing local food products. Due to the health and safety restrictions, he has found it harder to access suppliers and markets to sell his products. He has been offered the opportunity to partner with a friend and begin distributing masks in nearby communities but he is yet to find a way to access the new market with the current restrictions. Marco does not have a smartphone and cell-phone reception is unstable in his home. He is an active member of his community’s farmers association. Marco lives in the Andes, where many researchers have come over the years to study farmers’ livelihoods. He is wary of outsiders who often ask way too many questions and he is never sure what they do with all this information. He is less comfortable speaking with strangers nowadays as he lives with people who are high-risk of getting the virus.
The next step was conducting interviews.
We conducted several interviews with evaluators who were planning or had recently planned baseline data collections. We asked them what challenges they faced and what support they wish they had available to support them through this process. This activity was eye-opening and re-directed our initial thinking about the challenge. We found that many evaluators were being creative and using technology to overcome the data collection barriers imposed by COVID-19. However, in developing countries where there are additional challenges we heard using secondary data and collaborating with local organizations is essential. This led us to conclude:
- Using secondary data will be more efficient for evaluators (time and cost saving)
- Avoiding data collection will reduce infection risks
- Local partners have already established connections and trust with groups of interests. Establishing trust online, or through SMS or phone will be more challenging than in pre-COVID19 times.
- Local partners would have to rely less on travel to reach participants.
After conducting some research we found out that Ecuador was selected as part of an FAO global program to collect data about the implications of COVID-19 on the agricultural sector, livelihood of farmers, and provision of food products (FAO 2020). This sounded like the data our project evaluators would need to create the baseline measures for their evaluations.
At this stage, we redefined our goal.
After learning from our interviews we decided our focus should be on how can we coordinate existing data and evaluation efforts so that we can work around restrictions/challenges imposed by COVID-19 to solve the evaluation needs of economic reactivation projects in Ecuador.
We have assumed the following.
-Data collection in Ecuador is not coordinated. We did not find many open-source sites other than government and census datasets. In particular, we did not find a central source where current or future data collection efforts were listed.
-Key players have the capacity to collaborate with evaluators and other organizations.
-Evaluators are interested in getting their baseline measures in an efficient way even if this entails using secondary data.
Our solution ?
It is called EvalConnector. EvalConnector is digital platform that helps evaluators find and create successful partnerships in Ecuador.
EvalConnector will be a platform with information of INGOs, NGOs, State actors, Universities and evaluators including the region in Ecuador where they work, their policy area, activities, communities they serve and needs. Users will be able to filter the dashboard to find potential partners that can support their projects. Organizations’ profiles will also include a list of their projects, private datasets, and links to publicly available data they own.
Our initial prototype shows how different levels of filters an evaluator can use to identify an organization that aligns with their project.
Eval Connector will mainly benefit evaluators by:
- Making it easier for them to find partners that align with their projects
- Helping them identify existing data and avoid duplicating data collection efforts
- Facilitating their collaboration and learning from local experts
- Leveraging trust from local partners when data needs to be collected
- And facilitating fair partnerships where evaluators can offer alternative ways of compensating partners when funding is limited
The platform will include a specific section for organizations to include their needs which can include resources, training, capacity building, etc. A code of conduct will be developed with input from stakeholders with guidelines of fair and reciprocal partnerships. This will ensure that partners also benefit from the connections facilitated by our platform.
After testing our prototype, we came up with a visual of what the dashboard would look like.
The efficient implementation and use of our dashboard would change the way evaluators conduct work in Ecuador. We expect that by facilitating fair and efficient collaborations, our platforms would help evaluators face challenges that COVID-19 imposes in face-to-face interactions required during data collection in hard-to-reach areas. Limiting these activities is extremely important when working with vulnerable communities who have lower access to healthcare. Ecuador has been among the hardest-hit countries with COVID-19 and caution will be required for months or years to come.
The outcomes of our project include increasing access to private data that will support evaluations, increased use of existing evaluation studies, increased collaborations with local partners, evaluators seeing ‘competitors’ as partners, improved skills using secondary data, and increase use of local partners in data collection and validation.
Some of the challenges we would face include engaging stakeholders, the sustainability of the project and funding. We incorporated the following aspects in our plan to address these limitations:
- An engagement strategy- where key stakeholders will be reached out directly and informed of the benefits of joining the platform. We will select key stakeholders and use a snowball approach so the word spreads and other organizations join.
- An inclusive approach- We will make sure organizations with limited resources and those working with marginalized populations are included through umbrella organizations. Detailed user guides and webinars will be available to help organizations join the dashboard and upload their information. Volunteers will also be available to help organizations with limited resources over the phone.
- External funding- opportunities will be sought for the design and launch of the platform.
- Sustainability- Finally, we will reach out to partners like EvalYouth_LAC to develop a volunteer program where emerging evaluators can gain recognized experience for helping us update and maintain the dashboard.
- Our final project took an unexpected approach after learning what evaluators on the ground required
- Due to the infection risk of conducting data collection in the COVID-19 environment, evaluators might need to change their mindset about evaluations. We should ask ourselves if collecting our own data is the best way of evaluating programs. How can we work together to get over this crisis and support decision-makers to make timely decisions?
- COVID-19 has not only made data collection more difficult but has also increased the costs of conducting evaluations (investing in digital platforms, training evaluators to collect data through videos, calls or SMS, paying to access the appropriate platforms, increased time to build trust without face-to-face interaction). The economic crisis resulting from COVID-19 cannot be ignored, as evaluators, we can strive to find efficient ways of doing our work and ensure funds are used efficiently. We can also support local organizations by involving them and compensating them fairly. When funds are limited, we can be creative and ensure they benefit from our partnerships.
Our takeaways and learnings:
Whats's next? Here is a quick look at the next steps.
Resources that we used
FAO. 2020. Ecuador evalúa impactos de Covid-19 en el sector agropecuario con apoyo de la cooperación internacional. Retrieved from http://www.fao.org/ecuador/noticias/detail-events/en/c/1296791/