Note from Tanzania: Poverty Tools Testing, The Latest Chapter

Download the pdf version of this Note.

At the end of a demanding month of work, interviewers (from left) Marpa Tarima, Aika Edwin, and Phiona Ngowi wait to board a ferry at Lake Victoria, commencing a difficult 2-day trip back to Dar es Salaam.

Practicality testing of USAID’s Poverty Assessment Tools continues to reveal unique challenges on the ground. 

The latest chapter in the testing of USAID’s Poverty Assessment Tools comes from FINCA Tanzania (FT). Made possible by USAID’s Accelerated Microenterprise Advancement Program, the Poverty Assessment Tools project aims to create tools that gauge the poverty level of microfinance recipients and specifically differentiate the poor from the very poor, according to project definitions. The University of Maryland’s IRIS Center developed these tools and enlisted the assistance of microfinance and business development organizations to conduct the second phase of testing in the project: that of practicality. IRIS assigned each participating organization a specific tool and asked them to answer the following question: Is this tool cost-effective and easy to implement while maintaining accuracy? Future data analysis by IRIS will answer the question of accuracy, experience of the organization. For FT, assessing cost-efficiency did not pose a problem, but judging ease did. With commendable candor, Megan Gash explains some of the obstacles, missteps, and lessons learned from the testing process:

Even when a survey tool is short and simple to use, the method used to test the tool can make judging the ease of implementation a long and complicated process. FINCA International sent individuals from its headquarters in Washington, D.C. to carry out the project instead of using in-country staff as a way to improve upon techniques in its own poverty assessment methodology, the FINCA Client Assessment Tool (FCAT). FCAT consists of a client survey recorded with a Palm-Pilot that collects information on client demographics, outreach to the poor, business profitability and wealth creation, and clients’ social well-being, and then uses the data to calculate a client’s household poverty level.
Although using outside staff may come with advantages—such as a fresh perspective on the project, preventing current staff from being distracted from other works in progress, and maintaining closer management over implementation—we found it also brings challenges.

Client Overexposure
The biggest challenge we encountered during interviewing was the hazard that we like to call “interviewing fatigue.” The marketing team at FT conducts customer service surveys and focus group discussions on a regular basis, which we assumed would have familiarized the clients with the interviewing process and facilitated the exercise overall. Unfortunately, their experience created the opposite effect. Many clients claimed they were tired of giving their personal information and opinions and were reluctant to do it again. One banking group refused to be interviewed at all. To help encourage participation, we provided snacks during their meetings, but eventually some clients began demanding these treats regardless of interview participation. We soon ceased our “incentive program” and realized that we would need to visit more village banks than expected to fulfill our projected sample.

Risks in Outsourcing
We hired an employment agency to assist us in retaining project interviewers. Instead of expediting the process, this decision prolonged it. We were misled about the interviewers’ prior experience, their wage rates, their relationship with the representative, the representative’s fee, and receipt of payment for the services. We also encountered difficulty when dealing with the representative’s supervisor. Additionally, the man hired to translate the survey from English to Swahili managed to squander three weeks of our time before completing the job. Although the interviewers’ linguistic skills and cultural knowledge later proved to be an invaluable asset, we realized that much closer screening of outsourcing staff should be exercised in the future.

The Power of Probing
Lacking experience in interviewing, our surveyors did not fully understand the importance in probing clients for answers. When asked for estimations of household clothing expenditures and values of dishes and metal pots, one interviewer claimed that clients said they “don’t know how much and don’t want to lie.” Whereas two interviewers probed the clients for estimates, the third interviewer did not, resulting in the elimination of almost 30 percent of all surveys completed. We learned to stress probing not only in interviewer trainings but also in quality control checks throughout the data collection process.

Among the insights gained from probing include: some clients’ hesitation in giving us personal information stemmed from a fear that FT would go to their homes to verify their answers, or that the project staff would rob them of their claimed belongings; and some clients who claimed nine months as the “number of days ill and unable to participate in normal daily activity” were actually referring to pregnancy. Additionally, when asked “who is the main income earner in your household throughout the year,” one client became confused by the term ‘main income earner’ since in Swahili the same phrase also means OB/GYN. She responded by asking how the doctor could be considered part of her household since he works in the hospital and does not live with her. Our experience provides further support for the saying, “it never hurts to ask.”

Travel Planning Pains
Our sample plan required the interviewing team to travel to northern Tanzania to conclude the final rounds of interviewing. To cut costs, we opted to take buses and ferries to get us to our destinations instead of flying. Unfortunately, this experience included a non-air-conditioned 33-hour bus ride to the first city, an 8-hour ferry ride to the second city, a 4-hour (but precarious) return ferry ride to the first city, and a 27-hour return bus ride to Dar es Salaam. The route to and from included a not-so-convenient detour through Nairobi, Kenya, where the team incurred visa fees at the border high enough to close the savings gap in land versus air travel. Even though the trip included some fond moments, such as laughing along with the crowd while watching “The Gods Must Be Crazy” on the bus TV, we made a mental note to be more vigilant when making travel plans.

When writing our final report and responding to IRIS’s request to judge the ease of implementing the tool, the FT practicality testing team responded with a qualified answer. Although we accomplished all testing requirements and gathered useful data, using in-house staff could certainly ease the process.

Further information on USAID’s Poverty Assessment Tools project can be found at: www.povertytools.org.