Download the full technical guide to designing and implementing a FinNeeds survey or module.
Once you’ve chosen which data source(s) to use, how do you go about gathering the data? This section gives an overview of the practical steps to follow for each data source, with an emphasis on demand-side data.
The quality of FinNeeds insights largely depends on the questions that people answer and how well these questions are understood. Good survey practice requires that the questionnaire not take longer than 30-40 minutes to administer and the scope of questions should be tailored accordingly, as relevant to the local context.
The first step in structuring a dedicated FinNeeds survey is designing a module on each of the four financial needs. Each need module considers the incidence of use cases in that need category, the devices used towards each need and, where relevant, indicators around the recency and extent of the use cases experienced.
The use cases to consider under each need differ depending on the country context. It is therefore important to design the questionnaire with the local context in mind.
The device options listed per use case are determined based on the FinNeeds device taxonomy included in the technical guide, so that they can later be labelled as devices from either different product markets (credit, savings, payments, insurance) or provider categories (formal, informal, social, personal). So, for example, on the use case “providing for my children’s education”, the list of devices may include:
The first two options would classify as credit, the final three as savings. The first two would be social devices, the third and fifth a personal device and the fourth a formal device.
The use cases are followed by a usage module to track usage metrics for payments, savings, credit and insurance devices (considering frequency, recency, value and duration as relevant), as well as a drivers module to ask self-reported reasons for device choice.
Where a FinNeeds module or questions are included in a broader financial inclusion survey not dedicated to FinNeeds, the full scope of questions cannot be asked. Here, it is important to take stock of the financial inclusion policy objectives in the particular country to determine priority use cases, devices and drivers to explore.
Once the questionnaire is designed, it is rolled out using standard sampling methods. Using the correct sampling approach is important to ensure the results can be weighted to be representative at the national or regional level, depending on the scope of the survey. Demand-side survey data may be collected either through in-person interviews or via an SMS or other electronic survey:
The traditional approach to collecting demand-side data is through an in-person interview. The questionnaire is administered by an interviewer who captures the responses on each question. The face-to-face interaction enables the interviewer to get visual cues which may be helpful in probing for more accurate responses. The interviewer may also translate questions in cases where a FinNeeds module is presented in a language that is different from the respondent’s first language.
When is it appropriate? The advantage of in-person interviews is that they can increase the validity of responses through intervention by the interviewer in cases where the respondent fails to understand the question asked. The major limitation of face-to-face interviews is the high cost of implementation.
To ensure that questions are appropriately phrased in the local context and will solicit meaningful responses, it is important to pilot the questionnaire and, where possible, include cognitive interviewing of respondents.
What is cognitive interviewing?
Cognitive interviewing is a survey piloting technique that explores how people interpret survey questions. Once a question is asked and a response captured, in-depth probing of the question and answer provided helps to determine what people think is being asked of them and if the answers that they give are appropriate responses to the question being asked.
Cognitive interviewing is a best-practice technique, ideally used when developing new surveys or new survey questions, as it reveals what people believe is being asked of them and where misunderstandings could occur. Cognitive interviewing can dramatically reduce survey error created by questions that are unclear or open to interpretation.
The proliferation of mobile phones makes data collection via the mobile phone (through self-complete or interviewer-administered surveys or through a combination of both) an appealing option. Mobile phone-based surveys can be achieved through sms, calls, social media, or over websites. Mobile-data collection, especially the self-completion method, is significantly cheaper than face-to-face surveying, but it comes with many of its own limitations. We focus on SMS surveys based on recent insight2impact experience on collecting demand side data using mobile phone messages.
Typically, an SMS survey is structured into a limited number of targeted questions aimed at gauging insights on a specific topic. The first SMS establishes contact and notes that consumers will not be charged for their response SMSs. Each question then provides a simple menu of response options, often structured into “yes/no/don’t know” options.
When is it appropriate? The main advantage of an SMS survey is that it is cost-effective to implement. While a face-to-face interview in 2018 cost on average $60 USD per respondent to complete, an SMS survey could be implemented at only $5 USD per respondent. The limitation, however, is that mobile surveys are only able to reach those people who have mobile phones and are literate enough to complete a self-completion survey. It is also very easy for people to opt-out of completing an SMS survey as there is no social pressure placed on them by an interviewer standing with them and asking them to continue. This means that SMS surveys have to be short and engaging. Further, the SMS platform only allows for 160 characters per SMS – meaning that the questions themselves must be short and simple. The use of cognitive testing is particularly valuable to the design of SMS surveys. It can assist with ensuring that, in simplifying the ways in which we ask things, the overall meaning is retained, and people are able to give answers that correctly represent their lives, behaviours and attitudes.
Innovative and sustainable data collection
Qualitative research, such as in-depth individual interviews or focus group discussions, are a good way of probing underlying needs, perceptions and drivers of observed behaviour. Financial diaries apply qualitative techniques to build up data on a particular household’s financial behaviour over time. It is particularly well-suited to identify FinNeeds outcomes and usage, as they track actual money in and out of the household budget on an ongoing basis. Either of these techniques can be implemented in a targeted way to help answer a specific question related to the FinNeeds framework.
In addition, behavioural experiments can be used to render objective insights on triggers and drivers of usage behaviour. While not strictly speaking qualitative, such experiments do not produce representative data and therefore stand apart from survey techniques.
When is it appropriate? Qualitative techniques cannot produce representative findings. It may also not be possible to cover the full breadth of the FinNeeds framework in a single qualitative instrument. Though indicative only, qualitative techniques can give richer insights than would be possible in a survey. Thus, they are ideally rolled out in combination with quantitative data sources. For example, qualitative techniques may be used to form a better understanding of how perceptions and societal worldviews help to explain the drivers of observed usage behaviour.
Read our blog on innovations in qualitative research for an overview of qualitative techniques here.
The first step in collecting transactional data is securing buy-in from the financial service provider or regulatory authority. The next step is signing a non-disclosure agreement to set out the conditions for use of the proprietary data. Following this, it is important to engage the technical team at the financial service provider or authority to understand the nature of the data: the available datasets across different financial products and, for each dataset, the available fields (types of transactions, demographic data) and the most appropriate customer segments for which to extract data. An approach must also be agreed for anonymising the data via the allocation by the provider or authority of unique client identifiers in lieu of customer names.
Next, the financial service provider or authority will be asked to extract and share the data. The exact mode for data access will depend on the context and policies and procedures of the participating institution and must be negotiated with the participating financial service provider or authority. For example, in one of our pilot studies, the participating provider extracted, encrypted and shared a representative sample of credit card, debit card, loan and insurance customers’ transactions data for a 12-month period. Note that, depending on the ruling data protection policy or regulation, it may not be possible for the provider or authority to share the data externally. In this case, as in one of our pilot studies, the data analysts have to spend time on the provider or authority’s premises to analyse the data on their server.
Download a generic terms of reference for engaging with financial service providers here.
Learn how to apply the #FinNeeds approach to measuring financial services usage
Sign up here for our monthly Data Briefing on data and innovation in the financial services sector.
And, to receive insight2impact updates, subscribe to the insight2impact quarterly newsletter.
Please write to us and give us your feedback. info@i2ifacility.org
