As you might remember, I wrote two blogs about alternative data, as an effective way to better score people with no or only a small credit history. One was “Psychometrics in credit Analysis – threat or promise?” where I reflect on a presentation by the Entrepreneurial Finance Lab (EFL), a US alternative credit scoring company spun out of Harvard University. Please find a one pager on EFL and on their products. The other one was “A US study: Utility and telecom payment data predict loan repayment capacity!” Since my time at CGAP I think the topic is super exciting and opens a lot of doors to significantly increase well-structured lending to low-income population segments.
To remind everybody: EFL uses psychometric data to develop credit score creating creates a deep quantitative understanding of individual risk and opportunity in small business (MSME) and consumer financing.
On the EFL Website, you find some exciting information, e.g.
- that Equifax (EFX) set out to determine if EFL could add value to its MSME scoring in one of their biggest markets in Latin America: Peru. EFL proved the ability to reduce default rate by 50%!!
- With GMG, a leading Peruvian electronics retailer targeting lower and middle income consumers, they sought to expand its credit offering to the un-banked. Result: EFL enabled a 35% increase in approvals with no change in default rate
- In a Project with BTPN, Indonesia’s 4th largest MSME lender, EFL was to better leverage credit data analytics and streamline its credit screening processes. Result: EFL enabled a 79% reduction in end-to-end turnaround time
However, I might also have expressed my points probably not clear enough or might have been on occasion too critical or pre-occupied with regards to alternative data. I have received comments from EFL which I would like to share with you.
Please find one by Bailey Klinger, EFL from June 2 2015 on “Psychometrics in credit Analysis – threat or promise?” below. I have not convinced him about becoming a guest Blogger yet, but I am very tempted:
“Let me share with you and the forum some lessons 5 years after moving from my offices in Washington & Cambridge to the trenches of implementing solutions for financial inclusion, because I think they are relevant to this posting, which is falling into some of the same pitfalls I myself fell into.
We created a psychometric credit scoring tool because of the shortcomings of existing approaches. In our focus markets, credit bureaus covered barely 50% of the adult population in the best of cases, 0% in many other cases. Building custom scorecards off a bank’s historical application data is a good solution in theory, but in practice that data often doesn’t exist, is a mess, and is tremendously costly to turn into stable statistical insights and operationalize for lending to new applicants, particularly for smaller lenders & products. Psychometric credit scoring suffers from none of these shortcomings- it can be used to score anyone, anywhere, and because it’s a standardized application platform and dataset across banks, there are huge economies of scale in scorecard building, maintenance, and implementation. We had the solution to all the problems out there with existing approaches- a ‘game changer’.
But taking a solution from theory to practice reveals its own unique challenges. Unlike bureaus and historical application scorecards, psychometric credit scoring involves active data collection and therefore requires strong fraud monitoring systems, not against applicants (as most people assume) but against loan officers. This was a lesson we had to learn the hard way, and develop a suite of fraud detection algorithms after having seen numerous cases of fraud. It also requires rapidly building scores off small datasets, which in cases when used on a new lending product with rapidly changing qualification criteria and applicant pools, has to be rapidly updated when the population changes. Over time as the dataset grows, that problem diminishes, but that is after five hard and bloody years fighting for data, against fraudsters, and on rapidly shifting terrain.
So now I find myself on the other side of where I was five years ago. Psychometric credit scoring, like bureau scoring and historical application scorecards, is being used in the real world, enhancing financial inclusion and making a large number of lenders a large amount of money. But, in implementation, its practical challenges, like those of the other sources, have been laid bare. But have no fear, there is a new data source out there for alternative credit scoring that will solve all these practical problems: utility and telcom data. This time is different- it will be a game changer!
Let’s not repeat the same mistake. These other data sources, just like those that have come before them, solve some of the problems of existing approaches, but in exchange will have their own challenges. The winners in enhancing financial inclusion will know that going in, and will take a portfolio approach.
How do we know that is true? EFL isn’t a psychometrics credit scoring company, it is an alternative credit scoring company, pulling in any and every source of data to better understand risk and help the un/under-banked earn the credit they deserve. We have production models using psychometrics, credit bureau, and historical bank application and transaction data. And we also have existing models that can be used today for scoring with mobile phone data, online/social media data, and GIS data. But today, the psychometric score is the one having the largest impact, because we are seeing the practical challenges in implementing these other new scores as well, and taking our time to methodically fight through them.
Take for example telcom data, cited in this report. We were able to gather CDR data from a network operator with 98% market share in a Caribbean country, match it to loan data from one of the largest MFIs in that country, and make a highly predictive model with a 72% hit rate (compared to a 10% hit rate for social media data and 0% hit rate for credit bureau in that country). Why isn’t this new model a game changer rapidly creating financial inclusion around the world? The problem is that to actually implement this at scale using a CDR, you need to work out a complicated business agreement between a mobile network operator (MNO) and a lender, which turns out to be a herculean task when you actually attempt it in emerging markets. For that one country it took us 4 years, with a single MNO. If you are working in a different country like Indonesia, you aren’t dealing with one MNO, but many. There you need the largest 5 MNOs to cover just 80% of the subscriber base. Not to mention issues of data privacy and customer consent. You can go over the network directly to the consumers and their phones, but that strategy has its own challenges.
This is not to say that scoring using mobile phone data will not be a hugely valuable tool to enhance financial inclusion after these issues and other practical challenges of implementation that come up are solved- it will. But that will take a long time, and will expose a lot of warts. I wouldn’t be surprised if in five years there was a new blog post here dismissing mobile phone data scoring as dangerous because of these practical challenges have reared their ugly head, or because a lender using it was unsuccessful*, but this time it’s different: now there is this new thing that is the solution to all the problems out there with existing approaches- a ‘game changer’.
A smarter approach is to realize that every existing approach has practical challenges that are known. A good new approach will solve some of these challenges and reach a new segment of the population, but in exchange will have its own unique challenges that will rapidly emerge as theory meets practice and they are rolled out. The smart players will have a toolkit of approaches and use the right combination in the right circumstances.
At EFL we have a toolkit. Today, psychometrics is the dominant tool we are using, because despite all the problems above, it is the most practically useful tool for reaching new borrowers with controlled risk. Or to paraphrase Churchill, it the worst approach, except for all the others. We haven’t used our mobile phone or online scoring models that much yet because the pipes of data aren’t yet there in most markets where we work (nor are the pipes for water, for that matter). We’re building them, but that is a long-term project, and we can’t just wait for the future. Its 5pm right now in Lima, Peru. Today, one lender using our tool in 3 semi-formal neighborhoods in the south and eastern cones of the city has created a psychometric credit score for 84 people who would have previously been rejected because they don’t have sufficient history in the credit bureau for a traditional score. The top-scoring 51 of them got a loan. Over the past two years, that lender has seen that those top-scoring 60% are highly profitable to lend to. They gave test loans to some of the bottom scoring 40% that should have been rejected based on our score, just to see what happened, and they followed the others that were rejected in the central bank database, some of which managed to get a loan from another lender. Those bottom scoring 40% defaulted more than twice as much. This is profitable expansion of access to credit happening now, every day, only possible with psychometric credit scoring, with results audited and endorsed by the banking regulator. When the bureau is available, our client uses it. Data on utility bill payments is available, and the lender uses it. When we finish building the pipes for mobile phone data, GIS data, and social media data, we’ll use it. That future has to be created, but with open eyes to the fact that none of these new solutions by themselves will be flawless. And until then, the best tools available have to be used, one of which is proving to be psychometric credit scores.
*BTW it is faulty logic to argue that a credit score doesn’t work because a particular lender using it lost money. Thousands of lenders using FICO’s score in the United States have lost money, despite the fact that the score has revolutionized retail lending in that country. A score is only one part of the overall credit risk assessment, which is only one part of the entire chain of policy, process, and product design for successful lending.”
