Can we predict who is more likely to repay a Balloon loan?
Throughout 2017 we collected over 500 data points on each entrepreneur who successfully completed our programme. We collect this data to measure impact but also to make us smarter in answering key questions. One of these key questions is: what (if anything) predicts which entrepreneurs are more or less likely to repay the loans that Balloon gives them?
We recently started a project to comprehensively analyse this question and have some preliminary findings we wanted to share. With that in mind, the findings presented here are not conclusive, they are just what we’ve learnt so far.
What we did
Our starting point was to review the loan repayment records of all of our entrepreneurs who had completed programmes in the first half of 2017. We then grouped these entrepreneurs into two groups, those who were repaying their loans on time and those that were not (including those who defaulted and those who were late in their payments). Then we used statistical methods (something called stepwise logistic regression) to tell us whether any programme related data predicted whether an entrepreneur ended up in the ‘good repayments’ or ‘poor repayments’ group.
Before jumping into the findings, let’s look at what predictors we considered. There are two important bits we looked at in this analysis. The first, is data on the entrepreneur’s characteristics. As part of our programme, volunteers work with entrepreneurs for several weeks. Therefore, they get to know their entrepreneurs very well. To leverage this knowledge, we ask our volunteers to provide character references of the entrepreneurs they are working with. Amongst other things, this captures how passionate entrepreneurs are about their business, how opportunistic they are, their knowledge of finance, etc.
The second chunk of data comes from our pitching process. At the end of the programme, entrepreneurs who want to apply to Balloon for an interest free, flexible, unsecured loan are invited to pitch. This includes three parts: preparing a pitch document (a summary of the entrepreneur’s business and everything they’ve achieved on the programme); a visit to their business; and the pitch itself (a 30-minute presentation and Q&A).
An entrepreneur makes their pitch below:
An example pitch document:
At every step of the pitch, a panel of evaluators is gathering evidence to score the entrepreneur on Balloon’s 12 pitch criteria. These are criteria that we have learnt are important for entrepreneurial success. Examples include: proof there is demand for the entrepreneur’s product/service; the entrepreneur’s experience in running similar businesses; and the financial viability of the business.
A panel completes their ratings of the entrepreneur they’ve just evaluated:
So, in summary, we looked at whether the character references and pitch ratings of entrepreneurs predicted loan repayment performance. What did we find out?
What we learnt
The character reference ratings were important predictors of whether someone repaid or not. Interestingly, amongst all of them, the best single predictor was ‘appetite to take risks’.
For example, if an entrepreneur was rated a 4 (out of 5) instead of 3, on their appetite to take risks (i.e. they are riskier), they were five times less likely to repay their loan! This makes sense, people who are riskier might be more open to not repaying a loan (when they have the money to do so) but also might engage in overly risky behaviour which causes them to lose the money they would’ve used to repay their loan.
For pitch ratings, there were similarly positive findings. Here, the strongest individual rating of the 12 pitch criteria was: ‘The entrepreneur can repay the loan within twelve months, taking into account costs of the business and any salary or personal expenses’. Once again, a one-point difference on this rating was associated with an increased likelihood of repaying the loan of over four times.
This is promising because it means that our panels’ assessment of the likelihood of loan repayment is actually accurate!
Findings like these are exciting. They mean that we can become smarter in our decision making, using data and evidence to maximise impact. For example, we might in the future choose not to give loans to people who are rated as riskier.
To make those kinds of decisions we have to collect a lot more data to be absolutely sure that what we are seeing are true patterns in the data. We also have to check that ratings on these variables are not biased against any demographic as this might then make us unconsciously biased in how we allocate loans (you can read more about how we’ve tested this in our pitching data here).
For now, we are doing two things. First, we are finishing off the rest of the preliminary analysis to see what else we can find. Second, we are laying the groundwork to collect even more data to strengthen our conclusions. With this, we can begin to make practical improvements across our programmes to maximise the impact we create.
This piece was written by Nicholas Andreou. Nicholas leads the Insight & Impact function at Balloon Ventures. He holds a PhD from the University of Nottingham and has previously held research positions at Harvard University and the World Health Organization as a student.