Explainable income prediction based on transaction data
If you've ever applied for consumer credit, you’ll know that one of the most significant factors that lenders consider is financial stability. Typically, this means providing financial documents like bank statements or payslips. However, these documents can be both helpful and misleading, as they may not paint the full financial picture, especially if you have irregular income or expenses. Not to mention, self-employed individuals may face additional challenges in providing financial documents, since they don't receive regular payslips. As a result, the process of applying for consumer credit can become challenging and time-consuming.
That's where the RiskQuest Navigator comes in. This one-stop solution gives credit providers a complete insight into the financial stability of loan applicants while making the application process extremely fast and easy. And with the revised Payment Services Directive (PSD2) granting continuous access to vast amounts of transaction data, the RiskQuest Navigator is enabling credit providers to gain even deeper financial insights. Unlike traditional financial reports, transaction data provides a level of detail that makes it almost impossible to hide any concerning details. With the increasing amount of possibilities brought about by PSD2, the RiskQuest Navigator aims at developing advanced solutions to determine credit risk from transaction data.
As a guideline for consumer-based lending, the ‘Vereniging van Financieringsondernemingen in Nederland’ (VFN) determines a yearly norm based on factors such as fixed income, mortgage, rent, vehicle costs, and child-care. By looking into an individual’s historical transactions, many of these factors can be collected in the RiskQuest Navigator. But since certain factors, such as fixed income, are irregular in nature (holiday bonus, 13th month, raise), it would be wise to consider the bigger picture, taking into account past, present and future.
In this blog post, we'll conduct an experiment to test whether the RiskQuest Navigator is able to predict an individual's income based on historical transaction data. We'll explore the challenges of studying income dynamics, including prevalent structural regime changes and one-off shocks. Ultimately, we'll see how innovative tools like the RiskQuest Navigator can help credit providers make better-informed decisions based on real-time transaction data.
To illustrate, consider the income time series in the graph above. This individual, say Johan Jansen, has a normal job and was promoted in December 2022, which represents a significant shock to his income. Since we we cannot expect Johan’s income to go back to €2,030, the new, reasonable expectation is that Johan´s income will remain stable at €2,440. Furthermore, Johan’s employer pays out a holiday bonus in May, which is determined as a percentage of the income of the past year and should not be confused with a raise or another bonus.
Traditional time-series approaches, such as Autoregressive Integrated Moving Average (ARIMA) models, fail to capture structural changes and have no means for correcting for shocks (such as gifts or inheritance). Even more complex methods, such as Recurrent Neural Networks (RNN), are limited in flexibility. For example, a promotion or an annual bonus should not make prediction more difficult on paper, but is difficult for traditional models to handle. Even then, the more complex models have a clear limitation in that they are not explainable – how do we explain the result of a complex Neural Network to a loan applicant? How does the algorithm decide on income? When it comes to deciding whether someone is credit worthy enough to get a loan, the determining factors, and hence all the underlying results require some form of explainability.
Keeping explainability in mind, we handpick 6 simple predictors and then develop a method to choose between these predictors given a historic sample of transactions. This allows us to apply an advanced statistical method to the problem while maintaining a degree of explainability. From the 6 simple predictors below, we will try to predict the most suitable one for each income series.
1. Mode – the most common observation
2. Median
3. Exponentially weighted average with coefficient 0.5
4. Mean of last four observations
5. Mean of last two observations
6. Last observation
For evaluation, we calculate the average rank of the recommended predictor, this time counting equal predictions to have the same rank. This reflects the use of the model in practice – when forecasting, all we need is the best predictor, whereas for model selection we care to capture exactly how good the model is, not only how often it is perfectly right.
To create a theoretical base and avoid privacy restrictions from client data, we simulated a hundred-thousand income series based on publicly available income statistics from CBS, as well as the extensive knowledge of the RiskQuest Navigator team. We simulated both fixed and variable pay for self-employed persons. For model development, we restricted ourselves to four months of data per income series. This is because we noted that four months corresponds to the average transaction history that is processed by the Navigator. Specific month information was also fed into the model to inform about seasonal bonusses. The target label was the most suitable simple regressor given the input, and chosen to be the predictor that minimized the Mean Squared Error (MSE) with random tie breaks when multiple performed best. For example, in the extreme case where all observations are equal, every simple predictor will yield the same MSE. Then all predictors can be chosen with equal probability of 1/6.
After trying several different algorithms, including Naive Bayes, Adaboost, Multilayer perceptron (MLP), and a Convolutional Neural Network (CNN), we found that the best algorithm for predicting the best income predictor was the CNN with embeddings for month data, with an average rank of 1.598. Feeding in all respective month numbers allowed for missing values, due to for example a leave of absence.
It's worth noting that the other algorithms also performed well, with Naive Bayes having an average rank of 1.717, Adaboost having an average rank of 1.696, and the MLP also having an average rank of 1.69. However, the CNN outperformed them all.
The simplicity of the underlying predictors is crucial for underwriters to be able to clearly defend their decisions. However, using any single simple predictor would lead to a poor fit overall – the combined approach thus harnesses all the nuance of the data to select the most relevant predictor. The consequence is that the prediction is simple to explain, and at the same time much more accurate.
For example, in the below chart Johan’s son, Jaap, gets a bonus, almost doubling his income. A naïve forecast would be just taking the average of all observed salaries. This, however, leads to a significantly higher forecast – thus giving a false representation of the height of his income. Within the RiskQuest Navigator, this would not happen since our model has learned that one-off bonuses of this size are more likely than a structural break (such as a promotion), so simply taking the median is the best predictor.
Conversely, in the below chart there is a holiday bonus paid out in May, followed by a raise in July. Here, the RiskQuest model recommends taking the average of the last two observations for prediction – as opposed to overestimating the income by taking an average. The May holiday bonus is extremely common – meaning that a person would easily understand that this is based on the regular income (as in June), meaning the income increase between June and July is a likely promotion. It is exactly this kind of nuance that simpler models fail to capture.
We have seen that our income prediction approach can serve as a great extension to the transaction labelling capabilities of the RiskQuest Navigator. When properly trained, this approach has many use cases beyond private income, such as modelling different kinds of consumption, or even cashflow dynamics for company data. While things like power and gas have obvious seasonal effects, discretionary spending is prone to jumps (e.g. booking holiday travel), structural breaks (due to changes in income or family situation) and common patterns (spending boom before Christmas and crash after). The model can be further extended by adding other known covariates such as sources of income or specific quirks within different uses of the Navigator, allowing for a lot of customisation and even better performance.