A decade of credit risk modelling: what has changed?

On August 6th 2012, I entered the Rabobank to start my career in the field of credit risk modelling. This means that I am currently working in this field for a decade. I don’t know whether I am more surprised that I am still working in this field or that I am still fascinated by credit risk models!

Maybe it sounds monotonous to work within credit risk modelling for ten years. Especially, consultants prefer to work in diverse fields and would like to contribute to varied projects. However, in my view, credit risk modelling in itself is a very diverse field. Furthermore, credit risk modelling heavily developed in the last ten years. For example, in 2012 it was impossible to run a simple logistic regression model with a million observations, due to technical limitations. Can you imagine! Therefore, it is interesting to look back and see the development of credit risk in the past decade. Furthermore, without becoming sentimental, there might even be things that were better in the past.

The technological improvement does not only have impact on the number of observations that can be used during modelling. It has also a large impact on the collection and storage of data. This will be discussed in more detail in the next section. Afterwards the involvement of stakeholders is discussed. Before the financial crises, front office (usually referred to as “the business”) had a compelling opinion about the final model. Currently, it is more important whether the model will be rated as compliant by the regulator. In the last section, it will be discussed what the effect of all these changes is on the project organization of a credit risk model development.

It is all about the data

All of us know that data is an important part of a model development project. We are always wondering why the data quality is below standard for the majority of the portfolios. Furthermore, we participated more than once in an extensive data remediation project, before we were able to start modelling. These remediation projects collect and improve data manually and case-by-case.

These enormous efforts and costs are justified, because historical data is not up to (current) standards. This can be explained from technological perspective. Storage and use of large datasets was not possible ten years ago. Usually, data was stored in many different systems and not linked to each other. Data warehouses for risk purposes were non existing! Furthermore, departments that manage data and construct modelling datasets did not exist as well. Therefore, modelers did these tasks themselves during a development project. As you can expect, this led to many issues. For example, modeling datasets were constructed different every project, which lead to inconsistencies for scoping, default definition, loss amount and modeling level. Especially, when you realize that the development of PD and LGD were usually different projects.

A striking example, is a mismatch between modelling level for PD and LGD. PD was modelled on loan part level. Most loans existed of multiple loan parts and in many cases only one of the loan parts had significant arrears and was therefore in default. The other loan parts of the same loan were still performing. However, LGD was modeled on loan/facility level. If a client was foreclosed, write-off was divided over all loan parts. Obviously, this lead to incorrect expected and unexpected loss.

Another, less obvious, example that we have seen multiple times was a mismatch on the basis of the dataset. PD was based on the full portfolio, while LGD was based only on the losses in the special asset management administration. Not every default has necessarily a loss registered at special asset management.

Luckily, increased attention for the collection and preparation of data decreased these types of inconsistencies. Data engineering within risk leads to a solid basis for model development. Definitely, one of the most important improvements over the past ten years.

Who decides whether a model is appropriate

The first years that I worked in credit risk, the most important stakeholder for the models was front office (usually referred to as the ‘business’). Credit risk models were used for multiple purposes and therefore it was important that the calculated RWA was not too high. Business was very nervous that the bank would decline clients, because they were marked as too risky. Or the other way around, the client would decline the offer of the bank, because the (risk-adjusted) price was too high. Regularly, this lead to pressure on the modelling team to keep the impact of new models low.

A big change was the shift from DNB as regulator to ECB as regulator for ‘large’ banks. At the same time a large number of new Regulatory Technical Standards (RTS) and Guidelines (GL) were published by the ECB. This changed the view on models for all banks in the Netherlands.

Earlier versions of the regulations were more principle-based than rule-based. Therefore, there was a lot of freedom for both used definitions and development models. The Asset Quality Review, better known as AQR and later the Technical Review of Internal Models, better known as TRIM, lead to an enormous amount of findings and obligations. This narrowed down the possibilities to influence the outcome of RWA calculations.

Currently, almost all development projects are followed by an Internal Model Investigation (IMI) by the regulator. This makes compliancy with the regulations (one of) the most important requirement(s). In general, this gives more power to the risk departments and lead to less pressure from the business. However, sometimes all these detailed regulations are so blindly followed that the end result of the model development project does not make sense anymore. In these cases, a check with the business would have led to more sensible results.

How do we do it

As I already mentioned, in the past the data preparation for a model development was also conducted by the modeling team. Although we may not call it a ‘team’, because most model developments were conducted by only 2 or 3 modelers. Can you imagine that you will conduct a full model development with only 2 colleagues and that this includes data collection and preparation from source systems? This team composition can be explained by the historical differences mentioned earlier: less available data and less regulation. However, the modeling team conducted less analyses, less documentation and less substantiation.

Currently, model development projects exist sometimes of 40 modelers. In my view, this is also not ideal. As we say in the Netherlands: “you are not able to paint a restroom with 10 people”. Sometimes the tasks are so shredded and divided such small pieces that no one has the overview anymore. This could lead to inconsistencies, mistakes and unreadable documentation.

Given that you work with many more colleagues on a development project, you would expect that the project would take less time. However, that is definitely not the case! The last years, development projects seem to take more and more time. This also lead to an additional risk. Given the large number of modelers and the long horizon, often people leave the project during the development. More than once, at the end of the project the development team is totally from the team that you started with. Obviously, this makes the reproduction of the model, the substantiation of choices and the documentation more complex.

During such long an bumpy rides, I want to go back to less complex periods. However, when I realize that they will not come back, I start to feel old! Before I become sentimental, I will finish this blog. Last thing to say: I am curious what the next ten years will bring us!