Models are really good looking, which is one reason why everyone in their college years wants to date one. If you are like us and dated a couple of hundred of them back then (hey, there was no YouTube, so it could have happened) you realize that while they are very attractive on the outside, they might not necessarily the sharpest tools in the drawer (yes, we are generalizing). As with models in real life, no one can look that good and not have some faults, so here we focus on some of those faults as we discuss banking models of a different kind.
One key problem with models is that they tend to underestimate risk. This is because they are designed to act logically and are driven by data fed in by the bank. As we all know from recent experience, the world seldom acts logically and limited data going in means limited value coming out. Taking things further and getting a bit more technical for the "quants" in the crowd, normal distributions tend to underestimate the probability of extreme movements and losses can substantially exceed expectations. Go no further than Lehman or Bear Stearns to see a real-life example of this modeling risk.
Another issue we often see with models relates to complexity. It is nice to have really big print with a single number and a thumb's up right next to it, but that seldom explains all of the risk going on in today's complex industry situation. Models that take complex concepts and try to distill them down to something much less complex tend to underestimate the risks and give the bank a false sense of security that because things have been modeled and a report is held in one hand and can be waived in front of the board that all is under control.
One key to using modeling results is to understand the model itself is only as good as the assumptions going into it. Risks are complex by nature and simply trusting a black-box model to tell you what risks the bank is facing is not only ludicrous, but outright dangerous as well. Make sure you understand how the model works and test using multiple types of models to thoroughly understand the range of risks your bank could be facing before trusting any results.
A fourth trap we see banks get into over and over again relates to model structure itself. Many models use historical data to make future assumptions. While commonplace, when it comes to modeling processes, using the past to predict the future is not something that works in a changing world that is as interconnected and volatile as ours currently is. When times are quiet and volatility is low, models can do a fine job using historical data to project future expectations, but when things start to whip around, bankers would be better served to try and nail down expectations and projections based on a range of possibilities to be sure the management team and board are comfortable with the risk profile no matter the circumstances.
Finally, we close with a nuance related to complex modeling processes that is worthy of highlighting. Complex models often provide a false sense of security because of their complexity, but behind the curtains, some very complex models may actually be driven mostly by one or two assumptions feeding into them. Understanding the modeling process is good for any bank executive to know, so don't blindly accept results without double checking assumptions and stress testing to see where the wheels start to fall off.
The models at the bank may not be as attractive as the picture at the left, but they can deliver attractive results when used properly and weaknesses are thoroughly understood.