Remember when banking was easy? It used to be that management teams could look forward to Friday as the kick-off to the weekend. Fattening food would show up at the office, everyone walked around smiling and times were good. Today, it is difficult to find any community banker that doesn't take work home with them over the weekend. Bankers hunker down in their home office, pour over reports and email continually to try and stay ahead of the curve. There is no time for mistakes and no time period, so today we consider the impact of making decisions based on reports and analysis generated by poor modeling.
Model risk is still a fairly new area of focus at most community banks, however studies show 82% of large banks have a specific unit dedicated to model validation. While most large banks build many, many proprietary models (so they need this function to be robust), most community banks use off-the-shelf models, so it is assumed the model-maker does all of this.
The world has become more complex and the principles behind model validation are critically important to community bankers as everyone works to enhance risk management and reporting functions. To stay on top of things, here are some suggestions to help ensure your models and the reports they generate are producing information you can leverage and use effectively.
One best practice is to set at least an annual cycle to review any model that could impact financial results. At a bank, this could cover just about anything, but prioritizing the list is important when resources are scarce. Auditing functionality, assumptions and back-testing are all important considerations to ensure model result integrity.
Another key consideration is to have a well written enterprise wide policy for managing and validating model risk. Studies show about 84% of large banks do this and it is important for community banks to start down this road as well to stay ahead of the curve. This policy is usually owned by the Chief Risk Officer or equivalent and at least annually approved by the Audit Committee.
One "lesson learned" from the crisis is that while most cannot blame their demise on poor modeling, it plays a role. Having a consistent governance structure with tight standards and controls around modeling helps ensure risks are contained.
When it comes to modeling, another key consideration is to not only validate the model independently on a periodic basis, but also to do so upfront. Waiting until a model is installed and operational for a year before taking steps to test inputs, back-test or validate assumptions and results is fraught with risk. While sometimes this cannot be done during a changing economic cycle and historical relationships between key drivers are not as correlated today, this should be recognized by the modeler and duly noted in results. Executives and directors need to be able to rely on reports that are generated so it is critically important to install and test things upfront. At the large banks, studies show 75% require formal approval of all new models before installation and when making material modifications to any existing model.
To stay in peak condition, fashion models know it is important to work out periodically and eat right. When bankers work to ensure their models are in peak condition, the same principles apply. Testing conceptual soundness, reviewing documentation, maintaining a current inventory of significant models/key assumptions, conducting an independent review and comparing model results to real world events are all critical to prudently managing model risk.