Model risk is inevitable
The problem is, models per Definition is simplifications of reality. As British mathematician George E.P. Box pointed out, “Essentially, all models are wrong, but some are useful.”
The risk of model-supported decisions is divided into two areas:
- Does the model provide inaccurate output?
- Is the model abused?
This can be a misunderstanding of output or use for things outside the design area.
This risk of making the wrong decision is compounded by the fact that the analytical model developers / builders are often separated from the decision makers. This results in a “computer said yes” approach where the person using the results from the model does not necessarily understand the model or why it reached its answer.
The consequences of “poor” model guidance on significant business decisions are:
- The bank may have insufficient capital (or cash) and fail (or require a government rescue).
- The bank suffers significant losses. Even if these are not large enough to cause capital problems, they will damage the bank’s value and reputation.
A wake-up call for regulators and senior management
Regulators now have increased awareness of model risk and tackle the problem. The US Central Bank led the way in 2011 by issuing SR 11-7 Model Risk Management Guide. The European Banking Authority has followed US regulators and in December 2014 incorporated a specific model for risk management guidance in the latest Guide to the Control and Evaluation Process (SREP). This document is issued to all EU regulators to instruct how they conduct their periodic SREP review of each bank (typically annually) and take effect January 1, 2016.
In addition to the increased regulatory pressure, senior executives are now much more valued by the risk in the models used by their banks. This means that banks are now building a new approach to corporate governance and model risk management – and, crucially, mechanisms to provide regulators with evidence that this has been done.
Addressing model risk
The basic approach to managing model risk is to have regular, independent checks on models. A rigorous exercise must be performed when the model is first created or undergoes significant change (model validation). Thereafter, the model must be checked periodically to confirm that it still produces tolerance results (model review).
The challenge for a large bank (with perhaps 2,000 significant models across many teams and countries) is to ensure that the entire model portfolio receives the appropriate level and quality of model validation and review. The central risk team can issue a risk policy model – determine what needs to be done. But how does senior management enforce this policy?
Experience financial services use the new SAS® Model Management solution with excellent results. Before taking SAS, it took Discover four to five weeks to collect and prepare its data, documents and reports for a CCAR (US regulatory) review. With the SAS solution, it took less than a week and model mutual dependency and interconnection were readily available.
But the real value is providing systematic help that comes from centralizing model information management. Discover has six larger units and a few dozen smaller groups that are very involved in model development, implementation and use. With this approach, model risk management can be centralized, but not the actual development, testing and implementation of the models that are left to the business units.
As a result, Discover’s CRO may feel comfortable asking questions of regulators about the organization’s models and how it monitors and controls risk – plus provide the proof that it is happening.