Model Risk Management, industry practices and the software solution.

Meriem Kanzari

The Model Risk notion has been around for many years now; yet, it remains particularly complex to quantify and mitigate.

Some spectacular failures in risk management have demonstrated the consequences of neglecting model uncertainty. Instances trace back to 1997 with the Bank of Tokyo/Mitsubishi $83 million loss and NatWest Capital Markets £50 million loss. The Long-Term Capital Management (LTCM) debacle in 1998 was also attributed to model risk. More recently, around 2010 and 2021, JP Morgan endured a $6 billion loss [1] and AXA Rosenberg $242 million fine, not counting the reputational damage costs [2].

"Various case studies have indicated the importance of Model Risk Management in the derivatives industry."

Many large and small financial institutions are aware of this issue and have been developing in-house or proprietary methods to tackle it systematically. Through this article, we will first take a quick look at some definitions, then summarize what has been done on both the regulatory and corporate sides. Finally, we will indicate what we believe are the criteria and checkpoints for software to enable Model Risk Management.

1. Seeking a definition

Prices of complex financial derivatives and structured products need to be inferred from observable market prices of related instruments. This process, known as “marking-to-model,” involves both computations and human financial decision-making, thus exposing it to estimation errors [3]. There are multiple definitions of model risk. Following Derman (1996) research notes: [4]

  • Inapplicability of modeling
  • Incorrect model
  • Correct model, inappropriate use
  • Badly approximated solution
  • Bad implementation resulting in Software bugs
  • Incorrect use of data and unstable data

2. What do regulators bring to the table?

Some definitions

North America:
The most commonly used definition of model risk comes from the Federal Reserve’s letter on Model Risk Management (MRM) “SR11-7”. SR11-7 defines model risk as “the risk of adverse consequences (e.g. financial loss, poor business or strategic decisions, or reputational damage) arising from decisions based on incorrect or misused model outputs” [5].
Europe:
Model risk comprises two distinct forms of risk: risk relating to the underestimation of own funds requirements by regulatory approved models (e.g. internal ratings-based (IRB) models for credit risk); and risk of losses relating to the development, implementation or improper use of any other typically internal models by the institution for decision-making (e.g. product pricing, evaluation of financial instruments, monitoring of risk limits, etc.)” [8].

Some regulations

alt text Chart 1: Source: Deloitte, 2017: Model Risk Management Driving the Value in Modeling [7].

Some frameworks

Europe:
The Targeted Review of Internal Models (TRIM) is a large-scale project conducted by the ECB as a guide for banks to reduce risks associated with the use of internal models.
North America:
SR 11-7, the US regulatory model governance standard, provides a framework for model risk management. Both frameworks share some commonalities:

a) Model validation as a separated step from the use and development.
b) Model governance and policies including model inventory management and documentation.
c) Model development and use including IT implementation tests, accuracy and reliability of risk parameters, and internal validation.

3. What has been done on the financial institutions’ side?

One way to answer this question is to hear from some institutions. For the purpose of this blog post, we picked a Mckinsey survey of 75 banks from North America, Europe, Asia, and Australia [8].

"To date, there is no consensus on MRM or MRM metrics and the MRM framework is still hard to grasp."

North American institutions have led the adoption of MRM-specific technology solutions, mostly focused on inventory management and reporting.

European firms are more focused on core workflow and inventory management, and less on wider topics such as automation of testing, model monitoring technological platforms for MRM. However, banks both in Europe and at a global scale have understood that partnering with financial technology firms, analytics vendors and data providers has become the new norm [8].

4. How could financial software facilitate Model Risk Management?

"We have identified four levels by which a given software solution could enable Model Risk Management."


alt text Chart 2: Source: LexiFi 2021: Software MRM building Blocks. The software tools for an effective model risk management, from the most basic to the most advanced.

According to Mckinsey’s survey of 75 leading global banks [8], the biggest challenge banks have mentioned is the lack of flexibility of current technologies which does not allow them to cope with the fast-changing requirements of effective MRM.

At LexiFi, we have identified four levels by which a given software solution could enable Model Risk Management. The first being the most basic and the fourth the most advanced.

  1. The software solution should provide several different pricing models: because the notion of Model Risk cannot be treated in the absence of multiple models
  2. For a given pricing computation, the software should give users access to all intermediate calculation steps and results for auditing purposes or further investigation if needed
  3. The software solution should allow users to get many prices at once from all suitable models for a given contract
  4. The highest is obviously to provide a functionality to compute specific metrics for model uncertainty and risk analyses

5. LexiFi’s Approach

LexiFi’s quantitative team has been working over the past year on perfecting levels 1 to 3 of the Model Risk Management building blocks presented in the section above in Chart 2. We have recently reached level 4 as LexiFi’s engineers have developed a tool that provides a confidence interval for a given price. The certainty level is measured by a volatility indicator that we define as the Confidence Interval Equivalent Volatility (CIEV). Visit our dedicated Price Uncertainty page to know more.

Resources

[1] Rama Cont, 2004

[2] Larcker and Tayan, 2013

[3] Röman Janm, 2017

[4] Derman, 1996

[5] Federal Reserve System

[6] European Central Bank

[7] Deloitte, 2017

[8] Mckinsey, 2019