SS17/16 – Solvency II: internal models – assessment, model change and the role of non-executive directors

1.1

This supervisory statement is addressed to all UK firms that fall within the scope of the Solvency II Directive (‘the Directive’),¹ and to Lloyd’s. It sets out the Prudential Regulation Authority’s (PRA’s) expectations of firms regarding internal models.

1.2

This statement should be read in conjunction with the PRA’s rules in the Solvency II Sector of the PRA Rulebook, the Solvency 2 Regulations 2015 (2015/575)² and the PRA’s insurance approach document.³

1.3

This supervisory statement expands on the PRA’s general approach as set out in its insurance approach document. By clearly and consistently explaining its expectations of firms in relation to the particular areas addressed, the PRA seeks to advance its statutory objectives of ensuring the safety and soundness of the firms it regulates, and contributing to securing an appropriate degree of protection for policyholders.

1.4

In this supervisory statement, the PRA sets out its expectations for firms in the following areas:

• internal model applications;
• the assessment of credit risk;
• dealing with variability in premium provisions;
• the effect of stresses on the volatility adjustment;
• the role of non-executive directors;
• model justification and validation and the role of boards;
• the PRA’s use of quantitative analysis in approving models; and
• scope, identification and classification, governance and reporting of internal model changes.

2.1

Firms are reminded that once a formal internal model application has been submitted to the PRA, there is limited opportunity for firms to make substantive changes. Firms should therefore make sure their applications are stable and approved by their internal governance processes prior to formal application. Where firms become aware that they may need to make changes during the application period, these should be discussed with their usual supervisory contact as soon as possible. Where changes are material, a new application is likely to be required. Alternatively, firms themselves have an option to ‘stop the clock’ on the current application. Neither of these options should be approached lightly. If firms believe that significant model changes are likely to continue into the formal application phase, they are encouraged to consider delaying their application to the PRA and to discuss options with their usual supervisory contact. In this respect, the pre-application process is a means to help firms verify they are on the right path before they submit an application.

2.2

The PRA can approve an internal model application only where it is satisfied that the model has met all the Directive tests and standards (T&S). Approval must be based on this requirement⁴ and not an ‘on-balance’ judgement. Some firms have proposed applying internal management loadings to models to help deal with known areas of weakness which cannot be fully fixed ahead of the formal application. In some cases, such adjustments might help firms to demonstrate that specific areas of the model meet the relevant T&S (for example, the Directive calibration standard of 99.5% over one year).⁵ However, all areas of the model must meet the Directive requirements and the use of more generic management loadings cannot be used by firms as a mitigant where the model does not meet the required T&S.

2.3

Irrespective of the progress of the internal model application, firms should have an alternative approach that they can use if they fail to gain model approval after submitting an application, and ensure that they have a clear understanding of the actions they would take in those circumstances. For example, a merger or restructuring may make the existing standard formula inappropriate and therefore the applicant would need to have a contingency plan in the event of non-approval.

4.1

General insurance firms should consider variability in premium provisions on their year-end Solvency II balance sheet. In the same way that events can occur that cause claims provisions to vary, some of those same events will also cause the premium provision to vary. Examples include changes in court awards for liability claims or an unforeseen major natural catastrophe event, like the Tohoku earthquake.

4.2

Firms that do not consider this risk may fall short of the internal model T&S. The PRA considers this risk exists for all actively underwriting internal model firms who model on a one-year earned basis, although it may be that for some firms this risk will be small in the context of their total SCR.

6.1

This chapter sets out the PRA’s expectations regarding the role of the non-executive directors (NEDs) when considering a firm’s internal model.⁶ 

6.2

The use test⁷ is one of the T&S firms need to meet. As part of the use test, firms must ensure that members of the board and others involved in running the firm have an understanding of the model.⁸ One of the methods the PRA may use to assess whether firms are meeting the use test is to speak to NEDs (either individually or collectively) to gauge their understanding. The PRA does not believe that it is necessary for board members to be technical experts in modelling techniques in order to meet the use test requirements. Rather, the PRA expects board members to understand and be able to explain areas such as the:

• key strengths, limitations, and judgements within the model;
• assumptions and judgements that have the most material impact on the model output; and
• key sources of information and advice the board has relied upon in order to satisfy itself about the appropriateness of both the model design and the model output.

6.3

Board members should be supported by the executive management to understand and engage with the key features of models. Boards should draw on a wide range of sources to understand, challenge and make a decision on the validity of a model. For example, independent validation can play an important role in helping boards gain an overall understanding of a model and its strengths and limitations, as long as the validation work is focused appropriately on a critical appraisal of the most material aspects of the model, and if its conclusions are summarised and presented appropriately for a board-level audience.

6.4

Under the leadership of the chair, NEDs should consider and challenge, as appropriate, the executive management on all aspects of the firm’s strategy, which includes the viability and sustainability of the business model and the establishment, maintenance and use of the risk appetite and management framework. NEDs are expected to challenge how these elements are reflected in the internal model.

6.5

The executive management should be able to explain the firm’s internal model in simple and transparent terms to NEDs. This includes explaining the uncertainty around judgements, in what circumstances the results may deteriorate (eg the analysis or strategy could be wrong), and the implications on the internal model of different methods of measurement for issues such as liquidity. In order to explain complexity to the NEDs, the executive management should provide good management information to enable challenge and thus encourage accountability. If NEDs do not feel that they can meet these expectations, they should demand the time and support to enable them to do this.

7.1

Model justification and validation are two separate processes and represent important components of the Directive requirements. The PRA expects firms to demonstrate clearly this demarcation in their implementation.

7.2

The Statistical Quality Standards (SQS) for internal models in Solvency Capital Requirements – Internal Models 11 and Articles 228 to 237 of the Commission Delegated Regulation,⁹ set out requirements in particular on the methods and assumptions to be used in the internal model. Compliance with those requirements will have to be evidenced to the PRA (Solvency Capital Requirements – Internal Models 3.1).

7.3

For instance, firms will have to justify that the chosen actuarial and statistical methods are adequate¹⁰ and will have to justify the assumptions used taking into account the significance of those assumptions.¹¹ Such justifications need to be satisfied as part of the SQS requirements, and it is not the aim of the validation to create a substitute for these requirements. Depending on the firm, such justification either could be completed by the first line of defence (as part of the production of outputs within the calculation kernel), or by the second line.

7.4

In contrast, the validation is a regular and independent (from the development and operation of the model) process which includes reviewing the model in terms of the appropriateness of its specifications, the correspondence of its results against experience and its overall performance over time.¹²

7.5

Model justification, as part of the internal model development, is often covered by the validation using a bottom-up approach. Part of this includes the justification of why the modelling choices are reasonable and defensible. However, the bottom-up approach should cover the validation objectives fully and help boards and senior management to obtain an adequate understanding of the model.

7.6

The PRA expects validation to be a combination of detailed ‘bottom-up’ testing and ‘top-down’ ownership by boards.¹³ The PRA expects firms to be able to produce clear evidence showing how boards are overseeing and influencing the design of the validation process, how the findings from validation work are summarised and reported to them and how boards are then involved in tracking validation issues through to resolution. The PRA expects firms to demonstrate clearly that boards are using validation as a tool to enable them to gain a good understanding of a model and its strengths and weaknesses. The PRA has high-level expectations on board involvement in validation. See also Chapter 6 ‘Role of non-executive directors’.

7.7

Firms’ use of validation materials that are focussed primarily on the bottom-up justification of parameters and assumptions may not be aligned clearly enough to enable senior management and boards to challenge effectively the key assumptions and limitations of the model.

7.8

Although a bottom-up approach is an important aspect of the internal model validation, the PRA emphasises that boards should value the role that good validation can play in helping them to understand the key drivers and limitations of a model. The PRA expects that firms would be able to provide evidence that the board has:

• challenged the validation process and its results;
• understood and satisfied itself on the key assumptions and limitations of the model; 
• considered the possible quantification of these limitations; and
• taken appropriate mitigating actions.

7.9

The PRA expects boards to be tracking progress actively in addressing key issues identified by validation work.

7.10

The PRA does not expect boards or senior management to be able to discharge their duty in isolation. Boards should demand support from executive management to ensure that key features of models are explained in a way that directors can engage with properly, and they should draw on a wide range of sources, not limited to model developers, to ensure they are satisfied with the model.

7.11

To verify the robustness of the internal model, the T&S are designed to ensure that a model is well grounded in its technical content, with good sources of underlying data. The T&S also require that the model and its limitations are properly understood by its users and by senior management at firms, including the board.

7.12

In summary, a comprehensive validation process should put specific attention on those key assumptions and expert judgments that have a material impact on the model, and should also articulate how the sensitivity to the key assumptions and expert judgement are being assessed and taken into account in the decision process.

7.13

The PRA expects that validation of the internal model clearly evidences the review and challenge that has taken place in assisting the board to meet its objectives.

8.1

The PRA’s quantitative framework falls within the context of the overall model review process which is structured around the T&S. The internal toolkit the PRA uses in its assessments is framed around those requirements so that the PRA is able to satisfy itself that the model meets the T&S. Within this structure, the PRA applies a series of qualitative and quantitative tools to help guide areas that are in need of greater review focus. One of those tools is the quantitative framework for model reviews, which includes the use of specific quantitative indicators (‘QIs’) where risks are sufficiently homogeneous.

8.2

The PRA’s decision-making process is built around assessing the T&S, building from a granular assessment of each of the criteria into broader requirement categories (eg ‘documentation’) through to a final approval or rejection decision. As part of this overall assessment, the PRA has used its risk-based approach to supervision to focus additional scrutiny on a firm-by-firm basis. In this assessment process the PRA uses a series of indicators to determine the focus of review scrutiny: these are both qualitative (eg a view on the embedding of the model from previous supervisory engagement) and quantitative. 

8.3

One of the T&S categories relates to the calibration standards, which are set out in Solvency Capital Requirement – General Provisions 3 and Solvency Capital Requirement – Internal Models 12. An assessment of these requirements is geared towards ensuring that the SCR produced by the model corresponds to the value at risk (VaR) of the firm’s basic own funds at the 99.5% confidence level over one year.¹⁴

8.4

The PRA uses its quantitative framework as:

1. (a) a diagnostic tool to help assess model rigour and capital adequacy and hence highlight areas of potential concern;
2. (b) a prioritisation tool, to help inform where review teams should direct their attention, eg by identifying risks or correlations which may be under-calibrated; and
3. (c) one contributor to decision making as to whether a firm has met the T&S, and therefore whether its model should be approved.

8.5

Internal models are required to be calibrated to the standard specified in Solvency Capital Requirement – General Provisions 3.4. Where risks are homogeneous, a PRA quantitative assessment of the calibration of individual risks and their dependency structures can give an efficient diagnosis of whether there are areas of potential concern where the model has not been calibrated adequately to meet the T&S. Where the risks are largely (but not totally) homogeneous, the PRA has tailored its quantitative assessments to reflect a firm’s specific risk profile.

8.6

Quantitative tools are also important in helping the PRA to prioritise areas for early review where firms may not have calibrated their risks or correlations adequately. However, they are not determinative of the PRA’s final view of a model or model component. It is also worth noting that the PRA looks at the calibration of any model as a whole, as well as in its constituent parts, with particular consideration being given to whether the model remains appropriate in a range of conditions and over time when the balance of risks may change.

8.7

Finally, the outputs of this quantitative analysis constitute one of the many indicators that are taken into account by the PRA in concluding whether the model meets the T&S. Specifically, while the PRA’s quantitative analysis assists in verifying that the calibration standard in Solvency Capital Requirement – General Provisions 3 has been met it does not negate the need for other aspects of the model to be reviewed including SQS in Solvency Capital Requirement – Internal Models 11.

8.8

As set out above, the operation of the quantitative framework does not yield a mechanistic ‘pass/ fail’ decision. It is worth highlighting once more that model approval does not hinge on meeting any particular quantitative criterion, but on meeting the T&S. It must also be emphasised that the use of the tools underlying the PRA’s quantitative framework is always tailored having regard to a firm’s own risk profile.

9.1

Firms with an approved internal model are required to have an approved model change policy.¹⁵ Following approval, the model change policy is expected to play a central role in the wider governance of a firm’s internal model. For example, it should help ensure that the internal model continues to reflect the risks to which a firm is exposed and meet the requirements of the Directive.

9.2

It is important that the model change policy is of a good standard. Firms should consider all the relevant Directive requirements and the EIOPA Guidelines when developing and maintaining their model change policy.

9.3

This chapter outlines some further expectations for firms on setting out their internal model change policy.

9.4

When defining the scope of the policy, firms are reminded to consider Guideline 6 (scope of the policy for model changes) of the EIOPA Guidelines on the use of internal models. It is important for firms to consider whether the scope of the model change policy and model change definitions are sufficiently broad and appropriately flexible to be able to capture any changes that could have a material impact on the SCR or to enable the firm to meet the T&S. For example, the policy recognises that a particular change to a technical provision model may be within scope if that change leads to an impact on the internal model SCR.

9.5

There may also be situations where firms consider it appropriate to exclude something from the scope of the model change policy. In these circumstances, it is good practice for firms to justify these exclusions clearly.

9.6

Firms should also be mindful of monitoring circumstances that might necessitate the need to change the scope of the policy.

9.7

It is important for firms to recognise that the need for model changes may arise from a wide range of potential sources. For example, model changes may be instigated through a firm’s model development plans, validation activities, the own risk and solvency assessment (ORSA) or evolving use of the model. In addition, changes in a firm’s own risk profile and factors external to the firm, such as the economic or commercial environment, may be potential triggers of model changes. A good model change policy would establish a robust process to identify, collate and manage all sources of potential model changes.

9.8

The EIOPA Guidelines on the use of internal models¹⁶ expects firms to develop and use a number of key quantitative and qualitative indicators for major changes.

9.9

In terms of quantitative indicators, the majority of firms define major changes based on a percentage change in the total SCR. An improved approach, adopted by some firms, specifies additional indicators at a more granular level, for example, indicators that relate to changes in the strength of the marginal risk distribution at certain percentiles or the amount of pre-diversified capital requirements for that risk.

9.10

It is important for the model change policy to include qualitative indicators for major changes. An example of a qualitative indicator is where a major change is triggered after a fundamental change in the methodology or a key expert judgement relating to a particular risk regardless of the impact that the change has on the SCR. Another potential qualitative major change indicator is if a proposed model change needs to be signed off at, or above, a certain level of seniority within the firm. Firms may also wish to consider what indicators might be appropriate to use to determine whether a major change might be triggered through ongoing model validation.

9.11

When developing major change indicators, the PRA encourages firms to consider the appropriateness of having different indicators or threshold levels for different risks or components of the model. For example, it may be desirable to include specific change thresholds for certain elements of the model that are of key interest because they are highly material, highly judgemental or have known limitations.

9.12

Finally, it is important that firms justify their choice of major change indicators including why any thresholds chosen are at an appropriate level for the ongoing supervision of the model. In this regard, it can be helpful if firms provide examples of model changes (eg past model changes) that meet their major change indicators in order to demonstrate the appropriateness of thresholds chosen.

9.13

Firms may struggle to articulate how they would define the circumstances in which a combination of minor model changes would constitute a major model change. Better model change policies have specified at least the following:

• how the impact of minor changes will be accumulated together;
• the time period over which these changes will be accumulated; and
• the indicators or thresholds used to determine when such an accumulation becomes a major change.

9.14

A reasonable starting point for each of these may be to:

• accumulate the absolute values of the impact of the minor changes together, unless it could be demonstrated why it would be reasonable to allow the impact of two minor changes to offset each other;
• accumulate changes from the date of the latest minor model change accumulation reset being, either at the end of an annual cycle (that firms may specify) or at the point of receiving a major change application (if that application is approved); and
• use indicators similar to those defined for single major changes, where considered appropriate.

9.15

A further consideration firms may wish to make is whether it is informative to group minor model changes together by risk or other common feature of the model.

9.16

The better model change policies clearly articulate the governance framework covering the internal process for identifying, approving and implementing the model changes. These included an articulation of how the model change policy fits in within the wider model governance, risk management and validation processes.

9.17

The PRA generally expects firms’ executive management to be responsible for the internal sign-off of major model changes and at least to be made aware of minor changes where appropriate.

9.18

It is important that firms also ensure that there is a robust governance process to agree whether changes should be classified as either major or minor, especially in cases where the classification is borderline or subject to judgement.

9.19

In addition to submitting major changes for approval, according to the EIOPA Guidelines on the use of internal models, firms are expected to provide a quarterly summary of minor model changes to the PRA. It may be helpful for the summary to group related changes together, for example by risk area or function of the model.

9.19A

The PRA expects firms with an approved internal model to submit their quarterly minor model change reports via the Bank of England Electronic Data Submission (BEEDs) portal.

9.19B

Documentation should be submitted as ‘occasional submissions’.

9.19C

The PRA expects firms with an approved internal model to submit their quarterly minor model change reports in line with the PRA Solvency II quarterly submission schedule¹⁷ for groups or more frequently where appropriate. Where possible the PRA encourages earlier submission to allow early engagement.

19.9D

The PRA has provided a template that firms can complete and submit, containing key information regarding minor model changes (Appendix).¹⁸

9.19E

While the submission of the relevant information via the template is optional, the PRA encourages firms to submit information consistent with that outlined in the template as a minimum within their quarterly minor model change reports.

9.19F

Firms may wish to submit supporting quantitative and qualitative document with their template submission.

9.20

The PRA encourages firms to review the effectiveness of the model change policy on a regular basis to ensure that the internal model continues to reflect the firm’s risk profile and meets the T&S. Firms are also reminded that any change to the model change policy itself is subject to the PRA’s approval. Readers also are referred to SS12/16, ‘Changes to internal models used by UK insurance firms’.¹⁹