1. IOPS Toolkit for Risk-based Supervision Module 2: Quantitative Assessment of Risk

2. Overview – Quantitative Assessments Play an important part RBS – poor QA results imply higher levels of residual risk to be factored into the overall risk analysis or risk score Quantitative tools can provide a bridge between rules-based and risk-based approach to supervision Quantitative tests can be undertaken by the supervisory authority itself or by pension funds QA should focus on the risks relevant for the type of fund (i.e. funding and solvency issues for DB funds, investment returns and volatility and reliability of retirement income for DC funds) Not all risks can be measured in a quantitative fashion – the overall risk assessment will always be a combination of quantitative and qualitative factors

3. Models Quantitative assessment tools make use of models – 3 main limitations: what is modelled and how: including vital omitting factors, use of inappropriate data, only using recent data in development of models and their parameters, failing to take account of extreme events, using the same accepted view understanding the power and limitations of models: including their use outside their sphere of applicability, hidden assumptions and overestimating the power of models which are simplified representations operational risks around the use of models: poor documentation, lack of testing, and the misuse of data

4. Modelling - Recommendations It is recommended that models should: sufficiently represent those aspects of the real world that are relevant to the decision makers for which the information will be used include explanations of how the inputs are derived and what the outputs are intended to represent be fit for purpose in both theory and practice include explanations of their significant limitations

5. Quantitative Regulations Quantitative regulations are the starting point for quantitative risk analysis These can be straight forward limits (such as minimum funding rules for DB funds or investment restrictions for DC funds) Alternatively, these quantitative regulations can be risk-based themselves (e.g. factor based solvency rules, VaR calcuations) Under a risk-based approach, supervisors need to consider not only whether quantitative regulatory requirements are being met, but whether risks are being identified and managed in such a way that the requirements will continue to be met in the future

6. Quantitative Regulations Risk-based supervision can incorporate quantitative regulations in 3 ways:  Combine ‘rules-based’ and a ‘risk-based’ approach – compliance with quantitative restrictions is checked, and if not met a lower score would be factored into the overall risk assessment of the fund  Quantitative requirements could be made more ‘risk-based’ but testing whether compliance would still hold in adverse circumstances (i.e. by stress testing) -the results of these stress-tests would then be incorporated in the overall risk score  Where the quantitative regulations are already risk-based,compliance with these risk-based regulations would be fed into the overall risk score

7. Quantitative Regulations: DB Funds Valuation requirements : these can be tested and incorporated into an overall risk analysis by looking at the valuation assumptions, undertaking sensitivity testing of changes in valuation assumptions, and stress testing of risks such as high inflation Minimum funding requirements: these can be incorporated into an overall risk assessment either by checking for compliance (and scoring the fund accordingly) or by stress testing the funding position to see if the minimum requirements would be met under adverse circumstances (with the results of such tests fed into the risk score) Factor-based solvency margins: solvency margins can be risk-based by requiring higher amounts of capital to be held against risky assets (such as equities), thereby providing a buffer in case such assets decline in value. Either straight forward or stress-tested compliance with these margins can be incorporated into an overall risk score Stress-related solvency margins : require each entity to calculate the additional amount of assets it would need to be able to meet its obligations under a prescribed stress scenario or scenarios. The results are then fed into the overall risk assessment

9. Quantitative Regulations: DC Funds Investment limits: compliance with these limits forms part of the overall risk score Minimum return requirements: solvency requirements backing guarantees would be measured and assessed in the same way as DB fund promises Value at risk limits: these assess the volatility of investment returns. Results of such stress testing would be incorporated in the overall risk score. Where such limits are themselves regulatory requirements, compliance would be part of the overall risk assessment Alternative risk measures: attempt to measure risk against long-term income requirements (such as replacement ratios), with regulators devising optimal portfolios for achieving this target. The performance of the actual portfolio of a pension fund could then be assessed vs. this benchmark portfolio. Supervisors could then work this analysis into their overall risk assessment via a ‘traffic light’ system.

11. Techniques for Quantitative Risk Assessment Comparison of valuation assumptions – compare assumptions with peers, previous assumptions of entity, and consideration of current environment Analysis of surplus –compare actual experience to assumptions to assess appropriateness / accuracy of assumptions Roll-forward calculations –financial position projected under certain scenarios to assess exposure to adverse circumstances Duration analysis – project cash flows of assets and liabilities of fund to determining timing mismatches, as well as interest rate sensitivity Sensitivity testing – test sensitivity of valuation results to differences in assumptions by recalculating results using alternative assumptions Deterministic stress testing – calculate the financial position of a pension entity at current or future date to one or more adverse scenarios Stochastic stress testing –calculate the financial position of a pension entity at current or future date using computer generated adverse scenarios Value at risk (VaR) calculations – type of stochastic stress test measuring adverse market movement with a specified probability

12. Techniques for Quantitative Risk Assessment

13. Integrating Quantitative Tools Defined Benefit Solvency and funding ratios are the key quantitative tests for DB funds – above 100% resulting in low risk score, below implying a higher level of risk Roll forward and stress tests would be useful indicators – if funding remains over 100% after stress testing in negative scenarios, a low risk score would result ALM – seeks minimise and manage the asset related risks as a function of liabilities, thus ALM testing could indicate a sign of robust risk management at a pension fund

14. Integrating Quantitative Tools Defined Contribution VaR – measuring investment volatility can be used for DC funds, though is considered controversial ALM type measurements to see if DC funds can meet income replacement targets are still under development For DC funds offering guarantees, similar standards and solvency stress tests of DB funds can be used

15. Quantitative Measurement of Non- financial Risk Non-financial risk encompasses operational risks – the degree of complexity of the fund and the capacity to handle the complexity. It also includes governance, management, internal controls and independent review. Difficult to quantify, although very useful as “leading” indicators. DB funds are inherently complex due to benefit design, such as early retirement benefits, indexation etc., thus attracting a higher risk score. DC funds offering a large range of investment options or “life-cycle” investment; pooled investment returns “declaring” rates on a non- transparent smoothing approach rather than market basis and high levels of outsourcing, would attract a higher risk score.

16. Thank You Presentations of practical examples to follow