Testing of XBRL files for the supervisory reporting 5 May2014 Katrin Heinze Testing of XBRL files for the supervisory reporting Eurofiling Workshop in Rome

Objectives The objective of this presentation is to provide an overview on the testing strategy of the SUBA project to explain the test plan and sample test cases to inform about the next steps Eurofiling Workshop in Rome, 5th of May 2014

Agenda Overview 1 SUBA* test strategy 2 Sample XBRL test cases 3 Next steps * SUpervisory BAnking Data System Eurofiling Workshop in Rome, 5th of May 2014

Agenda Overview 1 SUBA Test Strategy 2 Sample XBRL test cases 3 Next steps Eurofiling Workshop in Rome, 5th of May 2014

Technical Test Execution Status SUBA Test Strategy Functional Tests Non-Functional Tests Regression Tests Based on User requirements Use cases Experience of users Technical requirements Source code (unit tests) … Categorised following Load tests (number of users) Performance tests (processing time) Volume test (quantity of data) Stress test Test of the (data) security Usability tests … Done after changes on Components XBRL taxonomy Validation rules Interfaces EXDI RIAD Infrastructure … Technical Test Execution Status 17.04.2014 Eurofiling Workshop in Rome, 5th of May 2014 Page 5

HP ALM usage within the SUBA project SUBA Test Strategy HP ALM usage within the SUBA project Scope Planning of releases and test cycles Enable traceability between requirements, tests, defects and according code changes Increase visibility with dashboards on analysis and KPIs Achievements Definition of test cases linked to the defined requirements Execution of test cases based on defined test data Defect tracking and documentation of bug fixes Way forward Increase the level of automation on test execution Customise reports and analysis based on the project needs Preparation of test cases for the next release Eurofiling Workshop in Rome, 5th of May 2014 Page 6

Test plan structure Conventions on numbering 1. level: Categories 2. level: Use cases 3. level: Test cases Conventions on numbering Structure in the Test Lab Eurofiling Workshop in Rome, 5th of May 2014 Page 7

Tracking of the test progress in HP ALM Eurofiling Workshop in Rome, 5th of May 2014 Page 8

Agenda Overview 1 SUBA Test Strategy 2 Sample XBRL test cases 3 Next steps Eurofiling Workshop in Rome, 5th of May 2014

Positive functional test cases Sample XBRL test cases Positive functional test cases Creation of test data for each module of the COREP taxonomy which is a valid XBRL instance document.   Valid according to XML specification XBRL 2.1 specification XBRL Dimension 1.0 specification XBRL Formula 1.0 specification EBA disabled validation rules EBA Filing Rules for COREP and FINREP By using different file sizes small medium large Eurofiling Workshop in Rome, 5th of May 2014 Page 10

Negative functional test cases Sample XBRL test cases Negative functional test cases Tests that ensure that the application shows a proper behaviour when invalid files are received. Creation of test data where the system is expected to provide correct error messages. Possible test cases : A reporting entity sends a file which is not a well-formed XML document. has not the correct format, i.e. jpg, txt, xls etc. has not the expected encoding. includes neither facts nor contexts. misses a namespace in the xbrli root element includes letters where numbers are expected as fact value misses an explicit member in the scenario element Includes a fact with no value but also no nil attribute is provided Refers to a non-existing module Refers to two different modules. Eurofiling Workshop in Rome, 5th of May 2014 Page 11

Agenda Overview 1 SUBA Test Strategy 2 Sample XBRL test cases 3 Next steps Eurofiling Workshop in Rome, 5th of May 2014

Voluntary functional tests with NCAs Next steps Voluntary functional tests with NCAs Test scope Test period: 13th to 15th of May Test activities: Acquisition NCA sends data to EXDI via WebUI or standard adapter. EXDI system sends a delivery message. SUBA system sends an acknowledgement message to NCA with the XBRL validation result NCA receives the messages in the scheduled time periods. Eurofiling Workshop in Rome, 5th of May 2014 Page 13

Next steps Load tests First estimation about the load expected for the first remittance date Number of reported entities 128 First remittance date NCAs to EBA/ECB 31.07.2014 Modules reported Number of files (per institute) Max. file size in MB Max. number of observations COREP consolidated Own Funds 1 115 177.536 Large Exposures 264 355.930 NSFR 9 7.510 Total 512 49.664 69.244.928 *) Estimations based on information of the Deutsche Bundesbank for one big German bank IT infrastructure for establishing a supervisory function at the ECB

Organisation and approach – Test case preparation Next steps Organisation and approach – Test case preparation Specification workshops Specification sign-off Implementation Development of software Definition of test cases (HP ALM) Tests execution Tests runs (HP ALM) Defects workflow (HP ALM) Bug fixing Control point A Control point B Control point C Control point Metric Target (by priority) Average High Medium Low A % reviewed requirements 100% B % requirements test coverage 75% 50% C % tests run 90% % tests succeeded Defect resolution time (working days) 3 5 Defects reopened ratio 5% 10% Eurofiling Workshop in Rome, 5th of May 2014 Page 15

Key issues and challenges Next steps Key issues and challenges Functional tests Supporting the voluntary functional tests with the NCAs, organized by WG SIM Load tests Defining scenarios for load tests on XBRL processing via automated test, simulating also peaks Preparation for next releases Defining test cases based on more precise user requirements EBA testing Participating in tests scheduled by EBA Eurofiling Workshop in Rome, 5th of May 2014 Page 16