Confidencial - TRACASA Automatize test [e- Reporting]

Confidencial - TRACASA INTRODUCTION  Data loading procedure is time consuming and complex  Aggregate calculation procedures are time- consuming and complex  Testing (even small) changes or new features in both procedures is time-consuming and tedious 2

Confidencial - TRACASA OBJECTIVE  Create automate testing in order to reduce  Bugs in data loading and aggregate calculations procedures  Facilitate data error detection  Reduce man power needed in these tasks 3

Confidencial - TRACASA APPROACH  Generic  Based on well-known best practices  Use of simple procedures and a structure capable of validating everything related to a SQL relational database: database existence, users, schema, data, results in a view, stored procedures,...  Anything included in a valid SELECT statement  Simple and understandable  An SQL test database with 3 tables and a generic store procedure  One of the tables storing a list of all "test cases" and the expected result  Each "test case" will test a requirement included in SELECT statement that must return maximum one row and one column  Multidatabase  The test cases can be launched against any database hosted in the same server as the SQL test database 4

Confidencial - TRACASA TEST TABLE STRUCTURE 5  Test_Plan  Table to store high level groups of test suits (Deployment version, FME data loading, Aggregate calculations and any other system that needs testing)  Test_Suite  Table to store for each [Test_Plan] groups of test cases  Test_Case  Table to store for each [Test_Suite], a specific unit validation Explained in more detail in the following slides

Confidencial - TRACASA Test_Plan 6 [definition]:Descriptive text [filter]: Optional text used to filter tests to execute. Each filter must be between the character ‘|’. e.g. |FME|deploy|

Confidencial - TRACASA Test_Suite 7 [definition]:Descriptive text [filter]: Optional text used to filter tests to execute. Each filter must be between the character ‘|’. e.g. |Load|Update|

Confidencial - TRACASA Test_Case 8 [definition]:Descriptive text [filter]: Optional text used to filter tests to execute. Each filter must be between the character ‘|’. e.g. |SP|Function|Index| [order_exec]: Integer to specify the execution sequence. Also to make the tests groupable [sql]: Text with t-sql command to execute. The sentence must return only one value which alias should be 'expectedresult‘ e.g. SELECT 1 AS expectedresult e.g. SELECT TOP 1 [field] AS expectedresult FROM [dbo].[table] [expectedresult]: Result expected from the sql statement

Confidencial - TRACASA HOW TO EXECUTE A TEST 9  The test database includes a general Stored Procedure [Exec_TestSuit]  Input parameters of the exec: name of the database which will be tested ID of the test suit. (optional) By default Null. Otherwise, filter the [Test_Case] where [filter] e.g. EXEC = = = Null; e.g. EXEC = = = ‘FME’; -- execute only the [test_case] where filter is like ‘|FME|’

Confidencial - TRACASA AIRQUALITY TESTS APPROACH  Design  Isolated test environment  Workflow including unattended test procedures  Setup  Implement empty test database  Identify, gather and store relevant and applicable data samples in files  Prepare and implement relevant queries to test procedures  Prepare cleaning test environment procedure 10

Confidencial - TRACASA AIRQUALITY TEST WORKFLOW 11 Initialize Environment Tests should always start from same starting point. Initial Test Check the database deployment before testing routines First Load (*) FME: Only Insert new data Agg: First generation Load = 1 Insert & Update (*) FME: Insert and update Agg: Generate again Load = 2 Insert & Update XX (*) FME: Insert and update Agg: Generate again Load = XX

Confidencial - TRACASA INSERT & UPDATE XX PROCESS DETAILED 12 FME Load Load a XML from UTD or CDR Workbench for metadata ‘D’ Workbench for measures ‘E’ Test FME Load Validate the Insert/Update in ‘D’ and ‘E’ SQL Job Aggregations Calculate the Agg. with the new delivery Test Aggregations Validate the result of the Aggregates

Confidencial - TRACASA AIRQUALITY TESTS IMPLEMENTATION 13  To ease the tests execution, three stored procedures has been created:  AQ_00_Deploy: to check database schema and master data  AQ_01_FME_Load: to check FME Load process  AQ_02_AGG_Load: to check Aggregates process  These three execute the general Stored Procedure [Exec_TestSuit] with specific parameters

Confidencial - TRACASA INITIALIZE ENVIROMENT 14  Environment under control  Restricted access  Manual process to:  Initialize the database to a clean status (tests should always start from same starting point)  Restore database or execute scripts to delete all data and restart the identity values e.g.: ‘AQ_clean_database_initial_state.sql’ Full workflow Workflow of each “Insert & Update XX” Overview

Confidencial - TRACASA INITIAL TEST 15  Execute AQ_00_Deploy to validate database deployment and specifically correctness of before testing process:  Stored procedures, indexes, functions, table structure, master data (eg: records in Unit table, timezone in AggregateType table…) USE [Test_SQL] GO EXEC = = NULL Full workflow Workflow of each “Insert & Update XX” Overview

Confidencial - TRACASA First Load - FME 16  Use specific XML files to load for the first time well known data  Launch the FME Load ‘0_Main.fmw’ to fill the metadata (D)  Launch the FME Load ‘E_Batch_Workbench.fmw’ to fill the measurements (E) Test with AQ_01_FME_Load (test cases not implemented yet) USE [Test_SQL] GO EXEC = = = NULL Full workflow Workflow of each “Insert & Update XX” Overview

Confidencial - TRACASA First Load - AGGREGATES 17  Launch aggregates calculations procedures Test with AQ_02_AGG_Load USE [Test_SQL] GO EXEC = = = NULL Full workflow Workflow of each “Insert & Update XX” Overview

Confidencial - TRACASA INSERT & UPDATE - FME 18  Use specific XML files to update and/or insert new well known data  Launch the FME Load ‘0_Main.fmw’ to fill the metadata (D)  Launch the FME Load ‘E_Batch_Workbench.fmw’ to fill the measurements (E) Test with AQ_01_FME_Load (test cases not implemented yet) USE [Test_SQL] GO EXEC = = = NULL Full workflow Workflow of each “Insert & Update XX” Overview

Confidencial - TRACASA INSERT & UPDATE - AGGREGATES 19  Launch aggregates calculations procedures  Test aggregates calculations procedures USE [Test_SQL] GO EXEC = = = NULL Full workflow Workflow of each “Insert & Update XX” Overview

Confidencial - TRACASA REPEAT 20  Loop through Load procedures, testing new updates and inserts using different data files and test all the casuistic as many times as needed to have a comprehensive testing Full workflow Workflow of each “Insert & Update XX” Overview

Confidencial - TRACASA KEEP IN MIND 21  When new features or changes are implemented in the Load or Aggregations procedures:  Check if tests cases are still valid or need changes  Check if new test cases are needed

Confidencial - TRACASA By David Alonso