Training – Creating Tests & Executions Green - #92C82B Orange - #FE6701 Blue - #2266A5 Grey - #626262 Training – Creating Tests & Executions May 2015

Content quilliup Training Getting to know the basics Advanced abilities Infrastructure abilities Green - #92C82B Orange - #FE6701 Blue - #2266A5 Grey - #626262

Getting to know the basics 1 Green - #92C82B Orange - #FE6701 Blue - #2266A5 Grey - #626262

Creating Tests & Executions Getting to know the basics Diving Deeper Infrastructure abilities Intro and Navigation Creating a new test Compare rule Threshold Test Result and History Private and library data sets Custom Rule Monitor Exporting & Emailing results Library variables Library scripts Creating an Execution flow Scheduling Advanced Execution options Compare rule – compare settings Logging the results Versions management Bulk operations Performance tracking, scalability and test settings

Creating Tests & Executions 3 Getting to know the basics Green - #92C82B Orange - #FE6701 Blue - #2266A5 Grey - #626262 Advanced abilities Infrastructure abilities

Creating Tests & Executions Getting to know the basics Intro and Navigation Logging in to the Application After logging in to the platform you will see your authorized modules. Our default home page is the “My Apps” page. You can set the default home page of each user at the Administration User Management and navigate directly to the relevant module entrance page. Side bar menu Each application contains a Projects area and Management settings area. A project will represent a unit of objects according to teams and initiatives and will also be a base for the permissions model. Management settings will contain different management entities relevant to the module.

Creating Tests & Executions Getting to know the basics Intro and Navigation Tags Tags management enable users to create multiple dimensions perspective that will enable to navigate and locate objects easily as well as analyzing the quality KPIs and results according to multiple perspectives. You will find the tag management under the management settings section where you can create tag groups and assign values. These tags will be available for use in any of the objects (Tests, Data sets, Execution flows, etc…), where you will be able to multiple tag each object.

Creating Tests & Executions Getting to know the basics Intro and Navigation Searching objects and creating customized lists Finding an object is quit easy. Under each project you will see the different objects types, If you enter Tests for example, you will reach the tests table, where you can search according to any information that appears in the tests table. We also have a main search page that searches different objects across the system. Quick filters on the left and the “Save as Favorite” option will make your life easy by creating pre-defined user customized objects lists.

Creating Tests & Executions Getting to know the basics Creating a New Test Adding a new test Go to the project  Tests and click on creating a new test You will be asked to provide the meta data for the test. It is highly important to tag the test

Creating Tests & Executions Getting to know the basics Creating a New Test Selecting the relevant data sets Select from the menu on the right side relevant data sets (queries) from shared library (pre defined data sets) or create a private tests data set One or more data sets will be followed by a rule and result settings When using the compare rule – you are allowed to use as many data sets for comparison as you wish, they can be from completely different data sources and types

Creating Tests & Executions Getting to know the basics Compare Rule Select compere rule any time you wish to compare between different data sets in any resolution. Compare type will enable you to select between different compare resolutions: Data Integrity – validate if all records exist or not according to a selected key (also alerts on key duplication) Aggregated Data – validate missing grouping keys and measures consistency Raw Data – validate missing records and different values at the row level, reveal the discrepancies root cause records and columns. Compare mapping step – map the keys, columns and measures you would like to compare according to the selected type

Creating Tests & Executions Getting to know the basics KPI Rule Select KPI rule any time you wish to validate your data sets with aggregation functions. KPI type will enable you to select between 2 resolutions: KPI range – aggregate data and choose fix values as threshold KPI snapshot – aggregate data and choose last run results as threshold KPI mapping step – map the keys, columns and measures you would like to calculate and measure KPI Range

Creating Tests & Executions Getting to know the basics KPI Rule Select KPI rule any time you wish to validate your data sets with aggregation functions. KPI type will enable you to select between 2 resolutions: KPI range – aggregate data and choose fix values as threshold KPI snapshot – aggregate data and choose specific run results as threshold KPI mapping step – map the keys, columns and measures you would like to calculate and measure KPI Snapshot

Compare column threshold Creating Tests & Executions Getting to know the basics Thresholds Set customized tolerance level and define the relevant statuses and range. There are two types of thresholds: Tests threshold – sets the final test status according to number or percent of failed records. User can also add an alert range. Column threshold – set the threshold for each column separately, user can add an alert range and define the relevant gaps and direction. *This threshold impact the status of the row and therefore also impact the general test threshold. Test threshold Compare column threshold KPI threshold

Creating Tests & Executions Getting to know the basics Test Results and Test History Test results are presented at the bottom of the page according to the compare type and compare settings. Compare rule results Failed values presented for all kpi types – the default test perform a validation of keys in the data set which to the total keys population pf all data sets (changing the compare settings as will be explained later, will impact the results). Different records appears at the second section and is relevant for agg compare or raw compare where column/measure values are being compared. The results displayed show in which column there are discrepancies out if those who were configured for comparison in the column mapping step of the rule.

Creating Tests & Executions Getting to know the basics Test Results and Test History KPI Range results Failed and success values presented for all KPI types – the default test perform a validation of keys in the data set which does not met the threshold value KPI Snapshot results Failed and success values presented for all KPI types – the default test perform a validation of keys in the data set which does not met the threshold value, comparing to last test’s execution

Creating Tests & Executions Getting to know the basics Test Results and Test History Results History Results history displays all previous runs. You can view the results and run statistics of a previous run by clicking on a specific row.

Creating Tests & Executions Getting to know the basics Private and library data sets A data set can be created or shared to the library. A library data set can be reused and managed centrally Library data sets appear in the test menu When creating a private data set within a test the option will appear A data set can receive a business name and description that describe its internal logic, technical columns can receive friendly names. Library data set can be used by business users without the need to know how to write the query logic Any library data set can be edited only from the library location Any library data set can be viewed or modified by creating a local copy in a test Library data set in project menu Library data set in test menu

Creating Tests & Executions Getting to know the basics Custom Rule Select custom rule any time you wish to use your own logic. All rows returned in the data set will be considered as failed rows. The custom will receive one data set as an input (the data set can be based on any data source type). Custom mapping step – map the fields to be returned in the results. 1 - Define test flow 2 – Map participating columns 3 – View results

Creating Tests & Executions Getting to know the basics Monitor The monitor enables you to be in control at any time knowing what’s currently running, and results of last executed objects. Navigate by clicking the monitor icon at the bar. Click on view in full page to go over to the dynamic monitor search. Test results are accessible by clicking the test name.

Advanced abilities 2 Green - #92C82B Orange - #FE6701 Blue - #2266A5 Grey - #626262

Creating Tests & Executions Getting to know the basics Diving Deeper Infrastructure abilities Intro and Navigation Creating a new test Compare rule Threshold Test Result and History Private and library data sets Custom Rule Monitor Exporting & Emailing results Library variables Library scripts Creating an Execution flow Scheduling Advanced Execution options Compare rule – compare settings Logging the results Versions management Bulk operations Performance tracking, scalability and test settings

Creating Tests & Executions Diving deeper Exporting and Emailing results Exporting results to Excel Results can be exported into an Excel file, all keys are presented with the compared columns results

Creating Tests & Executions Diving deeper Exporting and Emailing results Exporting results to Excel Test is presented with meta data The file has 2 sheets – different values and missing keys Status can be filtered in the file at the row level (key) or at the column level by filtering the column status

Creating Tests & Executions Diving deeper Exporting and Emailing results Emailing test results Each test can send an Email alert with the test status as well as sample and/or full results data attachment. Go to the share options on the right side of the test. Define recipients and Email definition at the Email settings. Enable/disable mailing while working or de-bugging the test. Email settings enables to define more than one group of definitions – several groups of recipients can be defined, each receiving a different Email/content.

Creating Tests & Executions Diving deeper Exporting and Emailing results Emailing test results Each group of recipients has its own set of definitions. Type an entity to be searched (a search button will appear) or type a full Email address (an Add button will appear). Search can be done directly in Active Directory to find users or Email groups of AD, or for application entities like users, groups and Email groups. Email template has to be chosen, Email templates can be added in the Administration. Email subject – can be modified, can combine test meta data and result variable by using the ‘@’ char . Email body – can be modified, can combine system variables as well. Email content – enable sample data display in the Email, enable script query as attachment, display detailed execution error if the test failed at execution. Send on status – which test result sends an Email Attachment – attach full data results (according to the limit at the test config) Some of the fields are disabled in display according to Administration Email settings.

Creating Tests & Executions Diving deeper Library variables Creating a new variables Variables are assigned values that can be combined within the Quality Gates processes. Variables are created at the project library You will be asked to provide the meta data for the variable. It is highly important to tag the variable

Creating Tests & Executions Diving deeper Library variables Creating a new variables Variables are assigned values that can be combined within the Quality Gates processes. Variables are created at the project library You will be asked to provide the meta data for the variable. It is highly important to tag the variable

Creating Tests & Executions Diving deeper Library variables Creating a new variable There are 2 types of variables: Fixed value – assign of a fixed value to a variable enables to manage “hard coded” values in one central place Single value query – a query (using the query builder or free command) that perform a calculation “on the fly” when the process that uses it runs, and returns a single value

Creating Tests & Executions Diving deeper Library variables Query variable options The calculated query variable is based on similar abilities to a data set. This means that query can be compiled in the query builder or by free command There are 3 options to define the returning value: Restrict to one column – if the query returns several rows and columns, it will use only the first row and column Concatenation of columns – if the query returns several columns, they will be concatenated into a single value Concatenation of rows – if the query returns an array of values, they will be concatenated into a single value Nested variables are supported – each query can use another variable

Creating Tests & Executions Diving deeper Library variables Using a variable within a process Variables can be combined in different process, at first releases they are can be combined in queries (data sets and calculated variables). For example in a data set query builder filter section For example in a data set free query

Creating Tests & Executions Diving deeper Library variables Assigning a value to a variable Assigning a value to a variable is available in all supported object by clicking on the right side of the page You can change the variable calculated/fixed value by assigning a specific fixed value for the variable from within the run process. You can also use a nested variable for calculation. The value assigned in the object will overcome the variable value, any external entity will overcome an internal one (for example assigning a value at the test level will overcome an assigned value at the data set level). Setting variables values will be available at the following objects according to order (internal to external) Variable query Data set Test Script Execution flow global variable (sets the value in all participating objects that use it within the flow) Execution flow private variable (sets the value in a specific object within a flow, will overcome a global variable value) – to set the variable click on inside the object External execution of an object can be performed with an assignment of a variable value (will be detailed in the external execution section)

Creating Tests & Executions Diving deeper Library scripts Creating a new script Scripts are commands that can be combined within the Quality Gates execution flows. Scripts are created at the project library. You will be asked to provide the meta data for the script. It is highly important to tag the script.

Creating Tests & Executions Diving deeper Library scripts Script types Supported script types: CMD Shell script/command Sql execute – SPs, transaction commands Sql query – return a value Script steps The script can contain different steps (of the same type) Steps order can be changed Dependency type of a following step in a previous step failure can be configured Return value The default return status of a script is a successful execution. If the scripts is returning a value, it is possible to specify a an expected success/failure value and use the script as a trigger

Creating Tests & Executions Diving deeper Creating an execution flow An execution flow is a process of tests (or scripts and other execution flows) with dependencies between them The available objects for use within an execution flow are located in the menu on the left side of the flow Each test can have an input or an output with the following dependencies types: Completion – previous test run and finished in any one of the statuses Failure – test status is failed Warning – test status is warning Success – test status is success The flow can have several roots (meaning that tests can run in parallel)

Creating Tests & Executions Diving deeper Creating an execution flow Adding a new execution flow Go to the project  Execution Flows and click on creating a new execution flow You will be asked to provide the meta data for the execution flow. It is highly important to tag the flow

Creating Tests & Executions Diving deeper Creating an execution flow Selecting the participating objects Select participating objects from the flow menu at the left side all project objects will appear for each type, you can filter by name Tests Execution flows – you can run nested execution flows and reuse part of the process in different flows Scripts – combine different types of commands within the testing flow Notes – add user notes to the process for documentation Creating dependencies You can connect between different objects and create a dependency run. Draw a line from one object to another, double click it and select one dependency type Any connection can be removed by selecting it and delete by keyboard

Creating Tests & Executions Diving deeper Schedule Scheduling an execution flow Scheduling process in Quality Gates is available within an execution flow by clicking on A scheduled flow will have an indication Scheduling options Every scheduler can be enabled/disabled A flow can be scheduled in any of the available intervals (minutes, hourly, daily, weekly, monthly, yearly) The scheduling can be narrowed to specific dates range Cron expression can be viewed and edited Recommended implementation We recommend you to create a master execution flow for each scheduling The master flow will run small units of execution flows to enable parallelism and organized view of the processes

Creating Tests & Executions Diving deeper Advanced execution options Execution from UI All objects can be run from UI using the will run on partial data according to the test config and will not be counted in any statistics Scheduled processes Objects can be scheduled within execution flows using the settings External execution Any runnable object (tests, scripts and execution flows) can be run by using a command (can be run from any type of server/application) that will run the process URL The execution can be parameterized Assigning variable values when operating the external command Assigning an internal batch identifier (like project or tag) that will run all related objects

Creating Tests & Executions Diving deeper Compare settings Compare settings Getting back to the compare rule: it contains 2 steps of configuration. Compare type – sets the compare resolution Compare settings – sets the join type Data Integrity Performs validation of missing records according on a selected key There are 2 types of compare settings Full Comparison – performs a validation that a key that exists in the unified keys population, exists in each data set (cross join) Compare to master – performs a validation that a key that exists in the master data set, exists in other data sets (left join)

Creating Tests & Executions Diving deeper Compare settings Aggregated Data/Raw Data In both tests types different values are being located so the settings are identical There are 3 types of compare settings Full Comparison – performs a validation that a key that exists in the unified keys population, exists in each data set (cross join), these records will appear in the missing rows section of the results. Different values are searched for keys that exist in the master data set and compared to it. Compare different values – performs only a validation of different values for keys that exist in all data sets (inner join). Compare to master – performs a validation that a key that exists in the master data set, exists in other data sets (left join), these records will appear in the missing rows section of the results. Different values are searched for keys that exist in the master data set and compared to it.

Creating Tests & Executions Diving deeper Compare settings Aggregated Data/Raw Data In both tests types different values are being located so the settings are identical There are 3 types of compare settings Full Comparison – performs a validation that a key that exists in the unified keys population, exists in each data set (cross join), these records will appear in the missing rows section of the results. Different values are searched for keys that exist in the master data set and compared to it. Compare different values – performs only a validation of different values for keys that exist in all data sets (inner join). Compare to master – performs a validation that a key that exists in the master data set, exists in other data sets (left join), these records will appear in the missing rows section of the results. Different values are searched for keys that exist in the master data set and compared to it.

Creating Tests & Executions Diving deeper Logging the Results Enabled your Data Source for logging  Create\Update Data Source and choose Enabled for logging. In the result cube choose Log Results button.  Choose the right Data Source and fill all the details. See the Log Tables in your Data Base

Infrastructure abilities 3 Green - #92C82B Orange - #FE6701 Blue - #2266A5 Grey - #626262

Creating Tests & Executions Getting to know the basics Diving Deeper Infrastructure abilities Intro and Navigation Creating a new test Compare rule Threshold Test Result and History Private and library data sets Custom Rule Monitor Exporting & Emailing results Library variables Library scripts Creating an Execution flow Scheduling Advanced Execution options Compare rule – compare settings Logging the results Versions management Bulk operations Performance tracking, scalability and test settings

Creating Tests & Executions Advanced abilities Version management All objects in the system have an internal version management mechanism that appears where the is available This means that you can view and interrogate history versions in case of problem, without performing a user initiated operation. Clicking on will display last versions feed specifying the changed characteristics, you can select and switch to history version. History run is available for viewing (you can view and go back to the current version by You can restore a history run and make it your current version by using the option.

Creating Tests & Executions Advanced abilities Bulk operations Bulk operations will appear in the objects grids (data sources, data sets, tests, execution flows, etc…) and in the search page. It will enable a user to perform different operations for a batch of objects. Available operations vary from object type to another and will usually include : copy, delete and update of specific characteristics Some of the operations are still blocked at first releases, but will be added in future versions.

Creating Tests & Executions Advanced abilities Performance tracking Run statistics details can be viewed at the run details at the lower section of the results. Each run is split into chunks according to a system decision based distribution key. Executions can split into multiple chunks, each chunk (sub process) can run on different nodes and servers. Defaults of chunk size and the distribution key can be customized at the test settings. Chunk details enables technical users and administrators viewing current run progress or perform performance tracking and analysis. The engine can run on any available memory. When Low memory is allocated, and the data is split into multiple chunks, the query is sent to the database for each chunk separately, and the processing time will increase – but the process will be able to finish the run with the limited resources.