1. Automating the testing of your BI solutions with NBi Cédric L. Charlier

2. Thank you to our sponsors!

3. Who am I?  Business Intelligence, Data & Information Architect  MVP SQL Server from Belgium  Principal committer for NBi  Open source contributor  Agile enthusiast  http://seddryck.wordpress.com  @Seddryck

4. Agenda  Why automating is a must?  Tests with queries (Sql/Mdx/Dax)  Tests with ETL packages (SSIS)  Testing models (SSAS and DB engine)  Automating the creation of test-suites

5. What’s automated testing? A piece of software written to reproduce steps/actions defined in a manual process

6. Benefits of automation Allow testing to happen more frequently Speed up testing to accelerate releases Improve test coverage Ensure consistency

7. Tests automation doesn’t apply to Business Intelligence

8. Wednesday, November 24, 2012 Est. 1869 Price 6d More than 33% of BI projects fail to deliver Lack of agility. Aenean commodo ligula eget dolor. Aenean massa. Cum sociis natoque penatibus et magnis dis Aenean commodo ligula eget dolor. Aenean massa. Cum sociis natoqueCum sociis natoqueCum sociis natoque Low quality of deliverables Aenean commodo ligula eget dolor. Aenean massa. Cum sociis natoque penatibus et magnis dis parturientAenean commodo ligula eget dolor. Aenean massa. Cum sociis natoqueCum sociis natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. Donec quam felis, ultricies nec, Simply Too late. Cum sociis natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. Donec quam felis, ultricies nec, Member of the Asscoiated Press. Aenean commodo ligula eget dolor. Aenean. Aenean commodo ligula eget dolor. Aenhswse. Cejhciebce fcdcdcd. ILLUSTRATED WEEKLY NEWSPAPER

9. We haven’t time to create these tests

10. Time to execute a test-suite Exponential slope Why? -New objects -Old objects

11. Time to write a test-suite Initial gap

12. No existing framework for automating tests dedicated to BI … I’m not proficient enough in C#

13. This framework exists: NBi No C#, just xml syntax Open- source

14. Agenda  Why automating is a must?  Tests with queries (Sql/Mdx/Dax)  Tests with ETL packages (SSIS)  Testing models (SSAS and DB engine)  Automate the creation of test-suites

15. Compare queries … result-sets System-under-test Assert Previous release Two different releases of a same data warehouse Tabular model versus Csv file DAX … or any brilliant idea Cube and data warehouse MDX SQL

16. Demo: compare queries’ result  Compare cube/operational database  101  Compare cube/flat files  Defaults and References  Tolerance  Intervals

17. Other testing goals for queries  Performance of a query  Executed in less than x seconds  Cache management  Syntax of a query  Extract queries from reports and assemblies

18. Query’s result and format  Result’s format  Example: all the cells have effectively 2 digits after the decimal separator and this separator is a comma.

19. Check calculations in your query  Assert a logical test, row by row, for you result-set  Example: Calculate the total price from the unit price, quantity and discount percentage.  If at least one row doesn’t validate this logical test, the whole test shift to red

20. Assert row count  Assert how many rows are returned in the result-set.  Equal, more-than, less-than

21. Agenda  Why automating is a must?  Tests with queries (Sql/Mdx/Dax)  Tests with ETL packages (SSIS)  Testing models (SSAS and DB engine)  Automate the creation of test-suites

22. Demo SSIS  Assert the execution status of an ETL  Alternatively test performances

23. Inside this ETL Dimension Calendar Dimension Scientist insert Unique index violation Dimension Geography

24. Setup and Cleanup  Setup:  Define the state of your testing environment before executing a test  Cleanup:  Remove consequences of your tests on the testing environment

25. Tasks and behaviors  Available tasks  Truncate/Load tables with csv files  Run sql batch, exe or batches  Run Etl packages  Move or delete files  Start/Stop Windows services  Run-once: execute only once this set of tasks  Parallel: run a set of tasks in parallel

26. Demo Setup  Setup to register a clean state before test’s execution  Assert database’s state after the execution of an ETL

27. Limitations  ETLs must be stored  SSIS Catalog  File share  SQL Server  Tests must be executed on the server running the ETL  No remote testing  v1.10 supports SQL Server 2014  v1.11 supports SQL Server 2008R2, 2012 and 2014 Currently in preview

28. Agenda  Why automating is a must?  Tests with queries (Sql/Mdx/Dax)  Tests with ETL packages (SSIS)  Testing models (SSAS and DB engine)  Automate the creation of test-suites

29. Testing dimension’s members  “Norway” is a member of dimension “Country”  “Norway” is not a member of the set “European Union”  Dimension “Country” has more than 190 members.  All members of dimension “Country” have at least 4 characters  Members of hierarchy “Month” are ordered chronologically (and not alphabetically)  Members are equivalent to members of this other instance  …

30. Example: members’ pattern

31. Testing structure (SSAS)  Assert that a dimension, hierarchy, measure, set, … is visible for end-user in a given perspective.  Assert the existence of a relation between a measure-group and a dimension.

32. Testing structure (DB engine)  Assert existence of a  schema,  Table  Column  Routine (stored procedure or function)  Assert the type of a column  Full type: decimal(10,3)  Light type: decimal Currently in preview

33. Agenda  Why automating is a must?  Tests with queries (Sql/Mdx/Dax)  Tests with ETL packages (SSIS)  Testing models (SSAS and DB engine)  Automate the creation of test-suites

34. Some tests are really repetitive It should be possible to automate the creation

35. It exists! Take a look at Genbi A template for the tests A set of test- cases

36. First steps with Genbi File loaded from Genbi Variables to be substituted (surrounded by $)

37. Demo: Genbi  Load test-cases  From a flat file  Select a template  Generate the test-suite  Grouping option

38. I’m scripting macros to automate the actions performed by genbi … With genbiL!

39. Sample for genbiL Subject Action Parameters

40. Demo for genbiL  Command line

41. Conclusions  Benefits of automated tests  Save Time (Time-on-market)  Better Quality (end-result)  More confidence  NBi  Supports large range of different tests  No need of compiler (neither C#)  Good feedback when some tests are failing  Open-Source  Automate the creation of your test-suites (genbi / genbiL)

42. Read further http://www.nbi.io/  Download releases  Read documentation  Report bugs  Suggest new features

43. Your feedback is important! Please let us know what you thought of this session http://sqlsaturday.com/414/sessions/sessionevaluation.aspx

44. Thank you to the organizers and volunteers!