Analysis Services in times of Continuous Integration

THANKS to all Sponsors! EVENT SPONSORS EXPO SPONSORS EXPO LIGHT SPONSORS

Meet me at the Community Zone After this session, you can speak with me in the Community Zone WE MIGHT Discuss additional questions Review parts of my session in more detail Network Take selfies…  IMPORTANT : ONLY SHOW THIS SLIDE IF YOU HAVE COMMITED TO BE AT THE COMMUNITY ZONE

Gabi Münster g.muenster@oh22.net @SQLMissSunshine BI consultant at oh22data AG since 2010 Experience with SQL Server since version 2000 Main topics: SSAS (Multidimensional/Tabular), SSRS, MDS Speaker at chapter meetings, SQL Saturdays, Pass Summit, SQL Bits g.muenster@oh22.net @SQLMissSunshine https://www.linkedin.com/in/gabimuenster/

Agenda Motivation - Some words about CI Motivation - CI Best practices Challenges for CI in BI A possible approach Continous deployment Demo and a warning – It‘s c# coding time Summary

Motivation - Some words about CI Continuous Integration (CI) is a process well-known and often practiced in software development. It‘s aim is to prevent integration problems at a very early stage of development. Originally it covers the part of merging development versions of all developers into a single version and testing the validity of the combined version. But today CI is often used as a synonym for the complete process of guaranteeing integrity of the current version througout the whole environment.

Motivation - CI Best practices Maintain a code repository Automate the build Support by tools like Jenkins, Travis CI, GoCD Make the build self-testing And log the results Everyone commits to the baseline every day Daily check in by all developers and merge into a single version Every commit (to baseline) should be built Implement automated build & deploy processes triggered by check-in

Motivation - CI Best practices Keep the build fast Errors should be identifiable quickly Test in a clone of the production environment The closer the environment to the productive system the more reliable are the tests Make it easy to get the latest deliverables Involve testers and stakeholders Everyone can see the results of the latest build Make build and test results visible Automate deployment Analogously to automated build Supported by tools like Octopus Deploy

Challenges for CI in BI (I) Data layer, Business Logic and Frontend different projects or technologies Integration tests over complete environment Data models prohibit parallel development XML based models barely suitable for merging Little support by standard tools High change frequency for requirements, especially during first phases of a project

Challenges for CI with Analysis Services (I) Maintain a code repository Developing with SSDT allows for integration of the projects into Source Control (e.G. TFS or GIT) Not commonly used at the moment Recommendable Automate the build Today: Manually out of SSDT or by script Make the build self-testing Not common yet Tests from technical perspective could be deploy and process Unit tests (e.g. nBI)

Challenges for CI with Analysis Services (II) Everyone commits to the baseline every day Parallel development in SSAS is tricky. Merge in SSAS Tabular possible, especially with the new data model Merge in SSAS Multidimensional not recommendable But either way even the single developer should commit each day  Every commit to baseline should be built Implement automated build & deploy processes triggered by check-in E.G. using MSBuild Support by tools like Jenkins, Travis CI, GoCD

Challenges for CI with Analysis Services (III) Keep the build fast Builds of SSAS projects normally are, tests might not be  Test in a clone of the production environment Keeping environments up-to-date containing all elements of the BI landscape might get hard Usage of Azure environments might help Make it easy to get the latest deliverables Keep versions available for QA and business approvers The smaller the change sets, the easier the test Automatic deployment of the complete environment?

Challenges for CI with Analysis Services (IV) Everyone can see the results of the latest build No difficulties Automate deployment Analogously to automated build Big improvement e.g. for large OLAP databases: Do small tests by processing dimensions and small portions of the cubes for first test, automatic deployment will take care of e.g. a nightly full deployment Supported by tools like Octopus Deploy

A possible approach Code repository Integrate your SSDT project into Visual Studio Online Automatic build VS online support for existing templates or individual build processes Automatic tests Create tests with nBI and integrate those into your build process Automatic deployment Use results from your build to deploy to your other environments And how?

Continuous deployment Build processes generate artifacts you can use for deployment You can use scripting languages like PowerShell or C# to deploy those artifacts With C# you can also investigate existing databases. This will allow you do a kind of schema compare. A very advanced example of this is the BISM Normalizer from Christian Wade: http://bism-normalizer.com/ This will allow you to do a partial deployment as well.

How to deploy your deploy your build artifacts using c# How to deploy your deploy your build artifacts using c#? First trials of a BI person – It‘s coding time  Demo

Summary Several reasons like a different way of requirement handling and a slightly different project lifecycle makes integration of standard software development approaches into BI sometimes quite complex. Support by software vendors is also still not on a satisfactory level. So applying CI approaches to your BI projects will require a lot of effort and development on your side. But more and more tools and technologies are helping us with this and improving your standard processes as well as the quality and stability of your development is in my opinion a worthy goal to pay that price.

Some useful links AMO for multidimensional databases: ASSL AMO Concepts and AMO Model ASSL Analysis Services Scripting Language (ASSL for XMLA) AMO for Tabular (TOM) Introduction to the Tabular Object Model (TOM) in Analysis Services AMO TMSL Tabular Model Scripting Language (TMSL) Reference Schema Compare for Tabular: BISM Normalizer http://bism-normalizer.com/

Some useful links Continuous Integration Testing for BI Build Tools https://en.wikipedia.org/wiki/Continuous_integration Testing for BI http://www.nbi.io/ Build Tools https://jenkins.io/index.html https://travis-ci.org/ https://www.gocd.io/ Deployment Tools https://octopus.com/

Any questions?

Please evaluate all sessions! QR / LINK on posters and in program