Telling Stories with Data Gogula G. Aryalingam

Speaker Profile Software Architect at Navantis MCT, MCSE (BI and Data Platform) Former SQL Server MVP Subject Matter Expert for developing SQL Server certification exams Author of a Chapter on the SQL Server MVP Deep Dives II book

Data is All Around… Telling Stories with Data

Data is all around Look around your organization… You’ll find data all over… …in various places 4 | Telling Stories with Data

Data is all around Look at the world… ? You’ll find data all over… …in various places ? Exchange Rates Other Stock Market 5 | Telling Stories with Data

A need to make sense of this data… Telling Stories with Data

Before… traditional We used Business Intelligence… ^ Heterogeneous data sources Data Marts/Data Warehouse Reports/Visualizations/Analysis Typically, a business intelligence solution looked like what is depicted in this slide. Raw data (i.e. data that is in its native structure, eg: transactional systems, spreadsheets etc.) are extracted from their source systems, brought together, transformed and loaded (a.k.a. ETL or Extract, Transform and Load) into a special type of database called data warehouses or data marts. These are essentially relational databases that are structured differently from a traditional OLTP structure. This structure which is usually designed using the dimensional modelling technique is optimal for reading. A data mart is essentially similar to a data warehouse, except that it contains data from a single department or silo of an organization, whereas a data warehouse contains information from across the organization. Data from the data warehouse/data mart is then pulled into a special type of database called OLAP databases, which have structures called cubes instead of tables. A cube has multiple dimensions, much unlike tables which have only two dimensions (rows and columns). A cube can have 2, 3, 4 or more dimensions; making it a very fast database for reading large amounts of data for reporting and analysis. Finally, the visualizations. From enterprise reports that are 10 pages long that no one would read to nifty dashboards that show the state of the business at-a-glance can be created from the data that is collected. Traditional BI (or the traditional way of doing BI, a.k.a. Corporate BI because business intelligence was usually used by corporates in a large scale) mostly stored data in a data warehouse. This data was periodically updated as new data came into the source systems. The updates were usually performed on a weekly, or nightly basis; and as time went by, and when technologies became better and cheaper, the updates were performed on an hourly and sometimes every minute. These systems usually provided hindsight into the data and in certain cases some insight as well. Still, traditional BI involved high costs for enterprise scale software and hardware. It was specialist-built or built by IT, and in a lot of cases did not exactly cater to what the business users wanted. These BI systems were usually used by top-level management who mostly looked at the very high level picture of the business (dashboards) and some business users who did some analysis on the data (reports, interactive reports, scorecards). And in most cases these systems were not used at all. Most BI projects take a quite a long time to complete. Some take as much as 2-3 years, whereas a few others take almost 5 years… A lot are abandoned part way through. Most of the time it is because of the ambitious nature of trying to build the system for the entire organization, and due to no proper understanding between the technical folk building the system and the business folk who are the stakeholders, and during this (long) time things happen: people move out, new ones come in, requirements change, technologies get better (and others go obsolete) – visions are lost. Changes take long to be incorporated etc. etc. Extract, Transform & Load OLAP/Cubes Source: http://is.gd/L7QCQc Telling Stories with Data

And in a lot of cases, not used at all… Top Management CxO Some business users Typically used by And in a lot of cases, not used at all… Traditional BI systems were usually used by the top management of an organization, including CxOs and some business users. These users mostly required high-level information such as dashboards and scorecards, and graphical reports. Decisions were usually taken based on information presented through these media. Rarely was any data discovery done – Since systems were too complicated for them to use. Certain business users though, took it upon themselves to perform further analysis and then presenting their findings to the top personnel. From: http://bit.do/devday2014ssbipooh Telling Stories with Data

Microsoft Tools for Enterprise BI SQL Server Integration Services Analysis Services Multidimensional Tabular Reporting Services SharePoint PerformancePoint Services Telling Stories with Data

Distributed storage and processing Social Media Distributed storage and processing 10 | Telling Stories with Data Telling Stories with Data

Then it was not enough… Data changed Requirements changed More people wanted to join in the fun… …on their own terms Telling Stories with Data

A shift from Business Intelligence to Data Discovery Image source: http://bit.do/devday2014ssbicookie Telling Stories with Data

Self-Service – The way to go… Insights through Data Discovery Mashup data from various sources Within the organization (without/with less IT involvement) Outside the organization Share findings, tell stories and take action Telling Stories with Data

Tools for the Data Discovery Age… Power BI Excel + Add-ons Azure Machine Learning Azure SQL Warehouse HDInsight Telling Stories with Data

Demo Telling Stories with Data

Thank you! Telling Stories with Data

Each complete feedback form gives you a chance at the raffle We want your feedback Each complete feedback form gives you a chance at the raffle Telling Stories with Data

Microsoft Corporation Our Gold Sponsors Microsoft Sri Lanka Pluralsight Microsoft Corporation Brought to you by: Telling Stories with Data