1. Week 5 – Nov 4, 2015 Data Analysis

2. Class today Last week & Homework review SQL: Purpose and functions Data Cleaning Pivot Tables, Power Pivots and Power Query (again) exercise Lab on Power Pivots Homework

3. HOMEWORK & LAB REVIEW

4. SQL: PURPOSE AND FUNCTIONS

5. SQL Introduction SQL stands for: Structured Query Language It is the most commonly used relational database language today. SQL works with a variety of different programming languages & vendors such as Visual Basic, Tableau, Excel, etc.

6. SQL is used for: Data Manipulation – Query 1+ tables – Insert/Delete/Modify tuples in tables Data Definition – Create/Alter/Delete tables and their attributes (won’t do this in this class) Data Administration All are expressed with a SQL statement or command

7. Tables Explained The schema of a table is the table name and its attributes: Product(PName, Price, Category, Manfacturer) A key is an attribute whose values are unique; we underline a key Product(PName, Price, Category, Manfacturer) The DAMA Dictionary of Data Management

8. Tables Explained A tuple = a record (all attributes of atomic type) A table = a set of tuples (like a list, but unordered)

9. Database Set Up Instructions to load Northwind DB and AdventureWorks is available on canvas DEMO

10. Basic Query Structure

12. 1) SELECT SELECT 1

13. 2) FROM SELECT * FROM Person.Address * asks for every column in a table

14. 3) COUNT(*) SELECT COUNT(*) FROM Person.Address Returns the number of records in a table

15. 4) WHERE SELECT COUNT(*) FROM Person.Address WHERE City=‘Bothell’ WHERE: helps us filter our dataset

16. 3) COUNT 5) GROUP BY 11)DISTINCT SELECT COUNT(PostalCode), City FROM Person.Address GROUP BY City SELECT DISTINCT PostalCode FROM Person.Address

17. 5) GROUP BY6) SUM() We can define an aggregation in SQL as well SELECT Size, AVG(ListPrice) FROM Production.Product WHERE Size IS NOT NULL GROUP BY Size

18. 7) LIKE SELECT City, PostalCode FROM Person.Address WHERE City LIKE ‘U%’ GROUP BY City SELECT * FROM Person.Address WHERE City LIKE '[bsp]%'; %: A substitute for 0+ characters _ : A substitute for a single character [charlist], [!charlist]: sets/ ranges of characters to match, not match

19. 8) TOP SELECT TOP 10 * FROM Person.Person SELECT * FROM Person.Person LIMIT 10 SELECT * FROM Person.Person WHERE ROWNUM <= 10 Microsoft SQL Server MySQL Oracle

20. 9) ORDER BY SELECT TOP 10 * FROM Person.Person ORDER BY LastName (DESC) Microsoft SQL

21. SQL Joins http://blog.codinghorror.com/a- visual-explanation-of-sql-joins/ http://blog.codinghorror.com/a- visual-explanation-of-sql-joins/ Cool visual interpretation of Joins Tableau example of how joins can go wrong

23. JOIN ON SELECT * FROM Customers JOIN Orders ON Customers.CustomerID = Orders.CustomerID SELECT * FROM Customers c JOIN Orders o ON c.CustomerID = o.CustomerID

24. INNER, LEFT, SELECT * FROM Customers INNER JOIN Orders ON Customers.CustomerID = Orders.CustomerID SELECT * FROM Customers c LEFT JOIN Orders o ON c.CustomerID = o.CustomerID

25. OUTER JOINS

28. Microsoft Architecture for Self-service Business Intelligence

29. Microsoft Power BI for Office 365 Analyze Visualize Share Find Q&AQ&A Mobile Discover Search, access, and transform public and internal data sources with Power Query Share data views and workbooks refreshable from on-premises and cloud based data sources, with Power BI Sites Easy data modeling and lightning fast in-memory analytics with Power Pivot Bold new interactive data visualizations with Power View and Power Map Ask questions and get immediate answers with natural language query Mobile access through HTML5 and touch optimized apps for Windows 8, RT and iOS tablets Scalable | Manageable | Trusted Microsoft slide

30. EXCEL PIVOT TABLES

31. Learning objectives Understanding pivot tables Learn how to create pivot tables Use pivot tables to quickly answer questions about data

32. What is a Pivot Table A way to summarize, filter, and explore data interactively When we have lots of data, with lots of columns, there is often too much detail and we lose sight of the questions we want answered

33. Intro to PivotTables A pivot table is essentially a dynamic summary report Data can be in the worksheet or in an external file Allow you to reorganize and summarize data dynamically PivotTables unlock awesome interactivity with the data

34. How do we create a PivotTable Insert > Tables > Recommended PivotTables Insert > Tables > PivotTable – If you create a table for the range, then Excel will refresh the pivot table without the need to manually indicate the new data range

35. PIVOT TABLES Sort Count Total Change Aggregation Percent of Total Visualizations Filter Directly Query Databases, including OLAP

36. PIVOT TABLES PROs/CONS Pros: – Flexible – Works with any SQL report – Calculate large range of metrics very quickly and easily – Easily segmented – Summarizes a large amount of data quickly Cons: – Can be manually time consuming – Excel can take time to churn through really large data sets – Sometimes flying blind – Limited style options – Text not allowed in the data values area

37. POWER QUERY

38. Why Power Query? Discovery and connectivity to a wide range of data sources, including publicly available data of all sizes & shapes. Highly interactive and intuitive experience for rapidly and iteratively building queries over any data source, any size. Consistency of experience, and parity of query capabilities over all data sources. Joins across different data sources; ability to create custom views over data that can then be shared with others in your organization Microsoft slide

39. POWER PIVOT

43. POWER PIVOTS

44. POWER PIVOT ADVANTAGES

45. POWER PIVOT LAB

46. HOMEWORK

47. Homework Two parts practice: Part A: More SQL practice Part B: Excel PivotTable and Power Pivot practice Both due by Thurs, 11/12 at 5pm Required reading Reading: Keeping up with the Quants Ch. 5, Excel Ch. TBD

48. Next week SQL Quiz More Excel (VLOOKUPS) Data collection, cleaning and modeling Stats Review Introduction To Tableau Possibly a Guest Speaker