3. “The half-life of BI is typically shorter than the life of the project needed for its implementation.” --Industry whitepaper (see references)

4. “Predicting is hard… …especially about the future” --Yogi Berra

6. A recent Gartner Group Advanced Technology Research Note listed data mining at the top of the five key technology areas that "will clearly have a major impact across a wide range of industries within the next 3 to 5 years."

7. Data Mining finds patterns in data

8. – Using Machine Learning Algorithms Don’t worry: the hard yards are done – A lot at Microsoft Research

9. Data Mining finds patterns in data Uses these patterns to make predictions

10. SSAS ≠ Cube

11. Dimensional Modelling: Build a Cube  Learn MDX   Construct Analyses …of the PAST Data Mining: Build Structure Use Model Make Predictions …about the Future

12. Cubes summarize facts: – For Example: Sums of Sales in all regions for all months Aggregated by Gender and Age For each Product …

13. Cubes summarize facts: – For Example: Sums of Sales in all regions for all months Aggregated by Gender and Age For each Product … – Data mining find patterns in data

14. Cubes summarize facts: – For Example: Sums of Sales in all regions for all months Aggregated by Gender and Age For each Product … – Data mining find patterns in data – Cubes abstract much of the interesting information

15. Cubes summarize facts: – For Example: Sums of Sales in all regions for all months Aggregated by Gender and Age For each Product … – Data mining find patterns in data – Cubes abstract much of the interesting information Facts that form the patterns are lost in the Cube’s summations

16. Connect to Data Source Highlight Exceptions Forecasting Key Influencers

17. Is it all just smoke and mirrors???

18. “Excel data mining add-in was invented to make astrology look respectable!” – Donald Data, industry pundit

22. Jargon: ADO = ActiveX Data Objects ADO MD = ADO Multidimensional AMO = Analysis Management Objects DSO = Decision Support Objects XMLA = XML for Analytics

23. Books Online  Contents or… Search For Data Mining Tutorials

25. Business Intelligence Development Studio Demo: Key Influencers – Models and Model Viewers Decision Tree Cluster Naïve Bayes Neural Network

26. Correlation  Tree Node

28. Hybrid Linear regression & association & classification

29. Hybrid Linear regression & association & classification Algorithm highlights Remove rare attributes (“Feature Selection”)

30. Hybrid Linear regression & association & classification Algorithm highlights Remove rare attributes (“Feature Selection”) Group values into bins for performance

31. Hybrid Linear regression & association & classification Algorithm highlights Remove rare attributes (“Feature Selection”) Group values into bins for performance Correlate input attributes with outcomes

32. Hybrid Linear regression & association & classification Algorithm highlights Remove rare attributes (“Feature Selection”) Group values into bins for performance Correlate input attributes with outcomes Find attribute separating outcomes with maximum information gain

33. Hybrid Linear regression & association & classification Algorithm highlights Remove rare attributes (“Feature Selection”) Group values into bins for performance Correlate input attributes with outcomes Find attribute separating outcomes with maximum information gain Split tree and re-apply

35. Algorithm options: Non-scalable (all records)

36. Algorithm options: Non-scalable (all records) Scalable (50,000 records + 50,000 more if needed) – 3 x faster than non-scalable

37. Algorithm options: Non-scalable (all records) Scalable (50,000 records + 50,000 more if needed) – 3 x faster than non-scalable K – means (hard)

38. Algorithm options: Non-scalable (all records) Scalable (50,000 records + 50,000 more if needed) – 3 x faster than non-scalable K – means (hard) Expectation Maximization (soft) (default)

39. Algorithm options: Non-scalable (all records) Scalable (50,000 records + 50,000 more if needed) – 3 x faster than non-scalable K – means (hard) Expectation Maximization (soft) (default) – Form initial cluster

40. Algorithm options: Non-scalable (all records) Scalable (50,000 records + 50,000 more if needed) – 3 x faster than non-scalable K – means (hard) Expectation Maximization (soft) (default) – Form initial cluster – Assign probability each attribute-value in each cluster

41. Algorithm options: Non-scalable (all records) Scalable (50,000 records + 50,000 more if needed) – 3 x faster than non-scalable K – means (hard) Expectation Maximization (soft) (default) – Form initial cluster – Assign probability each attribute-value in each cluster – Iterate until model = likelihood of data

42. Simple, fast, surprisingly accurate

43. “Naïve”: attributes assumed to be independent of each other

44. Simple, fast, surprisingly accurate “Naïve”: attributes assumed to be independent of each other Pervasive use throughout Data Mining

45. Simple, fast, surprisingly accurate “Naïve”: attributes assumed to be independent of each other Pervasive use throughout Data Mining P(Result | Data) = P(Data | Result) * P(Result) / P(Data)

46. P(Girl | Trousers) = ? P(Trousers | Girl) = 20/40 P(Girl) = 40/100 P(Trousers) = 80/100

47. P(Girl | Trousers) = ? P(Trousers | Girl) = 20/40 P(Girl) = 40/100 P(Trousers) = 80/100 P(Girl | Trousers) = P(Trousers | Girl) P(Girl) / P(Trousers)

48. P(Girl | Trousers) = ? P(Trousers | Girl) = 20/40 P(Girl) = 40/100 P(Trousers) = 80/100 P(Girl | Trousers) = P(Trousers | Girl) P(Girl) / P(Trousers) = (20/40)(40/100)/(80/100) = 20/80 = 0.25

49. Cars Weight 2 2 Cars Weight 3 3 Age Weight Buy No Input NeuronsHidden Neurons Output Neurons W W W W W W W W W W W W W W W W W W W W W W W W W W W W W W

50. Multilayer Perceptron Network =

51. Back-Propagated Delta Rule Network

52. Multilayer Perceptron Network = Back-Propagated Delta Rule Network Assign weights: assess importance of input on output using training dataset

53. Multilayer Perceptron Network = Back-Propagated Delta Rule Network Assign weights: assess importance of input on output using training dataset Batch Learning – Start at outputs and propagate back through the network:

54. Multilayer Perceptron Network = Back-Propagated Delta Rule Network Assign weights: assess importance of input on output using training dataset Batch Learning – Start at outputs and propagate back through the network: – Evaluate weight accuracy: predicted value vs. holdout value

55. Multilayer Perceptron Network = Back-Propagated Delta Rule Network Assign weights: assess importance of input on output using training dataset Batch Learning – Start at outputs and propagate back through the network: – Evaluate weight accuracy: predicted value vs. holdout value – Adjust weights to improve prediction

56. Multilayer Perceptron Network = Back-Propagated Delta Rule Network Assign weights: assess importance of input on output using training dataset Batch Learning – Start at outputs and propagate back through the network: – Evaluate weight accuracy: predicted value vs. holdout value – Adjust weights to improve prediction » Weight can be negative to show inhibiting influence

57. Multilayer Perceptron Network = Back-Propagated Delta Rule Network Assign weights: assess importance of input on output using training dataset Batch Learning – Start at outputs and propagate back through the network: – Evaluate weight accuracy: predicted value vs. holdout value – Adjust weights to improve prediction » Weight can be negative to show inhibiting influence Iterate using conjugate gradient algorithm to converge

59. SSMS (aka SQL Mangler) – Analysis Services Database Data Mining

60. Business Intelligence Development Studio Lift Chart: Key Influencers – Decision Tree – Cluster – Naïve Bayes – Neural Network

61. – Lift Chart (Profit Chart) Random Guess line Perfect Prediction line (“ideal”) Model prediction line (“lift” over random) – Example: Population 10,000 with 10% response rate Mail to 5,000, targeted with mining model Random response: 500 Ideal response: 1,000

62. Population Random: 50% Targeted Data Mining: 85% Ideal: 100% Lift Chart Operation Bike Buyers

64. Demo: Targeted Mailing – Find prospective customers – Save results to database – Import in a new Data Source View – Process again with Data Mining!

65. Fill By Example Goal Seek What If Highlight Exceptions Data Mining Tab: – Explore Data – Clean Data, etc….

66. Off-the-shelf toolkit No Cube required No code required Good default parameters Easily explored models – Change parameters, filter input, compare lift Excel Add-In

68. Data Mining Add-ins http://office.microsoft.com/en-us/excel-help/data-mining-add-ins- HA010342915.aspx#_Toc257717762 http://office.microsoft.com/en-us/excel-help/data-mining-add-ins- HA010342915.aspx#_Toc257717762 Analysis Services - Data Mining Videos http://msdn.microsoft.com/en-us/library/dd776389(v=SQL.100).aspx SQL Server Data Mining Home http://www.sqlserverdatamining.com/ssdm/ Microsoft Contoso BI Demo Dataset for Retail Industry http://www.microsoft.com/downloads/en/details.aspx?displaylang=en&FamilyID=868662dc -187a-4a85-b611-b7df7dc909fc http://www.microsoft.com/downloads/en/details.aspx?displaylang=en&FamilyID=868662dc -187a-4a85-b611-b7df7dc909fc What Every IT Manager Should Know About Business Users’ Real Needs for BI http://docs.media.bitpipe.com/io_25x/io_25515/item_392177/Tableau_S_MktgLtr_BI_IT.pdf An Introduction to Data Mining : Discovering hidden value in your data warehouse http://www.thearling.com/text/dmwhite/dmwhite.htm

69. Problems: – Data to old to be useful – Need for instantaneous feedback Solution: – StreamInsight Complex Event Processing

70. Processing and querying of event data streams Data queried while “in flight” May involve multiple concurrent event sources Works with high data rates Aims for near-zero latency

71. Relational Database Applications Financial Trading Applications Aggregate Data Rate (Events/sec) Operational Analytics Applications (e.g., Logistics) Manufacturing Applications Monitoring Applications CEP Target Scenarios Data Warehousing Applications Web Analytics Applications

72. Data Sources, Operations, Assets, Feeds, Sensors, Devices Monitor & Record Monitor & Record Operational Data Store & Archive CEP Engine f(x) g(y) CEP Engine f(x) f'(x) g(y) h(x,y) History Deploy Results f'(x) h(x,y) Manage & Benefit Manage & Benefit Mine & Design Mine & Design Input Data Streams Output Data Streams

74. Algorithmic trading Smart order routing Real-time profit and loss Rapid analysis of transactional cost Fraud detection Risk management

75. Often 100,000 events per second Automate – Page layout – Navigation – Presentation – Targeted advertising

76. Real-time network monitoring Quality of service monitoring Location-based services Fraud detection Intrusion detection

77. Battlefield control Monitoring of resource locations Intrusion detection Network traffic analysis – Emails – Network traffic – Watch lists – Financial movements

78. Asset monitoring Aggregation of machine-based sensor data Generation of alerts in error conditions Identifying the “golden batch”

79. Real-time monitoring Managing player interest Website traffic analysis Detecting and eliminating undesired behaviors Understanding behavioral patterns

80. Patient management Outbreak management Trend detection Insurance risk analysis

81. Vehicle management Supply chain forecasting and tracking Maritime logistics GPS tracking

82. Monitoring – Consumption – Variations Detecting outages Smart grid management Aggregating data across the grid

83. Gaming machine event analysis Card table analysis – Fraud detection – Profit and loss in real-time Targeted advertising – Player behavior – Loyalty system implementation