Challenges in Large Enterprise Data Management James Hamilton Microsoft SQL Server 2002.08.20.

Slides:



Advertisements
Similar presentations
Fabián E. Bustamante, Winter 2006 Recovery Oriented Computing Embracing Failure A. B. Brown and D. A. Patterson, Embracing failure: a case for recovery-
Advertisements

Key Challenges in Information Processing James Hamilton Microsoft SQL Server
MICROSOFT PLATFORM  Microsoft is a platform company is committed to providing a rich ecosystem for building and managing connected systems.  Microsoft.
DEV392: Extending SharePoint Products And Technologies Through Web Parts And ASP.NET Clint Covington, Program Manager Data And Developer Services - Office.
Active Server Availability Feedback James Hamilton Microsoft SQL Server
Unified Logs and Reporting for Hybrid Centralized Management
MyCloudIT Removes the Complexity of Moving Cloud Customers’ Entire IT Infrastructures to Microsoft Azure – Including the Desktop MICROSOFT AZURE ISV: MYCLOUDIT.
SQL Server 2005 Database Engine Sommarkollo Microsoft.
With the Help of the Microsoft Azure Platform, Devbridge Group Provides Powerful, Flexible, and Scalable Responsive Web Solutions MICROSOFT AZURE ISV PROFILE:
Design & Development Tools: Visual Studio 2005 SQL Server 2005 Biztalk Server 2006 David Gristwood, Mike Taulty Developer & Platform Group Microsoft Ltd.
Barracuda Networks Confidential1 Barracuda Backup Service Integrated Local & Offsite Data Backup.
Managing and Monitoring SQL Server 2005 Shankar Pal Program Manager SQL Server, Redmond.
Passage Three Introduction to Microsoft SQL Server 2000.
SQL Server 2008 for Hosting Key Questions to Address How can SQL Server save your costs? How can SQL Server help you increase customer base? How can.
Module 16: Software Maintenance Using Windows Server Update Services.
Copyright © 2007 Quest Software The Changing Role of SQL Server DBA’s Bryan Oliver SQL Server Domain Expert Quest Software.
Cross Platform Mobile Backend with Mobile Services James
BMC Software confidential. BMC Performance Manager Will Brown.
STEALTH Content Store for SharePoint using Caringo CAStor  Boosting your SharePoint to the MAX! "Optimizing your Business behind the scenes"
Microsoft TechForge 2009 SQL Server 2008 Unplugged Microsoft’s Data Platform Vinod Kumar Technology Evangelist – DB and BI
Website s Azure Websites is an enterprise class cloud solution for developing, testing and running web apps. Azure Websites allows you to focus on what.
James Akrigg Microsoft Ltd Integrating InfoPath Forms Into Workflow Solutions And Business Processes.
Techcello Provides SaaS Lifecycle Management Solution to “SaaS-ify” Your Application Efficiently on the Powerful Microsoft Azure Cloud Platform MICROSOFT.
Alert Logic Provides a Fully Managed Security and Compliance Solution Based in the Cloud, Powered by the Robust Microsoft Azure Platform MICROSOFT AZURE.
Securely Synchronize and Share Enterprise Files across Desktops, Web, and Mobile with EasiShare on the Powerful Microsoft Azure Cloud Platform MICROSOFT.
Accumulus Delivers Enterprise Class Subscription Billing and Automation Solutions for Gaming, Retail, and More on the Scalable Microsoft Azure Platform.
Microsoft Management Seminar Series SMS 2003 Change Management.
Corent’s SurPaaS Transforms Your Software into Scalable SaaS on Windows Azure – in Days! COMPANY PROFILE: CORENT TECHNOLOGY INC. Corent’s SurPaaS is a.
Datalayer Notebook Allows Data Scientists to Play with Big Data, Build Innovative Models, and Share Results Easily on Microsoft Azure MICROSOFT AZURE ISV.
An Overview of Microsoft.NET Todd M. Gagorik Technical Architect Microsoft Corporation.
SQL Server 2008 R2 Manageability. Challenges facing database administrators today: Scaling management to multiple data centers Proactively monitoring.
Built on the Powerful Microsoft Azure Platform, Mproof’s Clientele ITSM Provides Companies with a Complete Software Suite to Manage Services MICROSOFT.
Powered by Microsoft Azure, PointMatter Is a Flexible Solution to Move and Share Data between Business Groups and IT MICROSOFT AZURE ISV PROFILE: LOGICMATTER.
Managing and Monitoring the Microsoft Application Platform Damir Bersinic Ruth Morton IT Pro Advisor Microsoft Canada
Hosting Websites and Web Applications with Microsoft ® SQL Server ® 2008.
Multi-Tier Apps with Admin Access, RDP, Custom Installs Modern Scalable Web Sites Full Windows Server/Linux VMs Web Sites Virtual Machines Cloud Services.
Boost Developer Productivity with a 360- Degree View of Every Software Change by Using FinditEZ, Certified Microsoft Platform Ready for SQL Azure MICROSOFT.
Features Of SQL Server 2000: 1. Internet Integration: SQL Server 2000 works with other products to form a stable and secure data store for internet and.
+ Logentries Is a Real-Time Log Analytics Service for Aggregating, Analyzing, and Alerting on Log Data from Microsoft Azure Apps and Systems MICROSOFT.
Bring Your Own Security (BYOS™): Deploy Applications in a Manageable Java Container with Waratek Locker on Microsoft Azure MICROSOFT AZURE ISV PROFILE:
Microsoft Azure and DataStax: Start Anywhere and Scale to Any Size in the Cloud, On- Premises, or Both with a Leading Distributed Database MICROSOFT AZURE.
Saasabi’s Analytical Processing Engine in the Cloud Makes Business Intelligence Affordable for Everyone COMPANY PROFILE: Saasabi Saasabi is a BizSpark.
Norman SecureBackup Flexible backup for small and medium businesses.
Built on the Powerful Microsoft Azure Platform, HarmonyPSA Is a Cloud-Based Customer Service and Billing System for IT Solution Providers MICROSOFT AZURE.
DreamFactory for Microsoft Azure Is an Open Source REST API Platform That Enables Mobilization of Data in Minutes across Frameworks and Storage Methods.
Snip2Code: Search, Share and Collect Code Snippets Faster, Easier, Efficiently with Power of Microsoft Azure Platform MICROSOFT AZURE ISV PROFILE: SNIP2CODE.
SQL Database Management
Connected Infrastructure
Univa Grid Engine Makes Work Management Automatic and Efficient, Accelerates Deployment of Cloud Services with Power of Microsoft Azure MICROSOFT AZURE.
COMPANY PROFILE: CORENT TECHNOLOGY INC.
Lead SQL BankofAmerica Blog: SQLHarry.com
NeoFirma Taps into the Microsoft Azure Cloud Platform to Deliver Digital Oilfield SaaS to North American Independent Oil and Gas Producers MICROSOFT AZURE.
Connected Infrastructure
Built on the Powerful Microsoft Azure Platform, Lievestro Delivers Care Information, Capacity Management Solutions to Hospitals, Medical Field MICROSOFT.
Designed for Big Data Visual Analytics, Zoomdata Allows Business Users to Quickly Connect, Stream, and Visualize Data in the Microsoft Azure Platform MICROSOFT.
Scalable SoftNAS Cloud Protects Customers’ Mission-Critical Data in the Cloud with a Highly Available, Flexible Solution for Microsoft Azure MICROSOFT.
On-Premises, or Deployed in a Hybrid Environment
Auth0 Is Identity Made Simple for Developers, Built by Developers and Supported by the High Availability and Performance of Microsoft Azure MICROSOFT AZURE.
DeFacto Planning on the Powerful Microsoft Azure Platform Puts the Power of Intelligent and Timely Planning at Any Business Manager’s Fingertips Partner.
Accelerate Your Self-Service Data Analytics
MyCloudIT Enables Partners to Drive Their Cloud Profitability Using CSP-Enabled Desktop Hosting Automation with Microsoft Azure and Office 365 MICROSOFT.
Crypteron is a Developer-Friendly Data Breach Solution that Allows Organizations to Secure Applications on Microsoft Azure in Just Minutes MICROSOFT AZURE.
Appcelerator Arrow: Build APIs in Minutes. Connect to Any Data Source
XtremeData on the Microsoft Azure Cloud Platform:
Get your ETL flow under statistical process control
Last.Backend is a Continuous Delivery Platform for Developers and Dev Teams, Allowing Them to Manage and Deploy Applications Easier and Faster MICROSOFT.
Andy Puckett – Sales Engineer
Mark Quirk Head of Technology Developer & Platform Group
Microsoft Azure Services Platform
Designing Database Solutions for SQL Server
Presentation transcript:

Challenges in Large Enterprise Data Management James Hamilton Microsoft SQL Server

2 Enterprise Data Management Issues  Three leading enterprise data management challenges: 1. Availability/Cost of administration 2. Legacy integration & multi-tier applications 3. End-to-end system security  Industry making progress but the job isn’t yet done  Feature race partly diverts industry attention

3 1. Availability/Cost of Admin  1985 Tandem study (Gray):  Administration: 42%, Software: 25%, Hardware 18% downtime  1990 Tandem Study (Gray):  Software 62%, Administration: 15%,  Even server-side S/W is big:  Network Attached Storage: 1 mloc  Windows2000: Over 50 mloc  Database: 3 to 5 mloc  SAP: 37 mloc (4,200 engineers)  Example single outage costs ( Patterson HPTS02 & InternetWeek 4/3/00 ):  Brokerage: $6,500k  Credit Card Auth: $2,600k  Ebay: $225k  Amazon.com: $180k  Observations:  H/W downtime contribution trending to zero  Software & admin costs dominate & growing  Expensive: admin >5x S/W & H/W cost  Administration error prone: #1 or #2 cause of downtime

4 1. Availability/Cost of Admin  Solution summary:  Understand downtime/measure improvements:  Event Log Analyzer  Watson  Data Collection Agent  Ease of admin: “No knobs” design  Auto-administration: Index tuning wizard  Online utilities  No-reboot diagnostics  Uncertain admins just restart  Admin thread  Allow recovery without restart

5 2. Legacy Integration & Multi-tier Apps  Applications multi-tiered for many reasons:  Wrap legacy systems as web services  Improved availability through redundant mid-tier servers  Application scaling & DB offload through caching  Multi-tiered applications typically still hand crafted  Needed: object access layer, data cache, async queuing, data directed routing, mid-tier security support & integration,...  Solution Summary:  Visual Studio: App, mid-tier, & DB dev, deployment, & debug  Web Services: ASP.Net,.Net Framework, & Web Matrix  Distributed heterogeneous query support  Direct HTTP access into database  Multi-tier cache integration with notifications  Semi-structured & unstructured search:  XML <> Relational Mapping  Native XML datatype support  XQuery & XPath support  High-scale full text indexing and unstructured search  Asynchronous queuing in database  Security integration without fully provisioning back-end DB

6 3. System Security  DB Security is as important as data integrity  Protection against data damage, loss, manipulation, & unintentional disclosure  Rapidly changing threat environment:  DBs on public net rather than behind mediated access (CS rep)  Customers & partners integrating & directly sharing data  Security “experts” built publishing potential vulnerabilities  Security issues are under reported  Also under invested:  “Less than % of corp revenue invested in security” – Richard Clarke, Special security advisor to president  Solution summary:  Full review of server software stack (~$100m investment)  Full SQL organization for nearly 3 months  Automation: Code analysis tools targeting security vulnerabilities  Address Admin error: Easy to understand, policy driven  Fix delivery: Installer & auto-update (300m downloads/month)  Research on active security:  Detecting new threats & miss-configurations

Microsoft