1. Service-Oriented Data Architecture (SODA) Jon Jahren Produktsjef, Microsoft Norge jonjah@microsoft.com

2. Agenda Natural forces in data Data in a services world Introducing SODA SQL 2005 SODA features Wrap up

3. Wave 1 Processor Code Data New Data “Data Processing”

4. Wave 2 Enough storage to keep some data “online” Processor Code Data “OLTP”

5. Wave 3 Enterprise Process and Data models led to “Enterprise Applications” Enterprise Applications linked via “Integration” (EAI) EAI ERPHR

6. The Data Monolith Previous 30 years trended towards data consolidation SOA, connected systems, common schema, pulling data apart Data Processing OLTP EA Internet B2B Services Devices SOA Data Convergence Data Distribution

7. Data in a SOA World Implications of SOA on data Breaking of data monoliths Data becomes more distributed Recognition of data classes Different treatment for different data SOA → Event Driven Data Systems Loose coupling Heterogeneous systems TimeSecurity

8. Consistency vs. Distance Transactions take DB’s from one consistent state to another Doing this across space and time is hard! Attempts 2-phase commit Optimistic CC Replication DataConsistency“Distance”SpaceTimeAuthorityConnectedness

9. Resilience and Correctness Challenge: System Correctness System Correctness Distance: Distance: Space Space Time Time Authority Authority Implementation Implementation Loose coupling Loose coupling Approach: Acknowledge Separation Acknowledge Separation Authority Authority Transaction Transaction Implementation Implementation Recognize data classes Recognize data classes Reference data Reference data Result: Service Oriented Database Architecture (SODA) Service Oriented Database Architecture (SODA) Native Web Services Native Web Services Service Broker Service Broker Service Logic Host (CLR) Service Logic Host (CLR) Query Notifications Query Notifications

10. Order Processing circa 1950 Payment CatalogUpdates OrderForm Catalog OrderAck Invoice OrderCopy PickList Inventory Inventory Ledger Reference Data Resource Data Activity Data Service Interaction Data

11. Reference Data Defined Used to create service requests Format interpretable by all parties Anyone can read the catalog Has a stable identifier March 2005 catalog Referenced by service request Item 23, Page 64, March 2005 catalog Can be infinitely replicated and cached Doesn’t need to be 100% consistent Might be able to order from March catalog on April 1 st. Catalog

12. Activity Data Defined Lifetime generally bounded by business activity (order, itinerary, etc.) May be retained for reporting, etc. Format is private to service Generally keyed to business activity Backorder for order #268 Can be partitioned Can change partitioning without moving data (Ephemeral – evens out over time) PickList

13. Resource Data Defined Very long lifetime SKU’s, Customers, Accounts… Format and update private to service Highly concurrent One item’s “stock on hand” modified by many orders Re-partitioning requires moving entries Inventory

14. Service Interaction Data Defined The only way services communicate Format interpreted by all parties Everyone can read and interpret the “order form” It may get lost Whether mail, e-mail, internet, etc. If it gets lost, you send another copy The copy better be an exact replica of the lost one OrderForm

15. Observations on Data Types Different Service Data Classes Have Different Characteristics No Read Only None Yes No: Has Open Schema No Read Only None Yes No: Has Open Schema Yes Very Low Object Persistance Optimistic MaybeYes May Be High COM+ Transactional Pessimistic (Locking) MaybeYes Service Interaction Reference Activity Resource Private to Service? XML? Con- currency Needs En- capsulation? Object Style Concurrency Approach Kind of Data

16. Deployment Topologies Single Tier & Client/Server All data updated directly Service interaction data implicit (Order Data comes from form) All data treated alike Catalog OrderCopy PickList Inventory Ledger

17. Deployment Topologies Multi-Tier Reference Data Cloned & Cached Activity Data “Tightly Coupled” to back end Application connected to app server in a “Session” Inventory Ledger CatalogCache CatalogMaster OrderCopy PickList Application Server Replication CatalogCache CatalogCache

18. Deployment Topologies Fully Distributed Connected Application InventoryService LedgerService CatalogMaint.Service OrderService KittingService CatalogUpdates Order OrderAck Invoice Order Payment Payment Reference Data Cached Remotely Activity Data Managed by services All communications via Service Interaction Data (Messages) Supports Heterogeneous Implementations

19. What Role Databases? Role of databases in SOA? Reference Data Cache Management Service Interaction (Message) Data Handling Activity Data Management Connected Sessions → Event Driven Need Service Oriented Database Architecture (SODA)

20. Scaling Techniques differ with data Reference data Shared Scalable Database Snapshot replication Query notifications Activity data Dynamic partitioning Data dependent routing Service interaction data Inherently asynchronous Resource data Static partitioning Peer-to-peer or bi-directional replication

21. Storage strategies Techniques differ with data Reference data Master copy in central database Replicas online Activity data Keep copies for activity tracking, trending, mining Service interaction data Keep EXACT copies of request and response For retry and recovery Sometimes a legal requirement Resource data Divide processes Updates can be asynch with other categories

22. Session Oriented Database Input & Events processed by application Passive Data Little binding between data and “logic” Database “Application” Database Session Database “Application”

23. Service Oriented Database Service Logic bound to: Services Query Notifications Events Service Logic Triggered: By data changes By message arrival By timer firing Data becomes “active” Events Service Logic Database “Application”

24. SODA Requirements Event Driven Communication endpoint for services Great host for service logic Decoupled from messaging (asynch) Support logic inside or outside DB process Needs to support reference and activity data management Support service centric programming model

25. What about App Servers? Some SODA requirements implemented in Application Servers… Message Brokering Complex Logic Hosting Web Service Hosting Web Service Comm. Endpoints Web Service Mechanisms Caching support Analogy between TP-Lite and traditional TP

26. TP-Heavy vs. TP-Lite TP-Lite Early to mid 1990’s DB Vendors add TP features to database Thread pooling/management Logic hosting (stored procedures) Communication Management Transaction coordination Became “Good Enough” for most needs Same thing will happen with SODA

27. SQL Server 2005 SODA features Query notifications For scaling out data & presentation caches Reference data scaling Native web services Service Endpoint Service broker Service centric architecture Reliable messaging with complete database integration Service logic host CLR Integration

28. Service Broker Query Notifications for Web Caching Query Processor Response Request Query/ Subscription Result Notification IIS / ASP.Net SQL Server 2005 Dispatcher

29. Native SOAP access Standards based access to SQL Server No client dependency Improved Interoperability New “ENDPOINT AS HTTP” object Configure connection info Configure authentication Expose Functions & SPs Expose TSQL Batches http://server1/aspnet/default.aspx http://server1/sql/pubs?wsdl KernelModeListener

30. Asynchronous, Distributed, Decoupled programming environment Fully integrated into the database engine Programming Model - DDL and DML Integrated Management and Deployment Dependent SQL Server 2005 features – Events, Query Notifications, SQLiMail Customer Usage, benefits: Enables new scenarios – async SPs, Scalable distributed multi-db applications

31. Transaction Transaction Stored proc runs on Different transaction Different transaction Different thread Different thread Different security Different security Different time Different time ClientApp Persisted Dialog LaunchedStoredProc

32. Services Live In The Databases Ongoing work in the database Each Service “instance” is kept in a database State is kept in the database Messages are stored in the database Routing to a database Incoming messages are kept in the database The message is matched to the state and the computation is performed Routing incoming web service messages means delivering to the correct database Transaction Service Shared Stuff Private State TransactionTransaction

33. SQLCLR Integration CLR Deeply Integrated Hosting layer provides coordination of: ThreadsMemory Garbage Collection DDL, Metadata, Security Allows System Extension Scalar Functions Table Value Functions ProceduresTriggers Data Access Windows SQL-OS Host Layer SQL CLR SQL Engine

34. Conclusion Must consider classes of data in SOA Databases will evolve From: Connected & session oriented To: Event driven & service oriented TP-Heavy → TP-Lite evolution will repeat itself from App Server → SODA SQL 2005 is a SODA platform