Chap 1. What is it all About ?? Transactional Information Systems. Chap 1. What is it all About ?? 2007. 03. 27 (Thu) Database Lab. Kim Jinyoung

Contents. Goal and Overview. Application Examples. System Paradigms. 1 Application Examples. 2 System Paradigms. 3 Virtues of the Transaction Concept. 4 Concepts and Architecture of Database Servers. 5 Lessons Learned. 6 References. 7

Goal and Overview. Why transactions are a good idea? Why transactions form a reasonable abstraction concept for certain classes of real life data management and related problems. What can and what cannot be done with the transaction concept.

Goal and Overview. Transaction concept : A paradigm for dealing with concurrent accesses to a shared database and for handling failures. Application area ex) fund transfer in banking, travel industry with flight, hotel bookings. Etc. Key problems : - To cope with the subtle and often difficult issues of keeping data consistent even in the presence of highly concurrent data accesses and despite all sorts of failures. - Completely freed from the burden of dealing with such system issues. (abstraction concept)

OLTP : Online Transaction Processing Application Examples. /* debit / credit program */ Void main() { EXEC SQL BEGIN DECLARE SECTION; int accountid, amount; /* input variables */ int balance; /* intermediate variable */ EXEC SQL END DECLARE SECTION; /* read user input */ printf(“Enter Account ID, Amount for deposit (positive) or withdrawal (negative):”); scanf(“%d%d”, &accountid, &amount); /* determine current balance of the account, reading it into a local, intermediate, variable of the program */ EXEC SQL Select Account_Balance Into : balance From Account Where Account_Id = : accountid; /* add amount (negative for withdrawal) */ balance = balance + amount; /* update account balance in the database */ EXEC SQL Update Account Set Account_Balance = balance Where Account_Id = : accountid; EXEC SQL Commit work; } OLTP : Online Transaction Processing Online Transaction Processing : Debit / Credit Example funds transfer in a banking environment, a classical OLTP application. Account table : describes bank accounts. {account ID, customer name, branch name, balance} The typical structure of a debit / credit program. (p. 5)  embedding these commands in a C program. Consistancy. Transactional server provides recovery techniques to cope with failures. In addition to ensuring transaction atomicity, these techniques serve to ensure the durability of a transaction’s effects once the transaction is completed. atomicity, durability, and isolation.

Application Examples. EXAMPLE 1.1 Need to be addressed as well P1, P2 : debit/credit transactions - concurrently executed by processes - both operating on the same account x Account_Balance : Local program variable balance1, balance2 : Temporal variable of transaction P1, P2 P1 : withdraw $30. P2 : deposit $20. Initial account balance : $100 Upon completion of the execution, the balance of account x will be $120, although it should be $90 after execution of two transactions. Need to be addressed as well

Application Examples. EXAMPLE 1.2 /* funds transfer program */ Void main() { EXEC SQL BEGIN DECLARE SECTION; int sourceid, targetid, amount; /* input variables */ EXEC SQL END DECLARE SECTION; /* read user input */ printf(“Enter Source ID, Target ID, Amount to be transferred:”); scanf(“%d %d %d”, &sourceid, &targetid, &amount); /* subtract desired amount from source */ EXEC SQL Update Account Set Account_Balance = Account_Balance - :amount Where Account_Id = :sourceid; /* add desired amount to target */ EXEC SQL Update Account Set Account_balance = Account_Balance + :amount Where Account_Id = :targetid; EXEC SQL Commit Work; } This example illustrates that atomicity is a crucial requirement for being able to cope with failures. Now assume that the above funds transfer program has started executing and has already performed its first update statement, withdrawing the specified amount of money from the source. If there is a computer hardware of software failure, the remaining second part will not be performed anymore. Thus, the target account will not receive the money, so that money is effectively lost in transit. a recovery procedure, to be invoked after the system is restarted, could try to find out which updates were already made by ongoing transaction program executions and which ones were not yet done, and could try to fix the situation in some way. Intrrupts Recovery

Application Examples. Electronic Commerce Example consider what happens when a client intends to purchase something from an Internet-based bookstore. client requests my span multiple databases. The purchasing activity steps. - The client connects to the bookstore’s server, and starts browsing and querying the store’s catalog. - The client fills an electronic shopping cart. - The client make a final decision. - The client provides all the necessary information for placing a definitive order. This information may be encrypted, merchant can only verify its authenticity. - The merchant’s server forwards the payment information to the customer’s bank, etc. When the payment is accepted, the shipping is initiated by the merchant’s server, and the client is notified on the successful completion of the activity.

Application Examples. Electronic Commerce Example It is obviously important to keep certain data consistent, and this data is even distributed across different computers. Three parties agree on the data that tracks the overall outcome: - The merchant’s server must have records on both the order and the successfully certified payment. - At the same time, the clearinghouse must have a record on the payment. - Finally, the client must have received the notification that the ordered items are being shipped. It is obviously important for the entire e-Commerce industry on atomicity guarantees.

Application Examples. Electronic Commerce Example Nicely highlight the potential generalization of the transaction concept. - The entire application is distributed across multiple computers, and the software may be heterogeneous in that different database systems are used at the various servers. - The servers are not necessarily based on database systems. - The effects of a transaction may even include messages between computers.

Application Examples. Workflow Management: Travel Planning Example. Workflow = business process : A workflow is a set of activities that belong together in order to achieve a certain business goal. Workflow Management: Travel Planning Example. characteristic of workflows : - allows a company or other institution to largely automate the repetitive, stereotypic parts of its processes while retaining flexibility, and to quickly adjust these processes to changing business needs. - the activities are distributed across different responsible persons and different, independent information systems, possibly across different enterprises. Workflow management systems with such capabilities are commercially available and are gaining significant industrial relevance.

Specification of the travel planning workflow. following activities : - Select a conference, based of its subject, technical program, time, and place. If no suitable conference is found, the process is terminated. - Check out the cost of the trip to this conference, typically by delegation to a travel agency. - Check out the registration fee for the conference, which often depends on your memberships, tutorials that you may wish to attend, and so on. - Compare the total cost of attending the selected conference to the allowed budget, and decide to attend the conference only if the cost is within the budget. Application Examples. Workflow Management: Travel Planning Example. Example : consider the activities planning of a business trip. - Suppose your manager allows you to choose one scientific or developer’s conference that you can attend as part of a continuing- education program. following activities : - Select a conference, based of its subject, technical program, time, and place. If no suitable conference is found, the process is terminated. - Check out the cost of the trip to this conference, typically by delegation to a travel agency. - Check out the registration fee for the conference, which often depends on your memberships, tutorials that you may wish to attend, and so on. - Compare the total cost of attending the selected conference to the allowed budget, and decide to attend the conference only if the cost is within the budget. Specification of the travel planning workflow. /Budget := 1000; Trials := 1; CheckConfFee Final state Go Select tutorials Compute fee Initial state Select- Conference /Cost := [Cost ≤ Budget] ConfFee + TravelCost Check- Cost [ConfFound] /Cost := 0 Check airfare hotel [Cost > Budget and Trials ≥ 3] Final state No CheckTravelCost [!ConfFound] [Cost > Budget and Trials < 3] /Trials ++

Application Examples. Workflow Management: Travel Planning Example. The transitions between states are governed by event-condition-action rules that are attached to the transition arcs as labels. E [ C ] / A A B Event occurred → Condition satisfied → Action execution. In the example, the two states CheckConfFee and CheckTravelCost are executed in parallel. CheckConfFee has several steps, CheckTravelcost again leads to two parallel substates.

Application Examples. Workflow Management: Travel Planning Example. Check airfare hotel how a workflow application could possibly benefit from transaction-supporting services. - Related two or more steps need to be tied together in a single transaction. This transaction is a distributed one that involved two autonomous information systems. - The outcome of above reservations affects the further processing of the W.F. The state of the W.F. application should be under transactional control as well. - After all, the entire workflow should have an all-or-nothing, atomic effect. · Atomicity possible. But isolation property is impossible. (performance problem)  straightforward approach of turning an entire workflow into a single transaction absolutely infeasible !! But the one-transaction-per-activity approach is the only kind of transactional support for workflows. use the compensation activities. CheckTravelCost

Reference architecture System Paradigms. Three-Tier and Two-tier Architectures Application Program 1 Program 2 Request Reply Encapsulated data Objects Exposed data Users Clients server Data server Stored data (pages) Reference architecture

System Paradigms. Three-Tier and Two-tier Architectures two major options for such simpler architectures : - Client + Application server : client-server system with fat clients. (the application programs reside on the clients)  Problems : many data server have lacked the capabilities for maintaining a very large number of sessions and execution threads.  But, not every application needs such high degrees of concurrency, and many servers have made tremendous progress in this direction anyway. - Application server + Data server : client-server system with thin clients.  Problems : the data server may become less scalable and more susceptible to being overloaded.  But, extremely powerful hardware, we can use this Architecture. Our approach in this book will be based on computational models. - The two-tier architectures are easier to match the abstract computational models.  the two-tier cases over the more general tree-tier case

Federated system architecture System Paradigms. Federations of Servers Federated system architecture Users Clients Application server Data server Stored data (pages)

Virtues of the Transaction Concept. Transaction Properties and the Transaction Programming Interface application programs themselves can safely ignore a good portion of the complexity of the overall system. application programs are completely freed up from taking care of issues of : - concurrency : all effects that may result from concurrent or even parallel program executions and especially data accesses. - failures : all effects that would result from program executions being interrupted because of process or computer failures. Transaction form the “interface contract” in the following sense : - the boundaries of a transaction : Begin and Commit transaction call. - the server automatically considers all requests.

Virtues of the Transaction Concept. Transaction Properties and the Transaction Programming Interface ACID properties : a server guarantees for a transaction. - Atomicity : a transaction is executed completely or not at all. - Consistency preservation : a transaction leads from one consistent state to another. - Isolation : a transaction is isolated from other transactions. - Durability : when the application program is notified that a transaction has been successfully completed all updates that the transaction has made in the underlying data servers are guaranteed to survive subsequent software or hard failures.

Virtues of the Transaction Concept. Transaction Properties and the Transaction Programming Interface transaction : a set of operations issued by an application program and executed on one or more data servers, with the ACID properties guaranteed by the run-time system of the involved servers. the programming interface of a transactional information system conceptually needs to offer three calls : - Begin transaction : beginning of a transaction. - Commit transaction : successful end of a transaction. - Rollback transaction : unsuccessful end of a transaction with the request to abort the transaction.

Virtues of the Transaction Concept. Requirements on Transactional Servers. Core requirement : provide the ACID guarantees for set of operations. Two components : - concurrency control component : isolation. - recovery component : atomicity and durability. a transactional data server must provide good performance. - two metrics : · high throughput · short response times If performance were not an issue  much simpler to provide the ACID. The server must be reliable  virtually never loses data. Applications require high availability  short recovery time, small failures  Do not become a bottleneck.

Concepts and Architecture of Database Servers. Architectural Layers of Database Systems Layered architecture of a database system Clients Requests Language & Interface Layer Query Decomposition & Optimization Layer Request Execution Threads Query Execution Layer Access Layer Database Server Storage Layer Data accesses Each layer has an interface. All layers are within the same server. not visible to application programs. Database

Concepts and Architecture of Database Servers. Architectural Layers of Database Systems Language & Interface Layer : - client-server communication, Initial processing steps for request, checking authentication and authorization. Query Decomposition & Optimization Layer : - the request into smaller units that can be directly executed. (represented as a tree of operators) Query Execution Layer : - provides codes for operators and control, data flow among operators. Access Layer : - Both data records and index structures are ultimately mapped onto pages. Storage Layer : - responsible for managing the pages of a database.

Concepts and Architecture of Database Servers. All data objects are kept internally in the form of data records. Pages are the minimum unit of data transfer between a disk and main memory. Page Header has some purpose of pages, etc. Slot array : keep pointers to records in other places. Resides at end of the page. RID (record ID) : A pointer to a record. Do not span multiple pages. Extent and extent table. , free space management Concepts and Architecture of Database Servers. How data is Stored ? Storage layout of database pages. Database Page Page Header Ben 55 Las Vegas ... Sue 23 Seattle Joe 29 San Antonio Free space --- Extent Table Extents - Forwarding RID Slot array

Concepts and Architecture of Database Servers. How Data is Accessed ? Table scan vs. Index lookup. - the records of the given search key is through a sequential table scan. All records of the corresponding table are accessed and inspected. - Index lookup : Using an index structure for resolving a specified search condition. Index structure : Search trees, Hash table  Index scans - tree-based implementation lookup <index> where <indexed field> = <search key> - hash-based implementation lookup <index> where <indexed field> between <lower bound> and <upper bound>

Concepts and Architecture of Database Servers. How Data is Accessed ? Use b+ tree. ex) Search : Dick Examples of a b+ tree

Concepts and Architecture of Database Servers. How Queries and Updates Are Executed Example. Select Name, City, Zipcode, Street From Person Where Age < 30 And City = “Austin” Age Index does not exists 6 5 Projection Projection 5 4 RID access Filtering Range lookup Operator tree proceeds direction 3 3 RID list intersection RID access 4 1 2 1 2 Index scan on Age index Index scan on City index Fetch Person record Index scan on City index Fetch Person record Example of a query execution plan Alternative query execution plan for the query

Lessons Learned. Transaction. ACID properties. Users can rely on the consistency of the data that they see and maintain as part of their day-to-day business. Application developers greatly benefit from the transaction concept. in that the application programming is fundamentally simplified, thus boosting programming productivity and lowering the development and maintenance costs. A Concurrency control component that guarantees the isolation of transaction. A Recovery component that guarantees the atomicity and durability of T. Improve performance of the concurrency control and recovery components.

References. G.Weikum and G.Vossen, “Transactional Information Systems: Theory, Algorithms, and the Practice of Concurrency Control and Recovery,” Morgan Kaufmann, 2002

Thank You !

Application Examples. P1 Time P2 /* balance1 = 0, x.Account_Balance = 100, balance2 = 0 */ Select Account_Balance Into :balance1 From Account 1 Where Account_Id = x /* balance1 = 100, x.Account_Balance = 100, balance2 = 0 */ Select Account_Balance 2 Into :balance2 From Account Where Accont_Id = x /* balance1 = 100, x.Account_Balance = 100, balance2 = 100 */ Balance1 = balance1 – 30 3 /* balance1 = 70, x.Account_Balance = 100, balance2 = 100 */ 4 Balance2 = balance2 + 120 /* balance1 = 70, x.Account_Balance = 100, balance2 = 120 */ Update Account Set Account_Balance =:balance 5 Where Account_Id = x /* balance1 = 70, x.Account_Balance = 70, balance2 = 100 */ Update Account 6 Set Account_Balance =:balance2 Where Account_Id = x /* balance1 = 70, x.Account_Balance = 120, balance2 = 120 */