1. 1 ICS 214B: Transaction Processing and Distributed Data Management Course Project

2. ICS214BNotes 032 Client Distributed Travel Reservation System Resource Manager Flights Resource Manager Hotels Resource Manager Cars Resource Manager Customers Workflow Controller Transaction Manager

3. ICS214BNotes 033 Project Plan Two steps: –Build centralized travel reservation system  Components: Resource Manager –Add distributed functionality  Components: Workflow Controller, Transaction Manager

4. ICS214BNotes 034 Client Part 1: Simple Travel Resource Manager Resource Manager Flights, Hotels, Cars, Customers interface ResourceManager start(); queryFlightPrice(); reserveFlight(); queryCarPrice(); reserveCar(); … commit();

5. ICS214BNotes 035 Resource Manager Functionality Data Manipulation –Query and Update flight, car, and hotel data –Make reservations Transactions with ACID properties Technical/Testing Support

6. ICS214BNotes 036 Data Definition (fixed) Stores the following tables: –FLIGHTS(flightNum, price, numSeats, numAvail) –HOTELS(location, price, numRooms, numAvail) –CARS(location, price, numCars, numAvail) –CUSTOMERS(custName) –RESERVATIONS(custName, resvType, resvKey) Simple (unrealistic) assumptions –One hotel, rental car agency per location –One airline –All seats in a flight cost the same –All rooms and cars in a location cost the same

7. ICS214BNotes 037 Data Manipulation Operations Flights –Add flights –Add seats to flight –Remove seats from a flight –Cancel flight –Query number of seats available on a flight –Query price of a flight –Reserve seat in flight for a given customer Similar operations for Cars and Hotels –Described in interface ResourceManager.java

8. ICS214BNotes 038 Important concepts review We assume you’ve completed ICS214(A) or equivalent Here we present a short overview just to refresh your memory

9. ICS214BNotes 039 Consistent DBConsistent DB’ T Important concepts review

10. ICS214BNotes 0310 Big assumption: If T starts with consistent state + T executes in isolation  T leaves consistent state

11. ICS214BNotes 0311 What can go wrong? Transaction bug DBMS bug Hardware failure e.g., disk crash alters balance of account System crash –CPU halts, memory wiped out, disk intact Concurrency e.g.: T1: give 10% raise to programmers T2: change programmers  systems analysts

12. ICS214BNotes 0312 Key problem: Unfinished transaction ExampleConstraint: A=B T 1 : A  A  2 B  B  2

13. ICS214BNotes 0313 T 1 :Read (A,t); t  t  2 Write (A,t); Read (B,t); t  t  2 Write (B,t); Output (A); Output (B); A: 8 B: 8 A: 8 B: 8 memory disk 16 failure!

14. ICS214BNotes 0314 Need atomicity: execute all actions of a transaction or none at all

15. ICS214BNotes 0315 Implementing Atomicity In ICS214 (Fall 2002): –Logging In this programming project, use –Shadow paging –Why?  Shadow paging is simpler to implement  But performance is lower than logging

16. ICS214BNotes 0316 Shadow Paging Overview Each file is managed via a page table P –Each transaction T updates the file via a private page table –Commit T by replacing the public page table by a private one –Example: suppose DB has two files, “a” and “b”

17. ICS214BNotes 0317 P1a new Pt´[a] 1 2 3... Pt´[b] 1 2 3... P2a new P2b new Master Pointer abab P1a old Pt[a,1] 123123... Pt[b,1] 123123... P2a old P1b old P2b old DISKDISK Master´ a b Master MAIN MEMORY FOR T

18. ICS214BNotes 0318 P1a new Pt´[a] 1 2 3... Pt´[b] 1 2 3... P2a new P2b new Master Pointer abab P1a old Pt[a,1] 123123... Pt[b,1] 123123... P2a old P1b old P2b old DISKDISK Master´ a b Master

19. ICS214BNotes 0319 P1a new Pt´[a] 1 2 3... Pt´[b] 1 2 3... P2a new P2b new Master Pointer abab P1a old Pt[a,1] 123123... Pt[b,1] 123123... P2a old P1b old P2b old DISKDISK Master´ a b Master

20. ICS214BNotes 0320 It is a technique pioneered in System R changes are made to a copy of a page (block) When a transaction commits, the copy becomes the current page and the original is discarded SHADOW PAGING

21. ICS214BNotes 0321 Suppose transaction A starts up: the current page table (directory) is copied to the shadow page table (shadow directory) if the transaction updates a page, the original page is not altered, rather a copy is created and that is modified the copy is pointed to by the current page table - the shadow page table is never modified Single transaction

22. ICS214BNotes 0322 Write operation When transaction T i issues a write(A) command, the write operation is executed as follows (assuming data item A resides on page P A ): 1.If page P A is not already in main-memory then issue input(P A ) 2.If this is the first write operation on page P A by transaction T i then: (a) allocate a new disk page (call it tP A ) (b) copy P A into tP A (c) modify the current page table so that the entry corresponding to P A now points to tP A 3.perform the update on the page pointed to by tP A

23. ICS214BNotes 0323 Commit When transaction T i commits: 1.Ensure all buffer pages in memory that have been modified are flushed to the disk. 2.output the current page table to disk 3.change the current page table to become the new page table 4.free the pages of the old shadow page table that are no longer necessary 5.read the old shadow page table and free its pages

24. ICS214BNotes 0324 What is required if the system crashes while a transaction is executing? free up all modified pages discard the current page table reinstate the shadow page table as the current page table Crash recovery

25. ICS214BNotes 0325 appears simple for single transaction environments complexity increases for concurrent transactions clustering diminishes quickly not aware of any commercial implementations Pros and cons

26. ICS214BNotes 0326 What if two transactions update different pages of a file? –If they share their main memory copy of the page table, then committing one will commit the other’s updates too! Use a private copy of page table, per transaction. To commit T, within a critical section: –get a private copy of the last committed value of the page table of each file modified by T –update their entries for pages modified by T –store the updated page tables on disk –write a new master record and master pointer, thereby installing the update just for T (// end of critical section) Multiple transactions

27. ICS214BNotes 0327 Shadow Paging in Practice Reference: R. Lorie, “Physical Integrity in a Large Segmented Database”ACM Trans. on DB Sys., March 1977. Used in the Gemstone OO DBMS. Not good for high-performance TP systems –count disk updates per transaction –how to do record-level locking?

28. ICS214BNotes 0328 Concurrency Control T1T2…Tn DB (consistency constraints) Use the 3 rules (well- formed xacts, legal scheduler, 2PL) Support shared locks

29. ICS214BNotes 0329 System transparently acquires locks when transactions access data Holds all locks until transaction commits –Called “Strict 2-phase locking” –Strict 2PL automatically avoids cascading rollbacks # locks time Simple Locking System

30. ICS214BNotes 0330 Implementing the RM For simplicity, implement the tables using hashtables –Each row is a hashtable entry  Create a class for every kind of row e.g., Flight  Primary key is hashkey For the RESERVATIONS table, you can merge with the CUSTOMERS table: include a List of reservations in each hashtable entry.

31. ICS214BNotes 0331 Implementing the RM Java has convenient Hashtable and Vector classes For durability, write the database to disk –You can use Java serialization to directly write the hashtables to disk –The class java.io.ObjectOutputStream might be helpful

32. ICS214BNotes 0332 Implementing the RM Using serialized hashtables makes it for easy persistence But does not use block-based I/O model –Low-performance Need to adapt shadow paging for this model

33. ICS214BNotes 0333 Shadowing with hashtables Have two copies of file on disk, with master pointer pointing to last committed copy Last committed copy also cached in memory Each transaction has private update list: hashtable of entries it has read or written On commit: –Merge update list into in-memory table –Table written to disk using different filename –Master pointer updated to point to new file

34. ICS214BNotes 0334 In-Mem Copy A:3 B:5 C:2... Master Pointer File0 A:3 B:5 C:2... DISKDISK MAIN MEMORY Update List for T B:8 …...

35. ICS214BNotes 0335 In-Mem Copy A:3 B:8 C:2... Master Pointer File0 A:3 B:5 C:2... File1 A:3 B:8 C:2... DISKDISK MAIN MEMORY Update List for T B:8 …...

36. ICS214BNotes 0336 In-Mem Copy A:3 B:8 C:2... Master Pointer File0 A:3 B:5 C:2... File1 A:3 B:8 C:2... DISKDISK MAIN MEMORY Update List for T B:8 …...

37. ICS214BNotes 0337 Concurrency Control We provide a lock manager Supports two operations –lock(xid, itemName, READ|WRITE) –unlockAll(xid) Implement two-phase locking using the supplied lock manager

38. ICS214BNotes 0338 Resource Manager Transactions A client starts a transaction by calling the start() method –Returns a transaction id (xid) –Client includes xid in all data manipulation operation requests Client calls commit(xid) or abort(xid) to finish a transaction System crash or deadlock may forcibly abort transactions

39. ICS214BNotes 0339 Client Part 2: Distributed Travel Reservation System Resource Manager Flights Resource Manager Hotels Resource Manager Cars Resource Manager Customers Workflow Controller Transaction Manager

40. ICS214BNotes 0340 Part 2: Distributed Transactions Add Workflow Control and Transaction Manager components Implement 2-phase commit –To be covered later

41. ICS214BNotes 0341 Code base project directory with two packages lockmgr: do not change –lock & unlockAll in LockManager.java transaction: your code here –ResourceManager.java: do not change –ResourceManagerImpl.java –Client.java –Makefile

42. ICS214BNotes 0342 Java RMI RMI: Remote Method Invocation –The way Client talks to the RM 1. Start rmiregistry –Use your own port number 2. Start server (RM) –RM binds to a name at the registry 3. Start Client –Client queries registry using the bind name All taken care of in the code base

43. ICS214BNotes 0343 Java synchronized constructs Convenient critical section implementation Synchronized block: –Associated with any object –Before entering, thread obtains exclusive lock on obj. –no two threads inside synch. blocks belonging to same obj at same time Synchronized methods: lock on “this”

44. ICS214BNotes 0344 Java synchronized statements Synchronized block example: AAA synchronized (ht) { BBB } CCC Synchronized method example: public synchronized void foo() {SAME AS … } public void foo() { synchronized (this) { … }

45. ICS214BNotes 0345 Java Hashtable Table of (key,value) pairs Hashtable ht = new Hashtable(); … // Insertion ht.put(“Flight214, flt); … // Lookup flt = (Flight) ht.get(“Flight214”);

46. ICS214BNotes 0346 Java serialization Object Serialization is process of writing the state of an object to a byte stream –Serializable objects can be written out to disk and restored easily. –Needed by RMI Hashtable implements Serializable, so: ObjectOutputStream outS = new ObjectOutputStream(new FileOutputStream(“data/RMimage1”); outS.writeObject(ht); ---------------------------------------------- ObjectInputStream inS = new ObjectInputStream(new FileInputStream(“data/RMimage1”); Hashtable ht = (Hashtable) inS.readObject();

47. ICS214BNotes 0347 Project Logistics Getting started: class home page –http://www.ics.uci.edu/~ics214b/ Work on ICS SUN machines Collaboration policy –As always, each group can have up to 3 students Due dates: –part 1: Feb 15, Saturday –part 2: March 14, Friday Submit: source code + short README Grading: correct functionality –We’ll use our own Client.java to test your code

48. ICS214BNotes 0348 Acknowledgements This project is based on a similar course project developed by Anand Rajaraman at Stanford, which was based on a similar project developed by Phil Bernstein at the University of Washington