1. Basic Web Application Development Instructor: Matthew Schurr

2. Guest Lecturer  Professor Jennifer Widom  Ph.D. in Computer Science (Cornell ‘87)  Professor, Stanford University  Chair, Stanford Computer Science Department  We will be looking at excerpts from her Stanford course, CS 145 Introduction to Databases.

3. (Videos)  Introduction (14 minutes)  Relational Models (16 minutes)  Relational Algebra (38 minutes)

4. Real-World Implementations  Database Systems Ideal for large-scale multi-user applications ○ Example: Facebook, web applications Software: PostgreSQL, MySQL, Oracle  Database Storage Ideal for locally stored single-user software ○ Example: An iPhone application Software: ○ SQLite - lightweight, simple, small datasets, single file

5. (Videos)  Structured Query Language (SQL) Introduction (5 minutes) SELECT Statements (10 minutes) Table Variables, Set Operators (10 minutes) Sub-queries in WHERE (20 minutes) Sub-queries in SELECT (8 minutes) Aggregation (25 minutes) NULL Values (7 minutes) Modifications (14 minutes)

6. (Videos)  Indexes (15 minutes)

7. LIKE vs. =  WHERE `name` = 'Matt'; Returns records where the name field is exactly “Matt”.  WHERE `name` LIKE 'Matt'; Returns records where the name field is “Matt” (case- insensitive).

8. Wildcards, Searching  For string fields, we can use a wildcard symbol % to search the database.  SELECT * FROM `people` WHERE `name` LIKE '%Joe%';  Returns all records in people where the name field contains the case-insensitive substring “Joe”.  This operation is not incredibly efficient, so use sparingly.

9. Replaces  REPLACE works exactly like INSERT with one important difference.  If an existing row in the table has the same unique key as the inserted row, the existing row is deleted and replaced with the new one. If no existing record is present, acts like a normal insert.  Let’s say we have the following table, users: userid (primary key) username (string) password (string)

10. Replaces  users ( userid, name, password )  Initially, users has one record: (1, “Matthew”, “password!”)  We run the query: REPLACE INTO `users` (`userid`, `password`) VALUES (1, ‘new password’);  What do you think will happen?

11. Replaces  The table will still have one record: (1, NULL, ‘new password’)  What happened to the username? REPLACE deletes the existing record and replaces it with the new one. Thus, we can not omit fields and expect them to be unchanged after the query. We should have included the username field again in the replace into query.  Why are replaces useful?

12. Concurrency  We are operating a banking web application.  Let’s say that Bob makes a deposit into his account. 1) $oldBalance = SELECT `balance` FROM `accounts` WHERE `acc` = ‘1’ LIMIT 1; 2) $newBalance = $oldBalance + $amountDeposited; 3) UPDATE `accounts` SET `balance` = $newBalance WHERE `acc` = ‘1’;  What happens if Bob’s wife tries to make a deposit at the same time? 4) $oldBalance = SELECT `balance` FROM `accounts` WHERE `acc` = ‘1’ LIMIT 1; 5) $newBalance = $oldBalance + $amountDeposited2; 6) UPDATE `accounts` SET `balance` = $newBalance WHERE `acc` = ‘1’;  The database system may choose to execute concurrent queries in any order; the only constraints are that 1 comes before 3 and 4 before 6. What happens if the system chooses 1, 4, 3, 6?

13. Data Race Prevention  We could fix the example in the previous slide by making the query for deposits be: UPDATE `accounts` SET `balance` = `balance` + $amountDeposited WHERE `acc` = ‘1’;  At first glance, you may think that this still results in a data race. However, concurrency controls within the database system guarantee that the end result of two queries (Q1, Q2) executed in parallel must be either: Q1; Q2; Q2; Q1;  Why did this not help us on the previous slide?

14. Additional Problems  What if we absolutely need to execute a sequence of queries in order to update something and we want to be sure no concurrent queries altered the intermediate values? Example: Bob and his wife making deposits at the same time where each deposit requires two queries (a read and a write).  What if we need to execute a group of queries and we must be absolutely certain that every single query within the group succeeds? Example: user performs action that required us to write values on multiples tables; connection to database is lost after executing only the first query

15. Transactions (IMPORTANT)  In order for your application to be secure, you must ensure that its data remains in a safe, consistent state at all times.  Consider a banking website that allows its users to transfer money between one another online. 1. UPDATE `accounts` SET `balance` = `balance` - :amount WHERE `user` = 'bob'; 2. UPDATE `accounts` SET `balance` = `balance` + :amount WHERE `user` = 'alice';  What happens if the server crashes after (1) is executed?  What happens if (1) fails but (2) succeeds?

16. Transactions (continued)  A transaction is a set of operations to a database that are treated as a unit.  A transaction… runs in isolation. enforces the constraint that all of its operations must succeed. If any operation fails, the database is reverted to the state it was in when the transaction began.  Transactions can happen concurrently, provide system failure recovery, and ensure the database is always in a consistent state.

17. Transactions  If the system crashes (or the client loses connection) during a transaction, then the transaction is invalidated and the database is rolled back. The user might have to repeat actions, but the database state remains consistent.  By default, every statement you enter is treated as a transaction comprised of a single query. To disable this, SET `autocommit` = 0; If you disable this, you will need to manually COMMIT; your queries.

18. Transactions (continue)  To use a transaction (in raw SQL): START TRANSACTION; …do your work… COMMIT;  If one of the queries in your transaction failed: ROLLBACK;  If the connection closes while you have an open transaction, the system will automatically call ROLLBACK.

19. ACID  Transactions guarantee… Atomicity Consistency Isolation Durability

20. Atomicity  Deals with crashes during transactions  Each transaction is “all-or-nothing”. We are never in an unsafe state where something is half done.  Rollbacks (Aborts) Can be initiated by client or system. Reverts the database to the state it was in when transaction began.

21. Consistency  Guarantees serializibility - that all of the constraints you have placed on your database always hold.  For example, we can use transactions to guarantee that money is not created or destroyed during online transfers.

22. Isolation  Each client appears to operate in isolation (unaffected by any other users).  Transactions may “lock” certain areas of the database, preventing concurrent access to these areas. Do not leave them open for extended periods of time.  If T1 and T2 are executed simultaneously, end behavior is either T1; T2; or T2; T1; The order might matter; databases do not account for this. Your application must handle this. Beware of isolation levels.

23. Isolation  “Dirty” data – written by an uncommitted transaction  “Dirty” read – reading a value in a DIFFERENT uncommitted transaction  Isolation levels control the effects of dirty reads and dirty writes within a transaction and in relation to concurrent transactions. Certain levels may decrease performance.  You should watch the Stanford videos on transactions if you are interested in a deeper understanding (outside of scope of this class).

24. Isolation Levels  You can set the isolation level for the next transaction using: SET TRANSACTION ISOLATION LEVEL …; ○ REPEATABLE READ ○ READ COMITTED ○ READ UNCOMITTED ○ SERIALIZABLE  REPEATABLE READ is generally the default (though it may vary by system) which disallows dirty reads.  These are outside of the course’s scope; watch the Transactions and Isolation videos if you are interested.

25. Read Only Transactions  You may wish to declare certain transactions as read only. SET TRANSACTION READ ONLY;  This will provide the system with additional information to optimize performance (the system will know that there are no writes within the transaction).  Isolation level will still affect behavior of dirty reads.

26. Durability  If the system crashes after a transaction is successfully committed, all effects of the transaction remain in the database.  This prevents data loss.

27. Transactions in Programs  Generally, your database API will simplify these for you. // Queries up here are autocommited. // Denotes a transaction. If any query fails, then // an exception is thrown and database rolled back. $db->transaction(function($db){ // Queries here are all or nothing. $stmt = $db->prepare(…); $read = $db->execute($stmt); $stmt = $db-prepare(…); $write = $db->execute($stmt); }); // Queries down here are autocommitted.

28. Transactions in Programs // Queries up here are autocommitted. try { $db->beginTransaction(); $db->query(…); $db->commit(); } catch (DatabaseException $e) { // If the transaction failed, you must close // the transaction by rolling back. $db->rollback(); // You might also want to display an error to user. } // Queries down here are autocommitted.

29. Further Learning  Relational Databases are a huge topic; we could study them for years.  If you’re interested in learning more, explore the additional videos I have included (not required for this course).