JDBC

DBI 2008 HUJI-CS 2 Java Database Connectivity JDBC (Java Database Connectiveity) is an API (Application Programming Interface), –That is, a collection of classes and interfaces JDBC is used for accessing (mainly) databases from Java applications Information is transferred from relations to objects and vice-versa –databases optimized for searching/indexing –objects optimized for engineering/flexibility

DBI 2008 HUJI-CS 3 JDBC Architecture Java Application JDBC Oracle DB2 Postgres Oracle Driver DB2 Driver Postgres Driver These are Java classes Network We will use this one…

DBI 2008 HUJI-CS 4 JDBC Architecture (cont.) Java code calls JDBC library JDBC loads a driver The driver talks to a particular database An application can work with several databases by using all corresponding drivers Ideal: change database engines w/o changing any application code (not always in practice) ApplicationJDBCDriver

DBI 2008 HUJI-CS 5 Seven Steps Load the driver Define the connection URL Establish the connection Create a Statement object Execute a query using the Statement Process the result Close the connection

DBI 2008 HUJI-CS 6 Registering the Driver To use a specific driver, instantiate and register it within the driver manager: Driver driver = new org.postgresql.Driver(); DriverManager.registerDriver(driver); Driver driver = new org.postgresql.Driver(); DriverManager.registerDriver(driver);

DBI 2008 HUJI-CS 7 A Modular Alternative We can register the driver indirectly using Class.forName loads the given class dynamically When the driver is loaded, it automatically –creates an instance of itself –registers this instance within DriverManager Hence, the driver class can be given as an argument of the application Class.forName("org.postgresql.Driver");

DBI 2008 HUJI-CS 8 An Example // A driver for imaginary1 Class.forName("ORG.img.imgSQL1.imaginary1Driver"); // A driver for imaginary2 Driver driver = new ORG.img.imgSQL2.imaginary2Driver(); DriverManager.registerDriver(driver); //A driver for PostgreSQL Class.forName("org.postgresql.Driver"); imaginary1 imaginary2 Registered Drivers Postgres

DBI 2008 HUJI-CS 9 Connecting to the Database Every database is identified by a URL Given a URL, DriverManager looks for the driver that can talk to the corresponding database DriverManager tries all registered drivers, until a suitable one is found How is this done?

DBI 2008 HUJI-CS 10 Connecting to the Database Connection con = DriverManager.getConnection("jdbc:imaginaryDB1"); imaginary1 imaginary2 Registered Drivers Postgres    acceptsURL("jdbc:imaginaryDB1")?

DBI 2008 HUJI-CS 11 The URLs in HUJI-CS In CS, a URL has the following structure: The machine running PostgrSQL jdbc:postgresql://dbserver/public?user=?? Your login You can only access your own account!

DBI 2008 HUJI-CS 12 Interaction with the Database We use Statement objects in order to –Query the db –Update the db (insert, update, create, drop, …) Three different interfaces are used: Statement, PreparedStatement, CallableStatement All are interfaces, hence cannot be instantiated They are created by the Connection

DBI 2008 HUJI-CS 13 Querying with Statement executeQuery returns a ResultSet object representing the query result (discussed later…) String queryStr = "SELECT * FROM Member " + "WHERE Lower(Name) = 'harry potter'"; Statement stmt = con.createStatement(); ResultSet rs = stmt.executeQuery(queryStr);

DBI 2008 HUJI-CS 14 Changing DB with Statement String deleteStr = "DELETE FROM Member " + "WHERE Lower(Name) = 'harry potter'"; Statement stmt = con.createStatement(); int delnum = stmt.executeUpdate(deleteStr); executeUpdate is for data manipulation: insert, delete, update, create table, etc. –Anything other than querying! executeUpdate returns the number of rows modified (or 0 for DDL commands)

DBI 2008 HUJI-CS 15 About Prepared Statements Prepared statements are used for queries that are executed many times Parsed (compiled) by the DBMS only once Values of some columns are set after compilation Instead of values, use ‘?’ and setType methods Hence, prepared statements can be thought of as statements that contain placeholders to be substituted later with actual values

DBI 2008 HUJI-CS 16 Querying with PreparedStatement String q = "SELECT * FROM Items " + "WHERE Name = ? and Cost < ?"; PreparedStatement pstmt=con.prepareStatement(q); pstmt.setString(1, "t-shirt"); pstmt.setInt(2, 1000); ResultSet rs = pstmt.executeQuery();

DBI 2008 HUJI-CS 17 Updating with PreparedStatement String dq = "DELETE FROM Items " + "WHERE Name = ? and Cost > ?"; PreparedStatement pstmt = con.prepareStatement(dq); pstmt.setString(1, "t-shirt"); pstmt.setInt(2, 1000); int delnum = pstmt.executeUpdate();

DBI 2008 HUJI-CS 18 Statement vs. PreparedStatement : Be Careful! String val = "abc"; PreparedStatement pstmt = con.prepareStatement("select * from R where A=?"); pstmt.setString(1, val); ResultSet rs = pstmt.executeQuery(); String val = "abc"; PreparedStatement pstmt = con.prepareStatement("select * from R where A=?"); pstmt.setString(1, val); ResultSet rs = pstmt.executeQuery(); String val = "abc"; Statement stmt = con.createStatement( ); ResultSet rs = stmt.executeQuery("select * from R where A=" + val); String val = "abc"; Statement stmt = con.createStatement( ); ResultSet rs = stmt.executeQuery("select * from R where A=" + val); Are these the same? What do they do?

DBI 2008 HUJI-CS 19 What can be Assigned to “?” Will this work? No!!! “?” can only represent a column value (to enable pre-compilation) PreparedStatement pstmt = con.prepareStatement("select * from ?"); pstmt.setString(1, myFavoriteTableString ); PreparedStatement pstmt = con.prepareStatement("select * from ?"); pstmt.setString(1, myFavoriteTableString );

DBI 2008 HUJI-CS 20 PreparedStatement and Security Suppose Google was implemented in JDBC without a PreparedStatement. The main DB query might have been implemented like this: Statement s; s.executeQuery("SELECT URL,Title from Internet " + "WHERE Content LIKE ‘%" + searchString + "%’"); What would happen if a hacker searched for: aaaaa’ UNION SELECT Company AS URL, CreditCardNum AS Title FROM AdvertisingClients WHERE Company LIKE ‘ SQL InjectionThis technique is known as SQL Injection and is the main reason for using PreparedStatement s

DBI 2008 HUJI-CS 21Timeout Use setQueryTimeOut(int seconds) of Statement to set a timeout for the driver to wait for a query to be completed If the operation is not completed in the given time, an SQLException is thrown What is it good for?

DBI 2008 HUJI-CS 22ResultSet ResultSet objects provide access to the tables generated as results of executing Statement queries Only one ResultSet per Statement or PreparedStatement can be open at a given time! The table rows are retrieved in sequence –A ResultSet maintains a cursor pointing to its current row –next() moves the cursor to the next row

DBI 2008 HUJI-CS 23 ResultSet Methods boolean next() –Activates the next row –First call to next() activates the first row –Returns false if there are no more rows –Not all of the next calls actually involve the DB void close() –Disposes of the ResultSet –Allows to re-use the Statement that created it –Automatically called by most Statement methods

DBI 2008 HUJI-CS 24 ResultSet Methods (cont’d) Type getType(int columnIndex) –Returns the given field as the given type –Indices start at 1 and not 0! –Add the column name as a comment if it is known! Type getType(String columnName) –Same, but uses name of field –Less efficient (but may not be your bottleneck anyway) Examples: getString(5), getInt(“salary”), getTime(…), getBoolean(…),... int findColumn(String columnName) –Looks up column index given column name

DBI 2008 HUJI-CS 25 ResultSet Example Statement stmt = con.createStatement(); ResultSet rs = stmt. executeQuery("select name,age from Employees"); // Print the result while (rs.next()) { System.out.print(rs.getString(1) + ":"); System.out.println(rs.getShort("age")); }

DBI 2008 HUJI-CS 26 Mapping Java Types to SQL Types SQL TypeJava Type CHAR, VARCHAR, LONGVARCHARString NUMERIC, DECIMALjava.math.BigDecimal BITboolean TINYINTbyte SMALLINTshort INTEGERint BIGINTlong REALfloat FLOAT, DOUBLEdouble BINARY, VARBINARY, BYTEAbyte[] DATEjava.sql.Date TIMEjava.sql.Time TIMESTAMPjava.sql.Timestamp

DBI 2008 HUJI-CS 27 Null Values In SQL, NULL means the field is empty Not the same as 0 or “”! In JDBC, you must explicitly ask if the last- read field was null –ResultSet.wasNull(column) For example, getInt(column) will return 0 if the value is either 0 or NULL!

DBI 2008 HUJI-CS 28 Null Values When inserting null values into placeholders of a PreparedStatement : –Use setNull(index, Types.sqlType) for primitive types (e.g. INTEGER, REAL ); –For object types (e.g. STRING, DATE ) you may also use setType(index, null)

DBI 2008 HUJI-CS 29 Result-Set Meta-Data A ResultSetMetaData is an object that can be used to get information about the properties of the columns in a ResultSet object An example: Write the columns of the result set ResultSetMetaData rsmd = rs.getMetaData(); int numcols = rsmd.getColumnCount(); for (int i = 1 ; i <= numcols; i++) System.out.print(rsmd.getColumnLabel(i)+" ");

DBI 2008 HUJI-CS 30 Database Time Times in SQL are notoriously non-standard Java defines three classes to help java.sql.Date –year, month, day java.sql.Time –hours, minutes, seconds java.sql.Timestamp –year, month, day, hours, minutes, seconds, nanoseconds –Usually use this one

DBI 2008 HUJI-CS 31 Cleaning Up After Yourself Remember: close Connections, Statements, Prepared Statements and Result Sets con.close(); stmt.close(); pstmt.close(); rs.close()

DBI 2008 HUJI-CS 32 Dealing With Exceptions An SQLException is actually a list of exceptions catch (SQLException e) { while (e != null) { System.out.println(e.getSQLState()); System.out.println(e.getMessage()); System.out.println(e.getErrorCode()); e = e.getNextException(); }}

DBI 2008 HUJI-CS 33 General SQL Advise Take the time to carefully design your database before you start coding – this will save you time and frustration The same data can be organized in very different ways – When designing your DB and when deciding what your primary keys should be, always think about typical use cases and deduce from them which queries are most common and which queries must run fastest Add as many constraints as possible ( NOT NULL, UNIQUE, etc…) – this will help you debug prevent data corruption even if you missed a bug

Transaction Management

DBI 2008 HUJI-CS 35 Transactions and JDBC Transaction:Transaction: more than one statement that must all succeed (or all fail) together –e.g., updating several tables due to customer purchase Failure − System must reverse all previous actions Also can’t leave DB in inconsistent state halfway through a transaction COMMIT = complete transaction ROLLBACK = cancel all actions

DBI 2008 HUJI-CS 36 An Example Suppose that we want to transfer money from bank account 13 to account 72: PreparedStatement pstmt = con.prepareStatement("update BankAccount set amount = amount + ? where accountId = ?"); pstmt.setInt(1,-100); pstmt.setInt(2, 13); pstmt.executeUpdate(); pstmt.setInt(1, 100); pstmt.setInt(2, 72); pstmt.executeUpdate(); What happens if this update fails?

DBI 2008 HUJI-CS 37 Transaction Lifetime Transactions are not opened and closed explicitly A transaction starts on 1 st (successful) command –After a connection is established –After the previous transaction ends A transaction ends when COMMIT or ROLLBACK are applied –Either explicitly or implicitly (see next 4 slides)

DBI 2008 HUJI-CS 38 Committing a Transaction How do we commit? Explicitly invoking Connection.commit() Implicitly –After every query execution, if AutoCommit is true –When the user normally disconnects (i.e., appropriately closes the connection) –In some DBs: After invoking a DDL command (CREATE, DROP, RENAME, ALTER, …)

DBI 2008 HUJI-CS 39 Automatic Commitment A Connection object has a boolean AutoCommit If AutoCommit is true (default), then every statement is automatically committed If AutoCommit is false, then each statement is added to an ongoing transaction Change using setAutoCommit(boolean) If AutoCommit is false, need to explicitly commit or rollback the transaction using Connection.commit() and Connection.rollback()

DBI 2008 HUJI-CS 40 Rolling Back Rolling Back: Undoing any change to data within the current transaction The ROLLBACK command explicitly rolls back (and ends) the current transaction ROLLBACK is implicitly applied when the user abnormally disconnects (i.e., without appropriately closing the connection)

DBI 2008 HUJI-CS 41 Fixed Example con.setAutoCommit(false); try { PreparedStatement pstmt = con.prepareStatement("update BankAccount set amount = amount + ? where accountId = ?"); pstmt.setInt(1,-100); pstmt.setInt(2, 13); pstmt.executeUpdate(); pstmt.setInt(1, 100); pstmt.setInt(2, 72); pstmt.executeUpdate(); con.commit(); catch (SQLException e) { con.rollback(); …; }

DBI 2008 HUJI-CS 42 Transaction Isolation How do different transactions interact? Does a running transaction see uncommitted changes? Does it see committed changes? Don’t worry about this now – we will discuss it in future lessons (see the Appendix if you really can’t wait)

Managing Large Objects In PosgreSQL-JDBC

DBI 2008 HUJI-CS 44 LOBs: Large OBjects A database can store large pieces of data –e.g., images or other files Sometimes, two distinguished types: –CLOB: Character large object (a large number of characters) –BLOB: Binary large object (a lot of bytes) Actual data is not stored in the table, only a pointer (Object ID) to the data storage

DBI 2008 HUJI-CS 45 LOBs in PostgreSQL LOBs MyImages NameDescriptionconten t image1Human face53 image2Rabbit113 Create a new lob Get an in-stream to Obj. 24 LOB Manager Get an out-stream Obj. 53

DBI 2008 HUJI-CS 46 Storing LOBs (1) Given: A binary source (file, socket, etc.), readable through an InputStream object Goal: Store the content of the source in the table MyImages CREATE TABLE MyImages ( name VARCHAR(20), content OID); CREATE TABLE MyImages ( name VARCHAR(20), content OID);

DBI 2008 HUJI-CS 47 Storing LOBs (2) 1.Disable auto-commit (why???): 2.Obtain the LOB manager: Downcast con into org.postgresql.PGConnection 3.Create a new LOB LargeObjectManager lobm = con.getLargeObjectAPI() con.setAutoCommit(false) int oid = lobm.create(LargeObjectManager.READ | LargeObjectManager.WRITE)

DBI 2008 HUJI-CS 48 Storing LOBs (3) 4.Insert a new row, with the LOB’s oid PreparedStatement pstmt = con.prepareStatement ("INSERT INTO MyImages VALUES (?, ?)"); pstmt.setString(1, imageName); pstmt.setInt(2, oid); pstmt.executeUpdate(); pstmt.close();

DBI 2008 HUJI-CS 49 Storing LOBs (4) 5.Fill the LOB with the input bytes 6.Finalize LargeObject obj = lobm. open(oid, LargeObjectManager.WRITE); int bytesRead = 0; byte[] data = new byte[4096]; while ((bytesRead = iStream.read(data)) >= 0) obj.write(data,0,bytesRead); obj.close(); iStream.close(); con.commit();

DBI 2008 HUJI-CS 50 Retrieving LOBs (1) 1.Obtain the LOB ID from the relevant row: PreparedStatement pstmt = con.prepareStatement ("select bytes from MyImages where name = ?"); pstmt.setString(1, filename); ResultSet rs = pstmt.executeQuery(); rs.next(); int oid = rs.getInt(1); rs.close(); pstmt.close();

DBI 2008 HUJI-CS 51 Retrieving LOBs (2) 2.Using oid, obtain a the corresponding LargeObject instance from the LOB Manager Downcast con into org.postgresql.PGConnection LargeObjectManager lobm = con.getLargeObjectAPI(); LargeObject obj = lobm.open(oid,LargeObjectManager.READ);

DBI 2008 HUJI-CS 52 Retrieving LOBs (3) 3.Read the content of the large object (that acts as an input/output stream) while ((bytesRead = obj.read(data,0,data.length)) > 0) oStream.write(data, 0, bytesRead); obj.close(); oStream.close();

Appendix: PostgreSQL Command-Line Interface

DBI 2008 HUJI-CS 54 Entering the Command-Line UI psql -hdbserver public Server running PostgreSQL Database Name Automatically enters your own part of the database (based on your login) Recommendation: > echo ”alias psql ’psql –hdbserver public’” >> ~/.aliases

DBI 2008 HUJI-CS 55 A Screenshot You can now write SQL queries

DBI 2008 HUJI-CS 56 Additional Useful Commands \dt List all tables \d table-name Show table definition \dT List all types \i sql-file Include SQL from file \dl List all large objects \lo_export oid file Export a large object \lo_import file Import a large object \q Quit

DBI 2008 HUJI-CS 57 Transactions in Command Line By default, the command-line session runs in AUTOCOMMIT mode –That is, commit is implicitly applied after every command To use transactions, you need to explicitly specify the beginning and end of a transaction by the commands BEGIN and At the end of the transaction, you either apply COMMIT, ROLLBACK or END

Appendix: Byte-Arrays

DBI 2008 HUJI-CS 59 Representing Byte Arrays Use the SQL type BYTEA to hold byte arrays CREATE TABLE MyBStrings ( name VARCHAR(20), content BYTEA); CREATE TABLE MyBStrings ( name VARCHAR(20), content BYTEA); (works with PostgreSQL, may not work with other DBMSs)

DBI 2008 HUJI-CS 60 Storing Byte Arrays Store array b[] of bytes: 1.Prepare a row-inserting statement 2.Set the statement values: 3.Execute and close PreparedStatement pstmt =con.prepareStatement ("INSERT INTO MyBStrings VALUES (?, ?)"); pstmt.setString(1, desiredName ); pstmt. setBytes (2, b); pstmt.executeUpdate(); pstmt.close();

DBI 2008 HUJI-CS 61 Retrieving Byte Arrays Retrieve bytes with the name “myBytes”: 1.Apply a selection query 2.Obtain the byte-array from the result 3.Close resources PreparedStatement pstmt =con.prepareStatement (“SELECT content from MyBStrings where name=?"); pstmt.setString(1, "myBytes"); ResultSet rs = ps.executeQuery(); rs.next(); byte[] theBytes = rs. getBytes (1); rs.close(); pstmt.close();

Appendix: Transaction Isolation

DBI 2008 HUJI-CS 63 Transaction Isolation How do different transactions interact? Does a running transaction see uncommitted changes? Does it see committed changes?

DBI 2008 HUJI-CS 64 Transaction Isolation Levels The isolation level determines the capabilities of a transaction to read/write data that is accessed by other transactions In PostgreSQL, two levels of isolation: 1.READ COMMITTED (default) 2.SERIALIZABLE Each transaction determines its isolation level –Connection.setTransactionIsolation(int level)

DBI 2008 HUJI-CS 65 Isolation Level in the Command-Line UI SET SESSION CHARACTERISTICS AS TRANSACTION ISOLATION LEVEL READ COMMITTED; SET SESSION CHARACTERISTICS AS TRANSACTION ISOLATION LEVEL SERIALIZABLE; Changing to read committed: Changing to serializable:

DBI 2008 HUJI-CS 66 READ COMMITED & SERIALIZABLE In principle, a query in PostgreSQL never reads uncommitted (i.e. dirty) data SERIALIZABLE: During the whole transaction, statements read only the changes that were committed by the time the transaction begun (and the changes made by the transaction itself) READ COMMITTED: A statement reads the data that was committed by the time the statement (not the transaction) begun

DBI 2008 HUJI-CS 67 Update: Row-Level Locking PostgreSQL disables one from updating a row that is updated by an uncommitted transaction –The second updating transaction is blocked until the first one commits or rolls back If 2 nd update is within READ COMMITTED, then 2 nd update is done after 1 st transaction commits or rolls back If 2 nd update is within SERIALIZABLE, then 2 nd update fails if 1 st transaction commits; otherwise, it succeeds

DBI 2008 HUJI-CS 68 Some Definitions Dirty reads:Dirty reads: A transaction reads data that is written by another, uncommitted transaction Non-repeatable reads:Non-repeatable reads: A transaction rereads data it previously read and finds that a committed transaction has modified or deleted that data Phantom reads:Phantom reads: A transaction re-executes a query returning a set of rows satisfying a search condition and finds that a committed transaction inserted additional rows that satisfy the condition

DBI 2008 HUJI-CS 69 READ COMMITED vs. SERIALIZABLE SERIALIZABLE READ COMMITED Impossible Dirty Reads ImpossiblePossible Non-repeatable Reads ImpossiblePossible Phantom Reads

DBI 2008 HUJI-CS 70 What Happens Here (1)? CREATE TABLE pairs (x INTEGER, y INTEGER); select * from pairs insert into pairs values(1,1) select * from pairs COMMIT select * from pairs insert into pairs values(1,1) select * from pairs COMMIT select * from pairs insert into pairs values(1,2) select * from pairs COMMIT select * from pairs insert into pairs values(1,2) select * from pairs COMMIT T.1: R. COMMITTED T.2: SERIALIZABLE

DBI 2008 HUJI-CS 71 What Happens Here (2)? CREATE TABLE pairs (x INTEGER, y INTEGER); T.1: R. COMMITTED T.2: SERIALIZABLE insert into pairs values(1,1) COMMIT select * from pairs select * from pairs COMMIT insert into pairs values(1,1) COMMIT select * from pairs select * from pairs COMMIT select * from pairs select * from pairs insert into pairs values(1,2) COMMIT select * from pairs select * from pairs insert into pairs values(1,2) COMMIT

DBI 2008 HUJI-CS 72 What Happens Here (3)? CREATE TABLE pairs (x INTEGER, y INTEGER); insert into pairs values(1,1) COMMIT select * from pairs select * from pairs COMMIT insert into pairs values(1,1) COMMIT select * from pairs select * from pairs COMMIT select * from pairs select * from pairs insert into pairs values(1,2) COMMIT select * from pairs select * from pairs insert into pairs values(1,2) COMMIT T.1: SERIALIZABLE T.2: SERIALIZABLE Is it equivalent to any truly serial execution of the transactions?

DBI 2008 HUJI-CS 73 What Happens Here (4)? CREATE TABLE pairs (x INTEGER, y INTEGER); insert into pairs values(1,1) COMMIT update pairs set y=3 where x= select * from pairs COMMIT insert into pairs values(1,1) COMMIT update pairs set y=3 where x= select * from pairs COMMIT update pairs set y=2 where x= select * from pairs COMMIT update pairs set y=2 where x= select * from pairs COMMIT T.1: SERIALIZABLE T.2: SERIALIZABLE

DBI 2008 HUJI-CS 74 What Happens Here (5)? CREATE TABLE pairs (x INTEGER, y INTEGER); insert into pairs values(1,1) COMMIT update pairs set y=3 where x= select * from pairs COMMIT insert into pairs values(1,1) COMMIT update pairs set y=3 where x= select * from pairs COMMIT update pairs set y=2 where x= select * from pairs COMMIT update pairs set y=2 where x= select * from pairs COMMIT T.1: R. COMMITTED T.2: SERIALIZABLE

Appendix: Changing Passwords

DBI 2008 HUJI-CS 76 Changing Your Password Suppose that snoopy wants to change his password from “snoopy” to “snoopass”: ALTER USER snoopy WITH PASSWORD 'snoopass';