1. 1 Oracle Database Applications Database Connectivity

2. 2 Outline Motivation Architecture 7 steps Sample program Metadata

3. 3 Motivation Most popular form of database system is the relational database system. Structured Query Language (SQL) is used among relational databases to construct queries. Applications need to query databases

4. 4 Simple Database Application APPLICATIONAPPLICATION DBMSDB Oracle

5. 5 Multi-Databases APPLICATIONAPPLICATION DBMS 1 DBMS 2 DBMS 3 DB

6. 6 Standard Access to DB APPLICATIONAPPLICATION DRIVERMGRDRIVERMGR DBMS Driver 1 DBMS Driver 2 DBMS Driver 3 DBMS 1 DBMS 2 DBMS 3 DB

7. 7 ODBC Architecture Application Class1Class2 Driver Manager DriverType1DriverType2DriverType3 DataSource2 DataSource1 DataSource3 ODBC

8. 8 Open Database Connectivity (ODBC) Standard ODBC standard is an interface by which application programs can access and process SQL databases in a DBMS-independent manner. It contains: – A Data Source that is the database, its associated DBMS, operating system and network platform – A DBMS Driver that is supplied by the DBMS vendor (Oracle) or independent software companies – A Driver Manager that is supplied by the vendor of the O/S platform (e.g., Windows/UNIX/Mac) where the application is running – E.g Control Panel->Administrative Tools->Data Sources (ODBC) in Windows. You can use ODBC to access MS Access or even Excel documents using the corresponding drivers.

9. 9 ODBC Interface System-independent interface to database environment –requires an ODBC driver to be provided for each database system from which you want to manipulate data. The ODBC driver manager bridges the differences between –The ODBC DBMS-independent interface and the DBMS-dependent interface of the DBMS driver

10. 10 An Example ApplicationDriverManager Oracle DriverDB2 driver SQLServer driver odbc standard API

11. 11 Java Support for Database Connectivity When applications written in Java want to access data sources, they use classes and associated methods provided by Java DBC (JDBC) API. JDBC is specified an “interface.” An interface in Java can have many “implementations.” –it provides a convenient way to realize many “drivers” JDBC can be an interface to an ODBC driver manager. Also it provides a JDBC API as a standard way to directly connect to common relational databases.

12. 12 Application in Java Application in Java DriverManager odbc standard API jdbc API Oracle DriverDB2 driver SQLServer driver

13. 13 Application in Java Application in Java Oracle Driver Jdbc API

14. 14 Java Support for SQL Java supports embedded SQL. –Embedded SQL in Java (SQLJ is one version of it) provides a standardized way for Java programs to interact with multiple databases, but at a higher level than the existing JDBC API. –Embedded SQL allows connecting to a database by including SQL code right in the program. –An Embedded SQL preprocessor converts the SQL statements to Java code at pre-compile time. The preprocessor generates code that includes the driver functionality for direct connection to the DBMS via JDBC. java.sql package and an extensive exception hierarchy.

15. 15 Data Source and Driver Data source: database created using any of the common database applications. Your system should have the driver for the database you will be using. –E.g., MS SQL Server on a Windows system. There are a number of JDBC drivers available. http://industry.java.sun.com/products/jdbc/drivers For oracle: http://jdbc.oracle.org/http://jdbc.oracle.org/

16. 16 JDBC Components Driver Manager: Loads database drivers, and manages the connection between application & driver. Driver: Translates API calls to operations for a specific data source. Connection: A session between an application and a driver. Statement: A SQL statement to perform a query or an update operation. Metadata: Information about the returned data, driver and the database. Result Set : Logical set of columns and rows returned by executing a statement.

17. 17 JDBC Classes Java supports DB facilities by providing classes and interfaces for its components DriverManager class Connection interface (abstract class) Statement interface (to be instantiated with values from the actual SQL statement) ResultSet interface

18. 18 java.sql JDBC is implemented via classes in the java.sql package –Supports SQL-2 entry level Define objects for: –Remote connection to DB –Executing query 8 interfaces to define objects: –Statement, CallableStatement, PreparedStatement, DatabaseMetaData, ResultSetMetaData, ResultSet, Connection, Driver

19. 19 DriverManager Driver Connection Statement ResultSet Load the driver Define the Connection URL Establish the Connection Create a Statement object Execute a query Process the result Close the connection Seven Steps

20. 20 Loading the Driver Registering the driver directly automatically: Class.forName(“org.oracle.Driver"); Calling Class.forName, which automatically –creates an instance of the driver –registers the driver with the DriverManager

21. 21 Identifying Data Sources Gives the required information for making the connection to the database Specified using the URL format. : : Example: jdbc:oracle://foo.itu.edu/mydatabase jdbc:oracle://localhost/testdb

22. 22 Another Option Create an instance of the driver and register it with the Driver Manager: Driver driver = (Driver) Class.forName("org.oracle.Driver").newInstance(); DriverManager.registerDriver(driver);

23. 23 Connection A Connection represents a session with a specific database Within the context of a Connection, SQL statements are executed and results are returned

24. 24 Connections There can be multiple connections to a database A connection provides “metadata”, i.e., information about the database, tables, and fields Connection object has methods to deal with transactions

25. 25 Creating a Connection Use getConnection on the Driver Connection getConnection (String url, String user, String password) Connects to given JDBC URL with given user name and password Throws java.sql.SQLException returns a Connection object

26. 26 Creating a Connection Connection connection = DriverManager.getConnection("jdbc:or acle://localhost/moviedb", "testuser", "mypassword");

27. 27 Statements Statement createStatement() –returns a new Statement object –Used for general queries PreparedStatement prepareStatement(String sql) –returns a new PreparedStatement object –For a statement called multiple times with different values (precompiled to reduce parsing time) CallableStatement prepareCall(String sql) –returns a new CallableStatement object –For stored procedures

28. 28 Statements A Statement object is used for executing a static SQL statement and obtaining the results produced by it executeQuery is used for statements that return an output result executeUpdate is used for statements that need not return an output

29. 29 Executing Queries and Updates ResultSet executeQuery(String) –Execute a SQL statement that returns a single ResultSet int executeUpdate(String) –Execute a SQL INSERT, UPDATE or DELETE statement –Used for CREATE TABLE, DROP TABLE and ALTER TABLE –Returns the number of rows changed

30. 30 Timeout Use setQueryTimeOut to set a timeout for the driver to wait for a statement to be completed If the operation is not completed in the given time, an SQLException is thrown What is it good for?

31. 31 Cursor What is the result of a query? How can a database send the result of a query through communication lines? The answer: using a cursor

32. 32 Result Set A ResultSet provides access to a table of data generated by executing a Statement Only one ResultSet per Statement can be open at once The table rows are retrieved in sequence A ResultSet maintains a cursor pointing to its current row of data The 'next' method moves the cursor to the next row –you can’t rewind

33. 33 Working with ResultSet boolean next() –activates the next row –the first call to next() activates the first row –returns false if there are no more rows void close() –disposes of the ResultSet –automatically called by most Statement methods

34. 34 Type getType(int columnIndex) –returns the given field as the given type –E.g., int getInt(5); string getString(3); –fields indexed starting at 1 (not 0) Type getType(String columnName) –same, but uses name of field –less efficient int findColumn(String columnName) –looks up column index given column name Getting Values from Rows

35. 35 isNull In SQL, NULL means the field is empty Not the same as 0 or “” In JDBC, you must explicitly ask if a field is null by calling ResultSet.isNull(column)

36. 36 Mapping Java Types to SQL Types SQL type Java Type CHAR, VARCHAR, LONGVARCHARString NUMERIC, DECIMALjava.math.BigDecimal BITboolean TINYINTbyte SMALLINTshort INTEGERint BIGINTlong REALfloat FLOAT, DOUBLEdouble BINARY, VARBINARY, LONGVARBINARYbyte[] DATEjava.sql.Date TIMEjava.sql.Time TIMESTAMPjava.sql.Timestamp

37. 37 Database Time Times in SQL are nonstandard Java defines three classes to help java.sql.Date –year, month, day java.sql.Time –hours, minutes, seconds java.sql.Timestamp –year, month, day, hours, minutes, seconds, nanoseconds –usually use this one

38. 38 Optimized Statements Prepared Statements –SQL calls that you make again and again –allows driver to optimize (compile) queries –created with Connection.prepareStatement() Stored Procedures –written in DB-specific language –stored inside database –accessed with Connection.prepareCall()

39. 39 Prepared Statement Example PreparedStatement updateSales; String updateString = "update COFFEES " + "set SALES = ? where COF_NAME like ?"; updateSales = con.prepareStatement(updateString); int [] salesForWeek = {175, 150, 60, 155, 90}; String [] coffees = {"Colombian", "French_Roast", "Espresso","Colombian_Decaf","French_Roast_Decaf"}; int len = coffees.length; for(int i = 0; i < len; i++) { updateSales.setInt(1, salesForWeek[i]); updateSales.setString(2, coffees[i]); updateSales.executeUpdate(); }

40. 40 JDBC Class Diagram

41. 41 Metadata Connection: –DatabaseMetaData getMetaData() ResultSet: –ResultSetMetaData getMetaData()

42. 42 ResultSetMetaData What's the number of columns in the ResultSet? What's a column's name? What's a column's SQL type? What's the column's normal max width in chars? What's the suggested column title for use in printouts and displays? What's a column's number of decimal digits? Does a column's case matter? and so on...

43. 43 DatabaseMetaData What tables are available? What's our user name as known to the database? Is the database in read-only mode? If table correlation names are supported (association of a column with the table it comes from, when multiple columns of the same name appear in the same query - multi-table queries), are they restricted to be different from the names of the tables? and so on…

44. 44 Useful Methods of Metadata getColumnCount getColumnDisplaySize getColumnName getColumnType isNullabale Imagine the case where you want to print the result

45. 45 Transaction Management A transaction: a sequence of SQL statements Transactions are not explicitly opened and closed Instead, the connection has a state called AutoCommit mode if AutoCommit is true, then every statement is automatically committed default case: true

46. 46 AutoCommit Connection.setAutoCommit(boolean) if AutoCommit is false, then every statement is added to an ongoing transaction you must explicitly commit or rollback the transaction using Connection.commit() and Connection.rollback()

47. 47 Connection Manager For a large threaded database server, create a Connection Manager object It is responsible for maintaining a certain number of open connections to the database When your applications need a connection, they ask for one from the CM’s pool Why? Because opening and closing connections takes a long time Warning: the CM should always setAutoCommit(false) when a connection is returned

48. 48 More info Go through the tutorial at: http://developer.java.sun.com/developer/onlineTr aining/Database/JDBC20Intro/ http://developer.java.sun.com/developer/onlineTr aining/Database/JDBC20Intro/ http://java.sun.com/products/jdbc/learning.html

49. 49 Optimizing JDBC

50. 50 Goal –Illustrate techniques for optimizing JDBC API- based calls from the Java platform –Design better JDBC implementations –Recognize potential performance bottlenecks

51. 51 Outline 1)Why optimize? 2)Basic API techniques. 3)Design Strategies. 4)Advanced Driver Tuning methods.

52. 52 Why Optimize? On average, a web request performs 4 database queries. Experience has shown that database calls are typical performance bottleneck. Bad JDBC can overwhelm the database.

53. 53 JDBC API SQL: “SELECT * FROM TABLE” java.sql.PreparedStatement java.sql.CallableStatement Cache data on client. Most Versatile Most Optimized

54. 54 JDBC API SQL Statements Most flexible Least reliable Must be recompiled in database for each use

55. 55 JDBC API PreparedStatement Represents a precompiled SQL statement Can be used to efficiently execute statement multiple times Somewhat flexible – can create new ones as needed

56. 56 JDBC API PreparedStatement pstmt = con.prepareStatement("UPDATE EMPLOYEES SET SALARY = ? WHERE ID = ?"); pstmt.setBigDecimal(1, 153833.00); pstmt.setInt(2, 110592); pstmt.execute();

57. 57 JDBC API java.sql.CallableStatement Used to execute SQL stored procedures. Same syntax as PreparedStatement. Least flexible. Most optimized DB call.

58. 58 JDBC API Cache Keep data within client to reduce the number of round-trips to the database. Lesson: The less JDBC the better.

59. 59 Basic Design Techniques Use Database Connection Pool Don’t use DriverManager.getConnection() often. JDBC connections can take 0.5 to 2 seconds to create. Create Pool of Connections and reuse them. Necessity for any production system.

60. 60 Basic Design Techniques Use multi-threading with Connection Pooling to address network latency: Threads can issue queries over separate database connections. This improves performance to a point.

61. 61 Basic Design Techniques Single-batch Transactions Collect set of operations and submit transaction in one statement: BEGIN TRANSACTION UPDATE TABLE1... INSERT INTO TABLE2… DELETE TABLE3 COMMIT

62. 62 Basic Design Techniques Single-batch Transactions DB obtains necessary locks on rows and tables, uses and releases them in one step Depending on transaction type, separate statements and commits can result in more DB calls and hold DB locks longer

63. 63 Basic Design Techniques Don’t have transaction span user input Application sends BEGIN TRAN and SQL, locking rows or tables for update Application waits for user to press key before committing transaction

64. 64 Basic Design Techniques Smart Queries Make queries as specific as possible Put more logic into SQL statements DB are designed to use SQL efficiently Proper use of SQL can avoid performance problems

65. 65 Basic Design Techniques Smart Query Ex: get employees in ENG dept Instead of: SELECT * FROM employees; SELECT * FROM dept; (and joining on Java application side) Use database join: SELECT employees.* FROM employees E, dept D WHERE E.DEPTNO = D.DEPTNO AND D.DEPTTYPE = ‘ENG’;

66. 66 Basic Design Techniques Smart Queries Minimize ResultSet before crossing network Many performance problems come from moving raw data around needlessly

67. 67 Basic Design Techniques Smart Query Guidelines Use DB for filtering Use Java for business logic DB does filtering very well DB business logic is poor –At least very inconsistent between database vendors.

68. 68 Basic Design Techniques Keep operational data set small as possible Move non-current data to other tables and do joins for rarer historical queries Otherwise, index and cluster so frequently used data is logically and physically localized

69. 69 Advanced Driver Tuning Special options for each JDBC driver No common standard Improve performance by reducing round trips to the database. Example: Oracle driver performance extensions

70. 70 Advanced Driver Tuning Oracle Performance Extensions 1)Row Prefetch 2)Batch Updates

71. 71 Advanced Driver Tuning 1. Row Prefetch Use client-side buffers Replace round trips by local manipulation of rows returned by query Use OracleStatement.setRowPrefetch()

72. 72 Advanced Driver Tuning 2. Batch Updates Reverse Prefetch Does for data going to DB what prefetch does for data coming from it OraclePreparedStatement.setExecuteBatch

73. 73 Advanced Driver Tuning 2. Batch Updates Standard JDBC makes a trip to DB for each PreparedStatement.executeUpdate Batching: When number of queued requests reaches batch size, sends them to DB

74. 74 Summary Optimization Stages 1)Leverage the strengths of the DB 2)Use the full range of java.sql API 3)Design for Performance – Connection Pools, Multi-Threading, etc. 4)Implement driver performance extensions