Database Application Development Programming With Databases Basic question: how do we send sql commands? How do we get the answers back and process them? Give example of extending tetrad with relational database.

Overview Concepts covered in this lecture: SQL in application code Embedded SQL Cursors Dynamic SQL Stored procedures

Example: Course Enrolment Enters request: add course, drop course Sends query: Course availability, student info,… User/Client Application Database Returns query result Checks constraints returns confirmation for display

Programming With SQL

Overview Static Queries: Query form known at compile time Dynamic Queries Execution in Application Space Embedded SQL SQLJ API: Dynamic SQL ODBC, JDBC Server Execution Stored Procedure SQL/PSM Could also have dynamic stored procedures but we won’t discuss it.

SQL statements as Part of a larger software system So far: interactive SQL interface, pure “SQL programs”. In practice often: queries are not ad-hoc, but programmed once and executed repeatedly. need the greater flexibility of a general-purpose programming language: for complex calculations graphic user interfaces.

Key Questions How do we send SQL commands to a database management system from within an application program? How do we get the answer back in a way that can be processed by the application program? Rather than extending a programming language with SQL capability, how about extending SQL with programming capabilities? Prepare tetrad demo for my own application. Need to find files for loading on system.

Database APIs Add library with database calls (API) Special standardized interface: procedures/objects Pass SQL strings from language, presents result sets in a language-friendly way Sun’s JDBC: Java API Supposedly DBMS-neutral a driver traps the calls and translates them into DBMS-specific code database can be across a network. Source code and executable is independent of DBMS. API: application programming interface. Embedded sql gets compiled for a specific database system. Source code is independent of database system; executable is not. Driver translates sql commands into dbms commands.

SQL in Application Code SQL commands can be called from within a host language (e.g., C++ or Java) program. SQL statements can refer to host variables (including special variables used to return status). Must include a statement to connect to the right database. Two main integration approaches: Embed SQL in the host language (Embedded SQL, SQLJ) Create special API to call SQL commands (JDBC, Visual Studio).

SQL in Application Code (Contd.) Impedance mismatch: SQL relations are (multi-) sets of records, with no a priori bound on the number of records. No such data structure exists traditionally in procedural programming languages such as C++ or Java. SQL supports a mechanism called a cursor to handle this.

Sending SQL Queries to a database Dynamic SQL Queries Sending SQL Queries to a database

Dynamic SQL Often, the concrete SQL statement is known not at compile time, but only at runtime. Example 1: a program prompts user for parameters of SQL query, reads the parameters and executes query. Example 2: a program prompts user for an SQL query, reads and executes it. Construction of SQL statements on-the-fly: PREPARE: parse and compile SQL command. EXECUTE: execute command.

Dynamic SQL: Example char c_sqlstring[]= {“DELETE FROM Sailors WHERE rating > 5”}; EXEC SQL PREPARE readytogo FROM :c_sqlstring; EXEC SQL EXECUTE readytogo; “readytogo” is an SQL variable. Add example for visual studio?

Connecting to Databases

Connection Flow Chart Load Driver Manager do this once Driver Manager create Connection create Statement show also in MysQl workbench gui execute repeat this part Query = string return Cursor, ResultSet

Python Database Connectivity See CSIL website for how to connect. Python has the built-in concept of a tuple, so fetching one row from a cursor returns a tuple! Python Database Tutorial http://www.sfu.ca/content/dam/sfu/computing/csil/windows/python_sample_pymssql_1.py.txt # # # For testing SQL Server connection in CSIL through pymssql connection. # # Author: Johnny Zhang # # Last modified @ 2014.03.03 # # # Note: you should run this program and all of your CSIL SQL programs on a CSIL system. # # Please modify this program before using. # # alternation includes: # # the CSIL SQL Server standard login (which is formatted as s_<username>, e.g. s_helpdesk) # the password for the CSIL SQL Server standard login # the name of your database (which is formatted as <username><course#>, e.g helpdesk999) # import pymssql conn = pymssql.connect(host='cypress.csil.sfu.ca', user='s_username', password='***', database='username###') # ^^^ these 3 values must be changed in your own program. # cur = conn.cursor() # to validate the connection, there is no need to change the following line cur.execute('SELECT username from dbo.helpdesk') row = cur.fetchone() while row: print "SQL Server standard login name= %s" % (row[0]) row = cur.fetchone() conn.close() # This program will output your CSIL SQL Server standard login, # If you see the output as s_<yourusername>, it means the connection is a success. # # You can now start working on your assignment. #

Python Example >>> conn = pymssql.connect(host='cypress.csil.sfu.ca',user='s_oschulte',password=’J****’, database='oschulte354') >>> mycursor = conn.cursor() >>> tablename = 'sailors’ >>> myquery = 'SELECT * from '+tablename /*just another Python string */ >>> print myquery SELECT * from sailors >>> mycursor.execute(myquery) >>> row = mycursor.fetchone() /*just another Python tuple */ >>> print row (22, u'Dustin', 7, 45.0) >>> while row: print "Sailor ID and Sailor Name are", row[0:2] row = mycursor.fetchone() Sailor ID and Sailor Name are (22, u'Dustin') Sailor ID and Sailor Name are (29, u'Brutus') ..... Sailor ID and Sailor Name are (32, u'Andy') >>> conn.close() u denotes unicode string see http://docs.python.org/2/howto/unicode.html

Visual Studio Example Visual Studio Connection Example see course website.

JDBC: Architecture Four architectural components: Application (initiates and terminates connections, submits SQL statements) Driver manager (load JDBC driver) Driver (connects to data source, transmits requests and returns/translates results and error codes) Data source (processes SQL statements)

JDBC Example Connection con = DriverManager.getConnection(url, ”login", ”pass"); // connection object created by Driver Manger Statement stmt = con.createStatement(); // set up statement for the connection String query = "SELECT name, rating FROM Sailors"; ResultSet rs = stmt.executeQuery(query); try { // handle exceptions // loop through result tuples while (rs.next()) { // while there is a next tuple String s = rs.getString(“name"); // get value of “name” in current tuple Int n = rs.getFloat(“rating"); // get value of “rating” in current tuple System.out.println(s + " " + n); } } catch(SQLException ex) { System.out.println(ex.getMessage () + ex.getSQLState () + ex.getErrorCode ()); Also use try for Driver Manager to handle connection problems. See 2 slides down. We’ll see how to prepare a statement later (prepareStatement rather than createStatement)

JDBC Driver Management All drivers are managed by the DriverManager class Loading a JDBC driver: In the Java code: Class.forName(“oracle/jdbc.driver.Oracledriver”); Class.forName(“com.mysql.jdbc.Driver”); When starting the Java application: -Djdbc.drivers=oracle/jdbc.driver Check CSIL documentation! https://www.sfu.ca/computing/about/school-facilities/csil/windows/how-to-use-sql.html#connect-to-sqlsever

Connections in JDBC We interact with a data source through sessions. Each connection identifies a logical session. JDBC URL: jdbc:<subprotocol>:<otherParameters> Example: String url=“jdbc:oracle:www.bookstore.com:3083”; Connection con; try{ con = DriverManager.getConnection(url,usedId,password); } catch SQLException except { …}

Connection Class Interface public boolean getReadOnly() and void setReadOnly(boolean b) Specifies whether transactions in this connection are read-only public boolean getAutoCommit() and void setAutoCommit(boolean b) If autocommit is set, then each SQL statement is considered its own transaction. Otherwise, a transaction is committed using commit(), or aborted using rollback(). public boolean isClosed() Checks whether connection is still open.

Processing Query Results Cursors Processing Query Results

Cursors Can declare a cursor on a relation or query statement (which generates a relation). Can open a cursor, and repeatedly fetch a tuple then move the cursor, until all tuples have been retrieved. Can use a special clause, called ORDER BY, to control the order in which tuples are returned. Fields in ORDER BY clause must also appear in SELECT clause. Can also modify/delete tuple pointed to by a cursor. Basic idea: cursor points at a row; can then fetch that row. Also called resultset in Java. 16

Terminology “Cursor” is the official SQL term. Many programming languages have their term for the same concept. Language Term SQL Standard Cursor Stored Procedures PSM Java ResultSet Visual Studio (C##, Visual Basic) DataReader

Cursor that gets names of sailors who’ve reserved a red boat, in alphabetical order EXEC SQL DECLARE sinfo CURSOR FOR SELECT S.sname FROM Sailors S, Boats B, Reserves R WHERE S.sid=R.sid AND R.bid=B.bid AND B.color=‘red’ ORDER BY S.sname Most implementations differ from SQL standard: Can use order by without cursor. Can have many expressions for order criteria. S.Sid doesn’t appear in select clause. Seems to be legal in SQL Server, see http://msdn.microsoft.com/en-us/library/ms188385.aspx. http://developer.postgresql.org/pgdocs/postgres/sql-select.html#SQL-ORDERBY Cursor allows us to deal with sets of rows (It’s a cursor for those rows).

Java ResultSets A ResultSet is a very powerful cursor: previous(): moves one row back absolute(int num): moves to the row with the specified number relative (int num): moves forward or backward first() and last() RecordSet, DataReader in Visual Basic Many other methods defined, e.g. number of columns, find column headers.

Call ResultSets PreparedStatement.executeUpdate only returns the number of affected records PreparedStatement.executeQuery returns data, encapsulated in a ResultSet object (a cursor) ResultSet rs=pstmt.executeQuery(sql); // rs is now a cursor While (rs.next()) { // process the data } PreparedStatement escapes parameter values for defense against code injection https://www.owasp.org/index.php/Preventing_SQL_Injection_in_Java#Defense_Strategy . String concatenation should never be used to create dynamic SQL. All queries should be parametrized.

Stored Procedures

Stored Procedures A stored procedure is a function / procedure written in a general-purpose programming language that is executed within the DBS. Performs computations that cannot be expressed in SQL. Procedure executed through a single SQL statement. Executed in the process space of the DB server. SQL standard: PSM (Persistent Stored Modules). Extends SQL by basic concepts of a general-purpose programming language.

Advantages of Stored Procedures Can encapsulate application logic while staying close to the data. Reuse of application logic by different users. Avoid tuple-at-a-time return of records through cursors. Provides data security (like a view). Give example of time difference table in Spain. Also look at Microsoft SQL server functionality. Mention my own research problems: example stored procedure.

Stored Procedures: Examples CREATE PROCEDURE ShowNumReservations SELECT S.sid, S.sname, COUNT(*) FROM Sailors S, Reserves R WHERE S.sid = R.sid GROUP BY S.sid, S.sname Stored procedures can have parameters: Three different modes: IN, OUT, INOUT CREATE PROCEDURE IncreaseRating( IN sailor_sid INTEGER, IN increase INTEGER) UPDATE Sailors SET rating = rating + increase WHERE sid = sailor_sid In variables: input read-only. Output variables: output only. Inout: can be input and have their values changed. SELECT S.sid, S.sname, COUNT(*)FROM Sailors S, Reserves R WHERE S.sid = R.sid GROUP BY S.sid, S.sname works in mysql

Stored Procedures: Examples (Contd.) Stored procedure does not have to be written in SQL: CREATE PROCEDURE TopSailors( IN num INTEGER) LANGUAGE JAVA EXTERNAL NAME “file:///c:/storedProcs/rank.jar” But rarely supported (e.g., not in MySQL but postgress yes) SQLServer supports .NET procedures http://technet.microsoft.com/en-us/library/ms187926.aspx

Main SQL/PSM Constructs (Contd.) Local variables (DECLARE) RETURN values for FUNCTION Assign variables with SET Branches and loops: IF (condition) THEN statements; ELSEIF (condition) statements; … ELSE statements; END IF; LOOP statements; END LOOP Queries can be parts of expressions Can use cursors without “EXEC SQL” Cursor use: open, fetch, close. Works in MySQL, not really in SQL Server. PSM = persistent stored module

SQL/PSM Most DBMSs allow users to write stored procedures in a simple, general-purpose language (close to SQL)  SQL/PSM standard is a representative Declare a stored procedure: CREATE PROCEDURE name(p1, p2, …, pn) local variable declarations procedure code; Declare a function: CREATE FUNCTION name (p1, …, pn) RETURNS sqlDataType local variable declarations function code;

Main SQL/PSM Constructs CREATE FUNCTION rate Sailor (IN sailorId INTEGER) RETURNS INTEGER DECLARE rating INTEGER DECLARE numRes INTEGER SET numRes = (SELECT COUNT(*) FROM Reserves R WHERE R.sid = sailorId) IF (numRes > 10) THEN rating =1; ELSE rating = 0; END IF; RETURN rating; Can have out or inout variables as well. Can have loops as well. Show demo in Visual Studio.

SQL Server Version CREATE FUNCTION rateSailor (@sailorId INT) RETURNS INT AS BEGIN DECLARE @numRes INT DECLARE @rating INT SET @numRes = (SELECT COUNT(*) FROM Reserves R WHERE R.sid = @sailorId) IF @numRes > 10 SET @rating = 1 ELSE SET @rating = 0 RETURN @rating END GO; SELECT dbo.rateSailor(22); go look under Programmability, Functions, Scalar-Valued.

MySQL Version DELIMITER $$ CREATE FUNCTION rateSailor(sailorId INT) RETURNS INT BEGIN DECLARE numRes INT; DECLARE rating INT; SET numRes = (SELECT COUNT(*) FROM Reserves R WHERE R.sid = sailorId); IF numRes > 10 THEN SET rating = 1; ELSE SET rating = 0; END IF; RETURN rating; END $$ select rateSailor(22);

Calling Stored Procedures EXEC SQL BEGIN DECLARE SECTION Int sid; Int rating; EXEC SQL END DECLARE SECTION // now increase the rating of this sailor EXEC SQL CALL IncreaseRating(:sid,:rating); Demo in visual basic

Calling Stored Procedures (Contd.) JDBC: CallableStatement cstmt= con.prepareCall(“{call ShowSailors}”); ResultSet rs = cstmt.executeQuery(); while (rs.next()) { … } Iterator = cursor. Show demo for calling stored procedures in Visual Studio.

Summary

SQL Connectivity Embedded SQL allows execution of parametrized static queries within a host language Dynamic SQL allows execution of completely ad-hoc queries within a host language Cursor mechanism allows retrieval of one record at a time and bridges impedance mismatch between host language and SQL APIs such as JDBC introduce a layer of abstraction between application and DBMS

Stored Procedures Stored procedures execute application logic directly at the server SQL/PSM standard for writing stored procedures