Stored Routines Tips, Tricks and Solutions Alex Nozdrin Software Developer, Server team Alexander.Nozdrin@mysql.com George Trujillo Senior Instructor George.Trujillo@mysql.com

Agenda Stored Objects Overview 5 min Stored Procedures vs Stored Functions Features of triggers User and stored routine variables SQL SECURITY INVOKER / DEFINER When and why? Technical Details 10 min CREATE-time context Storing stored objects Current limitations Non-documented features Dynamic SQL 10 min SQL Injections Dealing with PS limitations Real-life Examples 25 min

Stored Objects Overview Stored Procedures vs Stored Functions Features of triggers User and stored routine variables SQL SECURITY INVOKER / DEFINER When and why? Technical Details Dynamic SQL DBA Examples Q & A

Stored Objects Overview Definition Stored Routines Stored Procedures Stored Functions Triggers Events*

Stored Objects Overview Stored Procedures vs Stored Functions - does not return a value - can generate a result set, or even multiple result sets - that result set can not be used directly in expressions - can not be used in expressions - is invoked by a CALL statement - supports IN, OUT and IN/OUT parameters - Stored function - returns a single value - which can be used in an expression I.e. a stored function is used the same way as a constant, a built-in function or a reference to a table. - can not generate a result set - can not be invoked by a CALL statement - supports only IN parameters

Stored Objects Overview Triggers Activation Time: BEFORE, AFTER Events: INSERT, UPDATE, DELETE Special variables: NEW, OLD FOR EACH ROW only One trigger / table for {activation time, event} No result set No CALL, which leads to generating result set Use OUT-parameters to pass data from stored procedures Can not begin or end transactions Explicitly or implicitly No DDL No temporary tables or views No prepared statements

Stored Objects Overview User and Stored Routine Variables User variable – an item in expression Store routine variable – a column in a temporary table

Stored Objects Overview SQL SECURITY: Definition sql security DEFINER SUID object Executes under the “definer” – the user who created the object sql security INVOKER DEFINER clause DEFINER = username@hostname Requires SUPER privilege Specifies the user which privileges will be used when executing the object A stored routine runs either with the privileges of the user who created it or the user who invoked it. The choice of which set of privileges to use id controlled by the value of the SQL SECURITY attribute. - A value of DEFINER causes the routine to have the privileges of the user who created it. - A value of INVOKER causes the routine to run with the privileges of its invoking user. This means the routine has access to database objects only if that user can already access them otherwise. SQL SECURITY DEFINER allows you to create routines that access information on behalf of users who otherwise would not be able to do so.

Stored Objects Overview SQL SECURITY: Example

Stored Objects Overview Stored Routines: Why? Security No need to grant privileges on the tables Code “security” – application developer can not easily modify SQL code Performance Cached on the server Can be designed to reduce network traffic Consistency No “copy & paste” Separation of interface and implementation Maintenance Implementation can change No need to change clients during upgrade

Stored Objects Overview Triggers: Why? Log Audit Change data of an operation (INSERT, UPDATE) Prevent some operations CREATE PROCEDURE throw_error() ROLLBACK; -- Any stmt causing COMMIT or ROLLBACK CREATE TABLE t1(c INT) ENGINE=InnoDB; CREATE TRIGGER t1_bi BEFORE INSERT ON t1 FOR EACH ROW BEGIN IF NEW.c = 2 THEN CALL throw_error(); END IF; END INSERT INTO t1 VALUES (10), (11), (12); -- Ok INSERT INTO t1 VALUES (1), (2), (3); -- Rollback

Stored Objects Overview What is BAD? Processing close to data – scale-up MySQL's way – scale-out Current limitations

Technical Details Stored Objects Technical Details CREATE-time context Storing stored objects SHOW CREATE and INFORMATION_SCHEMA Current limitations SHOW CODE SHOW PROFILE Dynamic SQL Real-life Examples Q & A

Technical Details CREATE-time context: Execution Overview

Technical Details CREATE-time context CREATE-time Context – a list of attributes that are fixed at CREATE time sql_mode The “main switch” of MySQL behavior. character_set_client Defines character set of the CREATE-statement. collation_connection Defines character set / collation rules for the execution. collation_database Owner database collation current database time_zone (events)

Technical Details Storing stored objects Stored routines – mysql.proc table Triggers – TRG, TRN files mysql – a system database Stored objects are stored in the original form Original character set Character set introducers untouched ... CREATE PROCEDURE p1() SELECT 'text1', _cp1257 'text2';

Technical Details SHOW CREATE vs INFORMATION_SCHEMA Returns original query in the original character set Query can be “binary” (consists of several charsets) CREATE PROCEDURE p1() SELECT 'text1', _cp1257 'text2'; INFORMATION_SCHEMA Returns “normalized” query in UTF8 “Normalized” == no character set introducers SELECT 'text1-in-utf8', 'text2-in- utf8'; Confused? Use SHOW CREATE for Dump / backup – mysqldump uses SHOW CREATE Use INFORMATION_SCHEMA for Reporting

Technical Details Limitations Common Prepared Statements Stored Routines

Technical Details Limitations: Common Meta-data is fixed at the compilation time i.e. Don't change meta-data on the fly i.e. NO ALTER BAD Examples: Prepare stmt & ALTER meta-data Execute a SR & ALTER meta-data Create a trigger & ALTER meta-data Why? The server does not track dependencies on the objects in cache (compiled objects)

Technical Details Limitations: Prepared Statements No support for OUT-parameters No query plan cache No arrays Placeholders are not supported everywhere Table names Column names Dynamic search condition

Technical Details Limitations: Stored Routines Cache Per connection No limit Almost any DDL flushes all caches No COLLATE for stored routine variable No SIGNAL / RESIGNAL SR-variable shadows column Prepared statements PREPARE uses only user variables (no SR vars) EXECUTE uses only user variables (no SR vars, no constants)

Technical Details Non-documented features SHOW PROCEDURE | FUNCTION CODE SHOW PROFILE (5.0, Community patch) CREATE PROCEDURE p1(x INT) BEGIN IF x < 0 THEN INSERT INTO TError VALUES (CONCAT('Negative value: ', x)); ELSEIF x > 100 THEN INSERT INTO TError VALUES (CONCAT('Too large value: ', x)); ELSE INSERT INTO Tvalues VALUES (x); END IF; END SHOW PROCEDURE CODE p1; +-----+---------------------------------------------------+ | Pos | Instruction | | 0 | jump_if_not 3(7) (x@0 < 0) | | 1 | stmt 5 "INSERT INTO TError VALUES (CONCAT('..." | | 2 | jump 7 | | 3 | jump_if_not 6(7) (x@0 > 100) | | 4 | stmt 5 "INSERT INTO TError VALUES (CONCAT('..." | | 5 | jump 7 | | 6 | stmt 5 "INSERT INTO Tvalues VALUES (x)" |

Dynamic SQL Stored Objects Technical Details Dynamic SQL SQL Injections Dealing with PS limitations Dynamic table / column names Dynamic WHERE Arrays Real-life Examples Q & A

Dynamic SQL Kinds of Dynamic SQL Dynamic queries Pros Good for prototyping Universal approach The only way to do some things Cons Cluttering the client code, copy & paste Query has to be parsed each time SQL Injections Prepared statements Pros A query is parsed only once No SQL injections Cons Extra job if query is executed only once Not all SQL statements supported Arrays are not supported Placeholders can not be used everywhere HINT: close PS after use

Dynamic SQL SQL Injections (1) Dynamic query my $name = ...; # read from HTML form my $s = “SELECT id FROM t1 WHERE name = '$name'”; mysql_query($mysql, $s); Arbitrary SQL query can be executed $name: “'; DELETE FROM t1 WHERE 'a' = 'a” $s: SELECT id FROM t1 WHERE name = ''; DELETE FROM t1 WHERE 'a' = 'a'; SQL query can be compromised $name: “' OR 'a' = 'a” SELECT id FROM t1 WHERE name = '' OR 'a' = 'a'

Dynamic SQL SQL Injections (2) Prepared statements Stored procedures – just for convenience here CREATE PROCEDURE check_user(name VARCHAR(255)) PREPARE stmt FROM 'SELECT id FROM t1 WHERE name = ?'; EXECUTE stmt USING name; CALL check_user("valid_user_name"); --> id --> 1 CALL check_user("'; DELETE * FROM t1 WHERE 'a' = 'a"); --> Nothing CALL check_user("' OR 'a' = 'a");

Dynamic SQL SQL Injections (3) C API (other instruments): mysql_real_escape_string() Quotes CR / LF NUL Slash / backslash mysql_real_query() One query at a time

Dynamic SQL Dynamic Table / Column names Avoid that Tables are types Columns are attributes Workaround Use prepared statement to pass parameters Check column or table names

Dynamic SQL Dynamic table name: Example create function check_table_name(table_name char(10)) returns bool return strcmp(lcase(table_name), 't1') = 0 or strcmp(lcase(table_name), 't2') = 0; instr(table_name, "'") = 0 and ... instr(table_name, '(') = 0; create procedure p1(table_name char(10), value char(255)) l1: begin if check_table_name(table_name) != 1 then leave l1; end if; SET @s = CONCAT('SELECT name FROM ', table_name, ' WHERE id = ?'); SET @id_var = id; PREPARE s1 FROM @s; EXECUTE s1 USING @id_var; end l1

Dynamic SQL Dynamic WHERE SELECT * FROM table_name WHERE ? Must not be used in a general case Can be replaced by a static SQL in common cases

Dynamic SQL Dynamic WHERE: Example CREATE TABLE t1(c1 INT, c2 INT, c3 TEXT); INSERT INTO t1 VALUES (1, 1, 'hello'), (1, 2, 'world'), (2, 1, 'foo'); CREATE PROCEDURE p1(v1 INT, v2 INT, v3 VARCHAR(255)) BEGIN SELECT c1, c2, c3 FROM t1 WHERE (v1 IS NULL OR c1 = v1) AND (v2 IS NULL OR c2 = v2) AND (v3 IS NULL OR STRCMP(c3, v3) = 0); END CALL p1(NULL, NULL, NULL); -- 3 rows CALL p1(1, NULL, NULL); -- 2 rows CALL p1(NULL, 1, 'foo'); -- 1 rows

Dynamic SQL Arrays The problem SELECT * FROM table WHERE id IN (?) Solution CSV-list: “123; 456.789;10;hello world; another list item” Proper escaping (delimiters) Extra spaces Malicious (illegal) input Empty items Slow Legacy or imported data Fixed-width list: “ red blue green whiteorange” (6 symbols) More memory Not always possible Usually faster than CSV-list Temporary table Probably, the best

Dynamic SQL MySQL Forge http://forge.mysql.com Split a Delimited String in SQL MySQL General Purpose Stored Routines Library Arrays and basic data structures "FOR EACH" loops Named parameters Syntax helpers

Real-life Examples Senior Instructor George.Trujillo@mysql.com

Questions ?

Thanks! Alexander.Nozdrin@mysql.com George.Trujillo@mysql.com