Oracle 10g & 11g for Dev Virtual Columns DML error logging SQL Pivot/Unpivot regular expressions(REGEXP_*) New Analytic Functions

Introducing Virtual Columns Beginning with Oracle 11g tables may now include virtual columns (dynamic values; not stored) Virtual columns obtain their value by evaluating an expression that might use: Columns from the same table Constants Function calls (user-defined functions or SQL functions) Virtual columns might be used to: Eliminate some views Control table partitioning (DBA stuff) Manage the new "binary" XMLType data Virtual columns may be indexed! Oracle has supported stored expressions for many years, in views and function-based indexes. Most commonly, views enable us to store and modularise computations and expressions based on their underlying tables' columns. In more recent versions (since around the 8i timeframe), we have been able to index expressions using function-based indexes. Now, with the release of 11g, Oracle enables us to store expressions directly in the base tables themselves as virtual columns. Ref http://www.oracle-developer.net/display.php?id=510

DML error logging Major new feature for for bulk SQL operations : DML error logging enables us to trap "bad data" and filter it to a log table without failing our overall DML statement Insert, Update, Delete, and Merge add ERROR logging allowing you to capture DML errors and log them INSERT …/* or UPDATE, DELETE, MERGE */ LOG ERRORS [ INTO [schema.] table ] [ (simple_expression) ] [ REJECT LIMIT { integer | UNLIMITED } –Default error table defined by DBMS_ERRLOG package: ERR$_ followed by first 25 characters of DML target table –Simple expression is value to be used as statement tag (may be result of SQL function call) –Reject limit default is zero Ref http://www.oracle-developer.net/display.php?id=329 http://www.oracle-developer.net/display.php?id=330 a major new feature of Oracle 10g Release 2 for bulk SQL operations. DML error logging enables us to trap "bad data" and filter it to a log table without failing our overall DML statement. This has never been possible in SQL before, although we could use complex constraint management and application code to achieve a slightly similar end-result. DML error logging is more similar in concept to the FORALL SAVE EXCEPTIONS construct in PL/SQL (new in Oracle 9i).

DML error logging Oracle provides a PL/SQL packaged procedure to create the logging table (for each table to be logged) execute DBMS_ERRLOG.CREATE_ERROR_LOG('myemp', 'myemplog'); myempTable DML is being applied to myemplogLogging table for rejected rows Output example: insert into emp select * from myempbig log errors into myemplog('Log test3') reject limit unlimited; 0 rows created. Error log table containing: ORA_ERR_NUMBER$Error number ORA_ERR_MESG$Error message ORA_ERR_ROWID$ Rowidof impacted rows ORA_ERR_OPTYP$ Operation type (I,U,D,M) ORA_ERR_TAG$ Text from LOG_ERRORS All column values (good & bad) as varchar2(4000) Ref http://www.oracle-developer.net/display.php?id=329 http://www.oracle-developer.net/display.php?id=330

Creating Virtual Column CREATE TABLE NEWEMP (EMPNO NUMBER(4) NOT NULL, ENAME VARCHAR2(10), JOB VARCHAR2(9), MGR NUMBER(4), HIREDATE DATE, SAL NUMBER(7, 2), COMM NUMBER(7, 2), INCOME NUMBER(9,2) GENERATED ALWAYS AS (NVL("SAL",0)+NVL("COMM",0)) VIRTUAL, DEPTNO NUMBER(2)); Datatype defaults if not specified (based upon expression) Expression result appears as data in table but is “generated always” (whether or not specified in table definition) “ VIRTUAL” is not required, but adds clarity Oracle 11g also allows specification of Virtual Columns via “ALTER TABLE” alter table myemp add (totpay as (nvl(sal,0)+nvl(comm,0)));

SELECT PIVOT/UNPIVOT Pivot queries involve transposing rows into columns (pivot) or columns into rows (unpivot) to generate results in crosstab format. Adding a PIVOT clause to a SELECT allows rotation of rows into columns while performing aggregation to create cross-tabulation queries The PIVOT clause: Computes aggregations (implicit GROUP BY of all columns not in PIVOT clause) Output of all implicit grouping columns followed by new columns generated by PIVOT UNPIVOT performs the same activity but converts columns into ROWS (does not “undo” PIVOT) Clever developers have used PL/SQL and/or CASE to achieve PIVOT results before, but now it is part of Oracle's standard SQL http://www.oracle-developer.net/display.php?id=506 For more information on the new PIVOT and UNPIVOT syntax, see the SQL Reference documentation. For some more examples of the use of pivot and unpivot queries, see the Data Warehousing Guide here and here. SQL Reference : http://download.oracle.com/docs/cd/B28359_01/server.111/b28286/statements_10002.htm#CHDCEJJE pivot document : http://download.oracle.com/docs/cd/B28359_01/server.111/b28313/analysis.htm#DWHSG0209 Unpivot document: http://download.oracle.com/docs/cd/B28359_01/server.111/b28313/analysis.htm#sthref1179

PIVOT Example select * from (select job,deptno,income from newemp) query1 pivot (avg(income) for deptno in (10 AS ACCOUNTING, 20 AS RESEARCH, 30 AS SALES)) order by job; Job ACCOUNTING RESEARCH SALES ----------------------------------------------- ANALYST 30000 CLERK 13000 9500 9500 MANAGER 24500 29750 28500 PRESIDENT 50000 SALESMAN 19500

UNPIVOT Example select * from pivot_emp_table unpivot include nulls (avgpay for dept in (ACCOUNTING,RESEARCH,SALES)) order by job; JOB DEPT AVGPAY -------------------------------------------- ANALYST ACCOUNTING ANALYST RESEARCH 30000 ANALYST SALES /*** more rows ***/ SALESMAN ACCOUNTING SALESMAN RESEARCH SALESMAN SALES 19500

regular expressions REGEXP_LIKE (10g) REGEXP_INSTR (10g) function return TRUE when it matches patterns using standard (or POSIX) notation. REGEXP_INSTR (10g) returns the position of a string that matches a given pattern. REGEXP_SUBSTR (10g) returns the part of the string that matches the pattern only. REGEXP_REPLACE (10g) this function searches for a specified number of occurrences (default all) of a pattern and replaces it with a given string or NULL. REGEXP_COUNT (11g) counts the number of times a pattern occurs in a source string Until 11g, Oracle's regular expression support comprised four functions (REGEXP_LIKE, REGEXP_SUBSTR, REGEXP_INSTR and REGEXP_REPLACE), but with the new version, Oracle has added a fifth, REGEXP_COUNT. In addition, 11g includes some usability enhancements to two of the existing functions. so for further reading, see the SQL Functions http://download.oracle.com/docs/cd/B14117_01/server.101/b10759/functions001.htm#i88893 section of the SQL Reference for any of the REGEXP_* functions. Also see Appendix C http://download.oracle.com/docs/cd/B14117_01/server.101/b10759/ap_posix.htm#g693775 of in the SQL Reference that describes all of the various meta-characters, POSIX character classes and notation covered in this article (and more…). Reference http://www.oracle-developer.net/display.php?id=302

REGEXP_COUNT REGEXP_COUNT counts the number of times a pattern occurs in a source string select ename,regexp_count(ename,'l',1,'i') from emp; ----------------- SMITH 0 ALLEN 2 BLAKE 1 /** more rows ***/ MILLER 2 string expression and/or column to match pattern Regular Expression pattern Beginning position in the source string (default=1) Match parameters (i = case insensitive, c = case sensitive, m = multiple line source delimited by ‘^’ or ‘$’, n = matches ‘.’ newline characters (default no), and x = ignore whitespace characters (default is to match)

New Analytics (11gR2) Oracle 11gR2 has improved upon the already-impressive analytic functions first introduced in Oracle 8i adding: LISTAGG NTH_VALUE

LISTAGG (11gR2) LISTAGG provides lists of lower-level columns after aggregation select department_id, listagg(last_name, ', ') within group (order by last_name) dept_employees from hr.employees where department_id in (20,30) group by department_id order by department_id; DEPARTMENT_ID DEPT_EMPLOYEES ------------- --------------------------- 20 Fay, Hartstein 30 Baida, Colmenares, Himuro, Khoo Ref http://www.oracle-developer.net/display.php?id=515

NTH_VALUE (11gR2) NTH_VALUE simplifies the process of retrieving the “n-th” value select distinct department_id ,first_value(salary) ignore nulls over (partition by department_id order by salary desc) "1st" ,nth_value(salary,2) ignore nulls over (partition by department_id order by salary desc) "2nd" ,nth_value(salary,3) ignore nulls over (partition by department_id order by salary desc) "3rd" from hr.employees where department_id = 80 order by department_id, "1st", "2nd", "3rd"; DEPARTMENT_ID 1ST 2nd 3rd ------------- ---------- ---------- ---------- 80 14000 13500 12000 80 14000 13500 80 14000