8 Copyright © 2005, Oracle. All rights reserved. Parallel Operations in Data Warehouses

8-2 Copyright © 2005, Oracle. All rights reserved. Objectives After completing this lesson, you should be able to do the following: Determine when partitionwise joins are used Use the ALTER SESSION command to force parallel operations Create indexes in parallel Set initialization parameters relating to parallel operations tuning Query dynamic performance views related to parallel operations tuning Tune PDML operations

8-3 Copyright © 2005, Oracle. All rights reserved. Parallel Query The various query types that can be parallelized are: Access methods: –Table scans, fast full index scans –Partitioned index range scans –Various SQL operations: Sorts, aggregations, and set operations Join methods: –Nested loop, sort merge –Hash, star transformation, partitionwise join

8-4 Copyright © 2005, Oracle. All rights reserved. Parallel Partitioned Table Scan Scan by ROWID for partitioned tables. SELECT /*+ PARALLEL(SALES,9)*/ * FROM SALES; SALES PQ1 PQ2 PQ3 PQ4PQ5 PQ6 PQ7 PQ8 PQ9

8-5 Copyright © 2005, Oracle. All rights reserved. Parallel Partitioned Index Scan Scans can be performed by partition for partitioned indexes and tables. The hint must contain the table name, index name, and degree of parallelism. SELECT /*+ PARALLEL_INDEX(c,ic,3)*/ * FROM customers c WHERE cust_city = 'MARSEILLE'; Nonprefixed index IC Indexed column cust_city PQ1 PQ2 PQ3

8-6 Copyright © 2005, Oracle. All rights reserved. Partitionwise Joins Query slavePartition Partitioned table Partial partitionwise join QS Full partitionwise join QS

8-7 Copyright © 2005, Oracle. All rights reserved. Parallel DDL The parallel DDL statements for nonpartitioned tables and indexes are: – CREATE INDEX – CREATE TABLE... AS SELECT – ALTER INDEX... REBUILD The parallel DDL statements for partitioned tables and indexes are: – CREATE INDEX – CREATE TABLE... AS SELECT – ALTER TABLE... MOVE PARTITION – ALTER TABLE... SPLIT PARTITION – ALTER TABLE... COALESCE PARTITION – ALTER INDEX... REBUILD PARTITION – ALTER INDEX... SPLIT PARTITION

8-8 Copyright © 2005, Oracle. All rights reserved. Creating Indexes in Parallel Object Parallel execution server Nonpartitioned index Index piece Data

8-9 Copyright © 2005, Oracle. All rights reserved. Creating Indexes in Parallel Global partitioned index Object Parallel execution server Index piece Data

8-10 Copyright © 2005, Oracle. All rights reserved. Creating Indexes in Parallel Local partitioned index Object Parallel execution server Index piece Data

8-11 Copyright © 2005, Oracle. All rights reserved. Parallel DDL: Example DDL statements are parallelized by specifying a PARALLEL clause: CREATE BITMAP INDEX fk_sales_prod ON sales(prod_id) PARALLEL 16 LOCAL NOLOGGING;

8-12 Copyright © 2005, Oracle. All rights reserved. Parallel DML: Overview Complement parallel query architecture by providing parallelization for: – INSERT – UPDATE – DELETE – MERGE Useful when changing big tables

8-13 Copyright © 2005, Oracle. All rights reserved. Performance Benefits of Parallel DML Can dramatically speed up DML transactions that can be parallelized Serial update: Parallel update: … … … … UPDATE sales SET amount_sold=amount_sold* ; alter session enable parallel DML; UPDATE /*+PARALLEL(sales,12)*/ sales SET amount_sold=amount_sold* ;

8-14 Copyright © 2005, Oracle. All rights reserved. Enabling Parallel DML You must enable PDML. The ALTER SESSION statement enables parallel DML mode: Parallel queries are still parallelized, even if parallel DML is disabled. The default mode of a session is DISABLE PARALLEL DML. ALTER SESSION DISABLE PARALLEL DML ENABLE ALTER SESSION PARALLEL DML FORCE PARALLEL n

8-15 Copyright © 2005, Oracle. All rights reserved. Parallel DML: Example MERGE /*+ PARALLEL(c,3) PARALLEL(d,3) */ INTO customers c USING diff_customers d ON (d.cust_id = c.cust_id) WHEN MATCHED THEN UPDATE SET c.cust_last_name = d.cust_last_name, c.cust_city = d.cust_city WHEN NOT MATCHED THEN INSERT (c.cust_id,c.cust_last_name) VALUES (d.cust_id,d.cust_last_name);

8-16 Copyright © 2005, Oracle. All rights reserved. When to Use Parallel DML Scenarios where parallel DML is used include: Refreshing large tables in a data warehouse Creating intermediate summary tables Using scoring tables Updating historical tables Running batch jobs

8-17 Copyright © 2005, Oracle. All rights reserved.

8-18 Copyright © 2005, Oracle. All rights reserved. Restrictions on Parallel DML After a PDML statement modifies an object, it is no longer possible to query or modify this object in the same transaction ( ORA ). Limited integrity constraint support is provided. Clustered tables are not supported. PDML is not allowed: –On tables with enabled triggers –On remote objects –When the operation is part of a distributed transaction

8-19 Copyright © 2005, Oracle. All rights reserved. Tuning Parameters for Parallel Execution The initial computed values of parallel execution parameters should be acceptable in most cases. They are based on the values of CPU_COUNT and PARALLEL_THREADS_PER_CPU at database startup. Oracle Corporation recommends that you use the default settings. Manual tuning of parallel execution is more complex than using default settings. –Manual parallel execution tuning requires more attentive administration than automated tuning. –Manual tuning is prone to user-load and system resource miscalculations.

8-20 Copyright © 2005, Oracle. All rights reserved. Using Default Parameter Settings Increasing 4K to 8K improves parallel performance if SGA is large enough 2 KB (port specific) PARALLEL_EXECUTION_ MESSAGE_SIZE Maximizes the number of processes used by parallel execution CPU_COUNT x PARALLEL_THREADS_ PER_CPU x (1; 2 if PGA_AGGREGATE_TARGET > 0) x 5 PARALLEL_MAX_SERVERS Throttles DOP requests to prevent system overload TRUEPARALLEL_ADAPTIVE_ MULTI_USER CommentsDefaultsParameter

8-21 Copyright © 2005, Oracle. All rights reserved. Balancing the Workload To optimize performance, all parallel execution servers should have equal workloads. For parallelization by block range or parallel execution servers, the workload is dynamically divided among the parallel execution servers. By choosing an appropriate DOP, you can: –Minimize workload skew –Optimize performance

8-22 Copyright © 2005, Oracle. All rights reserved.

8-23 Copyright © 2005, Oracle. All rights reserved. Adaptive Multiuser and DOP The adaptive multiuser feature adjusts the DOP on the basis of user load. PARALLEL_ADAPTIVE_MULTI_USER set to: – TRUE improves performance in a multiuser environment (default) – FALSE is used for batch processing PARALLEL_AUTOMATIC_TUNING has been deprecated in Oracle Database 10g. –Kept for backward compatibility only

8-24 Copyright © 2005, Oracle. All rights reserved. Resource Manager and the DOP Resource plan: NIGHT_PLAN Resource consumer group OLTP DSS Allocation method parameters CPU = 25% Max Degree of parallelism = 2 CPU = 75% Max Degree of parallelism = 20

8-25 Copyright © 2005, Oracle. All rights reserved.

8-26 Copyright © 2005, Oracle. All rights reserved. Are There Execution Problems? Use V$PQ_TQSTAT to find out whether there is data distribution skew or bad object statistics: –Contains traffic statistics between slaves at table queue level –Valid only when queried from the parallel session Check I/O and CPU bound at the operating system level and decide on the parallelism. Check for contention.

8-27 Copyright © 2005, Oracle. All rights reserved. Data Distribution and V$PQ_TQSTAT SELECT /*+PARALLEL*/ cust_city, sum(amount_sold) FROM sales s, customers c WHERE s.cust_id=c.cust_id GROUP BY cust_city;... PX SEND QC (RANDOM) :TQ10002 P->S QC (RAND) HASH GROUP BY PCWP PX RECEIVE PCWP PX SEND HASH :TQ10001 P->P HASH HASH GROUP BY PCWP HASH JOIN PCWP PX RECEIVE PCWP PX SEND BROADCAST :TQ10000 P->P BROADCAST PX BLOCK ITERATOR PCWC TABLE ACCESS FULL CUSTOMERS PCWP PX BLOCK ITERATOR PCWC TABLE ACCESS FULL SALES PCWP

8-28 Copyright © 2005, Oracle. All rights reserved. Data Distribution and V$PQ_TQSTAT DFO_NUMBER TQ_ID SERVER_TYP PROCESS NUM_ROWS Consumer P Consumer P Producer P Producer P Consumer P Consumer P Producer P Producer P Consumer QC Consumer P Producer P

8-29 Copyright © 2005, Oracle. All rights reserved. Using Other Dynamic Performance Views General information: – V$FILESTAT – V$SESSTAT, V$SYSSTAT Information about parallel execution: – V$PX_SESSION – V$PX_PROCESS – V$PX_PROCESS_SYSSTAT – V$PX_SESSTAT – V$PQ_SESSTAT – V$PX_BUFFER_ADVICE TIMED_STATISTICS should be set to TRUE.

8-30 Copyright © 2005, Oracle. All rights reserved.

8-31 Copyright © 2005, Oracle. All rights reserved. Using V$PX_SESSION QCSID SID Group Set Degree ReqDegree SELECT qcsid, sid, server_group "Group", server_set "Set", degree "Degree", req_degree "ReqDegree" FROM V$PX_SESSION ORDER BY 1,3,4;

8-32 Copyright © 2005, Oracle. All rights reserved. Using V$PX_SESSTAT QCSID SID Group Set StatName VALUE physical reads physical reads physical reads physical reads physical reads 2 SELECT qcsid, sid, server_group "Group", server_set "Set", name "StatName", VALUE FROM V$PX_SESSTAT A, V$STATNAME B WHERE A.statistic# = B.statistic# AND name = 'physical reads' AND value > 0 ORDER BY 1,3,4;

8-33 Copyright © 2005, Oracle. All rights reserved. Using V$PX_PROCESS SERV STATUS PID SPID SID SERIAL P002 IN USE P003 IN USE P004 AVAILABLE P005 AVAILABLE P000 IN USE P001 IN USE SELECT * FROM V$PX_PROCESS;

8-34 Copyright © 2005, Oracle. All rights reserved. Using V$SYSSTAT SELECT name, value FROM V$SYSSTAT WHERE UPPER(name) LIKE '%PARALLEL OPERATIONS%' OR UPPER(name) LIKE '%PARALLELIZED%' OR UPPER(name) LIKE '%PX%';

8-35 Copyright © 2005, Oracle. All rights reserved. Using V$SYSSTAT NAME VALUE queries parallelized 347 DML statements parallelized 0 DDL statements parallelized 0 DFO trees parallelized 463 Parallel operations not downgraded 28 Parallel operations downgraded to serial 31 Parallel operations downgraded 75 to 99 pct 252 Parallel operations downgraded 50 to 75 pct 128 Parallel operations downgraded 25 to 50 pct 43 Parallel operations downgraded 1 to 25 pct 12 PX local messages sent PX local messages recv'd PX remote messages sent 0 PX remote messages recv'd 0

8-36 Copyright © 2005, Oracle. All rights reserved. Tuning PDML Using local striping for local indexes Using global striping for global indexes Increasing INITRANS for global indexes Using NOLOGGING Using multiple archivers Using multiple DBWRs or I/O slaves

8-37 Copyright © 2005, Oracle. All rights reserved. Summary In this lesson, you should have learned how to: Determine when partitionwise joins are used Use the ALTER SESSION command to force parallel operations Create indexes in parallel Set initialization parameters relating to parallel operations tuning Query dynamic performance views related to parallel operations tuning Tune PDML operations

8-38 Copyright © 2005, Oracle. All rights reserved. Practice 8: Overview This practice covers the following topics: Forcing parallelization of queries Interpreting a partial parallel partitionwise join execution plan Forcing parallelization of DMLs Verifying the requested DOP for a parallel DDL operation