1. Michael R. Messina, Management Consultant
Partitioning, More Cost Savings, Better Performance, Better Management in 11g
(30 sec.)
Michael R. Messina, Management Consultant
Rolta-TUSC, Oracle Open World 2010 (60 min)

2. Introduction Michael Messina Management Consultant with Rolta-TUSC
Background includes Performance Tuning, High Availability and Disaster Recovery
Using Oracle for approximately 16 years
(30 sec)

3. Audience Experience
How many have utilized Partitioning and what have been some of your experiences?
How many have utilized Table Compression and index compression (prior to 11g Advanced Compression) and what are your thoughts/experiences?
How many are using Advanced Compression in Production? Thoughts?
(2 min.)

4. Goals Touch on industry challenges Explosive Data Growth Performance
Costs
Look at 2 Partitioning Features Introduced with 11g
Examine 11g Advanced Compression and impact using 11g partitioning features
Show how Partitioning and Advanced Compression together help address some of the industry challenges
(30 sec)

5. Industry Challenges Exploding Data Growth Performance
Got to keep up
Performance
Query Performance Degradation as data volumes increase
Backup time increases as data volumes increase
Costs - What are the True Costs?
Disk Space Purchase / Backup / Space Management / Power / Cooling
What can we do??
(30 sec)
Organizations are seeing explosive data growth and IT groups have to keep up
Organizations are seeing the performance decline as the data volumes increase and backup times increase making it more difficult to manage the backup to protect the larger volumes of data.
Cost can be significant, cost of additional storage is more then the cost of the disk purchase, so what are our true costs?
We need to address this we need to do something, What can we do about it? What options to we have?

6. Exploding Data Growth
If you think storing data is a challenge now, it's nothing compared to what it could be in just a few years. Data Growth of 60% is common.
(1 1/2 min.)
IDC and EMC have said that data requirements are growing at an annual rate of 60 percent, InternetNews.com
A typical database environment has a least 1 pre-production environment usually a copy of production, this environment will experience the same growth as the production environment. Therefore we will see 2GB of disk growth for every additional GB in the production system making the cost impact even larger.
If we use a typical 100GB database today and factor a 60% yearly growth we can see in year 5 that database is approaching 700GB. If we also consider that the typical organization also has a development and test database for each production environment with test being the size of production for load testing, etc. We are not experiencing a 500+ GB growth in our enterprise storage but GB growth. Imagine having multiple environments experiencing this type of growth with some environments today starting larger then the 100GB we used as an example here.
Are your experiences with storage cost and storage issues similar to that of your peers, are you seeing large data growth? Is it at around 60%?

7. Performance
“Storage capacity grows at over 60% per year while performance improves at less than 10% per year. This trend has existed for over 10 years and is expected to continue for the foreseeable future. “, BNET
(30 sec.)
The data growth trends are staggering. The growth IT shops are seeing around 60% a year, however technology is only improving performance at less then 10% per year. This means that the trend of declining performance as our data grows will continue for the foreseeable future.
This indicates a real challenge for some of our database environments and has a dba’s hair on end or has them pulling their hair out as the data volumes increase and performance is not keeping up in some our environments creates risk of severe performance issues to address impacting the businesses which rely on the data in the database. This means that going forward to handle our explosive data growth and maintain the performance level our users expect we as database administrators must find ways to handle the data growth,
How many have experienced this trend in their environment; where as the data volume increases performance degrades and performance and tuning starts to consume more and more time and does not yield needed results as data continues to grow?
Ask yourselves how is this impacting your business functions and demands on your DBAs time.

8. Releasing your Database Performance
All of these trends of database growth and other pressures constrain database performance almost like are database is being held in prison and constrained.
How can we release your database from it’s prison?

9. Performance Ref Partitioning what is advertised
Introduced with Oracle 11g
Improves performance for parent child relationships
Partitions the child with the parent
Interval Partitioning what is advertised
Introduced with 11g
Same performance Benefits as Range partitioning
(30 sec)
Oracle says that the Ref partitioning and Interval partitioning introduced with 11g offer some performance benefits. Ref Partitioning is advertised to improve performance of queries that access a parent child relationship. While interval partitioning is to offer the same performance improvements that ranger partitioning has offered in prior releases.

10. Manageability Interval Partitioning Ref Partitioning
Introduced with 11g
Defined using an interval
Works much like Range Partitioning
Partitions are created as needed eliminates need to manually add partitions.
Ref Partitioning
Simpler partition management, child partitions created automatically when parent partitions are created
(1 min)
The new partitioning options introduced with 11g that we are examining, Interval Partitioning and Ref Partitioning do offer some manageability improvements.
Interval partitioning improves manageability by automatically creating partitions as they are needed, this eliminates the need to manually allocate partitions or keep a set of partitions out into the future to keep track of.
Ref Partitioning eliminates the need to mange partitions for the child table as child table partitions are created with the partitions of the parent table at the time they are created.
There is a limitation that interval partitioning and ref partitioning can not be used together. Therefore when using ref partitioning range partitioning is your best option and interval is not yet supported when using ref partitioning.

11. True Disk Costs
“The cost of managing storage hardware ranges from two to ten times the acquisition cost of the storage hardware.”, BNET
Cost/GB
GB Util Rate
Real Cost per GB
True Business Storage Cost
DAS
$9.65
45%
$20.27
$40.54 – $202.70
SAN FC
$18.51
80%
$22.21
$44.42 – $222.10
SAN SATA
$6.33
$7.60
$ $76
(2 min.)
“The cost of managing storage hardware ranges from two to ten times the acquisition cost of the storage hardware.”, BNET we can not look at disk being cheap as the larger the data volume the larger the hidden costs become. Also remember our pre-production environment (s) that are the same size as production that will experience the same growth as production. Take the actual cost that we said could be 2-10 times disk purchase cost for our production environment and consider the multiple environments under and supporting production So actual costs could be multiple times higher due to the increase for each GB having to be added to these environments as well.
We need to examine that disk between management, floor space, power, cooling, disk replacement costs, etc. all add up. This can create a longer term cost for your data and need to be aware of the hidden and long term costs of our data growth.
A lot of organizations are struggling with large data growth in servers adding challenges for data centers that are short on floor space, power and cooling capabilities, additional disk only complicates this further.
As we can see the true cost per GB is really much larger then the disk purchase cost when we consider factors like usable storage, power costs, cooling costs, floor space costs, management costs and backup costs. These costs can very from organization to organization depending and costs for these items and this does no account for the costs of replacement in the future at the SAN end of life.
As an example day we are a middle of the road organization using a Fibre Channel SAN at an average of $100/gb therefore it could be truly costing the organization $50,000 per 500GB and $100,000 per TB of usable storage. Therefore an increase of 500GB in a production database could really be $100,000 cost for that 500GB growth. And the long term costs continue on.
This shows that the cost of storage over the long term can get quite expensive when we factor in the typical replacement cycle of 5 years for storage, which means these costs repeat every 5 years and when we factor in the growth rate over 5 years which has the potential of 5 times the size we are today the costs can be huge.
How many are facing power and cooling limitations in their current data centers that is posing a challenge to add disk, servers, etc. ?
Is the cost of disk storage an impact or consideration for your organization?
** The above costs are based on 16TB configuration. 
Monash University, Cost of Storage – Direct Attached vs. SAN

12. Compression Index Compression since 8i Table Compression since 9i
No Additional License Requirement
Only for direct inserts
Compression Not Maintained with updates and normal inserts
Had to re-org table to re-compress over time.
Advanced Compression 11g
Additional License Requirement
Compression Maintained with all DML activity
No re-orgs required after initial
compression
(30 sec.)
We have been able to utilize index compression since 8i, Table Compression Since 9i
Problem is table compression only worked for direct inserts and normal DML would cause the loss of compression over time therefore requiring continual maintenance on the table to maintain compression of the table. This added to the management overhead of the table and limited our options on where it could most effectively be utilized.
Now Enter 11g with Advanced compression.

13. Compression What can compression accomplish? Shrink size of tables?
Shrink Size of indexes?
Improve buffer cache utilization?
Improve I/O disk visits?
Improve performance?
(30 sec.)
What can compression accomplish?
Can we shrink the size of tables?
Can we shrink the size of indexes?
Can we improve buffer cache utilization?
Can we Improve I/O utilization?
Can we improve performance?

14. What can we do Reduce Size of Existing? Reduce Size of Future Data?
Can we get a 10%, 20%, 30% reduction?
Reduce Size of Future Data?
Can we impact growth by 10%, 20%, 30%
Minimize performance impact of larger data volumes?
Disk Space, Backup/Recovery, Server Resources
Can we do all this without adding significant management overhead to the DBA?
(2 min.)
With true storage costs in the neighborhood of $100,000 per terabyte of storage and data growth averaging 60% annually large organizations are seeing their true storage cost rising and IT infrastructure struggling to keep up.
We need to reduce the size of the existing data, reduce the size of future data which reduces the flow of data growth, then we need to minimize the performance impact of large data growth within the system while trying to keep additional management to a minimum.
Are your storage administrators and DBAs having problems keeping up with the storage demands in your organization?
If so are you considering adding staff to compensate for this explosion in storage needs or are you looking for ways to reduce the storage needs in your data growth plans?

15. Ref Partitioning Examine Space Impact of Partitioning
Show disk space impact partitioned and un-partitioned.
Examine the true performance gain from Ref Partitioning
Demonstrate the partitioned and un-partitioned performance
Demonstrate the partitioned and compressed performance
(30 sec)
We will examine the 11g New Partitioning Feature Ref partitioning, What impact does it have on table sizes and query performance, then we will examine what apply the 11g New Advanced compression feature has on this new partitioning feature and how does that affect the table size and query performance.

16. Ref Partitioning – Un-Partitioned Table Size
ORDERS (78880 rows)
SUM(BYTES)/1024
4096
ORDER_ITEMS ( rows)
16384
(30 sec)
To demonstrate the impact of Ref Partitioning and Compression utilized with Ref Partitioning we will utilize 2 tables that have a parent child relationship.

17. Ref Partitioning Impact on Table Sizes
ORDERS (78880 rows)
SUM(BYTES)/1024
4736
ORDER_ITEMS ( rows)
13950
* Surprisingly we see the child table size reduced
(15 sec)
Partitioning the 2 tables in our example we examine the new sizes of the tables once partitioning is applied.
Partitioning typically increases table size however here is a surprise, by implementing Ref Partitioning the size of the child table actually got smaller by about 14%, though we do see that the parent table did increase in size.

18. Ref - Non Partitioned Table Performance
SELECT o.order_date, sum(oi.unit_price*oi.quantity) order_total
FROM oe.orders o, oe.order_items oi
WHERE o.order_date BETWEEN TO_DATE('01-APR-1999','DD-MON-YYYY') AND TO_DATE('30-JUN-1999','DD-MON-YYYY') AND o.order_id = oi.order_id
GROUP BY order_date
ORDER BY order_date ; ..
16 rows selected.
Elapsed: 00:00:00.93
(15 sec)
From the example query utilizing the 2 tables un-partitioned we can see the query is utilizing .93 seconds of elapse time.

19. Ref - Non Partitioned Table Performance
Statistics
1 recursive calls
0 db block gets
1967 consistent gets
1964 physical reads
0 redo size
970 bytes sent via SQL*Net to client
427 bytes received via SQL*Net from client
3 SQL*Net roundtrips to/from client
1 sorts (memory)
0 sorts (disk)
16 rows processed
(15 sec)
When we examine the statistics we see utilization of 1967 logical I/O and 1964 physical I/O lets make a note of this for later

20. Ref Partitioning Impact
SELECT o.order_date, sum(oi.unit_price*oi.quantity) order_total
FROM oe.orders o, oe.order_items oi
WHERE o.order_date BETWEEN TO_DATE('01-APR-1999','DD-MON-YYYY') AND TO_DATE('30-JUN-1999','DD-MON-YYYY') AND o.order_id = oi.order_id
GROUP BY order_date
ORDER BY order_date ; ..
16 rows selected.
Elapsed: 00:00:00.57
* .93 to .57 / 38% Improvement
(30 sec)
After implementing ref-partitioning for the tables that have the parent child relationship we see an impact to performance and resource utilization.
We go from .93 to .57 seconds representing a 38% elapse time performance improvement
Any thoughts on what improved the elapse time?

21. Ref Partitioning Impact
Statistics
44 recursive calls
0 db block gets
1630 consistent gets
1621 physical reads
0 redo size
896 bytes sent via SQL*Net to client
427 bytes received via SQL*Net from client
3 SQL*Net roundtrips to/from client
1 sorts (memory)
0 sorts (disk)
16 rows processed
* PIO - from 1967 to 1630 / 17% Improvement
LIO – from 1964 to 1621 / 17% Improvement
(30 sec)
We also see a reduction in logical I/O from 1967 to 1630 representing a 17% reduction in logical I/O
and then also see a physical I/O reduction from 1964 to 1621 also a 17% reduction
While this might not sound like large reductions in absolute terms the percentages are more representative of the impact. Take these savings and factor in the number of times this type of activity is executed as well as factor in other areas where the same impacts can be made and the performance improvement and resource savings add up.

22. Interval Partitioning
Examine Space Impact of Partitioning
Show disk space impact partitioned and un-partitioned.
Examine the true performance gain from Interval Partitioning
Demonstrate the partitioned and un-partitioned performance
Demonstrate the partitioned and compressed performance
(30 sec)
Now we will examine the affect of Oracle 11g Partitioning New Feature Interval Partitioning and the affect it has on table size and query performance , then examine the affect the 11g New Feature Advanced Compression has.

23. Interval - Non-Partitioned Table Size
Un-Partitioned Table 889,888 Rows
SUM(BYTES)/1024
45056
(30 sec.)
To measure the impact of size we first look at the size of a table in our environment with 889,888 rows in it.
While this table is 45 MB in size and does not have a large impact on overall space necessarily imagine this table being only 1 of 2000 in your database and if each table is 45M, and we all know that we most likely have tables larger then 45M in our database, 45M each = 90G.

24. Interval Partitioning Impact on Table Size
Partitioned Table 889,888 Rows
SUM(BYTES)/1024
58816
* 45M to 58M represents and increase in size when table is partitioned.
(30 sec)
As we can see that partitioning a table does increase the overall size of the table taking the table from approximately 45M to 58M in size.
So if we were looking at just improving performance and I.O this increase in size may not pose a problem for us esp if the performance gains are good enough to justify the size increase.
However what did partitioning improve performance and I/O?

25. Interval - Non Partitioned Table Performance
Un-Partitioned Table 889,888 Rows
SQL> select deptno, avg(sal)
from emp
where hiredate
between to_date('01-JAN-1982', 'DD-MON-YYYY') and to_date('01-JAN-1983', 'DD-MON-YYYY')
group by deptno ; ..
Elapsed: 00:00:03.37
(30 sec.)
Lets examine a table with 889,888 rows to see what a query takes against the table.
The results indicate 3.37 seconds for the query

26. Interval - Non Partitioned Table Statistics
7 recursive calls
0 db block gets
5548 consistent gets
5674 physical reads
0 redo size
546 bytes sent via SQL*Net to client
416 bytes received via SQL*Net from client
2 SQL*Net roundtrips to/from client
0 sorts (memory)
0 sorts (disk)
3 rows processed
(30 sec.)
The statistics show the query utilizing 5,548 logical I/Os and 5,674 physical I/Os
While this in of itself my not look like a large impact take a system that may be executing this type of query 1,000 times and hour and suddenly the impact becomes much larger to the overall system. Typically in a database individual queries in of themselves do not make up the impact to a system that affects overall performance it is the number of queries and transactions by many users that make up the overall workload impacting performance over time.
How can we Make this better?

27. Interval Partitioning Impact
Partitioned Table 889,888 Rows
SQL> select deptno, avg(sal)
from emp_part
where hiredate
between to_date('01-JAN-1982', 'DD-MON-YYYY') and to_date('01-JAN-1983', 'DD-MON-YYYY')
group by deptno ; ..
Elapsed: 00:00:01.57
* 3.37 to 1.57 / 53% Improvement
(30 sec.)
Partitioning may have increased the size of the table, but made a huge impact on performance and I/O utilization.
Went from an elapse time of 3.37 seconds down to 1.57 seconds a 53% performance improvement.

28. Interval Partitioning Impact
Statistics
1 recursive calls
0 db block gets
658 consistent gets
652 physical reads
0 redo size
546 bytes sent via SQL*Net to client
416 bytes received via SQL*Net from client
2 SQL*Net roundtrips to/from client
0 sorts (memory)
0 sorts (disk)
3 rows processed
* LIO – 5548 to 658 / 88% Improvement
PIO – 5674 to 653 / 88% Improvement
(1 min)
Are I/O impact shows where the elapse speed improvement comes from
Reducing logical I/O from 5,548 logical I/Os down to 658 an 88% reduction
and physical I/Os from 5,674 down to 653 again an 88% reduction.
From a look if performance and I/O were our goal then partitioning works well, for a little increase in size we make a significant impact to I/O and performance. The issue is what if we have disk constraints? Power, cooling, budget, etc. We may not be able to implement partitioning where we would get the most benefit due to our space constraints.
Would you considering partitioning performance and I/O improvements worth the additional disk space?

29. Accomplished With Partitioning
Positive
Reduced logical I/O
Reduced Physical I/O
Improved elapse time
Negative
Increased the size of the table
(15 sec)
Though partitioning we achieved several positive outcomes, we improved I/O both physical and logical as well as reduced elapse time.
However we increased the size of the table. Unfortunately with exploding data growth we headed in the wrong direction. We improved performance and improved I/O utilization but at the cost of additional disk space. What can we do?

30. Impact of Compression on Size of Ref-Partitioned Tables
ORDERS (78880 rows)
SUM(BYTES)/1024
4352
* 8% reduction over partitioned table 5% increase on Original table.
ORDER_ITEMS ( rows)
11520
* 29% reduction over partitioned table
17% reduction over Original table
(30 sec)
We see that for the orders table the table did decrease in size over the partitioned table size of 4736, but not below that of the original un-partitioned table with a size of We do however we do see the order_items table did see a size difference over both the original un-partitioned table with a size of and the partitioned table with a size of
While the increase in size is slight over the original orders table the order_items table reduction in size more then makes up the difference here by reducing the order_items table size by 29% over the original table and 17% over the partitioned table with ref partitioning. The overall space savings is certainly of value here however if our performance impact is significant this space savings is certainly big bonus.

31. Impact of Ref Partitioning and Compression Together
Elapsed: 00:00:00.43
Statistics
1 recursive calls
0 db block gets
413 consistent gets
407 physical reads
0 redo size
896 bytes sent via SQL*Net to client
427 bytes received via SQL*Net from client
3 SQL*Net roundtrips to/from client
1 sorts (memory)
0 sorts (disk)
16 rows processed
* .43 seconds - 53% improvement to original /
24% improvement partitioned un-compressed
(2 min)
Utilizing the same query we go from .93 to .57 seconds representing a 38% elapse time performance improvement with just partition as we demonstrated earlier, now that we add compression we improve performance to .43 seconds a 53% improvement over the non-partitioned and a 24% improvement over the partitioned and un-compressed set.
We also saw a reduction in logical I/O from 1967 to 1630 representing a 17% reduction in logical I/O between the original and partitioned set now that we add compression we see logical I/O reduced to 413 a 79% improvement over non partitioned set and a 74% reduction over the partitioned set un-compressed.
Further we saw a physical I/O reduction from 1964 to 1621 also a 17% reduction however adding compression we see physical I/O reduced to 407 again a 79% reduction over the original set and a 74% reduction over the partitioned but un-compressed set.
This shows that combining compression with ref partitioned makes some significant gains in performance in our example.

32. Impact of Compression on Size of Interval Partitioned Table
Partitioned Table 889,888 Rows
SUM(BYTES)/1024
39168
* 39% reduction on partition and uncompressed table
13% reduction from original table
(30 sec.)
By adding compression to the partitioned table the table size reduced to 39M, the original uncompressed table was 45M and the partitioned table was 58M
Therefore compression made a 39% reduction in the partitioned tables size and a 13% reduction on the original table size. So if we look at compression alone vs. our original table size and we can reduce each table in our system between 13% and 39% would there be advantages there for you?
Think about a data warehouse with 3 TB (3000G / 30,000M) or more of used 13% we would reduce used space by 3,900M / 3.9 GB.
What happened to performance though, in the reducing the space utilized was our performance impacted negatively or positively?

33. Impact of Interval Partitioning and Compression Together
Elapsed: 00:00:00.59
Statistics
1 recursive calls
0 db block gets
373 consistent gets
367 physical reads
0 redo size
546 bytes sent via SQL*Net to client
416 bytes received via SQL*Net from client
2 SQL*Net roundtrips to/from client
0 sorts (memory)
0 sorts (disk)
3 rows processed
* PIO - Original 5548 to 373 a 93% Imprv.
Partitioned 658 to 373 a 43% Imprv.
LIO - Original 5677 to 367 a 93% Imprv.
Partitioned 653 to 367 a 43% Imprv.
(2 min.)
Wow, as we can see we further reduced physical and logical I/O as well as further reduced elapse time.
We now see that the original table query logical I/O at 5548 and partitioned table at 658 and now logical I/O down to 373 a 93% reduction over the non-partitioned and uncompressed table and a 43% reduction over the partitioned and uncompressed table.
We also see physical I/O from the original table at 5677 and partitioned table down to 653 and now compressed down to 367 again a 93% reduction over the non-partitioned and uncompressed table and again a 43% reduction over the partitioned and uncompressed table.
Not only did we improve I/O but further improved performance to an elapse time of .59 seconds which is a 82% performance boost over the non-partitioned and uncompressed table and a 62% performance improvement over the partitioned and uncompressed table.
Can we see how powerful this is, and what advantages we can bring to our environment by making this kind of impact?
What if we could make this kind of impact in several places we could improve performance, lower disk space utilization and improve I.O utilization for the system?

34. Partitioning and Compression Summary
What can partitioning accomplish
Improve Performance
Break large table into chunks reducing I/O
Minimize Management Cost
Utilize interval and Ref partitioning where new partitions are created automatically.
Manage though individual Partitions adding flexibility for Table and index Management
Improve Database backup Performance
Mark tablespaces holding older Data partitions Read-Only as it eliminates the need to backup with each full backup of the database.
(1 min.)
Partitioning has been successful in improving performance and helping reduce backup times by putting tablespaces holding partitions into read-only since 9i, however management time of partitioned tables was higher then that of non-partitioned tables.
Partitioned tables improved performance through reducing I/O operations, this was accomplished through partition elimination which reduced that amount of read activity required to retrieve data from the partitioned table as well as though parallel operations.
We can reduce management cost though 11g partitioning features that reduce the time required to manage the partitioned tables.
We can still improve backup performance through placing the old partitions of tables into read-only tablespaces to eliminate the need to continue backing up those tablespaces.

35. Partitioning and Compression Summary
What Can Compression Accomplish?
Reduce Disk Space Costs
Compress partitioned tables reducing the size of tables
Improve Performance
Compress tables to reduce I/O read operations
(30 sec)
With 11g and advanced compression we can minimize or even eliminate the space impact of partitioning and reduce the overall size of a table reducing the disk space needs in the database. We could do this with table compression in 10g however it require more maintenance activities to maintain a compressed level the management of compressed tables is improved with advanced compression in 11g.
With compression we can further improve performance and the I/O benefit achieving lower elapse times for queries.

36. Partitioning Conclusions
Partitioning can improve I/O utilization
Partitioning can improve performance
Partitioning increases space utilization
Compression reduces space utilization
Compression can improve performance
Compression with partitioning can improve performance more then either of them alone and can reduce space utilization.
Interval Partitioning and Ref Partitioning reduces maintenance impact for partitioning
(2 min)
Does everyone agree that we can make these statements?
Anyone disagree with these statements?

37. Questions/Discussion
THANK YOU
(15 min.)