1. Ben Davis- Information Governance Specialist 1 May 2012
Maximising your investment in your Oracle Applications with proven lifecycle management strategies
Ben Davis
Senior Data Governance Specialist
IBM Australia & New Zealand
Ben Davis- Information Governance Specialist
1 May 2012

2. Our World today

3. Big Data!

4. Some facts
In 2009, amid the "Great Recession," the amount of digital information grew 62% over 2008 to 800 billion gigabytes (0.8 Zettabytes).
90% of the world’s total data has been created just within the past two years 
By 2020, IT departments will be looking after 10 x more servers, 50 x more data and 75 x more files.  Meanwhile, the number of IT administrators keeping track of all that data growth with increase by 1.5 times.
By 2020, the percent of digital information requiring security beyond baseline levels will grow from 30% to 50%.
35% more digital information is created today than the capacity exists to store it. This number will jump to over 60% over the next several years.

5. So when will this data storm end?
The answer is never…
Data will get bigger
Data will become more complex
You cannot build a dam big enough to
hold all of your data
You need to be smarter
You need to be prepared

6. Strategies to maximise your investment

7. Understand & discover your data
Develop a Data Management lifecycle strategy
Test Data management practices

8. Understand & discover your data

9. You can’t govern what you don’t understand
Discover & Define
Develop & Test
Optimize & Archive
Consolidate & Retire
Information Governance Core Disciplines
Lifecycle Management ?
Data can be distributed over multiple applications, databases
and platforms
Where are those databases located?
Complex, poorly documented data relationships
Which data is sensitive, and which can be shared?
Whole and partial sensitive data elements can be found in hundreds of tables and fields
Data relationships not understood because:
Corporate memory is poor
Documentation is poor or nonexistent
Logical relationships (enforced through application logic
or business rules) are hidden ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?
In order to define a governance strategy and a process to achieve your organization’s goal, you first have to understand what you have. Without this, you cannot create an effective plan that will support your organization. This process begins with understanding the web of information represented in your enterprise applications and databases. You must understand:
- where the data exists and what data elements there are
- what relationships exists within systems
- what complex relationships exists across and between systems
- You have to understand the complex relationships because
- where is sensitive data located
Many organizations rely on documentation (which is often out-dated) or on system/application experts for this information. Sometimes, this information is built into application logic and is not apparent to anyone the hidden relationships that might be enforced behind the scenes.
Think about it as a using a current map to understand your heterogeneous landscape – very similar to driving in an automobile and trying to get from point A to point B. You would not use a map from 20 years ago because the road infrastructure has probably changed. Would you just start driving without any idea of the roads to take, how you are going to plan your stops for a long trip and knowing what risky areas of town you should avoid? Navigating data is the same concept.
It’s all about time, cost and risk. Trying to manually understand this information (or using the ‘spot check’ approach) can you lead you down the wrong path resulting in many lost hours in the future including potentially delays for project deployment.
An automated result can produce tremendous savings. For example, doing in one week what can take 10 people 10 months.
Distributed Data Landscape

10. Oracle Applications: The Challenge
Business Object
(e.g. Purchase Order)
Database schemas with thousands of tables & complex relationships
Almost No PK or FK Constraints
Database System Catalog does not hold useful metadata
Proprietary ERP/CRM metadata holds ‘Logical View’ of data
Customers extending the standard data model
Scoping the tables for data mgmt projects

11. InfoSphere Optim Application Repository Analyzer
Requirements
Analyze metadata repository of leading packaged Oracle ERP & CRM applications
Analyze application metadata for exploring data models and customizations
Compare data model structures across application versions & releases
Export the complete business object for use with other InfoSphere Optim solutions
Optim ARA
Benefits
Quickly identify and scope application data structures to speed ILM projects
Provide better impact analysis of application upgrades
Easily create complete business objects for Optim archive, subsetting & masking projects 11

12. Value to Our Customers
Accelerate understanding your existing distributed data landscape for:
Data Archiving
Test Data management
Sensitive Data
Application/Data Consolidation, Migration and Retirement
Master Data Management and Data Warehousing
10x reduction in risk, time and effort for the discovery phase projects
Automated discovery of business entities, cross-source business rules and anomalies
Increased repeatability
Verifiable results
Automated data discovery, audit and remediation
Manual Data Analysis and Validation (Time, Risk and $$)

13. Develop a Data Management lifecycle strategy

14. The Lifecycle of data Created Utilised Reports Infrequently accessed
Security
Development
Upgrades
Created
Search
Support
Legal Holds
App Perforrmance
Accessibility
Utilised
Storage
Ease of use
Longevity
Universal Access
File Attachments
SLA’s
Training
Testing
Reports
Auditability
Infrequently accessed
Archive Policy
App retirement
Evidentiary admissibility
Changing Data schemas
Metadata

15. Organizations have been increasingly challenged with successfully managing data growth
Increasing Costs
Poor Application Performance
Manage Risk & Compliance
3-10x
80%
50%
Cost of managing storage over the cost to procurea
The time DBA’s spend weekly on disk capacity issuesc
of firms retain structured data for 7+ yearse
$1.1 billion
250 hours
57%
Amount organizations will have spent in 2011 on storageb
The amount of time needed to run “daily” batch processesd
of firms use Back-up for data retention needse
(a) Merv Adrian, IT Market Strategies, “Data Growth Challenges Demand Proactive Data Management”, November 2009
(b) IDC, “Worldwide Archival Storage Solutions 2011–2015 Forecast: Archiving Needs Thrive in an Information-Thirsty World”, October 2011
(c) Simple-Talk, “Managing Data Growth in SQL Server ”, January 2010
(d) IBM Client Case Study: Toshiba TEC Europe; archiving reduced batch process time by 75%
(e) IDC Quick Poll Survey 2011, “Data Management for IT Optimization and Compliance”, November 2011
And the challenges are real:
The cost of *managing* storage can be 3-10 times the cost of procuring it
And IDC estimated that last year, organizations spent about $1.1 billion on storage costs
Ask the DBA how much time they’re spending each week on hardware capacity-related performance issues – it can be up to 80%
One InfoSphere Optim client had about 19,000 batch processes that took 250 hours to run – that’s more than 10 days. Archiving helped them reduce that time by over 75%
Talk to your clients about how long they’re keeping data in their production systems. How long are they *supposed* to keep it at all? At least 50% of companies are keeping data for 7 or more years.
And 57% of companies are leveraging their back-up copies for “data retention”. That’s a lot of data to sift through if you have an e-discovery request.
As organizations strive to stay ahead of the competition, a more “proactive” approach to data management is needed to ensure the data is accessible and trusted.

16. Organizations have been increasingly challenged with successfully managing data growth
Develop & Test
Discover& Define
Consolidate & Retire
Optimize & Archive
Information Governance Core Disciplines
Lifecycle Management
Poor Application Performance
Increasing Costs
Manage Risk & Compliance
Storage Capex 20%
Application Performance
Storage Operational Costs
80%
Buying more storage is not a “cheap” fix when you add the operational burden
Business users & customers wait for application response; DBA’s spend majority of time fixing performance issues
The “keep everything” strategy can impact disaster recovery and data retention & disposal compliance
Organizations today have been increasingly challenged with successfully managing data growth. They have large volumes of data stored in various data repositories across the organization — and this is likely to grow to exponentially in the coming years. As a result of this rampant growth, these companies store more and more data every year in production systems.
If your client has enterprise applications such HR, Finance, Customer Support systems – and they likely have many – then they’re dealing with the effects of increasing data volumes. Today’s organizations are met with 3 primary challenges:
Increasing costs: As the volumes of data increase, the “natural” response it to simply buy more storage for the enterprise application. After all, “storage is cheap”, right? However, while the acquisition of storage hardware is cheap, the operational costs associated with it are often underestimated. There’s also the costs associated with draining IT staff resource allocation as they try to locate and manage data growth issues.
Poor Application Performance: Over time, as data volumes increase, application performance will be an issue. And what often occurs is that your clients “self-diagnose” this performance issue with simply the hardware performance – “let’s buy faster systems”; or they may task their DBA’s to tune and re-tune the database. However, without properly managing the data volumes, companies will see the impact in the performance of their system over time. This is particularly a problem when application performance impacts employee productivity.
Risks associated with data retention and compliance: So, the “keep everything” in production methodology can also create a risk associated with accessing this data for the long-term. So, how can companies safely store data that is no longer needed in production systems, but that must be retained per data retention policies. How can organizations store this data so that it’s audit-ready, easily accessible, and available for any e-discovery requests?
And the true cost of data storage has many components as indicated by this chart – there is much more to consider than just disk storage. In face, storage is one of the most unplanned budget busters organizations grapple with. Its much more than disk storage and the results of not dealing with it extend well beyond $ but also negative performance and lost productivity. These numbers are based on our BVA studies and a case study from Gartner (
Said differently, for every $1 spent on storage, organization spend $4 on the operational elements of managing that stored data.
So how do organizations respond to this exponential data growth? And what’s the key to managing it WELL?
When you implement a system, you typically create hardware requirements based on the performance requirements of the business. Over time, performance slowly decreases due in large part to the increased data. So what do most businesses do? Invest in more hardware to get performance back to acceptable levels. But then what happens? Over time, the data continues to grow and performance continues to decrease. This happens over and over again, so you are essentially increasing costs to get the same results.
Does this methodology fix the cycle? Or delay the inevitable?
So, where does data growth management fit into your organization’s lifecycle management strategy? 16

17. Archive historical data for data growth management
Develop & Test
Discover& Define
Consolidate & Retire
Optimize & Archive
Information Governance Core Disciplines
Lifecycle Management
Production
Data Archives
Historical
Archive
Historical Data
Reference Data
Restored Data
Retrieve
Current
Can selectively restore archived data records
Universal Access to Application Data
ODBC / JDBC
XML
Report Writer
Application
When looking for an archive solution, again, you want to ensure you flexible access to this data and flexible storage options. So how do you do that?
Well, let’s first define what we mean by archiving. It should be an intelligent process for MOVING inactive/infrequently accessed data that still has value (e.g. for data retention needs, etc.). [Recall: Forrester’s statistic of 85% inactive data!]
PLUS, the archive process should provide the ability to search and retrieve the data in a manner that functional users need to consume the data.
This is a typical example of a Production environment prior to archiving. Initially, both Active and Inactive data is stored in the Production environment, taking up most of the space on the Production Server.
Safely move the inactive or historical data to an archive, capturing the complete business object for application independent access. This data can then be stored in a variety of environments.
The data can then also be easily retrieved to an application environment when additional business processing is required.
You then have universal access to this data through multiple methods, including Report Writers such as Cognos and Crystal Reports, XML, ODBC/JDBC, application-based access (Oracle, Siebel, etc.) or even IBM Mashup Center. 17

18. Effectively Archive and Manage Data Growth
Reduce Costs
Improve Performance
Minimize Risk
Reduce hardware, software, storage & maintenance costs of enterprise applications
Improve application performance & streamline back-ups and upgrades
Support data retention regulations & safely retire legacy/redundant applications
Discover & identify data record types to archive across heterogeneous environments
Intelligently archive data to improve application performance and support data retention
Capture & store historical data in its original business context
Define & maintain data retention policies consistently across the enterprise
Ensure long-term, application-independent access of archived data via multiple access methods
Support for custom & packaged ERP applications in heterogeneous environments
So, how can IBM help clients with these challenges? Through effective data growth management. InfoSphere Optim Data Growth Solution can help clients REDUCE COSTS, IMPROVE APPLICATION PERFORMANCE AND MINIMIZE THE RISKS associated with managing application data over its lifetime.
Reduce Costs: By archiving infrequently used data from production environments, that data can be stored on less expensive, tier 2 storage, and can be compressed to save even more storage space.
Improve Performance: With less data, application performance improves, searches and batch processes run faster, back-up processes run more efficiently. If your client is considering an application upgrade, archiving can streamline the associated data conversion process – less data to convert to the upgraded version, the less time the application is offline.
Minimize Risks: Intelligently archiving data out of production systems allows for data retention compliance, but also supports a better long-term solution for storing the application data, providing application independent access to that data.
InfoSphere Optim Data Growth solution is the proven, marketing leading solution that:
With InfoSphere Discovery, discovers & identifies data record types to archive across heterogeneous environments
Intelligently archives data, capturing & storing historical data in its original business context
Defines & maintains data retention policies consistently across the enterprise
Ensures long-term, application-independent access of archived data via multiple access methods, including third party reporting tools as well as InfoSphere Optim Data Find, a web based search engine
Supports for custom & packaged ERP applications in heterogeneous environments 18

19. An effective data archiving policy
Starts out small and targeted
Archive data object and related entities as a referentially intact set of data
Ensures archived data is still accessible
Ties into corporate records keeping requirements
IS an evolving project that MUST be flexible as requirements change

20. Test Data management practices

21. Information Governance Core Disciplines
Organizations continue to be challenged with building quality applications
Discover& Define
Optimize & Archive
Consolidate & Retire
Develop & Test
Information Governance Core Disciplines
Lifecycle Management
Time to Market
37%
Satisfied with speed of software developmentf
30-50%
Time testing teams spend on
setting up test environments,
instead of testingb
Increasing Costs
$300 billion Annual costs of software-related downtime.d
32%
Low success rate for software projectse
Increasing Risk
45,000+ Number of sensitive records exposed to 3rd party during testingc
62% companies use actual customer data to test applicationsa
a. The Ponemon Institute. The Insecurity of Test Data: The Unseen Crisis
b. NIST, Planning Report. The Economic Impacts of Inadequate Infrastructure for Software Testing c. Federal Aviation Administration: Exposes unprotected test data to a third party d. The Standish Group, Comparative Economic Normalization Technology Study, CHAOS Chronicles v12.3.9, June 30, 2008 e. The Standish Group, Chaos Report, April 2009
f. Forrester Research, “Corporate Software Development Fails To Satisfy On Speed Or Quality", 2005
These are some statistics supporting the challenges. For example, An FAA server used for application development & testing was breached, exposing the personally identifiable information of 45,000+ employees.
62% of companies use actual customer data to test applications exposing sensitive information to testers and developers.
30-50% time testing teams spend on setting up test environments instead of testing.
$300 Billion is the annual costs of software related downtime.

22. Organizations continue to be challenged with building quality applications
Discover& Define
Optimize & Archive
Consolidate & Retire
Develop & Test
Information Governance Core Disciplines
Lifecycle Management
Increasing Risk
Time to Market
Increasing Costs
Mandatory to protect data and comply with regulations
Lack of realistic test data and inadequate environments
Defects are caught late in the cycle
Organizations continue to be challenged with building and delivering quality applications. They are challenged with increasing risks associated with protecting data and complying with regulations. Time to market of these applications is critical to success of the businesses yet long testing cycles and resources often delay delivery of the software on time
…. inadequate test environments, lack of realistic test data are contributing reasons for this challenge. The cost go spiraling up as defects are caught late in the cycle where they are expensive to correct.

23. Employ effective test data management practices
Discover& Define
Optimize & Archive
Consolidate & Retire
Develop & Test
Information Governance Core Disciplines
Lifecycle Management
Production or Production Clone
Subset & Mask
-Compare
-Refresh
2TB
25 GB
Create targeted, right-sized test environments
Substitute sensitive data with fictionalized yet contextually accurate data
Easily refresh, reset and maintain test environments
Compare data to pinpoint and resolve application defects faster
Accelerate release schedules
25 GB
Development
Unit Test
100 GB
50 GB
Training
For generating the test data, it’s critical to productivity to create “right sized” subsets for all your testing needs, allowing testers and developers to easily extract, refresh and create properly sized data sets. After running tests, relationally compare results sets on the new data set and the actual production data to see the exact differences – and only the differences. This can help resolve application defects faster.
Part of effective test data management is the ability to protect the sensitive data within these non-production environments. Ensure sensitive test data is masked while maintaining the referential integrity of the data, while ensuring this data transformation is appropriate to the context of the application. That is, the results of data transformation have to make sense to the person reviewing the test results. For example, if an address is needed, you would like to use a street address that actually exists as opposed to using something meaningless like XXXXXX as a street name.
Integration Test 23

24. Improve application quality and delivery efficiency
Reduce Cost
Reduce Risk
Speed Delivery
Automate creation of realistic “right sized”
test data to reduce the size of test environments
Mask sensitive information for compliance and protection
Refresh test data through self-service, speeding testing and
application delivery
Understand what test data is needed for test cases
Create “right-sized” test data by subsetting
Ensure masked data is contextually appropriate to the data it replaced, so as not to impede testing
Easily refresh & maintain test environments through self service access by developers and testers
Automate test result comparisons to identify hidden errors
Support for custom & packaged ERP applications in heterogeneous environments
InfoSphere Optim Test Data Management Solution improves application quality and delivery efficiency.
Key value propositions include:
Reduce Cost through Automate creation of realistic “right sized" test data to reduce the size of test environments
Reduce Risk through Mask sensitive information for compliance and protection
Speed Delivery through Refresh test data through self-service, speeding testing and application delivery
Key differentiators include:
Understand what test data is needed for test cases
Create “right-sized” test data by subsetting
Ensure masked data is contextually appropriate to the data it replaced, so as not to impede testing
Easily refresh & maintain test environments through self service access by developers and testers
Automate test result comparisons to identify hidden errors
Support for custom & packaged ERP applications in heterogeneous environments 24

25. IBM meeting these challenges
Subset & Mask
Non Production Environments
Archive
Production Environments
OEM/ISV
Custom
Amdocs
SAP
JDEdwards
PeopleSoft
Oracle
Siebel
OEM/ISV
Custom
Amdocs
SAP
JDEdwards
PeopleSoft
Oracle
Siebel
Data Growth, Data Privacy, Test Data Management, Application Upgrades, Application Retirement
Optim ™
Oracle
SQL Server
Sybase
Informix
DB2 LUW
XML
IMS
VSAM
Adabas
DB2 z/OS
Teradata
Windows XP/ Solaris HP/UX Linux AIX OS/ Z/OS i-Series
NAS SAN ATA CAS Optical Tape
Single, scalable, interoperable EDM solution provides a central point to deploy policies to extract, store, port, and protect application data records from creation to deletion

26. Attend our other sessions later today
14:00-14:50- Rangimarie 1
Closing the data Privacy Gap: How safe is your data?
15:00-15:30- Rangimarie 3
Manage the data lifecycle of Big Data environments

27. Questions? Further Information www.optimsolution.com Ben Davis