Split-Brain Syndrome: Barriers To Oracle Database In The Cloud And How Data Virtualization Resolves Them Tim Gorman | Delphix | Technical Services

Senior Principal, Technical Services at Delphix Tim Gorman Senior Principal, Technical Services at Delphix 2.5 yrs at Delphix, 25 yrs as an Oracle DBA, 33 yrs in IT Co-author of two books on data warehousing, three books on database administration, one book on SQL Oracle ACE Director, OakTable Network, RMOUG Lives in Colorado, married to Kellyn (DBAKevlar), 2 kids, 3 step-kids Peter (26), Marika (23), Sam (21), Caitlyn (19), Josh (16) utoug.org

1 2 3 Methods to get from on-premise to IaaS cloud Barriers to IaaS cloud migration 3 Data virtualization and transformations as a solution utoug.org

Migrating from on-premise to IaaS How? Just copy stuff up there Straightforward approach Inefficient, non-incremental for large environments Open-source “bbcp” uses encryption, compresses, and multi-threads utoug.org

Migrating from on-premise to IaaS How? Just copy stuff up there Straightforward approach Inefficient, non-incremental for large environments Open-source “bbcp” uses encryption, compresses, and multi-threads Start with backups to IaaS storage, then populate re- hosted applications by restoring from those backups IaaS backups are easy, secure, unobtrusive utoug.org

Backups cloud service: from on-premises to IaaS Cloud Storage Database server utoug.org

Backups cloud service: from on-premises to IaaS, then restore Cloud Storage Database server Database server utoug.org

If you’re very lucky, yes… Migration solved? If you’re very lucky, yes… utoug.org

If you’re very lucky, yes… Migration solved? If you’re very lucky, yes… ...but realistically no, not by a long shot. utoug.org

What are the challenges? Assuming that non-production systems are populated from production systems… Different cost structures Doing common tasks the same way might end up costing more utoug.org

Different cost structures On-demand EC2 instances Range from $0.0065/hr to $8.184/hr Reserved instances provide discounts from 29% to 75% over on-demand Storage in Elastic Block Storage EBS General Purpose SSD (gp2) volumes $0.10 per GB-month of provisioned storage EBS Provisioned IOPS SSD (io1) volumes $0.125 per GB-month of provisioned storage $0.065 per provisioned IOPS-month EBS Throughput Optimized HDD (st1) volumes $0.045 per GB-month of provisioned storage Data transfer in EC2 DATA IN from “internet” is free, but internally within EC2 can be $0.01/GB Source: https://aws.amazon.com/ec2/pricing/ Sep 2016 utoug.org

Migrating from on-premise to IaaS How? Just copy stuff up there Straightforward approach Inefficient, non-incremental for large environments Open-source “bbcp” uses encryption, compresses, and multi-threads Start with backups to IaaS storage, then populate re- hosted applications by restoring from those backups IaaS backups are easy, secure, unobtrusive Data virtualization Easy, secure, revolutionary Optimized for faster tempo development and testing Optimized for new costing structures in IaaS utoug.org

Data Virtualization: linking to a source Delphix Virtualization Engine 10 TB storage RMAN Source DB server Oracle 9i, 10g, 11g, 12c 10TB storage utoug.org

Data Virtualization: provisioning a virtual database to a target Delphix Virtualization Engine 10 TB storage NFS Target DB server Oracle 9i, 10g, 11g, 12c No database storage utoug.org

Data Virtualization: provisioning a virtual database to a target Delphix Virtualization Engine 10 TB storage NFS NFS Target DB server Oracle 9i, 10g, 11g, 12c No database storage Target DB server Oracle 9i, 10g, 11g, 12c No database storage utoug.org

Data Virtualization: provisioning a virtual database to a target Delphix Virtualization Engine 10 TB storage NFS NFS NFS Target DB server Oracle 9i, 10g, 11g, 12c No database storage Target DB server Oracle 9i, 10g, 11g, 12c No database storage Target DB server Oracle 9i, 10g, 11g, 12c No database storage utoug.org

Data Virtualization: provisioning a virtual database to a target Delphix Virtualization Engine 10 TB storage NFS NFS RMAN NFS Target DB server Oracle 9i, 10g, 11g, 12c No database storage Target DB server Oracle 9i, 10g, 11g, 12c No database storage Target DB server Oracle 9i, 10g, 11g, 12c No database storage Source DB server Oracle 9i, 10g, 11g, 12c 10TB storage utoug.org

Snapshot 1 – initial load b c d e f g h i We start with a full backup. Includes transaction logs needed for recovery. Jonathan Lewis © 2015

Snapshot 2 – incoming new changes After initial link/load, Delphix is incremental forever Sn #1 Needs the sample rman script generated. Do I have is somewhere a b c d e f g h i b' c' Jonathan Lewis © 2015

Apply Snapshot 2 a b b' c c' d e f g h i The Delphix appliance unpacks the backup and "overwrites" the initial backup with the changed blocks. DxFS makes new copies of the changed blocks, never updates a block Jonathan Lewis © 2015

Snapshot 3 – more incoming changes b b' c c' d e f g h i i' Jonathan Lewis © 2015

Apply Snapshot 3 a b b' c c' d e f g h i i' Sn #1 Sn #2 Sn #3 Jonathan Lewis © 2015

Retention policies – dropping snapshots b b' c c' d e f g h i i' Jonathan Lewis © 2015

Retention policies – dropping snapshots b' c' d e f g h i i' Jonathan Lewis © 2015

Creating a VDB a b' c' d e f g h i i' VDB #1 Sn #2 Sn #3 Jonathan Lewis © 2015

Creating a VDB a b' c' d e f g h i i' VDB #1 Sn #2 Sn #3 Jonathan Lewis © 2015

Changes to a VDB a b' c' d e f g h i i' a‘' VDB #1 Sn #2 Sn #3 Jonathan Lewis © 2015

Changes to a VDB a a‘' b' c' d e f g h i i' VDB #1 Sn #2 Sn #3 Jonathan Lewis © 2015

Data Virtualization: on-premises Delphix Virtualization Engine 10 TB storage RMAN NFS Target DB server Oracle 9i, 10g, 11g, 12c No database storage Source DB server Oracle 9i, 10g, 11g, 12c 10TB storage utoug.org

Data virtualization: from on-premises into IaaS Delphix Virtualization Engine 10 TB storage Source DB server Oracle 9i, 10g, 11g, 12c 10TB storage Target DB server Oracle 9i, 10g, 11g, 12c No database storage utoug.org

OK, now is the migration solved? If you’re very lucky, yes… utoug.org

OK, now is the migration solved? If you’re very lucky, yes… ...but realistically still no, not by a long shot. utoug.org

What are the challenges? Assuming that non-production systems are populated from production systems… Different cost structures Doing common tasks the same way might end up costing more Confidential data All IaaS alternatives promote encryption for data in-flight and for data at-rest, but does encryption meet requirements? Platform Unless you choose a database/schema IaaS solution, your legacy platform might be a problem utoug.org

How does data virtualization meet these challenges? Optimized for the cloud in the first place… Different cost structures Much smaller storage footprint, much less data-transfer utoug.org

Different cost structures Traditional copy data management techniques Developed without concerns about infrastructure chargeback Less mindful of metering of usage of storage and data transfer IaaS vendors monitor storage and data transfers Help meet SLAs, garner profits Data virtualization minimizes… Storage costs Thin-provisioning storage avoidance Data transfer costs Far less data transferred during provisioning/refresh operations As opposed to the volume of data transferred using traditional cloning techniques Network-attached storage data transfers offset somewhat Most non-production usage modifies or accesses only a tiny proportion of the total database during lifetime utoug.org

How does data virtualization meet these challenges? Optimized for the cloud in the first place… Different cost structures Much smaller storage footprint, much less data-transfer Additional data transformations extending the base capability Confidential data Integrated data masking utoug.org

All IaaS solutions provide encryption in-flight and encryption at-rest Confidential data All IaaS solutions provide encryption in-flight and encryption at-rest But does encryption solve the problem facing the management of non-production systems? Your gut says “no”… The EU agrees with you... http://ec.europa.eu/justice/data-protection/article- 29/documentation/opinion- recommendation/files/2014/wp216_en.pdf utoug.org

Encryption is reversible data obfuscation Confidential data Encryption is reversible data obfuscation Does it resolve the security problem? Or does it shift the problem toward authentication and authorization? Is authentication and authorization in non- production in compliance with security goals? Should developers and QA testers and DBAs in non- production be able to access confidential production data? In order to do the job, they have to, don’t they? utoug.org

Masking is irreversible data obfuscation Confidential data Masking is irreversible data obfuscation Masking personally-identifiable information renders it permanently useless from a security standpoint Resolves both the technical issue as well as the people/process issue Mask the data while still in the production zone Then ship it to the non-production zone Confidential data in-flight is no longer at risk, encrypted or not Confidential data at-rest is no longer at risk, encrypted or not utoug.org

Confidential data Exposure Production Non-Production utoug.org

Confidential data Exposure Solution Production Encryption Non-Production Masking utoug.org

Data virtualization: transformation by masking Delphix Virtualization Engine 10 TB storage Linux Source 10 TB database Linux Target Oracle 9i, 10g, 11g, 12c No database storage Delphix Masking Engine utoug.org

Data virtualization: transformation by masking, then replicating to IaaS Delphix Virtualization Engine 10 TB storage Delphix Virtualization Engine 10 TB storage Linux Source 10 TB database Linux Target Oracle 9i, 10g, 11g, 12c No database storage Target DB server Oracle 9i, 10g, 11g, 12c No database storage Delphix Masking Engine utoug.org

How does data virtualization meet these challenges? Optimized for the cloud in the first place… Different cost structures Much smaller storage footprint, much less data-transfer Additional data transformations extending the base capability Confidential data Integrated data masking Platform Integrated cross-platform provisioning utoug.org

The following CPU architectures are big-endian… Platform The following CPU architectures are big-endian… SPARC, PowerPC, PA-RISC, Itanium The following CPU architectures are little-endian... x86 What does this mean for legacy UNIX platforms? Extinction Oracle RMAN supports cross-platform migration since 10g utoug.org

Data virtualization: transforming cross-platform Delphix Virtualization Engine 10 TB storage UNIX Target Oracle 9i, 10g, 11g, 12c No DB storage UNIX Source Oracle 9i, 10g, 11g, 12c 10 TB database Linux Target Oracle 9i, 10g, 11g, 12c No DB storage utoug.org

Data virtualization: transforming cross-platform, then masking Delphix Virtualization Engine 10 TB storage UNIX Target Oracle 9i, 10g, 11g, 12c No DB storage UNIX Source Oracle 9i, 10g, 11g, 12c 10 TB database Linux Target Oracle 9i, 10g, 11g, 12c No DB storage Delphix Masking Engine utoug.org

Selective replication of virtual databases and files Delphix appliances include replication For data protection, for scalability, and for migration Virtualized data is already compressed at-rest, further compressed and encrypted in-flight, incrementally replicated to another engine utoug.org

Data virtualization + transformations = complete solution Once data has been virtualized on-premises… Then transformed cross-platform on-premises... Then transformed by securely masking on-premises... Then... the securely virtualized data can be safely replicated to IaaS utoug.org

Data virtualization: transforming cross-platform, then masking, then replicating to IaaS Delphix Virtualization Engine 10 TB storage Delphix Virtualization Engine 10 TB storage UNIX Target Oracle 9i, 10g, 11g, 12c No DB storage UNIX Source Oracle 9i, 10g, 11g, 12c 10 TB database Linux Target Oracle 9i, 10g, 11g, 12c No DB storage Target DB server Oracle 9i, 10g, 11g, 12c No database storage Delphix Masking Engine utoug.org

Summary: challenges with solutions New chargebacks Data virtualization minimizes storage and data transfer Confidential data Data masking integrated with data virtualization Platform Cross-platform conversion integrated with data virtualization utoug.org

Tim Gorman Technical Services Tim.Gorman@delphix.com +1 (303) 885-4526 utoug.org