Freeware Replication from Oracle Proof of Product For Postgres & Mysql

Problem Create subsets of production data called used sets, and load them into freeware databases (postgres&mysql).

Issues Cdf had several prototype attempts at doing freeware replication from the cdfofprd database. Due to people leaving the project, no prototype was completed. Last attempt at Ohio State was handed to dsg last summer. Want to see if replicating to postgres and mysql is feasible, maintainable. Cdf would define ~5-6 used sets that would provide small subsets of production data and be sufficient for most end users analysis needs.

Project Requirements 1.Copy the calibration database application and used set provided by Ohio State to mysql & postgres. 2.Update the non-calibration tables by hand to mysql & postgres. Some non-calibration supporting data is needed for queries. 3.Make schema changes smooth 4.Update replica at some frequency

Status of Requirements Copy calibration data & validate – done for mysql and postgres. Update required non-calibration by hand – done for both databases. A property file is used to define what is selected.

Status of Requirements cont. Make schema changes smooth (truncate & refresh) – done for both databases. Added options to exclude some table columns and merge 2 or more oracle tables to 1 freeware table. Update replica with some frequency – complete, updates/refreshes are on demand.

Some Stats load 76 tables from oracle cdfofprd; total KB. in general less then an hour per 1 million records. noted it is slow during daytime. Some examples: CDF2_RUNSECTION_LUMINOSITIES 1,833,064 records for 43:58min. CDF2_RUNSECTIONS 1,885,126 records for 33:36min.