1. Data Archival
SunGard MF

2. Central Concepts Federated Engine Archival Engine
Enables creating logical pointers to data in remote mysql servers
While querying/inserting user is agnostic of the “real” source of data
Archival Engine
Compresses the data stored – 75% smaller than innodb engine
Ideal for auditing – supports only inserts and selects, no update

3. Suggested industry practice
Create a federated table in live server (separate db) with a logical pointer to the archive db in linux server
Create a view that “union”s live table and federated table – use it for reporting
During archival, move data from live table to federated table – data will be compressed and stored in remote archive engine
Remove the moved data from live server
src: dev.mysql.com

4. Solution Design
Setup
Will create a physical archive db and archive tables in the remote server (linux)
Will create a archive db and federated tables in live server pointing to the remote server
Archive tables will have the same schema as the mifos tables
Reporting views will be created during set up that unions the mifos tables and archive tables
Archivelog table will be created in mifos for maintaining archival history

5. Business Use Cases
Admin logs in to the app
Goes to Admin  Archival Screen
Chooses the sub-entity to archive – (for example customerschedule)
Configures the condition to archive – (all customerschedules before 2008 Jan)
Starts the archival action
Archival procedure runs which moves all customer schedules before 2008 Jan to the federated db (and thereby to compressed archive db)
Archival procedure deletes the moved records from live db
Entry added to archive log table in mifos

6. Is the process on demand or real time?
Questions
Is the process on demand or real time?
It will be on-demand
Can I have multiple copies of archive db – categorized by year of data?
Setup process will have to re-run to create different archive db. At any point one archive db will be current
How will schema modifications to mifos be handled
We may have to change the schema of federated and archived tables as well.. We may have to plug-in to the patch process of mifos.

7. How can we ensure that required data is not archived
Questions
How can we ensure that required data is not archived
For every sub-entity identified for archiving we need to have a rule to identify “archivable” data
The data archival procedure should not archive non-archivable data even if they satisfy the archive condition specified in the admin screen
Can we have the archival process automatic
Theoretically possible as we will know what all data is archivable
We can have a job that can take care of archiving data – for the first release let us not include this