1. Archiving and Document Transfer Utilities
SPS Spotlight Series
March 2015

2. This information was originally created for The SPS Spotlight Series project, which ran from November 2013 through November Updates to the original content may be incorporated to ensure accuracy. If you have any questions about this information, please contact the SPS Help Desk at

3. Overview Document Transfer Utility Archiving Utility
Preliminary and Final Transfer Modes
Preliminary Process Status Report window
Final Transfer
Process Status – 7 types
Cleanup Service
Archiving Utility
Purpose
Storage Database
Archive Eligible Date
Processes – 5 types
Preliminary and Final Modes
Initiate Transfer Process

4. PD² Document Transfer Utility v5.0
The purpose of the PD² Document Transfer Utility is to provide the ability to transfer data from one PD² database to another PD² database.
The PD² Document Transfer Utility is built using the webMethods Integration Platform, and is part of an application suite that is designed to work with minimal user intervention. The PD² Document Transfer Utility v5.0 supported baseline is PD² v4.2 Increment 2 SR15.
The originating and receiving databases need to be on the same version of PD² database, Standard Data Installer and Clause Installer.

5. JDBC Adapters Connection Page

6. PD² Document Transfer Utility

7. DTU - Preliminary vs Final Transfer Modes
Preliminary mode allows sites to run the process without actually transferring records to the target database.
A report is produced that provides a list of those documents meeting the defined criteria.
Provides the ability for an SA to review the list of potential transactions moving to the target database prior to the action taking place.
The preliminary mode status report contains a time estimate indicating how long the process takes to complete if run in final mode.
The preliminary mode status report also provides the ability to run the same process in final mode.
Allows the SA to run the final mode process without having to select the documents and the source/target database connections again.

8. DTU - Preliminary vs Final Transfer Modes
The option to choose Preliminary or Final

9. Preliminary vs Final Transfer Modes

10. DTU - Preliminary vs Final Transfer Modes
Time estimate indicating how long the process takes to complete if run in final mode.
The report that is produced that provides a list of those documents meeting the defined criteria. These are all of the documents that should be transferred.

11. DTU - Preliminary vs Final Transfer Modes
Process Status indicating at least one document selected for the process encountered an error.
To review which document(s) encountered an error, click on Error.
We’ll see a sample error later in this presentation.

12. DTU - Preliminary vs Final Transfer Modes
The ability to run this process in Final Mode without having to select the documents and the source/target database connections again.
The [Run Final Mode] button exists only in the Preliminary Mode Process Status Report window for processes that are in a "Complete" or "Complete with Errors" status.

13. DTU - Preliminary vs Final Transfer Modes
In Final mode, selected records are transferred to the target database and a report is produced listing those transactions that were moved.
In Final mode, the documents are inserted in the receiving database within a folder. This folder is titled as the document number with the latest creation date that is found in the transfer list.
Each transferred chain resides in its own folder in the target database.
All the folders that are created by the Document Transfer Utility in the receiving database reside within the team cabinet that is selected in the Select Team Cabinet from Target drop-down list box. Selecting the Team Cabinet is on a per process basis, so all documents in a given process are transferred to folders that reside within that Team Cabinet.

14. DTU - Preliminary vs Final Transfer Modes
Each transferred chain resides in its own folder in the target database. The cabinet in which the folder is created is selected in the Team Cabinet from Target drop-down list box.

15. DTU - Preliminary vs Final Transfer Modes
The following data is not transferred from one PD² database to another with the document chain:
Workload Assignments
Folders
Audit Tables
Multiple Contract Modifications
Tracking Codes
Document Codes
Line Item, Document, and Vendor Local Information

16. Process Status
Click the Process ID to open the Process Status Report window.

17. Types of Process Statuses
COMPLETE – the process has finished and no errors were encountered.
COMPLETE W/ ERRORS - the process has finished but at least one document selected for the process encountered an error.
SCHEDULED - the process is scheduled to run at a future time.
IN PROGRESS - the process is currently running.
CANCELLED - the process has been cancelled and will not run.
FAILED - the process was interrupted because of a fatal exception such as a power failure event or a database crash event.
ROLLED BACK - the process has been cleaned up by manually running the cleanup process on a failed process or automatically by the Document Transfer Utility when it encounters a validation error.

18. Process Status
Click the ERROR link to view the detailed error message.

19. Process Status

20. Cleanup Service
If there is a validation error, a cleanup service is automatically invoked that deletes all the documents that were inserted from the target and restore access to all the documents in the source.
There is also an option to manually run the cleanup service for failed processes. The cleanup option is only available for failed processes.
Select the target database where the cleanup is to be run against.
DTU then deletes the records from the target and resets the status in the source so that the documents are usable.
The status of the process is updated to “ROLLED BACK”.
When initiating the transfer in preliminary or final mode, the system displays any errors that prevent a transfer from being successfully completed.

21. Cleanup Service
When the DTU fails and part of the document chain is NOT transferred into the RECEIVING database, the RECEIVING database must be cleaned up before attempting the transfer again. The Help Desk has a script that can assist with this:
P_DSK_dtu_cleanup_delete_doc_chain_by_obj_id - C01681
For the documents that did transfer to the RECEIVING database, the SOURCE database document chain transfer flags were set to 1 and must be reset to 0 before attempting the document transfer again. The Help Desk has a script that can assist with this:
P_DSK_dtu_source_db_doc_chain_reset_by_obj_id - C01680

22. Moving on to the PD² Archiving Utility…

23. PD² Archiving v6.0
The purpose of the PD² Archiving Utility is to provide the ability to archive data from the PD² database.
The PD² Archiving Utility offers the following capabilities:
Move data from the PD² production database to its companion PD² storage database.
Restore data from a PD² storage database to its companion PD² production database.
Archive data from the PD² storage database to XML files.
Restore data from the archived XML files to the PD² storage database.
Purge documents from the PD² storage database.
These processes allow for documents to be moved off-line to provide for better production performance, allow for increased database space.
The storage database serves as a staging area for documents that may later be archived or destroyed.

24. PD² Archiving v6.0
During installation, a companion storage database is created for the site's production database.
There can only be one storage database for any production database and each storage database can only point to one production database.
The databases are linked during installation and are considered a pair when running or scheduling any processes.
The storage database has the same database and table structure as the production database so when documents are moved the data is cut from production and pasted to storage. This allows for the seamless restoring of data.
Moving forward, when the production database is upgraded the storage database is upgraded as well to ensure they are in synch.

25. PD² Archiving v6.0 – Archive Eligible Date
Archive Eligible Date in the PD² application
Not required in order to archive a document, nor does the system stop a document from being archived prior to the date that is set.
Is a way for the end user to specify that a certain document is available to be moved to storage on the given date.
In the PD² Archiving Utility, documents can be searched for based on the archive eligible date.
The Archive Eligible Date can only be populated on awards, agreements, orders, and modifications.
Setting Archive Eligible Date in the PD² application
After logging into PD², Highlight a document.
From the menu select Procurement  Archive Eligible Date.
Enter a date in the specified format and click [OK].

26. PD² Archiving v6.0 – Five Processes
Move from Production to Storage
Provides the ability for a site to move documents from the production database to its companion storage database.
Restore from Storage to Production
Restores data from the storage database back to production. This process restores the records to production and deletes them from storage.
Move from Storage to Archive
Provides the ability to archive data from the storage database to a specified archive location.
Creates XML files containing the data of the documents being archived, places the files in a designated archive location, and removes the data from the storage database.

27. PD² Archiving v6.0 – Five Processes
Restore from Archive to Storage
Provides the ability to restore archived XML files to the storage database.
Restores a single contract/document file at a time as selected by a System Administrator.
Destroy from Storage
Provides the ability for sites to purge documents within the storage database when it is determined that their retention period has expired.
The selected document chains are permanently deleted from the storage database.

28. PD² Archiving v6.0 – Preliminary vs Final Modes
Preliminary mode allows sites to run the process without actually deleting records from the source database.
A report is produced that provides a list of those documents meeting the defined criteria.
This provides the ability for a System Administrator to review the list of potential documents moving to archive, storage or production prior to the action taking place.
Final mode moves data and also generates a status report with any applicable errors.
In Final mode, any selected records are deleted from the source database and a report is produced listing those documents that were removed.

29. JDBC Adapters Page

30. PD² Archiving v6.0

31. PD² Archiving v6.0 – View Process Status

32. PD² Archiving v6.0 – View Process Status

33. PD² Archiving v6.0 – View Process Status

34. PD² Archiving v6.0 – Initiate Transfer

35. PD² Archiving v6.0 – Initiate Transfer
A default reason may be specified and is captured in the storage database for all non-closed out awards, orders and agreements that are archived as part of the configured process. The default reason is not required.
When selecting the Restore from Storage to Production option, The System Administrator is required to select the appropriate team cabinet from the Restore to Team Cabinet drop-down list box.

36. PD² Archiving v6.0 – Initiate Transfer
Notes on folders:
The PD² Archiving Utility does not move folders to the storage database.
Folders are automatically deleted from the production user’s desktop when all documents in the folder have been moved to storage.
There is not a need for folders in the storage database as the database does not have a front end to view documents.

37. PD² Archiving v6.0 – Initiate Transfer
All offices and not just offices designated as Issuing Offices display in this list because PD² allows any office type to be selected into the Issue Office block of the standard forms.

38. PD² Archiving v6.0 – Initiate Transfer

39. PD² Archiving v6.0 – Initiate Transfer

40. PD² Archiving v6.0 – Reviewing the Transfer