Primary Storage Manager Carlton Doe & Scott Pickett NA Informix Technical Sales WW Informix Technical Sales For questions about this presentation contact: or © 2014 IBM Corporation

Primary Storage Manager (PSM) Faster and more functional than ISM  Easy way to configure OnBar to take backups and restores  Easy way for embedded solutions to take parallel backups using OnBar  Easy way for Ontape customers to migrate to OnBar  Replaces ISM © 2014 IBM Corporation2

Primary Storage Manager  A brief history of Informix backup utilities –ontape For years, the only backup/restore (B/R) available Operated serially Supported a single device (tape or disk) for backing up the instance and logical logs In Informix 11, intelligent file naming for instance backups to disk to prevent overwriting a previous backup  Fixed with the introduction of backups to directories and the IFX_ONTAPE_FILE_PREFIX parameter  ontape worked well when database sizes were small but as they grew, Informix implemented a two-tier solution that provided more options –More sophisticated backup devices Disk, tape, jukeboxes –The ability to backup multiple dbspaces in parallel © 2014 IBM Corporation3

Primary Storage Manager  The first tier – ON-Bar –Informix’s implementation of the XBSA API defined by the X/Open organization An open interface between database servers and tape management systems to support B/R operations  Contains a number of services and components to support B/R tasks including communication, read/write tasks, metadata creation/searching etc. –ON-Bar is the pipe that moves data to / from the instance and the backup software Is used to connect to backup solutions like Tivoli Storage Manager  The second tier – the Informix Storage Manager (ISM) –A limited use and functionality backup solution included with the engine –A licensed port of a (now very old) B/R product –For customers who needed something between ontape and a full-fledged tape management system © 2014 IBM Corporation4

Primary Storage Manager  The ISM had a number of limits and disadvantages –Only 4 “simple” backup devices (tape or disk) directly connected to the physical server –Difficult to configure and administer –Code line was difficult to maintain since it was third-party and very, very old –IBM had to pay a license fee to use this ancient, difficult to maintain code  With the rapid decrease in disk-based storage, most small-to-mid sized customers are now backing up to disk –No longer need ISM with its problems –Need a faster, parallelized solution to get instance data to disk  Solution – The Primary Storage Manager (PSM)! © 2014 IBM Corporation5

Primary Storage Manager  The PSM –Replaces the ISM –Has better integration with the ON-Bar API –Is simple to configure Only 1 parameter at its most basic well maybe 2 –Is simple to administer using the onpsm utility –Provides a parallelizable backup/recovery stream to a disk device –Provides many of the features of a more full-featured tape management system Ability to define storage pools, including devices, sizes and priorities Expire and prune backups  Using the onsmsync utility Some export capabilities to perform a remote restore Object and meta data verification, locking and unlocking Supports optional backup compression and/or encryption  Through the BACKUP_FILTER parameter © 2014 IBM Corporation6

Primary Storage Manager  Components of the PSM – The XBSA library Single shared library, called dynamically by ON-Bar operations Default location Linux/Unix/Mac: $INFORMIXDIR/lib/libbsapsm. Windows: %INFORMIXDIR\bin\libbsapsm.dll – The onpsm utility CLI-based front end tool to administer objects in the storage manager – The PSM catalogs C-ISAM tables stored in $INFORMIXDIR/etc/psm Repository of information for all backups, devices, pools, etc. © 2014 IBM Corporation7

Primary Storage Manager  Components of the PSM (cont) –The ixbar. file (aka ON-Bar emergency boot file) Contains information to execute a cold restore What was backed up and when – The onsmsync utility Used to synch information between the sysutils database, the ixbar file and the PSM catalog repository © 2014 IBM Corporation8

Primary Storage Manager  There is very little to configure for the PSM to work – Required Create one (or more) directories where instance and logical backups will be created Can have different locations for logical logs and instance backups Fixed sizes can be assigned, as shown later, to prevent overflowing the file system Configure the PSM_DEVICE parameter with one fully pathed directory Default location for instance and logical backups Can be over-ridden with additional configuration – Optional, though highly recommended Configure the BAR_BSALIB_PATH parameter to point to the XBSA library Execute backups!!! © 2014 IBM Corporation9

Primary Storage Manager  There is very little to configure for the PSM to work (cont.) – It is possible to execute a backup without any configuration at all – Backups would be created in $INFORMIXDIR/backups Probably not the best location Would want to specify one or more locations via onpsm operations Full syntax discussed later – From a minimalist perspective, there is one optional, though highly recommended initial configuration setting Configure the BAR_BSALIB_PATH parameter to point to the XBSA library © 2014 IBM Corporation10

Primary Storage Manager  Additional configuration parameters to consider –PSM_ACT_LOG – log for PSM activities Not much is recorded here –PSM_DEBUG_LOG – debug file location –PSM_DEBUG – used to set the level of debug messages 0 (off) to 9 (high) –PSM_CATALOG_PATH – location of the PSM catalog repository if not in the default location of $INFORMIXDIR/etc/psm –PSM_DBS_POOL – the name of the storage pool for instance backups Default name: DBSPOOL Unique names are strongly encouraged if multiple instances are active on a server –PSM_LOG_POOL – the name of the storage pool for instance backups Default name: LOGPOOL Unique names are strongly encouraged if multiple instances are active on a server © 2014 IBM Corporation11

Primary Storage Manager  Additional configuration parameters to consider (cont.) –BAR_MAX_BACKUP – Controls the number of parallel data streams from the instance to the PSM Regardless of this value, only one output file for each space is created  Illustrated later –BAR_XFER_BUF_SIZE – Can be used to set a custom data transfer buffer size –LTAPEDEV – can not be set to /dev/null if backing up with ON-Bar Some restore operations, including specific moments in time, may require logical log restore operations © 2014 IBM Corporation12

Primary Storage Manager  Three interesting quirks to the PSM –First, critical instance files are backed up as part of an instance backup operation The ixbar.n, $ONCONFIG, sqlhosts, oncfg files Stored in a separate directory structure, shown later –Second, the PSM organizes data differently As administrators, used to dealing with backup “levels”  0  1  2 While the PSM supports these levels, the PSM organizes the backup output and meta data by “generation” and “descendants”  Some administrative operations affect entire generations >Such as export or expire –Third, the PSM uses a single name for all output objects, consists of Instance name Descendant version Part number Example: inst_2.2.1 © 2014 IBM Corporation13

Primary Storage Manager  In a newly created and populated instance, suppose these actions occur: – A L0 backup is created The PSM considers this the beginning of a new generation – A L2 backup is created The PSM considers this a “first descendant”  Is stored in a different location than the previous L0 backup – A L1 backup is created The PSM considers this as a “first descendant” as well  Is stored in the same location as the previous L2 backup The L2 descendant is NOT deleted  In order to support point-in-time restore – A L2 backup is created The PSM considers this a “second descendant”, linked to the previous L1  Is stored in a different location than the L1 and earlier L2 backup >It is a different descendant class than the earlier L2 © 2014 IBM Corporation14

Primary Storage Manager  Let’s see this in action with the inst_2 instance –The DBSPOOL is configured to /opt/IBM/informix/backups –First backup (L0) occurs at 5:13 pm Directory automatically created for the inst_2 instance objects Directories created for the instance spaces. The “2” directory contains logical log backups. Labeled “2” because it’s DBSERVERNUM of instance © 2014 IBM Corporation15

Primary Storage Manager  Looking at the directory for the rootdbs – A single directory for new backup generations is created – Populated with the L0 backup of the space For new generations Backup of the dbspace. Notice the file name. © 2014 IBM Corporation16

Primary Storage Manager  Look in more detail at what happens in the data_space_1 directory Output created by L0 backup in the new generation subdirectory for this space. Notice the output file name is identical to that in the rootdbs directory structure. © 2014 IBM Corporation17

Primary Storage Manager  Look in more detail at what happens in the data_space_1 directory (cont.) – Data changes occur in the dbspace – L2 backup executed at 5:22 pm Output created is created in the FIRST descendant directory. Notice the file name. Nothing in the SECOND descendant directory © 2014 IBM Corporation18

Primary Storage Manager  Look in more detail at what happens in the data_space_1 directory (cont.) – More data changes occur in the dbspace – L1 backup executed at 5:26 pm Additional output created is created in the FIRST descendant directory. Notice the file name. It’s the second version of a first descendant. Still nothing in the SECOND descendant directory © 2014 IBM Corporation19

Primary Storage Manager  Look in more detail at what happens in the data_space_1 directory (cont.) – Still more data changes occur in the dbspace – L2 backup executed at 5:28 pm Nothing new added to the FIRST descendant directory Finally, output in the SECOND descendant directory. Notice the file name. © 2014 IBM Corporation20

Primary Storage Manager  Conceptually this makes sense – To restore to very beginning: Only need the content of the new generation (aka 0) directories. – To restore to the first L2 backup (and changes in the data_space_1 space): Need the new generation AND The first descendant with the L2 backup. – To restore to the L1 backup Need the new generation AND Plus the second version of the first descendant with the L1 backup: Supersedes the L2 backup. – To restore to the last L2 backup Need the new generation, The second version of the first descendant with the L1 backup AND The second descendant with the L2 backup: Because the L2 backup contains changes occurring after the L1 backup. © 2014 IBM Corporation21

Primary Storage Manager  One last thing – logical log backups – Are created under the DBSERVERNUM directory – Each log has its own directory (named by log_id ) with the backup of that log © 2014 IBM Corporation22

Questions © 2014 IBM Corporation23

Primary Storage Manager  Let’s look at some of the other PSM components –The $INFORMIXDIR/etc/ ixbar.n file contains a list of every dbspace backed up for every operation: –n is the DBSERVERNUM © 2014 IBM Corporation24

Primary Storage Manager  Let’s look at some of the other PSM components (cont.) –If ixbar doesn’t exist when the first ON-Bar / PSM backup is executed, it is automatically created as part of the operation No ixbar file It is created and the backup continues © 2014 IBM Corporation25

Primary Storage Manager  Let’s look at some of the other PSM components (cont.) – The PSM C-ISAM catalog tables are in $INFORMIXDIR/etc/psm This data is synchronized with the sysutils database © 2014 IBM Corporation26

Questions © 2014 IBM Corporation27

Primary Storage Manager  The onpsm utility is used to manage backup devices and the files created by backup operations – Can add and delete “devices” or output locations – Can check the consistency of catalog tables Can also delete the catalog tables – Can unlock the catalog tables if a B/R operation terminates abnormally – Can list or delete backup files created by backup operations Not recommended as the primary method of deleting backups as shown later – Main syntax tree onpsm -C – check consistency of (or delete) the PSM catalog tables -D – manage backup devices and pools -L – manage locks on the PSM catalog tables -O – list (or delete) backup files created by a backup operation © 2014 IBM Corporation28

Primary Storage Manager  onpsm - device management -D – Parameters for adding a device or backup location Only one device in a pool can be classified with a HIGH priority  If multiple devices are configured with LOW, the actual device used for a backup operation will be randomly selected For tape sizes, 0 (zero) can be used to indicate “unlimited” or write until the device returns a “media full” error.  Other sizes can be entered using the B, K, M, G, T, or P suffix (for bytes, kilobytes, megabytes, gigabytes, terabytes, or petabytes) add full_path -p [ HIGH | LOW ] – the priority for the location -g pool_name -t [ FILE | TAPE ] -s size – optional, maximum output size if using a tape -b block_size – optional, the block size for the tape device © 2014 IBM Corporation29

Primary Storage Manager  onpsm - device management -D – Example, as shown earlier © 2014 IBM Corporation30

Primary Storage Manager  onpsm - device management -D – Use list to list configured devices © 2014 IBM Corporation31

Primary Storage Manager  onpsm - device management -D – Use list to list configured devices (cont.) With the –u (lowercase) file_name parameter, can export the device configuration to a named flat file With the –l (lowercase) file_name parameter, an exported list of devices can be imported into the PSM for use © 2014 IBM Corporation32

Primary Storage Manager  onpsm - device management -D – To delete a device from a pool del device_path –g pool_name [ -y ] – To modify device information update device_path -p [ HIGH | LOW ] – optional, if needed to change device prioritization -s size – optional, if needed to change tape device size With the B, K, M, G, T, or P suffix © 2014 IBM Corporation33

Primary Storage Manager  onpsm – backup object management -O (“oh”, uppercase) – Use the list parameter to show objects backed up © 2014 IBM Corporation34

Primary Storage Manager  onpsm – backup object management -O (“oh”, uppercase) –One or more backup objects can be deleted from disk using the del parameter This does NOT refresh information in the ixbar file, it still thinks the object is there –For example, in this list object 85 is the rootdbs for a third generation L0 backup Is also the last generation created Notice the file name – inst_2.3.1 © 2014 IBM Corporation35

Primary Storage Manager  onpsm – backup object management -O (“oh”, uppercase) – Delete syntax del –o object_id [ -y ] lowercase “oh” – New list shows the object is deleted © 2014 IBM Corporation36

Primary Storage Manager  onpsm – backup object management -O (“oh”, uppercase) – The file was deleted from disk as part of the operation No third generation output file © 2014 IBM Corporation37

Primary Storage Manager  onpsm – backup object management -O (“oh”, uppercase) – The ixbar file still contains a reference to the now missing backup object © 2014 IBM Corporation38

Primary Storage Manager  onpsm – backup object management -O (“oh”, uppercase) – Attempting to execute a cold restore fails without the rootdbs backup file © 2014 IBM Corporation39

Primary Storage Manager  onpsm – backup object management -O (“oh”, uppercase) – The only way, in this case, to successfully restore the system is to use point- in-time syntax to execute a second (or first) generation restore – Take away from the onpsm –O del discussion Should only be used for objects that have already been deleted (or expired) from the ixbar file but somehow didn’t get properly removed from disk Use the “expiration” functionality in onsmsync to properly remove “old” backups © 2014 IBM Corporation40

Primary Storage Manager  The onsmsync utility is used to perform several operations – Synchronize: The sysutils database The ixbar file The PSM catalog tables – Expire old backups Will delete the backup files (on disk) Prune the ixbar and catalog / sysutils tables as needed – Can recreate a missing or damaged ixbar file so a restore can occur – Can “export” a backup so it can be used to restore on a different server – Can “import” a backup so it can be used in a restore operation © 2014 IBM Corporation41

© 2013 IBM Corporation42 The Board of Directors of the International Informix Users Group (IIUG) announce the: 2014 IIUG Informix Conference 2014 IIUG Informix Conference April 27 – May 1, 2014 J.W. Marriott Hotel (Brickell) Miami, Florida, USA J.W. Marriott Hotel (Brickell) For more details visit the official conference web site Register before February 28, 2014 and save $ – See you in Miami! (remember IIUG members save an additional $100!) Attention Speakers We are now accepting Presentation Proposals thru Nov. 20, Details at the conference speaker page: All Non IBM speakers selected receive a Complimentary Conference pass!! !

Questions © 2014 IBM Corporation43

Logo © 2014 IBM Corporation44

Logo © 2014 IBM Corporation45