FlashCopy in a zSeries Environment Charlie Burger Storage Systems Advanced Technical Support 11/28/2018

Table of Contents FlashCopy Overview FlashCopy DS6000 & DS8000 How does FlashCopy work? FlashCopy Implementation Incremental FlashCopy Persistent FlashCopy Consistency Group FlashCopy Inband FlashCopy FlashCopy Nocopy to Copy Multiple Relationships Fast Reverse Restore Enabled and Fast Reverse Restore FlashCopy and DFSMSdss FlashCopy and TSO FlashCopy and ICKDSF Performance Considerations Addendum

FlashCopy Overview ® “Instant” T0 (Time 0) copy Source and target volumes immediately available for processing with full read/write access A hardware solution invoked by software z/OS DFSMSdss, TSO, ICKDSF, API, DS CLI, DS Storage Manager (GUI) z/VM and z/VSE ICKDSF, host commands, DS CLI, DS Storage Manager ® ® ®

FlashCopy – DS6000 and DS8000 All FlashCopy functions shipped with PTC feature Full volume FlashCopy (BACKGROUND COPY or NOCOPY) Incremental FlashCopy Inband FlashCopy Consistency Group FlashCopy FlashCopy NOCOPY to COPY Persistent FlashCopy Fast Reverse Restore Multiple Relationships Data Set FlashCopy (z/OS and z/VSE only) Target volume can be within any LSS within the same physical disk subsystem Target volume can be a Metro Mirror or Global Copy primary volume Must use parameters in the command that indicate you know that it is a primary Duplex volume falls into copy pending Target cannot be a primary in a Global Mirror session Functions can be combined…. Example: Inband, Incremental, Consistency Group

How does FlashCopy work? Request copy from source to target FlashCopy relationship created between the volumes Target is available for processing once the relationship is created BACKGROUND COPY Tracks are copied from the source to the target Attempts to read/write data already copied proceed as normal Attempts to read a target track not already copied intercepted and data obtained from source Attempts to write a source track not already copied intercepted and source track copied to target before update occurs BACKGROUND NOCOPY Attempts to read a target track not copied intercepted and data obtained from source Attempts to write a source track not copied intercepted and source track copied to target before update occurs Once the relationship is established, the source and target volumes are available for read/write operations. You do NOT have to wait for a background copy to complete in order to use the source/target. You do have to wait for the background copy to complete to reverse an incremental flash copy.

FlashCopy Implementation – BACKGROUND COPY Source Target Time Copy data command issued Copy immediately available Source Writes Target Read and write to both source and target possible Background copy is the default. Incremental FlashCopy requires BACKGROUND COPY. When copy is complete, relationship between source and target ends BACKGROUND COPY

FlashCopy Implementation – BACKGROUND NOCOPY Source Target Time Copy data command issued Copy immediately available Read and write to both source and target possible Source Writes Target For most situations, we recommend BACKGROUND NOCOPY for performance reasons. Global Mirror FlashCopy requires BACKGROUND NOCOPY. If all of the tracks have not been copied and the relationship is withdrawn, the target data is invalid. Relationship between source and target exists until withdrawn or all tracks copied BACKGROUND NOCOPY

Incremental FlashCopy ‘Change Recording’ keeps track of changes made to source and target volumes after establishment of FlashCopy relationship Use ‘Change Recording’ along with BACKGROUND COPY and PERSISTENT Supported only at full volume/LUN level There can only be one incremental relation per volume but can coexist with other non-incremental relationships During refresh: To maintain the incremental relationship, specify ‘Change Recording’ on each incremental FlashCopy Only changed data is copied in the background Previous increment BACKGROUND COPY does not have to complete before new increment is taken if the FlashCopy is in the same direction A new FlashCopy increment can be performed in the reverse direction Previous incremental BACKGROUND COPY going in the opposite direction must complete before performing an incremental in the other direction Once the pairs are established, both volumes are available for read/writes. Invoking with TSO forces inhibit target write.

Incremental FlashCopy Example Source Target Establish FlashCopy A>B Source Target Both A and B are updated Source Target Resynchronize: B becomes an exact copy of A with new updates from A and updated tracks on B overlaid from A Once the pair is established in the reverse direction, both volumes are available for reads/writes. If you reverse the incremental, what was the source becomes the target and what was the target becomes the source. or Source Target Reverse the increment and A becomes an exact copy of B with new updates from B and overlay updated tracks on A from B

Persistent FlashCopy Time Source Target Source Target Source Target FlashCopy command issued with BACKGROUND COPY and PERSISTENT BACKGROUND COPY Source Target When BACKGROUND COPY is complete, relationship between source and target persists The reason this function was developed was to allow customers to flash a volume using BACKGROUND COPY and after the COPY completed, maintain the relationship so that the source volume could be queried and the target identified. This was beneficial for customers who would flash the source to target1 on one night and target2 the next night. By being able to do the query, an accidental overlay of the most recent target is greatly reduced. Relationship exists until specifically withdrawn. Persistent relationships are used for Incremental FlashCopy and Revertable FlashCopy. Source Target Explicitly WITHDRAW the relationship

Consistency Group FlashCopy Hold off initiation/completion of write I/O to the source volumes until FlashCopy establish is completed Select source and target volumes with ‘freeze’ option Create Consistency Group Created command allows resumption of I/O One per LSS Enables creation of a consistent point-in-time copy across multiple volumes with minimum host impact Target of each source volume is within one physical disk subsystem but source volumes within a consistency group can span physical disk subsystems

Consistency Group FlashCopy - Example Source Target FlashCopy S1 to T1 Writes cannot proceed on S1 Any writes occurring on S2-S4 are not dependent writes FlashCopy S2 to T2 Writes cannot proceed on S1 or S2 Any writes occuring on S3-S4 are not dependent writes FlashCopy S3 to T3 and S4 to T4 T1-T4 contain a consistent copy Issue THAW Writes may proceed on S1-S4 S1 T1 Source Target S2 T2 Source Target S3 T3 Source Target S4 T4

Inband FlashCopy FlashCopy commands are issued to the primary volume of a PPRC pair at the local site PPRC pair acts as a conduit to the remote site for execution of the command at the remote site All FlashCopy functions can be invoked Inband except for multiple relationships and data set FlashCopy Volume at remote site that executes the command must be a PPRC secondary For Consistency Group FlashCopy, you can’t thaw using Inband. It will automatically thaw after a default of 120 seconds. A B C PPRC PPRC Secondary FlashCopy Source FlashCopy Target Tertiary PPRC Primary Host I/O Local Site Remote Site

FlashCopy NOCOPY to COPY Time Source Target FlashCopy command issued with NOCPY Change BACKGROUND from NOCOPY to COPY BACKGROUND NOCOPY Source Target When BACKGROUND COPY is complete, relationship is withdrawn BACKGROUND COPY

Multiple Relationships Target A source volume can participate in multiple FlashCopy relationships One source to up to 12 targets A target can only have one source A target cannot be a source at the same time Only one of the 12 relationships can be an incremental A volume, LUN, or data set can only be a source or target at any given time However, with data set FlashCopy, data set source and target can reside on the same volume T1 Source Target S1 T2 Target T3

FRR Enabled FlashCopy Establish FlashCopy A>B Source Target Establish FlashCopy A>B Persistent Change recording Background nocopy Inhibit target writes Source Target Updates to A causes the A track to Be copied to B prior to the update Unlike Incremental FlashCopy, only Tracks updated on A are copied to B The grids are not meant to represent bit maps but rather tracks on the volumes that have or have not been physically updated. The B volume has the tracks physically written to it that are required to be copied back to A to restore the A volume as it was before being updated. If the relationship were withdrawn now, the B volume could not be used. This form of FlashCopy is used at the secondary site for Global Mirror. When used in the Global Mirror environment, the FlashCopy volumes are typically referred to as the B and C volumes. It can also be used outside of the Global Mirror environment. For example, a user has a batch job where they wish to FlashCopy their source volumes to create a backup prior to submitting the batch job. If the job fails, a Fast Reverse Restore could be done. If the user waited for the background copy to complete from the Fast Reverse Restore, A could be flashed again to B. By being BACKGROUND NOCOPY, Fast Reverse Restore can be done immediately rather than waiting for a BACKGROUND COPY to complete before reversing an Incremental FlashCopy

FlashCopy Fast Reverse Restore Source Target Stop updates to the A volume Source Target Perform a Fast Reverse Restore B>A to create consistent data on the A volume Fast Reverse Restore reverses the direction of the FlashCopy relationship when FRR Enabled FlashCopy is being used. This restores the source volume to the state it was in when last flashed to the target. Changed tracks are copied back from the target to the source. Fast Reverse Restore converts the A > B background nocopy operation to a B > A background copy operation. Once the background copy completes the relationship is withdrawn. Source Target Once the background copy B>A is complete, Flash A back to B

FlashCopy and DFSMSdss FlashCopy BACKGROUND COPY and NOCOPY NOCOPY to COPY Incremental FlashCopy Consistency Group FlashCopy Data Set FlashCopy

DFSMSdss and FlashCopy Default is to use FlashCopy if possible FASTREPLication(PREFerred | REQuired | NOne DFSMSdss checks if source and target are eligible for FlashCopy FlashCopy eligible volumes preferred for new target data set allocation If not, DFSMSdss will do the copy using normal I/O Source and target volumes MUST be online DFSMSdss performs serialization Once the FlashCopy relationship is made, the DFSMSdss job completes It does not wait until the background copy is physically complete DFSMSdss performs BACKGROUND(COPY) by default Once the BACKGROUND(COPY) is physically complete, the relationship between source and target is withdrawn BACKGROUND(NOCOPY) invoked with FCNOCOPY on COPY command FCWITHDRAW specified on DUMP command will withdraw a relationship You can specify COPYVOLID or leave it out DFSMSdss dump conditioning APAR OW45674 allows this for SMS or non-SMS volumes Use DEBUG(FRMSG(MINimal | SUMmarized | DETAILED)) to determine why a FlashCopy was not invoked

DFSMSdss FlashCopy Example DFSMSdss command //COPYFULL JOB ..... //* //FC EXEC PGM=ADRDSSU //DASDI1 DD UNIT=3390,VOL=SER=SORCV1,DISP=SHR //DASDI2 DD UNIT=3390,VOL=SER=SORCV2,DISP=SHR //DASDO1 DD UNIT=3390,VOL=SER=TARGT1,DISP=SHR //DASDO2 DD UNIT=3390,VOL=SER=TARGT2,DISP=SHR //SYSPRINT DD SYSOUT=* //SYSUDUMP DD SYSOUT=V,OUTLIM=3000 //SYSIN DD * PARALLEL COPY FULL INDD(DASDI1) OUTDD(DASDO1) ADMIN DUMPCONDITIONING COPY FULL INDD(DASDI1) OUTDD(DASDO2) ADMIN DUMPCONDITIONING /*

FlashCopy Results for DFSMSdss - COPYVOLID OFFLINE VOL001 VOL001 VTOC VTOC SYS1.VTOCIX.VOL001 SYS1.VTOCIX.VOL001 SYS1.VVDS.VVOL001 SYS1.VVDS.VVOL001 Benefits: Volser and VVDS are in sync Individual VSAM data sets can be RESTOREd Data sets are cataloged if their catalogs are available at time of RESTORE Issues: Duplicate volser Volume must be dumped from another MVS image

FlaschCopy Results for DFSMSdss - No COPYVOLID - No DUMPCONDITIONING VTOC VTOC SYS1.VTOCIX.VOL001 SYS1.VTOCIX.VOL002 SYS1.VVDS.VVOL001 SYS1.VVDS.VVOL001 Benefits: Volume remains online to system performing COPY and therefore can be DUMPed by that system Can RESTORE entire volume and clip volser back to the source volser Issues: Volser and VVDS are not in sync VSAM data sets on target volume are inaccessible

FlashCopy Results for DFSMSdss - No COPYVOLID - DUMPCONDITIONING VTOC VTOC SYS1.VTOCIX.VOL001 SYS1.VTOCIX.VOL001 SYS1.VVDS.VVOL001 SYS1.VVDS.VVOL001 Benefits: Volume remains online to system performing COPY and therefore can be DUMPed by that system Can RESTORE entire volume and since DUMPCONDITIONING was used, will be RESTOREd as VOL001 Will be able to RESTORE individual VSAM data sets from a full volume DUMP of the target volume Issues: Volser and VVDS are not in sync VSAM data sets on target volume are inaccessiblee

Sample Procedure Tape Tape COPY FULL INDD(SRCVOL) OUTDD(TGTVOL) DUMPCONDITIONING FCNOCOPY VOL002 VOL001 Relationship VOL001 VTOC VTOC SYS1.VTOCIX.VOL001 SYS1.VTOCIX.VOL001 SYS1.VVDS.VVOL001 SYS1.VVDS.VVOL001 DUMP FULL INDD(TGTVOL) OUTDDNAME(TAPE1) FCWITHDRAW VOL001 VOL002 Relationship VOL001 VOL001 SYS1.VTOCIX.VOL001 SYS1.VVDS.VVOL001 VTOC VTOC Tape SYS1.VTOCIX.VOL001 SYS1.VTOCIX.VOL001 SYS1.VVDS.VVOL001 SYS1.VVDS.VVOL001 RESTORE INDD(TAPE) OUTDD(SRCVOL) VOL001 VOL001 SYS1.VTOCIX.VOL001 SYS1.VVDS.VVOL001 VTOC Tape SYS1.VTOCIX.VOL001 SYS1.VVDS.VVOL001

DFSMSdss NOCOPY to COPY New FCNOCOPYTOCOPY parameter to change a BACKGROUND NOCOPY to BACKGROUND COPY

DFSMSdss Incremental FlashCopy FCINCRemental Establish the initial Incremental FlashCopy relationship Refresh the existing Incremental FlashCopy relationship Reverse the direction of the FlashCopy relationship and restore (copy) the changed tracks Swap the original INDD/INDYNAM and OUTDD/OUTDYNAM FCINCREMENTALLAST Refresh the existing Incremental FlashCopy relationship and then end Reverse the direction of the FlashCopy relationship and restore (copy) the last (final) increment FCINCRVERIFY(REVERSE | NOREVERSE) Optionally verify the FlashCopy direction before reversing it FCWAIT(numsecs,retries) Time in seconds to wait before checking for physical background copy completion Number of additional queries to make on physical background completion DFSMSdss will not target inhibit write.

FCINCRemental & FCINCREMENTALLAST Do not specify with: CONCURRENT DATASET FASTREP(PREFERRED | NONE) FCNOCOPYTOCOPY OUTTRACKS Ignored with: ALLDATA ALLEXCP

DFSMSdss Consistency Group FlashCopy FCCGFREEZE Source volume placed in ‘extended long busy for consistency grouping’ when relationships established I/O activity held until CGCREATE command issued to LSS or Consistency Group timer in LSS expires CGCREATE ACCESSVOLume(volser,) Signals to an LSS that the creation of consistent data spanning a group of volumes has completed ACCESSVOLume is used to communicate to the LSS Volume must be mounted and online FCCGVERIFY(volser) Option on the CGCREATE command Verification volume used to check the state of the Consistency Group Usually first source volume established in Consistency Group When LSSs have different Consistency Group timer values, specify volser of a volume in the LSS with smallest Consistency Group timer value

FCCGFREEZE Do not specify with: CONCURRENT DATASET FASTREP(PREFERRED | NONE) FCNOCOPYTOCOPY

DFSMSdss Consistency Group FlashCopy Example //COPYCGRP JOB ..... //* //FC EXEC PGM=ADRDSSU //SRC101 DD UNIT=3390,VOL=SER=SRC101,DISP=SHR //SRC102 DD UNIT=3390,VOL=SER=SRC102,DISP=SHR //SRC203 DD UNIT=3390,VOL=SER=SRC103 //TGT101 DD UNIT=3390,VOL=SER=TGT101,DISP=SHR //TGT102 DD UNIT=3390,VOL=SER=TGT102,DISP=SHR //TGT103 DD UNIT=3390,VOL=SER=TGT203 //SYSPRINT DD SYSOUT=* //SYSIN COPY FULL INDYNAM(SRC101) OUTDYNAM(TGT101) ADMIN DUMPCOND FCCGFREEZE PARALLEL COPY FULL INDYNAM(SRC102) OUTDYNAM(TGT102) ADMIN DUMPDOND FCCGFREEZE COPY FULL INDYNAM(SRC203) OUTDYNAM(TGT203) ADMIN DUMPCOND FCCGFREEZE SERIAL CGCREATE FCCGVFY(SRC101) ACCVOL(SRC101,SRC203)

FlashCopy and TSO FlashCopy BACKGROUND COPY and NOCOPY NOCOPY to COPY FCESTABL SDEVN(X’uadd’) TDEVN(X’uadd’) FCESTABL SDEVN(X’uadd’) TDEVN(X’uadd’) MODE(NOCOPY) NOCOPY to COPY FCESTABL SDEVN(X’uadd’) TDEVN(X’uadd’) FCNOCOPYTOCOPY Incremental FlashCopy Consistency Group FlashCopy Inband FlashCopy FRR Enabled FlashCopy FlashCopy Fast Reverse Restore (FRR) FlashCopy Query

TSO and FlashCopy TSO does NOT perform any serialization TSO FCESTABL source and/or target can be offline TSO FCESTABL uses BACKGROUND(COPY) by default TSO FCESTABL has no COPYVOLID option Copies entire volume by default so source volser is copied to target If you wish to FlashCopy to a different volser and keep that target volser, FlashCopy extents excluding cyl 0 trk 0

TSO Incremental FlashCopy INCREMENTAL(YES) on the FCESTABL command Target is write inhibited for the duration of the Incremental FlashCopy relationship Swap source and target specifications to reverse the increment

TSO Consistency Group FlashCopy ACTION(FREEZE) on the FCESTABL command I/O activity held until FCWITHDR ACTION(THAW) command issued to LSS or Consistency Group timer in LSS expires FCWITHDR ACTION(THAW) Must use DEVN with SOURCE and TARGET parameters

TSO Inband FlashCopy Command is issued to PPRC primary to FlashCopy its secondary to a tertiary FCESTABL DEVN(devno) SOURCE(serialno lss cca|lun) TARGET(serialno lss cca|lun) MODE(COPY) SSID(ssid) REMOTE(YES)

TSO FRR Enabled and Fast Reverse Restore FRR Enabled FlashCopy FCESTABL SDEVN(ssss) TDEVN(tttt) MODE(ASYNC) Fast Reverse Restore FlashCopy Switch the original source and target addresses FCESTABL SDEVN(ssss) TDEVN(tttt) ACTION(FRR)

FlashCopy and ICKDSF FlashCopy BACKGROUND COPY and NOCOPY FLASHCPY ESTABLISH DDNAME(ddname) TARGETVOL(lss,cca,cuuu) FLASHCPY ESTABLISH DDNAME(ddname) TARGETVOL(lss,cca,cuuu) MODE(NOCOPY) Incremental FlashCopy Inband FlashCopy FRR Enabled FlashCopy FlashCopy Fast Reverse Restore (FRR) FlashCopy Query

ICKDSF Incremental FlashCopy Establishing the increment FLASHCPY ESTABLISH DDNAME(ddname) TARGETVOL(lss,cca,cuuu) CHANGERECORDING(YES) Resync the increment FLASHCPY ESTABLISH DDNAME(ddname) TARGETVOL(lss,cca,cuuu) CHANGERECORDING(YES) RESTORE Reversing the increment FLASHCPY ESTABLISH UNIT(ccuu) TARGETVOL(lss,cca,cuuu) CHANGERECORDING(YES) RESTORE

ICKDSF Inband FlashCopy Creating a tertiary copy of a PPRC secondary by issuing a FlashCopy command to the PPRC primary FLASHCOPY ESTABLISH UNIT(ccuu) TARGETVOL(lss,cca,ccuu) Issuing an inband FlashCopy command to a PPRC primary but have the action take place on a PPRC secondary other than the PPRC secondary of the device the command is issued to FLASHCPY ESTABLISH UNIT(ccuu) SOURCEVOL(lss,cca,ser#,ccuu) TARGETVOL(lss,cca,ccuu)

ICKDSF FRR Enabled and Fast Reverse Restore FRR Enabled FlashCopy FLASHCPY ESTABLISH UNIT(ccuu) TARGETVOL(lss,cca,cuuu) CHANGERECORDING(YES) MODE(NOCOPY) INHIBITTARGETWRITES(YES) Fast Reverse Restore FlashCopy Unit is the original target and TARGETVOL is the original source FLASHCPY ESTABLISH UNIT(ccuu) TARGETVOL(lss,cca,cuuu) FASTREVERSERESTORE TGTOKPRIME(YES)

FlashCopy Source & Target Placement In general: Spread evenly across disk subsystems Within each disk subsystem, spread evenly across clusters Within each cluster, spread evenly across device adapters Within each device adapter, spread evenly across ranks Place FlashCopy target in same cluster as source If using BACKGROUND COPY, target on a different device adapter

FlashCopy Source/Target Placement Table Cluster Device Adapter Rank FlashCopy Establish Performance Same cluster Doesn’t matter Different ranks BACKGROUND COPY Performance Different device adapter FlashCopy Impact to Applications

FlashCopy Establish Establish is the period of time when all of the necessary preparations are being made to create the FlashCopy relationship FlashCopy target should be in the same cluster and on a different rank Incremental FlashCopy introduces modest performance impact for FlashCopy establish The impact is negligible in most cases

FlashCopy BACKGROUND COPY Performance BACKGROUND COPY physically copies the data from the source volume/LUN to the target volume/LUN Only write updates to the source volume/LUN are copied if BACKGROUND NOCOPY is used Do not expect to see all pairs actively copying data if you have FlashCopied many volumes at one time Microcode will attempt to try and balance active copy pairs across DS8000 device adapter resources Microcode gives higher preference to application activity than BACKGROUND COPY activity FlashCopy target should be in the same cluster, different device adapter, different rank

Background Copy Rate – Turbo R2 This figure compares the end to end background copy rate for the previously released code to the latest code release. The end to end background copy rate is the average data rate from the time the FlashCopy command is issued until the copy is complete. The improvement was due to the optimization of the FlashCopy algorithm. For larger configurations using more disk drives and DA pairs, the percentage improvement is greater.

Background Copy Rate – Turbo R2 FlashCopy background copy rate scaled up linearly with additional DA pairs for both CKD RAID 5 and FB RAID 10. With eight DA pairs, the end to end FlashCopy background copy rate reached over 1400 MB/s with a peak rate over 1800 MB/s for FB.

FlashCopy Full Box Copy Implication that all rank resources are involved in the copy process Either all or nearly all ranks have both source and target volumes or half of the ranks are source volumes/LUNs and half the ranks have target volumes/LUNs Common at disaster recovery sites where tertiary copies are made Place source and targets in different ranks For example, source volumes/LUNs on rank 0 flashed to targets volumes/LUNs on rank 1 and volumes on rank 1 to rank 0 If there is heavy application activity on a rank, BACKGROUND COPY will have less effect if the target is on a rank with little application activity If BACKGROUND COPY performance is critical, consider using Incremental FlashCopy as less data will be copied in the background

Full Box Background Copy Rate – Turbo R2 This figure indicates that the FlashCopy background copy rate dropped from 1433 MB/s without IO to 486 MB/s with IO, which was as expected. FlashCopy is designed to favor host IO over background copy operations so the copy rate decreases as the host IO load increases. Note that while Figure 11 displays the average overall background copy rate, the instantaneous copy rate starts out lower but tends to increase and reach a steady state over the course of the copy.

To COPY or to NOCOPY?…. That is the question! BACKGROUND NOCOPY is typically the best choice to minimize rank and DA activity within the physical box But…. You must ask why are you making a copy? And…. What type of application workload do I have? For example: Is the copy only going to be used for creating a tape backup? BACKGROUND NOCOPY should be used and the relationship withdrawn after the tape backup is complete Is the copy going to be used for testing or development? NOCOPY again is typically the best choice Will you need a copy of the copy? BACKGROUND COPY must be used so that the target will be withdrawn from its relationship after all of the tracks are copied thereby allowing it to be a source in a new relationship Possibly use NOCOPY to COPY option Is the workload OLTP (NOCOPY typically is the choice) or are there a large number of random writes and are not cache friendly (COPY may be the better choice)

Addendum FlashCopy Status – TSO FlashCopy Status - ICKDSF FlashCopy and DS CLI Full volume FlashCopy Invocation FlashCopy NOCOPY to COPY Invocation Incremental FlashCopy Invocation Incremental FlashCopy Reverse Invocation Consistency Group FlashCopy Invocation Persistent FlashCopy Invocation Inband FlashCopy Invocation FRR Enabled FlashCopy Invocation Fast Reverse Restore FlashCopy Invocation Copy Services Matrix References

Displaying FlashCopy Status – TSO FCQUERY DEVN(devno) FORMATTED Note that the TSO FCQUERY only tells you how many relationships there are when issued under FC V2… it does not identify the targets.

Displaying FlashCopy Status – ICKDSF FLASHCPY UNIT(ccuu) QUERY RELATIONS

FlashCopy and DS CLI Invoking FlashCopy BACKGROUND COPY and NOCOPY Invoking FlashCopy NOCOPY to COPY Invoking Incremental FlashCopy Invoking Consistency Group FlashCopy Invoking Persistent FlashCopy Invoking FRR Enabled FlashCopy Invoking FlashCopy Fast Reverse Restore Invoking Inband FlashCopy DS CLI Display Status

Full Volume FlashCopy Invocation DS CLI mkflash –dev storage_image_id source_volume_ID:target_volume_ID mkflash –dev –nocp storage_image_id source_volume_ID:target_volume_ID DEVN must specify a CKD access volume located in the same subsystem cluster as the fixed block device identified by SOURCE in this command.

FlashCopy NOCOPY to COPY Invocation DS CLI rmflash –dev storage_image_id –cp source_volume_ID:target_volume_ID DUMPCONDITIONING would be ignored in this case.

Incremental FlashCopy Invocation DS CLI resyncflash –dev storage_image_id –record source_volume_ID:target_volume_ID The -persist flag automatically selected when –record flag is set. CHANGERECORDING(YES) will also activate persistence. TSO INCREMENTAL(YES) will also set ‘inhibit target write’.

Incremental FlashCopy Reverse Invocation DS CLI reverseflash –dev storage_image_id –record source_volume_ID:target_volume_ID The -persist flag automatically selected when –record flag is set. CHANGERECORDING(YES) will also activate persistence.

Consistency Group FlashCopy Invocation DS CLI mkflash –dev storage_image_id –freeze source_volume_ID:target_volume_ID unfreezeflash –dev storage_image_id source_LSS_ID

Persistent FlashCopy Invocation DS CLI mkflash –dev storage_image_id –persist source_volume_ID:target_volume_ID

Inband FlashCopy Invocation DS CLI mkremoteflash –dev storage_image_id -conduit LSS_ID source_volume_ID:target_volume_ID resyncremoteflash –dev storage_image_id -conduit LSS_ID -record source_volume_ID:target_volume_ID The format of a volume ID is hexadecimal XYZZ, where XY is a logical subsystem (LSS) number (00–FE), and ZZ is a volume number (00–FF) that is contained by a logical subsystem.

FRR Enabled FlashCopy Invocation DS CLI mkflash –dev storage_image_id –record –tgtinhibit -nocp source_volume_ID:target_volume_ID

Fast Reverse Restore FlashCopy Invocation DS CLI setflashrevertable –dev storage_image_id source_volume_ID Commitflash –dev storage_image_id source_volume_ID revertflash –dev storage_image_id source_volume_ID DS CLI – You have to enable revertable flash first. ICKDSF – the TARGETVOL was the source volume (B) when the Global Mirror FlashCopy relationship was first established.

Displaying FlashCopy Status – DS CLI -s (Optional) Displays FlashCopy pair IDs. You cannot use the -l and the -s flags together. -l (Optional) Displays the default output plus copy indicator, out of sync tracks, date created, and date synchronized. You cannot use the -l and the -s flags together. lsremoteflash is used to query remote FlashCopy.

lsflash –l Output lsflash –dev IBM.2107-75FA120 –l 0100:0200 0101:0201 0102:0202 0103:0203 Report field definitions ID Specifies the FlashCopy pair ID. This ID consists of two volume IDs, one designated as the source and the other as the target volume for a FlashCopy relationship. | SrcLSS Specifies the Consistency Group LSS ID that is associated with the source volume of this FlashCopy relationship. Sequence Num Specifies the sequence number that is associated with the FlashCopy relationship. Note: This item is not supported on the 2105. Timeout (secs) Specifies the consistency group Long Busy Timeout setting for the LSS ID that is associated with the source volume of this FlashCopy relationship. You can specify a value in the range of 1 - 600 000. The default value is 120 seconds. ActiveCopy Specifies (Enabled or Disabled) whether the background copy process is active for this FlashCopy relationship. Recording Specifies (Enabled or Disabled) whether this FlashCopy relationship was established with the record changes option. Persistent Specifies (Enabled or Disabled) whether this FlashCopy relationship was established with the persistent option. Revertible Specifies (Enabled or Disabled) whether this FlashCopy relationship was established with the revertible option. SourceWriteEnabled Specifies (Enabled or Disabled) whether this FlashCopy relationship was established with the allow source writes option. TargetWriteEnabled Specifies (Enabled or Disabled) whether this FlashCopy relationship was established with the allow target writes option. BackgroundCopy Specifies (Enabled or Disabled) whether this FlashCopy relationship was established with the run background copy option. CopyIndicator Specifies (Yes or No) whether the CopyIndicator is set for this FlashCopy relationship. OutOfSyncTracks Specifies the number of tracks that are not synchronized for this FlashCopy relationship. The maximum value that can be displayed is dependent on the source volume size. DateCreated Specifies the date and the time that the FlashCopy relationship was established. DateSynced Specifies the date and time this FlashCopy relationship was synchronized, or null (-) if the relationship is not synchronized.

Copy Services Matrix GMz10 (XRC) Primary (XRC) Secondary Metro Mirror or Global Copy Primary Metro Mirror or Global Copy Secondary Global Mirror Primary Global Mirror Secondary FlashCopy Source FlashCopy Target Incremental FLC Source Incremental FLC Target Concurrent Copy Source No Yes11 Yes No5 Metro Mirror or Global Copy Primary Yes 1 No6 Yes1 Yes8 Yes9 Yes 3,4 Yes 4 Yes3 Yes 2 Yes4 No7 Device Is May Become

Notes: Only in a Metro/Global Copy (supported on ESS) or a Metro/Global Mirror Environment (supported on ESS and DS8000). FlashCopy V2 at LIC 2.4.0 and higher on ESS800 (DS6000 and DS8000 utilize FlashCopy V2 by default). You must specify the proper parameter to perform this Metro Mirror primary will go from full duplex to copy pending until all of the flashed data is transmitted to remote Global Mirror primary cannot be a FlashCopy target FlashCopy V2 Multiple Relationship. FlashCopy V2 Data Set FlashCopy (only available for z/OS volumes). The Storage Controller will not enforce this restriction, but it is not recommended. A volume may be converted between the states Global Mirror primary, Metro Mirror primary and Global Copy primary via commands, but it two relations cannot exist at the same time (i.e. multi-target). GMz (XRC) Primary, Global Mirror Secondary, Incremental FlashCopy Source and Incremental FlashCopy Target all use the Change Recording Function. For a particular volume only one of these relationships may exist. Updates to the affected extents will result in the implicit removal of the FlashCopy relationship, if the relationship is not persistent. This relationship must be the FlashCopy relationship associated with Global Mirror – i.e. there may not be a separate Incremental FlashCopy relationship. Global Mirror for zOS (GMz) is supported on ESS and DS8000 In order to ensure Data Consistency, the XRC Journal volumes must also be copied.

IBM Copy Services Technologies – DS6K/DS8K FlashCopy Point in time copy Available on: DS8000, DS6000, ESS SAN Volume Controller DS4000 N Series Primary Site A Metro distance <300km Site B Metro Mirror Synchronous mirroring Available on: DS8000, DS6000, ESS SAN Volume Controller DS4000 N Series Out of Region Site B Primary Site A Global Mirror Asynchronous mirroring Available on: DS8000, DS6000, ESS SAN Volume Controller DS4000 N Series Primary Site A Metro Site B Out of Region Site C Metro / Global Mirror Three site synchronous and asynchronous mirroring Available on: DS8000, ESS N Series Within Storage System Asynchronous processing The separation of data transmission from the signaling of I/O complete Distance between primary and secondary has little impact upon the response time of the primary volume Helps minimize impact to application performance Examples are PPRC Extended Distance or XRC

References SC35-0428 DFSMS Advanced Copy Services GC35-0033 Device Support Facilities User’s Guide and Reference SC26-7625 Command Line Interface User’s Guide SG24-6788 IBM System Storage DS8000 Series: Copy Services in Open Environments SG24-6787 IBM System Storage DS8000 Series: Copy Services with IBM System z SG24-6783 IBM System Storage DS6000 Series: Copy Services in Open Environments SG24-6782 DS6000 Series: Copy Services with IBM System z Servers http://www-03.ibm.com/support/techdocs/atsmastr.nsf/Web/TechDocs FLASH10305 Enhanced z/OS Support for ESS Advanced Copy Services Functions FlashCopy and PPRC FLASH10319 Using TSO Incremental FlashCopy FLASH10461 DFSMSdss Support for Incremental FlashCopy, FlashCopy Consistency Groups, and FlashCopy NOCOPY to COPY Performance White Paper http://w3-1.ibm.com/sales/systems/portal/_s.155/254?navID=f320s260&geoID=All&prodID=System%20Storage&docID=ditlDS8000PerfWPPower DS8000/DS6000 Copy Services: Getting Started WP100905 http://www-03.ibm.com/support/techdocs/atsmastr.nsf/WebDocs/?Search&Query=[HTMLDocumentName=WM*]+AND+(burger)&Start=1&Count=50&SearchOrder=1&SearchMax=10000

Disclaimers and Trademarks 11/28/2018

Disclaimers No part of this document may be reproduced or transmitted in any form without written permission from IBM Corporation. Product data has been reviewed for accuracy as of the date of initial publication. Product data is subject to change without notice. This information could include technical inaccuracies or typographical errors. IBM may make improvements and/or changes in the product(s) and/or program(s) at any time without notice. Any statements regarding IBM's future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only. The performance data contained herein was obtained in a controlled, isolated environment. Actual results that may be obtained in other operating environments may vary significantly. While IBM has reviewed each item for accuracy in a specific situation, there is no guarantee that the same or similar results will be obtained elsewhere. Customer experiences described herein are based upon information and opinions provided by the customer. The same results may not be obtained by every user. Reference in this document to IBM products, programs, or services does not imply that IBM intends to make such products, programs or services available in all countries in which IBM operates or does business. Any reference to an IBM Program Product in this document is not intended to state or imply that only that program product may be used. Any functionally equivalent program, that does not infringe IBM's intellectual property rights, may be used instead. It is the user's responsibility to evaluate and verify the operation on any non-IBM product, program or service. THE INFORMATION PROVIDED IN THIS DOCUMENT IS DISTRIBUTED "AS IS" WITHOUT ANY WARRANTY, EITHER EXPRESS OR IMPLIED. IBM EXPRESSLY DISCLAIMS ANY WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE OR INFRINGEMENT. IBM shall have no responsibility to update this information. IBM products are warranted according to the terms and conditions of the agreements (e.g. IBM Customer Agreement, Statement of Limited Warranty, International Program License Agreement, etc.) under which they are provided. IBM is not responsible for the performance or interoperability of any non-IBM products discussed herein. Information concerning non-IBM products was obtained from the suppliers of those products, their published announcements or other publicly available sources. IBM has not tested those products in connection with this publication and cannot confirm the accuracy of performance, compatibility or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products. The providing of the information contained herein is not intended to, and does not, grant any right or license under any IBM patents or copyrights. Inquiries regarding patent or copyright licenses should be made, in writing, to: IBM Director of Licensing IBM Corporation North Castle Drive Armonk, NY 10504-1785 USA

Trademarks The following terms are trademarks or registered trademarks of the IBM Corporation in either the United States, other countries or both. Linear Tape-Open, LTO, LTO Logo, Ultrium logo, Ultrium 2 Logo and Ultrium 3 logo are trademarks in the United States and other countries of Certance, Hewlett-Packard, and IBM. Java and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States and/or other countries. Microsoft, Windows, Windows NT, and the Windows logo are trademarks of Microsoft Corporation in the United States and/or other countries. Intel, Intel Inside (logos), MMX and Pentium are trademarks of Intel Corporation in the United States and/or other countries. UNIX is a registered trademark of The Open Group in the United States and other countries. Linux is a trademark of Linus Torvalds in the United States and other countries. Other company, product, or service names may be trademarks or service marks of others. AIX AIX 5L BladeCenter Chipkill DB2 DB2 Universal Database DFSMSdss DFSMShsm DFSMSrmm Domino e-business logo Enterprise Storage Server ESCON eServer FICON FlashCopy GDPS Geographically Dispersed Parallel Sysplex HiperSockets i5/OS IBM IBM eServer IBM logo iSeries Lotus ON (button device) On demand business OnForever OpenPower OS/390 OS/400 Parallel Sysplex POWER POWER5 Predictive Failure Analysis pSeries S/390 Seascape ServerProven System z9 System p5 System Storage Tivoli TotalStorage TotalStorage Proven TPF Virtualization Engine X-Architecture xSeries z/OS z/VM zSeries

Disclaimers IBM customers are responsible for ensuring their own compliance with legal requirements. It is the customer's sole responsibility to obtain advice of competent legal counsel as to the identification and interpretation of any relevant laws and regulatory requirements that may affect the customer's business and any actions the customer may need to take to comply with such laws. IBM does not provide legal advice or represent or warrant that its services or products will ensure that the customer is in compliance with any law. The information contained in this documentation is provided for informational purposes only. While efforts were made to verify the completeness and accuracy of the information provided, it is provided “as is” without warranty of any kind, express or implied. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, this documentation or any other documentation. Nothing contained in this documentation is intended to, nor shall have the effect of, creating any warranties or representations from IBM (or its suppliers or licensors), or altering the terms and conditions of the applicable license agreement governing the use of IBM software.