Changing modes on #5887 EXP24S SFF GEN2-BAY DRAWER Material compiled by: Pat Buckland PatBuck@us.ibm.com and Mark Olson. OlsonM@us.ibm.com

Cautions !!! Very infrequently there may be reasons to change the mode after the EXP24S drawer has been shipped from IBM. NOTE that by changing the mode you are changing the SAS topology. This changes what the SAS adapters see and what drives they run. You may also be changing SAS adapters and cables as well. Bad things can happen if you blindly make changes without understanding what you are doing including possible loss of data. These “bad things might happen” scenarios are the reason why changing modes was not and is not announced or made generally available. However if you understand what you are doing, the risk of a problem is small, especially assuming you have removed any content or at least backed up the content. It is recommended that you back up your data before changing modes. Better yet, move all content off the EXP24S drawer before changing modes. If you change to a “larger” mode (example mode 1 to mode 2), the original SAS adapter port will see fewer drives and this could break some RAID arrays. The problem you might see is somewhat similar to someone accidentally pulling two drives out of a production RAID-5 array and moving those two drives into a different array and then reformatting or writing over those two drives before you realized the original array was destroyed. Or if you change to a another example, if you go to a “smaller” mode (example mode 4 to mode 2), one partition may be able to see data that was on another partition. When you make the change you will temporarily go into a special manufacturing mode 0 that is used to avoid some configurations where adapters would not configure. As you go into mode 0 and then to the final mode there will probably be some SAS fabric errors reported. And the stored VPD data may no longer be correct. So it is recommended that the user should unmount the drawer and all drives then reconfigure it after changing the mode. Also the user should modify the mode label to show the correct mode. After changing the mode remember to remove the wrap plug. If you leave it on the drawer will go back into mode 0 at the next power cycle. It would be good to update AAS/WTAAS records using a no-charge RPO (record purpose only) MES to reflect the changed mode. That will help avoid future confusion as to what is really installed. An RPO MES ships no hardware, it just merely updates the IBM records.

To change the mode plug in a wrap plug (IBM PN 44X1419) to an open SAS port on the left ESM C1) IMPORTANT Note -- No IBM ordering structure such as a feature code or RPQ has been created to order/ship this part from IBM. A negligible number of requests for the capability have been identified so far. But if you really need one, the vendor from which IBM buys the wrap plug as of March 2013 is Molex Incorporated. The Molex part number is 74547-0399 (iPass™ Loopback Assembly, Key 2,4,6, Hot Pluggable, Pull-to-Release Latch). See www.molex.com Molex sells these in batches of 25 plus shipping. Authors of this note guess the cost of a batch would be around 500 USD plus shipping. See the Molex web site for specific country contact points and prices. Wrap plug If there is not cable on the T1 connecter then plug it in there If there is a cable on the T1 connecter then plug it in connecter T2 ESM C1 T2 T3 T1 ESM C2 T2 T3 T1 When the wrap plug is detected, both green LEDs above the connecter will come on

Finding mode on AIX Changing mode on AIX ses7 U5887.001.Z01K200-P1-C2 SAS Enclosure S SAS Expander: FRU Number..................45W7653 Serial Number...............YL10DZ07T013 Customer Card ID Number.....50B1 Zoning Mode.................1 ROM Level.(alterable).......5xxG Hardware Location Code......U5887.001.Z01K200-P1-C2  Changing mode on AIX After adding the wrap plug the two ESMs will go to a special manufacturing mode 0. This may cause some SAS fabric errors to be reported. Need to be at AIX level of 61N or 71B .At the AIX command line use command /usr/lpp/diagnostics/bin/ses_mode . The syntax is: /usr/lpp/diagnostics/bin/ses_mode -d device [-z <n>] -d device where device is the SAS disk enclosure that is zoned to share disks to different partitions. -z <n> set the zone mode to <n> (n is 1, 2 or 4) After changing the mode remember to remove the wrap plug

Changing modes in Linux Required Linux levels RHEL 5.7 - July 21, 2011 RHEL 6.2 - November 28, 2011 SLES 11 SP2 - December 12, 2011 After adding the wrap plug the two ESMs will go to a special manufacturing mode 0.. This may cause some SAS fabric errors to be reported To check mode iprconfig -c query_ses_mode sg# To set mode iprconfig -c set_ses_mode sg# <mode> sg# is the sg device, such as sg5, and <mode> is the mode number, such as 2. After changing the mode remember to remove the wrap plug.

Changing modes in IBM i IBM i only supports mode 1 Suggest changing modes on the “from” machine before attaching to an IBM i partition. Author did not research the IBM i commands to change mode.

Mode label PN = 44X1419 Note -- After the change, you will probably cover the existing label with another adhesive label you have hand-cut to fit and manually write. No way to order replacement labels is available at this time

Supported or Not Supported There is no formal documentation or cook books how to do this process in the info center. There are no menus guiding you how to do this and helping you to avoid making an error. There is no monitor/warning if you leave the wrap plug in the SAS bay accidentally. There is no IBM-provided way to order the wrap plug. So from the above perspective changing modes in the EXP24S is “unsupported”. -------------------------------------- But moving/changing SAS adapters and/or EXP24S drawers and their SAS drives around and reconfiguring adapters and cables – this is a very common type of activity done by IBMers, Business Partners and clients. Like any change involving storage that is holding your data or your system, you can cause serious problems if you don’t know what you are doing. So in this sense, changing modes on an EXP24S is a typical activity (with the initial caveats on this page). Similarly changing modes in the #5802/5803/5873/5877 12X-attached I/O drawers (which is supported) can potentially have some of the same problems. But those mode changes are better documented and don’t require a separate wrap plug. If you change mode and there is a problem, IBM Service will treat it like a normal service call. If IBM Service determines a problem was caused by client error, the time and perhaps parts could be billable. Normal wear/tear of a drive or other part would be covered under standard warranty/service. Recovery of data (if possible) caused by a client error is a billable service.

Repeat Cautions !!! Very infrequently there may be reasons to change the mode after the EXP24S drawer has been shipped from IBM. NOTE that by changing the mode you are changing the SAS topology. This changes what the SAS adapters see and what drives they run. You may also be changing SAS adapters and cables as well. Bad things can happen if you blindly make changes without understanding what you are doing including possible loss of data. These “bad things might happen” scenarios are the reason why changing modes was not and is not announced or made generally available. However if you understand what you are doing, the risk of a problem is small, especially assuming you have removed any content or at least backed up the content. It is recommended that you back up your data before changing modes. Better yet, move all content off the EXP24S drawer before changing modes. If you change to a “larger” mode (example mode 1 to mode 2), the original SAS adapter port will see fewer drives and this could break some RAID arrays. The problem you might see is somewhat similar to someone accidentally pulling two drives out of a production RAID-5 array and moving those two drives into a different array and then reformatting or writing over those two drives before you realized the original array was destroyed. Or if you change to a another example, if you go to a “smaller” mode (example mode 4 to mode 2), one partition may be able to see data that was on another partition. When you make the change you will temporarily go into a special manufacturing mode 0 that is used to avoid some configurations where adapters would not configure. As you go into mode 0 and then to the final mode there will probably be some SAS fabric errors reported. And the stored VPD data may no longer be correct. So it is recommended that the user should unmount the drawer and all drives then reconfigure it after changing the mode. Also the user should modify the mode label to show the correct mode. After changing the mode remember to remove the wrap plug. If you leave it on the drawer will go back into mode 0 at the next power cycle. It would be good to update AAS/WTAAS records using a no-charge RPO (record purpose only) MES to reflect the changed mode. That will help avoid future confusion as to what is really installed. An RPO MES ships no hardware, it just merely updates the IBM records.