1. Bruno Giovanini Manesco © 2016
iSCSI TOI
Bruno Giovanini Manesco © 2016

2. About this TOI
Purpose: It has been created to get new hires up to speed on the iSCSI technology in VMware solutions and its common issues.
Benefits: This TOI will give you a base on how VMware is utilizing iSCSI in its set ups.
iSCSI TOI

3. What will be covered What is iSCSI? How does it work?
How does it work in VMware?
Improving Performance
Practical Examples
Troubleshooting
Errors and Logs
iSCSI TOI

4. What is iSCSI?

5. iSCSI IBM’s Internet Small Computer System Interface.
SCSI based, Computer and Storage act as Client/Server set up.
Sends SCSI commands over the TCP protocol, using any kind of Network
Allows the transportation of block level data, between initiator and target
iSCSI TOI

6. How does it work?

7. iSCSI structure Initiator-target model, naming standard
iSCSI encapsulation
Requirements on the client side
CHAP Authentication, IP Sec
Variations Hardware iSCSI, Hardware assisted (TOE) and Software iSCSI.
iSCSI TOI

8. How does it work in VMware?

9. iSCSI and ESXi One software iSCSI adapter per ESXi
The initiator itself is automatically configured
iSCSI TOI

10. iSCSI and ESXi
The configuration interface is simple and present you with a couple of options to configure.
iSCSI TOI

11. iSCSI and ESXi
The CHAP can be configured in the iSCSI properties, but rarely presents a problem
iSCSI TOI

12. iSCSI and ESXi
As iSCSI is encapsulation over TCP/IP it requires a working network connection, counting on VMKs
The vmk being configured have to be in the same subnet as the SAN targets
iSCSI TOI

13. iSCSI and ESXi Create a vmk port with:
No traffic assigned to it
Same IP subnet of the target
In a vSwitch that has the vmnic in the Storage Network
The ESXi will automatically pick up the first vmk in the same subnet to use to communicate with the target
Do not use the same subnet as management!!!
Demo.
iSCSI TOI

14. iSCSI and ESXi
Dynamic and Static discovery can be used to add targets:
iSCSI TOI

15. Improving the Performance and Availability

16. iSCSI Performance and Redundancy
Once this technology is Storage related:
It is a best practice to have redundancy
It WILL be accused of being the bottleneck of the environment
CU usually wants to take full advantage of the offered solution trying unsupported things
Majority of the cases
iSCSI TOI

17. iSCSI Redundancy Redundancy achieved through multipathing
Configuring multipathing in VMware
From host: multiple vmks;
From SAN: multiple storage portals;
Multipathing does not requires port binding in all situations
iSCSI TOI

18. iSCSI Port binding
"The primary use case of this feature (Port binding) is to create a multipath configuration with storage that only presents a single storage portal, such as the DELL EqualLogic and the HP/Lefthand. Without iSCSI multipathing, these type of storage would only have one path between the ESX host and each volume. iSCSI multipathing allows us to multipath to this type of clustered storage.“
Requirements:
Array Target iSCSI ports must reside in the same broadcast domain and IP subnet as the VMkernel port (GOLDEN RULE).
All VMkernel ports used for iSCSI connectivity must reside in the same broadcast domain and IP subnet.
All VMkernel ports used for iSCSI connectivity must reside in the same vSwitch.
Currently, port binding does not support network routing.
iSCSI TOI

19. iSCSI Port binding
iSCSI TOI

20. iSCSI Port binding Configuring the Port Binding:
The vmks has to be in the same vSwitch
Each vmk has to be designated to a specific vmnic, set it in nic teaming, failover order
Add those vmks to the iSCSI software adapter Network tab
iSCSI TOI

21. iSCSI Common Mistakes
Reality:
Expected:
iSCSI TOI

22. iSCSI Performance improvements (PSP RR)
Once you have multipathing Establish, you are able to choose the path selection policy (PSP) according to your environment:
Fixed
MRU – Most Recently Used
Round Robin (Performance Improvement)
You can change the default RR IOPS limit from 1000 to 1 (KB )
DELL has its own PSP for EqualLogic
“Configuring and Installing the PS Series Multipathing Extension Module for VMware vSphere and PS Series SANs” pdf file.
iSCSI TOI

23. iSCSI Performance improvements (PSP RR)
CONFIDENTIAL

24. iSCSI Performance improvements - Jumbo Frames
Huge amount of data being dealt by the iSCSI
Might take a toll in the processing resources.
Jumbo frames enable bigger packets.
Maximum transmit unit (MTU) size from default 1500 up to 9000.
1GB of data needs approximately:
packets to be transferred on a 1500 MTU;
packets to be transferred on a 9000 MTU;
9000/1500 = 6, six times less encapsulation, acknowledgements
iSCSI TOI

25. iSCSI Performance improvements - Jumbo Frames
Requirements:
NIC has to support that MTU size;
Switch has to support that MTU size;
SAN has to support that MTU size;
All the Network component participating on the packets exchange have to be configure for dealing with the same MTU size;
iSCSI TOI

26. iSCSI Performance improvements - Jumbo Frames
On the ESXi has to be set in 2 levels:
vSwitch
Vmk port
iSCSI TOI

27. Troubleshooting

28. iSCSI Troubleshooting
As any troubleshooting scoping is the first step
Establish in which level is the problem occurring;
Network Connectivity
LUN Presentation
Authentication
FileSystem
Performance
iSCSI TOI

29. iSCSI Troubleshooting
A standardized approach is always advised
Start with the basic info:
ESXi version
ESXi Hardware spec and Vendor
vCenter version
SAN vendor and Model
SAN Firmware
Network Configuration
iSCSI TOI

30. iSCSI Troubleshooting – Port Binding
Taking too long to reboot.
Connection marked as Offline, different subnets usually.
Test connectivity with the vmks set to Port Binding
#vmkping –I vmk<#> <target_IP>
iSCSI TOI

31. iSCSI Troubleshooting – Port Binding
If you need to do all in cmd:
Get the target IP address:
#esxcli iscsi adapter target portal list
Get the vmk number knowing the label
#esxcli network ip interface get | egrep "(iSCSI|vmk)“
Get the vmk IP knowing the vmk number
#esxcli network ip interface ipv4 get | grep vmk1
Test ping between vmk and target
#vmkping -I vmk
To investigate more deeply which interface is dealing with which vmk, relating to which specific storage path use the following KB (and rate it 5 stars)
iSCSI TOI

32. iSCSI Troubleshooting – Jumbo Frames
Be sure to have all the requirements satisfied
Check the connectivity to the target using:
# vmkping -I vmk -s d IP_ADDRESS
Obs: 28 bytes are for IP header of the packet
Check if from VMware side everything is configured correctly
Vmk and vSwitch level
iSCSI TOI

33. iSCSI Troubleshooting – Software adapter
Even though rare issues with software adapter can happen:
iSCSI not starting or seeing targets
KB , iSCSI DB corrupt.
Recreate the adapter loosing all the connections
iSCSI TOI

34. iSCSI Troubleshooting – Datastore Unavailable
Dell’s EqualLogic SAN is known for having issues that can cause Datastore to be completely unavailable, causing Production down situations. KB
Often suggests to disable the Delayed Ack, which ESXi has enabled by default once their SAN does not deal with that well.
iSCSI session time outs could also cause the same unavailability, advanced settings can be tweaked to resolve the issue.
iSCSI TOI

35. Sources
binding.pdf
enhancements.html d= d=
29&sliceId=1&docTypeID=DT_KB_1_1&dialogID= &stateId=0%200%
CONFIDENTIAL

36. Questions?
Thank you!
CONFIDENTIAL