1. Secure Asymmetric iSCSI For Online Storage
Sarah A. Summers
Here to talk to you about my Masters project – Secure Asymmetric iSCSI for Online Storage
4/25/2019
Summers - SAiSCSI

2. Outline of Presentation
Goals of the Project
Overview of SCSI, iSCSI and IPsec
Review of Efficient Asymmetric Secure iSCSI
Testing of Efficient Asymmetric Secure iSCSI
Motivation for Enhancements
Proposed Enhancements
Results
Disaster Recovery
Additional Research
Lessons Learnt, Future Work, Conclusions
I would like first to give you a general outline to this presentation.
The items covered will be:
Goals of the project
Brief overview of SCSI, iSCSI and IPSec as relevant to the project
Review of the previous work on Efficient Asymmetric Secure iSCSI
My testing of the Efficient Asymmetric Secure iSCSI scheme
Motivation for enhancements
The proposed enhancements
Performance results obtained from testing the enhancements
Additional Research that I undertook
Then I will finish up with lessons learnt during the project, some suggestions for future work and finally some general conclusions.
4/25/2019
Summers - SAiSCSI

3. Goals
Enhance the existing Efficient Asymmetric Secure iSCSI scheme to:
Enable the transfer of files of arbitrary size
Allow files to be transferred to two target storage devices for mirroring and backup.
Develop and implement a user interface to simplify usage of the scheme
Consider the use of the scheme for disaster recovery
Consider the use of the scheme for mirroring and backup
The goals of the project can be split into 3 areas:
Enhancement of the existing Efficient Asymmetric Secure iSCSI scheme to:
Enable the transfer of files of arbitrary size
Allow files to be transferred to two target storage devices
Develop and implement a graphical user interface to simplify usage of the scheme
Consider the use of the scheme for disaster recovery
Consider the use of the scheme for mirroring and backup
Chow: add mirroring and backup.
4/25/2019
Summers - SAiSCSI

4. Overview of SCSI, iSCSI, IPsec, sg Interface, sg3_utils and sg_dd
The SCSI, iSCSI and IPsec protocols are used either directly or indirectly in this project.
As such, it is necessary to have a general understanding of them before considering details of the project itself.
In addition, the sg_dd utility in the sg3_utils package is used and needs some explanation.
4/25/2019
Summers - SAiSCSI

5. SCSI Application Layer Storage Protocol
Standard device interface bus enabling block data I/O
Logically addresses blocks
Utilizes a client/server (initiator/target) architecture
Data transferred in Command Descriptor Blocks
Limitations
Length of SCSI bus limits distance over which SCSI can operate
Limited number of devices can be connected to it
SCSI (Small Computer Systems Interface) is an application layer storage protocol.
It was developed as a standard device interface bus to enable block data I/O
Additional slide at end showing CDB
4/25/2019
Summers - SAiSCSI

6. iSCSI
End-to-end protocol to enable transportation of storage I/O block data over IP networks
Maps SCSI functionality to the TCP/IP protocol
SCSI CDBs encapsulated in iSCSI Protocol Data Units
Can be implemented in software and hardware
Limitations
No built in security, relies on IPsec
Due to the limitations of the SCSI protocol and the desire to be able to use online storage, the internet SCSI (iSCSCI) protocol was developed.
The iSCSI protocol is an end-to-end protocol that enables the transportation of storage I/O block data over IP networks.
It maps SCSI functionality to the TCP/IP protocol. This is achieved by encapsulating SCSI Command Descriptor Blocks in iSCSI Protocol Data Units.
The iSCSI protocol can be implemented in software or hardware. In this project software implementations are utilized.
A limitaton of the iSCSI protocol is that it has no built in security. However, it was designed allow a variety of security implementations, the one of relevance to this project is IPsec which I will now discuss briefly.
Include slides of PDU’s in extra section at end
4/25/2019
Summers - SAiSCSI

7. IPsec
Extension of the IP protocol that provides security to IP and upper layers of the OSI model
Encapsulating Security Protocol (ESP) – provides confidentiality and optionally authentication
Performs 3 basic steps
Header calculation and placement
Trailer calculation and placement
ESP authentication field calculation and placement
Internet Key Exchange (IKE)
Transport Mode
As I have already said iSCSI does not provide security to data nor does the IP protocol in its standard form.
However security can be implemented using IPsec which is an extension to the IP protocol.
IPsec can provide security to IP and the upper layer OSI protocols during transmission of data.
Security can be provided in terms of authentication, integrity and confidentiality.
IPsec is comprised of three sub-protocols: Authentication Header, Encapsulating Security Payload and Internet Key Exchange. For the purposes of this project only ESP and IKE were utilized.
ESP provides confidentiality and optionally authentication to data. When ESP is used 3 basic steps are performed.
These are:
Header calculation and placement
Trailer calculation and placement
ESP authentication field calculation and placement
The Internet Key Exchange sub-protocol is used to generate and exchange security proposals and negotiates the associations based on the Internet Security Association and Key Management protocol.
IPsec can be used in 2 modes transport and tunnel. The transport mode has been used in this project. When used in the transport mode, data is protected end-to-end between the hosts. Only the data contained in the IP packet is encrypted and/or authentic
Possible include diagram or something at end of presentation
4/25/2019
Summers - SAiSCSI

8. sg Interface
Generic driver which allows access to all types of devices
Allows user applications to send SCSI commands to devices by using commands in the sg3_utils package
Located in the upper layer of the SCSI subsystem in the Linux kernel
The sg interface is a generic driver which is very versatile in that all types of devices can be accessed though it.
It has an additional advantage in that it allows user applications to send scsi commands to devices using commands in the sg3_utils package
It is located in the upper layer of the SCSI subsystem located in the Linux kernel.
4/25/2019
Summers - SAiSCSI

9. sg3_utils and sg_dd sg3_utils package sg_dd
Contains low level utilities, such as sg_dd for devices using the SCSI command set
Utilizes the sg interface
sg_dd
Variant of the Unix dd command for copying files
specialized for block oriented devices that use the SCSI command set
sg_dd if=test.txt of=/dev/sda bpt=1 odir=1 skip=0 seek=0
Mr Andukuri used the sg_dd utility from the sg3_utils package for writing and reading data to/from the target storage.
It was chosen for that project and for the enhancements in this project since it allows user applications to send SCSI commands to devices which is what we want to do
The sg3_utils package contains low level utilities, such as sg_dd, for devices using the SCSI command sets.
It utilizes the sg interface which has already been described.
The sg_dd utility is a variant of the Unix dd command for copying files. It is specialized for block oriented devices that use the SCSI command set.
A typical example of a command to transfer a file using sg_dd is shown.
When the command is issued a SCSI command descriptor block is constructed, this is then passed to the lower level of the SCSI subsystem where it is handled by the iSCSI driver.
bpt = blocks per transfer
odir = direct output
skip = the block number in the input file to commence reading
seek = the block number in the output file to commence writing
Chow: Need to explain what sg is. SCSI Generic.
May want to give url,
why pick sg3_utils? Explain how is it related to iSCSI.
4/25/2019
Summers - SAiSCSI

10. Review of Efficient Asymmetric Secure iSCSI
This project has focused on the use of iSCSI as a secure online storage solution. It builds on the Efficient Asymmetric Secure iSCSI scheme developed and implemented by Mr. Andukuri. As such it was necessary to review his work to develop suitable enhancements for this project.
As I have already said, iSCSI does not provide security to user data when it is transmitted although it does allow for the provision of security during transit by using IPsec. That being said, neither iSCSI nor IPsec provide security of data once it is saved on the storage device. The work of Mr. Andukuri addressed this latter issue.
4/25/2019
Summers - SAiSCSI

11. Efficient Asymmetric Secure iSCSI
Utilizes the sg_dd command to achieve the transfer of files between initiator and target
Dual key cryptographic enhancement to IPsec
Custom key to encrypt data
IKE generated keys to encrypt headers
Mr Andukuri’s scheme provides security to data both during transmission and while stored. His scheme uses the sg_dd utility from the sg3_utils package to transfer the files.
The security of the data is achieved through a dual key cryptographic enhancement to IPsec. A custom key is used to encrypt the user data at the IPsec layer of the initiator. This key is not shared with the target. The TCP and iSCSI header combination of the packet containing the user data is then encrypted using keys generated using the Internet Key Exchange.
When the packet is received at the target, only the TCP and iSCSI headers are decrypted and the payload is passed through the upper layers and stored in an encrypted form.
When retrieving data from the storage only the TCP and iSCSI headers are encrypted at the IPsec layer of the target, again using keys generated using the Internet Key Exchange. Then on receipt at the initiator the headers are decrypted using the IKE keys and then the payload is decrypted using the custom key before being passed to its final destination.
4/25/2019
Summers - SAiSCSI

12. EASI Packet Modification
The EASI implementation requires that the packets in which the data is transmitted are modified.
The implementation requires that a file be an integer multiple of block size. The block size used is 1024 bytes.
In addition to the user data being an integer multiple of block size the combination of the TCP and iSCSI headers must also be block size. This is achieved by moving the TCP header and inserting the appropriate amount of padding in the gap created between the TCP and iSCSI headers.
4/25/2019
Summers - SAiSCSI

13. Testing of Efficient Asymmetric Secure iSCSI
Having established how the Efficient Asymmetric Secure iSCSI scheme worked, it was necessary to test the scheme in order to determine what factors may affect proposed enhancements.
4/25/2019
Summers - SAiSCSI

14. VMware Virtual Machine Test Bed
In order to test the implementation, it was decided to create a virtual machine test bed using the Vmware Server
Efforts were made to recreate both the initiator and target machines by installing the Linux kernel from scratch and to incorporate the changes to the IPsec code. Unfortunately these attempts were unsuccessful.
In view of the problems that were encountered, I decided to clone the physical test for use as a virtual machine. This was done using UltimateP2V which incorporates BartPE and Symantec Ghost. The Figure shows the test bed that was created. An additional target machine was also created for the project.
4/25/2019
Summers - SAiSCSI

15. Tests
Transferring files to/from the target storage file using the sg_dd utility
Transferring files to/from the target storage file using the cp utility
Transferring files to a mounted target storage device
Mr. Andukuri’s implementation uses a file as a target storage device and the sg_dd utility to permit the user to transfer the files. He had suggested that it would be necessary to utilize the cp utility to be able to transfer files of arbitrary size.
Therefore, I decided to test the implementation as was and also attempt transfers with the cp utility in order to determine problems which would have to be resolved.
In addition, initial testing suggested that using a target file as a storage device may have limitations. So I decided to attempt to mount a disk as target storage and transfer files to it using both sg_dd and cp.
4/25/2019
Summers - SAiSCSI

16. Limitations of the Existing Scheme
User data must be an integer multiple of block size (1024 bytes)
Target Storage File
Only one file can be stored at a time
Pre-set size of target storage file limits the size of the user data that can be stored
sg_dd Utility
Count and block size values must be specified when issuing the sg_dd command
cp Utility
Reading data from target results in entire target file being retrieved
Mounted Target Storage
Not possible with existing scheme
Information obtained from the review and testing of the EASI scheme revealed the following limitations:
User data to be transferred must be comprised of integer mutiples of block size. The block size being 1024 bytes.
Use of a file as the target storage means that:
Only one file can be stored at a time
The preset size of the storage files limits the size of user data that can be stored.
When using the standard sg_dd utility with the scheme requires that the user specify both a count value and a block size value
When using the cp utility ………
Mounting a target storage disk is not possible with the existing implmentation.
First reference of cp utility. Need explanation.
4/25/2019
Summers - SAiSCSI

17. Motivations for Enhancements
Bring existing implementation closer to a complete and usable secure data transfer/storage system
Allow transfer of files of arbitrary size
Allow duplicate transfer of files to second target storage device
Configuring and starting the initiator and target software and transferring user data requires significant command line interaction which can be complex and could be simplified with a GUI
The limitations identified in the previous slide along with the proposals for future work suggested by Mr. Andukuri suggested the following motivations for enhancing the scheme
To bring the existing implementation closer to a complete and usable secure data transfer/storage system
Allow the transfer of files of arbitrary size
Allow duplicate transfer of files to a second target storage device
Configuring and starting the initiator and target software and transferring user data requires significant command line interaction which can be complex and could be simplified with a GUI
It is not clear what you mean by “Relative complexity of existing scheme in terms of user interaction”
Bringing  Bring
Allowing  Allow
4/25/2019
Summers - SAiSCSI

18. Proposed Enhancements
4/25/2019
Summers - SAiSCSI

19. Secure Asymmetric iSCSI for Online Storage Enhancements
Enhancements to the sg_dd utility to:
Transfer of files of arbitrary size
Transfer to two targets
Graphical User Interface
Initiator interface
Target interface
4/25/2019
Summers - SAiSCSI

20. Implementation Logic for Arbitrary File Transfer
Set block_size = 1024
If input_file != target_storage_device
Determine size of file in bytes
If ((size % 1024) != 0)
{ while((size_file_in_bytes % 1024) != 0)
{ size++; } }
count = size/1024
Create CDB using count value
Changing the default ‘bs’ value in the sg_dd code to 1024
Adding a function to determine the size of the file in bytes. If the returned value is not block size (1024 bytes) or an integer multiple of block size, the count value must be set to the next integer multiple of the number of blocks to be written. This value is then used for constructing the command descriptor blocks, reading the data from the initiator, and writing to the target.
4/25/2019
Summers - SAiSCSI

21. Implementation Logic for File Transfer to Two Targets
Set block_size = 1024
If (second output file == TRUE) {
Determine device type
Determine number of blocks in second target
create command descriptor block for second target
} else {
set second output file to be /dev/null }
Add another command line argument to allow the user to specify a second target device.
Process the command line options to determine whether one or two target devices are specified.
If a second target is specified, determine the type of device it is.
Determine the number of blocks and the block size of the second target.
Create a second command descriptor block for the second target.
If a second target file is not specified, the second target is set to /dev/null. As in the case of a specified second target, a command descriptor block is created.
The data is written to a special file that discards all data written to it (but reports that the write operation succeeded)
Chow: Write to /dev/null still takes some instruction. Should it be implemented in actual operation by testing the flag of two targets.
4/25/2019
Summers - SAiSCSI

22. File Transfer Performance Results
4/25/2019
Summers - SAiSCSI

23. Comparison of Real Times for Arbitrary File Sized Transfers
4/25/2019
Summers - SAiSCSI

24. Comparison of System Times for Arbitrary File Sized Transfers
4/25/2019
Summers - SAiSCSI

25. Comparison of User Times fot Arbitrary Sized File Transfers
4/25/2019
Summers - SAiSCSI

26. Comparison of Real Times for Transfers to Two Targets
4/25/2019
Summers - SAiSCSI

27. Comparison of System Times for Transfer to Two Targets
4/25/2019
Summers - SAiSCSI

28. Comparison of User Times for Transfers to Two Targets
4/25/2019
Summers - SAiSCSI

29. User Interface
4/25/2019
Summers - SAiSCSI

30. User Interface Developed and implemented using Python and Tkinter
Comprised of two components
Initiator Interface
Target Interface
4/25/2019
Summers - SAiSCSI

31. Initiator Interface Functionality
Generate IPsec keys
Generate SAD and SPD entries
Start initiator software
Login/Logout to/from the target(s)
Transfer user data to target storage
Retrieve user data from target storage
4/25/2019
Summers - SAiSCSI

32. Initiator User Interface
4/25/2019
Summers - SAiSCSI

33. Target Interface Functionality
Create additional target storage file(s)
Configure ietd.conf file for additional target(s)
Generate SAD and SPD entries
Start/stop iscsitarget software
4/25/2019
Summers - SAiSCSI

34. Target User Interface
4/25/2019
Summers - SAiSCSI

35. Secure Asymmetric iSCSI for Disaster Recovery
HIPPA and SOX require security, privacy and accountability of data
Standard online storage techniques may not be sufficient
Secure Asymmetric iSCSI for online storage may be the solution
Secure storage of Custom key
Initiator and custom key can be duplicated
Store key with trusted third party
HIPPA and SOX require security, privacy and accountability of stored data. There is also the requirement to disclose data with a specified time period. The advent of online storage goes some way to addressing these issues. However, standard online storage techniques may not be sufficient. The Secure Asymmetric iSCSI for Online Storage scheme offers a potential solution.
However, a number of issues remain to be resolved.
In the current implementation the custom key is stored on the initiator. If the initiator is destroyed there is no way to decrypt the data.
One solution is to duplicate the initiator and thus the custom key. However, doing so effectively dilutes the security provided by the key.
A second alternative is to store the key with a trusted third party. However, in the event of a disaster the time taken to recover the key from the third party may be unacceptable in terms of the delay in restoring availability of data.
Obviously, this is an area that needs further consideration.
4/25/2019
Summers - SAiSCSI

36. Additional Research
During testing of the Efficient Asymmetric iSCSI scheme, it was discovered that the scheme could not be used with a mounted target storage device. Since the mounting of a target storage device is highly desirable, it was decided to examine network protocol analyzer logs for amongst other things the mount command.
Since the Linux kernel, open-iscsi and iscsitarget software has undergone significant changes since the implementation of EASI, it was decided to create an updated test bed with more recent versions of the kernel and software
4/25/2019
Summers - SAiSCSI

37. Mounting a Target Storage Device
Network Protocol Analyzer (Wireshark) Results
Discovery – only iSCSI/SCSI commands issued
Login – iSCSI/SCSI Read and PDU but PUSH flag not set
Mounting results in the issuing of iSCSI/SCSI Writes and Reads
iSCSI/SCSI Writes/Reads interpreted by EASI as file transfers
strace of mount command
kernel reads file system information which results in invocation of iSCSI/SCSI commands
It was know that both discovery and login are successful with the existing scheme therefore network protocol analyzer logs were taken during these processes for comparison purposes against the mount log.
During discovery only iSCSI/SCSI commands were found to be issued. In comparison during the login process an iSCSI/SCSI Read and its associated Data-in PDU were issued, examination of these packets revealed that the PSH flag was not set and therefore, the packet would not be processed by the EASI scheme.
Examination of the mount log revealed that a number of iSCSI/SCSI Reads and writes and associated Data-in and Data-out PDUs were issued. Closer examination of the packets suggested that there packets would be processed by EASI. In order to understand why the Reads and Writes occurred the mount command was examined further and an strace log was obtained. It was found that mount is a file system operation handled by the Virtual File System. During the mount the kernel reads file system information and sets up file descriptors. As a result, read and write commands are issued which may result in iSCSI/SCSI Reads and Writes being issued.
4/25/2019
Summers - SAiSCSI

38. Using cp to Transfer to a Mounted Storage Device
Writing to Target
Various iSCSI/SCSI Writes and Data-out PDUs
In addition to user payload, file and directory structure of storage device written
Reading from Target
Various iSCSI/SCSI Reads and Writes
Plain vanilla TCP packet after first iSCSI/SCSI Read contains names of directories and files currently on target storage
Unidentified data in some Data-out PDUs
4/25/2019
Summers - SAiSCSI

39. Lessons Learnt/Observations
sg_dd uses defaults of stdin for input file and stdout for output file.
If no output is required use /dev/null
Linux kernel, open-iscsi and iscsitarget under constant development.
Therefore changes to these codes need to be incorporated in new releases.
Wireshark and Ethereal appear to interpret/display identical log files in a different manner.
Retrieving data from target using sg_dd requires count value to be specified.
sg_dd code uses defaults of standard input stream (stdin) for input file and standard output stream (stdout) for output file. If no output file is required /dev/null is used. This information was used when reading back from the target.
4/25/2019
Summers - SAiSCSI

40. Future Directions
Modify the EASI scheme to use the most current Linux kernel version, open-iscsi and iscsitarget code.
Re-implement the EASI scheme in a way that mounted target storage disk can be used.
Improve the simplification of the setup and use of the implementation through the development of an API.
Enhance the implementation to utilize a dynamic method, such as ‘racoon’, for establishing security associations between the initiator and target
4/25/2019
Summers - SAiSCSI

41. Conclusions Enhancements to the sg_dd utility has enabled:
Simulated transfer of files of arbitrary size
Transfer of files to two targets
Graphical user interface simplifies user interaction:
Keys for the encryption of the headers has been simplified
Interaction with the initiator and target software has been simplified
Writing and reading user data to /from the target has been simplified
4/25/2019
Summers - SAiSCSI

42. Conclusions (continued)
Limitations
Arbitrary file transfer still requires user to specify a count value when reading data from target storage
True arbitrary file transfer is not achieved
4/25/2019
Summers - SAiSCSI

43. ADDITIONAL SLIDES SCSI Command Descriptor Blocks
Phases of SCSI I/O Operation
SCSI Architecture in Linux Kernel
General Structure of iSCSI PDU
iSCSI PDU Basic Header Segment
iSCSI Protocol Layering Model
Data Encapsulation
Open-iSCSI
Iscsitarget
Packet for IPsec Transport Mode
Write Processing on the Initiator
Write Processing on the Target
Read Processing on the Target
Read Processing on the Initiator
4/25/2019
Summers - SAiSCSI

44. SCSI Command Descriptor Block
Operation Code – the operation being requested e.g., read or write and length of CDB
Logical Block Address – Identifies the location of the data on the physical medium
Transfer Length – The amount of data to be transferred (number of blocks) Marked as if required since some commands transfer no data
Parameter List Length – Identifies the number of bytes that an application client wishes to transfer. If the value is zero no data is transferred.
Allocation length – The maximum number of bytes an application client wishes to transfer
4/25/2019
Summers - SAiSCSI

45. Phases of SCSI I/O Operation
3 main phases:
Command
Initiator sends command and parameters to the target in a CDB
Data
Data is transferred in accordance with command issued in CDB
Status
Provides confirmation that command executed is received
Bare minimum command and status no data is transferred
4/25/2019
Summers - SAiSCSI

46. SCSI Subsystem in Linux Kernel
Upper Layer – Main entry point
Mid Layer – Unifying layer
Lower Layer - Drivers
4/25/2019
Summers - SAiSCSI

47. General Structure of iSCSI PDU
4/25/2019
Summers - SAiSCSI

48. iSCSI PDU Basic Header Segment
4/25/2019
Summers - SAiSCSI

49. iSCSI Protocol Layering Model
4/25/2019
Summers - SAiSCSI

50. Data Encapsulation
4/25/2019
Summers - SAiSCSI

51. Open-iSCSI Open source software implementation of iSCSI initiator
Kernel Portion
iSCSI data path
User Portion
Control path of iSCSI
Management facilities
Release used in EASI: open-iscsi
Current Release: open-iscsi
4/25/2019
Summers - SAiSCSI

52. iscsitarget Open source software implementation of iSCSI target
Kernel Portion
User Portion
Release used in EASI: iscsitarget
Current Release: iscsitarget
4/25/2019
Summers - SAiSCSI

53. Packet for IPsec Transport Mode
4/25/2019
Summers - SAiSCSI

54. Write Processing on the Initiator
Identify iSCSI traffic – source port 3260
Determine if iSCSI packet contains user data
Encrypt packets not containing user data
Encrypt packets containing user data
4/25/2019
Summers - SAiSCSI

55. Write Processing on the Target
Identify iSCSI traffic
Identify packets containing user data
Decrypt packets not containing user data
Process packets containing user data
4/25/2019
Summers - SAiSCSI

56. Read Processing on the Target
Identify iSCSI traffic – source port 3260
Determine if iSCSI packet contains user data
Encrypt packets not containing user data
Encrypt packets containing user data
4/25/2019
Summers - SAiSCSI

57. Read Processing on the Initiator
Identify iSCSI traffic
Determine if packet contains user data
Decrypt packets containing user data
Decrypt packets not containing user data
4/25/2019
Summers - SAiSCSI

58. Comparison of Standard IPsec and EASI at Packet Level
4/25/2019
Summers - SAiSCSI

59. Transfer Times Arbitrary Sized Files
4/25/2019
Summers - SAiSCSI

60. Transfer Times for 2 Targets
4/25/2019
Summers - SAiSCSI