1. Ken Cantrell / NetApp Mark Rogov / EMC July 30, 2015

2. © 2015 Storage Networking Industry Association. All Rights Reserved. About The Speakers 2 Mark Rogov EMC Systems Engineer @rogovmark Dr. J Metz Cisco R&D Engineer @drjmetz Ken Cantrell NetApp Manager Perf Engineering @kencantrelljr

3. © 2015 Storage Networking Industry Association. All Rights Reserved. SNIA Legal Notice The material contained in this tutorial is copyrighted by the SNIA unless otherwise noted. Member companies and individual members may use this material in presentations and literature under the following conditions: Any slide or slides used must be reproduced in their entirety without modification The SNIA must be acknowledged as the source of any material used in the body of any document containing material from these presentations. This presentation is a project of the SNIA Education Committee. Neither the author nor the presenter is an attorney and nothing in this presentation is intended to be, or should be construed as legal advice or an opinion of counsel. If you need legal advice or a legal opinion please contact your attorney. The information presented herein represents the author's personal opinion and current understanding of the relevant issues involved. The author, the presenter, and the SNIA do not assume any responsibility or liability for damages arising out of any reliance on or use of this information. NO WARRANTIES, EXPRESS OR IMPLIED. USE AT YOUR OWN RISK. 3

4. © 2015 Storage Networking Industry Association. All Rights Reserved. Storage Performance Benchmarking 4 SYSTEM UNDER TEST METRICS AND TERMINOLOGY FILE COMPONENTS BLOCK COMPONENTS WORKLOAD DEFINITIONS

5. © 2015 Storage Networking Industry Association. All Rights Reserved. Storage Performance Benchmarking 5 SYSTEM UNDER TEST METRICS AND TERMINOLOGY FILE COMPONENTS BLOCK COMPONENTS WORKLOAD DEFINITIONS TODAY FUTURE WEBCASTS INTROBLOCKFILEGRAPH

6. © 2015 Storage Networking Industry Association. All Rights Reserved. BUSINESS RUN PRODUCTION APPLICATIONS 6 Three Great Buckets Of Testing DATABASE COMPUTE FARM VDI INTROBLOCKFILEGRAPH

7. © 2015 Storage Networking Industry Association. All Rights Reserved. BUSINESS RUN PRODUCTION APPLICATIONS BAKE-OFFS SINGLE OBSCURE METRIC 7 Three Great Buckets Of Testing UN*X DD IOMETER IOZONE VDBENCH DATABASE COMPUTE FARM VDI INTROBLOCKFILEGRAPH

8. © 2015 Storage Networking Industry Association. All Rights Reserved. BUSINESS RUN PRODUCTION APPLICATIONS BAKE-OFFS SINGLE OBSCURE METRIC 8 Three Great Buckets Of Testing UN*X DD IOMETER IOZONE VDBENCH DATABASE COMPUTE FARM VDI INTROBLOCKFILEGRAPH

9. © 2015 Storage Networking Industry Association. All Rights Reserved. BENCHMARKS BALANCED PROFESSIONAL APPROACH BUSINESS RUN PRODUCTION APPLICATIONS BAKE-OFFS SINGLE OBSCURE METRIC 9 Three Great Buckets Of Testing UN*X DD IOMETER IOZONE VDBENCH DATABASE COMPUTE FARM VDI LOGIN VSI VMMARK SPEC SFS SPC1 INTROBLOCKFILEGRAPH

10. © 2015 Storage Networking Industry Association. All Rights Reserved. 10 Block / SAN Terminology IO BLOCK / SAN PROTOCOL FC/iSCSI/FCoE Target Initiator INTROBLOCKFILEGRAPH

11. © 2015 Storage Networking Industry Association. All Rights Reserved. 11 Block / SAN Terminology IO Target BLOCK / SAN PROTOCOL FC/iSCSI/FCoE IO OPERATION AKA TYPE: READ WRITE Initiator INTROBLOCKFILEGRAPH

12. © 2015 Storage Networking Industry Association. All Rights Reserved. Block / SAN Operations 12 Source: http://www.t10.org/cgi-bin/ac.pl?t=f&f=sbc4r07c.pdf Graphic: http://www.wordle.net/create INTROBLOCKFILEGRAPH

13. © 2015 Storage Networking Industry Association. All Rights Reserved. 13 Block / SAN Terminology IO Target BLOCK / SAN PROTOCOL FC/iSCSI/FCoE IO OPERATION AKA TYPE: READ WRITE Initiator IOPS COUNT EVERY IO OPERATION PER SECOND THROUGHPUT SAME AS IOPS INTROBLOCKFILEGRAPH

14. © 2015 Storage Networking Industry Association. All Rights Reserved. Where To Measure IOPS? 14 HOSTS S TORAGE CONTROLLER D ISK DRIVES INTROBLOCKFILEGRAPH

15. © 2015 Storage Networking Industry Association. All Rights Reserved. Where To Measure IOPS? 15 HOSTS S TORAGE CONTROLLER D ISK DRIVES 500 400 300 200 100 F RONT E ND 1500 INTROBLOCKFILEGRAPH

16. © 2015 Storage Networking Industry Association. All Rights Reserved. Where To Measure IOPS? 16 HOSTS S TORAGE CONTROLLER D ISK DRIVES 500 400 300 200 100 1000 200 IDLE 15002200 F RONT E ND B ACK E ND INTROBLOCKFILEGRAPH

17. © 2015 Storage Networking Industry Association. All Rights Reserved. 17 Block / SAN Terminology IO Target BLOCK / SAN PROTOCOL FC/iSCSI/FCoE IO OPERATION AKA TYPE: READ WRITE Initiator IOPS COUNT EVERY IO OPERATION PER SECOND MB/S PAYLOAD SUM OF EVERY OPERATION PER SECOND BANDWIDTH SAME AS MB/S INTROBLOCKFILEGRAPH THROUGHPUT SAME AS IOPS

18. © 2015 Storage Networking Industry Association. All Rights Reserved. 1000 4 TERABYTES TB 1000 3 GIGABYTES GB MB/s Vs. MiB/s 18 MEGABYTES MB 1000 2 1000 1 KILOBYTES KB 1024 4 TEBIBYTES TiB 18 1024 3 GIBIBYTES GiB 1024 2 MEBIBYTES MiB 1024 1 KIBIBYTES KiB 10% 7% 5% 2% DECIMALBINARY INTROBLOCKFILEGRAPH

19. © 2015 Storage Networking Industry Association. All Rights Reserved. 19 Block / SAN Terminology IO Target BLOCK / SAN PROTOCOL FC/iSCSI/FCoE IO OPERATION AKA TYPE: READ WRITE IO SIZE SIZE OF DATA PAYLOAD IN AN IO OPERATION Initiator IOPS COUNT EVERY IO OPERATION PER SECOND MB/S PAYLOAD SUM OF EVERY OPERATION PER SECOND BANDWIDTH SAME AS MB/S INTROBLOCKFILEGRAPH THROUGHPUT SAME AS IOPS

20. © 2015 Storage Networking Industry Association. All Rights Reserved. MB/s (or MiB/s), IO Size, And IOPS 20 KEEPING IOPS CONSTANT, L ARGER I/O SIZES PRODUCE GREATER MiB/ S 512B 1KiB 4KiB INTROBLOCKFILEGRAPH

21. © 2015 Storage Networking Industry Association. All Rights Reserved. MB/s (or MiB/s), IO Size, And IOPS 21 AS IO S IZE INCREASES IOPS DECREASE FOR THE SAME MiB/ S AS IO S IZE INCREASES IOPS DECREASE FOR THE SAME MiB/ S 16KiB8KiB4KiB INTROBLOCKFILEGRAPH

22. © 2015 Storage Networking Industry Association. All Rights Reserved. IO SIZE SIZE OF DATA PAYLOAD IN AN IO OPERATION 22 Block / SAN Terminology IO Target BLOCK / SAN PROTOCOL FC/iSCSI/FCoE IO OPERATION AKA TYPE: READ WRITE Initiator IOPS COUNT EVERY IO OPERATION PER SECOND MB/S PAYLOAD SUM OF EVERY OPERATION PER SECOND BANDWIDTH SAME AS MB/S RESPONSE TIME TIME TARGET TAKES TO REPLY TO AN IO LATENCY SAME AS RESPONSE TIME INTROBLOCKFILEGRAPH THROUGHPUT SAME AS IOPS

23. © 2015 Storage Networking Industry Association. All Rights Reserved. Aggregate Property Of Response Time 23 D ISK DRIVES S TORAGE CONTROLLER HOSTS INTROBLOCKFILEGRAPH

24. © 2015 Storage Networking Industry Association. All Rights Reserved. Aggregate Property Of Response Time 24 D ISK DRIVES S TORAGE CONTROLLER HOSTS  WRITE ≥ 1ms 1ms INTROBLOCKFILEGRAPH

25. © 2015 Storage Networking Industry Association. All Rights Reserved. Aggregate Property Of Response Time 25 D ISK DRIVES S TORAGE CONTROLLER HOSTS   READ  WRITE ≥ 1.5ms ≥ 1ms 1ms 0.5ms INTROBLOCKFILEGRAPH

26. © 2015 Storage Networking Industry Association. All Rights Reserved. Aggregate Property Of Response Time 26 D ISK DRIVES S TORAGE CONTROLLER HOSTS   READ  WRITE ≥ 1.5ms ≥ 1ms READ  1ms ≥ 5ms 4ms 0.5ms INTROBLOCKFILEGRAPH

27. © 2015 Storage Networking Industry Association. All Rights Reserved. 27 File / NAS Terminology OP FILE PROTOCOL NFS/SMB/FTP Client Server INTROBLOCKFILEGRAPH

28. © 2015 Storage Networking Industry Association. All Rights Reserved. File / NAS Terminology 28 OP FILE PROTOCOL NFS/SMB/FTP Client Server FILE OPERATION SEVERAL DOZEN DEPENDING ON PROTOCOL INTROBLOCKFILEGRAPH

29. © 2015 Storage Networking Industry Association. All Rights Reserved. NFSv4 Operations 29 Source: https://www.ietf.org/rfc/rfc3530.txt Graphic: http://www.wordle.net/create INTROBLOCKFILEGRAPH

30. © 2015 Storage Networking Industry Association. All Rights Reserved. SMB3 Operations 30 Source: https://msdn.microsoft.com/en-us/library/cc246482.aspx (use [MS-SMB2].pdf) Graphic: http://www.wordle.net/create INTROBLOCKFILEGRAPH

31. © 2015 Storage Networking Industry Association. All Rights Reserved. File / NAS Terminology 31 OP FILE PROTOCOL NFS/SMB/FTP Client Server FILE OPERATION SEVERAL DOZEN DEPENDING ON PROTOCOL OPS COUNT EVERY PROTOCOL OPERATION PER SECOND INTROBLOCKFILEGRAPH

32. © 2015 Storage Networking Industry Association. All Rights Reserved. File / NAS Terminology 32 OP FILE PROTOCOL NFS/SMB/FTP Client Server FILE OPERATION SEVERAL DOZEN DEPENDING ON PROTOCOL OPS COUNT EVERY PROTOCOL OPERATION PER SECOND MB/S PAYLOAD SUM OF EVERY OPERATION PER SECOND THROUGHPUT SAME AS MB/S BANDWIDTH SAME AS MB/S INTROBLOCKFILEGRAPH

33. © 2015 Storage Networking Industry Association. All Rights Reserved. File / NAS Terminology 33 OP FILE PROTOCOL NFS/SMB/FTP Client Server FILE OPERATION SEVERAL DOZEN DEPENDING ON PROTOCOL OPS COUNT EVERY PROTOCOL OPERATION PER SECOND MB/S PAYLOAD SUM OF EVERY OPERATION PER SECOND THROUGHPUT SAME AS MB/S BANDWIDTH SAME AS MB/S INTROBLOCKFILEGRAPH

34. © 2015 Storage Networking Industry Association. All Rights Reserved. File / NAS Terminology 34 OP FILE PROTOCOL NFS/SMB/FTP Client Server FILE OPERATION SEVERAL DOZEN DEPENDING ON PROTOCOL OPS COUNT EVERY PROTOCOL OPERATION PER SECOND MB/S PAYLOAD SUM OF EVERY OPERATION PER SECOND THROUGHPUT SAME AS MB/S BANDWIDTH SAME AS MB/S RESPONSE TIME TIME TARGET TAKES TO REPLY TO AN IO LATENCY SAME AS RESPONSE TIME INTROBLOCKFILEGRAPH

35. © 2015 Storage Networking Industry Association. All Rights Reserved. Difference Between OPS and MB/s (MiB/s) Two Otherwise Identical Runs With Different NFS Operations Mix (64 KiB IO Size) 35 INTROBLOCKFILEGRAPH

36. © 2015 Storage Networking Industry Association. All Rights Reserved. Difference Between OPS and MB/s (MiB/s) Two Otherwise Identical Runs With Different NFS Operations Mix (64 KiB IO Size) 36 High MetaData Operations High Data Operations INTROBLOCKFILEGRAPH

37. © 2015 Storage Networking Industry Association. All Rights Reserved. Difference Between OPS and MB/s (MiB/s) Two Otherwise Identical Runs With Different NFS Operations Mix (64 KiB IO Size) 37 High MetaData Operations High Data Operations INTROBLOCKFILEGRAPH

38. © 2015 Storage Networking Industry Association. All Rights Reserved. Graph Fun Or Why “It Depends” Is Relevant? 38 5,000 10,000 INTROBLOCKFILEGRAPH

39. © 2015 Storage Networking Industry Association. All Rights Reserved. Graph Fun Or Why “It Depends” Is Relevant? 39 INTROBLOCKFILEGRAPH

40. © 2015 Storage Networking Industry Association. All Rights Reserved. Graph Fun Or Why “It Depends” Is Relevant? 40 Operations per sec, OPS Response time, ms INTROBLOCKFILEGRAPH

41. © 2015 Storage Networking Industry Association. All Rights Reserved. Graph Fun Or Why “It Depends” Is Relevant? 41 Operations per sec, OPS Response time, ms INTROBLOCKFILEGRAPH

42. © 2015 Storage Networking Industry Association. All Rights Reserved. M AXIMUM PERFORMANCE NEED M AXIMUM PERFORMANCE NEED Graph Fun Or Why “It Depends” Is Relevant? 42 Operations per sec, OPS Response time, ms INTROBLOCKFILEGRAPH

43. © 2015 Storage Networking Industry Association. All Rights Reserved. M INIMUM PERFORMANCE REQUIREMENT M INIMUM PERFORMANCE REQUIREMENT Graph Fun Or Why “It Depends” Is Relevant? 43 Operations per sec, OPS Response time, ms INTROBLOCKFILEGRAPH

44. © 2015 Storage Networking Industry Association. All Rights Reserved. A CCEPTABLE L ATENCY B AND Graph Fun Or Why “It Depends” Is Relevant? 44 Operations per sec, OPS Response time, ms INTROBLOCKFILEGRAPH

45. © 2015 Storage Networking Industry Association. All Rights Reserved. A CCEPTABLE L ATENCY B AND Variance 45 Operations per sec, OPS Response time, ms INTROBLOCKFILEGRAPH

46. © 2015 Storage Networking Industry Association. All Rights Reserved. Other Metrics To Consider 46 Operations per sec, OPS Response time, ms $2/OP $10/OP $1.1/OP $5.5/OP INTROBLOCKFILEGRAPH

47. © 2015 Storage Networking Industry Association. All Rights Reserved. Other Metrics To Consider 47 Operations per sec, OPS Response time, ms $1.1/OP $5.5/OP $25K/U $1K/U $40/W $140/W } @9000 OPS @ 17 ms } @9000 OPS @ 10 ms INTROBLOCKFILEGRAPH

48. © 2015 Storage Networking Industry Association. All Rights Reserved. Other Metrics To Consider 48 Operations per sec, OPS Response time, ms $1.1/OP $5.5/OP $25K/U $1K/U $40/W $140/W } @9000 OPS @ 17 ms } @9000 OPS @ 10 ms CONTEXT MAKES METRICS MATTER CONTEXT MAKES METRICS MATTER INTROBLOCKFILEGRAPH

49. © 2015 Storage Networking Industry Association. All Rights Reserved. Summary 49 GRAPH FUN CONTEXT MAKES METRICS MATTER CONTEXT MAKES METRICS MATTER OPS COUNT EVERY PROTOCOL OPERATION PER SECOND MB/S PAYLOAD SUM OF EVERY OPERATION PER SECOND TERMINOLOGY IOPS COUNT EVERY IO OPERATION PER SECOND RESPONSE TIME TIME TARGET TAKES TO REPLY TO AN IO INTROBLOCKFILEGRAPH

50. © 2015 Storage Networking Industry Association. All Rights Reserved. Storage Performance Benchmarking 50 SYSTEM UNDER TEST METRICS AND TERMINOLOGY FILE COMPONENTS BLOCK COMPONENTS WORKLOAD DEFINITIONS TODAY FUTURE WEBCASTS

51. © 2015 Storage Networking Industry Association. All Rights Reserved. After This Webcast 51 This Webcast and a PDF of the slides will be posted to the SNIA Ethernet Storage Forum (ESF) website and available on-demand http://www.snia.org/forums/esf/knowledge/webcasts A full Q&A from this webcast, including answers to questions we couldn't get to today, will be posted to the SNIA-ESF blog http://sniaesfblog.org/ Follow us on Twitter @SNIAESFSNIAESF Next Webcast - October 21 st “Storage Performance Benchmarking: Part 2”