StorPerf - Cinder Storage Performance Measurement

Slides:



Advertisements
Similar presentations
Performance Testing - Kanwalpreet Singh.
Advertisements

CloudStack Scalability Testing, Development, Results, and Futures Anthony Xu Apache CloudStack contributor.
Agile Infrastructure built on OpenStack Building The Next Generation Data Center with OpenStack John Griffith, Senior Software Engineer,
Orchestration of Fibre Channel Technologies for Private Cloud Deployments OpenStack Summit; Ecosystem Track April 15 th, 2013 Oregon Convention Center.
1 NETE4631 Cloud deployment models and migration Lecture Notes #4.
DataGrid is a project funded by the European Union 22 September 2003 – n° 1 EDG WP4 Fabric Management: Fabric Monitoring and Fault Tolerance
  Copyright 2003 by SPAN Technologies. Performance Assessments of Internet Systems By Kishore G. Kamath SPAN Technologies Testing solutions for the enterprise.
An Adaptable Benchmark for MPFS Performance Testing A Master Thesis Presentation Yubing Wang Advisor: Prof. Mark Claypool.
Public and Private Clouds: Working Together
IETF 90: VNF PERFORMANCE BENCHMARKING METHODOLOGY Contributors: Sarah Muhammad Durrani: Mike Chen:
Distributed Data Stores – Facebook Presented by Ben Gooding University of Arkansas – April 21, 2015.
Computer System Lifecycle Chapter 1. Introduction Computer System users, administrators, and designers are all interested in performance evaluation. Whether.
GOVERNMENT SERVICES INTEGRATION INDUSTRY SOLUTION.
Toolbox for Dimensioning Windows Storage Systems Jalil Boukhobza, Claude Timsit 12/09/2006 Versailles Saint Quentin University.
Key Perf considerations & bottlenecks Windows Azure VM characteristics Monitoring TroubleshootingBest practices.
Storage Benchmark Proposal Edgar StPierre, EMC. Proposal Project Name: STORPERF Repo Name: STORPERF Category: Requirements Project Lead: Edgar StPierre,
FlashSystem family 2014 © 2014 IBM Corporation IBM® FlashSystem™ V840 Product Overview.
An I/O Simulator for Windows Systems Jalil Boukhobza, Claude Timsit 27/10/2004 Versailles Saint Quentin University laboratory.
INNOV-10 Progress® Event Engine™ Technical Overview Prashant Thumma Principal Software Engineer.
Maryam Tahhan Al Morton Benchmarking Virtual Switches in OPNFV draft-vsperf-bmwg-vswitch-opnfv-01.
Scalability == Capacity * Density.
PRESENTATION TITLE GOES HERE Emerald NAS Extensions Chuck Paridon Performance Architect H-P Enterprise Data Contributed by Nick Principe – EMC, Demartek.
Smart Grid Big Data: Automating Analysis of Distribution Systems Steve Pascoe Manager Business Development E&O - NISC.
REMINDER Check in on the COLLABORATE mobile app Oracle Performance Management with vCenter Operations Manager and Oracle Enterprise Manager (OEM) Adapter.
Statistical process model Workshop in Ukraine October 2015 Karin Blix Quality coordinator
StorPerf Overview Mark Beierl May 9, Agenda Goals of the StorPerf Project Architecture Challenges Executing a Performance Run Displaying the Results.
Tintri Overview Terry Curtis Systems Engineer. Adoption of Virtualization 2% % % 2018.
Canadian Bioinformatics Workshops
Learnings from the first Plugfest
Microsoft Cloud Adoption Framework Foundation
Outline Introduction and motivation, The architecture of Tycho,
Monitoring Windows Server 2012
Chapter 19: Network Management
Software Architecture in Practice
Diskpool and cloud storage benchmarks used in IT-DSS
Using OpenStack to Measure OpenStack Cinder Performance
Parametric Bottlenecks Testing Catalogue (POSCA)
Managing your IT Environment
Windows Server* 2016 & Intel® Technologies
HPE Persistent Memory Microsoft Ignite 2017
The Client/Server Database Environment
Performance Testing Methodology for Cloud Based Applications
Manufacturing system design (MSD)
OpenStack Ani Bicaku 18/04/ © (SG)² Konsortium.
Lecture 11: DMBS Internals
Direct Attached Storage and Introduction to SCSI
Real IBM C exam questions and answers
SQL Server 2012 Licensing Overview.
Northbound API Dan Shmidt | January 2017
Dev Test on Windows Azure Solution in a Box
ASM-based storage to scale out the Database Services for Physics
20409A 7: Installing and Configuring System Center 2012 R2 Virtual Machine Manager Module 7 Installing and Configuring System Center 2012 R2 Virtual.
Chapter 9: IOS Images and Licensing
Building a Database on S3
Lecture 1: Multi-tier Architecture Overview
Operating Systems : Overview
Cloud computing mechanisms
Operating Systems : Overview
Operating Systems : Overview
Operating Systems : Overview
IP Control Gateway (IPCG)
Data Warehousing Concepts
Performance And Scalability In Oracle9i And SQL Server 2000
OpenStack Summit Berlin – November 14, 2018
PerformanceBridge Application Suite and Practice 2.0 IT Specifications
Mr. M. D. Jamadar Assistant Professor
The StarlingX Story Learn, Try, Get Involved!
OpenStack for the Enterprise
Bending Ironic for Big Iron
Presentation transcript:

StorPerf - Cinder Storage Performance Measurement Mark Beierl, EMC

Where Did We Start SNIA[1] Solid State Storage Performance Test Specification “Manufacturers need to set, and customers need to compare, the performance of Solid State Storage (SSS) devices. This Specification defines a set of device level tests and methodologies intended to enable comparative testing of SSS devices in Enterprise systems” Guide to obtain reliable and comparative measurements Tests and methodologies Use known and repeatable test stimulus Based on guidelines from SNIA SSS Technical Working Group Isolates the device being tested from the test platform [1]Storage Networking Industry Association: http://www.snia.org

Goals of the StorPerf Project To provide a report based on SNIA’s Performance Test Specification Tests Cinder block storage using specified Cinder drivers Measures Latency Throughput in Bytes/Second IOPS Across a matrix of Block Sizes Queue Depths this spec was designed for testing SSDs, but we’ve extended/applied it to this test case because of the insidious effects of SSD behavior from initialization to steady state. To talk about how we see SSD as being an important technology for deployment in remote POPs due to reduced maintenance. And, even given all that, how we’ve extended and applied the testing provides relevance to HDD testing as well, as is currently the case in Pharos Pods.

Architecture Single Docker container controls test run Using Heat, creates: Volumes in Cinder VMs in Nova Private network and security group for VMs Floating IP addresses Container has ReST API (Swagger) for control and reports Graphite can also be exposed for ad hoc queries Carbon DB stores all metrics

Architecture

Challenges Logical Volumes Time SSD and HD have very different performance characteristics SSD exhibits transient elevated performance when fresh HD exhibits better performance after cache warm up Logical Volumes Not testing raw disk Cannot control cache or initialization Each storage implementation has its own performance characteristics Time Initializing large Cinder volumes can take a long time

Test Definitions Steady State Transition Zone A device is in Steady State when for a given (y) Max(y) – Min(y) is less than 20% of avg(y) Slope(y) is less than 10% Transition Zone State when volume’s performance is changing, such as during initial data fill Max Slope Min

Test Concepts Steady State Verification Ensure the initial performance or transition states are not reported as typical Do not report warm up or volume data fill Capture meaningful indication of volume’s performance during the bulk of its operating life Verification By inspection, while issuing IO requests Statistical analysis of last N rounds of sample data to determine when steady state has been reached

Test Concepts Active Range Data Patterns Queue Depth Definition of how much of the underlying Cinder pool is under test Can span from 1GB to 100% of all available block storage Data Patterns All tests are run with random data patterns FIO is the data generation utility See: http://freecode.com/projects/fio Queue Depth The number of outstanding I/O operations kept in flight throughout the test duration

Test Concepts Latency IOPS Workload Caching Duration between issuance of I/O request and its completion IOPS I/O operations per second, regardless of block size Workload Generation of I/O requests Specifies Read, Write, or mix of both operations Specifies sequential or random volume data access Caching StorPerf relies on Cinder Driver for tuning

Overview of Test Flow Configure environment: Number of agents. This controls the width of the run and simulates heavier load on Cinder Size of Cinder volumes per agent Floating IP network to use Creation of Cinder volumes (one per agent) Creation of Agent VMs Attach volumes to agents Boot Ubuntu 14.04 stock distribution Copy FIO to agents

Overview of Test Flow Initial Volume Fill Run Test Execute sequential write to fill 100% of each volume Volumes may be lazily created, therefore read tests prior to filling volume is invalid Transitional state - do not report these numbers Run Test Matrix of block size, queue depth and workload specifies Run workload up to 100% of volume size across all agents simultaneously Record intermediate data results of workload Examine past 5 test results for Steady State If reached, terminate workload and move on to next If 100% reached, record lack of Steady State and move on

What is Available Today? Ability to execute test across Workloads (Read, Write, Mix) and access (random, sequential) Queue depths Block sizes (note: Ceph is limited to 16K max block size) Report on average latencies 3D plot for matrix JSON for external reporting Built in Graphite browser Custom reporting of all raw data Statistics gathered once every 60 seconds

Existing Reports OPNFV Test DB Text results (http://storperf:5000/api/v1.0/job?id=NNNN) "rw.queue-depth.1.block-size.16384.duration": 1841, "rw.queue-depth.1.block-size.16384.read.iops": 51.52010714285715, "rw.queue-depth.1.block-size.16384.read.latency": 1505.8304166666665, "rw.queue-depth.1.block-size.16384.read.throughput": 823.8154761904763, Graph results (http://storperf:5000/results/NNNN)

Where to From Here? Linux Foundation Intern Project StorPerf is helping to define framework for LF Interns Tim Rault at CENGN paving the way Implementing steady state analysis Generating interest from other students Further adoption of SNIA reports Steady state convergence plot Steady state verification plot Per measurement value plots

Steady State Detection Currently: StorPerf reports on average across the entire run Transitional state may be included in final result at this time Target: Report only once steady state has been reached SNIA has guidelines on what determines steady state Statistically Valid Warm up, volume fill, etc. Statistically invalid

Steady State Report Steady State Convergence Report When 5 samples are found that fit the definition of steady state, plot the proof of convergence Max / Min values for the series Average values Calculated slope of the series

More Reports Measurement Tabular Report 2D Measurement Plot Read / Write Mix Block Sizes IOPS, Throughput or latency 2D Measurement Plot IOPS, Throughput or latency

In Summary StorPerf … what’s in your storage subsystem? “Standing on the shoulders”... A common theme! Based on SNIA, using accepted methodologies Characterization of Virtual vs. Physical Testing from inside the OpenStack infrastructure Still more work ahead Refinement of reporting, statistically relevant data points Yardstick integration and OPNFV test database dashboards Paying it forward Student and Intern friendly Open and welcome for new contributors these tests are marginally helpful to Pod testing today, but should be of significant value to standing up larger data centers and applying tests such as defined by Bottlenecks, or just run standalone in any larger scale lab or pre-production deployment

More Info Bi-weekly meetings Wiki: Mailing list tag [StorPerf] http://wiki.opnfv.org/display/StorPerf Developer guide Installation guide Performance test execution guide SNIA References and targets Mailing list tag [StorPerf] IRC Channel: #opnfv-storperf Bi-weekly meetings Wednesdays at 7:00 Pacific. Next meeting July 6

StorPerf - Cinder Storage Performance Measurement Mark Beierl, EMC