© 2012 Whamcloud, Inc. Whamcloud and Quality Chris Gearing & Mike Stok Software Engineers Whamcloud, Inc Lustre User Group Austin TX April 2012 Version.

Slides:



Advertisements
Similar presentations
Distributed Data Processing
Advertisements

DELIVERING SHAREPOINT AS A SERVICE
Beyond the Help Desk Getting ahead of the game Mihaela Damian Dan Sexton 11 th July 2013 CSCS > School of Clinical Medicine > University of Cambridge.
1 Planetary Network Testbed Larry Peterson Princeton University.
® IBM Software Group © 2010 IBM Corporation What’s New in Profiling & Code Coverage RAD V8 April 21, 2011 Kathy Chan
2004 Cross-Platform Automated Regression Test Framework Ramkumar Ramalingam, Rispna Jain IBM Software Labs, India.
All content in this presentation is protected – © 2008 American Power Conversion Corporation Rael Haiboullin System Engineer Change Manager.
The Digital Preservation Network at UT Austin Chris Jordan Texas Advanced Computing Center.
Managing Change in Production BW at General Mills Mike Ruesewald Cindy Burlet Chris Gunn.
“Open Benchmark for Lustre” Initiative November 14,
WORKDAY TECHNOLOGY Stan Swete CTO - Workday 1.
Welcome Course 20410B Module 0: Introduction Audience
Software Testing Test Design and Implementation. Agenda Test Design Test Implementation Test Design Sources Automated Testing 2.
Configuration Management Process and Environment MACS Review 1 February 5th, 2010 Roland Moser PR a-RMO, February 5 th, 2010 R. Moser 1 R. Gutleber.
MDC Open Information Model West Virginia University CS486 Presentation Feb 18, 2000 Lijian Liu (OIM:
Data Mining on the Web via Cloud Computing COMS E6125 Web Enhanced Information Management Presented By Hemanth Murthy.
Esri UC2013. Technical Workshop. Technical Workshop 2013 Esri International User Conference July 8–12, 2013 | San Diego, California ArcGIS for Local Government.
Continuous Delivery Ajey Gore Head of Technology ThoughtWorks India.
ArcGIS Workflow Manager An Introduction
Yury Kissin Infrastructure Consultant Storage improvements Dynamic Memory Hyper-V Replica VM Mobility New and Improved Networking Capabilities.
Drilling Data Management System
Implementation of HUBzero as a Knowledge Management System in a Large Organization HUBBUB Conference 2012 September 24 th, 2012 Gaurav Nanda, Jonathan.
Framework for Automated Builds Natalia Ratnikova CHEP’03.
VeribisCRM CUSTOMER RELATIONSHIP MANAGEMENT Engin Duran Experience is our know how.
Copyright © 2006 by The McGraw-Hill Companies, Inc. All rights reserved. McGraw-Hill Technology Education Copyright © 2006 by The McGraw-Hill Companies,
Data platform market will be 36.2B by 2012Database key in hosted scenarios.
March 26, 2003The Navigo Project Hans C. Masing, The University of Michigan Lance D. Speelmon, Indiana University An IMS and OKI Compliant Open Source.
© 2011 Whamcloud, Inc. Whamcloud Quality Chris Gearing Sr. Software Engineer Whamcloud, Inc. Lustre User Group Orlando Fl April 2011.
May 29 th, 2003 Curtis Anderson Sivaprasad Padisetty.
Software Development Software Testing. Testing Definitions There are many tests going under various names. The following is a general list to get a feel.
Test Case Manager v 3.0 Pierce Business Systems High Bridge Road Monroe, WA with customization by Ron Utz of Esker, Inc.
8.1 Lawson Security Overview Del Dehn Product Manager.
Contents 1.Introduction, architecture 2.Live demonstration 3.Extensibility.
© 2010 Voltaire Inc. HPCFS AT ORLANDO LUG 2011 BILL BOAS PATH FORWARD FOR LUSTRE COMMUNITY System Fabric Works.
Tivoli Software © 2009 IBM Corporation Maximo Mobile Suite David Havican Lead Product Designer & Architect.
Michael Still Google Inc. October, Managing Unix servers the slack way Tools and techniques for managing large numbers of Unix machines Michael.
SOA-14: Deploying your SOA Application David Cleary Principal Software Engineer.
Celine DONDEYNAZ, Joint Research Centre- Italy A. Leone, C. Carmona, P. Mainardi, M.Giacomassi and Prof. Daoyi Chen A Web knowledge Management Platform.
System Center Operations Manager 2007 Overview Amit Gatenyo Infrastructure & Security Team Lead Dario.
Open Scalable File Systems, Inc. (Nonprofit, User Group, “CO-OP,” User Community, Mutual Benefit Corp) (OpenSFS) – Status Update Norman.
Chapter 6 CASE Tools Software Engineering Chapter 6-- CASE TOOLS
GCRC Meeting 2004 BIRN Coordinating Center Software Development Vicky Rowley.
Terascala – Lustre for the Rest of Us  Delivering high performance, Lustre-based parallel storage appliances  Simplifies deployment, management and tuning.
Rob Davidson, Partner Technology Specialist Microsoft Management Servers: Using management to stay secure.
© 2011 Whamcloud, Inc. Whamcloud Overview Brent Gorda President and CEO Whamcloud, Inc. LUG2011 Orlando.
Infrastructure as code. “Enable the reconstruction of the business from nothing but a source code repository, an application data backup, and bare metal.
Changing Developer Behavior Using Automatic Test Intel Agile and Lean Development Conference Chris Gearing 23 rd May 2014 Version 1.0.
Ahmed Idris Tahir Waseel Application Service Provider.
70-293: MCSE Guide to Planning a Microsoft Windows Server 2003 Network, Enhanced Chapter 6: Planning, Configuring, And Troubleshooting WINS.
1 Comments to SPI. 2 General remarks Impressed by progress since last review Widespread adoption by experiments and projects Savannah, ExtSoft Build system.
Professional Services NAME Roland de Pijper Principal Consultant Openedge TITLE TECHNICAL EXPERTISE.
1 Punishment Through Continuous Delivery If it hurts, do it more often…
Windows Certification Paths OR MCSA Windows Server 2012 Installing and Configuring Windows Server 2012 Exam (20410) Administering Windows Server.
숭실대 IoT , Openstack, DevOps 숭실대
Platform & Engineering Services CERN IT Department CH-1211 Geneva 23 Switzerland t PES Agile Infrastructure Project Overview : Status and.
An assessment framework for Intrusion Prevention System (IPS)
Navigating the options for Data Redundancy
AlwaysOn Mirroring, Clustering
70-293: MCSE Guide to Planning a Microsoft Windows Server 2003 Network, Enhanced Chapter 6: Planning, Configuring, And Troubleshooting WINS.
X in [Integration, Delivery, Deployment]
IMPACTED TESTS BASED ON
LO2 - Be Able to Design IT Systems to Meet Business Needs
Course: Module: Lesson # & Name Instructional Material 1 of 32 Lesson Delivery Mode: Lesson Duration: Document Name: 1. Professional Diploma in ERP Systems.
Data Migration Assessment Jump Start – Engagement Kickoff
Enterprise Program Management Office
Continuous Integration
AEM Operations Dec 2017.
Key Takeaways.
UML Design for an Automated Registration System
Exploratory Testing Techniques : Finding Software Defects.
Presentation transcript:

© 2012 Whamcloud, Inc. Whamcloud and Quality Chris Gearing & Mike Stok Software Engineers Whamcloud, Inc Lustre User Group Austin TX April 2012 Version 1.0

© 2012 Whamcloud, Inc. Whamcloud’s View Of Quality A Year’s Progress The Year Ahead Maloo ‘Your Window onto Test’ – Mike Stok Agenda 2

© 2012 Whamcloud, Inc. Whamcloud’s View Of Quality 3

© 2012 Whamcloud, Inc. ‟ The only thing I’m non-Linear about is Whamcloud not delivering what it says it will deliver ” Whamcloud’s View Of Quality 4 Eric Barton CTO Whamcloud

© 2012 Whamcloud, Inc. Whamcloud’s view of quality development Whamcloud’s View Of Quality Stability Quality 5 Features Performance Time

© 2012 Whamcloud, Inc. Whamcloud is investing money, time and expertise in Quality …is continuing to develop tools and invest in infrastructure to enhance the Lustre community The whole of its engineering team is orientated around quality principles Whamcloud’s View Of Quality 6

© 2012 Whamcloud, Inc. A Year’s Progress 7

© 2012 Whamcloud, Inc. For landing testing we have carried out; –687 days of landing testing –2381 landing sessions – individual tests – passed Statistically –1425 passes –956 failures –59.85% Session passed Landing Test Performance 8

© 2012 Whamcloud, Inc. This chart shows the percentage of landings that passed all tests since LUG 2011 Landing Test Performance 9

© 2012 Whamcloud, Inc. Distributed Test 10 Source code repository Public Sector ISVs Resellers

© 2012 Whamcloud, Inc. Distributed Test 11 Source code repository Public Sector ISVs Public Sector Resellers But!

© 2012 Whamcloud, Inc. Juelich financed a fully equipped cluster Used for testing all head releases Specialises in failover testing –36TB of multi attached storage Good for performance regression tests –Private network with no contention so results are repeatable Also used for manual large Lun testing for 2.2 release I’d like to thank Juelich and in particular Frank Heckes for making this happen Juelich Supercomputer Centre 12

© 2012 Whamcloud, Inc. Indiana provided a 36 node cluster for the development and rollout of a backup test system Used for secondary 2.2 release testing Enabled the transfer of the primary test cluster from the West Coast to Colorado without any break in the Lustre test and landing processes Again I’d like to thank Indiana and Steve Simms for enabling this Indiana University 13

© 2012 Whamcloud, Inc. The Year Ahead 14

© 2012 Whamcloud, Inc. Development vs. Landing Test 15 Which is landing test? Which is development test?

© 2012 Whamcloud, Inc. Distinct paths for development vs. landing –Similar process but distinct purpose Provide for maximum flexibility in development testing –Encourage systematic test to be part of the development process –Test during development does improve product quality Encourage peer review as the code is written –Earlier review leads to better code and more opportunity for education Be auditable –Improvement requires knowledge of the past –Development test part of the landing collateral 100% pass rate for landing test –Developers should push tried and tested code for landing Development vs. Landing Test 16

© 2012 Whamcloud, Inc. Development Test Cloud 17

© 2012 Whamcloud, Inc. The Goal 18

© 2012 Whamcloud, Inc. Whamcloud’s Quality Approach A Review Of The Last Year Our Plans For The Coming Year Summary 19

© 2012 Whamcloud, Inc. Maloo ‘Your Window onto Test’ 20

© 2012 Whamcloud, Inc. What is Maloo? Recent changes to Maloo –High points since last LUG Development priorities –Tool quality –Easy access to timely, accurate data Some planned features –User preferences –Automated scanning of incoming log files for “interesting” data Maloo Agenda 21 LUG Austin, TX - April 2012

© 2012 Whamcloud, Inc. A repository for lustre test result data –Collects the test results and the logs generated –Allows users to query the database –Contains about 1TB of log files What is Maloo? 22 LUG Austin, TX - April 2012

© 2012 Whamcloud, Inc. Recent changes to Maloo LUG Austin, TX - April

© 2012 Whamcloud, Inc. Release report 24 LUG Austin, TX - April

© 2012 Whamcloud, Inc. Node utilization report 25 LUG Austin, TX - April

© 2012 Whamcloud, Inc. Internal changes Invisible work –Development practices –Testing –Packaging –Deployment –… 26 LUG Austin, TX - April 2012

© 2012 Whamcloud, Inc. Development priorities LUG Austin, TX - April

© 2012 Whamcloud, Inc. Tool quality –Mechanics of Maloo and its development –Visibility into the tool –Usability Development priorities 28 LUG Austin, TX - April 2012

© 2012 Whamcloud, Inc. Data quality –Accurate –Timely –Accessible Development priorities 29 LUG Austin, TX - April 2012

© 2012 Whamcloud, Inc. New features LUG Austin, TX - April

© 2012 Whamcloud, Inc. Log file scanning Automates a tedious task New features 31 LUG Austin, TX - April 2012

© 2012 Whamcloud, Inc. Maloo Footprints

© 2012 Whamcloud, Inc. Maloo Footprints 33

© 2012 Whamcloud, Inc. Maloo Footprints 34

© 2012 Whamcloud, Inc. Maloo Footprints 35

© 2012 Whamcloud, Inc. User profiles One size needn’t fit all New features 36 LUG Austin, TX - April 2012

© 2012 Whamcloud, Inc. What was covered Changes in Maloo since last year Our development priorities A couple of the planned changes Wrap up and questions 37 LUG Austin, TX - April 2012

© 2012 Whamcloud, Inc. Thank you very much 38