1. © 2012 Whamcloud, Inc. Whamcloud and Quality Chris Gearing & Mike Stok Software Engineers Whamcloud, Inc Lustre User Group Austin TX April 2012 Version 1.0

2. © 2012 Whamcloud, Inc. Whamcloud’s View Of Quality A Year’s Progress The Year Ahead Maloo ‘Your Window onto Test’ – Mike Stok Agenda 2

3. © 2012 Whamcloud, Inc. Whamcloud’s View Of Quality 3

4. © 2012 Whamcloud, Inc. ‟ The only thing I’m non-Linear about is Whamcloud not delivering what it says it will deliver ” Whamcloud’s View Of Quality 4 Eric Barton CTO Whamcloud

5. © 2012 Whamcloud, Inc. Whamcloud’s view of quality development Whamcloud’s View Of Quality Stability Quality 5 Features Performance Time

6. © 2012 Whamcloud, Inc. Whamcloud is investing money, time and expertise in Quality …is continuing to develop tools and invest in infrastructure to enhance the Lustre community The whole of its engineering team is orientated around quality principles Whamcloud’s View Of Quality 6

7. © 2012 Whamcloud, Inc. A Year’s Progress 7

8. © 2012 Whamcloud, Inc. For landing testing we have carried out; –687 days of landing testing –2381 landing sessions –2092758 individual tests –2087607 passed Statistically –1425 passes –956 failures –59.85% Session passed Landing Test Performance 8

9. © 2012 Whamcloud, Inc. This chart shows the percentage of landings that passed all tests since LUG 2011 Landing Test Performance 9

10. © 2012 Whamcloud, Inc. Distributed Test 10 Source code repository Public Sector ISVs Resellers

11. © 2012 Whamcloud, Inc. Distributed Test 11 Source code repository Public Sector ISVs Public Sector Resellers But!

12. © 2012 Whamcloud, Inc. Juelich financed a fully equipped cluster Used for testing all head releases Specialises in failover testing –36TB of multi attached storage Good for performance regression tests –Private network with no contention so results are repeatable Also used for manual large Lun testing for 2.2 release I’d like to thank Juelich and in particular Frank Heckes for making this happen Juelich Supercomputer Centre 12

13. © 2012 Whamcloud, Inc. Indiana provided a 36 node cluster for the development and rollout of a backup test system Used for secondary 2.2 release testing Enabled the transfer of the primary test cluster from the West Coast to Colorado without any break in the Lustre test and landing processes Again I’d like to thank Indiana and Steve Simms for enabling this Indiana University 13

14. © 2012 Whamcloud, Inc. The Year Ahead 14

15. © 2012 Whamcloud, Inc. Development vs. Landing Test 15 Which is landing test? Which is development test?

16. © 2012 Whamcloud, Inc. Distinct paths for development vs. landing –Similar process but distinct purpose Provide for maximum flexibility in development testing –Encourage systematic test to be part of the development process –Test during development does improve product quality Encourage peer review as the code is written –Earlier review leads to better code and more opportunity for education Be auditable –Improvement requires knowledge of the past –Development test part of the landing collateral 100% pass rate for landing test –Developers should push tried and tested code for landing Development vs. Landing Test 16

17. © 2012 Whamcloud, Inc. Development Test Cloud 17

18. © 2012 Whamcloud, Inc. The Goal 18

19. © 2012 Whamcloud, Inc. Whamcloud’s Quality Approach A Review Of The Last Year Our Plans For The Coming Year Summary 19

20. © 2012 Whamcloud, Inc. Maloo ‘Your Window onto Test’ 20

21. © 2012 Whamcloud, Inc. What is Maloo? Recent changes to Maloo –High points since last LUG Development priorities –Tool quality –Easy access to timely, accurate data Some planned features –User preferences –Automated scanning of incoming log files for “interesting” data Maloo Agenda 21 LUG Austin, TX - April 2012

22. © 2012 Whamcloud, Inc. A repository for lustre test result data –Collects the test results and the logs generated –Allows users to query the database –Contains about 1TB of log files https://maloo.whamcloud.com What is Maloo? 22 LUG Austin, TX - April 2012

23. © 2012 Whamcloud, Inc. Recent changes to Maloo LUG Austin, TX - April 2012 23

24. © 2012 Whamcloud, Inc. Release report 24 LUG Austin, TX - April 2012 https://maloo.whamcloud.com/reports

25. © 2012 Whamcloud, Inc. Node utilization report 25 LUG Austin, TX - April 2012 https://maloo.whamcloud.com/reports/show_node_utilization_report

26. © 2012 Whamcloud, Inc. Internal changes Invisible work –Development practices –Testing –Packaging –Deployment –… 26 LUG Austin, TX - April 2012

27. © 2012 Whamcloud, Inc. Development priorities LUG Austin, TX - April 2012 27

28. © 2012 Whamcloud, Inc. Tool quality –Mechanics of Maloo and its development –Visibility into the tool –Usability Development priorities 28 LUG Austin, TX - April 2012

29. © 2012 Whamcloud, Inc. Data quality –Accurate –Timely –Accessible Development priorities 29 LUG Austin, TX - April 2012

30. © 2012 Whamcloud, Inc. New features LUG Austin, TX - April 2012 30

31. © 2012 Whamcloud, Inc. Log file scanning Automates a tedious task New features 31 LUG Austin, TX - April 2012

32. © 2012 Whamcloud, Inc. Maloo Footprints

33. © 2012 Whamcloud, Inc. Maloo Footprints 33

34. © 2012 Whamcloud, Inc. Maloo Footprints 34

35. © 2012 Whamcloud, Inc. Maloo Footprints 35

36. © 2012 Whamcloud, Inc. User profiles One size needn’t fit all New features 36 LUG Austin, TX - April 2012

37. © 2012 Whamcloud, Inc. What was covered Changes in Maloo since last year Our development priorities A couple of the planned changes Wrap up and questions 37 LUG Austin, TX - April 2012

38. © 2012 Whamcloud, Inc. Thank you very much 38