6 Ways to Do More Performance Testing in Less Time NovaTAIG March 9, 2011 Steve Sturtevant, OC Systems James Pulley, Newcoe Performance Engineering

Presenters Steve Sturtevant Product Manager OC Systems, Inc. James Pulley CTO Newcoe Performance Engineering

Most organizations don’t establish performance requirements up front. – Architects, platform engineers, developers, functional test all may have different assumptions. Need to treat performance as you would any other non-functional requirement. – Want something that can be objectively measured. – Need to go through consensus-building process. Performance is a requirement… require it.

Test early, measure often. – Testing is always happening. – Got data? Look to your tools! – Word of caution: some developers live in Animal House. But why? – Validate architecture early. – Low hanging fruit is fruitful. – Your schedule will thank you. Measure twice, test once.

Get developers to buy into performance. – Create a personal connection. – Create an understanding within your metrics and methods. – Go public, point fingers. Involvement early pays dividends later. Accountability is king.

Get performance testing done before the performance testing window. – Add a couple additional elements that measure timing to functional tests. – If it doesn’t scale for one, it will never scale for many. – Send performance issues back just as you would functional defects. Functional testing provides performance smoke testing.

At the end of the day you need to fix, not find. – “More with Less,” sound familiar? “Here’s what I think happened.” – Context is King; be factual. Eliminate “I can’t reproduce.” Don’t just test, diagnose.

Need data that is as real, as dirty as what you have in production. Be prepared to deal with data issues: – Restricted data sets – data state may change. – Resource lock problems. – Testing ghosts. Seeding larger data sets takes less time than having engineers figure out if they have a bug or testing ghost. Don’t forget about your data.

6 Ways to Do More Performance Testing in Less Time 1.Performance is a requirement… require it. 2.Measure twice, test once. 3.Accountability is king. 4.Functional testing provides performance smoke testing. 5.Don’t just test, diagnose. 6.Don’t forget about your data.

Questions & Answers

Contact Information Steve Sturtevant Product Manager OC Systems, Inc James Pulley CTO Newcoe Performance Engineering