Plugin Quality Check Penny Wyatt JIRA QA Team Lead Atlassian

Quality Assistance Help devs to deliver value to customers Testing and testing advice Bug prevention

Plugin Quality Process Make it work Make it lustworthy Make it maintainable Make it safe

Setting the Quality Bar Tradeoff between time, scope and quality Mismatched quality expectations - unhappy customers

Make it work “But I’ve already tested it, it works!” in the real world

Happy Path Fallacy “We’ll only worry about the happy path, for now” Only test the actions a normal, reasonable user would perform.

Happy Path Fallacy Quality bar defines what you fix, not what you test. OK not to support everything. Set customer expectations!

Valid Scenarios Sample Plugin Purpose-built but not contrived Completely unrelated to similar plugins on PAC! Only a short list, more detail on CAC

Valid Scenarios Different types of data Special characters Large data sets Long strings

Valid Scenarios Different browsers Layout issues Broken functionality Behaviour in unsupported browsers

Valid Scenarios Different use cases Session timeouts Deletions/Moves/Edits Anonymous access

Make it Lustworthy

User experience Guide users towards the actions they should do. Avoid empty boxes - use appropriate controls. Help them to avoid mistakes. Reading documentation should not be required.

Administration experience Clear flow for configuration Provide helpful error information Don’t show stack traces Use logging sparingly

Make it Maintainable

Manual and Automated Testing Manual testing Fast, effective, broad. Only tests the current state. Automated testing Scalable, sustainable. Takes time, limited assertions.

Automated Testing Good for long-term regression tests Optimise for: Scenarios most likely to break Integration with code out of your control Use page objects for UI tests for maintainability

Version Numbering Versions are constant A version number refers to one state of the code only. Never re-release a plugin with the same version number. 25

26 v2.5 Latest? Yes! v2.5 Help! v2.5 ??? v2.5 Argh! !!!!!!!!!

27

28

Version Numbering One version number means one binary. Always bump up the version number. 29

Make it Safe

How plugins break apps XSS Inserting user-supplied data into HTML without HTML-encoding it. Allows an attacker to gain control of the victim’s browser.

How plugins break apps Insufficient permission checking Exposing data to anonymous users Not respecting permission schemes

How plugins break apps Performance Synchronous external requests Unbounded memory allocation

How plugins break apps Lack of CSS scoping Scope every item in the plugin CSS Avoid overriding built-in styles Lack of JavaScript scoping (function () { // code goes here })();

Final check Made it work Made it lustworthy Made it maintainable Made it safe

Ship it!

#atlascamp TAKE-AWAYS Penny Wyatt