VISUAL AUTOMATED REGRESSION TESTING - Sriram Angajala SKILLS MATTER – 11 AUG 2016

Sriram Angajala Sriram.angajala@gmail.com https://uk.linkedin.com/in/sriram-angajala-91a77852

I know Automated Testing, How is it different Why VRT? I know Automated Testing, How is it different Introduction How was it ever tested in past? Agenda Negative Side of WCSS Questions? Demo Tools

Introduction About Visual Regression Testing Tools Demo

Visual Regression Testing? Why What How

Why VRT? Content – Looks – and Feel, Colour contrasting… Every website – need to Rich Contrasting colours Content should look great First impressions

Why VRT? Beautiful on all browsers Testers not concluding their test on Chrome only?? IE also?

Why VRT? Beautiful on all resolutions (X*Y) Lot of effort to make it look beautiful in all sizes and resolutions

Playing around with CSS is costly – It may break ? That’s it? Any thing more… Playing around with CSS is costly – It may break ? @ fonts @height and size @relative heights @margins. Just because an element is on a page does not mean it’s in the right location (or the right size or color). Responsive Websites. Style Guide Based Development – Need more testing on style guides upfront

CSS – How the layout works Styles guides great but when its combined with other elements layout breaks

How to do VRT then – Try manual Lets Start with Unit Testing Css Unit testing is well always there but with compositional classes, you want to test the behaviour, not the implementation Compare page by Page Manually Manually take a baseline images and compare them manually – Manually with Human eye – Lets do a test then User open source image comparison tools Lots of websites and tools ready to do image comparisons just by providing urls – Its great – Even able to find an issue with apple.com – Great – But its not fully automated

Find atleast one difference

Lets see a sample(apple) website http://www.browsera.com/tests/5293#page=35171

Problems with manual VRT? Plan A - Visually Check All Pages X All browsers X All Devices X All resolutions Plan B Visually Check on All Pages X All Browsers X Resolutions Plan C Visually Check Few Pages X Few Browsers Plan D Visually Check Home Page on Chrome and FF

Finally – Automated VRT No manual Efforts on visual testing also. Fit into current automated testing process Test in all resolutions Support mobile testing Go hand and hand with automated testing Focus on specific elements ignoring dynamic areas Generate reports with comparisons Tailor along with selenium tests Ignore browser changes and screen changes

How was it ever Automated tested in past? NO its was not Automated before – Why? Selenium don’t support Visual testing Only functional automated tests across devices to see if every thing work Manual UI testing

How ? Choose a tool based in below parameters CLI – Gulp Support your preferred language Cloud supporting BDD including support GWT Focus on scripting Reports Continuous integration Developers local machine Minimize false positives

I know Automated Testing, How is it different Its the same – that’s what this meetup is all about. Still use your same framework – Just add a Step – THEN – “Everything looks GOOD” in your scenarios

Tools

Tools? No scripting required Browsera Backtac Javascript Cactus CSSCritic CSSLint Fighting Layout Bugs GhostStory Huxley BackstopJS PhantomCSS Wraith Grunt-Photobox

Tools? Java Selenium2-visualdiff  Applitools Ruby https://percy.io/

PhantomCSS PhantomCSS – Oldest Visual Regression Testing Tool based on Javascript. Uses Casper.js and Phontom.js Helps to navigate and take visual screen shots and compare. Cons No reporting tools No test management tools Not along with Webdriver Headless only

Wraith Yes its BBC’s entry, built for performing visual diffs on responsive websites. It allows authors to test various viewport widths, but tests are performed against full screenshots ONLY. One thing it has over PhantomCSS, though, is the ability to ‘fuzz’ the diffs so that tests don’t fail due to platform-specific antialiasing differences. Being written by the BBC it’s received quite a bit of publicity and I suspect it will become fairly popular. It’s built on Ruby, so setting it up on a Windows environment could be a pain Cons Full Page only No Reporting Tool

Selenium2-visualdiff is a Maven plugin for taking screenshots in Selenium webdriver tests, performing a diff and providing a comparative report. It uses some familiar technologies (Maven, Java and Selenium) Cons Not maintained well Full Screen only Cant Ignore dynamic content Link – http://kreyssel.github.io/selenium2-visualdiff/visualdiff.html

AppliTools It integrates with selenium and has the most advanced visual diff algorithm I’ve seen, with support for different resolutions, OS, browsers, and even branches, and exclusion of specific regions Pros Many Well maintained Strong reporting Nice Documentation Recommended for Java and also supports many languages Cons Not open source Cannot well fit with reporting tools

WebdriverCSS? Open source Javascript Based on famous WebdriverIO Extracted from PhantomCSS Uses Webdriver commands to navigate Easy customization Across real Browsers Focus on specific element based on CSS Cons No Report – But Webdriver Admin Panel Can do it

WebdriverCSS how to install? Node Package Manager • NodeJS • Selenium Standalone Server • GraphicsMagick or ImageMagick • webdriver.io • WebdriverCSS plugin for webdriver.io • PhantomJS, Chrome/ChromeDriver, Firefox (Iceweasel may work) • Font Packages (for Linux; OS X and Windows may have plenty installed) • Testing framework (I use Mocha; Jasmine and Cucumber also work) DEMO

Negative Side of AVRT? How to update baseline How to match screen sizes with mobiles False Positives WebdriverCSS not compatible with WebdriverIO Reports not available for Webdriver CSS – Webdriver Admin Panel

Questions?

Please go to meetup page click on link below Feedback? Please go to meetup page click on link below https://www.surveymonkey.co.uk/r/7QQYHGN

Drinks?