1. VISUAL AUTOMATED REGRESSION TESTING - Sriram Angajala
SKILLS MATTER – 11 AUG 2016

2. Sriram Angajala Sriram.angajala@gmail.com

3. I know Automated Testing, How is it different
Why VRT?
I know Automated Testing, How is it different
Introduction
How was it ever tested in past?
Agenda
Negative Side of WCSS
Questions?
Demo
Tools

4. Introduction
About Visual Regression Testing
Tools
Demo

5. Visual Regression Testing?
Why
What
How

6. Why VRT? Content – Looks – and Feel, Colour contrasting…
Every website – need to Rich
Contrasting colours
Content should look great
First impressions

7. Why VRT? Beautiful on all browsers
Testers not concluding their test on Chrome only?? IE also?

8. Why VRT? Beautiful on all resolutions (X*Y)
Lot of effort to make it look beautiful in all sizes and resolutions

9. Playing around with CSS is costly – It may break ?
That’s it? Any thing more…
Playing around with CSS is costly – It may break ?
@ fonts
@height and size
@relative heights
@margins.
Just because an element is on a page does not mean it’s in the right location (or the right size or color).
Responsive Websites.
Style Guide Based Development – Need more testing on style guides upfront

10. CSS – How the layout works
Styles guides great but when its combined with other elements layout breaks

11. How to do VRT then – Try manual
Lets Start with Unit Testing
Css Unit testing is well always there but with compositional classes, you want to test the behaviour, not the implementation
Compare page by Page Manually
Manually take a baseline images and compare them manually – Manually with Human eye – Lets do a test then
User open source image comparison tools
Lots of websites and tools ready to do image comparisons just by providing urls – Its great – Even able to find an issue with apple.com – Great – But its not fully automated

12. Find atleast one difference

14. Lets see a sample(apple) website

15. Problems with manual VRT?
Plan A - Visually Check
All Pages X All browsers X All Devices X All resolutions
Plan B
Visually Check on All Pages X All Browsers X Resolutions
Plan C
Visually Check Few Pages X Few Browsers
Plan D
Visually Check Home Page on Chrome and FF

16. Finally – Automated VRT
No manual Efforts on visual testing also.
Fit into current automated testing process
Test in all resolutions
Support mobile testing
Go hand and hand with automated testing
Focus on specific elements ignoring dynamic areas
Generate reports with comparisons
Tailor along with selenium tests
Ignore browser changes and screen changes

17. How was it ever Automated tested in past?
NO its was not Automated before – Why?
Selenium don’t support Visual testing
Only functional automated tests across devices to see if every thing work
Manual UI testing

18. How ? Choose a tool based in below parameters CLI – Gulp
Support your preferred language
Cloud supporting
BDD including support GWT
Focus on scripting
Reports
Continuous integration
Developers local machine
Minimize false positives

19. I know Automated Testing, How is it different
Its the same – that’s what this meetup is all about.
Still use your same framework – Just add a Step – THEN – “Everything looks GOOD” in your scenarios

20. Tools

21. Tools? No scripting required Browsera Backtac Javascript Cactus
CSSCritic
CSSLint
Fighting Layout Bugs
GhostStory
Huxley
BackstopJS
PhantomCSS
Wraith
Grunt-Photobox

22. Tools?
Java
Selenium2-visualdiff 
Applitools
Ruby

23. PhantomCSS
PhantomCSS – Oldest Visual Regression Testing Tool based on Javascript.
Uses Casper.js and Phontom.js
Helps to navigate and take visual screen shots and compare.
Cons
No reporting tools
No test management tools
Not along with Webdriver
Headless only

24. Wraith
Yes its BBC’s entry, built for performing visual diffs on responsive websites. It allows authors to test various viewport widths, but tests are performed against full screenshots ONLY. One thing it has over PhantomCSS, though, is the ability to ‘fuzz’ the diffs so that tests don’t fail due to platform-specific antialiasing differences. Being written by the BBC it’s received quite a bit of publicity and I suspect it will become fairly popular. It’s built on Ruby, so setting it up on a Windows environment could be a pain
Cons
Full Page only
No Reporting Tool

25. Selenium2-visualdiff
is a Maven plugin for taking screenshots in Selenium webdriver tests, performing a diff and providing a comparative report. It uses some familiar technologies (Maven, Java and Selenium)
Cons
Not maintained well
Full Screen only
Cant Ignore dynamic content
Link –

26. AppliTools
It integrates with selenium and has the most advanced visual diff algorithm I’ve seen, with support for different resolutions, OS, browsers, and even branches, and exclusion of specific regions
Pros
Many
Well maintained
Strong reporting
Nice Documentation
Recommended for Java and also supports many languages
Cons
Not open source
Cannot well fit with reporting tools

27. WebdriverCSS? Open source Javascript Based on famous WebdriverIO
Extracted from PhantomCSS
Uses Webdriver commands to navigate
Easy customization
Across real Browsers
Focus on specific element based on CSS
Cons
No Report – But Webdriver Admin Panel Can do it

28. WebdriverCSS how to install?
Node Package Manager
• NodeJS
• Selenium Standalone Server
• GraphicsMagick or ImageMagick
• webdriver.io
• WebdriverCSS plugin for webdriver.io
• PhantomJS, Chrome/ChromeDriver, Firefox (Iceweasel may work)
• Font Packages (for Linux; OS X and Windows may have plenty installed)
• Testing framework (I use Mocha; Jasmine and Cucumber also work)
DEMO

29. Negative Side of AVRT? How to update baseline
How to match screen sizes with mobiles
False Positives
WebdriverCSS not compatible with WebdriverIO
Reports not available for Webdriver CSS – Webdriver Admin Panel

30. Questions?

31. Please go to meetup page click on link below
Feedback?
Please go to meetup page click on link below

32. Drinks?