1. DevOps Meetup | Test Automation | 3/19/2012

2. Requirements and Purposes
Open source tools
Multiple browser and OS support
Regression / functional tests
Parallel execution
Tool for easily write tests (long term)

3. Used software Selenium (webdriver) in Java TestNG (+ReportNG) from Ant
Jenkins

4. Framework design Window driver pattern
Locators / Translations / Fixtures
WebElement locator strategy
id > css ( > xpath)
DynamicLocator

5. Framework design Organize tests by groups
Own page factory pattern design
Easy to read Test code
Wrap what you can ("wait for" utilities)
Report what you do

6. Framework design DLocator
<WebElement name="GeneralVideoDiv">
<!-- This is a general locator for videos, can be used as a dynamic locator to address videos. -->
<Locators>
<css>#MyVideosList > li</css>
<class>+ li</class>
</Locators>
</WebElement>
<WebElement name="VideoSpecificVideoDiv">
<!-- This is a specific locator for a video, can be used as a dynamic locator to address videos. -->
<css>#Video-VIDEO_ID</css>
<value>VIDEO_ID</value>
<WebElement name="VideoDivEditButton">
<!-- Edit button on a video div, can be used as second part of a dynamic
locator. -->
<css>button.button-bar.editButton</css>

7. Framework design TestCode
/**
* Tests error messages and text format in edit video overlay price field. */
@Test(description = "Tests video edit overlay functionalities", groups = { "PPVTests", "PPVVod" })
public void testVodEditVideoOverlay() {
final int videoToEdit = 3;
HeaderPage headerPage = getHeaderPage();
DashboardPage dashboardPage = getDashboardPage();
PPVEventUser ppvUser = userFactory.getPpvEventUser();
DynamicLocator editButton = dashboardPage.getDLocator("GeneralVideoDiv").append("VideoDivEditButton");
headerPage.loginUser(ppvUser.getName(), ppvUser.getPassword());
headerPage.openDashboard();
dashboardPage.loadManageYourShow();
dashboardPage.loadShowToManage(ppvUser.getVodPPVEventShowName());
dashboardPage.click("ShowYourVideos");
dashboardPage.waitForPageLoad();
dashboardPage.click(editButton, videoToEdit);
dashboardPage.waitForAjaxRequestDone();

8. Framework design TestGrps
<!DOCTYPE suite SYSTEM " >
<suite name="Social Pages Login Tests" verbose="1" parallel="tests" thread-count="4">
<test name="Social Pages Login Tests in Firefox" parallel="methods" thread-count="12">
<parameter name="browserName" value="Firefox" />
<parameter name="browserVersion" value="8" />
<groups>
<run>
<include name="SocialPagesLoginLogout" />
<exclude name="NotRunOnFirefox" />
</run>
</groups>
<packages>
<package name="com.ustreamv3.tests" />
</packages>
</test>
<test name="Social Pages Login Tests in Chrome" parallel="methods" thread-count="12">
<parameter name="browserName" value="Chrome" />
<exclude name="NotRunOnChrome" />

9. Framework design Report

10. Flash Automation JsApi is our friend getProperty(...) setProperty(...)
callMethod(...)
addListener(...)
Needs strong team cooperation

11. Flash Automation @Sequential(groupClasses = { MetricsUser.class })
@MultiBrowser
@Test(description = "Tests the broadcast start and stop effect on channel page.")
public void testBroadcastStartStop() {
MetricsUser metricsUser = userFactory.getMetricsUser();
ShowPage showPage = getShowPage();
JsAPI viewerApi = getJsAPI();
showPage.openShowPage(metricsUser.getShowName());
viewerApi.checkProperty(ViewerProperties.IS_PLAYING, false);
incrementSteps();
final int lastSession = startNewSession();
HeaderPage headerPage = getHeaderPage(lastSession);
JsAPI bcApi = getJsAPI(lastSession);
headerPage.loginUser(metricsUser.getName(), metricsUser.getPassword());
headerPage.openBroadcastConsole();
bcApi.checkProperty(BroadcasterProperties.IS_PUBLISHER_ACTIVE, true);
bcApi.callMethod(BroadcasterMethods.START_BROADCAST);
viewerApi.checkProperty(ViewerProperties.IS_PLAYING, true);
bcApi.callMethod(BroadcasterMethods.STOP_BROADCAST); }

12. Dynamic vs static fixtures
Dynamic fixture
backdoor setup
gui setup
internal api setup
Static fixture = manual shared fixture

13. Dynamic vs static fixtures Backdoor setup
Pros
full parallel
fast setup
accurate result
Cons
infrastructure
maintenance

14. Dynamic vs static fixtures GUI setup
Pros
black-box test
parallel
Cons
s l o w setup
fragile

15. Dynamic vs static fixtures Internal API setup
Pros
can be fast
parallel
custom
Cons
development
maintenance

16. Dynamic vs static fixtures Manual Shared Fixture
Pros
flexible
fast test
Cons
sequential
creation
maintenance

17. Parallel execution
Tests with multiple browser instances → sequential job execution
Native events = one browser / VM
IE = one instance / VM

18. Multiple browser support
Safari 5.x with selenium → Fail
IE webdriver = jvm killer app :(

19. Lessons learned seeing is believing be prepared for exceptional cases
multiple browser = Firefox
test a new selenium version before use
clean up trash that selenium forgot to
memory > cpu (cores > frequency)

20. Lessons learned Parallel is better than fast tests
Tests must run on a daily basis
Keep locators organized
Automation must be planned carefully
Keep tests in good health

21. Questions / comments ?

22. Thank you
Happy testing!