DevOps Meetup | Test Automation | 3/19/2012

Requirements and Purposes Open source tools Multiple browser and OS support Regression / functional tests Parallel execution Tool for easily write tests (long term)

Used software Selenium (webdriver) in Java TestNG (+ReportNG) from Ant Jenkins

Framework design Window driver pattern Locators / Translations / Fixtures WebElement locator strategy id > css ( > xpath) DynamicLocator

Framework design Organize tests by groups Own page factory pattern design Easy to read Test code Wrap what you can ("wait for" utilities) Report what you do

Framework design DLocator <WebElement name="GeneralVideoDiv"> <!-- This is a general locator for videos, can be used as a dynamic locator to address videos. --> <Locators> <css>#MyVideosList > li</css> <class>+ li</class> </Locators> </WebElement> <WebElement name="VideoSpecificVideoDiv"> <!-- This is a specific locator for a video, can be used as a dynamic locator to address videos. --> <css>#Video-VIDEO_ID</css> <value>VIDEO_ID</value> <WebElement name="VideoDivEditButton"> <!-- Edit button on a video div, can be used as second part of a dynamic locator. --> <css>button.button-bar.editButton</css>

Framework design TestCode /** * Tests error messages and text format in edit video overlay price field. */ @Test(description = "Tests video edit overlay functionalities", groups = { "PPVTests", "PPVVod" }) public void testVodEditVideoOverlay() { final int videoToEdit = 3; HeaderPage headerPage = getHeaderPage(); DashboardPage dashboardPage = getDashboardPage(); PPVEventUser ppvUser = userFactory.getPpvEventUser(); DynamicLocator editButton = dashboardPage.getDLocator("GeneralVideoDiv").append("VideoDivEditButton"); headerPage.loginUser(ppvUser.getName(), ppvUser.getPassword()); headerPage.openDashboard(); dashboardPage.loadManageYourShow(); dashboardPage.loadShowToManage(ppvUser.getVodPPVEventShowName()); dashboardPage.click("ShowYourVideos"); dashboardPage.waitForPageLoad(); dashboardPage.click(editButton, videoToEdit); dashboardPage.waitForAjaxRequestDone();

Framework design TestGrps <!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" > <suite name="Social Pages Login Tests" verbose="1" parallel="tests" thread-count="4"> <test name="Social Pages Login Tests in Firefox" parallel="methods" thread-count="12"> <parameter name="browserName" value="Firefox" /> <parameter name="browserVersion" value="8" /> <groups> <run> <include name="SocialPagesLoginLogout" /> <exclude name="NotRunOnFirefox" /> </run> </groups> <packages> <package name="com.ustreamv3.tests" /> </packages> </test> <test name="Social Pages Login Tests in Chrome" parallel="methods" thread-count="12"> <parameter name="browserName" value="Chrome" /> <exclude name="NotRunOnChrome" />

Framework design Report

Flash Automation JsApi is our friend getProperty(...) setProperty(...) callMethod(...) addListener(...) Needs strong team cooperation

Flash Automation @Sequential(groupClasses = { MetricsUser.class }) @MultiBrowser @Test(description = "Tests the broadcast start and stop effect on channel page.") public void testBroadcastStartStop() { MetricsUser metricsUser = userFactory.getMetricsUser(); ShowPage showPage = getShowPage(); JsAPI viewerApi = getJsAPI(); showPage.openShowPage(metricsUser.getShowName()); viewerApi.checkProperty(ViewerProperties.IS_PLAYING, false); incrementSteps(); final int lastSession = startNewSession(); HeaderPage headerPage = getHeaderPage(lastSession); JsAPI bcApi = getJsAPI(lastSession); headerPage.loginUser(metricsUser.getName(), metricsUser.getPassword()); headerPage.openBroadcastConsole(); bcApi.checkProperty(BroadcasterProperties.IS_PUBLISHER_ACTIVE, true); bcApi.callMethod(BroadcasterMethods.START_BROADCAST); viewerApi.checkProperty(ViewerProperties.IS_PLAYING, true); bcApi.callMethod(BroadcasterMethods.STOP_BROADCAST); }

Dynamic vs static fixtures Dynamic fixture backdoor setup gui setup internal api setup Static fixture = manual shared fixture

Dynamic vs static fixtures Backdoor setup Pros full parallel fast setup accurate result Cons infrastructure maintenance

Dynamic vs static fixtures GUI setup Pros black-box test parallel Cons s l o w setup fragile

Dynamic vs static fixtures Internal API setup Pros can be fast parallel custom Cons development maintenance

Dynamic vs static fixtures Manual Shared Fixture Pros flexible fast test Cons sequential creation maintenance

Parallel execution Tests with multiple browser instances → sequential job execution Native events = one browser / VM IE = one instance / VM

Multiple browser support Safari 5.x with selenium → Fail IE webdriver = jvm killer app :(

Lessons learned seeing is believing be prepared for exceptional cases multiple browser = Firefox test a new selenium version before use clean up trash that selenium forgot to memory > cpu (cores > frequency)

Lessons learned Parallel is better than fast tests Tests must run on a daily basis Keep locators organized Automation must be planned carefully Keep tests in good health

Questions / comments ?

Thank you Happy testing!