1. - Selenium -
Browser-Based Automated Testing of Web Apps Under Continuous Integration
Presented By
Chris Bedford
Founder & Lackey at Large
Build Lackey Labs
Company Overview
Founded
Services
Release engineering & Build/Test Automation for Java Environments
Spring/Hibernate & Groovy/Grails App Development
Training: Spring/Groovy/Grails & Build/Test Automation Technologies
Employees
Mike Jackson

2. How to write functional tests with
Agenda
KEY
TAKE-AWAYS
Different Ways To Test a Grails App
Unit, Integration, Functional
Selenium
Demo of IDE
Architecture
Alternatives (Canoo WebTest)
Automating Testing Using Selenium
Maven
Demo
Cargo
Viewing Test Results
Lab 1 – maven project
Ant
Continuous Integration With Hudson
How to write functional tests with
Selenium
How to set up
Selenium Testing
On Grails Apps
In Continuous
Integration
Building with maven
Building
With Ant
Continuous
Integration
With Hudson

3. Different Ways To Test a Java Web App
Unit
Unit Tests are created and run developer and run using framework like Junit or TestNG
test class or package functionality in isolation
heavy use of mocks and stubs
tests should execute quickly and be run often to catch problems early on
Integration
If you are dependent on someone else’s stuff, the test includes your stuff and theirs
Verifies that two or more components work together,
usually directly, and not with mocks
To simplify test set-up components typically execute same JVM process
Functional Testing
Run application in container
Exercise functionality via a client side agent that executes Javascript.
Agent could be either:
your actual target browser (Firefox, IE, etc.) [Selenium's approach]
a Javascript engine embedded into test framework [Canoo Webtest's approach]
Package or class level scope
Mock http requests
increasingly
coarse
grained
components
under
test
Client requests from same process
Separate client
Real http requests

4. Manual Steps Involved In Running Selenium
Before running Selenium Functional Tests we need to
Compile classes
Run unit and integration tests
bail on functional tests if we catch problems with lighter weight tests
Package .war file
Install and start the container in which we want to run the .war
Deploy the .war to the container
unless your running Jetty, which runs embedded in your app
Launch target browser
Launch Selenium on desired test suite

5. Launching Selenium

6. Recording New Selenium Tests

7. Exporting test commands to 3GL (Java, etc.)

8. Saving New or Modified Selenium Tests
Individual tests referenced by the suite are recorded using their paths relative to the suite. For simplicity put your suite and all tests in the same directory (to start)

9. Selenium Components Selenium IDE Selenese Commands
Enable test author to
simulate navigation, clicks
Make assertions about expected responses
Selenium RC
Client side library that enables you to program more sophistication into your tests than the IDE allows (conditions, looping, error handling)
The Selenium Server which launches and kills browsers, interprets and runs the Selenese commands passed from the test program, and acts as an HTTP proxy,

10. Selenium Client Side Library & Server In Action
(javascript)
(client side java script /
Ajax portion of application under
Test – originates from here )
HTTP
Application
under test
(server side)
Selenium-server.jar
Reports back results of test to client
HTTP
Source:

11. A Look Same Origin Policy (Which Selenium’s Architecture Circumvents)
Forbids JavaScript code running on a web page from interacting with resources which originate from any web site other than the one which served up that web page

12. Selenium RC Server Acting As Proxy To Avoid Same Origin Policy Restrictions
4) Server interprets the command and then triggers the corresponding javascript execution to execute that command within the browser (say open page in app under test)s
What happens when a test suite starts ?
Proxy forwards request
to app
server
5) Request to open the page is routed through proxy server 6)
2) Selenium-RC server launches a browser (or reuses an old one) with URL that injects Selenium-Core’s javascript into browser-loaded web page. 7)
App server returns
response
1) client/driver establishes
connection w.selenium-RC
3) client-driver passes a Selenese command to the server e.g.: open command

13. Functional Test Alternatives: Canoo vs Selenium
Canoo Web Test
built on HtmlUnit
pros:
excellent test reporting allows you to pin point errors in test very easily.
faster to run (no browser spin up overhead)
better support for non HTML content (like spread sheets)
cons:
Weaker IDE (for test recording and playback)
develop tests in Ant or Gant only
Selenium
develop tests in HTML markup or 3 GL's like Java, Ruby, etc.
run test in actual browser
vs.embedded Javascript engine used by NO popular browser platform
slower to start.
see 'pros' listed Canoo
RECOMMENDATION: I’d go with Canoo for testing a REST-ful API

14. Canoo Web Test Reports
Canoo's reports show overall test results and let you drill down into any test
Canoo’s test results reports really excel at providing you drill down capability into the exact step of a test that failed.
With Canoo’s reports you see a clickable entries for each test run toward the bottom of the page together with a summary of tests run.
(pass fail percentages of both tests and individual steps within tests).
If you click on an entry for an individual test, you see a page…. (next slide)…

15. Canoo Web Test Reports (cont.)
Click to review a copy of the response HTML page corresponding to the first test step that failed
That gives you the pass/fail status of each individual step within the test.
You can go right to the failing test and view the resulting response page from the server.. Then you can figure out why the returned content did not match your assertions about what that content should have been.

16. Gluing together the steps in your build process
Before running Functional Tests need to
Compile
Run unit and integration tests
bail on functional tests if we catch problems with lighter weight tests
Package .war file
Install the container in which we want to run the .war (optional)
Deploy the .war to the container (optional – can just do ‘grails run-app’)
Launch your target browser
Launch Selenium on your desired test suite
To automate this process you can use
ant
maven
Groovy/Gant scripts
Gradle
Our example uses Maven
(demo)
(tour of pom.xml files that wire together our build steps)

17. Download
And Install
Tomcat
deploy
Compile
Unit Test
Integration Test
Package .war file

18. Structure of our demo project
Maven Repository
org.example:demo:1.1
Lives in $HOME/.m2/repostitory or
/Docuemts and Settings/<user>/.m2/repository
mvn install
declares a
a dependency on
org.example:demo:1.1
declares the artifact it produces to be
org.example:demo:1.1

19. Demo
All Tests Pass
Some Tests Fail

20. Maven Basics Convention over configuration
Standardizes where things live and what they are named
Lets you know where to look for the things you need…
and what to do when you find them
Project Object Model (pom.xml) specifies the complete ‘recipe for your build’
what artifact type are your producing ? (the name, the version…)
what are the dependencies (things you need) to produce that artifact ?
What plug-ins activate at what phases in the build life cycle ?
Shared Repository for storing artifacts that result from a build
Convention for naming artifacts
Build Life Cycle
each project is typically responsible for producing a distinct artifact (a.k.a. packaging) type
.jar, .war, .ear, etc.
each packaging type has an associated life cycle (ordered set of build phases)
Nested build project (module) structure
overall driver for your build project lives at top level of a directory hierarchy
sub-modules underneath the parent can be either
individual components of your product …or….
key phases in your build

21. Maven Nested Module Structure, Dependencies and Shared Repo
Maven Repository
org.example:demo:1.1
Lives in $HOME/.m2/repostitory or
/Docuemts and Settings/<user>/.m2/repository
mvn install
declares a
a dependency on
org.example:demo:1.1
declares the artifact it produces to be
org.example:demo:1.1

22. Maven pom.xml – Nested Module Structure

23. Maven pom.xml – Dependency Relationships
Maven Repository
org.example:demo:1.1

24. Hooking Maven Plug-ins Maven Into the Build Life Cycle
pom.xml
Build Life Cycle Phases
validate
generate/process-sources
process-sources
generate/process-resources
compile
test
prepare-package
package
pre-integration-test
integration-test
post-integration-test
verify
install
deploy
Canoo’s test results reports really excel at providing you drill down capability into the exact step of a test that failed.
With Canoo’s reports you see a clickable entries for each test run toward the bottom of the page together with a summary of tests run.
(pass fail percentages of both tests and individual steps within tests).
If you click on an entry for an individual test, you see a page…. (next slide)…

25. Cargo A set of APIs that assists in
installing web containers (such as Tomcat, JBoss)
booting and shutting them down
deploying web applications (.wars and .ears)
Invokable via
ant tasks
maven plugin
Java API
<target name="functional-test" >
<cargo containerId="tomcat6x" action="start" … >
<zipurlinstaller installurl="
<configuration type="standalone" home="${tomcatdir}">
<deployable type="war" file="foo.war"/>
</configuration>
</cargo>
<plugin> ...
<artifactId>cargo-maven2-plugin</artifactId>
<config>
<wait>false</wait>
<container>
<containerId>tomcat6x</containerId>
<zipUrlInstaller>
<url>
...
Installer installer =
new URL("
installer.iZipURLInstaller(new nstall();
LocalConfiguration configuration =
new DefaultConfigurationFactory().createConfiguration("tomcat6x")...)
container = new DefaultContainerFactory() .createContainer("tomcat6x"....);
container.setHome(installer.getHome());
WAR deployable = new WAR("foo.war);
deployable.setContext("ROOT");
configuration.addDeployable(deployable);

26. Launching Selenium Via Ant
Walk through of ant script that launches Selenium server in separate JVM,
Waits for server ready, then launches Selenium tests, then waits for shut down

29. Continuous Integration
Dedicated box runs regular full builds (including tests) of your software
Build triggers whenever any developer checks into SCM
Team is notified of any failures

30. Continuous Integration Benefits
Replaces big (and long) integration cycles with small frequent ones.
Same benefits as continous compilation in Eclipse
Immediate feedback when an error is introduced.
Fewer deltas between when it worked and when it broke
=> easier resolution of failures
Lower ripple through impact when colleagues checks in broken code
CI server build fails & team is notified
Other developers know it is not safe to pull from source
Mail sent out when build goes back to normal

31. Workflow After Adopting Continous Integration
Developer:
0- Check CI server for current build status
broken ? don't update ! otherwise..
1- check out from SCM
2- code new feature
3- run automated build on your box
4- Tests pass ?
no? go back to 2 !
5- Commit changes
CI Server:
0- Receive notification of change in SCM repo
1- Check out latest sources
2- Build from scratch
3 Build passes?
yes: publish, e.g., deploy .war to QA server
no: send out , flash red lights ...
Build ok?
broken !
SCM Repo

32. Continuous Integration With Hudson
Hudson orchestrates the execution of repeatable jobs
periodic builds (e.g., nightly)
builds that are triggered by some event (like checkin to source control)
Keeps History of Past Builds (Trend Reports)
Notifies When Builds Fail
Written in Java and runs from a .jar (no install required)
Integrates with wide range of SCM systems (SVN, P4, etc.)
Supports ant, maven, shell script based builds

33. Small Sample of Available Hudson Plug-ins
Project Metrics
Code Coverage (clover, cobertura, emma)
Checkstyle
Frameworks
Grails (invoke grails tasks as build steps)
Gradle
Build Wrappers
Locks and Latches - allows orchestration of multiple jobs
Build Time Out

34. Resources Selenium Mailing List
Selenium Web Site
Ant and Maven CI Tips & Tricks

35. Thank You !

36. Content Licensing Terms for This Work
You may present, distribute, copy, or incorporate into your own work, any and all portions of this presentation as long as such copies, or derivative works are made available without charge.
If you would like to redistribute this work on any type of fee-for-use or subscription basis, or if you wish incorporate any or all portions of this work into content which you charge for, please contact info <at> buildlackey.com to discuss licensing terms.