Selenium vs. Watir Evaluation William Mosteller Test Automation Specialist Copyright © 2009, Eloqua Inc.

The Initial Challenge Who is Eloqua? Leading marketing automation platform (SaaS) Global operations Over 200 employees, and 500+ customers We’re testing a re-write of the web application Evaluate Selenium and Watir and decide which is better for our needs

Similarities Both are open source Both support browsers only We drive both with Ruby (Selenium RC) Ruby reports errors by line number You need an editor like NotePad++ Or SCITE, which comes with Ruby - C:\ruby\scite\SciTE.exe Unlike SilkTest, no window definitions Unlike SilkTest, tags may be fuzzy

Differences Selenium RC requires that you start a server on your computer Easily implemented with a.bat file in the Startup folder Selenium supports methods with x, y offsets Selenium supports more browsers and operating systems So we started with a Selenium bias

Our Strategy “Scientific Method” Comparison: – Multi Language Support (non-ascii characters) – Docs – Maturity – Frames – Learning Curve – Existing team knowledge – Price/License – Community – Etc.

Initial Experiences Early Trials Selenium Scripting with preliminary versions of our application Watir Scripting with LinkedIn Both looked promising

Scripting the Application Developed in SproutCore New open source development framework In both tools… Click doesn’t work Double-Click doesn’t work “No battle plan survives contact with the enemy!” My new thought: We may have to settle for the one we can get to work!

More Issues Handling Modal Dialog Boxes Handling Application All Actions Button Entering Text in the YUI Editor Selecting Text in the YUI Editor Drag ‘n Drop

Aside - Information Resources Open source documentation is distributed, you need research skills Watir - wiki.openqa.org/display/WTR/Project+Hme Selenium - seleniumhq.org/projects/core/ & wiki.openqa.org/display/SEL/Home Google Watir group on Google Selenium Forum on Open QA clearspace.openqa.org/community/selenium Selenium and Watir groups on LinkedIn Friends

Click Resolution In Selenium (unless otherwise noted, code examples here are Selenium) mouse_down(locator) mouse_up(locator)

Double Click Resolution SproutCore Changes Required MS-IE Implementation fire_event(locator, "DblClick") FireFox Implementation paired MouseDown/MouseUp events

Dialog Box Resolution Watir solutions are based on Click_No_Wait Selenium provides functions for “are you sure” case Use AutoIt based scripts, running separately Programmed in Ruby, but with different methods May need to register AutoitX3.dll using regsvr32 Windows only

All Actions Resolution MouseDown/MouseUp opens the drop-down Click chooses the actual selection

Aside - Tagging Tools With JavaScript, View Source isn’t helpful MS-IE IE Developer Toolbar FireFox Firebug DOM Inspector Xpather (Install these globally in FireFox)

YUI Text Entry Resolution Proper Tagging Note terse tag coding style

YUI Text Selection SilkTest and Watir Two ways to put text in a field (Watir examples) Set() Send_Keys() o Special syntax for control characters o "+{END}{DELETE}" + CannedText + "+{HOME}" Selenium doesn’t work this way

YUI Text Selection Resolution.key_down_native("17")# cntl down.key_press_native("65")# “A” press.key_up_native("17")# cntl up Need JavaScript Character Codes 83ac-25f0306e3e25/Javascript-Char-Codes-Key- Codes.aspx

Drag ‘n Drop Resolution drag_and_drop_to_object() Animation prettier with MS-IE Have suggestions for Watir

We Chose Selenium – March, 2009 We solved all our issues there Probably could also do so in Watir, but it looked more expensive Drag ‘n drop worked intermittently Send_Keys() not available in FireWatir And the initial Selenium plusses still matter Selenium supports methods with x, y offsets Selenium supports more browsers and operating systems

Since Selenium Selection Now testing our legacy application with Watir Selenium has trouble with modal browser windows, and legacy app has lots of them Watir handles MS-IE tables faster than Selenium Frames in legacy app complicate Watir tags Must mention every frame above your widget

Syntax Matters Is a.b.c.d present on the screen? Selenium: …element?(a.b.c.d) Watir: …a.b.c.d.exists? If c does not exist, or something weird happens, you still get an exception o Use nested.exists? or o While/Begin/Break/Rescue/End around the code