Posts Tagged ‘selenium tips’

Lessons Learned: Migrating from Selenium 1 to Selenium 2

July 21st, 2011 by Roger Hu

This guest blog post was written by Roger Hu, Software Engineer at Hearsay Social.

At Hearsay Social, we’ve upgraded our testing environment to use Selenium 2. We made the switch because there was enough evidence to suggest a huge 2-4x performance increase. Having learned a few lessons along the way, we thought it would be helpful to share what we found, especially for those who are considering making the transition.

  • Since Selenium 2 is redesigned to leverage what works best for the browser, whether it’s a NPAPI plugin in Firefox or a DLL module for IE, we’ve discovered a huge performance gain, especially in Internet Explorer (IE) browsers that have much slower JavaScript engines. The new approach seems to allow us to run Selenium more conveniently on Internet Explorer browsers without the hassle of changing the security options because of all the exceptions that were thrown as a result of the older JavaScript-based architecture.
  • Selenium 2 gets closer to simulating the behavior of a user on a browser.  In Selenium 2, the DOM element that is actually clicked is determined by the X/Y coordinates of the mouse event. Therefore, if you attempt to search for a DOM element that is hidden or obstructed by another element, the top element will always be fired and you might encounter ElementNotVisibleException errors from the Selenium server. You need to keep this issue in mind when rewriting your tests, since Selenium 1 versions may not have had this restriction. (We use the Django web framework and the popular django-debug-toolbar, which adds a popup overlay in our web application that has to be disabled in our application during Selenium tests.)
  • We’ve found that the new Selenium 2 WebDriver-based API is easier to train our developers to use. The documentation for Selenium 2 is still somewhat sparse, especially for the updated Python bindings, so digging into the source code (in our case, remote/ and remote/ code) is still the best way to learn what API commands are available. While Java developers may have access to WebDriverBackedSeleniumclass that can use existing Selenium 1 code while leveraging the WebDriver-based API, we didn’t find any similar support for Python. So we took the plunge and refactored most of our tests.


        def tag_name(self):
            """Gets this element's tagName property."""
            return self._execute(Command.GET_ELEMENT_TAG_NAME)['value']
        def text(self):
            """Gets the text of the element."""
            return self._execute(Command.GET_ELEMENT_TEXT)['value']
        def click(self):
            """Clicks the element."""
        def submit(self):
            """Submits a form."""
        def clear(self):
            """Clears the text if it's a text entry element."""

On the server-end, it’s important to study how the client API is sending remote commands by reviewing the JsonWireProtocol document posted on the Selenium Wiki, especially since Sauce Labs provides you with the raw logs to see what commands are actually being issued by the client.

  • While experimenting with Selenium 2, we found it much easier to test out the new WebDriver API by downloading and running the Selenium server locally. This way, your connection won’t constantly timeout as a result of using your Sauce Labs account, giving you more freedom to experiment with all the various commands. If you need to run browser tests against an external site while using your own machine to drive the browser interactions, you can setup a reverse SSH tunnel and then experiment with Selenium 2 API by setting debugger breakpoints and testing out the API bindings. In the long-term, though, you definitely want to use Sauce Labs for hosting all the virtual machines in the cloud for running your browser tests!
  • If you’re interested in using Firebug to help debug your application, Selenium 2 also provides a way to inject Firefox profiles. You can create a Firefox profile with this plug-in extension, and Selenium 2 includes an API that will base-64, zip-encode the profile that will be downloaded by the remote host. Note that this approach works best if you’re running the Selenium server locally, since using it over a Sauce Labs instance only gives you access to view the video.
  • Selenium 2 continues to be a moving target with its API, so you’ll want to keep up to date with any release notes posted on the Selenium HQ blog. Most recently, we found that the toggle() and select() commands have not only been deprecated but removed completely from the implementation. If you try to issue these commands, the Selenium server simply doesn’t recognize the commands and WebDriverExceptions are raised. The best thing to do is look at the Selenium version number. In this particular example, version 2.0.0 (three decimal places) are used to represent the release candidate of the latest Selenium build. You may also instantiate your .jar files with the -debug flag to watch how your client bindings execute API commands to the Selenium server.
20:38:02.687 INFO - Java: Sun Microsystems Inc. 20.1-b02
20:38:02.687 INFO - OS: Windows XP 5.1 x86
20:38:02.703 INFO - v2.0.0, with Core v2.0.0. Built from revision 12817
  • Selenium 1 users will find that is_text_present() and wait_for_condition() commands no longer exist, and are replaced by a more DOM-driven approach of selecting the element first before firing click() events or retrieving attribute values through get_attribute(). You no longer have to have wait_for_condition() for page loads. Instead, you set implicitly_wait() to a certain timeout limit to rely on find_element_by_id() to wait for the presence of DOM elements to appear to between page loads.
  • Lastly, we’ve noticed in the Selenium discussion groups that often there are questions about how to deal with concurrent Ajax requests during your tests.  In many test frameworks, there’s the concept of setup and tear down of the database between each set of tests.  One issue that we encountered is that if your browser is issuing multiple requests, you’re better off waiting for the Ajax requests to complete in your tear down function since the requests could arrive when the database is an unknown state. If this happens, then your Selenium tests will fail and you’re going to spend extra time trying to track down these race conditions. If you’re using jQuery, you can check the state to determine whether to proceed between pages (i.e. execute_script(“return === 0”)). You’ll want to keep looping until this condition is satisfied (for an example of implementing your own wait_for_condition() command, click here.)

Hope you find these tips helpful for migrating over to Selenium 2. Happy testing!

Why CSS Locators are the way to go vs XPath

May 17th, 2011 by Ashley Wilson

Last week, our own Santiago Suarez Ordoñez gave a presentation to the San Francisco Selenium Meetup group that convinced us all to say no (for the most part) to XPath and yes to CSS Locators in our Selenium testing.

In his role as official Sauce Ninja and as a prolific poster in the Selenium forums, Santi has helped more users solve locator issues than possibly anyone else in the world. He’s previously written a number of blog posts on ways to improve locator performance. As Sauce CEO John Dunham puts it, “If there was a foursquare mayorship for locators, Santi would have it for a lifetime.”

Drawing from this experience, he gave us these four reasons for using CSS Locators:

1. They’re faster
2. They’re more readable
3. CSS is jQuery’s locating strategy
4. No one else uses XPATH anyways!

I can’t speak for everyone, but Santi sure sold me on point number one when he showed off the performance metric script he wrote a script that tested the speed of XPath vs the speed of CSS Locators. There wasn’t much of a difference in Firefox, Safari, or Chrome, but with IE, the results were undeniable. Take a look:

To underscore this even further, he also recorded a video in Sauce OnDemand that uses one heck of a cute kitten to illustrate just how slow XPath can be. The cat’s paw movements represent the test clicking through the different locators. The first batch of clicks uses CSS Locators and completes in under 30 seconds. The second batch, the XPath one, continues on for another eight minutes. Eight minutes!

During the rest of the presentation, Santi dives into writing both basic and more advanced CSS Locators. He also spends some time talking about when you shouldn’t use CSS Locators (yes, there are a few cases where it is not the right tool for the job). To see the talk in its entirety, check out the recording below. And if you’re thinking of switching over from XPath, but unsure of how to go about it, check out the nifty tool Santi wrote called cssify. It does the handy work of translating your XPaths to CSS automatically.

Helpful links
View Santi’s Presentation Slides
Follow him on Twitter
Follow Sauce on Twitter
Join the San Francisco Selenium Meetup group!


How To Minimize The Pain Of Locator Breakage

September 29th, 2010 by Adam Goucher

In the Agile world, a common saying is ‘never break the build.’ But this thought can actually hinder agility if taken too literally. Sometimes, you want the build to break, as that means things are changing and progressing.

A better phrase is ‘don’t leave the build broken’. The same could be said for Selenium and element locators. They will break, and we want them to…periodically. The measures that count are:

  • How long it takes to go from broken to fixed – this should be small; really, really small
  • How big is the change to fix them – ideally, only one line per element. Total. Throughout the whole code base.

So how do we go about fixing these intermittent breakages?

The easiest locator to use is ‘id.’ To be most successful with this strategy, a static id attribute is put on each html element being interacted with. And since we don’t know what elements will be needed to automate in the future, it’s best to put an id on every element. The scripts will always know how to find the elements on the page uniquely, as WSC standards require that ids be unique in a single page.

While this solves the html structure change problems that often come from using XPath and CSS selectors by finding elements independently of their position, it still doesn’t solve id change issues. Solving that requires discipline and communication among the development and automation teams. One geographically dispersed team I worked with created the following two-part policy for this.

  1. Anyone can add an id to an element that does not have one
  2. If you change an id, the change must be communicated to both the dev and automation teams

Of course, knowing about a change is a bitter pill to swallow when it involves updating hundreds of scripts. A better solution is to separate the locator definitions from the main test code.

Selenium IDE

Separation of scripts from locators is achievable in Selenium IDE through the use of user-extensions. A previous Sauce Labs blog post, Parametrizing Selenese tests, explains how to make use of these extensions, but rather than provide the ‘Value’ column, the ‘Target’ is what could [should] be provided by the user-extension.

storedVars["registrationFirstname"] = "first";
storedVars["registrationLastname"] = "last";
storedVars["registrationSubmit"] = "register";

A script that interacts with a registration form might look like:

open /registration
type ${registrationFirstname} fred
type ${registrationFirstname} flintstone
click ${registrationSubmit}

Going back and updating Selenese scripts is a pain, but using user-extensions to provide the locators to your scripts makes it as simple as updating an external JS file.

Selenium RC

Most Selenium RC languages have a way of reading run-time properties from external files in some native manner:

  • Java – Properties
  • Python – ConfigParser or maybe just python dictionaries
  • Ruby – YAML
  • C# – App.Config

If you’re writing your scripts in Java, you should create a file, which would include something along the lines of:

# uses the id locator
# xpath
# css
registration.submit=css=form submit

Here is a Java class that interacts with our very simplistic registration form, and uses the external properties file (above) for its locators.

import java.util.Properties;

import org.junit.Before;
import org.junit.Test;
import org.junit.After;

import com.thoughtworks.selenium.DefaultSelenium;
import com.thoughtworks.selenium.Selenium;

public class TestRegistration.
    private Selenium selenium;
    public static Properties locators = new Properties();

    public void setUp() throws Exception
        InputStream is = TestRegistration.class.getResourceAsStream("/");

        selenium = new DefaultSelenium("localhost", 4444, "*firefox", "http://localhost:3000");

    public void testVisibleElement() throws Exception
        selenium.type(locators.getProperty("registration.firstname"), "fred");
        selenium.type(locators.getProperty("registration.lastname"), "flintstone");"registration.firstname");

    public void tearDown() throws Exception

Notice there are no hardcoded locator strings in the test, so we can change locators without changing the script. And if a consistent, well-thought-out naming convention is used for the property names, it CAN actually increase the readability of the scripts.

By using the same throughout the whole suite of scripts, a single change will cascade to all scripts. It also means that as ids become available on elements that didn’t have them previously, they can quickly be replaced with more structurally dependent methods, such as XPath and CSS. A further benefit is that, by quickly scanning the file, one can determine where in the suite XPath is still being used; converting from XPath to CSS is currently a recommended practice for performance reasons.

Page Objects

Page Objects are a way of representing a web page as a class (or number of classes) to bring Object Oriented techniques and patterns to Selenium. Most Page Object patterns have the locators abstracted out of the actual Page Objects and into a higher level parent class. But even then, those locators should be loaded from an external mechanism to allow for locator changes without recompilation.

Page Objects are beyond the scope of this article, but for an introduction and a sample implementation pattern of them in Python, see Page Objects in Python – Automating Page Checking without Brittleness.

Next Steps

Separating scripts from the locators being used is an important step towards reducing the amount of time teams invest in keeping their automation running. Too often, though, teams jump on this idea and invest valuable time converting their whole suite, which sets them further behind what the features development is producing. A better solution for the team is to collectively decide that scripts and locators will be separate, and from then on, only create scripts in this model, and only backport the abstraction of locator string when they break in existing code. The scripts will be ‘hybrid’ for a time, but ultimately it’s a much faster approach.

Selenium Tips: How to Coordinate Multiple Browsers in Sauce OnDemand

September 3rd, 2010 by Santiago Suarez Ordoñez

Let’s imagine your team is making a chat application. Nothing too fancy, just something simple in which you need to make sure multiple users get together, talk, and read each other.

Doesn’t sound that bad, huh? But how about coordinating those browsers? You need to have different sessions running at the same time in coordination and interacting with each other through your web app.

Now, when we think about parallelization and multiple browsers, Sauce OnDemand is, of course, our first answer, but here, there can be a small limitation. Sauce OnDemand has a special timeout that waits for new commands for no more than 90 seconds*. If it doesn’t get any new command, it will kill the job and assume something went wrong.

This effects us, as we need to get multiple browsers running in parallel and in coordination. As we request them one after another, depending on the amount of browsers and the load our service is put under, the first ones might timeout while waiting for the others to come.

To avoid this, the best solution is to write a multi-threaded script, in which browsers will send heartbeat commands while they are waiting for the rest. Multi-threading can be done with every programming language, but I decided to write my example in Python because I think it’s understandable for almost anyone, even if you’ve never seen Python before. The complete example is in github as a gist, and here is a brief description of what each interesting part is doing:

def get_browser_and_wait(browser, browser_num):
    print "starting browser %s" % browser_num
    browser.start()"/") # if we get here, OnDemand already gave us the browser
    print "browser %s ready" % browser_num
    while len(browsers_waiting) < len(browsers):
        print "browser %s sending heartbeat while waiting" % browser_num"/")

get_browser_and_wait is the function that will take care of the ‘start the browser’ request in OnDemand and wait till it comes back. Once the server is ready, it will keep sending open commands as heartbeats and waiting until the other requested browsers are ready and waiting too.

thread_list = []
for i, browser in enumerate(browsers):
    t = Thread(target=get_browser_and_wait, args=[browser, i + 1])

for t in thread_list:

This is the magic multithreading part, in which we iterate over every Selenium instance and call get_browser_and_wait with it. Once we send each browser to a thread, the main thread will continue and get to the t.join() part. By this method, the main thread will wait for every browser thread to complete in order to proceed with the rest of the code.

Again, the full example is up in github: If you’re interested in learning what you can use this kind of test for, stay tuned!

* This helps avoid running (and charging) tests that get disconnected or crash.

Selenium Tips: Finding elements by their inner text using :contains, a CSS pseudo-class

March 19th, 2010 by Santiago Suarez Ordoñez

As we already mentioned in our previous posts CSS Selectors in Selenium Demystified and Start improving your locators, CSS as a location strategy generally far outperforms XPATH.

That’s why for this Tip of the Week, we are presenting our users one more CSS pseudo-class to keep moving their test suites to this faster and cleaner location strategy.

Let’s imagine you have an element like the following:

<div>Click here</div>

The easiest way to locate it is using it’s “Click here” inner text and in XPATH you would do it using XPATH’s text() function:

//div[text() = "Click here"]

or even better (because the previous version can sometimes fail depending on the browser):

//div[contains(text(), "Click here")]

Now, if you want to move your locators to CSS in situations like this one, all you have to do is use the :contains() pseudo-class, with which your tests would end up using the following locator:

css=div:contains("Click here")

Did you find this useful? Are you still fighting with another XPATH functionality to completely move your tests to CSS? Let us know about it in the comments.

Selenium Tips: Parametrizing Selenese tests

March 12th, 2010 by Santiago Suarez Ordoñez

Even though I believe Selenium IDE should just be a transition environment for new Selenium users to reach Selenium RC, we know a lot of our users keep all their tests in Selenese and run them using Selenium IDE as their main testing tool.

That’s why at Sauce we have developed Sauce IDE, our own extension of Selenium IDE that combines Selenium RC’s cross-browser capabilities with the simplicity and interactive design of Selenium IDE.

To continue empowering the IDE users, I’m writing this tip of the week that will help making Selenese tests easier to maintain and write.

As you all know, Selenese is not a real programming language. Variable storage in it is pretty rough and there isn’t an easy way to share information between different Test Cases in the same Test Suite.

However there is a way to do it, through the use of the user-extensions.js file. Let me show you how in two easy steps:

First, create a user-extensions.js file with the following content:

storedVars["url"] = "/staging/search";
storedVars["title"] = "Search Your Item";
storedVars["search-string"] = "234234kjkj";

Then open Selenium IDE, or even better Sauce IDE, and in the Options menu, select the file in the “Selenium Core Extensions” field.
Close and open Selenium IDE again and you should be ready to use that variable in any test that you write. Even better, throughout your whole test suite.

For example:

open ${url}
assertText h1 ${title}
type search-field ${search-string}
click btnSearch
assertTextPresent Search Results for ${search-string}

This may not look super useful at first, but once your test suite grows, keeping things that change from time to time in a single place, makes it really easy to maintain, so if your application’s URL changes from “/staging/search” to “/search”, you will only need to fix the user-extensions.js file and all the hundreds of tests you could have will pass the same way as they did before.

Check the wikipedia’s page on DRY for more information about this programming technique.

Hope you found this useful. If you did, please let us know in the comments.


Selenium Tips: CSS Selectors in Selenium Demystified

January 29th, 2010 by Santiago Suarez Ordoñez

Following my previous Selenium testing TOTW about improving your locators, this blog post will show you some advanced CSS rules and pseudo-classes that will help you move your XPATH locators to CSS, a native approach on all browsers.

Next sibling

Our first example is useful for navigating lists of elements, such as forms or ul items. The next sibling will tell selenium to find the next adjacent element on the page that’s inside the same parent. Let’s show an example using a form to select the field after username.

</input> </input>

Let’s write a css selector that will choose the input field after “username”. This will select the “alias” input, or will select a different element if the form is reordered.

css=form input.username + input

Attribute values

If you don’t care about the ordering of child elements, you can use an attribute selector in selenium to choose elements based on any attribute value. A good example would be choosing the ‘username’ element of the form without adding a class.

</input> </input> </input> </input>

We can easily select the username element without adding a class or an id to the element.

css=form input[name='username']

We can even chain filters to be more specific with our selections.


Here Selenium will act on the input field with name=”continue” and type=”button”

Choosing a specific match

CSS selectors in Selenium allow us to navigate lists with more finess that the above methods. If we have a ul and we want to select its fourth li element without regard to any other elements, we should use nth-child or nth-of-type.


  • Cat
  • Dog
  • Car
  • Goat

If we want to select the fourth li element (Goat) in this list, we can use the nth-of-type, which will find the fourth li in the list.

css=ul#recordlist li:nth-of-type(4)

On the other hand, if we want to get the fourth element only if it is a li element, we can use a filtered nth-child which will select (Car) in this case.

css=ul#recordlist li:nth-child(4)

Note, if you don’t specify a child type for nth-child it will allow you to select the fourth child without regard to type. This may be useful in testing css layout in selenium.

css=ul#recordlist *:nth-child(4)

Sub-string matches

CSS in Selenium has an interesting feature of allowing partial string matches using ^=, $=, or *=. I’ll define them, then show an example of each:

^= Match a prefix
$= Match a suffix
*= Match a substring

A link with an “id” that starts with the text “id_prefix_”


A link with an “id” that ends with the text “_id_sufix”


A link with an “id” that contains the text “id_pattern”

Matching by inner text

And last, one of the more useful pseudo-classes, :contains() will match elements with the desired text block:

css=a:contains('Log Out')

This will find the log out button on your page no matter where it’s located. This is by far my favorite CSS selector and I find it greatly simplifies a lot of my test code.

Tune in next week for more Selenium testing tips from Sauce Labs.

Sauce Labs is the leading cloud testing platform that allows you to test across 170+ browsers/OS platforms in the cloud. With Selenium IDE and Selenium RC compatibilities, you can get complete cross-platform browser testing today.

Selenium Tips: HTTPS and self-signed certificate exceptions

January 22nd, 2010 by Santiago Suarez Ordoñez

This week in our support email,, we’ve seen a considerable number of cases of problems with Firefox and the Invalid Certificate warning thrown when a development environment is using the production’s certificate for HTTPS URLs, which causes the browser to wonder about the website’s identity.

Invalid Certificate warning

Warning displayed by Firefox 3.5 in this situation

If you do much browser-based testing, you have surely dealt with this situation more than once. In the manual testing world, you just tell your browser to add an exception for that certificate and ignore further errors, and you can forget about seeing that annoying warning anymore.

But when you move on to automation in the Selenium world, things are not always that simple. In most browsers, adding a certificate exception will work, as Selenium shares the same session as the user and will find the exception as you do. In Firefox, though, Selenium RC creates a special profile each time the browser is started and there’s no trail of the user settings in it.

The workaround for this problem is to create your own Firefox profile, with the specific certificate added on it by hand, and then tell Selenium to launch the browser based on that profile.

Another interesting approach, the one we take at Sauce Labs, where we can’t do this kind of trick, because we just don’t know which certificate the user will need before their test starts, is the use of RCE (Remember Certificate Exception), which is  a plugin that will automatically detect the warning and make the browser go through it, returning the control to Selenium after 4 or 5 seconds. Notice that if you use this approach, you will need to make sure your tests will tolerate this 5 secs additional delay to open the page.

Note: We currently support RCE in our Firefox 3.0 machines, and will be porting this extension to Firefox 3.5 soon.

You can find more info about RCE on its plugin page or in the author’s blog post.

Update: Adam Goucher, one of the big minds in the testing world just wrote a blog post that we couldn’t have written better ourselves about the first and most important advice regarding HTTPS and testing: Do yourself a favour and don’t test using HTTPS

Selenium Tips: Capturing Screenshots vs. Scrollbars!

December 11th, 2009 by Santiago Suarez Ordoñez

Capturing screenshots of your tests is one of the most important features you can give to an automation engineer. It’s the easiest way to actually understand from a report, why a test failed and how to reproduce it.

While our OnDemand will help you record videos of your entire test, sometimes you just want a screenshot. Selenium supports capturing screenshots on remote machines, as Sean discussed in a previous Tip of the Week (Taking ScreenShots on the Server).

Selenium has two methods dedicated to screenshot taking:

  • CaptureScreenshot
  • CaptureEntirePageScreenshot

An example of using the later function shows that it’s pretty amazing, giving you a full rendering of the page:

Entire Page Screenshot

While this function is perfect, it only works with Firefox, which greatly limits it’s usefulness. Unfortunately due to limited extension support in other browsers it’s going to be a while before we see this functionality ported.

If you’re stuck with the cross-browser version you’ll get limited success. We can see up to the fold after opening a new page, but we have no idea what’s going on in the depths of our page.

Regular Screenshot

This just isn’t going to work if we want to test something below the fold. There is a partial workaround, focus(). Focus will force all browsers to scroll the page so the selected element is visible. Here’s an example of a test we run to view the bottom of our homepage:

def test_case(self):
        browser = self.browser"/")
        png = browser.capture_screenshot_to_string()
        f = open('screenshot.png', 'wb')

Regular Plus Focus Screenshot

While this requires more code than Firefox + selenium, it will let you debug those hard to find rendering issues without having to boot up a virtual machine.

I hope you find this useful and start taking screenshots as soon your tests get harder to debug. If you need more visual debugging tools, our service Sauce OnDemand will capture full screen videos which will record every single selection event without any extra code!

Thank for tuning in, and check back after the new year for more Selenium Tip of The Week from Sauce Labs!

PS: You should follow us on twitter


Selenium Tips: Efficiently removing cookies

November 20th, 2009 by Santiago Suarez Ordoñez

We all know that when your tests run in firefox, things are easy. Even for sessions and cookies. Each time you run the start() method, Selenium will generate a clean Firefox profile and you can forget about previous tests or sessions.

Well, life is not always easy, and we don’t always have to test just Firefox… When IE comes to the game, Selenium can’t create clean profiles and cookies from previous tests will remain for future ones, causing unexpected issues.

For this, Selenium API’s has 2 methods you should use:

sel.delete_cookie(name, optionsString)

This method is useful when you only want to remove a specific cookie. The parameters it receives are sometimes hard to write and can take some time to actually make it work the way you need.


This one is my personal favorite. It just blows away any cookie that the current domain could have created in your browser.

But wait! I said “current domain”, not all domains. So, if your tests go though several domains, and you want to keep things clean, you’ll need something like the following:

def clean_history(sel, domains):
    temp = sel.get_location()
    for domain in domains:

This is a python function you can add anywhere, and by calling it with the selenium browser as the first parameter and a list of all domains used in the second one, will make you session clean without too much hassle. Off course you can always make this part of your own version of the selenium client, for then being able to call:


or even better, make selenium keep a record of each domain and then you can call:


But that will have to go in another TOTW.