Posts Tagged ‘selenium’

Ask a Selenium Expert: Strategies for Cross-Browser Testing

April 23rd, 2014 by Amber Kaplan

selenium testing & sauceThis is part 4 of 8 in a mini series of follow-up Q&A’s from Selenium expert Dave Haeffner. You can read up on the firstsecond, and third.

Dave discussed  how to build out a well factored, maintainable, resilient, and parallelized suite of tests that run locally, on a Continuous Integration system, and in the cloud in our recent webinar, “Selenium Bootcamp“.

Following the webinar, we had several follow-up questions. Dave’s agreed to respond to 8. Below you’ll find the third Q&A. Stay tuned next Wednesday for the next question.

3. ­I would like to see strategies for getting tests to work in multiple browers. For example, if my test works in Chrome but not Firefox, what do I do?

There are two things you’re likely to run into ­when running your tests across multiple browsers: speed of execution, and locator limitations.

Speed of execution issues occur when things execute too quickly (which indicates that you need to add explicit waits to your test code) or timeout (which indicates that the explicit waits you have are too tight and need to be loosened). The best approach to take is an iterative one. Run your tests and find the failures. Take each failed test, adjust your code as needed, and run it against the browsers you care about. Repeat until everything’s green.

In older browsers (e.g., Internet Explorer 8) you’ll be limited by the locators you can use (e.g., CSS3 nth locators like nth-child, nth-of-type, etc) and likely run into issues with some dynamic functionality (e.g., hovers). In cases like this, it’s simple enough to find an alternative set of locators to use that are specific to this browser and update your test to use them only when this browser is being used.

-Dave Haeffner, April 9, 2014

Can’t wait to see the rest of the Q&A? Read the whole post here.  Get more info on Selenium with Dave’s book, The Selenium Guidebook, or follow him on Twitter orGithub.

Have an idea for a blog post, webinar, or more? We want to hear from you! Submit topic ideas (or questions!)  here.

Ask a Selenium Expert: Selenium Test Report Example

April 16th, 2014 by Amber Kaplan

selenium testing & sauce

This is part 2 of 8 in a mini series of follow-up Q&A’s from Selenium expert Dave Haeffner. Read the first Q&A here.

Dave joined us and led our recent webinar, “Selenium Bootcamp“, wherein he discussed  how to build out a well factored, maintainable, resilient, and parallelized suite of tests that will run locally, on a Continuous Integration system, and in the cloud.

He’s also agreed to respond to 8 of the many follow-up questions we received post-webinar. Below you’ll find the second Q&A. Stay tuned next Wednesday for the next question.

2. ­Can you please show some examples of a Selenium test report?

Here are two examples:

-Dave Haeffner, April 9, 2014

Can’t wait to see the rest of the Q&A? Read the whole post here.  Get more info on Selenium with Dave’s book, The Selenium Guidebook, or follow him on Twitter orGithub.

Have an idea for a blog post, webinar, or more? We want to hear from you! Submit topic ideas (or questions!)  here.

Ask a Selenium Expert: Convert a Selenium IDE Script to Use Webdriver

April 9th, 2014 by Amber Kaplan

selenium testing & sauceThis is part 1 of  8 in a mini series of follow-up Q&A’s from Selenium expert Dave Haeffner.

Dave was kind enough to join us for a recent webinar, “Selenium Bootcamp“.   Topics included how to build out a well factored, maintainable, resilient, and parallelized suite of tests that will run locally, on a Continuous Integration system, and in the cloud.

He’s also agreed to respond to 8 of the many follow-up questions we received post-webinar. Below you’ll find the first Q&A. Stay tuned next Wednesday for the next question.

1. Is converting my Selenium IDE script to a programming language to start using webdriver sufficient?

You may get some value out of exporting your existing IDE tests, but they will likely require a good amount of clean-up. You’re likely to get more value out of identifying a few pieces of core functionality in the application you’re testing, and writing new tests for this functionality in a programming language from scratch.

-Dave Haeffner, April 9, 2014

Can’t wait to see the rest of the Q&A? Read the whole post here.  Get more info on Selenium with Dave’s book, The Selenium Guidebook, or follow him on Twitter or Github.

Have an idea for a blog post, webinar, or more? We want to hear from you! Submit topic ideas (or questions!)  here.

Webinar: Selenium at Salesforce Scale – Register Today

April 3rd, 2014 by Amber Kaplan

sfdc-logoEver wondered how an Enterprise company like Salesforce runs their QA tests? Yup, there’s a webinar for that.

After participating in this successful meetup, we wanted to cast this knowledge even farther. You’re invited to learn about Salesforce’s inventory of Selenium tests, how they run them at scale, and how to architect your test harness for success.

Join Salesforce engineers David Louvton and Sagar Wanaselja on Wednesday, April 16th, 2014 at 10:00am PDT for our latest webinar, Selenium at Salesforce Scale.

David and Sagar will cover the details of their engineering team, running their “Selenium farm,” scaling issues, Selenium design patterns, and how to overcome test failures.

A Q&A session will follow. Register today!

As usual, this event is FREE.

Do you want to share your knowledge with our community? We want to hear from you! Submit topic ideas for guest posts, webinars and more here

Slide into Selenium! “How To Use Selenium, Successfully” by Dave Haeffner

April 1st, 2014 by Amber Kaplan

Thanks to all of you who attended our last webinar, Selenium Bootcamp, with expert Dave Haeffner! Click here to listen to the recording, and check out the slide deck below from his presentation if you haven’t already done so.

Want to share your thoughts or expertise with our community? We want to hear it! Submit your blog post or webinar topic here.

Webinar: Selenium Bootcamp with Expert Dave Haeffner

March 20th, 2014 by Amber Kaplan

Whether you want to start fresh, learn more, or fill in gaps about QA testing using Selenium, you won’t want to miss our latest webinar! Join us for Selenium Bootcamp with expert Dave Haeffner on Thursday, March 27th, 2014 10:00am PDT.

QA testing with Selenium can help to dramatically improve your business’ productivity by minimizing testing time, cost, and hassle. Dave will walk you through all the necessary steps to learn how to build out a well-factored, maintainable, resilient, and parallelized suite of tests that will run locally, on a Continuous Integration system, and in the cloud.  A live Q&A session will follow. Register today!

Dave is the author of Elemental Selenium (a free, once weekly Selenium tip newsletter that is read by hundreds of testing professionals) as well as a new book, The Selenium Guidebook. He is also the creator and maintainer of ChemistryKit (an open-source Selenium framework). He has helped numerous companies successfully implement automated acceptance testing; including The Motley Fool, ManTech International, Sittercity, and Animoto. He is a founder and co-organizer of the Selenium Hangout and has spoken at numerous conferences and meetups about acceptance testing.

WebDriver: The Final Frontier – Selenium Camp 2014 (Presentation)

March 17th, 2014 by Amber Kaplan

Automated testing is the solution for a fast dev cycle that maintains high quality. Learn about the mobile JSON wire protocol, a collaboration between Selenium, Appium, ios driver, Selendroid, and firefoxOS in this presentation by Sauce Labs’ Bernie Kobos today.

WebDriver: The Final Frontier – Selenium Camp 2014

Getting Started With Selenium – Chapter 6: Packaging for Use

March 11th, 2014 by Bill McGee

selenium-logo-160x144This post is the sixth in a series of “Getting Started with Selenium Testing” posts from Dave Haeffner, a noted expert on Selenium and automated testing, and a frequent contributor to the Sauce blog and Selenium community. This series is for those who are brand new to test automation with Selenium and a new chapter will be posted every Tuesday (eight chapters in all).

Packaging For Use

In order to get the most out of our tests and page objects, we’ll need to package them into a more useful structure.

Let’s do that using the examples from the previous write-ups.

Global Setup And Teardown

First we’ll need to pull the test setup and teardown actions out of our tests and into a central place. In RSpec this is straight-forward through the use of a ‘spec_helper’ file.

# filename: spec_helper.rb

require 'selenium-webdriver'

RSpec.configure do |config|

  config.before(:each) do
    @driver = Selenium::WebDriver.for :firefox
  end

  config.after(:each) do
    @driver.quit
  end

end

We need to include the Selenium library here, and by doing so, can remove it from our tests. And by having our test configuration here, we can remove it from the `before(:each)` and `after(:each)` in our tests as well — replacing them with a simple require statement at the top of the file (`require_relative ‘spec_helper’`).

Base URL

Rather than hard-coding a URL in our tests and page objects, We’ll want to put it someplace central, and we’ll want it to be configurable. So let’s create a ‘config.rb’ file in the parent directory and place it there.

# filename: config.rb

ENV['base_url'] ||= 'http://the-internet.herokuapp.com'

By using a conditional when setting the environment variable (`||=`) we are making it so we can override this value when launching our test suite (e.g., `base_url=’http://localhost:4567′`). It essentially means if the environment variable already exists and contains a value, use it. Otherwise, set it to ‘http://the-internet.herokuapp.com’. We can then reference this variable in our page objects where necessary (if we haven’t already).

For instance:

# filename: login.rb

class Login
...

def initialize(driver)
@driver = driver
@driver.get ENV['base_url'] + '/login'
end

...

Folder Organization

It’s about time we create some folders for our specs and page objects. To err on the side of simplicity, let’s call the folders ‘spec’ (for our tests) and ‘pages’ (for our page objects). We are using ‘spec’ since it is a default folder that RSpec will look for.

Here’s everything we should have after creating folders and moving files around:

.
|-- config.rb
|-- Gemfile
|-- pages
| |-- dynamic_loading.rb
| `-- login.rb
`-- spec
|-- dynamic_loading_spec.rb
|-- login_spec.rb
`-- spec_helper.rb

Updating Require Statements

As a result of doing this, we will need to update the require statements in our tests.

# filename: spec/login_spec.rb

require_relative 'spec_helper'
require_relative '../pages/login'

describe 'Login' do
...

# filename: spec/dynamic_loading_spec.rb

require_relative 'spec_helper'
require_relative '../pages/dynamic_loading'

describe 'Dynamic Loading' do
...

Note the use of double-dots (`..`) in the page object require statement. This is how we tell Ruby to traverse up a directory (from our spec directory) before trying to access the page objects folder. The `spec_helper` require remains unchanged since this file lives in the same directory as our tests.

Running Everything

Now that things are cleaned up, we can run everything. To do that we’ll want to make sure to include our new config file. We can do that by specifying it at run time with `rspec –require ./config.rb`, or, `rspec -r ./config.rb` (for short).

Note the `./` before `config.rb`. This tells RSpec that the config file is in the current directory.

Give it a shot. All of the tests should run and pass.

For more examples like this (along with complete working code) — grab your copy of  The Selenium Guidebook.

Previous Chapters: 1. Getting Started | 2. Writing a Good Acceptance Test | 3. Writing Your First Selenium Test | 4. How To Reuse Your Test Code | 5. Writing Resilient Test Code

Dave is the author of Elemental Selenium (a free, once weekly Selenium tip newsletter that is read by hundreds of testing professionals) as well as a new book, The Selenium Guidebook. He is also the creator and maintainer of ChemistryKit (an open-source Selenium framework). He has helped numerous companies successfully implement automated acceptance testing; including The Motley Fool, ManTech International, Sittercity, and Animoto. He is a founder and co-organizer of the Selenium Hangout and has spoken at numerous conferences and meetups about acceptance testing.

Getting Started with Selenium: Chapter 5 – Writing Resilient Test Code

March 4th, 2014 by Bill McGee

selenium-logo-160x144This post is the fifth in a series of “Getting Started with Selenium Testing” posts from Dave Haeffner, a noted expert on Selenium and automated testing, and a frequent contributor to the Sauce blog and Selenium community. This series is for those who are brand new to test automation with Selenium and a new chapter will be posted every Tuesday (eight chapters in all).

Writing Resilient Test Code

Ideally, you should be able to write your tests once and run them across all supported browsers. While this is a rosy proposition, there is some work to make this a reliable success. And sometimes there may be a hack or two involved. But the lengths you must go really depends on the browsers you care about and the functionality you’re dealing with.

By using high quality locators you will be well ahead of the pack, but there are still some persnickity issues to deal with. Most notably – timing. This is especially true when working with dynamic, JavaScript heavy pages (which is more the rule than the exception in a majority of applications you’ll deal with).

But there is a simple approach that makes up the bedrock of reliable and resilient Selenium tests — and that’s how you wait and interact with elements. The best way to accomplish this is through the use of explicit waits.

An Example

Let’s step through an example that demonstrates this against a dynamic page on the-internet. The functionality is pretty simple — there is a button. When you click it a loading bar appears for 5 seconds, then disappears, and gets replaced with the text ‘Hello World!’.

Let’s start by looking at the markup on the page.

Dynamically Loaded Page Elements

Example 1: Element on page that is hidden


At a glance it’s simple enough to tell that there are unique idattributes that we can use to reference the start button and finish text. Let’s add a page object for Dynamic Loading.

Part 1: Create A Page Object

# filename: dynamic_loading.rb

class DynamicLoading

  START_BUTTON  = { css: '#start button' }
  FINISH_TEXT   = { id: 'finish' }

  def initialize(driver)
    @driver = driver
    @driver.get "http://the-internet.herokuapp.com/dynamic_loading/1"
  end

  def start
    @driver.find_element(START_BUTTON).click
  end

  def finish_text_present?
    wait_for { is_displayed? FINISH_TEXT }
  end

  def is_displayed?(locator)
    @driver.find_element(locator).displayed?
  end

  def wait_for(timeout = 15)
    Selenium::WebDriver::Wait.new(:timeout => timeout).until { yield }
  end

end

This approach should look familiar to you if you checked out the last write-up. The thing which is new is the wait_for method. In it we are using a built-in Selenium wait action. This is the mechanism with which we will perform explicit waits.

More On Explicit Waits

It’s important to set a reasonably sized default timeout for the explicit wait. But you want to be careful not to make it too high. Otherwise you run into a lot of the same timing issues you get from implicit waits. But set it too low and your tests will be brittle, forcing you to run down trivial and transient issues.

In our page object when we’re usingwait_for { is_displayed? FINISH_TEXT } we are telling Selenium to to see if the finish text is displayed on the page. It will keep trying until it either returns true or reaches fifteen seconds — whichever comes first.

If the behavior on the page takes longer than we expect (e.g., due to slow inherently slow load times), we can simply adjust this one wait time to fix the test (e.g.,wait_for(30) { is_displayed? FINISH_TEXT }) — rather than increase a blanket wait time (which impacts every test). And since it’s dynamic, it won’t always take the full amount of time to complete.

Part 2: Write A Test To Use The New Page Object

Now that we have our page object we can wire this up in a new test file.

# filename: dynamic_loading_spec.rb

require 'selenium-webdriver'
require_relative 'dynamic_loading'

describe 'Dynamic Loading' do

  before(:each) do
    @driver = Selenium::WebDriver.for :firefox
    @dynamic_loading = DynamicLoading.new(@driver)
  end

  after(:each) do
    @driver.quit
  end

  it 'Waited for Hidden Element' do
    @dynamic_loading.start
    @dynamic_loading.finish_text_present?.should be_true
  end

end

When we run it (rspec dynamic_loading_page.rb from the command-line) it should pass, rather than throwing an exception (like in the last write-up when the element wasn’t present).

As an aside — an alternative approach would be to rescue the exception like this:

  def is_displayed?(locator)
    begin
      @driver.find_element(locator).displayed?
    rescue Selenium::WebDriver::Error::NoSuchElementError
      false
    end
  end

This would enable you to check the negative condition for whether or not an element is displayed. And it can be used with an explicit wait as well (it won’t change it’s behavior).

Part 3: Add A Second Test

Let’s step through one more dynamic page example to see if our explicit wait approach holds up.

Our second example is laid out similarly to the last one, the main difference is that it will render the final result after the progress bar completes. Here’s the markup for it.

Dynamically Loaded Page Elements

Example 2: Element rendered after the fact



In order to find the selector for the finish text element we need to inspect the page after the loading bar sequence finishes. Here’s what it looks like.

Hello World!

Before we add our test, we need to modify our page object to accommodate visiting the different example URLs.

# filename: dynamic_loading.rb

class DynamicLoading

  START_BUTTON  = { css: '#start button' }
  FINISH_TEXT   = { id: 'finish' }

  def initialize(driver)
    @driver = driver
  end

  def visit_example(example_number)
    @driver.get "http://the-internet.herokuapp.com/dynamic_loading/#{example_number}"
  end

  ...

Now that we have that sorted, let’s add a new test to reference the markup shown above (and update our existing test to use the new.visit_example method).

# filename: dynamic_loading_spec.rb


require_relative 'dynamic_loading'

describe 'Dynamic Loading' do

  ...

  it 'Waited for Hidden Element' do
    @dynamic_loading.visit_example 1
    @dynamic_loading.start
    @dynamic_loading.finish_text_present?.should be_true
  end

  it 'Waited for Element To Render' do
    @dynamic_loading.visit_example 2
    @dynamic_loading.start
    @dynamic_loading.finish_text_present?.should be_true
  end

end

If we run these tests (rspec dynamic_loading_spec.rb from the command-line) then the same approach will work for both cases.

Explicit waits are one of the most important concepts in testing with Selenium. Use them often.

For a more in-depth look at explicit waits and how to address cross-browser timing issues — grab your copy of The Selenium Guidebook.

Previous Chapters: 1. Getting Started | 2. Writing a Good Acceptance Test | 3. Writing Your First Selenium Test | 4. How To Reuse Your Test Code

Dave is the author of Elemental Selenium (a free, once weekly Selenium tip newsletter that is read by hundreds of testing professionals) as well as a new book, The Selenium Guidebook. He is also the creator and maintainer of ChemistryKit (an open-source Selenium framework). He has helped numerous companies successfully implement automated acceptance testing; including The Motley Fool, ManTech International, Sittercity, and Animoto. He is a founder and co-organizer of the Selenium Hangout and has spoken at numerous conferences and meetups about acceptance testing.

Ohai, 100 Million Tests on Sauce Labs!

February 27th, 2014 by Amber Kaplan

Today we’re ecstatic to announce we surpassed 100 million test runs on Sauce Labs. Yep, that’s right. 100 MILLION tests have been executed on our cloud of real browsers and platforms. Now that’s a whole lotta testing, don’t ya think?!

It’s been an awesome couple years for Sauce, Selenium and automated testing in general. So we thought we’d use this milestone to look back at the growth both internally and within the testing community at large with a fun infographic below. Some of our favorite highlights include:

  • Support for browsers / OS platforms grew from 41 in 2010 to 270+ in 2014
  • Selenium IDE has been downloaded more than 10 million times
  • 10,000+ ounces of Saucy hot sauce distributed all over the world

It’s been a heck of a ride over the last five years, and we couldn’t be happier to be part of the movement. Onward to 200 Million!

Click to enlarge

Click to enlarge