Posts Tagged ‘selenium’

New! Support for OS X 10.11 and iOS 8.3, 8.4 & 9.0

August 12th, 2015 by Ken Drachnik

El Capitan 200 px wideIn our continuing effort to make Sauce Labs the best place to test, we’ve just added some additional OSes and Browsers. In addition to Windows 10 and the Edge Browser we announced last week, today we are extending the platforms we support to include released and upcoming iOS and OS X operating systems:

OSX 10.11 El Capitan (beta)
iOS 8.3 and 8.4
iOS 9 (beta)
Safari 8.0.7

To use any of these platforms visit our platforms configurator.

When we ask our users what it is they love about Sauce one of the most common responses is the wide variety of platforms we provide our users. In an ecosystem of increasing complexity, we know our customers have a lot of devices and platforms they need to test against. Our humble aspiration is to make your life a just little bit easier by having the platforms you need when you need them.

Sign in to start testing

How To Test Responsive Web Apps with Selenium

June 8th, 2015 by Dave Haeffner

The Problem

When testing a web application with a responsive layout you’ll want to verify that it renders the page correctly in the common resolutions your users use. But how do you do it?

Historically this type of verification has been done manually at the end of a development workflow — which tends to lead to delays and visual defects getting released into production.

A Solution

We can easily sidestep these concerns by automating responsive layout testing so we can get feedback fast. This can be done with a Selenium test, Applitools Eyes, and Sauce Labs.

Let’s dig in with an example.

An Example

NOTE: This example is built using Ruby and the RSpec testing framework. To play along, you’ll need Applitools Eyes and Sauce Labs accounts. They both have free trial accounts which you can sign up for here and here (no credit card required).

Let’s test the responsive layout for the login of a website (e.g., the one found on the-internet).

In RSpec, a test file is referred to as a “spec” and ends _spec.rb. So our test file will be login_spec.rb. We’ll start it by requiring our requisite libraries (e.g., selenium-webdriver to drive the browser and eyes_selenium to connect to Applitools Eyes) and specifying some initial configuration values with sensible defaults. (more…)

Video: “Eliminate Rollbacks” Talk by Neil Manvar

May 6th, 2015 by Amber Kaplan

Missed Velocity Conf SF? We’ve got you covered. Check out this talk by Neil Manvar, Sauce Lab’s new Professional Services Lead. View “Eliminate Rollbacks” below and let us know what you think in the comments.

Video: Jason Huggins – Fixing HealthCare.gov, One Test at a Time

April 23rd, 2015 by Amber Kaplan

This week, Sauce Labs co-founder and Selenium creator Jason Huggins came to visit to chat about his leave of absence to help fix HealthCare.gov.  For those who missed his talk at the Selenium meetup, we’re happy to report that we got our hands on a recording.  Check out the video below to watch.

In the video, Jason references a federal “digital playbook” that was written and published following the healthcare.gov overhaul. Point # 10 in the playbook –  “Automate Testing and Deployments” – was suggested and driven by Jason. #TestAllTheThings

 

ABSTRACT

In late 2013, Selenium creator Jason Huggins joined President Obama’s “tech surge” team to help fix HealthCare.gov. In D.C. during the height of the crisis in November and December 2013, Jason had a behind-the-scenes view into a unique period in American history when a website’s quality (or lack thereof) had the attention of the nation, the press, the President, and Congress.

In this talk, Jason will share some of his stories from the HealthCare.gov turnaround and the “HealthCare 2.0″ effort in mid-2014. Jason will talk about the newly created U.S. Digital Services and how it was created out of the original HealthCare.gov crisis. He’ll also cover the U.S. Digital Services Playbook and what the role of automated testing and deployment will be in future U.S. Government projects.

Lastly, Jason will talk about opportunites for how Silicon Valley can help government build effective digital services in the future.

BIO

Jason is a software engineer living in Chicago. He started the Selenium project in 2004 at ThoughtWorks. He later joined Google to work on large-scale web testing for Gmail, Google Maps, and other teams. He left Google to co-found Sauce Labs as CTO to create a cloud-based Selenium service. In late 2013, Jason took leave from Sauce to help with the HealthCare.gov turnaround. He is also the creator of Tapster, a mobile app testing robot that’s been featured in Popular Science, Wired,Tech Crunch, and the MIT Technology Review.

Speed up Selenium with Application Mocks

April 8th, 2015 by Chris Riley

Let’s face it, we all know Selenium is slow. Not the Selenium scripts themselves, but the process of interacting with the operating system and browser drivers, in addition to your application’s own responsiveness. Luckily for us, there’s a great way to speed up functional testing in Selenium by combining a comprehensive parallel testing grid with application mocks. (more…)

Practical Tips + Tricks for Selenium Test Automation [RECAP]

April 7th, 2015 by Amber Kaplan

selenium testing & sauceThanks to all of you who joined us for Practical Tips + Tricks for Selenium Test Automation with expert Dave Haeffner – author of The Selenium Guidebook.

In this webinar, Dave explained the best and most useful tips & tricks from his weekly Selenium tip newsletter (elementalselenium.com).

Topics covered included: (more…)

Applitools Recognizes 16 Influencers + Thought Leaders in Test Automation

April 2nd, 2015 by Amber Kaplan

Test Automation Top Influencers and Thought Leaders_500wSaucesome news in the automated testing world! Our friends at Applitools recognized several Sauce employees and friends in their Top 16 Influencers and Thought Leaders in Test Automation list, including our Co-Founder Jason Huggins, Director of Ecosystems and Integrations Jonathan Lipps, and Selenium expert Dave Haeffner.

Applitools stated, (more…)

Practical Tips + Tricks for Selenium Test Automation [WEBINAR]

April 1st, 2015 by Amber Kaplan

Have unanswered Selenium questions? Want to learn how to use Selenium like a Pro? Join Dave Haeffner – author of The Selenium Guidebook – as he steps through the best and most useful tips & tricks from his weekly Selenium tip newsletter (elementalselenium.com).

Topics covered include: (more…)

Jason Huggins: Fixing HealthCare.gov, One Test at a Time [MEETUP]

March 24th, 2015 by Amber Kaplan

jason-hugginsYou may recall that our co-founder Jason Huggins took a leave of absence to help fix HealthCare.gov. He’ll be back on April 21 to talk about his experiences there at the next Selenium Meetup in San Francisco. RSVP before it fills up! Info below.

Date + Time: Tuesday, April 21, 2015 at 6:30 PM

Location: If (we): 848 Battery Street, San Francisco, CA (more…)

How To Add Visual Testing To Existing Selenium Tests

February 27th, 2015 by Dave Haeffner

Thanks again to those of you who attended our recent webinar with Applitools on automated visual testing.  If you want to share it or if you happened to miss it, you can catch the audio and slides hereWe also worked with Selenium expert Dave Haeffner to provide the how-to on the subject. Enjoy his post below.

 

The Problem

In previous write-ups I covered what automated visual testing is and how to do it. Unfortunately, based on the examples demonstrated, it may be unclear how automated visual testing fits into your existing automated testing practice.

Do you need to write and maintain a separate set of tests? What about your existing Selenium tests? What do you do if there isn’t a sufficient library for the programming language you’re currently using?

A Solution

You can rest easy knowing that you can build automated visual testing checks into your existing Selenium tests. By leveraging a third-party platform like Applitools Eyes, this is a simple feat.

And when coupled with Sauce Labs, you can quickly add coverage for those hard to reach browser, device, and platform combinations.

Let’s step through an example.

An Example

NOTE: This example is written in Java with the JUnit testing framework.

Let’s start with an existing Selenium test. A simple one that logs into a website.

// filename: Login.java

import org.junit.After;
import org.junit.Assert;
import org.junit.Before;
import org.junit.Test;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.firefox.FirefoxDriver;

public class Login {

    private WebDriver driver;

    @Before
    public void setup() {
        driver =  new FirefoxDriver();
    }

    @Test
    public void succeeded() {
        driver.get("http://the-internet.herokuapp.com/login");
        driver.findElement(By.id("username")).sendKeys("tomsmith");
        driver.findElement(By.id("password")).sendKeys("SuperSecretPassword!");
        driver.findElement(By.id("login")).submit();
        Assert.assertTrue("success message should be present after logging in",
                driver.findElement(By.cssSelector(".flash.success")).isDisplayed());
    }

    @After
    public void teardown() {
        driver.quit();
    }
}

In it we’re loading an instance of Firefox, visiting the login page on the-internet, inputting the username & password, submitting the form, asserting that we reached a logged in state, and closing the browser.

Now let’s add in Applitools Eyes support.

If you haven’t already done so, you’ll need to create a free Applitools Eyes account (no credit-card required). You’ll then need to install the Applitools Eyes Java SDK and import it into the test.

// filename: pom.xml

<dependency>
  <groupId>com.applitools</groupId>
  <artifactId>eyes-selenium-java</artifactId>
  <version>RELEASE</version>
</dependency>
// filename: Login.java

import com.applitools.eyes.Eyes;
...

Next, we’ll need to add a variable (to store the instance of Applitools Eyes) and modify our test setup.

// filename: Login.java
...
public class Login {

    private WebDriver driver;
    private Eyes eyes;

    @Before
    public void setup() {
        WebDriver browser =  new FirefoxDriver();
        eyes = new Eyes();
        eyes.setApiKey("YOUR_APPLITOOLS_API_KEY");
        driver = eyes.open(browser, "the-internet", "Login succeeded");
    }
...

Rather than storing the Selenium instance in the driver variable, we’re now storing it in a localbrowser variable and passing it into eyes.open — storing the WebDriver object that eyes.openreturns in the driver variable instead.

This way the Eyes platform will be able to capture what our test is doing when we ask it to capture a screenshot. The Selenium actions in our test will not need to be modified.

Before calling eyes.open we provide the API key (which can be found on your Account Details page in Applitools). When calling eyes.open, we pass it the Selenium instance, the name of the app we’re testing (e.g., "the-internet"), and the name of the test (e.g., "Login succeeded").

Now we’re ready to add some visual checks to our test.

// filename: Login.java
...
    @Test
    public void succeeded() {
        driver.get("http://the-internet.herokuapp.com/login");
        eyes.checkWindow("Login");
        driver.findElement(By.id("username")).sendKeys("tomsmith");
        driver.findElement(By.id("password")).sendKeys("SuperSecretPassword!");
        driver.findElement(By.id("login")).submit();
        eyes.checkWindow("Logged In");
        Assert.assertTrue("success message should be present after logging in",
                driver.findElement(By.cssSelector(".flash.success")).isDisplayed());
        eyes.close();
    }
...

With eyes.checkWindow(); we are specifying when in the test’s workflow we’d like Applitools Eyes to capture a screenshot (along with some description text). For this test we want to check the page before logging in, and then the screen just after logging in — so we use eyes.checkWindow(); two times.

NOTE: These visual checks are effectively doing the same work as the pre-existing assertion (e.g., where we’re asking Selenium if a success notification is displayed and asserting on the Boolean result) — in addition to reviewing other visual aspects of the page. So once we verify that our test is working correctly we can remove this assertion and still be covered.

We end the test with eyes.close. You may feel the urge to place this in teardown, but in addition to closing the session with Eyes, it acts like an assertion. If Eyes finds a failure in the app (or if a baseline image approval is required), then eyes.close will throw an exception; failing the test. So it’s best suited to live in the test.

NOTE: An exceptions from eyes.close will include a URL to the Applitools Eyes job in your test output. The job will include screenshots from each test step and enable you to play back the keystrokes and mouse movements from your Selenium tests.

When an exception gets thrown by eyes.close, the Eyes session will close. But if an exception occurs before eyes.close can fire, the session will remain open. To handle that, we’ll need to add an additional command to our teardown.

// filename: Login.java
...
    @After
    public void teardown() {
        eyes.abortIfNotClosed();
        driver.quit();
    }
}

eyes.abortIfNotClosed(); will make sure the Eyes session terminates properly regardless of what happens in the test.

Now when we run the test, it will execute locally while also performing visual checks in Applitools Eyes.

What About Other Browsers?

If we want to run our test with it’s newly added visual checks against other browsers and operating systems, it’s simple enough to add in Sauce Labs support.

NOTE: If you don’t already have a Sauce Labs account, sign up for a free trial account here.

First we’ll need to import the relevant classes.

// filename: Login.java
...
import org.openqa.selenium.Platform;
import org.openqa.selenium.remote.DesiredCapabilities;
import org.openqa.selenium.remote.RemoteWebDriver;
import java.net.URL;
...

We’ll then need to modify the test setup to load a Sauce browser instance (via Selenium Remote) instead of a local Firefox one.

// filename: Login.java
...
    @Before
    public void setup() throws Exception {
        DesiredCapabilities capabilities = DesiredCapabilities.internetExplorer();
        capabilities.setCapability("platform", Platform.XP);
        capabilities.setCapability("version", "8");
        capabilities.setCapability("name", "Login succeeded");
        String sauceUrl = String.format(
                "http://%s:%s@ondemand.saucelabs.com:80/wd/hub",
                "YOUR_SAUCE_USERNAME",
                "YOUR_SAUCE_ACCESS_KEY");
        WebDriver browser = new RemoteWebDriver(new URL(sauceUrl), capabilities);
        eyes = new Eyes();
        eyes.setApiKey(System.getenv("APPLITOOLS_API_KEY"));
        driver = eyes.open(browser, "the-internet", "Login succeeded");
    }
...

We tell Sauce what we want in our test instance through DesiredCapabilities. The main things we want to specify are the browser, browser version, operating system (OS), and name of the test. You can see a full list of the available browser and OS combinations here.

In order to connect to Sauce, we need to provide an account username and access key. The access key can be found on your account page. These values get concatenated into a URL that points to Sauce’s on-demand Grid.

Once we have the DesiredCapabilities and concatenated URL, we create a Selenium Remote instance with them and store it in a local browser variable. Just like in our previous example, we feedbrowser to eyes.open and store the return object in the driver variable.

Now when we run this test, it will execute against Internet Explorer 8 on Windows XP. You can see the test while it’s running in your Sauce Labs account dashboard. And you can see the images captured on your Applitools account dashboard.

A Small Bit of Cleanup

Both Applitools and Sauce Labs require you to specify a test name. Up until now, we’ve been hard-coding a value. Let’s change it so it gets set automatically.

We can do this by leveraging a JUnit TestWatcher and a public variable.

// filename: Login.java
...
import org.junit.rules.TestRule;
import org.junit.rules.TestWatcher;
import org.junit.runner.Description;
...
public class Login {

    private WebDriver driver;
    private Eyes eyes;
    public String testName;

    @Rule
    public TestRule watcher = new TestWatcher() {
        protected void starting(Description description) {
            testName = description.getDisplayName();
        }
    };
...

Each time a test starts, the TestWatcher starting function will grab the display name of the test and store it in the testName variable.

Let’s clean up our setup to use this variable instead of a hard-coded value.

// filename: Login.java
...
    @Before
    public void setup() throws Exception {
        DesiredCapabilities capabilities = DesiredCapabilities.internetExplorer();
        capabilities.setCapability("platform", Platform.XP);
        capabilities.setCapability("version", "8");
        capabilities.setCapability("name", testName);
        String sauceUrl = String.format(
                "http://%s:%s@ondemand.saucelabs.com:80/wd/hub",
                System.getenv("SAUCE_USERNAME"),
                System.getenv("SAUCE_ACCESS_KEY"));
        WebDriver browser = new RemoteWebDriver(new URL(sauceUrl), capabilities);
        eyes = new Eyes();
        eyes.setApiKey(System.getenv("APPLITOOLS_API_KEY"));
        driver = eyes.open(browser, "the-internet", testName);
    }
...

Now when we run our test, the name will automatically appear. This will come in handy with additional tests.

One More Thing

When a job fails in Applitools Eyes, it automatically returns a URL for it in the test output. It would be nice if we could also get the Sauce Labs job URL in the output. So let’s add it.

First, we’ll need a public variable to store the session ID of the Selenium job.

// filename: Login.java
...
public class Login {

    private WebDriver driver;
    private Eyes eyes;
    public String testName;
    public String sessionId;
...

Next we’ll add an additional function to TestWatcher that will trigger when there’s a failure. In it, we’ll display the Sauce job URL in standard output.

// filename: Login.java
...
    @Rule
    public TestRule watcher = new TestWatcher() {
        protected void starting(Description description) {
            testName = description.getDisplayName();
        }

        @Override
        protected void failed(Throwable e, Description description) {
            System.out.println(String.format("https://saucelabs.com/tests/%s", sessionId));
        }
    };
...

Lastly, we’ll grab the session ID from the Sauce browser instance just after it’s created.

// filename: Login.java
...
        WebDriver browser = new RemoteWebDriver(new URL(sauceUrl), capabilities);
        sessionId = ((RemoteWebDriver) browser).getSessionId().toString();
...

Now when we run our test, if there’s a Selenium failure, a URL to the Sauce job will be returned in the test output.

Expected Outcome

  • Connect to Applitools Eyes
  • Load an instance of Selenium in Sauce Labs
  • Run the test, performing visual checks at specified points
  • Close the Applitools session
  • Close the Sauce Labs session
  • Return a URL to a failed job in either Applitools Eyes or Sauce Labs

Outro

Happy Testing!

 

About Dave Haeffner: Dave is the author of Elemental Selenium (a free, once weekly Selenium tip newsletter that is read by hundreds of testing professionals) as well as a new book, The Selenium Guidebook. He is also the creator and maintainer of ChemistryKit (an open-source Selenium framework). He has helped numerous companies successfully implement automated acceptance testing; including The Motley Fool, ManTech International, Sittercity, and Animoto. He is a founder and co-organizer of the Selenium Hangout and has spoken at numerous conferences and meetups about acceptance testing.