Posts Tagged ‘Functional testing’

3 Simple Strategies to Get Started With Automation

March 24th, 2016 by Joe Nolan

If your test automation team’s directive is to automate X amount of tests, and you have no strategy as to which tests they should focus on, you are wasting your time. Before you begin writing your first line of automation code, make sure you have a strategy in place. Otherwise, you will have a ton of ineffective tests to maintain.

Don’t Choose a Random Goal

How many times have you been told that the goal of the team is to have X amount of test coverage? This is an arbitrary value picked out of the sky. What is it based on? If a UI automation team were to cover 80% of the stories in a sprint, they would never get done in time.

We all know how fragile UI automation is! How many times will a designer make a change that directly affects the UI and breaks the test? This is almost manageable during a sprint while you are working closely together, but how about when the product is sent to be translated to another language? The translator inevitably comes back with suggestions to allow for phrases more common and translatable. Bugs might be entered and UI changes made by a maintenance team with no heads up to the automation team, and Bam! — You have broken tests that need to be investigated. (more…)

Translating Web App Functional Testing To Mobile

April 27th, 2015 by Greg Sypolt

Technological advancements and the explosion of devices across multiple platforms means hardware and software developers have a much more difficult job. They have to keep up with the demand to develop and roll out new products. One of the most significant issues is accounting for the differences in system response, when responding to mobile traffic rather than to internet traffic. (more…)

How Yahoo! Mail Transformed Its Functional Testing + Continuous Delivery Process [RECAP]

October 24th, 2014 by Bill McGee

Yahoo Mail 2 (1)Thanks to those of you who joined us for our last webinar, How Yahoo! Mail Transformed Its Functional Testing + Continuous Delivery Process with Front End Developer Neil Manvar. (more…)

How Yahoo! Mail Transformed Its Functional Testing + Continuous Delivery Process [WEBINAR]

October 8th, 2014 by Bill McGee

Yahoo Mail 2 (1)

Learn how one Engineer helped transform internet giant Yahoo! Mail’s quality engineering process through functional testing and continuous delivery – and the role Sauce Labs plays in their process.

Be sure to sign up for our next webinar on Wednesday, October 22nd, 2014, at 11:00am PDT. You won’t want to miss this one! 

Click here to register.


Re-Blog: Add Some Sauce To Your IE Tests

September 4th, 2014 by Bill McGee

Sauce Labs hearts ThoughtWorks! And apparently the feeling’s mutual. Check out this great blog post mentioning Sauce Labs by Tom Clement Oketch.

See an excerpt below:


Run Your Selenium Tests Completely in the Cloud Using Travis CI and Sauce Labs

March 18th, 2013 by Santiago Suarez Ordoñez

Passing builds!This is a quick tutorial to get you started running automated tests completely in the cloud as part of your development process using Travis CI and real browsers in Sauce Labs.

As a guinea pig during this blog post, I used a small, open source Python project of my own called cssify. You can find the cssify build results on Travis and even watch running jobs or videos of the browsers in action in the project’s Open Sauce page.

Steps involved:

  • Set up your tests to run on Travis CI
  • Configure Travis CI to start Sauce Connect
  • Configure your Selenium tests to run on Sauce Labs
  • Provide additional details to Sauce

What is Travis CI?

Travis CI logo
Travis CI is a hosted continuous integration service for the open source community. It is integrated with GitHub and offers first class support for:
C, C++, Clojure, Erlang, Go, Groovy, Haskell, Java, JavaScript (with Node.js), Perl, PHP, Python, Ruby and Scala.
Their CI environment provides multiple runtimes (e.g. Node.js or PHP versions), data stores and so on. Because of this, hosting your project on Travis means you can effortlessly test web applications against multiple runtimes and data stores without even having all of them installed locally.

Setup Your Tests to Run on Travis CI

There’s multiple tutorials out there to get started using Travis CI. One of my favorite things about Travis is how easy it is to configure and get started.
Check out the official Travis CI docs to get started in your own programming language.

For running your Selenium tests, remember to install the right dependencies, including the Selenium client bindings, like I did in my project. Also, make sure you make Travis start your webapp before the tests start. Your application needs to be serving pages to be able to test it using our browsers :)

Configure Travis CI to Start Sauce Connect

Sauce Connect creates a magical rainbow tunnel that will allow the Sauce browsers driven by your Selenium tests to access the Web Application that your Travis build starts. Once a Sauce Connect tunnel is running, browsers in our cloud can use it to hit localhost, internal IPs or even to just proxy the public internet through the right location.

I’ve created a small Bash script that takes care of downloading, starting and waiting for Connect to be ready. To use Sauce Connect in your build, all you need to do is curl and run this script in your .travis.yml before_script section as I did in my project:

  - curl | bash

Configure Your Selenium Tests to Run on Sauce Labs

Selenium LogoOnce your builds are happily running and Sauce Connect is started automatically by Travis, the next step is having your Selenium scripts run in the Sauce Labs cloud and hit the Application Under Test (AUT) through the tunnel.

To start running real browsers in the Sauce Cloud, make your scripts use Remote WebDriver and point the command executor to localhost:4445 (Sauce Connect will listen in that port and securely and efficiently relay to Sauce). You can find an example of this on the cssify codebase.

self.username = os.environ['SAUCE_USERNAME']
self.key = os.environ['SAUCE_ACCESS_KEY']
hub_url = "%s:%s@localhost:4445" % (self.username, self.key)
self.driver = webdriver.Remote(desired_capabilities=self.caps,
                               command_executor="http://%s/wd/hub" % hub_url)

Now for these browsers to be able to hit the AUT when opening localhost, you need to specify an additional Desired Capability, tunnel-identifier.
The value this capability should be set to is stored in the environment variable TRAVIS_JOB_NUMBER. You can find an example in the cssify codebase for this as well.

self.caps['tunnel-identifier'] = os.environ['TRAVIS_JOB_NUMBER']

Provide Additional Details to Sauce

In addition to running your tests in real browsers using the cloud, there’s additional details you can provide to speed up debugging and navigating test results.
In the cssify tests you can find some examples Desired Capabilities that Sauce uses and that you can pull from Travis CI’s environment variables.

self.caps['build'] = os.environ['TRAVIS_BUILD_NUMBER']
self.caps['tags'] = [os.environ['TRAVIS_PYTHON_VERSION'], 'CI']

That’s it!

You’re set! You are now running your tests using Travis CI, they start a Sauce tunnel on their own and drive real browsers in the cloud. Time to start thinking about running these in parallel to speed up your builds now!

Here are examples of what your Travis CI build should look and what you’ll see from the Sauce side.

Upcoming Selenium Events: Page Object Model, Optimizing Test Performance, and More

June 21st, 2012 by Ashley Wilson

June and July are busy months for the Selenium and Sauce community. Check out these meetups happening across the country (and world), and let us know if we’re missing anything. Hope to see you there!

June 21: PDX Selenium Meetup
Beyond Page Object Patterns
By Sam Woods, Software Development Engineer in Test at WebTrends
Learn about the evolution of automated UI tests from “scripts” to fully functional, maintainable, reliable abstracted automated test cases using the page object pattern.

June 27: Colombo Selenium Meetup
Selenium’s Journey
By Jason Huggins, Co-Founder of Sauce Labs
Get an introduction to Selenium’s colorful history in this virtual meetup with Jason Huggins, original creator of Selenium.

July 9th: Boston Selenium Meetup
July 11th: DC Selenium Meetup
Optimizing Selenium for Executed Performance
By Santiago Suarez Ordoñez, Selenium Ninja at Sauce Labs
This talk will cover test writing tips gathered over the years from helping hundreds of Selenium users and engineers make their tests run faster and more reliably, independently of the technology stack used. We’ll be talking about things such data independence, making atomic tests and generating application state.

July 10th: Continuous Delivery NYC Meetup
Continuous Delivery at New Relic
By: Bjorn Freeman-Benson, VP of Engineering at New Relic
Bjorn will talk about New Relic’s experiences in growing from a few developers supporting a few thousand users to 70+ developers delivering continuously to thousands of large and small commercial customers.

July 17th: San Jose Selenium Meetup
HARdy Har Har: How to generate HTTP Archive Files
By Adam Goucher, Selenium Expert and Consultant at Element 34
This talk will show you how to generate HTTP Archive (HAR) files in two different ways; using a custom Firefox WebDriver profile and with the BrowserMob Proxy.

July 18th: San Diego Web Performance Meetup
Capturing Performance Data from your Selenium Tests
By Patrick Lightbody, Director of Product Management at Neustar Enterprise Services
Learn from one of the original Selenium developers how you can use open source projects and new specifics, such as Navigation Timings, to augment your existing test suite and capture performance in your existing continuous integration environment.

July 18th: OC PHP Meetup
Selenium: Less Testing and More Coding
By Jonathan Lipps, Front-End Engineer at Sauce Labs
Jonathan will give a demo of a basic PHP web application and go through the motivations for using Selenium as a way to consistently and reliably test the application. He’ll then describe how Sauce Labs’s cloud service enables testing to transparently scale to multiple browsers and platforms and reduce several frustrations with running tests locally.

Javascript + Selenium: The Rockstar Combination of Testing

July 25th, 2011 by Ashley Wilson

For our July Selenium meetup, held last Thursday, we wanted to give attendees something a little different to chew on. Thanks to our good friends at Yammer, who co-hosted the event with us, we did so not only with delicious catered Mexican food, but also plenty of Javascript & Selenium testing goodness to go around.

Bob Remeika, senior engineer at Yammer, gave a spirited presentation that left no one questioning his stance on testing (his opening slide – “Test your shit” – really said it all). He gave us an inside look at how Yammer tests using a combination of Jellyfish and Sauce OnDemand, and gave some great advice on knowing what and how to test when you’re just starting out.


We also had Adam Christian, Sauce Labs’ Javascript Aficionado and the creator of Jellyfish, give two talks. The first, a lightning talk titled “Javascript Via Selenium: The Good, The Bad, The Obvious”, covered some of the lesser known things about Javascript testing via Selenium.

The second showed off how you can use Jellyfish, the open source Javascript runner that he announced a few weeks ago, to run your JS unit tests in any environment.

Thanks to Adam, Bob, and Yammer for making this quite the fun and memorable meetup. As always, the San Francisco Selenium Meetup group is free to join & we meet monthly at different venues around the Bay Area to talk all things testing. See you in August!

Sauce Builder Webinar: From zero to creating, storing and running automated tests in under 30 mins

May 3rd, 2011 by The Sauce Labs Team

As you may have heard, we recently released a new, free tool that makes building and running Selenium tests easier than before. Sauce Builder is now available for download, and with it comes the ability to create Selenium tests in your Firefox browser by simply clicking around your application. You can then run those tests in Sauce OnDemand with just a few clicks, no Selenium expertise necessary.

To explain more about Sauce Builder and how automated testing can accelerate your development velocity, we held a webinar with Adam Christian, a Sauce Labs developer and project lead for Sauce Builder.

With Sauce Builder, we’re hoping to simplify and improve the cross-browser testing process for development and QA teams here is an updated video that will demonstrate how to use

  • Build Selenium tests with zero programming – Simply click through your application and Sauce Builder writes a Selenium scripts that reflect your actions.
  • Export results in the language of your choice – HTML, JAVA, Groovy, C#, Perl, PHP, Python and Ruby so your tests speak the same language as your application and dev team.
  • Eliminate bugs faster – Use immediate video playback of your tests in action, and share them with your teammates.
  • Remove test infrastructure headaches – Sauce Builder makes it a snap to either run tests locally in Firefox or in the cloud with access to all the browser / OS combinations supported in the super scalable Sauce OnDemand service.

Watch an updated video presentation that will show you how to build a CI system using Selenium Builder, GitHub, Travis and Sauce Labs.


Sauce Labs Eliminates Barrier to Automated Cross-Browser Testing with Sauce Builder

March 31st, 2011 by The Sauce Labs Team

New tool enables QA pros to build and run Selenium tests without Selenium expertise or infrastructure

SAN FRANCISCO, CA — (Mar. 31, 2011) — Sauce Labs, the web application testing company, today introduced Sauce Builder, a free testing tool that makes it easy for users to build Selenium tests without Selenium expertise and run them with the Sauce OnDemand service. Sauce Builder allows users to build automated Selenium tests simply by clicking through an application. By eliminating the complexity of hand-coding Selenium scripts, Sauce Builder accelerates the adoption of automated testing for QA and development teams that have been craving the development productivity Selenium offers. Sauce Builder is free and available for immediate download.

“Automated testing has historically been one of the most complicated, yet most valuable, technologies for companies that build software. Automated testing is doubly challenging because teams need to build and maintain a testing environment and on top of that, building tests can require significant technical skill,” said John Dunham, CEO of Sauce Labs. “We launched our Sauce OnDemand cloud service last year to eliminate the headache of maintaining a test infrastructure. Now with Builder, we’ve removed the next barrier to the adoption of automated testing and we’re very excited to see how this combination can help QA and development teams achieve their goals.”

With Sauce Builder, Sauce Labs continues to simplify and improve the cross-browser testing process for development and QA teams. Sauce Builder’s benefits include:

  • Build Selenium tests with zero programming – Simply click through your application and Sauce Builder writes a Selenium scripts that reflect your actions
  • Export results in the language of your choice – HTML, Java, Groovy, C#, Perl, PHP, Python and Ruby so your tests speak the same language as your application and dev team
  • Eliminate bugs faster – Use immediate video playback of your tests in action, and share them with your teammates
  • Remove test infrastructure headaches – Sauce Builder makes it a snap to either run tests locally in Firefox or in the cloud with access to all the browser / operating system combinations supported in the super scalable Sauce OnDemand service

With over four million downloads in just four years, the Selenium project is the world’s most popular functional testing framework for web applications. Designed to further expand Selenium adoption, Sauce Builder is the first web-based Selenium tool of its kind, including technology Sauce Labs acquired from Go Test It in 2010. After becoming more familiar with the technology post-acquisition, Sauce Labs elected to open source the code under the name “Se Builder” earlier this year because the technology held so much promise for the Selenium community.

Sauce Builder expands the capabilities of Se Builder by enabling users to directly access Sauce OnDemand, the cloud-based Selenium service, to run their tests. Sauce OnDemand is free to try for up to 200 testing minutes every month. Sauce Labs is also leading a collaborative effort with the Selenium community to deliver a new generalized plug-in architecture for Se Builder that among other things will support integrated plug-ins for testing services like Sauce OnDemand.

To learn more about how automated testing can accelerate your development velocity, please join Adam Christian,a Sauce Labs developer and project lead for Sauce Builder, on Tuesday, April 19th at 10AM Pacific for the webinar, “From zero to creating, storing and running automated tests in under 30 minutes”.

“Debugging takes up valuable time that developers could be using to focus on their applications,” said Sauce Labs’ Christian. “Now with Sauce Builder, developers can leverage this great development environment through our cloud testing infrastructure and not worry about dealing with building or maintaining their own costly testing infrastructure.”
About Sauce Labs
Sauce Labs, web application testing company, provides Sauce OnDemand, a cloud based service that allows users to run automated cross-browser functional tests faster and eliminating the need to maintain their own test infrastructure. To date, over four million Sauce OnDemand tests have been run in the Sauce cloud. The lead investor of Sauce Labs is the Contrarian Group, Peter Ueberroth’s investment management firm. Sauce Labs is headquartered in San Francisco, California. For more information, visit

Media Contact
Chantal Yang
LEWIS Pulse for Sauce Labs