[Webinar] What DevOps Is – and Why You Should Care

July 8th, 2015 by Bill McGee

DevOps has become the newest buzzword. You’ve probably seen or heard the term DevOps once, twice, a thousand times. But even the biggest supporters of DevOps will admit that the concept is creating as much noise and confusion as converts.

The practice of DevOps is not new, yet in the last two years it has seemingly dominated chatter within software development. But what does DevOps really mean? And what is the impact of DevOps on QA teams, if any at all?

Join us as DevOps Analyst Chris Riley shares the meaning and history of DevOps, his perspective on the movement, and his ideas about its future. He will share the knowledge that he has gathered from tools vendors and practitioners, all to help you navigate the sea of DevOps conversations to maximize the movement to your advantage.

This webinar will cover:

  • The difference between the practice of DevOps and the movement
  • What the future of DevOps holds
  • The intersection of DevOps and QA

Join us for this presentation next Tuesday, July 14 at 11am PDT/2pm EDT. There will be a Q&A with Chris afterwards.

Click HERE to register today.

Want to read more about DevOps and Continuous Integration? Download this free GigaOm white paper.

Appium + Sauce Labs Bootcamp: Chapter 2, Touch Actions

June 15th, 2015 by Isaac Murchie

One aspect of mobile devices that needs to be automated in order to fully test applications, whether native, hybrid, or web, is utilizing gestures to interact with elements. In Appium this is done through the Touch Action and Multi Touch APIs. These two APIs come from an early draft of the WebDriver W3C Specification, and are an attempt to atomize the individual actions that make up complex actions. That is to say, it provides the building blocks for any particular gesture that might be of interest.

The specification has changed recently and the current implementation will be deprecated in favor of an implementation of the latest specification. That said, the following API will remain for some time within Appium, even as the new API is rapidly adopted in the server.

Touch Actions

The Touch Action API provides the basis of all gestures that can be automated in Appium. At its core is the ability to chain together _ad hoc_ individual actions, which will then be applied to an element in the application on the device. The basic actions that can be used are:

  • press
  • longPress
  • tap
  • moveTo
  • wait
  • release
  • cancel
  • perform

Of these, the last deserves special mention. The action perform actually sends the chain of actions to the server. Before calling perform, the client is simply recording the actions in a local data structure, but nothing is done to the application under test. Once perform is called, the actions are wrapped up in JSON and sent to the server where they are actually performed! (more…)

More VMs for Open Sauce!

June 2nd, 2015 by Ken Drachnik

We love open source. As open source advocates and contributors ourselves, we know it’s important to support projects that we benefit from on a regular basis. And what better way to do it than by providing the infrastructure that helps ensure new releases are fully tested?

We have long teamed up with Travis-CI to provide a seamless experience across our platforms. However there was a disconnect in that the # of VMs each platform could support was not the same – Travis supported 5 instances and Sauce just 3. Well we just fixed that and have increased the number of VMs that you can use in Open Sauce to 5. This will allow users of Travis and Sauce to maximize their test concurrencies and complete tests faster so you can speed your apps to market.  Not using Travis CI? Don’t worry, the 5 VMs still apply.

Learn more about Open Sauce.

Learn more about Travis CI

Sauce Labs Appoints Technology Veteran Charles Ramsey as Company’s First Chief Revenue Officer

February 24th, 2015 by Amber Kaplan

SAN FRANCISCO, CA-  Sauce Labs, Inc., the leading cloud-based web and mobile application testing platform, today announced that it has appointed Charles Ramsey as the company’s first Chief Revenue Officer (CRO). Ramsey brings more than 25 years of industry experience and insight to his role at Sauce Labs. He will be responsible for all customer-facing areas, including sales, business development, and marketing, as well as continuing to build on Sauce Labs record 2014 results as the company extends its leadership position in the booming automated testing market.

“Charles is an innovative strategist, committed to building strong relationships with customers and partners,” said Jim Cerna, CEO of Sauce Labs. “His demonstrated ability to strategically grow companies will help us address the exploding demand we are seeing in the market for our technology. With Charles’ experience, leadership, and track record of repeated successes, we are poised to continue our rapid growth trajectory through 2015 and beyond.”

“Sauce Labs is well-positioned to take advantage of the dramatic proliferation of web and mobile apps across a variety of devices and operating systems. I look forward to working with the entire Sauce team to further the company’s market leadership and growth by bringing new levels of innovation, customer experience, and value to the marketplace,” said Ramsey.

Sauce Labs provides an instantly scalable testing cloud that is optimized for continuous integration (CI) and continuous delivery (CD). When tests are automated and run in parallel on virtual machines across multiple browsers and platforms, testing time is reduced and developer time is freed up from managing infrastructure. When paired with a CI system, developers can easily test web, hybrid and native applications early on in their development cycles, continuously and affordably. Sauce Labs currently supports more than 480 browser, operating system and device platform combinations.

Prior to joining Sauce Labs, Ramsey was an early member of the Quest Software management team, where he served as vice president of World Wide Marketing and Sales. He is a former Venture Partner at JMI Equity and has also served on the Board of Directors at notable companies such as Configuresoft, Inc. and ServiceNow, Inc. Early in his career, Ramsey rose to vice president, North America Sales for Computer Intelligence, a division of Ziff Davis, after beginning his career with the IBM National Accounts Division in a variety of sales assignments. He has a Bachelor of Arts from the University of California, San Diego and a Master of Information Management from the American Graduate School of International Management.

Helpful Links

About Sauce Labs
Sauce Labs is the leading cloud-based web and mobile application automated testing platform. Its secure and reliable testing infrastructure enables users to run JavaScript unit and functional tests written with Selenium and Appium, eliminating the time and expense of maintaining a test grid. With Sauce Labs, organizations can achieve success with continuous integration and delivery, increase developer productivity and reduce infrastructure costs for software teams of all sizes.

Sauce Labs is a privately-held company funded by Toba Capital, Salesforce Ventures, Triage Ventures and the Contrarian Group. For more information, please visit http://saucelabs.com.

Cross-Browser Layout Testing With Galen Framework And Sauce Labs

January 8th, 2015 by Amber Kaplan

Every web developer faces this problem once the website is implemented: cross-browser layout testing. This is a very annoying task as you have to look at your website on a lot of different browsers. We used to perform a manual check on all these browsers at the end of website development, and as a result we get some layout issues and have to find hacks to fix these deffects. But what if we could automate layout testing of the website and perform it always in all major browsers? Imagine you follow a TDD technique in your front-end development and you want to have an early feedback on layout issues. To solve this problem you could have a look at Galen Framework.

In short Galen Framework is a layout and functional testing framework that has its own language for describing the layout. The way you test layout using this approach is by declaring a set of elements on the page and rules describing where each element is expected to be. The elements are being tested relatively to each other. This way you can make your tests very flexible and run them on different browsers and even mobile devices in case your website is responsive.

In this article I would like to show you how to write a basic layout test and run it in the SauceLabs cloud. SauceLabs offers various platforms and browsers and even allows you to choose browser versions. Lets take the major browsers like: Internet Explorer 11, Firefox, Chrome, Opera and Safari on Mac. The website under test will be http://testapp.galenframework.com.

(more…)

Announcing Support for Mac OS X Yosemite

November 7th, 2014 by lauren nguyen

selenium testingWe’ve just released support for Mac OS X Yosemite on the Sauce cloud. Browsers supported include the latest two versions of Firefox and Chrome, Safari 8.0, and iOS 7.0, 7.1, and 8.0. Yosemite sees Apple bringing OS X closer to iOS, with features like Handoff to switch between devices and the ability to initiate Instant Hotspots from your iPhone. As always, we’ll continue to add platform support so you can ensure your app works for all your users. (more…)

Open Source Stories: The Selenium Project & Sauce

March 21st, 2014 by lauren nguyen

selenium testing & sauceWe LOVE open source. So much so that we created Open Sauce, to give open source projects the ability to test their projects for free on our cloud. And in just a year, we now have over 800 OSS projects testing on Sauce! To celebrate, we’ll be exploring some of the different projects test on Sauce and what makes them awesome.

Naturally, the first project we wanted to cover was The Selenium Project. You know you can run your Selenium tests on the Sauce cloud, but beyond that, the Selenium Project and Sauce have a long history of working together, and they were one of the first OSS projects to ever test on Sauce. So where did it all begin?

The history

Years ago, the Selenium Project was using Bamboo for CI on one machine, with a VM for Windows and a VM for Mac. The project wanted to move toward releasing weekly updates to Selenium. For them, it meant that they could be more iterative and only needed to fix a week’s worth of problems at a time, rather than months worth of issues. But before being able to release, the project had to vet each release for different browser/OS versions. But maintaining all the supported browser/OS versions for testing was a pain. A classic problem if there ever was one! The project also wanted to get a better CI process into place and integrate reliable tests into their process.

They began by switching from Bamboo to Jenkins with Google Compute Engine to see what they could get running. Then, they began to work to get their tests running on Sauce. On the Sauce side, Santi made some modifications to the Sauce cloud that would allow the Selenium Project to test a custom version of Selenium, since they were testing unreleased versions. The project modified their tests to be able to run in parallel. And, with that, the Selenium Project had instant access to all the browser/OS  versions they needed, whenever they needed them. Magic!

Hooked on Sauce

Fast forward to today, and the Selenium Project now tests about 30 different browser/OS configurations in parallel with every commit. According to the project, testing that took a week now takes an hour, and weekly releases have become easy. They also run JS unit tests on Sauce, and have much less manual testing to do now that much of the testing is automated. The ability to consistently run tests is the greatest benefit Sauce has given the project, as well as having access to log files and videos that are shareable to different contributors. As with most people, they deal with some flaky tests, but rarely have a Sauce-related issue.

Selenium wisdom

Even if you’re the Selenium Project, you can almost always improve the way you’re testing, so we asked the project to give us tips on automated testing. Here’s what they had to say:

  • Try to keep your testing hermetic – for example, have a bunch of test pages, and serve them individually for every test run that happens
  • Don’t rely on other services for your tests. The Facebook API or Twitter API may be down, which can cause unnecessary failures and cause you to lose the connection between the code you’re testing and what you see.
  • Be smart and think through the browsers you test on. Have the right conversations about what needs to be tested.
  • Don’t test on every browser, only the necessary ones. The Selenium Project only tests on 1 version of Chrome; running in 10 different versions is wasteful. Tests will always have some amount of flakiness, and if you test unneeded browsers you might get a lot more failures.
  • Do test functionally different browser versions. The project tests both IE 8 and 9 because there are major differences between them.

Finally, try not to need Selenium tests. Sound crazy coming from the Selenium Project, doesn’t it? But the key here is that the more you try to be efficient, economical, and thoughtful about your tests, the better off your remaining tests will be. You’ll probably never be able to not test at all, but it’s a good exercise in restraint. Make sure there’s good reasoning behind every test in your suite. Otherwise, it’s easy for your test suite to balloon until its unmanageable.

Want to get involved with the Selenium Project?

According to the project, the best way to get involved is to find something you want to fix. There are hundreds of open issues, find one that gets you fired up. Another great way to get involved is to look through the wiki and how-to-build code and write a small failing unit test for the thing you want to improve. That’ll motivate people to get involved. Finally, hang out in the IRC channel: #selenium.

What’s next for the project?

There are some great things for the Selenium Project on the horizon, including the release of Selenium 3, RCA APIs being ripped out, and a W3C spec. Learn more on Selenium HQ.

JavaScript Unit Testing API Revamped

February 19th, 2014 by Jonah Stiennon

Sauce provides a shortcut API for running Javascript unit tests (like Jasmine, Qunit, Mocha, and YUI Test) on all browsers using our cloud.

The old way of doing things:

Before the unit test API was added, running frontend javascript tests using Selenium was pretty messy. One had to point the Selenium browser at the page which ran/reported the tests then inspect DOM elements on the page looking for the test results.

DOM unit testing

See how we’re getting the number of passing tests from the element with id=”mocha-stats”?

This was pretty dependent on the styling of the page, and can be an intensive amount of logic to put into a Selenium test, especially if you want to parse all the individual assertions.

The new way of doing things:

Now you can let Sauce Labs take care of all the tedium!

Instead of setting up a Webdriver and sending Selenium commands to our servers, just fire off a single HTTP request:

curl -X POST https://saucelabs.com/rest/v1/$SAUCE_USERNAME/js-tests         
     -u $SAUCE_USERNAME:$SAUCE_ACCESS_KEY -H 'Content-Type: application/json'
     --data '{
        "platforms": [["Windows 7", "firefox", "20"],
                      ["Linux", "googlechrome", ""]],        
        "url": "https://saucelabs.com/test_helpers/front_tests/index.html",
        "framework": "jasmine"}'

The Sauce servers point a browser at the test page and get the results. We parse the results depending on the framework you’re using and display them in a friendly manner on the Job Details page.

Failing Mocha and Qunit test reports on Sauce Labs

Failing Mocha and Qunit test reports on Sauce Labs

We now report the specific tests which fail, no more hunting through screenshots/videos.

The bad news:

Sauce doesn’t inspect DOM elements to get your test results, it’s much more robust. Buuuuuuut, it relies on you making the test results available in the global scope of the Javascript on the page. Once you add the code appropriate for your framework, our servers gather the data, parse it, and display it.

An extra feature we get from this is support for an arbitrary “custom” unit test report. If you set `window.global_test_restults` to an object that looks like this:

{
  "passed": 4,
  "failed": 0,
  "total": 4,
  "duration": 4321,
  "tests": [
    {
      "name": "foo test",
      "result": true,
      "message": "so foo",
      "duration": 4000
    },
    {
      "name": "bar test",
      "result": true,
      "message": "passed the bar exam",
      "duration": 300
    },
    {
      "name": "baz test",
      "result": true,
      "message": "passed",
      "duration": 20
    },
    {
      "name": "qux test",
      "result": true,
      "message": "past",
      "duration": 1
    }
  ]
}

We’ll display the results and report the test status automatically.

customUnitTest

Enjoy the new reporting, if this gets enough use we can expand support to more frameworks and see if we can inject the reporting code into test pages when we test them, lessening the work for the developer.

Remote file uploads with Selenium & PHP

January 8th, 2014 by Isaac Murchie

When testing file uploading functionality we have to consider two scenarios… running browsers locally and running them remotely. Locally there is no problem: test browsers have access to the files the tests specify. When running the same tests remotely, with the browser running on a different machine than the test programs, we run into a significant problem: the files we specify to upload are not available to the browser to upload!

So, simply sending the name of the file to upload will not work:

    public function testFileUpload() {
        $filebox = $this->byName('myfile');
        $this->sendKeys($filebox, "./kirk.jpg");
        $this->byId('submit')->submit();

        $this->assertTextPresent("kirk.jpg (image/jpeg)");
    }

The remote browser will not be able to do whatever it does with the file, in this case printing out the file name and content type.

Selenium 2 solves this problem by providing a method to upload the file to the server, and then giving the page the remote path the the file, and it does this transparently. For local tests you just use the sendKeys method to enter text into the file upload form element. For remote tests you do the same, but while setting up the tests you call the fileDetector method in order to let the WebDriver system know that a local file is being added rather than just the name of the file.

While all the Selenium WebDriver language bindings developed by the Selenium project have this functionality exposed (on Java and Ruby, see this tutorial, and for more information on Ruby, see this discussion), the PHP bindings do not. In order to facilitate remote testing in the Sauce Labs Selenium 2 Cloud, this functionality has been added to Sausage, allowing you to use PHPUnit to run your tests both locally and in the Sauce Cloud.

The crux of the solution is to give the system a way to know that a file might be uploaded, and how to discern this, by passing it a function that will be called when trying to send keys to form elements. This function takes a string and returns something truthy if it should be interpreted as the name of a file, however that is to be determined on your system. The most basic form, obviously, simply tests for the existence of the string as a file on the local file system. This function is then sent to the test library through a call to fileDetector:

    $this->fileDetector(function($filename) {
        if(file_exists($filename)) {
            return $filename;
        } else {
            return NULL;
        }
    });

If, when sending a string to a form element, the supplied function returns true, the system will, before changing the value of the function, transparently read the file, encode it in Base64, and send it to the Selenium server. It then puts the remote path, rather than the local one, into the file upload form element. This ensures the file contents to be available to the remote browser just as if it were running locally, allowing tests to proceed without trouble!

A full example (also available on GitHub):

  <?php
  require_once 'vendor/autoload.php';

  class WebDriverDemo extends Sauce\Sausage\WebDriverTestCase {
      public static $browsers = array(
          // run FF15 on Windows 8 on Sauce
          array(
              'browserName' => 'firefox',
              'desiredCapabilities' => array(
                  'version' => '15',
                  'platform' => 'Windows 2012',
              )
          )
      );

      public function setUpPage() {
          $this->url("http://sl-test.herokuapp.com/guinea_pig/file_upload");

          // set the method which knows if this is a file we're trying to upload
          $this->fileDetector(function($filename) {
              if(file_exists($filename)) {
                  return $filename;
              } else {
                  return NULL;
              }
          });
      }

      public function testFileUpload() {
          $filebox = $this->byName('myfile');
          $this->sendKeys($filebox, "./kirk.jpg");
          $this->byId('submit')->submit();

          $this->assertTextPresent("kirk.jpg (image/jpeg)");
      }
  }

Online Workshop – Learn How Gilt Groupe Uses Appium to Automate Their Mobile Testing

December 19th, 2013 by Bill McGee

1/14/2014 Update: We’ll be posting a link to the recorded webinar in several days.

We’re teaming up with the Gilt Groupe for our next online workshop! Join us on Tuesday, January 14th, 2014 at 11:00am PST to learn about how Gilt uses Appium to automate their mobile testing. Today, mobile purchases make up more than 40% of Gilt’s sales, and the mobile e-commerce market continues to rage – in the U.S. alone, mobile commerce will grow an estimated 63% to around $34.2 billion, up from $21 billion in 2012.

Find out how Gilt has streamlined testing of this vital part of their business, and what they are doing today on the testing front.

In the workshop, Matt Isaacs, Engineer at Gilt Groupe will cover how mobile testing at Gilt has changed since the early days of Gilt mobile. Then Mike Redman, Director of Sales Engineering at Sauce, will explain how to run your Appium tests on the Sauce cloud. This will be followed by a live Q&A. Register today!

Gilt-Man-smaller