Updated Jenkins CI Plugin for Sauce

July 29th, 2015 by Ken Drachnik

Because enterprises are under extreme internal and external pressure to deliver better software, faster, development teams are transitioning to new software delivery processes such as Continuous Integration (CI) and Continuous Delivery (CD). Both practices have proven their value in accelerating software production beyond traditional approaches, yet achieving continuous testing to support these processes is often the most difficult hurdle. Our updated Jenkins Plugin offers developers the combined power of the Sauce platform with Jenkins CI, the world’s leading CI tool, to further simplify and accelerate the development process.

The latest release of the Jenkins CI Plugin offers a set of rich features to help developers better utilize their CI systems to increase developer productivity so they can release better software, faster.

Here’s a summary of what’s new:

Updated UI
The latest release of the Sauce OnDemand Jenkins plugin includes an updated UI with features to make it easier to run tests faster in parallel and understand the results.

Updated Browser Selection Tool
This makes it easier to speed testing by running tests in parallel. The new UI makes it very simple to select multiple platforms and browsers at one time.

Enhance Reporting
Reporting enhancements include test job details so that users get more detailed information about their test results on the Jenkins build page. Users now can view a detailed list of Sauce jobs in Jenkins by name, OS, browser and version, pass/fail status, link to the test video and log.

Latest Version of Sauce Connect
Includes the latest version of Sauce Connect v4.3.9 (Sauce Labs’ secure tunnel) that gives users the latest security enhancements when running tests on applications located behind their firewall.

Automated Support Log Generation
Users can now create a zip file that contains the Sauce Connect log and Jenkins build output that makes support and debugging tests easier.

Updated Jenkins Build Messaging
Includes log information to indicate the status of the job such as when it has started, stopped and finished processing. This gives you more detailed information about the status of the job so you can debug more effectively.

The updated plug in is available today.
Download the new plugin
Download from the Jenkins Plugin Marketplace:
Read the docs:

Windows 10 Support

July 28th, 2015 by Ken Drachnik

Windows 10 150 pxWindows 10 is scheduled to launch tomorrow so today Sauce Labs is pleased to announce support for manual and automated Windows 10 testing – get your Win10 tests set up with our Automated Test Configurator.

We often release new OS / browser combinations ahead of their scheduled launch so you can begin testing to make sure your apps work on the shiny new systems ahead of time. Unfortunately, this release was a bit different as we did not get access to the bits we needed until later in the cycle.

Happy Testing!

[Webinar] What DevOps Is – and Why You Should Care

July 8th, 2015 by Bill McGee

DevOps has become the newest buzzword. You’ve probably seen or heard the term DevOps once, twice, a thousand times. But even the biggest supporters of DevOps will admit that the concept is creating as much noise and confusion as converts.

The practice of DevOps is not new, yet in the last two years it has seemingly dominated chatter within software development. But what does DevOps really mean? And what is the impact of DevOps on QA teams, if any at all?

Join us as DevOps Analyst Chris Riley shares the meaning and history of DevOps, his perspective on the movement, and his ideas about its future. He will share the knowledge that he has gathered from tools vendors and practitioners, all to help you navigate the sea of DevOps conversations to maximize the movement to your advantage.

This webinar will cover:

  • The difference between the practice of DevOps and the movement
  • What the future of DevOps holds
  • The intersection of DevOps and QA

Join us for this presentation next Tuesday, July 14 at 11am PDT/2pm EDT. There will be a Q&A with Chris afterwards.

Click HERE to register today.

Want to read more about DevOps and Continuous Integration? Download this free GigaOm white paper.

Appium + Sauce Labs Bootcamp: Chapter 2, Touch Actions

June 15th, 2015 by Isaac Murchie

One aspect of mobile devices that needs to be automated in order to fully test applications, whether native, hybrid, or web, is utilizing gestures to interact with elements. In Appium this is done through the Touch Action and Multi Touch APIs. These two APIs come from an early draft of the WebDriver W3C Specification, and are an attempt to atomize the individual actions that make up complex actions. That is to say, it provides the building blocks for any particular gesture that might be of interest.

The specification has changed recently and the current implementation will be deprecated in favor of an implementation of the latest specification. That said, the following API will remain for some time within Appium, even as the new API is rapidly adopted in the server.

Touch Actions

The Touch Action API provides the basis of all gestures that can be automated in Appium. At its core is the ability to chain together _ad hoc_ individual actions, which will then be applied to an element in the application on the device. The basic actions that can be used are:

  • press
  • longPress
  • tap
  • moveTo
  • wait
  • release
  • cancel
  • perform

Of these, the last deserves special mention. The action perform actually sends the chain of actions to the server. Before calling perform, the client is simply recording the actions in a local data structure, but nothing is done to the application under test. Once perform is called, the actions are wrapped up in JSON and sent to the server where they are actually performed! (more…)

More VMs for Open Sauce!

June 2nd, 2015 by Ken Drachnik

We love open source. As open source advocates and contributors ourselves, we know it’s important to support projects that we benefit from on a regular basis. And what better way to do it than by providing the infrastructure that helps ensure new releases are fully tested?

We have long teamed up with Travis-CI to provide a seamless experience across our platforms. However there was a disconnect in that the # of VMs each platform could support was not the same – Travis supported 5 instances and Sauce just 3. Well we just fixed that and have increased the number of VMs that you can use in Open Sauce to 5. This will allow users of Travis and Sauce to maximize their test concurrencies and complete tests faster so you can speed your apps to market.  Not using Travis CI? Don’t worry, the 5 VMs still apply.

Learn more about Open Sauce.

Learn more about Travis CI

Sauce Labs Appoints Technology Veteran Charles Ramsey as Company’s First Chief Revenue Officer

February 24th, 2015 by Amber Kaplan

SAN FRANCISCO, CA-  Sauce Labs, Inc., the leading cloud-based web and mobile application testing platform, today announced that it has appointed Charles Ramsey as the company’s first Chief Revenue Officer (CRO). Ramsey brings more than 25 years of industry experience and insight to his role at Sauce Labs. He will be responsible for all customer-facing areas, including sales, business development, and marketing, as well as continuing to build on Sauce Labs record 2014 results as the company extends its leadership position in the booming automated testing market.

“Charles is an innovative strategist, committed to building strong relationships with customers and partners,” said Jim Cerna, CEO of Sauce Labs. “His demonstrated ability to strategically grow companies will help us address the exploding demand we are seeing in the market for our technology. With Charles’ experience, leadership, and track record of repeated successes, we are poised to continue our rapid growth trajectory through 2015 and beyond.”

“Sauce Labs is well-positioned to take advantage of the dramatic proliferation of web and mobile apps across a variety of devices and operating systems. I look forward to working with the entire Sauce team to further the company’s market leadership and growth by bringing new levels of innovation, customer experience, and value to the marketplace,” said Ramsey.

Sauce Labs provides an instantly scalable testing cloud that is optimized for continuous integration (CI) and continuous delivery (CD). When tests are automated and run in parallel on virtual machines across multiple browsers and platforms, testing time is reduced and developer time is freed up from managing infrastructure. When paired with a CI system, developers can easily test web, hybrid and native applications early on in their development cycles, continuously and affordably. Sauce Labs currently supports more than 480 browser, operating system and device platform combinations.

Prior to joining Sauce Labs, Ramsey was an early member of the Quest Software management team, where he served as vice president of World Wide Marketing and Sales. He is a former Venture Partner at JMI Equity and has also served on the Board of Directors at notable companies such as Configuresoft, Inc. and ServiceNow, Inc. Early in his career, Ramsey rose to vice president, North America Sales for Computer Intelligence, a division of Ziff Davis, after beginning his career with the IBM National Accounts Division in a variety of sales assignments. He has a Bachelor of Arts from the University of California, San Diego and a Master of Information Management from the American Graduate School of International Management.

Helpful Links

About Sauce Labs
Sauce Labs is the leading cloud-based web and mobile application automated testing platform. Its secure and reliable testing infrastructure enables users to run JavaScript unit and functional tests written with Selenium and Appium, eliminating the time and expense of maintaining a test grid. With Sauce Labs, organizations can achieve success with continuous integration and delivery, increase developer productivity and reduce infrastructure costs for software teams of all sizes.

Sauce Labs is a privately-held company funded by Toba Capital, Salesforce Ventures, Triage Ventures and the Contrarian Group. For more information, please visit http://saucelabs.com.

Cross-Browser Layout Testing With Galen Framework And Sauce Labs

January 8th, 2015 by Amber Kaplan

Every web developer faces this problem once the website is implemented: cross-browser layout testing. This is a very annoying task as you have to look at your website on a lot of different browsers. We used to perform a manual check on all these browsers at the end of website development, and as a result we get some layout issues and have to find hacks to fix these deffects. But what if we could automate layout testing of the website and perform it always in all major browsers? Imagine you follow a TDD technique in your front-end development and you want to have an early feedback on layout issues. To solve this problem you could have a look at Galen Framework.

In short Galen Framework is a layout and functional testing framework that has its own language for describing the layout. The way you test layout using this approach is by declaring a set of elements on the page and rules describing where each element is expected to be. The elements are being tested relatively to each other. This way you can make your tests very flexible and run them on different browsers and even mobile devices in case your website is responsive.

In this article I would like to show you how to write a basic layout test and run it in the SauceLabs cloud. SauceLabs offers various platforms and browsers and even allows you to choose browser versions. Lets take the major browsers like: Internet Explorer 11, Firefox, Chrome, Opera and Safari on Mac. The website under test will be http://testapp.galenframework.com.

(more…)

Announcing Support for Mac OS X Yosemite

November 7th, 2014 by lauren nguyen

selenium testingWe’ve just released support for Mac OS X Yosemite on the Sauce cloud. Browsers supported include the latest two versions of Firefox and Chrome, Safari 8.0, and iOS 7.0, 7.1, and 8.0. Yosemite sees Apple bringing OS X closer to iOS, with features like Handoff to switch between devices and the ability to initiate Instant Hotspots from your iPhone. As always, we’ll continue to add platform support so you can ensure your app works for all your users. (more…)

Open Source Stories: The Selenium Project & Sauce

March 21st, 2014 by lauren nguyen

selenium testing & sauceWe LOVE open source. So much so that we created Open Sauce, to give open source projects the ability to test their projects for free on our cloud. And in just a year, we now have over 800 OSS projects testing on Sauce! To celebrate, we’ll be exploring some of the different projects test on Sauce and what makes them awesome.

Naturally, the first project we wanted to cover was The Selenium Project. You know you can run your Selenium tests on the Sauce cloud, but beyond that, the Selenium Project and Sauce have a long history of working together, and they were one of the first OSS projects to ever test on Sauce. So where did it all begin?

The history

Years ago, the Selenium Project was using Bamboo for CI on one machine, with a VM for Windows and a VM for Mac. The project wanted to move toward releasing weekly updates to Selenium. For them, it meant that they could be more iterative and only needed to fix a week’s worth of problems at a time, rather than months worth of issues. But before being able to release, the project had to vet each release for different browser/OS versions. But maintaining all the supported browser/OS versions for testing was a pain. A classic problem if there ever was one! The project also wanted to get a better CI process into place and integrate reliable tests into their process.

They began by switching from Bamboo to Jenkins with Google Compute Engine to see what they could get running. Then, they began to work to get their tests running on Sauce. On the Sauce side, Santi made some modifications to the Sauce cloud that would allow the Selenium Project to test a custom version of Selenium, since they were testing unreleased versions. The project modified their tests to be able to run in parallel. And, with that, the Selenium Project had instant access to all the browser/OS  versions they needed, whenever they needed them. Magic!

Hooked on Sauce

Fast forward to today, and the Selenium Project now tests about 30 different browser/OS configurations in parallel with every commit. According to the project, testing that took a week now takes an hour, and weekly releases have become easy. They also run JS unit tests on Sauce, and have much less manual testing to do now that much of the testing is automated. The ability to consistently run tests is the greatest benefit Sauce has given the project, as well as having access to log files and videos that are shareable to different contributors. As with most people, they deal with some flaky tests, but rarely have a Sauce-related issue.

Selenium wisdom

Even if you’re the Selenium Project, you can almost always improve the way you’re testing, so we asked the project to give us tips on automated testing. Here’s what they had to say:

  • Try to keep your testing hermetic – for example, have a bunch of test pages, and serve them individually for every test run that happens
  • Don’t rely on other services for your tests. The Facebook API or Twitter API may be down, which can cause unnecessary failures and cause you to lose the connection between the code you’re testing and what you see.
  • Be smart and think through the browsers you test on. Have the right conversations about what needs to be tested.
  • Don’t test on every browser, only the necessary ones. The Selenium Project only tests on 1 version of Chrome; running in 10 different versions is wasteful. Tests will always have some amount of flakiness, and if you test unneeded browsers you might get a lot more failures.
  • Do test functionally different browser versions. The project tests both IE 8 and 9 because there are major differences between them.

Finally, try not to need Selenium tests. Sound crazy coming from the Selenium Project, doesn’t it? But the key here is that the more you try to be efficient, economical, and thoughtful about your tests, the better off your remaining tests will be. You’ll probably never be able to not test at all, but it’s a good exercise in restraint. Make sure there’s good reasoning behind every test in your suite. Otherwise, it’s easy for your test suite to balloon until its unmanageable.

Want to get involved with the Selenium Project?

According to the project, the best way to get involved is to find something you want to fix. There are hundreds of open issues, find one that gets you fired up. Another great way to get involved is to look through the wiki and how-to-build code and write a small failing unit test for the thing you want to improve. That’ll motivate people to get involved. Finally, hang out in the IRC channel: #selenium.

What’s next for the project?

There are some great things for the Selenium Project on the horizon, including the release of Selenium 3, RCA APIs being ripped out, and a W3C spec. Learn more on Selenium HQ.

JavaScript Unit Testing API Revamped

February 19th, 2014 by Jonah Stiennon

Sauce provides a shortcut API for running Javascript unit tests (like Jasmine, Qunit, Mocha, and YUI Test) on all browsers using our cloud.

The old way of doing things:

Before the unit test API was added, running frontend javascript tests using Selenium was pretty messy. One had to point the Selenium browser at the page which ran/reported the tests then inspect DOM elements on the page looking for the test results.

DOM unit testing

See how we’re getting the number of passing tests from the element with id=”mocha-stats”?

This was pretty dependent on the styling of the page, and can be an intensive amount of logic to put into a Selenium test, especially if you want to parse all the individual assertions.

The new way of doing things:

Now you can let Sauce Labs take care of all the tedium!

Instead of setting up a Webdriver and sending Selenium commands to our servers, just fire off a single HTTP request:

curl -X POST https://saucelabs.com/rest/v1/$SAUCE_USERNAME/js-tests         
     -u $SAUCE_USERNAME:$SAUCE_ACCESS_KEY -H 'Content-Type: application/json'
     --data '{
        "platforms": [["Windows 7", "firefox", "20"],
                      ["Linux", "googlechrome", ""]],        
        "url": "https://saucelabs.com/test_helpers/front_tests/index.html",
        "framework": "jasmine"}'

The Sauce servers point a browser at the test page and get the results. We parse the results depending on the framework you’re using and display them in a friendly manner on the Job Details page.

Failing Mocha and Qunit test reports on Sauce Labs

Failing Mocha and Qunit test reports on Sauce Labs

We now report the specific tests which fail, no more hunting through screenshots/videos.

The bad news:

Sauce doesn’t inspect DOM elements to get your test results, it’s much more robust. Buuuuuuut, it relies on you making the test results available in the global scope of the Javascript on the page. Once you add the code appropriate for your framework, our servers gather the data, parse it, and display it.

An extra feature we get from this is support for an arbitrary “custom” unit test report. If you set `window.global_test_restults` to an object that looks like this:

{
  "passed": 4,
  "failed": 0,
  "total": 4,
  "duration": 4321,
  "tests": [
    {
      "name": "foo test",
      "result": true,
      "message": "so foo",
      "duration": 4000
    },
    {
      "name": "bar test",
      "result": true,
      "message": "passed the bar exam",
      "duration": 300
    },
    {
      "name": "baz test",
      "result": true,
      "message": "passed",
      "duration": 20
    },
    {
      "name": "qux test",
      "result": true,
      "message": "past",
      "duration": 1
    }
  ]
}

We’ll display the results and report the test status automatically.

customUnitTest

Enjoy the new reporting, if this gets enough use we can expand support to more frameworks and see if we can inject the reporting code into test pages when we test them, lessening the work for the developer.