Sauce Updates OnDemand Plugin for Jenkins Pipeline

August 10th, 2016 by Ken Drachnik

We have updated our OnDemand plugin for Jenkins with explicit support for Jenkins Pipeline. This update enables development and testing teams to easily create, manage, and run automated tests using scripts on Jenkins, resulting in faster testing.

The newly enhanced OnDemand plugin enables Sauce Labs users to skip the Jenkins UI and run tests directly via scripts. Previously, users had to work within the Jenkins UI to create testing tasks. While this is initially convenient, as automation grows and becomes more complex, the process of creating testing tasks becomes more complex, too. Now OnDemand enables testers to automate releases for complex and non-sequential tasks that previously required manual intervention.

Key benefits of the update include:

Create scripts to automate Jenkins operations
Save time by scripting test sequences that can automate testing, retry tests automatically and run processes in parallel. Developers and QA can more easily customize their Jenkins workflows and reporting results.

Automatically create test scripts for Jenkins
Use the Jenkins snippet generator to easily create Pipeline scripts to execute Sauce Commands via Jenkins. Users don’t need to know how to script in Pipeline – simply copy the code snippets to program workflows

The Sauce OnDemand Jenkins Plugin is available as a free integration. Download the plugin from Jenkins and read how to configure it on our docs page.

Happy Testing!

Appium 1.5.3 Released on Sauce

June 8th, 2016 by Isaac Murchie

We are pleased to announce the release of Appium 1.5.3 through npm and on Sauce Labs!

Appium 1.5.3 is primarily a bug-fix release. However, it also adds functionality to support for iOS WKWebViews and for automating the iOS Calendar app, as well as adding a capability to use abd for screenshots in Android web/hybrid sessions, and providing a list of Android activities to wait for on session startup.

General

  • Fix handling of statuses from proxied drivers (Chrome, Selendroid, etc.).
  • Fix intermittent bug where creating a directory might crash the server.

iOS

  • Add support for WKWebView-based webviews.
  • Add support for automating the built-in Calandar app.
  • Fix bug where scrolling in Safari would crash.
  • Fix handling of Safari in “new window” state.
  • Fix handling of arguments for keys method so that clients that send an array do not fail.
  • Clean up handling of ipa files for real device tests.
  • Make sure processArguments parsing works.

Android

  • Add nativeWebScreenshot capability, to use adb screenshots instead of ChromeDriver if necessary.
  • Add support to auto-select device based on platformVersion rather than using first available device.
  • Allow appWaitActivity to be a list of activities.
  • Fix setting wifi and data state.
  • Implement initAutoWebView capability.
  • Report appPackage, appActivity, and deviceUDID when requesting session details, so tests have more information about running state.
  • Make sure unlock app is closed at the end of a session so that the device works again.
  • Fix handling of unlock to make less flakey.
  • Make sure unicode encoding works correctly when strings are long.
  • Fix handling of disableAndroidWatchers capability.
  • Fix bug where early ChromeDriver failure crashed Appium.

New: Use Atlassian Bitbucket with Sauce Labs

May 24th, 2016 by Ken Drachnik

The Sauce Labs Ecosystem & Integrations team is proud to announce our latest ecosystem integration: Sauce Labs for Atlassian Bitbucket Pipelines. Bitbucket Pipelines is a continuous delivery service built right within Bitbucket Cloud that gives developers end-to-end visibility from coding to deployment.

As more and more devs work in agile teams, they increasingly rely on their code repos to manage code, threads and bug fixes and many use Bitbucket. As a code collaboration tool, Bitbucket is an amazing tool for managing code across large teams.

With Sauce Labs and Bitbucket Pipelines, it is easy to launch your automated Selenium and Appium tests as part of your Bitbucket Pipeline build process. Sauce Labs allows users to reduce their overall build time to as low as a couple minutes through cloud based test execution and massive parallelization. You can also leverage the over 700 browser and OS combinations including real mobile devices through Sauce Labs for Bitbucket Pipelines.

Bitbucket Pipelines is currently in Beta. To enable Bitbucket Pipelines visit Bitbucket Pipelines Integrations. You can learn more about the Sauce Labs integration on our documentation wiki.

Also, visit the Atlassian Marketplace to see additional Sauce integrations available for JIRA and Bamboo.

Happy Testing!

Announcing Sauce Connect Launcher

April 8th, 2016 by Ken Drachnik

We’re proud to announce the release of Sauce Connect Launcher!

Sauce Connect is a secure tunnel which lets users run tests against applications behind the firewall or on localhost. You can use Sauce Connect to run both manual and automated tests.

Sauce Connect Launcher (SCL) is a Firefox add-on which allows you to run a Sauce Connect tunnel directly from Firefox. With SCL you don’t have to download Sauce Connect or go to the command line to install and run Sauce Connect. A common use case for this tool would be to use the Firefox add-on to launch a Sauce Connect tunnel and then run a manual test from Firefox on localhost.

To use the tool, simply download the add-on from the Firefox marketplace. When prompted, input your Sauce username and access key (these can be found on your User Settings page).

Once this is done, to launch a test simply go to Tools > Web Developer > Launch SC.

Sauce Connect Launcher

 

Now start your tunnel. You can stop the tunnel by going to Tools > Web Developer > Shutdown Sauce Connect.

Once this tunnel is established, you can run tests on local host.

Happy Testing!

Recap: Achieve True Continuous Integration with Sauce Labs and Microsoft Visual Studio Team Services (Webinar)

March 11th, 2016 by Bill McGee

Thanks to everyone who joined us for our recent webinar, “Achieve True Continuous Integration with Sauce Labs and Microsoft Visual Studio Team Services”, featuring Sauce Labs Product Manager Jack Moxon. In his presentation. Jack shows the new Sauce Labs plugin for Visual Studio Team Services (VSTS) and how to launch tests on Sauce Labs as part of their VSTS build, enabling Continuous Integration (CI).

Jack also shows how the plugin allows users to launch Sauce Connect – a tunnel that allows customers to securely test pre-production web and mobile apps in the cloud. The plugin also integrates Sauce Labs test results and detailed reports (like videos, screenshots and logs) back into Visual Studio and TFS to enable collaboration and expedite the debugging process.

The webinar covers:

  • How Sauce Labs and Visual Studio Team Services (VSTS) are integrated for Continuous Integration
  • Launching automated tests as a part of a VSTS build
  • How to test pre-production apps securely behind your firewall
  • Sharing detailed test results and reports to collaborate and debug faster

Want to learn more about the Sauce Labs plugin for VSTS? We’ll be exhibiting at Microsoft Build in San Francisco March 30 – April 1, 2016. The conference is sold out but you can add a reminder to your calendar to make sure you don’t miss the live stream.

Access the recording HERE and view the slides below: 

Appium 1.5 Released on Sauce Labs

February 26th, 2016 by Jonathan Lipps

Appium 1.5

The Appium team is extremely proud to announce the release of Appium 1.5! Appium 1.5 has been in the works for over half a year, and we would like to share why it’s such an important release for us. Appium 1.5 is primarily a technical rearchitecture. As the project grew from infancy through to 1.0 and beyond, the core team did their best to keep the code organized, to keep it easy for new contributors to come to the project, and to fix bugs and add features in a timely fashion. However, as is the case with many software projects, the time came when despite our best efforts, Appium’s fundamental architecture was working against us, not with us.

Given the relatively stable nature of the project in the 1.4.x series, we felt it was a good time to invest in the future velocity of Appium’s development. When we began work on Appium 1.5 we had the following goals:

  • Take a hard look at the current code organization and reconceptualize the relationships between different subsections of Appium with an eye to making distinctions clearer and concerns more separate.
  • Leverage the NPM ecosystem’s best practices and break Appium up into packages based on responsibility, maintenance, etc…, whenever appropriate.
  • Rewrite the entire codebase from ES5 JavaScript to ES2015 JavaScript with async/await (and in so doing wipe away a huge source of hard-to-trace Appium bugs).
  • Standardize subprocess management. Appium is basically a combination API server and subprocess manager (we manage instruments, uiautomator, chromedriver, selendroid, etc…), and so standardizing subprocess management makes our code cleaner and more readable throughout the project.
  • Abstract common driver behavior: Appium’s IOS support and Android support really derive from the same common interface, so we wanted to factor that interface out to reduce duplication of work and boilerplate across Appium’s different drivers.
  • Clean up our build process so that we are not using a handful of different build tools in different situations. Ensure we’re running CI at the subpackage level to save time when releasing Appium.
  • Create a more friendly architectural environment for new devs to contribute to Appium. Make sure that one type of change needs to happen in one place and that READMEs abound.
  • …and many more very specific technical goals. For more information on how Appium is now put together from a developer perspective, see our developer’s overview.

(more…)

Announcing New REST API Rate Limits

February 3rd, 2016 by Yaroslav Borets

Sauce Labs is introducing a new rate limiting on our REST endpoints in order to ensure a great experience for all of our customers. In addition to the recent limits placed on the number of requests per second we will be implementing further restrictions with dedicated hourly request limits for each endpoint. The new restrictions will limit the access to all endpoints to 10 reqs/s or 3500 reqs/hour if the user is logged in and 2 reqs/minute if user is logged out.  The limits will be tracked on a per account basis for both logged in and logged out users.

The new limits will go in the effect on Tuesday, March 1st 2016. We strongly encourage customers who use the REST API to modify their code to be able to gracefully handle a new set of restrictions. Please refer to the code samples below on how to prepare for the new limits as well as the headers to use.

The addition of more restrictive rate limits will be handled in a multi-stage process as follows:

  1. Starting February 1st , customers can opt-in to the new rate limits in order to test how their code handles rate limiting. The opt-in capability will be provided via a new header.
  2. On March 1st, the new rate limits will be in place by default, but customers can opt out using a dedicated header.
  3. Finally, in the beginning of April the new rate limits will be in place, and customers will no longer be able to opt out.

(more…)

Introducing Stability Improvements to Sauce Connect

January 5th, 2016 by Yaroslav Borets

Starting with Sauce Connect v4.3.13, new features have been added which aim to increase stability when testing websites behind the firewall.

1) Prevent Sauce Connect from shutting down when actively being used by job(s)

Argument:  --wait-tunnel-shutdown

Adding the above argument prevents users from shutting down their tunnels while jobs are still running.  Should the user attempt to close the tunnel, they will receive a warning as well as a count of tests currently using the tunnel (see example output below). Once the test(s) using the tunnel complete, the tunnel will close itself.

[code]05 Jan 09:08:20 – Sauce Connect is up, you may start your tests.
05 Jan 09:11:13 – Cleaning up.
05 Jan 09:11:13 – Removing tunnel 3a248df76eb14145ad0401c6c4aaf690.
05 Jan 09:11:13 – Waiting for any active job using this tunnel to finish.
05 Jan 09:11:13 – Press CTRL-C again to shut down immediately.
05 Jan 09:11:13 – Number of jobs using tunnel: 1.
05 Jan 09:11:19 – Number of jobs using tunnel: 1.
05 Jan 09:11:25 – Number of jobs using tunnel: 1.
05 Jan 09:11:33 – Number of jobs using tunnel: 1.
05 Jan 09:11:41 – All jobs using tunnel have finished.
05 Jan 09:11:41 – Waiting for the connection to terminate…
05 Jan 09:11:42 – Connection closed (8).
05 Jan 09:11:42 – Goodbye.[/code]

If the user wants to force shut the tunnel, they can do so by sending another close command (e.g. Ctrl + C)

2) Prevent Sauce Connect shut down due to colliding tunnels

Argument: --no-remove-colliding-tunnels

When this argument is added, any tunnel started with the same username and tunnel-identifier will be pooled together creating failover/load balancing for Sauce Connect. Once a pool of tunnels is established, newly started tests will be assigned to an active tunnel in the pool at random.  

Note: In order to join a pool, each tunnel must be started with this argument.

(more…)

iOS 9.1 Beta Support and Speed Improvements

October 16th, 2015 by Ken Drachnik

With a number of reported problems on iOS 9.01 and 9.02, Apple is in beta with a 9.1 version that is supposed to fix issues and become the stable release.  We have iOS 9.1 beta ready now, so you can start testing and be ready for the general release predicted for late October or November.  Check out all our browser / OS combinations at our platform configurator.

At Sauce we are constantly improving our service by making it easier to integrate with CI systems, upgrading our UI and, lately, improving reliability and speed.   We’ve just finished part one of a project to speed up the generation of VMs so you can test even faster.  Our Windows VMs are now faster than ever, so check ’em out.

Automated Mobile Testing with Real Devices

September 8th, 2015 by Ken Drachnik

Mobile testing has long been a manual process – limited by the devices you have close at hand or at best, a painfully slow process in the cloud.  Sauce has long had emulators and simulators to let you speed your tests by automating them in the cloud. Sometimes, however, emulators are not enough and you need to test on real devices.  We are happy to announce the public beta of our Real Device Cloud. That’s right, we’ve put hundreds of Android and iOS devices up in our cloud so you can automate your tests across emulators and real devices with massive parallelism. That means you can now test fast – just break your tests up into manageable bits and run them in parallel. There’s no waiting, there’s no reservation system – they are available and ready to test whenever you are.

We anticipate opening The Real Device Cloud by the end of the month so you can start testing all your mobile apps.  Download the datasheet.

The Real Device Cloud

  • Instant Availability – Get access to the most popular* iOS or Android devices with no waiting
  • Over 80 mobile emulators and simulators
  • Massive Concurrency – Run your tests in parallel to reduce your total test time
  • Integrate with your CI tool of choice – automate all your tests using the top CI tools like Jenkins, Bamboo, Travis, Team City or Circle CI
  • Test Native, Mobile Web and Hybrid apps
  • Access back end databases – that’s right, you can test your app in our cloud and have secure access to your backend data and websites for a true end-to-end test.
  • Full reporting – instant access to videos, screenshots and all the data for your tests so you can analyse your results quickly
  • Enterprise features – account and team management lets you provision test resources as needed and SSO integration means you don’t have to go to IT to add another tester to your account
  • Professional Services and Training – we have people and partners to help you get started with Appium or if you’re already proficient, our experts can help your team become super users

* The Real Device Cloud will open with Apple iPhone 6 and Samsung Galaxy S4 and S5 phones.  We will be adding more device types in the near future.