Announcing Support For Android 4.4 (KitKat)

July 25th, 2014 by Amber Kaplan

Hungry for moar Android? We thought so. kitkat

Announcing Sauce Labs support for Android 4.4 (KitKat)! Now you can test mobile web apps with Selenium Webdriver and native and hybrid apps with Appium using our Android emulators. Nom!

Before you start testing, there are a couple of caveats you should know about:

A. When an Android 4.4. emulator is in landscape mode, its apps are unable to recognize the orientation [1]. Further, while the Android 4.4 emulator looks like it’s in landscape mode, apps will not rotate to reflect this. At this time, there is no known workaround.

B. Certain Appium test runs on Android 4.4 emulators will error out when the Selenium `clear` command is sent [2]. This will only affect test runs that end up using the Selendroid testing framework under the hood [3]. The accepted workaround until a fix can be landed is, instead of using the `clear` command, use the `sendKeys` command, and send multiple DELETE (backspace) keys to clear out any text fields.

Can’t wait to get testing? Visit https://saucelabs.com/platforms to get started now.

Sauce + KitKat FTW.

[1] https://code.google.com/p/android/issues/detail?id=61671
[2] https://github.com/appium/appium/issues/3186
[3] https://github.com/selendroid/selendroid/issues/492

sauce_supports_android_4_4_kitkat

Announcing Cloud9 Preview: Instantly Preview Your Cloud9 Project In Any Browser (Powered by Sauce Labs)

July 24th, 2014 by Jonathan Lipps

Sauce + C9 IntegrationToday our friends at Cloud9 have released a brand new version of their powerful Cloud9 Development Environment, the web-based IDE that gives you unlimited flexibility to develop and test your applications directly from your browser.

As part of this release, we’re excited to announce an integration that we’ve been working on together for a while—the ability to instantly check out the site or app you’re developing in any desktop or mobile browser that Sauce Labs supports, directly in the Cloud9 interface.

Cloud9 has always given you the ability to easily preview the site you’re working on using your current browser. For example, in the screenshot below, you can see the preview window conveniently located where you can work on your code and see the changes reflected immediately:

cloud9 screenshot 1

Our integration simply adds another option to the drop-down menu to the right of the URL bar in the preview pane. If you click “Browser”, you’ll see additional options (it even remembers the last four browsers you used for quick access):

c9browsers

You can select any of our desktop or mobile browsers right from this interface. And if you’re already logged in to Sauce Labs, you’ll see the browser loading immediately. Now you can see whether your “responsive” website actually works as expected on, say, an iPhone Simulator, without ever leaving your IDE:

c9+saucelabs

We think this integration opens up a ton of possibilities for your development workflow, and are proud to be a part of the web-based IDE revolution with Cloud9. Check it out and let us know what you think!

Online Workshop – Testing Behind a Firewall

March 6th, 2014 by Amber Kaplan

Learn all about best practices and strategies for testing apps behind a firewall and how to use Sauce Connect during our next online workshop! Join speaker Mike Redman, Director of Sales Engineering at Sauce, on Tuesday, March 11, 2014, at 11:00 AM Pacific Time for the latest.

Whether you’re at the enterprise or startup level, security is a hot topic. That’s why we created Sauce Connect. Sauce Connect creates a secure tunnel between your firewalled app and the Sauce cloud so you can run your tests knowing that your data is encrypted through industry standard TLS.

Keeping our security standards in mind, we completely rewrote the app. With the launch of our latest version, Sauce Connect 4, your tests will now run faster than ever, even with heavy loads. It’s better performing, more reliable, and gives broader support for a wider range of web standards, including Websockets.

Mike will walk you through testing behind a firewall and how to use Sauce Connect 4.  A live Q&A session will follow.  Register today!

Meet the New Sauce Connect 4

February 26th, 2014 by lauren nguyen

Sauce Connect 4We know a thing or two about testing securely, and we know that testing behind a firewall is crucial for many of our users. That’s why we’re excited to announce that we just released Sauce Connect 4, a brand spanking new version of our secure tunneling app.

Sauce Connect creates a secure tunnel between your firewalled app and the Sauce cloud, enabling you to run your tests on Sauce while keeping your data secured. Data transmitted with Sauce Connect is encrypted through industry-standard TLS. So what’s new with v4?

To start off, we rewrote the entire app from the ground up to make it simpler and better performing, especially under high loads. That means your tests run faster than ever, even if you’re running large numbers of tests. And it’s more reliable than ever. The rewrite gives SC broader support for a wider range of web standards, including Websockets.

Sauce Connect 4 also supports working entirely through a proxy, if that’s what you’re into.  And of course we squashed some bugs.

Sauce Connect secure tunneling architecture

We’ll be holding an online workshop on Tuesday Mar. 11th, 11AM PST about testing behind a firewall, where we’ll cover best practices and strategies for testing apps behind a firewall and discuss everything you need to know about Sauce Connect. Register here.

Want to get started using Sauce Connect or learn more? Check out our Sauce Connect docs page.

TeamCity Plugin Now Available for Sauce

January 23rd, 2014 by lauren nguyen

teamcity sauce integration CIWe just finished a plugin that allows you to integrate your Sauce tests with JetBrains TeamCity! Hooray! We hope the new integration will help anyone using TeamCity for CI use Sauce to test all the things. The integration will allow you to run your tests through TeamCity and view the test results, including steps performed, screencast, and screenshots, inside TeamCity. You can get the integration set up here.

A Word About Our Growing Pains

November 7th, 2013 by Adam Christian

Over the last year, the Sauce customer base has been growing tremendously. We have heard your concern about our recent service interruptions. We have always had our sights set on 99.99% uptime, and are continually doing everything we can to achieve that. Examples include our recent datacenter move, constant capacity upgrades, and complete re-architecture of our Mac cloud.

Unfortunately, our growth has caused some hiccups that have negatively impacted your experience using Sauce. Knowing ourselves how important a smooth development process is to productivity, our recent quality of service is simply unacceptable and we vow to do better. We want it to be 100% clear that reliability is our top priority, and every time we can’t run jobs within single digit seconds, it becomes a red alert of the highest priority for our team.

To give you insight into how we are assessing situations such as the one yesterday: when our average launch time for a job exceeds 30 seconds, we consider that event an outage and launch an investigation to fully understand what happened, and why, with the onus on not only fixing the problem, but improving our process going forward. These include such things as eliminating single points of failure, scaling our cloud capacity far beyond the foreseen needs of our users, and introducing more stringent testing and analysis into the development and pre-deploy processes.

In terms of communication, when an outage happens we update @sauceops immediately and also status.saucelabs.com, which shows the average wait time for jobs. Our status page is continuously monitoring the situation and updating every few seconds. We also take care to update @sauceops when we notice that job wait times are exceeding an average of 10 seconds, though we don’t count that as an outage.

Additionally, we’d like to make a suggestion to help isolate your build from the effects of unexpected elevated wait times and internet latency should they happen in the future. If you are using a CI system or test runner that times out and fails tests, please consider increasing your time-out threshold to five minutes to isolate your suite for when the Sauce Labs cloud experiences longer wait times.  This will allow your jobs to remain queued rather than fail on Sauce. Again, you can go to http://status.saucelabs.com at any time to see average wait times and tweets from @sauceops.

Rest assured that we take events like yesterday extremely seriously, and are working to prevent this from happening in the future. We are fortunate to have very loyal customers and don’t want to let you down.

- Adam Christian
VP of Engineering
Sauce Labs

Announcing iOS 7 support in the Sauce Labs Cloud

October 31st, 2013 by Ashley Wilson

ios7_01

Today we’re excited to let you know that you can now test your hybrid and native apps with iOS 7 on Sauce. With more than 200M downloads since its release, we know that this is an important platform for our users to test on.

To get started, visit our browsers and platforms page for Appium and copy and paste the DesiredCapabilities provided for either version of iOS (or Android) you want to test on. And if you’re new to mobile test automation in general, check out our Getting Started with Appium guide.

If you have any issues, give us a shout. Otherwise, happy (iOS 7) testing!

Error Messages on Sauce and What They Mean

October 21st, 2013 by lauren nguyen

Ever run into an error message on Sauce you don’t understand? Alex put this handy list of error messages on Sauce together. You can find the original doc here. This will soon be available for reference on Sauce’s website, but in the meantime, brush up on the error messages you might see on Sauce and what they mean.

==== AUTOMATED JOB ERRORS: ====

– Invalid credentials –

Some combination of the following error messages will be thrown:
OpenQA.Selenium.WebDriverException : Unexpected error. Unknown username.
You sent username ‘None’ in your browser string, which is not a valid Sauce Labs account.
OpenQA.Selenium.WebDriverException : Unexpected error. Invalid Credentials.
org.openqa.selenium.UnsupportedCommandException: Invalid Credentials.
You sent the access key ‘None’ but it does not match the access key associated with your account. Please login to saucelabs.com to retrieve a valid access key.
This indicates that Selenium could not parse the credentials you sent. If you’re using “access-key” in your test setup, try replacing it with “accessKey”, or vice versa, as it’s language-dependent.

– User Terminated –

This means that you ended the job, either through the “Cancel Job” button, by jumping into an automated session from the Sauce website, or via a breakpoint. Since this terminates the job immediately and hands over control to you, the test will not upload any screenshots, video, or logs that were collected previously.

– Timeout errors –

1. “Test did not see a new command for 90 seconds. Timing out.”

2. “Selenium took too long to run your command”

3. “Test exceeded maximum duration after 1800 seconds”

Sauce builds in a few default timeouts to keep uncontrolled tests from eating up your minutes. Here’s a brief rundown on what each of the available timeout settings does:

1. The “idle-timeout” (default 90 seconds) ends the job if no commands are received from your Selenium script.

2. The “command-timeout” (default 5 minutes) ends the job if a single command takes too long to run.

3. And the “max-duration” timeout (default 30 minutes) ends the job if your entire test takes longer than you want.

You can set these to different values using “desired capabilities” settings in the setup of your test, as described at the URL above.

– Unsupported OS/browser/version combo –

Check to make sure that the browser, browser-version, and platform settings you’re using are in our supported list.
And note that there’s a separate list for Appium tests (as they use a different mechanic to drive the tests).

– Browser failed to start –

The twin sibling of “Unsupported version”, this message means that something a little more unusual is off in your test setup. Usually, this means that you’re specifying a Selenium version that isn’t compatible with the browser/version/OS you’ve selected. (For example, you should not be setting this for any mobile tests.)

Try simply omitting that setting, and if you still see the issue, feel free to write to help@saucelabs.com with a description of the issue and a copy of your setup code.

– Client disconnected during getNewBrowserSession request –

This means that your test runner decided to end the job before it had fully initialized on Sauce’s end. There are a few potential causes:

1. You’re running too many tests at a time: Check the left sidebar on your Account page. It shows a “Parallel tests” number, which is the maximum number of tests you can run at a time, based on your subscription level. If your account can run 2 parallel tests, and you’re launching 10, 8 will be “queued” until one of your tests finishes and a slot frees up. However, if this takes a long time, your test runner may choose to end the queued jobs after a few minutes instead of waiting. Just make sure you’re launching a reasonable number of simultaneous tests for your account.

2. High job wait times: Check our status page and/or follow @sauceops on Twitter for up-to-the-minute news about any issues within the service. If something causes demand for certain VMs to stack up, your jobs may be queued and (as above) terminated by your test runner.

Tests that end this way are never taken out of your minutes.

– Runaway execution. Please contact help@saucelabs.com for assistance. –

This message means that an error has been detected in the VM or in Selenium, which caused the test to behave abnormally, and Sauce detected this and shut down the job. These are very rare and usually do not recur. If you do see more than one on the same test, let us know.

==== MANUAL JOB ERRORS: ====

– Job does not load –

There are two common scenarios here:
• Error message: “Uh oh! Some error occurred while connecting to the browser”
• The job seems to start, but you see only a white text box in the middle of a black screen.
Both of these indicate that your browser is having trouble displaying the VNC stream from the remote machine. Take the following steps to troubleshoot:
• Check the video on Sauce: If the recorded video after the job shows a steady video stream, this indicates that the issue is in your computer or connection to Sauce. However, if the Sauce video shows the same issue, that indicates an issue in our service. In that case, send us the URL for the job page and a screenshot of the issue.
• Check that your browser is up to date: If you’re on an older version, this may cause incompatibilities. Update your browser and try again.
• Check your firewall: make sure that your machine allows full access for the interactive stream over the required ports.
• Check that the Internet connection is stable: We recommend running Sauce tests from a machine with a wired Ethernet connection, to ensure a steady connection. If the connection flickers, this error could be thrown.

==== FRAMEWORK-SPECIFIC ERRORS: ====

– Appium: Failed to download mobile app –

First, if you’re having trouble, try using Sauce’s temporary storage. Uploading your app to our servers makes it download faster.
If you’re using this, make sure that you’re specifying the “APP_PATH” as “sauce-storage:your_file_name” — NOT the saucelabs.com/rest/v1/storage… URL.
And when you upload your file to our server, check the JSON response. It should be formatted like this:
{“username”: “USERNAME”, “size”: 99999, “md5″: “bc01aad123456789glhf9ee6e8c7eab3″, “filename”: “AppName.zip”}
Make sure that the size is something reasonable for the .zip file you uploaded. If not, check your file and try again. This should resolve the issue.

Re-introducing the Sauce Launcher Plugin for Google Chrome

April 24th, 2013 by lauren nguyen

For all of you who run manual test sessions with Sauce Labs, we’ve got some great news. We recently released our updated Sauce Launcher plugin for Google Chrome, with lots of improvements to help you run manual jobs more easily.

Sauce Launcher for Google Chrome

Sauce Launcher makes launching a manual test as easy as clicking a button, and we think the update makes it even better. We updated the plugin’s look, feel, and layout to better match saucelabs.com, in order to make it easier to use. Sauce Launcher now also offers all the browsers and platforms we support, and it now automatically updates when we add newly supported platforms and features so it will never go out of date again! Download it in the Chrome webstore, and let us know what you think. Happy testing!

Announcing Support for Lynx in the Sauce Labs Cloud

April 1st, 2013 by Steven Hazel

Today we are extremely excited to announce the initial release of an important browser platform on Sauce: the classic — but innovative — text-based browser Lynx. While Lynx is not as popular as more modern, graphical browsers like Internet Explorer 6, it has a devoted following and a compelling feature set, and we feel it’s important that Sauce customers be able to test their web apps in a text-based browser.Screen Shot 2013-03-31 at 9.23.05 PM

Ever since Twitter demonstrated the power of simple, textual interfaces, we have been looking at Lynx as a bold way forward for the web. We think its less-is-more approach represents the future of sustainable web technologies. Because it spends less energy on graphics, Lynx probably has the smallest carbon footprint of any web browser out there, and lightning-fast performance.

Lynx also has advantages for developers: its javascript support is easier to understand and has fewer pitfalls than many modern browsers, and its support for key web technologies like Flash and Java Applets was decades ahead of its time, and even today is on par with the latest browsers for iOS and Android.

And because Lynx is so different from other browsers, we believe it’s extra important that developers be able to test their web apps against it. That’s why we’re excited to extend Sauce Labs’ cross-browser functional testing cloud to include interactive and Selenium testing for Lynx.

Give it a try today. Happy testing!