We’ve just released support for emulator testing on Android 5.0 (Lollipop). Android 5 includes some big changes to the look and feel with the move to Material. (more…)
We’ve just released our new platform configurator that easily generates the code you need to configure your automated tests to run on Sauce. You can select the automation API, device type, operating system, and browser version you want to test with, and it will automatically generate a snippet of code for those desired capabilities. You can toggle between code generated for different languages. (more…)
Announcing Sauce Labs support for Android 4.4 (KitKat)! Now you can test mobile web apps with Selenium Webdriver and native and hybrid apps with Appium using our Android emulators. Nom!
Before you start testing, there are a couple of caveats you should know about:
A. When an Android 4.4. emulator is in landscape mode, its apps are unable to recognize the orientation . Further, while the Android 4.4 emulator looks like it’s in landscape mode, apps will not rotate to reflect this. At this time, there is no known workaround.
B. Certain Appium test runs on Android 4.4 emulators will error out when the Selenium `clear` command is sent . This will only affect test runs that end up using the Selendroid testing framework under the hood . The accepted workaround until a fix can be landed is, instead of using the `clear` command, use the `sendKeys` command, and send multiple DELETE (backspace) keys to clear out any text fields.
Can’t wait to get testing? Visit https://saucelabs.com/platforms to get started now.
Sauce + KitKat FTW.
Announcing Cloud9 Preview: Instantly Preview Your Cloud9 Project In Any Browser (Powered by Sauce Labs)July 24th, 2014 by Jonathan Lipps
Today our friends at Cloud9 have released a brand new version of their powerful Cloud9 Development Environment, the web-based IDE that gives you unlimited flexibility to develop and test your applications directly from your browser.
As part of this release, we’re excited to announce an integration that we’ve been working on together for a while—the ability to instantly check out the site or app you’re developing in any desktop or mobile browser that Sauce Labs supports, directly in the Cloud9 interface.
Cloud9 has always given you the ability to easily preview the site you’re working on using your current browser. For example, in the screenshot below, you can see the preview window conveniently located where you can work on your code and see the changes reflected immediately:
Our integration simply adds another option to the drop-down menu to the right of the URL bar in the preview pane. If you click “Browser”, you’ll see additional options (it even remembers the last four browsers you used for quick access):
You can select any of our desktop or mobile browsers right from this interface. And if you’re already logged in to Sauce Labs, you’ll see the browser loading immediately. Now you can see whether your “responsive” website actually works as expected on, say, an iPhone Simulator, without ever leaving your IDE:
We think this integration opens up a ton of possibilities for your development workflow, and are proud to be a part of the web-based IDE revolution with Cloud9. Check it out and let us know what you think!
Learn all about best practices and strategies for testing apps behind a firewall and how to use Sauce Connect during our next online workshop! Join speaker Mike Redman, Director of Sales Engineering at Sauce, on Tuesday, March 11, 2014, at 11:00 AM Pacific Time for the latest.
Whether you’re at the enterprise or startup level, security is a hot topic. That’s why we created Sauce Connect. Sauce Connect creates a secure tunnel between your firewalled app and the Sauce cloud so you can run your tests knowing that your data is encrypted through industry standard TLS.
Keeping our security standards in mind, we completely rewrote the app. With the launch of our latest version, Sauce Connect 4, your tests will now run faster than ever, even with heavy loads. It’s better performing, more reliable, and gives broader support for a wider range of web standards, including Websockets.
Mike will walk you through testing behind a firewall and how to use Sauce Connect 4. A live Q&A session will follow. Register today!
We know a thing or two about testing securely, and we know that testing behind a firewall is crucial for many of our users. That’s why we’re excited to announce that we just released Sauce Connect 4, a brand spanking new version of our secure tunneling app.
Sauce Connect creates a secure tunnel between your firewalled app and the Sauce cloud, enabling you to run your tests on Sauce while keeping your data secured. Data transmitted with Sauce Connect is encrypted through industry-standard TLS. So what’s new with v4?
To start off, we rewrote the entire app from the ground up to make it simpler and better performing, especially under high loads. That means your tests run faster than ever, even if you’re running large numbers of tests. And it’s more reliable than ever. The rewrite gives SC broader support for a wider range of web standards.
Sauce Connect 4 also supports working entirely through a proxy, if that’s what you’re into. And of course we squashed some bugs.
We’ll be holding an online workshop on Tuesday Mar. 11th, 11AM PST about testing behind a firewall, where we’ll cover best practices and strategies for testing apps behind a firewall and discuss everything you need to know about Sauce Connect. Register here.
Want to get started using Sauce Connect or learn more? Check out our Sauce Connect docs page.
We just finished a plugin that allows you to integrate your Sauce tests with JetBrains TeamCity! Hooray! We hope the new integration will help anyone using TeamCity for CI use Sauce to test all the things. The integration will allow you to run your tests through TeamCity and view the test results, including steps performed, screencast, and screenshots, inside TeamCity. You can get the integration set up here.
Over the last year, the Sauce customer base has been growing tremendously. We have heard your concern about our recent service interruptions. We have always had our sights set on 99.99% uptime, and are continually doing everything we can to achieve that. Examples include our recent datacenter move, constant capacity upgrades, and complete re-architecture of our Mac cloud.
Unfortunately, our growth has caused some hiccups that have negatively impacted your experience using Sauce. Knowing ourselves how important a smooth development process is to productivity, our recent quality of service is simply unacceptable and we vow to do better. We want it to be 100% clear that reliability is our top priority, and every time we can’t run jobs within single digit seconds, it becomes a red alert of the highest priority for our team.
To give you insight into how we are assessing situations such as the one yesterday: when our average launch time for a job exceeds 30 seconds, we consider that event an outage and launch an investigation to fully understand what happened, and why, with the onus on not only fixing the problem, but improving our process going forward. These include such things as eliminating single points of failure, scaling our cloud capacity far beyond the foreseen needs of our users, and introducing more stringent testing and analysis into the development and pre-deploy processes.
In terms of communication, when an outage happens we update @sauceops immediately and also status.saucelabs.com, which shows the average wait time for jobs. Our status page is continuously monitoring the situation and updating every few seconds. We also take care to update @sauceops when we notice that job wait times are exceeding an average of 10 seconds, though we don’t count that as an outage.
Additionally, we’d like to make a suggestion to help isolate your build from the effects of unexpected elevated wait times and internet latency should they happen in the future. If you are using a CI system or test runner that times out and fails tests, please consider increasing your time-out threshold to five minutes to isolate your suite for when the Sauce Labs cloud experiences longer wait times. This will allow your jobs to remain queued rather than fail on Sauce. Again, you can go to http://status.saucelabs.com
Rest assured that we take events like yesterday extremely seriously, and are working to prevent this from happening in the future. We are fortunate to have very loyal customers and don’t want to let you down.
– Adam Christian
VP of Engineering
Today we’re excited to let you know that you can now test your hybrid and native apps with iOS 7 on Sauce. With more than 200M downloads since its release, we know that this is an important platform for our users to test on.
To get started, visit our browsers and platforms page for Appium and copy and paste the DesiredCapabilities provided for either version of iOS (or Android) you want to test on. And if you’re new to mobile test automation in general, check out our Getting Started with Appium guide.
If you have any issues, give us a shout. Otherwise, happy (iOS 7) testing!
Ever run into an error message on Sauce you don’t understand? Alex put this handy list of error messages on Sauce together. You can find the original doc here. This will soon be available for reference on Sauce’s website, but in the meantime, brush up on the error messages you might see on Sauce and what they mean.
==== AUTOMATED JOB ERRORS: ====
– Invalid credentials –
Some combination of the following error messages will be thrown:
OpenQA.Selenium.WebDriverException : Unexpected error. Unknown username.
You sent username ‘None’ in your browser string, which is not a valid Sauce Labs account.
OpenQA.Selenium.WebDriverException : Unexpected error. Invalid Credentials.
org.openqa.selenium.UnsupportedCommandException: Invalid Credentials.
You sent the access key ‘None’ but it does not match the access key associated with your account. Please login to saucelabs.com to retrieve a valid access key.
This indicates that Selenium could not parse the credentials you sent. If you’re using “access-key” in your test setup, try replacing it with “accessKey”, or vice versa, as it’s language-dependent.
– User Terminated –
This means that you ended the job, either through the “Cancel Job” button, by jumping into an automated session from the Sauce website, or via a breakpoint. Since this terminates the job immediately and hands over control to you, the test will not upload any screenshots, video, or logs that were collected previously.
– Timeout errors –
1. “Test did not see a new command for 90 seconds. Timing out.”
2. “Selenium took too long to run your command”
3. “Test exceeded maximum duration after 1800 seconds”
Sauce builds in a few default timeouts to keep uncontrolled tests from eating up your minutes. Here’s a brief rundown on what each of the available timeout settings does:
1. The “idle-timeout” (default 90 seconds) ends the job if no commands are received from your Selenium script.
2. The “command-timeout” (default 5 minutes) ends the job if a single command takes too long to run.
3. And the “max-duration” timeout (default 30 minutes) ends the job if your entire test takes longer than you want.
You can set these to different values using “desired capabilities” settings in the setup of your test, as described at the URL above.
– Unsupported OS/browser/version combo –
Check to make sure that the browser, browser-version, and platform settings you’re using are in our supported list.
And note that there’s a separate list for Appium tests (as they use a different mechanic to drive the tests).
– Browser failed to start –
The twin sibling of “Unsupported version”, this message means that something a little more unusual is off in your test setup. Usually, this means that you’re specifying a Selenium version that isn’t compatible with the browser/version/OS you’ve selected. (For example, you should not be setting this for any mobile tests.)
Try simply omitting that setting, and if you still see the issue, feel free to write to email@example.com with a description of the issue and a copy of your setup code.
– Client disconnected during getNewBrowserSession request –
This means that your test runner decided to end the job before it had fully initialized on Sauce’s end. There are a few potential causes:
1. You’re running too many tests at a time: Check the left sidebar on your Account page. It shows a “Parallel tests” number, which is the maximum number of tests you can run at a time, based on your subscription level. If your account can run 2 parallel tests, and you’re launching 10, 8 will be “queued” until one of your tests finishes and a slot frees up. However, if this takes a long time, your test runner may choose to end the queued jobs after a few minutes instead of waiting. Just make sure you’re launching a reasonable number of simultaneous tests for your account.
2. High job wait times: Check our status page and/or follow @sauceops on Twitter for up-to-the-minute news about any issues within the service. If something causes demand for certain VMs to stack up, your jobs may be queued and (as above) terminated by your test runner.
Tests that end this way are never taken out of your minutes.
– Runaway execution. Please contact firstname.lastname@example.org for assistance. –
This message means that an error has been detected in the VM or in Selenium, which caused the test to behave abnormally, and Sauce detected this and shut down the job. These are very rare and usually do not recur. If you do see more than one on the same test, let us know.