Today, Sauce Labs is very happy to announce the general availability of OS X, iOS, and Android in the Sauce cloud testing services. Mobile web and desktop web developers can now use our service to run their functional tests on iOS (iPhone, iPad), Android, Safari+OS X, Firefox+OS X, Google Chrome+OS X.
If you want to start using it NOW, you can stop reading and go straight to our Getting Started docs. Click on the Mac icon for iOS and the Linux icon for Android, and follow the steps to run your mobile tests in the cloud.
When my co-founders and I launched Sauce Labs in 2008 to create our test automation cloud (aka “Selenium as a Service” — pronounced “Sauce”), we knew we’d eventually face a particular technical problem someday—support for Mac OS X and the iPhone.
Supporting OS X is important because your test infrastructure should match the same combination of your users’ operating systems and browsers, which likely includes Safari, Firefox, and Google Chrome. OS X is doubly important as a prerequisite to supporting iPhone and iPad testing via the iOS Simulator. Many developers use the iOS Simulator to test their applications. And of course, the iOS Simulator only runs on OS X.
We already use the Kernel-based Virtual Machine (KVM) hypervisor for our existing Windows and Linux cloud. It’s the best toolkit for a large-scale and highly-distributed test lab. Given our roots, we wanted to explore whether we could virtualize OS X with KVM as the hypervisor.
Planting a field of Mac Minis—with no virtualization—was one way we could have built out our new OS X capability. But virtualizing OS X leads to a more secure environment. Every test in Sauce runs on a new, pristine virtual machine that’s destroyed after each use. We also use the Copy on Write (qcow2) disk format so changes are never written back to the original virtual machine image. This ensures users’ tests are completely sandboxed. It clears up a lot of customer concerns regarding privacy and security of their tests and the applications they’re testing.
We started the “Apple Sauce” project with patches Alexander Graf had published for the KVM project. By 2011, however, the patches were starting to bit-rot as the underlying codebase changed. We contacted Alexander and asked if we could sponsor his further development of the patches. He brought in René Rebe, another excellent KVM contributor. With Alexander’s and René’s blessing, we’re pleased to now release these updated patches to the KVM community. We don’t know yet if the patches will be incorporated into the KVM project, but in keeping with our commitment to open source, we want the community to have the option.
With Android, we didn’t have to move as many mountains to get it working. But we did have to speed it up. We’re running the Android emulators on the most recent version of Ubuntu – 12.04. This includes a very recent version of KVM, which in turn includes hardware virtualization optimizations to make the Android Emulator run a bit faster.
To use the new browser and OS combinations in your automated tests, you use the Selenium 2 API. A huge thank you also goes to the Selenium core contributors, and especially Google, for contributing the original iOS and Android patches.
We’re proud of the work everyone has put into making this all work. It wasn’t easy. Selenium tests running on OS X, iOS Simulator, and Android have been running in private beta in our cloud for some months. But the job of a test lab operator is never done. Operating systems are always getting updated, new browser versions are getting released on faster and faster schedules, and cool new HTML5 features are getting rolled out, pushing the limits of what testing tools support. Running a test infrastructure service is definitely a marathon, not a sprint. So now that we’ve done the hard work for you, go run some tests already! Your users will thank you.
* And a footnote for the lawyers: Apple licensing allows virtualized OS X so long as it’s run on Apple hardware. So we purchased a whole bunch of Mac Pros and added them to our existing server farm. We boot directly to Linux (Ubuntu 12.04) on those Mac Pros, run KVM on Linux, then run OS X as guest VMs. And again, following the terms of the license, we purchased OS X Server editions for each guest VM.