Thanks to those of you who joined us for our last webinar, How Yahoo! Mail Transformed Its Functional Testing + Continuous Delivery Process with Front End Developer Neil Manvar. (more…)
Posts Tagged ‘Sauce Labs’
Thanks to InfoWorld for naming Appium as the mobile testing framework of choice in the category of the Best Open Source Application Development Tools in this year’s Bossie Awards! Sauce Labs is proud to sponsor the development and maintenance of this world class open source project. Cheers to our Ecosystems team and to the external committers who have helped drive its success. For more about Appium, read below.
Appium is at its heart a Node.js Web server that exposes a REST API. It receives connections from a client, listens for commands, executes those commands on a mobile device, and responds with an HTTP response representing the result of the command execution. If you don’t want to fuss with Node.js, you can install GUI-driven Appium servers on Windows and OS X.
— Martin Heller
The Bossy Awards showcase InfoWorld’s picks of the year among languages, frameworks, libraries, and all the other tools that programmers use. To learn who else won, click here.
Learn how one Engineer helped transform internet giant Yahoo! Mail’s quality engineering process through functional testing and continuous delivery – and the role Sauce Labs plays in their process.
Be sure to sign up for our next webinar on Wednesday, October 22nd, 2014, at 11:00am PDT. You won’t want to miss this one!
Over the last two years, Front End Developer Neil Manvar’s most important contribution to Yahoo! has been developing a modern functional testing framework that is based on open-source technologies, plays well with legacy code, supports many browsers, does not need maintenance, is readable to product managers, and makes writing a pleasure.
In his quest to build this framework, he’s also learned to navigate structural and organizational challenges; most significantly convincing upper management to require each developer to write and run tests on their code as their new standard operating procedure.
Neil will talk through and highlight:
• What their tech stack looks like (Cucumber, Page-Object, Watir-Webriver, Sauce Labs)
• How they flip features on and off and test against them
• How features get pushed, and the role of functional testing
• Why they structurally implemented testing into each build, even for devs
• How the shift towards cloud-based services, open-source technologies, and devs who write QA code is changing the face of the entire organization
A Q&A will follow. All registrants will receive a link to the presentation and recording, regardless of attendance. Sign up today!
Sauce is thrilled to announce that we’ve integrated with Siesta by Bryntum!
Sauce is a cloud-based tool that enables you to securely test your web and mobile apps across 385+ browser/OS/platform/device combinations.
How does Siesta work with Sauce?
Siesta’s integration with Sauce is super easy since they’ve done most of the work under the hood. The main difference from the usual way you run WebDriver tests in Siesta is the addition of a –saucelabs argument, which will define the necessary credentials for connecting to Sauce and the –cap arguments which will specify the OS and browser combinations on which you want your tests to run. When you run Siesta tests using WebDriver, a standard command might look like this:
For running the same test on Sauce’s hosted browsers, you’ll need to add your Sauce credentials and desired capabilities, like so:
__SIESTA_DIR__/bin/webdriver http://localhost/myproject/tests/harness.html --saucelabs SL_USERNAME,SL_KEY --cap browserName=firefox --cap platform=windows
This command will then arrange a few things behind the scenes, like setting up the Sauce Connect tunnel so these tests can run locally and securely, and direct Sauce to execute the test specs on the latest version of Firefox on Windows. Of course, there are many desired capabilities and platforms you can choose to make sure you’re running tests according to the coverage that you need, and you can see all the different options on the Sauce Labs Platforms page.
For more information and the nitty gritty details, go ahead and check out the instructions provided in the Siesta docs: http://bryntum.com/docs/siesta/#!/guide/saucelabs_integration
What does the integration mean?
For more information, check out the Bryntum blog post.
– Michael Sage, Principal Technology Evangelist, Sauce Labs
Michael Sage is a Principal Technology Evangelist at Sauce Labs who helps software teams develop, deliver, and care for great apps. He’s spent over 15 years as a solutions architect and consultant with software companies including Mercury Interactive, Hewlett Packard, and New Relic. He lives in San Francisco, CA.
Sauce Labs’ Support Team wants to help our customers easily ramp up their mobile automated testing, so we put together this easy guide to Sauce integration with Appium. We hope you find it to be informative.
If you have any remaining questions about Sauce Labs’ integration with Appium, feel free to leave a note in the comments.
– Cheers, the Support Team at Sauce
What is Appium? Appium is an open-source tool that you can use to automate tests for mobile native, mobile web, and mobile hybrid applications. Just like Selenium Webdriver – which is an open-source tool that you can use to to automate web app tests – Appium is an automation library that you can use to automate tests for mobile applications. Other details:
- It is cross-platform. This allows you to write tests against multiple platforms (iOS, Android), using the same API.
- It is not locked into a specific language or framework for writing and running your tests.
- It was created by extending the Selenium Webdriver JSON Wire Protocol with extra API methods that are useful for mobile automation.
- Mobile Native Applications vs Mobile Web Applications vs Mobile Hybrid Applications
Types of mobile applications:
- Mobile Native Applications: Native apps can use all the features from your device, such as the camera, audio, microphone, and more. These applications work on a specific platform (i.e iOS or Android). Use the respective platform Software Development Kit (SDK) to develop them. Once they’re released for public use, users install the application on their respective devices through an application store (e.g. Google Play or Apple’s App Store).
- Mobile Web Applications: These ones are a little tricky; instead of being an application that you download on your device, mobile web applications are a mobile website that you can access through a browser on your mobile device (e.g. Safari or Chrome). The advantage to creating and using these is that they are not platform-specific, which means that you should be able to access this application from any mobile platform (i.e. iOS or Android). Unfortunately, these applications have limitations when it comes to using a device’s internal features.
- Mobile Hybrid Applications: As the name indicates, mobile hybrid applications are part mobile native app and part mobile web app. Just as with mobile native applications, you can find and download mobile hybrid applications using an application store (e.g. Google Play or Apple’s App Store), you can access them through icons on your device, and the app is be able to use all of a device’s features, such as the camera, audio, microphone, etc. In the same manner as a mobile web app, a mobile hybrid application looks like a mobile website that can be accessed through a browser, but in this case the browser is an embedded webview within the application that would just allow to display some HTML. A good example of a mobile hybrid app is that from Bank of America. It provides a user with the mobile native application perks while simply rendering pages from their website.
Appium with Sauce Labs
In certain cases, Sauce Labs uses Appium as the server for our latest iOS simulators and Android emulators. This means that whenever you run a test in Sauce Labs specifying any of the following mobile platforms, Appium is the automation tool in the background that starts and drives the simulator or emulator for your test.
Sauce Labs platforms that use Appium
Mobile Native Applications
- all iOS platforms
- all Android platforms
Mobile Web Applications (currently only available for iOS)
- iPhone 6.1
- iPhone 7.0
- iPhone 7.1
- iPad 6.1
- iPhone 7.0
- iPhone 7.1
Start testing with Appium + Sauce Labs
Appium Bootcamp: Want to jumpstart automated mobile testing on Sauce with Appium? Check out this bootcamp series by noted Selenium expert Dave Haeffner:
Chapter 1: Get Started
Chapter 2: The Console
Chapter 3: Interrogate Your App
Chapter 4: Your First Test
Chapter 5: Write And Refactor Your Tests
Chapter 6: Run Your Tests
Chapter 7: Automate Your Tests
Chapter 8: Additional Information
Dave recently immersed himself in the open source Appium project and collaborated with leading Appium contributor Matthew Edwards to bring us this material. Appium Bootcamp is for those who are brand new to mobile test automation with Appium. No familiarity with Selenium is required, although it may be useful.
Now that you’re up and running with Appium locally, in the cloud, and on a CI solution, it’s best to show you where you can find more information. Below is a collection of some great resources to help you find your way when it comes to mobile testing.
These are the official tutorials for the Appium project for Android and iOS. They served as inspiration and a base for this getting started series. They are great follow-on material since they cover various topics in more depth, and include Java examples as well.
If you have an issue or a question, this is a great place to turn to. Before posting an issue, be sure to read through the Appium Troubleshooting docs and search the group to see if your question has already been asked/answered.
In addition to the Google Discussion Group, you can hop on the Appium HipChat chat room and ask questions from others in the Appium community.
This is a follow-up post answering loads of questions from a webinar from just after thet Appium 1.0 release. It’s chocked full of a lot of great information.
In this video, Jonathan Lipps (Appium’s Chief Architect) explains mobile automation with Appium.
This is an open-source book that is a work in progress; authored by Jonathan Lipps. It’s working title is “Appium: Mobile Automation Made Awesome”.
- GTAC 2013: Breaking the Matrix – Android Testing at Scale
- GTAC 2013: Breaking the Matrix Q&A
- How the Google+ Team Tests Mobile Apps
These links (a video, Q&A, and a blog post) cover how Google approaches Android testing.
uiautomator is a crucial component of Android test automation. In this video, the engineers behind it talk about it’s future.
This video is a walk through Google’s newest Android testing framework. This isn’t directly related to Appium, but it contains some useful information.
- Automating User Interface Testing With Instruments, by Apple at 2010 WWDC
- Apple’s iOS Accessibility Guide
Appium relies on Apple’s UI Automation support, and these are some solid resources for understanding it better.
If you are a Sauce customer and encounter an issue when using their platform with Appium, be sure to open a support ticket.
If you’re using Appium and you think you’ve found a bug specific to either Android or iOS, then let Google and/or Apple know. In either case it’s best to make sure that the bug is not an Appium issue before filing an issue.
For Google, file an issue here.
These are great instructions on how to search through the Appium source code to find more information.
There are over 600 Appium questions posted on Stack Overflow for you to peruse.
Xamarin has a free cheat sheet comparing popular mobile app controls. Definitely worth a look.
Now you’re ready, armed with all the information you need to continue your mobile testing journey.
About Dave Haeffner: Dave is a recent Appium convert and the author of Elemental Selenium (a free, once weekly Selenium tip newsletter that is read by thousands of testing professionals) as well as The Selenium Guidebook (a step-by-step guide on how to use Selenium Successfully). He is also the creator and maintainer of ChemistryKit (an open-source Selenium framework). He has helped numerous companies successfully implement automated acceptance testing; including The Motley Fool, ManTech International, Sittercity, and Animoto. He is a founder and co-organizer of the Selenium Hangout and has spoken at numerous conferences and meetups about acceptance testing.
Follow Dave on Twitter – @tourdedave
“Recently at Sauce Labs we decided to retool our documentation system. This decision came after accumulating docs in a number of template systems and repos which were difficult to standardize and maintain. The result of this effort was a new markdown-based docs site available at docs.saucelabs.com.”
For all the details, be sure to check out Chris’ post – just click the image below to view.
Want to contribute to Sauce Labs’ documentation? In the spirit of open source, we’ve housed them in GitHub. Submit away.
Last week our friends at CircleCI showed us how to securely run cross-browser front-end tests on the Sauce Labs cloud using their hosted Continuous Integration service. We’ve long been advocates of good continuous integration practices and have developed a few plugins for some of the more common CI servers out there. We’re super excited to add CircleCI to our list and even more excited about how easy it is to get it going!
Continuous Integration in the Cloud
Continuous Integration, if you don’t already know, is the process of building your app frequently from a shared source code repository and running tests to make sure everything works. If your tests don’t pass and the build is not successful, the code that was checked in since the last good build is where the defects were introduced, and so problems are much easier to find and fix quickly.
Maintaining a local CI server can be a hassle. Anyone who’s spent any considerable time configuring Jenkins jobs with all it’s various plugins and tasks can tell you all about it. CircleCI, on the other hand, integrates directly with GitHub and can actually *infer* necessary settings directly from your code (if you’re following good development practices for that language and environment) and so many projects just magically build themselves on CircleCI without any additional configuration on your part. It’s like three clicks from zero to CI. Pretty sweet! If you do need to tweak or customize any settings, you can easily do so by describing those settings in a circle.yml file placed in your repo.
Running Tests on Sauce Labs Browsers
Sauce Labs is the world’s largest cross-browser client cloud. We host over 375 different configurations of operating systems and browsers so you can ensure that your app works on all the specific platforms and versions you need to support. These days that’s an ever-growing list! So it makes sense to run these tests with your continuous integration process so you know things work across the board and you don’t end up spending a bunch of time and trouble trying to hunt down bugs that were introduced much earlier in the development cycle.
Now, if your build deploys your code to a publicly accessible environment, CircleCI will simply execute your Selenium tests and you probably won’t need to configure anything, since Sauce Labs browsers will be able connect to that environment over the public network. However, if you want CircleCI to execute your tests locally in it’s build containers, you’ll need to use Sauce Connect.
Sauce Connect is a secure tunneling utility which allows you to execute tests behind firewalls via an encrypted connection between Sauce Labs and your testing environment. When you run Sauce Connect yourself, you typically do it from a command line or shell script and supply your Sauce Labs account credentials as parameters. With a CircleCI build, you’ll need to set it up in the circle.yml file so it can be launched as part of the build process and those tests can run inside the local container.
All that’s really involved here is telling the build task where to find Sauce Connect and how to start it up with your account credentials.
The first part is downloading and unpacking the Sauce Connect file, which you add as a custom dependency entry in your circle.yml:
dependencies: post: - wget https://saucelabs.com/downloads/sc-latest-linux.tar.gz - tar -xzf sc-latest-linux.tar.gz
The second part is to add your credentials, launch the tunnel, and check that it’s running before kicking off the tests. You’ll put these lines in the `test` section of circle.yml:
test: override: - ./bin/sc -u $SAUCE_USERNAME -k $SAUCE_ACCESS_KEY -f ~/sc_ready: background: true pwd: sc-*-linux # Wait for tunnel to be ready - while [ ! -e ~/sc_ready ]; do sleep 1; done
That’s all there is to it. You can find out the details here and see a full example on GitHub. And CircleCI has a nice little utility to help you add your credentials as environment variables so that they are not visible as plain text in the repo.
With CircleCI tackling all the necessary work involved in supporting your continuous integration process and Sauce Labs hosting the world’s most extensive cross-browser client cloud, you can be free of the costs and hassles of managing all these systems and grids and get back to focusing on the business of making great software.
– Michael Sage, Principal Technology Evanglist, Sauce Labs
Michael Sage is a Principal Technology Evangelist at Sauce Labs. He helps software teams develop, deliver, and care for great apps. He’s lived through two or three technology revolutions and expects a few more to come, and finds the prospect incredibly exciting. A native of Philadelphia, he’s made San Francisco his home for over 25 years, but still can’t find a good hoagie there.
Ever wanted to develop and test applications directly from your browser? Cloud9 enables users to do just this using their powerful cloud-based development environment. With their recent release and new integration with Sauce Labs, users can now instantly test mobile and web apps across any browser that Sauce Labs supports – without ever leaving the Cloud9 interface.
Join us for our latest webinar showcasing the integration on Friday, August 29, at 11am PST.
Ruben Daniels, Cloud9’s founder and CEO, and Jonathan Lipps, Sauce Labs’ Director of Ecosystems and Integrations, will walk you through Cloud9’s setup and how to test and debug across multiple browsers and platforms with Sauce Labs.
This 30 minute webinar includes a Q&A. Click here to sign up today!
All registrants will receive a link to the recording and other assets following the webinar, regardless of attendance.
On Saturday, August 2nd, we will update our Selenium and Chrome default versions to meet current, stable implementations. This update affects users that run automated Selenium tests on Sauce.
Default versions of Selenium and Chrome are used only for tests that don’t have a specified browser version. Users who choose to assign Selenium and Chrome versions to their tests will remain unaffected.
Below you’ll find more details about the updates.
Currently the default Selenium version is 2.30.0. Following the update on August 2, the new default Selenium version will be 2.42.2. We advise you to test the new version (2.42.2) in advance using the following desired capability:
If you run into any issues with the new default, note you can continue using the previous version (2.30.0) after Saturday by making the test request the selenium-version desired capability referred to below:
Currently the default Chrome versions are Chrome 27 and Chromedriver 1. Following the update on August 2, the new default Chrome versions will be Chrome 35 and Chromedriver 2.10. We advise you to test the new versions (Chrome 35, Chromedriver 2) in advance using the following desired capabilities:
"browserName": "chrome" "version": "35"
By requesting Chrome 35, Chromedriver 2.10 will be used automatically.
If you run into any issues with the new default, you can continue using the previous versions (Chrome 27, Chromedriver 1) after Saturday by making the test request the “version” desired capabilities referred to below:
"browserName": "chrome" "version": "27"
If you see any issues after moving your tests to these new versions, we suggest checking for known issues on https://code.google.com/p/selenium/issues/list or contacting the Chromedriver and Selenium user groups.