Announcing Sauce Integration With Siesta

September 16th, 2014 by Amber Kaplan

siesta-logoSauce is thrilled to announce that we’ve integrated with Siesta!

Siesta is a JavaScript unit testing tool that can help you test any JavaScript code and also perform testing of the DOM and simulate user interactions. The tool can be used together with any type of JavaScript codebase – jQuery, Ext JS, NodeJS, Dojo, YUI etc. Using the API, you can choose from many types of assertions ranging from simple logical JS object comparisons to verifying that an HTML element is visible in the DOM. It comes in two versions: Lite and Standard. With Lite you can easily test your JavaScript in the browser, and with Standard you can also automate your tests (highly recommended).

Sauce is a cloud-based tool that enables you to securely test your web and mobile apps across 385+ browser/OS/platform/device combinations.

How does Siesta work with Sauce?

Siesta’s integration with Sauce is super easy since they’ve done most of the work under the hood.

The main difference from the usual way you run WebDriver tests in Siesta is the addition of a –saucelabs argument, which will define the necessary credentials for connecting to Sauce and the –cap arguments which will specify the OS and browser combinations on which you want your tests to run.

When you run Siesta tests using WebDriver, a standard command might look like this:

__SIESTA_DIR__/bin/webdriver http://localhost/myproject/tests/harness.html

For running the same test on Sauce’s hosted browsers, you’ll need to add your Sauce credentials and desired capabilities, like so:

__SIESTA_DIR__/bin/webdriver http://localhost/myproject/tests/harness.html --saucelabs SL_USERNAME,SL_KEY
--cap browserName=firefox --cap platform=windows

This command will then arrange a few things behind the scenes, like setting up the Sauce Connect tunnel so these tests can run locally and securely, and direct Sauce to execute the test specs on the latest version of Firefox on Windows. Of course, there are many desired capabilities and platforms you can choose to make sure you’re running tests according to the coverage that you need, and you can see all the different options on the Sauce Labs Platforms page.

For more information and the nitty gritty details, go ahead and check out the instructions provided in the Siesta docs: http://bryntum.com/docs/siesta/#!/guide/saucelabs_integration

What does the integration mean?

It’s become more than clear in the last few years that managing all the different browsers and operating systems out there, both desktop and mobile, is a whole lotta hassle and clearly better handled by a cloud-based service like Sauce Labs. But you need to ensure your JavaScript works from everywhere, and since you can now run your Siesta tests on any or all of Sauce’s 385 platform configurations, you can get that assurance with minimal effort and stay focused on the JavaScript code itself without having to invest your time and energy managing infrastructure.

Now you can test your JavaScript code across all of Sauce’s instantly available browsers and platforms without setting up or maintaining your own VMs. Magic!

For more information, check out the Brynthum blog post.

Michael Sage, Principal Technology Evangelist, Sauce Labs

Michael Sage is a Principal Technology Evangelist at Sauce Labs who helps software teams develop, deliver, and care for great apps. He’s spent over 15 years as a solutions architect and consultant with software companies including Mercury Interactive, Hewlett Packard, and New Relic. He lives in San Francisco, CA.

Recap: How To Combine Front-End and Back-End Testing [WEBINAR]

September 15th, 2014 by Amber Kaplan

sauce_labs_blazemeterThanks to those of you who attended our last webinar, How To Combine Front-End and Back-End Testing, featuring our  Chief Technology Evangelist Michael Sage and BlazeMeter‘s VP of Customer Success, Ophir Prusak. 

Everyone knows that front-end testing is crucial to make sure your web and mobile apps are meeting the needs of your users and customers. But how do you know what will happen to your front end when your web or mobile app is under heavy load?

Michael and Ophir set out to help answer this question, plus more. Together, they covered many valuable topics, including:

  • The fundamentals of approaching performance vs. front-end testing
  • Step-by-step instructions on getting real-world results from your front-end while applying load to the back-end
  • Critical issues you need to know about performance testing

They also showed a real-world test in real time using JMeter and Selenium.

Missed the webinar? You can watch it in its entirety below.

Below you’ll find the top Q&A’s post-presentation:

Q: Why do we need JMeter? Why can’t we do performance testing with Selenium browsers?

A: [Ophir] Great question! The answer is really all to do with scalability. When you’re doing a test with Selenium, you’re usually using real browsers –and real browsers are very resource intensive. On the other hand, when you’re doing a test in JMeter by using virtual users, I can exponentially get a lot more users per machine. Just to give you an idea of ballpark numbers, on a single low-end Amazon EC2 machine, I can get 3,4 or 5 virtual users using browsers in parallel until I hit a bottleneck on the machine. If I’m doing it with a JMeter, I can get 1,000 people in parallel. So you’re looking at around 250X more users when I’m doing a JMeter test per machine than Selenium. So if you’re looking at ten or 100 users, you can do it with Selenium but when you’re looking at tens of thousands of users, it just won’t be able to support it.

Q: Can I test behind the firewall?

A: [Michael] Yeah, absolutely We have a utility called ‘Sauce Connect’, which creates an encrypted tunnel between your environment inside your firewall or your DMZ and our grid and you get a dedicated virtual machine to act as the tunnel property. You run this utility and it appears as if the Sauce Labs grid is inside your network and it’s all done in an encrypted fashion.
[Ophir] At BlazeMeter, we also have the ability to run behind the firewall. It’s a different type of solution. It does require having something we call a BlazeMeter agent, which is basically a load generator which sits behind the firewall, which you can still control through your browser but you have a local machine which is creating the requests for you.

Lastly, please follow our friends at BlazeMeter at @blazemeter, Sauce Labs at @saucelabs, Michel Sage at @mondosage, and Ophir Prusak at @prutwo to keep up with the latest.  Feel free to share this webinar using the hashtag #frontandbackendtesting.

 

 

Sharecare Scales Mobile Automated Testing With Sauce Labs [VIDEO]

September 10th, 2014 by Amber Kaplan

We took a whirlwind trip to New Haven, CT to sit down with Daniel Gempesaw, Software Testing Architect at Sharecare. Sharecare is a wellness platform founded in part by Dr. Oz and Oprah.

We learned that after Sharecare started automating their testing process while using Sauce, time spent for each deploy fell from 7 days to 1 day. With more free time, the team is able to focus on scaling their mobile testing with Appium and employing new best practices such as mobile CI.

Watch this video to learn how they scaled their mobile testing with Sauce Labs.

Be sure to check out the case study, too.

Happy testing!

Re-Blog: Add Some Sauce To Your IE Tests

September 4th, 2014 by Amber Kaplan

Sauce Labs hearts ThoughtWorks! And apparently the feeling’s mutual. Check out this great blog post mentioning Sauce Labs by Tom Clement Oketch.

See an excerpt below:

Using Sauce Labs with a Continuous Integration (CI) Service

Running your tests only locally will not get you much mileage, especially if you are working with a sizeable team. Using a Continuous Integration service is therefore essential. Fortunately, Sauce Labs has first class support for a number of Continuous Integration services including JenkinsBambooTravis and TeamCity. The preceding links should contain sufficient information to integrate Sauce Labs with each of those CI services. In our case however, we had already set up Snap-CI as our CI service of choice. Snap-CI currently does not provide such integration with Sauce Labs. We therefore made the following adjustments to include Sauce Labs in our build pipeline:

  • As part of our functional test stage on Snap-CI, it was necessary to set up a tunnel to Sauce Labs using Sauce Connect, otherwise the browsers at Sauce Labs would not be able to run against our application instance in the Snap-CI Build Pipeline. We came across this gist which we altered to suit the requirements of our Snap Build. The gist takes care of downloading, starting and waiting for Sauce Connect to establish a tunnel before the functional tests are actually run
  • The terrain.py setup remained largely unchanged, except for the use of environment variables rather than the explicit declaration of the Sauce Username and API Access Key. Given that Snap-CI exports a number of additional environment variables during each build, it was also possible to annotate the test descriptions with these variables. Using annotations such as the pipeline counter and the git commit subsequently made it easier to identify the appropriate test in the Sauce Labs test dashboard.

Are you in a position where you need to run IE tests without getting your hands dirty? If so, maybe Sauce Labs can save your time as well. If on the other handyou are more interested in evaluating some of the other options out there, then this guide is a good place to start.

Don’t miss the entire post HERE for more code and instruction.

Have an idea for a blog post, webinar, or more? We want to hear from you! Submit topic ideas (or questions!) here.

How to Combine Backend & Front End Testing [WEBINAR]

August 28th, 2014 by Amber Kaplan

sauce_labs_blazemeterEveryone knows that front end testing is crucial to make sure your web and mobile apps are meeting the needs of your users and customers. But how do you know what will happen to your front end when your web or mobile app is under heavy load?

Sauce Labs and BlazeMeter are teaming up for an awesome webinar, in which we’ll be giving you step-by-step instructions on how to get real-world results from your front end while applying load to the backend.

We will:

  • Reveal the 3 critical things you need to know about performance testing
  • Run a real-world test in real time using JMeter and Selenium
  • Cover the fundamentals of how to approach performance vs. front end testing

Our experts will stay online for a live Q&A session. Plus, at the end of the webinar, we’ll be giving away an exclusive coupon from BlazeMeter & Sauce Labs for a great discount on our solutions.

Sign up today for our event on Tuesday, September 9th, 2014, at  11:00 am Pacific Time.

The Appium Philosophy

August 26th, 2014 by Amber Kaplan

Appium is a rising star in the mobile test automation landscape, and since a few of our developers here at Sauce Labs are regular committers to the project, it’s pretty close to our hearts. You’ll often find Sauce devs hanging out on the Appium Google group answering questions, musing about testing strategies, and helping folks tweak their configurations or hunt down bugs.

When it comes to mobile test automation, in many ways we’re still figuring out what the best approaches are, and with Appium, we’ve had the benefit of lessons learned from early automation solutions that didn’t quite work as well as we’d hoped. Some of these lessons are nicely summarized in Appium’s four-point philosophy:

  1. You shouldn’t have to recompile your app or modify it in any way in order to automate it.
  2. You shouldn’t be locked into a specific language or framework to write and run your tests.
  3. A mobile automation framework shouldn’t reinvent the wheel when it comes to automation APIs.
  4. A mobile automation framework should be open source, in spirit and practice as well as in name!

Let’s explore these four points, shall we?

You shouldn’t have to recompile your app or modify it in any way in order to automate it.

Some early attempts at mobile test automation were based around OCR and pixel-based interactions, which history tells us are never very reliable. After that we saw the introduction of in-app agents that can execute the underlying code that would otherwise be triggered by a user interacting with the app. These agents then make calls to the same code triggered by user actions, like swipes and taps and pinches. It’s not a bad solution, but the gotcha is that you have to compile these agents into your app while it’s being tested, and you probably want to take it out after the testing is done. The result? You’re not actually testing the code that you release.

The agent approach is a bit odd, too, since the real-world user actions are always from outside the app itself, and so if you’re simulating those actions inside the code rather than triggering them by their corresponding UI action, you’re that much further away from the real experience of an actual human user.

In order to solve this problem and provide for testing with these more real-world style user interactions, both Apple and Google have created user interface automation frameworks for their respective development environments. Google provides uiautomator, which is a Java API for simulating these UI actions on Android devices and emulators, and Apple provides UI Automation Instruments, a JavaScript programming interface for use with iOS devices and simulators.

Appium works by interfacing with these vendor-provided automation frameworks, translating your test code into the platform-specific interactions. We think this is a better approach than using an agent, since you don’t have to compile a test build that contains code which will not be in the production build. You’re best off testing the same code you will release.

The next two points of the Appium philosophy are covered together since they both derive from the decision to implement the WebDriver API and JSON Wire Protocol:

You shouldn’t be locked into a specific language or framework to write and run your tests.

A mobile automation framework shouldn’t reinvent the wheel when it comes to automation APIs.

WebDriver is already well-known as the engine behind Selenium 2 test automation for Web apps. In the Appium implementation, instead of driving a Web browser on a desktop operating system as Selenium does, Appium drives native apps and browsers on mobile operating systems.

Many testers are already very familiar with writing and running tests locally using Selenium WebDriver, where your test will open a browser, run through some interactions with the app, and verify that the test case passes. But the magic of WebDriver really happens when it’s used in a distributed fashion. Instead of making local calls directly to the browser, the WebDriver test becomes an HTTP client and makes requests to a WebDriver server, which in turn actually makes the necessary calls to the browser and app. The elegance of Appium is that it utilizes this framework to interact, not with a desktop browser, but with Android’s UI Automator or Apple’s UIAutomation Instruments, which then perform user actions on the native app or mobile browser running in either an emulator or a real device. And of course Sauce Labs’ infrastructure can manage all of this for you!

(Appium architecture for iOS testing on Sauce)

(Appium architecture for iOS testing on Sauce)

One of the great features of WebDriver is that you can write your test code in your language of choice, since there are WebDriver client libraries for pretty much every popular language out there. The code you write represents the actions users perform on your application under test, and WebDriver can be used with most test runners and frameworks, so chances are your Appium tests can simply be added to your team’s existing workflows.

And so the real-world benefits of the second two points of the Appium philosophy unfold from there. Because Appium implements the WebDriver API, you don’t need to reinvent that wheel. Because WebDriver is so widely used, you can choose any language and test framework and thus incorporate Appium tests easily into your existing process.

A mobile automation framework should be open source, in spirit and practice as well as in name!

Finally, Appium is proudly Open Source and hosted on GitHub. With over 4800 commits, 2000 issues closed, 3400 pull requests, 1100 forks, 1300 stars and dozens of contributors, Appium is a hotbed of excitement and activity in the open source world. The results are spectacular!

And the fun has only just begun!  Look for Appium to continue to evolve rapidly, guided by its core philosophy, and helping you do what you do best, which is delivering great apps to your users!

- Michael Sage, Principal Technology Evangelist, Sauce Labs

Michael Sage is a Principal Technology Evangelist at Sauce Labs who helps software teams develop, deliver, and care for great apps. He’s spent over 15 years as a solutions architect and consultant with software companies including Mercury Interactive, Hewlett Packard, and New Relic. He lives in San Francisco, CA.

 

Appium + Sauce Labs Integration

August 15th, 2014 by Amber Kaplan

appium_logo‘Appy Friday!

Sauce Labs’ Support Team wants to help our customers easily ramp up their mobile automated testing, so we put together this easy guide to Sauce integration with Appium. We hope you find it to be informative.

If you have any remaining questions about Sauce Labs’ integration with Appium, feel free to leave a note in the comments.

- Cheers, the Support Team at Sauce

 

Background info:

What is Appium?  Appium is an open-source tool that you can use to automate tests for mobile native, mobile web, and mobile hybrid applications. Just like Selenium Webdriver – which is an open-source tool that you can use to to automate web app tests – Appium is an automation library that you can use to automate tests for mobile applications. Other details:

  • It is cross-platform. This allows you to write tests against multiple platforms (iOS, Android), using the same API.
  • It is not locked into a specific language or framework for writing and running your tests.
  • It was created by extending the Selenium Webdriver JSON Wire Protocol with extra API methods that are useful for mobile automation.
  • Mobile Native Applications vs Mobile Web Applications vs Mobile Hybrid Applications

Types of mobile applications:

  • Mobile Native Applications:  Native apps can use all the features from your device, such as the camera, audio, microphone, and more. These applications work on a specific platform (i.e iOS or Android). Use the respective platform Software Development Kit (SDK) to develop them. Once they’re released for public use, users install the application on their respective devices through an application store (e.g. Google Play or Apple’s App Store).
  • Mobile Web Applications: These ones are a little tricky; instead of being an application that you download on your device, mobile web applications are a mobile website that you can access through a browser on your mobile device (e.g. Safari or Chrome). The advantage to creating and using these is that they are not platform-specific, which means that you should be able to access this application from any mobile platform (i.e. iOS or Android). Unfortunately, these applications have limitations when it comes to using a device’s internal features.
  • Mobile Hybrid Applications: As the name indicates, mobile hybrid applications are part mobile native app and part mobile web app. Just as with mobile native applications, you can find and download mobile hybrid applications using an application store (e.g. Google Play or Apple’s App Store), you can access them through icons on your device, and the app is be able to use all of a device’s features, such as the camera, audio, microphone, etc. In the same manner as a mobile web app, a mobile hybrid application looks like a mobile website that can be accessed through a browser, but in this case the browser is an embedded webview within the application that would just allow to display some HTML. A good example of a mobile hybrid app is that from Bank of America. It provides a user with the mobile native application perks while simply rendering pages from their website.

Appium with Sauce Labs

In certain cases, Sauce Labs uses Appium as the server for our latest iOS simulators and Android emulators. This means that whenever you run a test in Sauce Labs specifying any of the following mobile platforms, Appium is the automation tool in the background that starts and drives the simulator or emulator for your test.

Sauce Labs platforms that use Appium

Mobile Native Applications

  • all iOS platforms
  • all Android platforms

Mobile Web Applications (currently only available for iOS)

  • iPhone 6.1
  • iPhone 7.0
  • iPhone 7.1
  • iPad 6.1
  • iPhone 7.0
  • iPhone 7.1

Start testing with Appium + Sauce Labs

 

Additional Resources:

Appium Bootcamp: Want to jumpstart automated mobile testing on Sauce with Appium? Check out this bootcamp series by noted Selenium expert Dave Haeffner:

Chapter 1: Get Started
Chapter 2: The Console
Chapter 3: Interrogate Your App
Chapter 4: Your First Test
Chapter 5: Write And Refactor Your Tests
Chapter 6: Run Your Tests
Chapter 7: Automate Your Tests
Chapter 8: Additional Information

 

Appium Bootcamp – Chapter 8: Additional Information + Resources

August 14th, 2014 by Amber Kaplan

appium_logoThis is the eighth and final post in a series called Appium Bootcamp by noted Selenium expert Dave Haeffner. 

Read:  Chapter 1 Chapter 2 | Chapter 3 | Chapter 4 | Chapter 5 | Chapter 6 | Chapter 7

Dave recently immersed himself in the open source Appium project and collaborated with leading Appium contributor Matthew Edwards to bring us this material. Appium Bootcamp is for those who are brand new to mobile test automation with Appium. No familiarity with Selenium is required, although it may be useful. 

Now that you’re up and running with Appium locally, in the cloud, and on a CI solution, it’s best to show you where you can find more information. Below is a collection of some great resources to help you find your way when it comes to mobile testing.

Community Support

These are the official tutorials for the Appium project for Android and iOS. They served as inspiration and a base for this getting started series. They are great follow-on material since they cover various topics in more depth, and include Java examples as well.

If you have an issue or a question, this is a great place to turn to. Before posting an issue, be sure to read through the Appium Troubleshooting docs and search the group to see if your question has already been asked/answered.

In addition to the Google Discussion Group, you can hop on the Appium HipChat chat room and ask questions from others in the Appium community.

This is a follow-up post answering loads of questions from a webinar from just after thet Appium 1.0 release. It’s chocked full of a lot of great information.

In this video, Jonathan Lipps (Appium’s Chief Architect) explains mobile automation with Appium.

This is an open-source book that is a work in progress; authored by Jonathan Lipps. It’s working title is “Appium: Mobile Automation Made Awesome”.

Some Android Specific Resources

These links (a video, Q&A, and a blog post) cover how Google approaches Android testing.

uiautomator is a crucial component of Android test automation. In this video, the engineers behind it talk about it’s future.

This video is a walk through Google’s newest Android testing framework. This isn’t directly related to Appium, but it contains some useful information.

Some iOS Specific Resources

Appium relies on Apple’s UI Automation support, and these are some solid resources for understanding it better.

Professional Support

If you are a Sauce customer and encounter an issue when using their platform with Appium, be sure to open a support ticket.

If you’re using Appium and you think you’ve found a bug specific to either Android or iOS, then let Google and/or Apple know. In either case it’s best to make sure that the bug is not an Appium issue before filing an issue.

For Google, file an issue here.

For Apple, file an issue here. Apple keeps all bugs private, so it’s worth also filing a duplicate issue here.

Straight To The Source

These are great instructions on how to search through the Appium source code to find more information.

Some Other Resources

There are over 600 Appium questions posted on Stack Overflow for you to peruse.

Xamarin has a free cheat sheet comparing popular mobile app controls. Definitely worth a look.

Outro

Now you’re ready, armed with all the information you need to continue your mobile testing journey.

Happy Testing!

Read:  Chapter 1 Chapter 2 | Chapter 3 | Chapter 4 | Chapter 5 | Chapter 6 | Chapter 7

About Dave Haeffner: Dave is a recent Appium convert and the author of Elemental Selenium (a free, once weekly Selenium tip newsletter that is read by thousands of testing professionals) as well as The Selenium Guidebook (a step-by-step guide on how to use Selenium Successfully). He is also the creator and maintainer of ChemistryKit (an open-source Selenium framework). He has helped numerous companies successfully implement automated acceptance testing; including The Motley Fool, ManTech International, Sittercity, and Animoto. He is a founder and co-organizer of the Selenium Hangout and has spoken at numerous conferences and meetups about acceptance testing.

Follow Dave on Twitter - @tourdedave

Re-Blog: Building Markdown-Based Developer Docs

August 13th, 2014 by Amber Kaplan

Sauce Labs developer Chris Wren and his team have been working tirelessly to improve our documentation system, so we thought we’d share what they’ve been up to. Says Chris:

“Recently at Sauce Labs we decided to retool our documentation system. This decision came after accumulating docs in a number of template systems and repos which were difficult to standardize and maintain. The result of this effort was a new markdown-based docs site available at docs.saucelabs.com.”

For all the details, be sure to check out Chris’ post – just click the image below to view.

Building markdown-based developer docs

Want to contribute to Sauce Labs’ documentation? In the spirit of open source, we’ve housed them in GitHub. Submit away.

Appium Bootcamp – Chapter 7: Automate Your Test Runs

August 12th, 2014 by Amber Kaplan

appium_logoThis is the seventh post in a series called Appium Bootcamp by noted Selenium expert Dave Haeffner.

Read:  Chapter 1 Chapter 2 | Chapter 3 | Chapter 4 | Chapter 5 | Chapter 6 | Chapter 7

Dave recently immersed himself in the open source Appium project and collaborated with leading Appium contributor Matthew Edwards to bring us this material. Appium Bootcamp is for those who are brand new to mobile test automation with Appium. No familiarity with Selenium is required, although it may be useful. This is the seventh of eight posts; two new posts will be released each week.

To make our tests as useful as possible, we’ll want to automate when they get run. To do that, we’ll use a Continuous Integration (CI) Server.

A Continuous Integration Server Primer

A Continous Integration server (a.k.a. CI) is responsible for merging code that is actively being developed into a central place (e.g., “trunk” or “master”) frequently (e.g., several times a day, or on every code commit) to find issues early so they can be addressed quickly — all for the sake of releasing working software in a timely fashion.

With CI, we can automate our test runs so they can happen as part of the development workflow. The lion’s share of tests that are typically run on a CI Server are unit (and potentially integration) tests. But we can very easily add in our automated mobile tests.

There are numerous CI Servers available for use today. Let’s pick one and step through an example.

A CI Example

Jenkins is a fully functional, widely adopted, and open-source CI server. It’s a great candidate for us to step through.

Let’s start by setting it up on the same machine as our Appium Server. Keep in mind that this isn’t the “proper” way to go about this — it’s merely beneficial for this example. To do it right, the Jenkins server (e.g., master node) would live on a machine of its own.

Quick Setup

A simple way to get started is to grab the latest Jenkins war file. You can grab it from the Jenkins homepage, or from this direct download link.

Once downloaded, launch it from your terminal.

java -jar /path/to/jenkins.war

 

You will now be able to use Jenkins by visiting http://localhost:8080/ in your browser.

Running Tests Locally

After loading Jenkins in the browser, we’ll create a Job and configure it to run our Appium tests. Let’s start with the Android tests first.

  1. Click New Item in the top-left corner
  2. Type a name into the Item name input field (e.g., Appium Android)
  3. Select Build a free-style software project
  4. Click OK

This will load a configuration screen for the Jenkins Job.

 

  1. Scroll down until you reach the Build section (near the bottom of the page)
  2. Click Add build step
  3. Select Execute shell
  4. Input the following into the Command input box
cd /path/to/your/appium/test/code
bundle update
rake android

In this set of commands we are telling Jenkins to change directories to our test code, make sure we have the necessary libraries, and then launch the Android tests.

Click Save at the bottom of the page, make sure your Appium Server is running (if not, load up the Appium GUI and click Launch), and click Build Now on the left-hand side of the Jenkins Job screen.

Once it’s running, you can click on the job under Build History, and then click Console Output (from the left-hand panel). In it, you should see something similar to this:

Started by user anonymous
Building in workspace /Users/tourdedave/.jenkins/jobs/Appium Android/workspace
[workspace] $ /bin/sh -xe /var/folders/yt/h7v9k6px7jl68q81c9sqrd9h0000gn/T/hudson6140596697737249507.sh
+ cd /Users/tourdedave/Dropbox/_dev/appium/appium-getting-started/code-examples/7/1
+ bundle update
Fetching gem metadata from https://rubygems.org/...........
Fetching additional metadata from https://rubygems.org/..
Resolving dependencies...
Using rake 10.3.2
Using awesome_print 1.2.0
Using json 1.8.1
Using mini_portile 0.6.0
Using nokogiri 1.6.3.1
Using ffi 1.9.3
Using childprocess 0.5.3
Using multi_json 1.10.1
Using rubyzip 1.1.6
Using websocket 1.0.7
Using selenium-webdriver 2.42.0
Using blankslate 2.1.2.4
Using parslet 1.5.0
Using toml 0.1.1
Using appium_lib 4.0.0
Using bond 0.5.1
Using coderay 1.1.0
Using method_source 0.8.2
Using slop 3.6.0
Using pry 0.9.12.6
Using numerizer 0.1.1
Using chronic_duration 0.10.5
Using spec 5.3.4
Using appium_console 1.0.1
Using diff-lcs 1.2.5
Using mime-types 1.25.1
Using rdoc 4.1.1
Using rest-client 1.6.8
Using rspec-support 3.0.3
Using rspec-core 3.0.3
Using rspec-expectations 3.0.3
Using rspec-mocks 3.0.3
Using rspec 3.0.0
Using sauce_whisk 0.0.13
Using bundler 1.6.2
Your bundle is updated!
+ rake android
.

Finished in 38.39 seconds (files took 1.52 seconds to load)
1 example, 0 failures
Finished: SUCCESS

Making Sure We Have A Clean Finish

We now have a working job in Jenkins. But we’re not there yet. While the job was runnning you should have seen the Android Emulator open, load the test app, and perform the test actions. Unfortunately, after the job completed, the emulator didn’t close.

Closing the Android Emulator is something that Appium doesn’t handle, so we’ll need to account for this in our Jenkins build configuration. Otherwise, we won’t leave things in a clean state for future test runs.

The simplest way to close the emulator is by issuing a kill command against the name of the process (ensuring that the command always returns true). That way we cover our bases in case there is more than one emulator process running or if we try to kill a process that doesn’t exist. So let’s go ahead and add the kill command to our existing commands under the Build section of our job. For good measure, let’s add it before and after our test execution commands.

To get back to the job configuration screen, click Configure from the main job screen.

killall -9 emulator64-x86 || true

cd /path/to/your/appium/test/code
bundle update
rake android

killall -9 emulator64-x86 || true

Now let’s save the job and build it again. The job will run just like before, but now the emulator will close after the test run completes.

Creating Another Job

Now let’s create a second job to run our tests against iOS.

To save a step, let’s create a copy of our existing job and modify the build commands as needed.

  1. Click the Jenkins logo at the top of the screen (it will take you to the main page)
  2. Click New Item in the top-left corner
  3. Type a name into the Item name input field (e.g., Appium iOS)
  4. Select Copy existing Item
  5. Start to type in the name of the other job in the Copy from input field (e.g., Appium Android)
  6. Select the job from the drop-down as it appears
  7. Click OK

 

This will take us to a configuration screen for the new (copied) job. Let’s scroll down to the Build section and modify the Command input field under Execute Shell.

killall -9 "iPhone Simulator" &> /dev/null || true
killall -9 instruments &> /dev/null || true

cd /path/to/your/appium/test/code
bundle update
rake ios

killall -9 "iPhone Simulator" &> /dev/null || true
killall -9 instruments &> /dev/null || true

Similar to the Android job, we’re using kill to end a process (in this case two processes) and making sure the command returns true if it doesn’t exist. This protects us in the event that the test suite doesn’t complete as planned (leaving a simulator around) or if the simulator doesn’t close instruments cleanly (which can happen).

If we save this and build it, then we will see the iPhone Simulator load, launch the app, run the tests, and then close the simulator.

Running Tests On Sauce

We’ve covered running things locally on the CI server, now let’s create a job to run our tests on Sauce.

Let’s create another copy of the Appium Android job and modify the build commands.

Since we’re not going to be running locally, we can remove the kill line. We’ll then specify our Sauce credentials (through environment variables) and update the rake command to specify 'sauce' as a location. When we’re done, our Command window should look like this:

export SAUCE_USERNAME=your-username
export SAUCE_ACCESS_KEY=your-access-key

cd /path/to/your/appium/test/code
bundle update
rake android['sauce']

If we save this and build it, our tests will now run on Sauce Labs. And you can view them as they happen on your Sauce Labs Account Page.

An iOS job would be identical to this, except for the job name (e.g., Appium iOS Sauce) and the rake incantation (which would be rake ios['sauce'].

Outro

Now that we have our Appium tests wired up for automatic execution, we’re now able to configure them to run based on various triggers (e.g., other CI jobs, a schedule, etc.). Find what works for you and your development team’s workflow, and make it happen.

Read:  Chapter 1 Chapter 2 | Chapter 3 | Chapter 4 | Chapter 5 | Chapter 6 | Chapter 7 | Chapter 8

About Dave Haeffner: Dave is a recent Appium convert and the author of Elemental Selenium (a free, once weekly Selenium tip newsletter that is read by thousands of testing professionals) as well as The Selenium Guidebook (a step-by-step guide on how to use Selenium Successfully). He is also the creator and maintainer of ChemistryKit (an open-source Selenium framework). He has helped numerous companies successfully implement automated acceptance testing; including The Motley Fool, ManTech International, Sittercity, and Animoto. He is a founder and co-organizer of the Selenium Hangout and has spoken at numerous conferences and meetups about acceptance testing.

Follow Dave on Twitter - @tourdedave