Appium + Sauce Labs Integration

August 15th, 2014 by Amber Kaplan

appium_logo‘Appy Friday!

Sauce Labs’ Support Team wants to help our customers easily ramp up their mobile automated testing, so we put together this easy guide to Sauce integration with Appium. We hope you find it to be informative.

If you have any remaining questions about Sauce Labs’ integration with Appium, feel free to leave a note in the comments.

- Cheers, the Support Team at Sauce

 

Background info:

What is Appium?  Appium is an open-source tool that you can use to automate tests for mobile native, mobile web, and mobile hybrid applications. Just like Selenium Webdriver – which is an open-source tool that you can use to to automate web app tests – Appium is an automation library that you can use to automate tests for mobile applications. Other details:

  • It is cross-platform. This allows you to write tests against multiple platforms (iOS, Android), using the same API.
  • It is not locked into a specific language or framework for writing and running your tests.
  • It was created by extending the Selenium Webdriver JSON Wire Protocol with extra API methods that are useful for mobile automation.
  • Mobile Native Applications vs Mobile Web Applications vs Mobile Hybrid Applications

Types of mobile applications:

  • Mobile Native Applications:  Native apps can use all the features from your device, such as the camera, audio, microphone, and more. These applications work on a specific platform (i.e iOS or Android). Use the respective platform Software Development Kit (SDK) to develop them. Once they’re released for public use, users install the application on their respective devices through an application store (e.g. Google Play or Apple’s App Store).
  • Mobile Web Applications: These ones are a little tricky; instead of being an application that you download on your device, mobile web applications are a mobile website that you can access through a browser on your mobile device (e.g. Safari or Chrome). The advantage to creating and using these is that they are not platform-specific, which means that you should be able to access this application from any mobile platform (i.e. iOS or Android). Unfortunately, these applications have limitations when it comes to using a device’s internal features.
  • Mobile Hybrid Applications: As the name indicates, mobile hybrid applications are part mobile native app and part mobile web app. Just as with mobile native applications, you can find and download mobile hybrid applications using an application store (e.g. Google Play or Apple’s App Store), you can access them through icons on your device, and the app is be able to use all of a device’s features, such as the camera, audio, microphone, etc. In the same manner as a mobile web app, a mobile hybrid application looks like a mobile website that can be accessed through a browser, but in this case the browser is an embedded webview within the application that would just allow to display some HTML. A good example of a mobile hybrid app is that from Bank of America. It provides a user with the mobile native application perks while simply rendering pages from their website.

Appium with Sauce Labs

In certain cases, Sauce Labs uses Appium as the server for our latest iOS simulators and Android emulators. This means that whenever you run a test in Sauce Labs specifying any of the following mobile platforms, Appium is the automation tool in the background that starts and drives the simulator or emulator for your test.

Sauce Labs platforms that use Appium

Mobile Native Applications

  • all iOS platforms
  • all Android platforms

Mobile Web Applications (currently only available for iOS)

  • iPhone 6.1
  • iPhone 7.0
  • iPhone 7.1
  • iPad 6.1
  • iPhone 7.0
  • iPhone 7.1

Start testing with Appium + Sauce Labs

 

Additional Resources:

Appium Bootcamp: Want to jumpstart automated mobile testing on Sauce with Appium? Check out this bootcamp series by noted Selenium expert Dave Haeffner:

Chapter 1: Get Started
Chapter 2: The Console
Chapter 3: Interrogate Your App
Chapter 4: Your First Test
Chapter 5: Write And Refactor Your Tests
Chapter 6: Run Your Tests
Chapter 7: Automate Your Tests
Chapter 8: Additional Information

 

Appium Bootcamp – Chapter 8: Additional Information + Resources

August 14th, 2014 by Amber Kaplan

appium_logoThis is the eighth and final post in a series called Appium Bootcamp by noted Selenium expert Dave Haeffner. 

Read:  Chapter 1 Chapter 2 | Chapter 3 | Chapter 4 | Chapter 5 | Chapter 6 | Chapter 7

Dave recently immersed himself in the open source Appium project and collaborated with leading Appium contributor Matthew Edwards to bring us this material. Appium Bootcamp is for those who are brand new to mobile test automation with Appium. No familiarity with Selenium is required, although it may be useful. 

Now that you’re up and running with Appium locally, in the cloud, and on a CI solution, it’s best to show you where you can find more information. Below is a collection of some great resources to help you find your way when it comes to mobile testing.

Community Support

These are the official tutorials for the Appium project for Android and iOS. They served as inspiration and a base for this getting started series. They are great follow-on material since they cover various topics in more depth, and include Java examples as well.

If you have an issue or a question, this is a great place to turn to. Before posting an issue, be sure to read through the Appium Troubleshooting docs and search the group to see if your question has already been asked/answered.

In addition to the Google Discussion Group, you can hop on the Appium HipChat chat room and ask questions from others in the Appium community.

This is a follow-up post answering loads of questions from a webinar from just after thet Appium 1.0 release. It’s chocked full of a lot of great information.

In this video, Jonathan Lipps (Appium’s Chief Architect) explains mobile automation with Appium.

This is an open-source book that is a work in progress; authored by Jonathan Lipps. It’s working title is “Appium: Mobile Automation Made Awesome”.

Some Android Specific Resources

These links (a video, Q&A, and a blog post) cover how Google approaches Android testing.

uiautomator is a crucial component of Android test automation. In this video, the engineers behind it talk about it’s future.

This video is a walk through Google’s newest Android testing framework. This isn’t directly related to Appium, but it contains some useful information.

Some iOS Specific Resources

Appium relies on Apple’s UI Automation support, and these are some solid resources for understanding it better.

Professional Support

If you are a Sauce customer and encounter an issue when using their platform with Appium, be sure to open a support ticket.

If you’re using Appium and you think you’ve found a bug specific to either Android or iOS, then let Google and/or Apple know. In either case it’s best to make sure that the bug is not an Appium issue before filing an issue.

For Google, file an issue here.

For Apple, file an issue here. Apple keeps all bugs private, so it’s worth also filing a duplicate issue here.

Straight To The Source

These are great instructions on how to search through the Appium source code to find more information.

Some Other Resources

There are over 600 Appium questions posted on Stack Overflow for you to peruse.

Xamarin has a free cheat sheet comparing popular mobile app controls. Definitely worth a look.

Outro

Now you’re ready, armed with all the information you need to continue your mobile testing journey.

Happy Testing!

Read:  Chapter 1 Chapter 2 | Chapter 3 | Chapter 4 | Chapter 5 | Chapter 6 | Chapter 7

About Dave Haeffner: Dave is a recent Appium convert and the author of Elemental Selenium (a free, once weekly Selenium tip newsletter that is read by thousands of testing professionals) as well as The Selenium Guidebook (a step-by-step guide on how to use Selenium Successfully). He is also the creator and maintainer of ChemistryKit (an open-source Selenium framework). He has helped numerous companies successfully implement automated acceptance testing; including The Motley Fool, ManTech International, Sittercity, and Animoto. He is a founder and co-organizer of the Selenium Hangout and has spoken at numerous conferences and meetups about acceptance testing.

Follow Dave on Twitter - @tourdedave

Re-Blog: Building Markdown-Based Developer Docs

August 13th, 2014 by Amber Kaplan

Sauce Labs developer Chris Wren and his team have been working tirelessly to improve our documentation system, so we thought we’d share what they’ve been up to. Says Chris:

“Recently at Sauce Labs we decided to retool our documentation system. This decision came after accumulating docs in a number of template systems and repos which were difficult to standardize and maintain. The result of this effort was a new markdown-based docs site available at docs.saucelabs.com.”

For all the details, be sure to check out Chris’ post – just click the image below to view.

Building markdown-based developer docs

Want to contribute to Sauce Labs’ documentation? In the spirit of open source, we’ve housed them in GitHub. Submit away.

Appium Bootcamp – Chapter 7: Automate Your Test Runs

August 12th, 2014 by Amber Kaplan

appium_logoThis is the seventh post in a series called Appium Bootcamp by noted Selenium expert Dave Haeffner.

Read:  Chapter 1 Chapter 2 | Chapter 3 | Chapter 4 | Chapter 5 | Chapter 6 | Chapter 7

Dave recently immersed himself in the open source Appium project and collaborated with leading Appium contributor Matthew Edwards to bring us this material. Appium Bootcamp is for those who are brand new to mobile test automation with Appium. No familiarity with Selenium is required, although it may be useful. This is the seventh of eight posts; two new posts will be released each week.

To make our tests as useful as possible, we’ll want to automate when they get run. To do that, we’ll use a Continuous Integration (CI) Server.

A Continuous Integration Server Primer

A Continous Integration server (a.k.a. CI) is responsible for merging code that is actively being developed into a central place (e.g., “trunk” or “master”) frequently (e.g., several times a day, or on every code commit) to find issues early so they can be addressed quickly — all for the sake of releasing working software in a timely fashion.

With CI, we can automate our test runs so they can happen as part of the development workflow. The lion’s share of tests that are typically run on a CI Server are unit (and potentially integration) tests. But we can very easily add in our automated mobile tests.

There are numerous CI Servers available for use today. Let’s pick one and step through an example.

A CI Example

Jenkins is a fully functional, widely adopted, and open-source CI server. It’s a great candidate for us to step through.

Let’s start by setting it up on the same machine as our Appium Server. Keep in mind that this isn’t the “proper” way to go about this — it’s merely beneficial for this example. To do it right, the Jenkins server (e.g., master node) would live on a machine of its own.

Quick Setup

A simple way to get started is to grab the latest Jenkins war file. You can grab it from the Jenkins homepage, or from this direct download link.

Once downloaded, launch it from your terminal.

java -jar /path/to/jenkins.war

 

You will now be able to use Jenkins by visiting http://localhost:8080/ in your browser.

Running Tests Locally

After loading Jenkins in the browser, we’ll create a Job and configure it to run our Appium tests. Let’s start with the Android tests first.

  1. Click New Item in the top-left corner
  2. Type a name into the Item name input field (e.g., Appium Android)
  3. Select Build a free-style software project
  4. Click OK

This will load a configuration screen for the Jenkins Job.

 

  1. Scroll down until you reach the Build section (near the bottom of the page)
  2. Click Add build step
  3. Select Execute shell
  4. Input the following into the Command input box
cd /path/to/your/appium/test/code
bundle update
rake android

In this set of commands we are telling Jenkins to change directories to our test code, make sure we have the necessary libraries, and then launch the Android tests.

Click Save at the bottom of the page, make sure your Appium Server is running (if not, load up the Appium GUI and click Launch), and click Build Now on the left-hand side of the Jenkins Job screen.

Once it’s running, you can click on the job under Build History, and then click Console Output (from the left-hand panel). In it, you should see something similar to this:

Started by user anonymous
Building in workspace /Users/tourdedave/.jenkins/jobs/Appium Android/workspace
[workspace] $ /bin/sh -xe /var/folders/yt/h7v9k6px7jl68q81c9sqrd9h0000gn/T/hudson6140596697737249507.sh
+ cd /Users/tourdedave/Dropbox/_dev/appium/appium-getting-started/code-examples/7/1
+ bundle update
Fetching gem metadata from https://rubygems.org/...........
Fetching additional metadata from https://rubygems.org/..
Resolving dependencies...
Using rake 10.3.2
Using awesome_print 1.2.0
Using json 1.8.1
Using mini_portile 0.6.0
Using nokogiri 1.6.3.1
Using ffi 1.9.3
Using childprocess 0.5.3
Using multi_json 1.10.1
Using rubyzip 1.1.6
Using websocket 1.0.7
Using selenium-webdriver 2.42.0
Using blankslate 2.1.2.4
Using parslet 1.5.0
Using toml 0.1.1
Using appium_lib 4.0.0
Using bond 0.5.1
Using coderay 1.1.0
Using method_source 0.8.2
Using slop 3.6.0
Using pry 0.9.12.6
Using numerizer 0.1.1
Using chronic_duration 0.10.5
Using spec 5.3.4
Using appium_console 1.0.1
Using diff-lcs 1.2.5
Using mime-types 1.25.1
Using rdoc 4.1.1
Using rest-client 1.6.8
Using rspec-support 3.0.3
Using rspec-core 3.0.3
Using rspec-expectations 3.0.3
Using rspec-mocks 3.0.3
Using rspec 3.0.0
Using sauce_whisk 0.0.13
Using bundler 1.6.2
Your bundle is updated!
+ rake android
.

Finished in 38.39 seconds (files took 1.52 seconds to load)
1 example, 0 failures
Finished: SUCCESS

Making Sure We Have A Clean Finish

We now have a working job in Jenkins. But we’re not there yet. While the job was runnning you should have seen the Android Emulator open, load the test app, and perform the test actions. Unfortunately, after the job completed, the emulator didn’t close.

Closing the Android Emulator is something that Appium doesn’t handle, so we’ll need to account for this in our Jenkins build configuration. Otherwise, we won’t leave things in a clean state for future test runs.

The simplest way to close the emulator is by issuing a kill command against the name of the process (ensuring that the command always returns true). That way we cover our bases in case there is more than one emulator process running or if we try to kill a process that doesn’t exist. So let’s go ahead and add the kill command to our existing commands under the Build section of our job. For good measure, let’s add it before and after our test execution commands.

To get back to the job configuration screen, click Configure from the main job screen.

killall -9 emulator64-x86 || true

cd /path/to/your/appium/test/code
bundle update
rake android

killall -9 emulator64-x86 || true

Now let’s save the job and build it again. The job will run just like before, but now the emulator will close after the test run completes.

Creating Another Job

Now let’s create a second job to run our tests against iOS.

To save a step, let’s create a copy of our existing job and modify the build commands as needed.

  1. Click the Jenkins logo at the top of the screen (it will take you to the main page)
  2. Click New Item in the top-left corner
  3. Type a name into the Item name input field (e.g., Appium iOS)
  4. Select Copy existing Item
  5. Start to type in the name of the other job in the Copy from input field (e.g., Appium Android)
  6. Select the job from the drop-down as it appears
  7. Click OK

 

This will take us to a configuration screen for the new (copied) job. Let’s scroll down to the Build section and modify the Command input field under Execute Shell.

killall -9 "iPhone Simulator" &> /dev/null || true
killall -9 instruments &> /dev/null || true

cd /path/to/your/appium/test/code
bundle update
rake ios

killall -9 "iPhone Simulator" &> /dev/null || true
killall -9 instruments &> /dev/null || true

Similar to the Android job, we’re using kill to end a process (in this case two processes) and making sure the command returns true if it doesn’t exist. This protects us in the event that the test suite doesn’t complete as planned (leaving a simulator around) or if the simulator doesn’t close instruments cleanly (which can happen).

If we save this and build it, then we will see the iPhone Simulator load, launch the app, run the tests, and then close the simulator.

Running Tests On Sauce

We’ve covered running things locally on the CI server, now let’s create a job to run our tests on Sauce.

Let’s create another copy of the Appium Android job and modify the build commands.

Since we’re not going to be running locally, we can remove the kill line. We’ll then specify our Sauce credentials (through environment variables) and update the rake command to specify 'sauce' as a location. When we’re done, our Command window should look like this:

export SAUCE_USERNAME=your-username
export SAUCE_ACCESS_KEY=your-access-key

cd /path/to/your/appium/test/code
bundle update
rake android['sauce']

If we save this and build it, our tests will now run on Sauce Labs. And you can view them as they happen on your Sauce Labs Account Page.

An iOS job would be identical to this, except for the job name (e.g., Appium iOS Sauce) and the rake incantation (which would be rake ios['sauce'].

Outro

Now that we have our Appium tests wired up for automatic execution, we’re now able to configure them to run based on various triggers (e.g., other CI jobs, a schedule, etc.). Find what works for you and your development team’s workflow, and make it happen.

Read:  Chapter 1 Chapter 2 | Chapter 3 | Chapter 4 | Chapter 5 | Chapter 6 | Chapter 7 | Chapter 8

About Dave Haeffner: Dave is a recent Appium convert and the author of Elemental Selenium (a free, once weekly Selenium tip newsletter that is read by thousands of testing professionals) as well as The Selenium Guidebook (a step-by-step guide on how to use Selenium Successfully). He is also the creator and maintainer of ChemistryKit (an open-source Selenium framework). He has helped numerous companies successfully implement automated acceptance testing; including The Motley Fool, ManTech International, Sittercity, and Animoto. He is a founder and co-organizer of the Selenium Hangout and has spoken at numerous conferences and meetups about acceptance testing.

Follow Dave on Twitter - @tourdedave

Announcing CircleCI Integration on Sauce Labs

August 11th, 2014 by Amber Kaplan

Last week our friends at CircleCI showed us how to securely run cross-browser front-end tests on the Sauce Labs cloud using their hosted Continuous Integration service. We’ve long been advocates of good continuous integration practices and have developed a few plugins for some of the more common CI servers out there. We’re super excited to add CircleCI to our list and even more excited about how easy it is to get it going!

Continuous Integration in the Cloud

Continuous Integration, if you don’t already know, is the process of building your app frequently from a shared source code repository and running tests to make sure everything works. If your tests don’t pass and the build is not successful, the code that was checked in since the last good build is where the defects were introduced, and so problems are much easier to find and fix quickly.

Maintaining a local CI server can be a hassle. Anyone who’s spent any considerable time configuring Jenkins jobs with all it’s various plugins and tasks can tell you all about it. CircleCI, on the other hand, integrates directly with GitHub and can actually *infer* necessary settings directly from your code (if you’re following good development practices for that language and environment) and so many projects just magically build themselves on CircleCI without any additional configuration on your part. It’s like three clicks from zero to CI. Pretty sweet! If you do need to tweak or customize any settings, you can easily do so by describing those settings in a circle.yml file placed in your repo.

Running Tests on Sauce Labs Browsers

Sauce Labs is the world’s largest cross-browser client cloud. We host over 375 different configurations of operating systems and browsers so you can ensure that your app works on all the specific platforms and versions you need to support. These days that’s an ever-growing list! So it makes sense to run these tests with your continuous integration process so you know things work across the board and you don’t end up spending a bunch of time and trouble trying to hunt down bugs that were introduced much earlier in the development cycle.

Now, if your build deploys your code to a publicly accessible environment, CircleCI will simply execute your Selenium tests and you probably won’t need to configure anything, since Sauce Labs browsers will be able connect to that environment over the public network. However, if you want CircleCI to execute your tests locally in it’s build containers, you’ll need to use Sauce Connect.

Sauce Connect is a secure tunneling utility which allows you to execute tests behind firewalls via an encrypted connection between Sauce Labs and your testing environment. When you run Sauce Connect yourself, you typically do it from a command line or shell script and supply your Sauce Labs account credentials as parameters. With a CircleCI build, you’ll need to set it up in the circle.yml file so it can be launched as part of the build process and those tests can run inside the local container.

All that’s really involved here is telling the build task where to find Sauce Connect and how to start it up with your account credentials.

The first part is downloading and unpacking the Sauce Connect file, which you add as a custom dependency entry in your circle.yml:

	dependencies:
	  post:
		   - wget https://saucelabs.com/downloads/sc-latest-linux.tar.gz
		   - tar -xzf sc-latest-linux.tar.gz

The second part is to add your credentials, launch the tunnel, and check that it’s running before kicking off the tests. You’ll put these lines in the `test` section of circle.yml:

test:
     override:
        - ./bin/sc -u $SAUCE_USERNAME -k $SAUCE_ACCESS_KEY -f ~/sc_ready:
            background: true
            pwd: sc-*-linux
        # Wait for tunnel to be ready
        - while [ ! -e ~/sc_ready ]; do sleep 1; done

That’s all there is to it. You can find out the details here and see a full example on GitHub. And CircleCI has a nice little utility to help you add your credentials as environment variables so that they are not visible as plain text in the repo.

With CircleCI tackling all the necessary work involved in supporting your continuous integration process and Sauce Labs hosting the world’s most extensive cross-browser client cloud, you can be free of the costs and hassles of managing all these systems and grids and get back to focusing on the business of making great software.

- Michael Sage, Principal Technology Evanglist, Sauce Labs

Michael Sage is a Principal Technology Evangelist at Sauce Labs. He helps software teams develop, deliver, and care for great apps. He’s lived through two or three technology revolutions and expects a few more to come, and finds the prospect incredibly exciting. A native of Philadelphia, he’s made San Francisco his home for over 25 years, but still can’t find a good hoagie there.

Recap: Fearless Browser Test Automation [WEBINAR]

August 8th, 2014 by Amber Kaplan

john_david_daltonThanks to those of you who attended our last webinar, Fearless Browser Test Automation, featuring John-David Dalton. This webinar was presented by O’Reilly and Sauce Labs, a provider of the world’s largest automation cloud for testing web and native/hybrid mobile applications.

We hope you found John-David’s perspectives helpful, and that if you’re now doing manual testing on a limited range of browsers – or no testing at all – you’re ready for the awesomeness of automated cross-browser testing.

Missed the webinar? You can watch it in its entirety HERE.

Still scared? Never fear: you can get more tips and tools at the Sauce Labs Documentation Center. If you’re new to JavaScript testing, here are some resources to get you started.

Lastly, please follow our friends at O’Reilly at @oreillymedia, Sauce Labs at @saucelabs, and John-David at @jdalton to keep up with the latest, and feel free to share this webinar using the hashtag #fearlesstesting.

Appium Bootcamp – Chapter 6: Run Your Tests

August 7th, 2014 by Amber Kaplan

appium_logoThis is the sixth post in a series called Appium Bootcamp by noted Selenium expert Dave HaeffnerRead:  Chapter 1 Chapter 2 | Chapter 3 | Chapter 4 | Chapter 5 | Chapter 6 | Chapter 7 | Chapter 8

Dave recently immersed himself in the open source Appium project and collaborated with leading Appium contributor Matthew Edwards to bring us this material. Appium Bootcamp is for those who are brand new to mobile test automation with Appium. No familiarity with Selenium is required, although it may be useful. This is the sixth of eight posts; two new posts will be released each week.

Now that we have our tests written, refactored, and running locally it’s time to make them simple to launch by wrapping them with a command-line executor. After that, we’ll be able to easily add in the ability to run them in the cloud.

Quick Setup

appium_lib comes pre-wired with the ability to run our tests in Sauce Labs, but we’re still going to need two additional libraries to accomplish everything; rake for command-line execution, and sauce_whisk for some additional tasks not covered by appium_lib.

Let’s add these to our Gemfile and run bundle install.

# filename: Gemfile

source 'https://rubygems.org'

gem 'rspec', '~> 3.0.0'
gem 'appium_lib', '~> 4.0.0'
gem 'appium_console', '~> 1.0.1'
gem 'rake', '~> 10.3.2'
gem 'sauce_whisk', '~> 0.0.13'

Simple Rake Tasks

Now that we have our requisite libraries let’s create a new file in the project root called Rakefile and add tasks to launch our tests.

# filename: Rakefile

desc 'Run iOS tests'
task :ios do
  Dir.chdir 'ios'
  exec 'rspec'
end

desc 'Run Android tests'
task :android do
  Dir.chdir 'android'
  exec 'rspec'
end

Notice that the syntax in this file reads a lot like Ruby — that’s because it is (along with some Rake specific syntax). For a primer on Rake, read this.

In this file we’ve created two tasks. One to run our iOS tests, and another for the Android tests. Each task changes directories into the correct device folder (e.g., Dir.chdir) and then launches the tests (e.g., exec 'rspec').

If we save this file and run rake -T from the command-line, we will see these tasks listed along with their descriptions.

> rake -T
rake android  # Run Android tests
rake ios      # Run iOS tests

If we run either of these tasks (e.g., rake android or rake ios), they will execute the tests locally for each of the devices.

Running Your Tests In Sauce

As I mentioned before, appium_lib comes with the ability to run Appium tests in Sauce Labs. We just need to specify a Sauce account username and access key. To obtain an access key, you first need to have an account (if you don’t have one you can create a free trial one here). After that, log into the account and go to the bottom left of your dashboard; your access key will be listed there.

We’ll also need to make our apps available to Sauce. This can be accomplished by either uploading the app to Sauce, or, making the app available from a publicly available URL. The prior approach is easy enough to accomplish with the help of sauce_whisk.

Let’s go ahead and update our spec_helper.rb to add in this new upload capability (along with a couple of other bits).

# filename: common/spec_helper.rb

require 'rspec'
require 'appium_lib'
require 'sauce_whisk'

def using_sauce
  user = ENV['SAUCE_USERNAME']
  key  = ENV['SAUCE_ACCESS_KEY']
  user && !user.empty? && key && !key.empty?
end

def upload_app
  storage = SauceWhisk::Storage.new
  app = @caps[:caps][:app]
  storage.upload app

  @caps[:caps][:app] = "sauce-storage:#{File.basename(app)}"
end

def setup_driver
  return if $driver
  @caps = Appium.load_appium_txt file: File.join(Dir.pwd, 'appium.txt')
  if using_sauce
    upload_app
    @caps[:caps].delete :avd # re: https://github.com/appium/ruby_lib/issues/241
  end
  Appium::Driver.new @caps
end

def promote_methods
  Appium.promote_singleton_appium_methods Pages
  Appium.promote_appium_methods RSpec::Core::ExampleGroup
end

setup_driver
promote_methods

RSpec.configure do |config|

  config.before(:each) do
    $driver.start_driver
  end

  config.after(:each) do
    driver_quit
  end

end

Near the top of the file we pull in sauce_whisk. We then add in a couple of helper methods (using_sauce and upload_app). using_sauce checks to see if Sauce credentials have been set properly. upload_app uploads the application from local disk and then updates the capabilities to reference the path to the app on Sauce’s storage.

We put these to use in setup_driver by wrapping them in a conditional to see if we are using Sauce. If so, we upload the app. We’re also removing the avd capability since it will cause issues with our Sauce run if we keep it in.

Next we’ll need to update our appium.txt files so they’ll play nice with Sauce.

 

# filename: android/appium.txt

[caps]
appium-version = "1.2.0"
deviceName = "Android"
platformName = "Android"
platformVersion = "4.3"
app = "../../../apps/api.apk"
avd = "training"

[appium_lib]
require = ["./spec/requires.rb"]
# filename: ios/appium.txt

[caps]
appium-version = "1.2.0"
deviceName = "iPhone Simulator"
platformName = "ios"
platformVersion = "7.1"
app = "../../../apps/UICatalog.app.zip"

[appium_lib]
require = ["./spec/requires.rb"]

In order to work with Sauce we need to specify the appium-version and the platformVersion. Everything else stays the same. You can see a full list of Sauce’s supported platforms and configuration options here.

Now let’s update our Rake tasks to be cloud aware. That way we can specify at run time whether to run things locally or in Sauce.

desc 'Run iOS tests'
task :ios, :location do |t, args|
  location_helper args[:location]
  Dir.chdir 'ios'
  exec 'rspec'
end

desc 'Run Android tests'
task :android, :location do |t, args|
  location_helper args[:location]
  Dir.chdir 'android'
  exec 'rspec'
end

def location_helper(location)
  if location != 'sauce'
    ENV['SAUCE_USERNAME'], ENV['SAUCE_ACCESS_KEY'] = nil, nil
  end
end

We’ve updated our Rake tasks so they can take an argument for the location. We then use this argument value and pass it to location_helper. The location_helper looks at the location value — if it is not set to 'sauce'then the Sauce credentials get set to nil. This helps us ensure that we really do want to run our tests on Sauce (e.g., we have to specify both the Sauce credentials AND the location).

Now we can launch our tests locally just like before (e.g., rake ios) or in Sauce by specifying it as a location (e.g., rake ios['sauce'])

But in order for the tests to fire in Sauce Labs, we need to specify our credentials somehow. We’ve opted to keep them out of our Rakefile (and our test code) so that we can maintain future flexibility by not having them hard-coded; which is also more secure since we won’t be committing them to our repository.

Specifying Sauce Credentials

There are a few ways we can go about specifying our credentials.

Specify them at run-time

SAUCE_USERNAME=your-username SAUCE_ACCESS_KEY=your-access-key rake ios['sauce']

Export the values into the current command-line session

export SAUCE_USERNAME=your-username
export SAUCE_ACCESS_KEY=your-access-key

Set the values in your bash profile (recommended)

# filename: ~/*.bash_profile

...
export SAUCE_USERNAME=your-username
export SAUCE_ACCESS_KEY=your-access-key

After choosing a method for specifying your credentials, run your tests with one of the Rake task and specify 'sauce' for the location. Then log into your Sauce Account to see the test results and a video of the execution.

Making Your Sauce Runs Descriptive

It’s great that our tests are now running in Sauce. But it’s tough to sift through the test results since the name and test status are nondescript and all the same. Let’s fix that.

Fortunately, we can dynamically set the Sauce Labs job name and test status in our test code. We just need to provide this information before and after our test runs. To do that we’ll need to update the RSpec configuration incommon/spec_helper.rb.

 

# filename: common/spec_helper.rb

...
RSpec.configure do |config|

  config.before(:each) do |example|
    $driver.caps[:name] = example.metadata[:full_description] if using_sauce
    $driver.start_driver
  end

  config.after(:each) do |example|
    if using_sauce
      SauceWhisk::Jobs.change_status $driver.driver.session_id, example.exception.nil?
    end
    driver_quit
  end

end

In before(:each) we update the name attribute of our capabilities (e.g., caps[:name]) with the name of the test. We get this name by tapping into the test’s metadata (e.g., example.metadata[:full_description]). And since we only want this to run if we’re using Sauce we wrap it in a conditional.

In after(:each) we leverage sauce_whisk to set the job status based on the test result, which we get by checking to see if any exceptions were raised. Again, we only want this to run if we’re using Sauce, so we wrap it in a conditional too.

Now if we run our tests in Sauce we will see them execute with the correct name and job status.

Outro

Now that we have local and cloud execution covered, it’s time to automate our test runs by plugging them into a Continuous Integration (CI) server.

Read:  Chapter 1 Chapter 2 | Chapter 3 | Chapter 4 | Chapter 5 | Chapter 6 | Chapter 7 | Chapter 8

About Dave Haeffner: Dave is a recent Appium convert and the author of Elemental Selenium (a free, once weekly Selenium tip newsletter that is read by thousands of testing professionals) as well as The Selenium Guidebook (a step-by-step guide on how to use Selenium Successfully). He is also the creator and maintainer of ChemistryKit (an open-source Selenium framework). He has helped numerous companies successfully implement automated acceptance testing; including The Motley Fool, ManTech International, Sittercity, and Animoto. He is a founder and co-organizer of the Selenium Hangout and has spoken at numerous conferences and meetups about acceptance testing.

Follow Dave on Twitter - @tourdedave

Cloud9 + Sauce Labs Integration: Learn How It Works [WEBINAR]

August 6th, 2014 by Amber Kaplan

Sauce + C9 IntegrationEver wanted to develop and test applications directly from your browser? Cloud9 enables users to do just this using their powerful cloud-based development environment. With their recent release and new integration with Sauce Labs, users can now instantly test mobile and web apps across any browser that Sauce Labs supports – without ever leaving the Cloud9 interface.

Join us for our latest webinar showcasing the integration on Friday, August 29, at 11am PST.

Ruben Daniels, Cloud9′s founder and CEO, and Jonathan Lipps, Sauce Labs’ Director of Ecosystems and Integrations, will walk you through Cloud9′s setup and how to test and debug across multiple browsers and platforms with Sauce Labs.

This 30 minute webinar includes a Q&A. Click here to sign up today!

All registrants will receive a link to the recording and other assets following the webinar, regardless of attendance.

Appium Bootcamp – Chapter 5: Writing and Refactoring Your Tests

August 5th, 2014 by Amber Kaplan

appium_logoThis is the fifth post in a series called Appium Bootcamp by noted Selenium expert Dave Haeffner

Read:  Chapter 1 Chapter 2 | Chapter 3 | Chapter 4 | Chapter 5 | Chapter 6 | Chapter 7 | Chapter 8

Dave recently immersed himself in the open source Appium project and collaborated with leading Appium contributor Matthew Edwards to bring us this material. Appium Bootcamp is for those who are brand new to mobile test automation with Appium. No familiarity with Selenium is required, although it may be useful. This is the fifth of eight posts; two new posts will be released each week.

Now that we’ve identified some test actions in our apps, let’s put them to work by wiring them up in code.

We’ll start with the iOS app and then move onto Android. But first, we’ll need to do a quick bit of setup.

Quick Setup

Since we’re setting up our test code from scratch, we’ll need to make sure we have the necessary gems installed — and done so in a way that is repeatable (which will come in handy for other team members and for use with Continuous Integration).

In Ruby, this is easy to do with Bundler. With it you can specify a list of gems and their versions to install and update from for your project.

Install Bundler by running gem install bundler from the command-line and then create a file called Gemfile with the following contents:

# filename: Gemfile

source 'https://rubygems.org'

gem 'rspec', '~> 3.0.0'
gem 'appium_lib', '~> 4.0.0'
gem 'appium_console', '~> 1.0.1'

After creating the Gemfile run bundle install. This will make sure rspec (our testing framework), appium_lib (the Appium Ruby bindings), and appium_console (our interactive test console) are installed and ready for use in this directory.

Capabilities

In order to run our tests, we will need to specify the capabilities of our app. We can either do this in our test code, or we can leverage the appium.txt files we used for the Appium Console.

Let’s do the latter approach. But first, we’ll want to create two new folders; one for Android and another for iOS. Once they’re created, let’s place each of the appium.txt files into their respective folders.

├── Gemfile
├── Gemfile.lock
├── android
│   └── appium.txt
└── ios
    └── appium.txt

Be sure to update the app capability in your appium.txt files if you’re using a relative path.

Writing Your First Test

With our initial setup taken care of, let’s create our first test file (a.k.a. “spec” in RSpec). The test actions we identified in the previous post were focused on navigation in the app. So let’s call this spec file navigation_spec.rband place it in the ios folder.

├── Gemfile
├── Gemfile.lock
├── android
│   └── appium.txt
└── ios
    └── appium.txt
    └── navigation_spec.rb

Now let’s write our test to launch Appium for iOS and perform a simple navigation test.

In RSpec, describe denotes the beginning of a test file, whereas it denotes a test. So what we have is a test file with a single test in it.

In this test file, we are starting our Appium session before each test (e.g., before(:each)) and ending it after each test (e.g., after(:each)). More specifically, in before(:each), we are finding the path to the iOSappium.txt file and then loading it. After that we start the Appium session and promote the Appium commands so they will be available for use within our test. We then issue driver_quit in after(:each) to cleanly end the Appium session. This is equivalent to submitting an x command in the Appium console.

The commands in our test (it 'First cell' do) should look familiar from the last post. We’re finding the first cell, grabbing it’s title, click on the cell, and then looking to see if the title appeared on the inner screen.

After saving this file, let’s change directories into the ios folder (e.g., cd ios), and run the test (assuming your Appium Server is running — if not, load up the Appum GUI and click Launch) with rspec navigation_spec.rb. When it’s running, you will see the iOS simulator launch, load up the test app, click the first cell, and then close.

This is a good start, but we can clean this code up a bit by leveraging some simple page objects and a central configuration.

A Page Objects Primer

Automated tests can quickly become brittle and hard to maintain. This is largely due to the fact that we are testing functionality that will constantly change. In order to combat this, we can use page objects.

Page Objects are simple objects that model the behavior of an application. So rather than writing your tests directly against your app, you can write them against these objects. This will make your test code more reusable, maintainable, and easier to fix when the app changes.

You can learn more about page objects here and here.

Refactoring Your First Test

Let’s create a new directory called pages within our ios directory and create two new files in it: home.rb and inner_screen.rb. And while we’re at it, let’s create a new folder to store our test files (called spec – which is a folder RSpec will know to look for at run time) and move our navigation_spec.rb into it.

├── Gemfile
├── Gemfile.lock
├── android
│   └── appium.txt
└── ios
    ├── appium.txt
    ├── pages
    │   ├── home.rb
    │   └── inner_screen.rb
    └── spec
        ├── navigation_spec.rb

Let’s open up ios/pages/home.rb to create our first page object.

 

# filename: ios/pages/home.rb

module Pages
  module Home
    class << self

      def first_cell
        @found_cell = wait { text 2 }
        self
      end

      def title
        @found_cell.name.split(',').first
      end

      def click
        @found_cell.click
      end

    end
  end
end

module Kernel
  def home
    Pages::Home
  end
end

Since the Appium commands are getting promoted for use (instead of passing around a driver object), storing our page objects in a module is a cleaner approach (rather than keeping them in a class that we would need to instantiate).

To create the Home module we first wrap it in another module called Pages. This helps prevent any namespace collisions as well simplify the promotion of Appium methods.

In Home, we’ve created some simple static methods to mimic the behavior of the home screen (e.g., first_celltitleclick). By storing the found cell in an instance variable (e.g., @found_cell) and returning self, we will be able to chain these methods together in our test (e.g., first_cell.title). And in order to cleanly reference the page object in our test, we’ve made the home method available globally (which references this module).

Now let’s open up ios/pages/inner_screen.rb and create our second page object.

# filename: pages/inner_screen.rb

module Pages
  module InnerScreen
    class << self

      def has_text(text)
        wait { text_exact text }
      end

    end
  end
end

module Kernel
  def inner_screen
    Pages::InnerScreen
  end
end

This is the same structure as our previous page object. In it, we’re performing an exact text search.

Let’s go ahead and update our test to use these page objects.

# filename: ios/spec/navigation_spec.rb

require 'appium_lib'
require_relative '../pages/home'
require_relative '../pages/inner_screen'

describe 'Home Screen Navigation' do

  before(:each) do
    appium_txt = File.join(Dir.pwd, 'appium.txt')
    caps = Appium.load_appium_txt file: appium_txt
    Appium::Driver.new(caps).start_driver
    Appium.promote_appium_methods RSpec::Core::ExampleGroup
    Appium.promote_singleton_appium_methods Pages
  end

  after(:each) do
    driver_quit
  end

  it 'First cell' do
    cell_title = home.first_cell.title
    home.first_cell.click
    inner_screen.has_text cell_title
  end

end

We first require the page objects (note the use of require_relative at the top of the file). We then promote the Appium methods to our page objects (e.g., Appium.promote_singleton_appium_methods Pages). Lastly, we update our test.

Now when we run our test from within the ios directory (e.g., cd ios then rspec) then it will run just the same as it did before.

Now the test is more readable and in better shape. But there is still some refactoring to do to round things out. Let’s pull our test setup out of this test file and into a central config that we will be able to leverage for both iOS and Android.

Central Config

In RSpec, we can configure our test suite from a central location. This is typically done in a file called spec_helper.rb. Let’s create a folder called common in the root of our project and add a spec_helper.rb file to it.

├── Gemfile
├── Gemfile.lock
├── android
│   └── appium.txt
├── common
│   └── spec_helper.rb
└── ios
    ├── appium.txt
    ├── pages
    │   ├── home.rb
    │   └── inner_screen.rb
    └── spec
        ├── navigation_spec.rb

Let’s open up common/spec_helper.rb, add our test setup to it, and polish it up.

 

# filename: common/spec_helper.rb

require 'rspec'
require 'appium_lib'

def setup_driver
  return if $driver
  caps = Appium.load_appium_txt file: File.join(Dir.pwd, 'appium.txt')
  Appium::Driver.new caps
end

def promote_methods
  Appium.promote_singleton_appium_methods Pages
  Appium.promote_appium_methods RSpec::Core::ExampleGroup
end

setup_driver
promote_methods

RSpec.configure do |config|

  config.before(:each) do
    $driver.start_driver
  end

  config.after(:each) do
    driver_quit
  end

end

After requiring our requisite libraries, we’ve created a couple of methods that get executed when the file is loaded. One is to setup (but not start) Appium and another is to promote the methods to our page objects and tests. This approach is taken to make sure that only one instance of Appium is loaded at any one time.

We then configure our test actions so they run before and after each test. In them we are starting an Appium session and then ending it.

In order to use this central config, we will need to require it (and remove the unnecessary bits) in our test.

# filename: ios/spec/navigation_spec.rb

require_relative '../pages/home'
require_relative '../pages/inner_screen'
require_relative '../../common/spec_helper'

describe 'Home Screen Navigation' do

  it 'First cell' do
    cell_title = home.first_cell.title
    home.first_cell.click
    inner_screen.has_text cell_title
  end

end

Note the order of the require_relative statements – they are important. We need to load our page objects before we can load our spec_helper, or else the test won’t run.

If we run the tests from within the ios directory with rspec, we can see everything execute just like it did before.

Now that we have iOS covered, let’s wire up an Android test, some page objects, and make sure our test code to supports both devices.

Including Android

It’s worth noting that in your real world apps you may be able to have a single set of tests and segmented page objects to help make things run seamlessly behind the scenes for both devices. And while the behavior in our Android test app is similar to our iOS test app, it’s design is different enough that we’ll need to create a separate test and page objects.

Let’s start by creating spec and pages folders within the android directory and then creating page objects in pages (e.g., home.rb and inner_screen.rb) and a test file in spec (e.g., navigation_spec.rb).

├── Gemfile
├── Gemfile.lock
├── android
│   ├── appium.txt
│   ├── pages
│   │   ├── home.rb
│   │   └── inner_screen.rb
│   └── spec
│       ├── navigation_spec.rb
├── common
│   └── spec_helper.rb
└── ios
    ├── appium.txt
    ├── pages
    │   ├── home.rb
    │   └── inner_screen.rb
    └── spec
        ├── navigation_spec.rb

Now let’s open and populate our page objects and test file.

module Pages
  module Home
    class << self

      def first_cell
        @found_cell = wait { text 2 }
        self
      end

      def click
        @found_cell.click
      end

    end
  end
end

module Kernel
  def home
    Pages::Home
  end
end

This page object is similar to the iOS one except there’s no title search (since we won’t be needing it).

module Pages
  module InnerScreen
    class << self

      def has_text(text)
        wait { find_exact text }
      end

    end
  end
end

module Kernel
  def inner_screen
    Pages::InnerScreen
  end
end

In this page object we’re performing a search for an element by text (similar to the iOS example), but using find_exact instead of text_exact because of how the app is designed (we need to perform a broader search that will search across multiple attributes, not just the text attribute).

Now let’s wire up our test.

require_relative '../pages/home'
require_relative '../pages/inner_screen'
require_relative '../../common/spec_helper'

describe 'Home Screen Navigation' do

  it 'First cell' do
    home.first_cell.click
    inner_screen.has_text 'Accessibility Node Provider'
  end

end

Now if we cd into the android directory and run our test with rspec it should launch the Android emulator, load the app, click the first cell, and then end the session. The emulator will remain open, but that’s something we’ll address in a future post.

One More Thing

If we use the console with the code that we have right now, we won’t be able to reference the page objects we’ve created — which will be a bit of a pain if we want to reference them when debugging test failures. Let’s fix that.

Let’s create a new file in our android/spec and ios/spec directories called requires.rb. We’ll move our require statements out of our test files and into these files instead.

├── Gemfile
├── Gemfile.lock
├── android
│   ├── appium.txt
│   ├── pages
│   │   ├── home.rb
│   │   └── inner_screen.rb
│   └── spec
│       ├── navigation_spec.rb
│       └── requires.rb
├── common
│   └── spec_helper.rb
└── ios
    ├── appium.txt
    ├── pages
    │   ├── home.rb
    │   └── inner_screen.rb
    └── spec
        ├── navigation_spec.rb
        └── requires.rb

Here’s what one of them should look like:

# filename: ios/spec/requires.rb

# require the ios pages
require_relative '../pages/home'
require_relative '../pages/inner_screen'

# setup rspec
require_relative '../../common/spec_helper'

Next, we’ll want to update our tests to use this file.

require_relative 'requires'

describe 'Home Screen Navigation' do

  it 'First cell' do
    cell_title = home.first_cell.title
    home.first_cell.click
    inner_screen.has_text cell_title
  end

end
# filename: android/spec/navigation_spec.rb

require_relative 'requires'

describe 'Home Screen Navigation' do

  it 'First cell' do
    home.first_cell.click
    inner_screen.has_text 'Accessibility Node Provider'
  end

end

Now that we have a central requires.rb for each device, we can tell the Appium Console to use it. To do that, we’ll need to add some additional info to our appium.txt files.

 

# filename: ios/appium.txt

[caps]
deviceName = "iPhone Simulator"
platformName = "ios"
app = "../../../apps/UICatalog.app.zip"

[appium_lib]
require = ["./spec/requires.rb"]
# filename: android/appium.txt

[caps]
platformName = "android"
app = "../../../apps/api.apk"
avd = "training"
deviceName = "Android"

[appium_lib]
require = ["./spec/requires.rb"]

This new require value is only used by the Appium Console. Now if we run arc from either the ios or android directories, we’ll be able to access the page objects just like in our tests.

And if we run our tests from either directory, they will still work as directed.

Outro

Now that we have our tests, page objects, and central configuration all sorted, it’s time to look at wrapping our test execution and make it so we can run our tests in the cloud.

Read:  Chapter 1 Chapter 2 | Chapter 3 | Chapter 4 | Chapter 5 | Chapter 6 | Chapter 7 | Chapter 8

About Dave Haeffner: Dave is a recent Appium convert and the author of Elemental Selenium (a free, once weekly Selenium tip newsletter that is read by thousands of testing professionals) as well as The Selenium Guidebook (a step-by-step guide on how to use Selenium Successfully). He is also the creator and maintainer of ChemistryKit (an open-source Selenium framework). He has helped numerous companies successfully implement automated acceptance testing; including The Motley Fool, ManTech International, Sittercity, and Animoto. He is a founder and co-organizer of the Selenium Hangout and has spoken at numerous conferences and meetups about acceptance testing.

Follow Dave on Twitter - @tourdedave

Puppet Labs + Sauce Labs: Testing Made Awesome [VIDEO]

August 1st, 2014 by Amber Kaplan

A few weeks back, we sat down with Engineering Manager Dominic Maraglia of Puppet Labs to hear how they use Sauce.

Puppet Labs is the leading provider of IT automation software, so naturally automating processes internally is a high priority. We were pleased to hear that Sauce Labs fit easily into their testing process, and that after automating their tests and using Sauce’s infrastructure, the time they spent on testing went from 30 days to 3 days.

Watch the video below to learn more.

Want to share your story? We want to hear from you! Submit a request here.