Automated Mobile App Testing with Python and Nose

May 16th, 2013 by Jason Carr

One of the coolest parts about using Appium for automated functional testing is that you can write your tests in whatever language has the testing tools that suit you best. One of the most popular choices for testing is Python. Writing tests for iOS and Android apps with Appium and Python is easy.

In this post we’ll walk through the steps involved in testing an iOS sample app using Appium’s Python example tests, but the steps for running tests on android are very similar.

To start, fork and clone Appium from https://github.com/appium/appium, and follow the installation instructions to set up Appium on your machine.

We’ll need to install Appium’s dependencies and build the sample apps. To do this, run the following command from the Appium working directory:

$ ./reset.sh --ios

Once the project is built, you can start Appium by running:

$ grunt appium

Now that Appium is running, switch to the sample-code/examples/python directory. Install the dependencies with pip (if you are not in a virtualenv, you will need to use sudo):

$ pip install -r requirements.txt

Then, run the example test:

$ nosetests simple.py

Now that you’re set up to run tests, let’s take a closer look at the example test that we just ran to get an idea of how Appium works. This test launches our demo app, fills in some boxes, clicks a button, and checks for an expected result. First, we create the test class and a setUp method:

class TestSequenceFunctions(unittest.TestCase):

    def setUp(self):
        app = os.path.join(os.path.dirname(__file__),
                           '../../apps/TestApp/build/Release-iphonesimulator',
                           'TestApp.app')
        app = os.path.abspath(app)
        self.driver = webdriver.Remote(
            command_executor='http://127.0.0.1:4723/wd/hub',
            desired_capabilities={
                'browserName': 'iOS',
                'platform': 'Mac',
                'version': '6.0',
                'app': app
            })
        self._values = []

The “desired_capabilities” parameter is used to specify the platform (iOS 6.0) and the app we want to test. Next we’ll add a tearDown method to send the quit command after each test:

    def tearDown(self):
        self.driver.quit()

Finally, we’ll add a helper method that fills in the forms, and the main test method:

    def _populate(self):
        # populate text fields with two random number
        elems = self.driver.find_elements_by_tag_name('textField')
        for elem in elems:
            rndNum = randint(0, 10)
            elem.send_keys(rndNum)
            self._values.append(rndNum)

    def test_ui_computation(self):
        # populate text fields with values
        self._populate()
        # trigger computation by using the button
        buttons = self.driver.find_elements_by_tag_name("button")
        buttons[0].click()
        # is sum equal ?
        texts = self.driver.find_elements_by_tag_name("staticText")
        self.assertEqual(int(texts[0].text), self._values[0] + self._values[1])

That’s it! There are more Python examples in Appium’s sample tests. Please let us know if you have any questions or comments about running Appium’s tests with Nose and Python.

Share

Announcing iOS and Android App Testing on Sauce

May 14th, 2013 by Jonathan Lipps

appiumonsauceToday we are very excited to announce the general availability of Appium on Sauce! Appium is the open source cross-platform mobile automation platform for native mobile, hybrid, and mobile web apps, and it runs on iOS (iPhone and iPad) and Android devices and emulators.

Appium represents a huge step forward in the fight to make mobile automation a first-class citizen in mobile development. We’ve been running our iOS-based private beta for Appium on Sauce for a while, and have made many improvements while supporting the development of the Appium open source community. Since then, Android support has matured to the point where we are now offering it in our cloud alongside iPhone and iPad. Soon enough, we’ll be releasing the ability to run your Appium tests not only on our cloud of emulators and simulators, but also on real devices (stay tuned for more on that later!).

If you want to get started with Appium right now, simply visit the Appium on Sauce tutorial. And check out the Appium iPhone demo below.

If you’re interested in what Appium’s all about, read on.

After contributing thousands of lines of code to the open-source project and helping to breathe fire into what has become a very active community (see our issue tracker, our discussion group, or join us in #appium on freenode), it’s clear to us that Appium’s philosophy is compelling to a wide array of developers, test engineers, and QA professionals. Appium’s philosophy can be stated pretty succinctly in the form of four principles:

  1. Test the same app you submit to the marketplace. (You shouldn’t have to recompile your app with a 3rd-party library to test it.)
  2. Write your tests in any language, using any framework. (You shouldn’t have to learn a new language.)
  3. Use a standard automation specification and API. (Don’t re-invent the automation model wheel.)
  4. Build a large and thriving open-source community effort. (Say no to vendorware and skunkworks.)

Based on the huge amount of energy we’ve been receiving from the growth of the open source community, I’d say these principles are withstanding scrutiny.

While Appium on its own is a leap forward in mobile automation, allowing true cross-platform testing and scaling across the different dimensions of apps and devices (simulator vs. real devices, iOS vs. Android, native vs. hybrid), it is especially powerful when combined with Sauce Labs. Sauce’s value has historically been most apparent in freeing web developers and testers from managing their own test and reporting infrastructure. With Appium, this instant scalability is available to mobile-focused teams as well. Only on Sauce can you run your mobile tests across platforms in massive parallel, opening up a world of potential for continuous integration and QA in general. Continuous deployment for mobile apps, anyone? It’s possible, and with Appium on Sauce it can become the standard.

Writing Appium tests is as easy as writing the Selenium WebDriver tests you may be familiar with or may be already running on Sauce. You can write them in any language (Python, Java, PHP, Ruby, Node.js, C#, Go, Haskell, etc…) or test framework (RSpec, PHPUnit, JUnit, Nose, Mocha, Vows, Cucumber, Capybara, etc…). And of course, they come with the same great reporting (screenshots, videos, and command logs) as browser-based test sessions.

I’ll leave you with an example test, running a notes application on Android (taken from this example):

import os
from selenium import webdriver

desired_caps = {}
desired_caps['device'] = 'Android'
desired_caps['browserName'] = ''
desired_caps['version'] = '4.2'
desired_caps['app'] = 'http://appium.s3.amazonaws.com/NotesList.apk'
desired_caps['app-package'] = 'com.example.android.notepad'
desired_caps['app-activity'] = 'NotesList'

SAUCE_USERNAME = os.environ.get('SAUCE_USERNAME')
SAUCE_ACCESS_KEY = os.environ.get('SAUCE_ACCESS_KEY')

driver = webdriver.Remote('http://%s:%s@ondemand.saucelabs.com:80/wd/hub' % (SAUCE_USERNAME, SAUCE_ACCESS_KEY), desired_caps)

el = driver.find_element_by_name("New note")
el.click()

el = driver.find_element_by_tag_name("textfield")
el.send_keys("This is a new note!")

el = driver.find_element_by_name("Save")
el.click()

els = driver.find_elements_by_tag_name("text")
assert els[2].text == "This is a new note!"
els[2].click()

driver.quit()

You can also take a look at an iPhone example test.

Want to run Appium on Sauce right away? Simply create a Sauce account and you’ll be good to go. If you’re already a Sauce user or customer, you have access to Appium testing capability without any further steps.

Once you’re ready to start writing your Appium tests, take a look at the Appium on Sauce tutorial to get started. And don’t forget to let us know how things are going for you in your quest to total mobile automation.

Happy native app testing from all of us at Sauce!

Share

Steve Hazel on FLOSS Weekly

May 10th, 2013 by lauren nguyen

Our co-founder and VP of Product Steven Hazel was interviewed this week on FLOSS Weekly, the seminal weekly open source podcast. If you didn’t catch the show live, you can watch or listen here, and hear Steve talk Appium, Sauce Labs, and his love of Emacs. Steve joins a long list of distinguished open source guests, including Sauce co-founder Jason Huggins, who’s been on the show to talk about Selenium.

 

Share

Recap: GTAC 2013

May 9th, 2013 by lauren nguyen

If you weren’t at GTAC in NYC last month, you may have missed Jonathan Lipps‘ dazzling talk on Appium and mobile app automation. But fear not, you too can experience (or re-experience) the magic of his talk as many times as you like! You can watch his talk on YouTube, and boggle the ridiculousness that is his surprise Firefox OS demo. Jonathan and the Appium community have been hard at work improving Appium, and they’ve made a lot of great progress in building a great framework for testing native mobile apps. You can also see his deck from the talk on Slideshare.

Share

Come Play in the Sandbox with Us at Google I/O!

May 7th, 2013 by lauren nguyen

Google-I-O-13If you’ll be in San Francisco for Google I/O this year, don’t forget to come see us in the Developer Sandbox! Google I/O is an annual developer conference featuring highly technical, in-depth sessions, and showcasing the latest from Google’s product teams and partners, and this year will be held May 15-17 at the Moscone Center. The Developer Sandbox showcases demos from a wide range of developers who’ve built applications based on technologies and products featured at I/O. We’re pumped to be invited to demo Sauce Labs this year, and hope you’ll come hang out with us and talk about all things testing.

Adam Christian and Jonathan Lipps will be on site to talk about how using Sauce Labs and Selenium can make developing and testing on Chrome faster and easier.

We’ll be demo-ing at the Sandbox the first day of the conference, May 15th, starting around 11AM. See you there!

Share

Seven Reasons to Come to Our Ruby Hackathon

May 6th, 2013 by Dylan

In the Social Media age, there’s nothing, nothing more important then being sharable. And as the continued success of the Gawker network shows, nothing is as sharable as a list of stuff.

So! In order to help you convince your friends to come to our Hackathon (because you were already coming, right?  3pm, Thursday the 9th of May at our office in San Francisco, like we discussed?), we’re prepared a handy dandy tweetable pinnable shareable hackernewsable list of reasons why they should.

Oh, and just in case you want to know WHICH Google+ Circle to post it to, our hackathon is open to anyone who wants to play with Sauce Labs, anyone who wants help to get started, anyone who wants to work on the gem and anyone who just wants to hang out and talk testing and Ruby. All that after the jump!    

HA! No jump (Blog post by M Night Shaymalan)

Reason One:  Help to run Integration tests on multiple platforms

If you’ve ever wanted to play with multiple platforms, or see how your site renders on XP with IE7 without walking your mum through the screenshot process over the phone, or you’re just having problems getting the gem working, there’ll be lots of time to get one on one help.

Reason Two:  WAAAAAY Faster bootstrapping

4950445842_daabb0ae20_b
There’s no cost and no obligation to our hackathon, so it’s a great place to figure out if and how to get started using Sauce Labs to make your integration tests *awesome*.  Saucers and friends will have suggestions on structure, we know the tricks and features, we know how our cloud testing service works, and most importantly: How it sometimes doesn’t.  Chances are, if you’re having a problem getting started with Sauce Labs, other people have too, and they can stab it through the face for you.

Reasons Three:  Get expert Sauce help or just play around

It can be tough to experiment with a tool you’re using commercially.  There’s always a tension that you’ll damage your build or cost your company a bunch of money.  Plus, maybe the thing you’re trying will wind up being a waste of time and your team will be all “duuuuuuude… What about that important thing?” What’s great about hackathons is that there’s implicit permission to go play with this stuff.  Got something you want to try with Sauce Labs?  Go ahead.  Need a hand fetching screenshots to integrate into your dashboard?  Get it.

Reason Four:  Hanging with your peeps is cool

481-hrink-i-shall-play-you-the-song-of-my-peopleDevelopers are fun, creative people.  It’s good for the soul to spend time with your tribe, even without the safety of the internet between you.  It can be tough hanging with completely new people, but having a small amount in common makes it easier.  It’s also tough only spending time with your cohackers at work, even if you get along great.  Hackathons are the best of both worlds; they bring together a group of disparate people who still have something in common.  You’re almost guaranteed to find someone awesome to chat too and broaden your horizons.

Reason Five: Experiment with SPEEEEEEEEEEEEED

Parallel tests are FASTER tests.  Come talk and experiment with running your app and tests in parallel.  Parallelism can be such a hard thing to get going nicely that being able to sound it out with other people is a real boon.

Reason Six: Tour the Sauce Labs office and get free stuff

If you want to see what we offer, this is a great way to do it!  We’ll be giving people access to our products to play around with. We have a big, airy warehouse space with a bunch of couches and fun people.  We also have stickers.  Who doesn’t love stickers?

Reason Seven:  Add another notch to your Open Source belt

to-the-INTERNETWe have a gem for working with our product, allowing you to easily switch your existing RSpec, Test::Unit or Capybara tests to run on Sauce Labs.  Working on Open Source projects is good for the users, but it’s also good for the developers.  You get your name out there, and provable capability when people need examples (Psst:  Like job hunting).  It’s also a great way to influence the tools you use, and building things is FUN.  I often have problems coming up with things I want to build, so helping someone else build something can be very satisfying and less stressful

Reason Seven (a):  Free Pizza, Beer and Wifi

Yeah, OK, so this is a “gimme” on tech events in SF.  But we’ll have them, come consume them!  (No Torrenting :P)   That’s our list!  To the Socials!  Oh, and don’t forget to register for your spot!

Share

Automated iOS and Android Acceptance Testing with RSpec and Appium

May 6th, 2013 by Steven Hazel

If you’re a fan of RSpec for writing concise, readable tests in Ruby, you might be interested to know that RSpec tests can now easily automate iOS and Android mobile apps using Appium, an open source mobile test automation tool. Appium is compatible with Selenium WebDriver client libraries, so an Appium test is similar to a Selenium test.

In this post we’ll walk through the steps involved in testing an iOS sample app using Appium’s RSpec example tests.

To start, fork and clone Appium from https://github.com/appium/appium, and follow the installation instructions to set up Appium in your environment.

Install Appium’s dependencies and build the sample apps by running the following from the Appium working directory:

$ ./reset.sh --ios

Once the sample projects have have been built, you can then start Appium by running the following:

$ grunt appium

Once Appium is running, get a new terminal and go to the sample-code/examples/rspec directory.

First, install the test dependencies with bundler:

$ bundle install

Then, run the example RSpec tests:

$ rspec simple_test.rb

Now that you’re set up and able to run example tests, it’s time to learn how they work. Let’s look at a simple example of a test that clicks a button and interacts with an alert box in an iOS app. First, we open a “describe” block, and set up a “before” block to initialize the Selenium client we’ll use to talk to Appium:

 
describe "sample app" do
  before :all do
    @driver = Selenium::WebDriver.for(
        :remote, 
        :desired_capabilities => {'browserName' => 'iOS',
                                  'platform' => 'Mac',
                                  'version' => '6.0',
                                  'app' => '/path/to/TestApp.app'}
        :url => "http://127.0.0.1:4723/wd/hub")
  end

The “desired_capabilities” parameter here specifies the platform (iOS 6.0) and the app we want to test. Next, we’ll add an “after” block to quit the Appium session at the end of each test:

 
  after :all do
    @driver.quit
  end

And finally, we’ll open an “it” block and write the core of our test. This test finds all the ‘button’ elements on the page, selects the second one (index 1), and clicks it. That action will bring up an alert in the app, which we switch to so that we can examine its text field:

 
  it "should handle alerts" do
    els = @driver.find_elements(:tag_name, 'button')
    els[1].click
    a = @driver.switch_to.alert
    a.text.should eq("Cool title")
    a.accept
  end
end

And those are the basics! There are more examples to look at in Appium’s simple_test.rb. Please let us know if you have any questions or comments about running Appium tests with RSpec.

Share

Join Us for a Ruby Hackathon

May 3rd, 2013 by lauren nguyen

rubyMore events? You bet! It’s spring, and we’re in the mood to party, so we’re throwing our first Ruby hackathon! Stop by our office next Thursday, May 9th, starting at 3PM, to eat, drink, and hack with us. Come get started with running your existing Selenium integration tests on Sauce using the Sauce Gem, enhance the gem, or just hang out with the Saucers and Ruby community. Cross-browser testing on Ruby with Sauce is a cinch, and we want to make it even better! Our beloved Ruby dude Dylan will be in town all the way from Australia; if you ask him nicely enough he may agree to do a jig.

We’re providing pizza and beer, so bring yourself and your laptop down to our office to hang, eat, drink, and maybe get some work done! Register here.

We’re located at 500 3rd St. (cross street Bryant), Ste 240, San Francisco.

See you there!

Share

Swing by the San Francisco Python Meetup Next Week!

May 2nd, 2013 by lauren nguyen


python
If you’re in San Francisco next week, Saucer Santi will be giving a featured talk on Selenium and best practices for writing automated functional tests for your Python Web Apps on Thursday, 5/8 at 6:15PM. The meetup will be hosted at the Yelp SF office, 706 Mission St. Yelp is also generously sponsoring pizza and beer. :)

Drop by to say hi, hear some awesome talks, and connect with other Bay Area Python lovers.

You can find more info about the meetup and RSVP here.

See you there!

Share

Recap: Seattle Mobile Development Meetup

April 29th, 2013 by Jason Carr

mobiletestingIn mid April, I was welcomed by the Seattle community to speak about Appium at their Mobile Development Meetup. In addition to having a great time and great conversations, there were some key takeaways from the meetup that are worthy of some thought. Of course, there’s not time to summarize all the amazing conversations that I had with the wonderful Seattle-ites, but here are the high points.

Mobile testing is more important now than it’s ever been. There are few things as painful as submitting your app to the app store, getting it accepted and published, then realizing that your app has a showstopper bug that’s getting lots of 1 star reviews. Of course, more testing would solve that problem, but testing everything manually is a long, slow process. So we turn to automated testing, but the situation there is little better. Between all the different frameworks (with different limitations), it starts to feel like you’re learning a new framework every time you want to write a test. So it’s back to manual testing… and the cycle repeats itself.

There’s a deep need for a cross platform testing tool. People are writing apps for Android and iOS now. Not just one or the other. And you’re probably being asked to test on both platforms. So rather than manually test on every device and OS you need to support, what can you do? You need a tool that lets you write your app so you can write one test and run it on many platforms. It was great to see people’s enthusiasm for Appium because it lets you run tests on iOS and Android.

Accessibility is not optional. This is a sticky issue for a lot of developers. On one hand, accessibility is a key technology for the web, and something we all ought to have in our apps anyway. On the other hand, it takes time to build in accessibility and your project was due yesterday. Tipping the scales in accessibility’s favor is the fact that nearly all iOS and Android testing tools use accessibility functionality in some way to automate the device. Much like the early days of browser automation, these labels and hooks are all that exist to allow for automating the device. So if you’re stuck on accessibility, remember that it makes testing your apps easier too.

Mobile Development is complex. If you’ve gone to a mobile meetup, odds are you left with your head spinning. Keeping all the frameworks, concepts, and tools straight is a full-time job, and you’ve got code to write! Don’t feel bad. The mobile world is complex and moves at an astonishing speed. It’s worth some time to get the basic concepts straight: Hybrid v. Native apps, the major app development frameworks, and the top testing tools (like Appium!).

Windows Phone and Firefox OS cannot just be ignored. While neither of these platforms has the market share to command a lot of attention yet, it’s important that mobile devs keep their eyes on the future. You don’t know what the market will choose as the next great platform, and it’s absolutely key to keep a strong awareness of what is coming in the future, and how you can write killer apps with great tests on that hot new platform. So important that Sauce Labs’ Jonathan Lipps spent his time at GTAC working with some Mozillians to hack Firefox OS into Appium.

Seattle’s mobile developers are amazing! I had some great conversations and spent the day after my talk hacking with some cool engineers, getting them set up to use Appium. Props to Carter Rabasa for setting up an amazing meetup. If you’re in the Seattle area, check them out!

Share