Posts Tagged ‘ruby’

Setting up iOS Automation on Sauce Labs with Cucumber and Appium

October 7th, 2013 by Shashikant Jagtap

Automated mobile testing with Cucumber & AppiumAbstract

Sauce Labs has recently announced Appium support which makes it easier to test mobile apps in the cloud. Appium is a mobile test automation framework for hybrid and native mobile apps. Cucumber is a behaviour driven development a.k.a BDD tool used with different programming languages. The combination of Cucumber and Appium can be used for automating iOS apps in the cloud using Sauce Labs. This is a repost of the original post. In this post, we will see how to set up test automation of our iOS app in the cloud using Sauce Labs.

Getting Started

In order to get started, we need to have the initial setup handy. This includes the following:

  • Mac OSX 10.7.4 or higher with Xcode installed with command line tools.
  • Your app source code or a prebuilt .app bundle for your app. Browse wide range of open-source iOS apps
  • Saucelabs Username and API key. Signup for Saucelabs free account.
  • Web development environment on Mac OSX for Ruby including Xcode, RVM, HomeBrew, Git and Ruby. Follow Moncef’s blog
  • Appium-Ruby-console  with Node and npm (Node must be >= v0.8)
  • Ruby 1.9.3

Get Appium and Your iOS App

Before we go ahead, let’s get the Appium server up and running. There are two ways to do it-

  •  You can download Mac OSX Appium.dmg package and launch Appium.
  • You can run it from source. Follow instructions. You need to make sure, You have authorized use of the iOS Simulator. If you are running Appium from NPM, you’ll do this by running
$ sudo authorize_ios

There are a wide range of open source iOS mobile apps available here, we are going to use PlainNote iOS app for this tutorial.

Compile and Upload iOS App on SauceLabs

Now we need to compile PlainNote App with Sauce. [ Note Additional parameter TARGET_DEVICE_FAMILY]

$ git clone https://github.com/vkoser/PlainNote
$ cd PlainNote
$ xcodebuild -sdk iphonesimulator6.0 TARGETED_DEVICE_FAMILY=1

In Sauce, there are optional parameters like TARGETED_DEVICE_FAMILY parameter . To build an app for iPhone, we use  TARGETED_DEVICE_FAMILY= 1, for iPad  TARGETED_DEVICE_FAMILY=2 .

Now, once the build is successful, it will create “PlaneNote.app” at ‘/build/Release-iphonesimulator‘.

$ cd /build/Release-iphonesimulator/
$ ls
PlainNote.app PlainNote.app.dSYM
  •  Zip the PlainNote App & Upload to SauceLabs 

Now that we have ‘PlainNote.app‘, we need to zip it by navigating to that directory

$ cd /build/Release-iphonesimulator/
$ zip -r PlainNote.zip PlainNote.app

Now you will see ‘PlainNote.zip‘ file in that directory

$ ls
PlainNote.app      PlainNote.app.dSYM PlainNote.zip

Now, we need to upload this file to Sauce Labs temporary storage using the Sauce REST API. I am using my Username and API key here.

$ curl -u Shashikant86:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxbfc3 -X POST "http://saucelabs.com/rest/v1/storage/Shashikant86/PlainNote.zip?overwrite=true" -H "Content-Type: application/octet-stream" --data-binary @/path/to/PlainNote/build/Release-iphonesimulator/PlainNote.zip

It will show you response something like this,

{"username": "Shashikant86", "size": 42190, "md5": "6ef42125b024188976af9d6b8a104105", "filename": "PlainNote.zip"}

It will be now uploaded to “sauce-storage:PlainNote.zip“. Now we are all set for writing tests for the application with Cucumber.

Setup Cucumber project

Now that we have already uploaded our app on SauceLabs temporary storage, we can setup Cucumber project to talk to the mobile app in the cloud. I assume that you are familiar with the BDD Code and code structure for the Cucumber project. Please refer my old post if you are not aware of BDD code.

  • Create Gemfile

We need a Gemfile in order to specify all our dependencies

$ mkdir sauce-cucumber-appium
$ cd sauce-cucumber-appium
$ rvm use 1.9.3
$ vim Gemfile

Now insert the following dependencies into the Gemfile

source "https://www.rubygems.org"
gem "rest-client"
gem "rspec"
gem "selenium-webdriver"
gem "cucumber"
gem "rspec-expectations"

Now we need to install the bundle to download all the dependencies.

$ bundle install

This will create a ‘Gemfile.lock’ file.

  •  Create Feature File 

Now, we will write a feature file using Given When Then format. The feature file is written using the Gherkin Domain Specific Language.

Let’s create ‘features/plain_note_sauce.feature‘ file.

$ vim features/plain_note_sauce.feature

The PlainNote app feature will look something like this

Feature: Notes 
  As iOS automation specialist 
  I want to setup iOS app automation in the cloud using Saucelabs, Appium and cucumber 

Scenario: Add new Note using PlainNote App 

  Given I have App running with appium on Sauce 
  When click + button using sauce driver
  And I enter text "Data" and saved it on sauce
  Then I should see "Data" note added on home page in the sauce cloud

This feature is self explanatory, we are going to add a new note and make sure it displayed on the Home page.

  • Setup Cucumber Environment  

Let’s create ‘features/support/env.rb‘ where we can put our support code. We need to add sauce_capabilities mentioned in the Sauce Labs Appium tutorial.

$ vim features/support/env.rb

Insert the following code in the file.

require 'rspec/expectations'
require 'selenium-webdriver'

def sauce_capabilities
  {
    'app' => 'sauce-storage:PlainNote.zip',
    'device' => 'iPhone Simulator',
    'username' => 'Shashikant86',
    'access-key' => 'a0e37e25-e2f3-4cba-95d3-936007d8bfc3',
    'platform' => 'OS X 10.8',
    'version' => '6.0',
    'name' => 'Running PlainNote wit Cucumber and Appium', 
    'passed' => 'true'
  }
end

def sauce_url
  "http://Shashikant86:a0e37e25-e2f3-4cba-95d3-936007d8bfc3@ondemand.saucelabs.com:80/wd/hub"
end

def sauce
  @sauce ||= Selenium::WebDriver.for(:remote, :desired_capabilities => sauce_capabilities, :url => sauce_url)
end

After { @sauce.quit }

Now that we have  a created ‘sauce’ driver with all required desired capabilities, ee will using the ‘sauce’ object in our step_definitions

  •  Write Step definitions using Selenium-Webdriver  JSON Wire Protocol

At this point if you run the ‘bundle exec cucumber’ command it will tell you steps that are not implemented yet. We need to implement these step definitions using Selenium-Webdriver JSON Wire Protocol for Appium. Now we will create a step definition file and implement it

$ vim features/step_definitions/plain_note.rb

Now add these step definitions to the file.

Given(/^I have App running with appium on Sauce$/) do
end

When(/^click \+ button using sauce driver$/) do
sauce.find_element(:name, "Add").click
end

When(/^I enter text "(.*?)" and saved it on sauce$/) do |data|
sauce.find_element(:xpath, "//window[1]/scrollview[1]/textview[1]").send_keys data
sauce.find_element(:name, "Done").click
sauce.find_element(:name, "Save").click
end

Then(/^I should see "(.*?)" note added on home page in the sauce cloud$/) do |text|
note = sauce.find_element(:xpath,  "//window[1]/tableview[1]/cell[1]/text[1]")
note.attribute("value").should match text 
end

Appium Inspector 

Appium Inspector is a feature of the Appium OSX app which allows you to inspect elements on your mobile app. You can also record tests in the different languages. Writing the Ruby code is easy if you have used Appium Inspector locally to record tests. Watch this video to know ‘How to use Appium Inspector‘.

Now, we are all set to run cucumber to execute tests on the SauceLabs

$ bundle exec cucumber features/plain_note_sauce.feature

Now you will see the tests running on Sauce Labs and in your terminal you will see something like this

Feature: Notes
  As iOS automation specialist
  I want to setup iOS app automation in the cloud using Saucelabs, Appium and cucumber

  Scenario: Add new Note using PlainNote App                            # features/plain_note_sauce.feature:5
    Given I have App running with appium on Sauce                       # features/step_definitions/plain_note_sauce.rb:1
    When click + button using sauce driver                              # features/step_definitions/plain_note_sauce.rb:4
    And I enter text "Data" and saved it on sauce                       # features/step_definitions/plain_note_sauce.rb:8
    Then I should see "Data" note added on home page in the sauce cloud # features/step_definitions/plain_note_sauce.rb:14

1 scenario (1 passed)
4 steps (4 passed)
0m35.213s

You can watch video and screenshots of the job.

Cucumber-Appium-SauceLabs

You can find source code on GitHub:

GitHub : cucumber-appium

Watch this video get a clear idea of the cucumber-appium setup on Sauce Labs. If any questions, feel free to contact me.

Break Things Faster with Ruby Parallelization

February 25th, 2013 by Dylan

You’re probably wasting two things when you’re testing. Your time… because the other thing is your extra CPUs. Any time you’re waiting on a relatively slow resource, your CPU is just sitting there, twiddling its silicon thumbs. If you’re only using one CPU core at a time, the other cores are doing much the same. Unfortunately for web developers who do things involving CRUD operations, slow resources includes databases. Most unfortunately for you, Dear Reader (and us, Dear… Us), they also include the browsers you’re integrating with for Selenium testing.

Source: http://www.flickr.com/photos/calliope/440681335/

This is what waiting for browsers and io should be measured with

One of the best ways to get more test for your tokens is to run more tests at once. If you’ve got several tests going, even if some are waiting for one slow resource, the others can use the CPU. The more tests you can run at once, the shorter your test cycles will be, especially if you could, say, spin up more then one copy of the slow resource to test with (Psst: I’m talking about Sauce Labs Parallelization).

I’ve been looking for ways to make parallelization easier when testing with Ruby, and I stumbled across the Parallel Tests gem. It’s actively developed, has some nice documentation and integrates with rspec, rspec-rails and test:unit. Their benchmarks show that using the gem, the Rails ActionPack test suite time was cut in half, from 88 seconds to 44, with just 4 test runners. This, conveniently, is the number of parallel tests you can run with a Mild plan.

Source: http://www.flickr.com/photos/59937401@N07/5857826966/

Checkmate. Wait. Snap. Game Set Match?

So for many of you, there’s already a possibility of making your tests take half the time. Which means you can run twice as many. Which means a much faster turnaround time for TDD, BDD, and release testing.

I’d say that’s a win.

A Reading from the Book of Ruby

February 21st, 2013 by Dylan

The following is a post by Dylan Lacey. Kinda. He’s chosen to do it in interpretive dance  easily digestible image format.

Source - http://www.flickr.com/photos/jeffgunn/6663212147/

This is San Francisco.

I am currently in San Francisco

 

I was recently there…

http://www.flickr.com/photos/hostingreviews/8057709725/sizes/m/in/photostream/


http://images.wikia.com/uncyclopedia/images/c/cf/Developers.gif
http://www.flickr.com/photos/glasgowamateur/6268228233/in/photostream/
sauce_horiz_340_over

…because I am the new Ruby Developer Evangelist at Sauce Labs.

http://www.flickr.com/photos/antichrist/132623505/

My job involves taking a fair few of these;

http://www.flickr.com/photos/cogocogo/6204578958/

And sharing a lot of these. It’s a burden.

http://www.flickr.com/photos/hostingreviews/8057709725/sizes/m/in/photostream/

http://www.flickr.com/photos/nasonurb/8291759460/http://www.flickr.com/photos/infomatique/8136256947http://www.flickr.com/photos/infomatique/8136256947

I’m here to help get Ruby developers up and running, helping them to Drink the Sauce.

http://www.flickr.com/photos/otto-yamamoto/3655977700/

It’s SO awesome.

http://www.flickr.com/photos/farnea/395147027/

I’m insanely thrilled to be working with such smart people on an amazing product!

My bailiwick is to make it better for Ruby developers to use Sauce Labs’ stuff, including improving the gems, writing better documentation and building the community. Plus, it’s a unique opportunity to use my personality to insult people all over the world! If you want to offend your manager, disrupt your office and get kicked out of your favorite bar for getting shouty about whether RSpec is better than Test::Unit, let me know. And if you want help with the Sauce Gem or tests from Ruby land, hit me up.

You can find me on (T) or (E)

(This post was originally from Dylan’s Blog and all images herein are CC Commercial Licensed, you’ll find their attribution in their alt-text)

Video Demo: RSpec + Selenium + Sauce OnDemand

October 6th, 2010 by Ashley Wilson

Thanks to all who tuned in to our live web demo last week, Using RSpec + Selenium to Test Your Ruby Builds Faster. Sauce Labs Developer Sean Grove walked attendees through the steps of writing an RSpec test and for Selenium testing, and then showed how to run that test across multiple browsers in the cloud using Sauce OnDemand.

For those of you who missed it, check out Sean’s sample code on Github (http://github.com/sgrove/rspec_sauce_webinar), and then follow along with the video below. We’re planning another Ruby-based webinar in the future, so make sure to subscribe to our blog’s RSS feed to stay up to date!

RSpec + Selenium Demonstration 9/30 at 11 am PST

September 21st, 2010 by Ashley Wilson

Join Sean Grove, senior developer at Sauce Labs, as he shows you how to take your RSpec tests and pit ’em against the real world. RSpec is a Ruby framework that provides a Domain Specific Language that expresses executable examples of the expected behavior of your code.

Learn how to launch and control real browsers directly from RSpec tests using Selenium and Sauce OnDemand. In this live video demontration on Sept. 30 at 11 am PST, he’ll demonstrate and discuss:

* Writing your first RSpec test
* Configuring it to use Selenium
* Cross-browser testing and spec parallelization

Be sure to stick around for the Q&A following the presentation. Register Here!

Cucumber Sauce: Cross-browser testing in parallel

September 19th, 2010 by The Sauce Labs Team

I’ve been meaning to release a “standard” cucumber-in-parallel cross-browser project for awhile now, but I just got around to it this weekend. It’s called Cucumber Sauce.

Highlights:

  • Run your cucumber tests across different browsers all at the same time
  • Browsers are configurable via a yaml file
  • Multiple browser config files are easy to setup, and one can be passed in when running the rake task. Have a browsers_core.yml file to run while developing, and a browsers_full.yml to run before pushing to production
  • Designed to run tests ultra-cleanly. Adds time overhead initially, but tests are all independent of one another, and can be made to run in parallel at the scenario level in the future. Follow this template or you’ll lose that ability to speed up your tests.

Check it out at the Cucumber Sauce github repo.

Setting up Cucumber + Webrat + Selenium

June 15th, 2010 by The Sauce Labs Team

There’s quite a bit of information out there on getting these disparate tools to work together, but a great deal of it is out of date. To clear things up a bit, I’ve documented all the gems and modifications necessary to get these pieces of open source software up and running together. As time goes on, I’ll be expanding this blog post with notes about the pitfalls and various platform issues that may be discovered (I’m looking at you, Snow Leopard), but this should get most people up and running right away.

This is all using a clean REE environment via the poorly named but wonderfully written rvm, or Ruby Version Manager.

Gems you’ll need:

gem install actionmailer actionpack activerecord activeresource activesupport builder cgi_multipart_eof_fix cucumber cucumber-rails daemons database_cleaner diff-lcs fastthread gem_plugin gherkin json json_pure mime-types mongrel net-ssh net-ssh-gateway nokogiri rack rack-test rails rake rdoc rest-client rspec rspec-rails Selenium selenium-client sqlite3-ruby term-ansicolor trollop webrat

Some of those are not strictly necessary, but simply nice to have, while others solved some unexpected problems with the bare necessities. I’ll prune this list as feedback comes in from people’s experiences.

Points to watch out for:
Nokogiri: This was easily the worst on my Snow Leopard machine. It relies on the native libxml2, which had problems with 32/64 bit compatibility. No matter what I tried, errors kept coming up. I had to clean everything out with my MacPorts installation and force a universal installation.
Webrat: The Selenium server jar that was included by default caused no end of headaches. I had to manually go in to the directory, remove the default server jar, and download the newest version from the seleniumhq download page.

Once those are set up, you should be able to use Cucumber, Webrat, and Selenium together without too much headache. Then you can refer to our webinar video (which will be posted to the blog later this week) to learn how to use Cucumber to easily run Webrat and Selenium.

A bit of sugar and parallelism for Rails and RSpec

March 9th, 2010 by The Sauce Labs Team

Even though we focus very heavily on full-stack acceptance testing for the rails world, we know other forms of automated tests are critical as well. Our rails developers here make pretty heavy use of RSpec unit tests, and it’s nice to understand how to run those in parallel as well.

If you’re looking at how to setup a rails and selenium testing environment, check out our last post.

Parallelize the specs

We’ll use the excellent parallel_specs to beat a bit of parallelism into our specs. It prepares a separate database for each test environment, groups the specs to divide amongst processes, and then starts up a rails environment with a separate database for each group of processes.

I’ll paraphrase the installation instructions for convenience.

Install the required plug-in/gem:

sudo gem install parallel
script/plugin install git://github.com/grosser/parallel_specs.git

Here’s the semi-ingenious point – yaml can interpret ERB, so we can pass in an environment variable to the database.yml specifying at launch which database we want it to connect to.

Open config/database.yml and add the following:

test:
  adapter: sqlite3
  database: db/xxx_test<% ENV['TEST_ENV_NUMBER'] %>.sqlite3
  pool: 5
  timeout: 5000

(You can of course replace xxx_ with your project name)
So for example, to have our tests run against the xxx_test2 database, we would use:

export TEST_ENV_NUMBER=2; rake db:test:prepare

But it doesn’t make much sense to invoke it manually. That’s what plug-ins are for! Let’s go head and create/migrate a few test databases:

export TEST_ENV_NUMBER=0; rake db:test:create; rake db:test:migrate;
export TEST_ENV_NUMBER=1; rake db:test:create; rake db:test:migrate;
export TEST_ENV_NUMBER=2; rake db:test:create; rake db:test:migrate;

Great, now you’re able to run your non-Selenium tests in parallel!

But what about Selenium tests?

Stay tuned for our article on Sauce Labs’ SpecStorm plugin, that allows you to run your Selenium tests in true parallel fashion with Selenium Grid or our very own Sauce OnDemand service.

Running Selenium RSpec tests for Rails 2.3.5

February 9th, 2010 by The Sauce Labs Team

This article will get you setup with the bare-minimum environment to run Selenium tests with RSpec, for automated, full-stack testing of Rails apps. Once that’s done, we’ll work on polishing it for a nicer experience. In this post you find amalgamations of mostly outdated articles from around the internet updated to work with a modern rails system.

It’s written on a clean environment courtesy of rvm, so you may have some of the gems already setup.

A new Rails app, RSpec, and rspec-rails

First off, we’ll start with a new rails project:

gem install rails --no-ri --no-rdoc
rails rspec_saucerc
gem install rspec
gem install rspec-rails

Edit config/environment.rb and add:

  config.gem "rspec", :lib => false, :version => ">= 1.2.9"
  config.gem "rspec-rails", :lib => false, :version => ">= 1.2.9"

This sets up all the required files for RSpec to get off the ground, but we need to integrate it into Rails:

script/generate rspec

That will add the rake tasks, create the appropriate directories, and basic files. Now let’s get Selenium up and running!

Selenium

We’ll need to create a separate selenium database. in config/database.yml, let’s add:

selenium:
  adapter: sqlite3
  database: db/selenium.sqlite3
  encoding: utf8
  timeout: 5000

To get Selenium running under our RSpec stories, we’ll need the Selenium gem (note the capital “S” — it’s case-sensitive). Let’s also make sure we have the sqlite3 gem installed, and then prepare an appropriate environment for our selenium tests:

gem install Selenium
gem install sqlite3-ruby
cp config/environments/test.rb config/environments/selenium.rb

Open config/environments/selenium.rb and remove the last line that reads:

config.gem 'rspec-rails', :version => '>= 1.3.2', :lib => false unless File.directory?(File.join(Rails.root, 'vendor/plugins/rspec-rails'))

Sauce RC and Selenium RC

Sauce RC will broker all communication between rails and any browsers you might be driving. Get it from our downloads page for Windows or Mac, or use Selenium RC for Linux, and start it up so we can run our tests.

Testables

We’ll need some fodder to test. Use the RSpec generator, then create and migrate the database:

script/generate rspec_scaffold person name:string age:string language:string
rake RAILS_ENV=selenium db:create
rake RAILS_ENV=selenium db:migrate

Startup our test server

Rails tests do not normally bind to a webserver, so there is no way for Selenium to access the frontend. As a temporary workaround, we’ll manually invoke a Rails server using the selenium environment:

script/server -e selenium

Our spec helper

We will be using the Selenium gem, so let’s add it to the spec helper:

gem "selenium-client"
require "selenium/client"
require "selenium/rspec/spec_helper"

Example story

We’ll gloss over which types of tests should include selenium front-end testing for now, and just say it belongs in integration tests. Let’s run an example story that will fail the first time through, and we’ll then fix it.

Put the following in spec/integration/people_spec.rb:

require 'spec_helper'
describe "People" do
  before(:all) do
    @verification_errors = []

    @browser = Selenium::Client::Driver.new(
      :host => "localhost",
      :port => 4444,
      :browser => "*firefox"
      :url => "http://localhost:3000",
      :timeout_in_second => 90)

    @browser.start
  end

  before(:each) do
    @browser.start_new_browser_session
  end

  append_after(:each) do
    @browser.close_current_browser_session
    @verification_errors.should == []
  end

  it "should create a new Person with valid input" do
    @browser.open "/people"
    @browser.click "link=New person"
    @browser.wait_for_page_to_load "2000"
    @browser.type "person_name", "Jason Huggins"
    @browser.type "person_age", "26"
    @browser.type "person_language", "Albanian"
    @browser.click "person_submit"
    @browser.wait_for_page_to_load "30000"
    @browser.is_alert_present.should be_true
  end
end

Let’s try it out:

rake spec:integration
F
1) 'People should create a new Person with valid input' FAILED
expected false to be true
./spec/integration/people_spec.rb:22:

Finished in 7.443904 seconds

1 example, 1 failure

It’s failing as we expected it to (there shouldn’t be a javascript alert on submit). Let’s change the last line:

    @browser.is_alert_present.should be_false

… and try our test again:

rake spec:integration
.

Finished in 7.079062 seconds

1 example, 0 failures

Looks good! We finally have a working rails project with rspec runners and Selenium.

Weaknesses

Although we’ve got Rails, RSpec, and Selenium all working together, the relationship is not harmonious.

Automated tests in rails are not meant to serve the outside world, and as such they don’t bind to a webserver/port. Tests are run within the same process, which makes them quite fast, but prevents us from using browser-based frontend tools such as Selenium. To get around that, we manually fired up a rails server instance with the selenium environment, but this is messy for a number of reasons:

  • We have to manually start/stop the test server that selenium wants to access
  • We have to manually create the selenium environment’s database
  • We have to manually migrate the selenium environment each time there’s a schema change
  • Worse, we have to manually reset the database after each run
  • Running Selenium tests in serial is slow

We‘ve developed a plugin to automate some of these issues, called SpecStorm. We’ll go over installing it to get the most out of your tests (including running them in parallel) in the next post.

Notes:

Selenium matchers are case-sensitive: @browser.click “link=New Person” won’t match a link with “New person”