Posts Tagged ‘Best Practices’

JIRA is Just a Click Away with Our New Plugin

May 3rd, 2016 by Ken Drachnik

As more and more devs work in agile teams, they need tools to plan, track, and release software and many of them use JIRA. As a tracking tool, JIRA is amazing for collaboration and project planning. For many teams, JIRA is the place of record for everything in the software development lifecycle. We have found that many of our customers use JIRA and the #1 product ask was to integrate Sauce Labs’ test results with JIRA so it would be simple to track all the tests associated with a project in one place.

Let’s say you are running an automated or manual test on Sauce Labs and find a bug. You want to add it to JIRA so that someone on your team can take a look or so that it can be prioritized in the backlog. Historically, one would have to download all the Sauce assets, login to JIRA, create a ticket, and upload the assets again. This can be tedious when you’re running lots of tests.

With Sauce Labs for JIRA, this is all simplified and automated. With the click of a button you can now create a JIRA ticket directly from your Test Details page. The plugin gives you the option to upload the screenshots, logs, and video link to your tests, making it easy to share out among your team.

To download Sauce Labs for JIRA, visit the Atlassian Marketplace: https://marketplace.atlassian.com/plugins/sauce-jira-integration/cloud/overview.  To read more, visit our JIRA integration Docs page.

Happy Testing!

The Sauce Labs Ecosystems & Integrations Team

3 Simple Strategies to Get Started With Automation

March 24th, 2016 by Joe Nolan

If your test automation team’s directive is to automate X amount of tests, and you have no strategy as to which tests they should focus on, you are wasting your time. Before you begin writing your first line of automation code, make sure you have a strategy in place. Otherwise, you will have a ton of ineffective tests to maintain.

Don’t Choose a Random Goal

How many times have you been told that the goal of the team is to have X amount of test coverage? This is an arbitrary value picked out of the sky. What is it based on? If a UI automation team were to cover 80% of the stories in a sprint, they would never get done in time.

We all know how fragile UI automation is! How many times will a designer make a change that directly affects the UI and breaks the test? This is almost manageable during a sprint while you are working closely together, but how about when the product is sent to be translated to another language? The translator inevitably comes back with suggestions to allow for phrases more common and translatable. Bugs might be entered and UI changes made by a maintenance team with no heads up to the automation team, and Bam! — You have broken tests that need to be investigated. (more…)

Evangelizing Quality

November 6th, 2015 by Ashley Hunsberger

When I hear the word evangelist, all I can think about are those people on TV, preaching away. Not necessarily my cup of tea. As defined, an evangelist is someone who seeks to convert others to what he or she believes, especially by public preaching. I’ve never considered myself an evangelist — it goes against my general keep-to-myself nature. (Public speaking. Blech.) But I want to share what I know, and what I’ve learned, and I hope that others see the value in it. So, in the Church of Quality, I guess that makes me an evangelist. And we need more.

How do you get people to think more about quality?

It usually starts with a bad experience. Whether a customer issue came in, or you just had a very bad day testing, something negative tends to trigger a look at the quality of not just the product, but also the processes that broke. You need people to understand WHY quality is important, and why they should care. (more…)

Try Making It Smaller – It’s The Agile Way

November 4th, 2015 by Greg Sypolt

Agile is a must for development shops. Agile is a mature, iterative, collaborative methodology that breaks the development process down into shorter sprints. At its core, Agile development is about small iterations, test automation and a continuous integration pipeline.

Waterfall Was Created For a Perfect World, But We Don’t Live In One

Agile is a reaction to the slower, sequential approach known as Waterfall. Where Waterfall requires upfront planning to ensure that all details are accounted for, with no room for surprises or changes, Agile accounts for the inevitability of change, adapting to the project as it unfolds.

“Imagine a waterfall on the cliff of a steep mountain. Once the water has flowed over the edge of the cliff and has begun its journey down the side of the mountain, it cannot turn back. It is the same with waterfall development.”  (Search Software Quality)

To understand the advantages of Agile, it’s important to first understand the more traditional Waterfall methodology:

  • It is a sequential design process: discover, plan, build, test, review
  • Each project is based on the extensive collection of clear documentation gathered at the beginning
  • The whole product is only tested at the end of the cycle
  • It doesn’t take into account a client’s evolving needs or leave any room for revision (more…)

What is your definition of “Done”?

September 17th, 2015 by Greg Sypolt

Introduction

Why does a daily standup or scrum team have a definition of done (DoD)? It’s simple – everyone involved in a project needs to know and understand what “done” means.

What is DoD? It is a clear and concise list of requirements a software increment must adhere to in order to be considered a completed user story, sprint, or be considered ready for release. However, for organizations just starting to apply Agile methods, it might be impossible to reach immediately. Your organization needs to identify the problems and work as a team to build your version of DoD to solve them.

The Problem

The following conversation occurs during your daily standup:

Product Manager (PM): “Is the user story done?”
Developer (Dev): “Yes!”
Quality Assurance (QA): “Okay, we will execute our manual and/or automated tests today.”

Later that same day:

QA: “We found several issues, did Dev perform any code reviews or write any unit tests?”
PM (to Dev): “QA found several issues, did you do any code reviews or unit testing?”
Dev: “No, the code was simple. It was going to take too much time to write unit tests.”

Has this ever happened to you? (more…)

Getting the Existing Team On Board with Automation (Scripts)

August 27th, 2015 by Greg Sypolt

Introduction

In an attempt to do more with less, organizations want to test their software adequately, as quickly as possible. Businesses are demanding quick turnaround, pushing new features and bug fixes to production within days, along with quality. Everyone knows manual testing is labor-intensive and error-prone, and does not support the same kind of quality checks that are possible through the use of an automated test. Nobody likes change, but it’s time to educate your team on the importance of onboard automated testing.

The only way to make sense out of change is to plunge into it, move with it, and join the dance.  – Alan Watts

Everyone has had a job interview at some point in their lives, right? It is important to be prepared! The first few minutes of an interview are a make or break moment. Why? Because first impressions can have long-lasting effects. Never underestimate the power of first impressions. The same principle applies when onboarding automation to an existing manual testing team. Your initial presentation to your team or organization should be treated like a job interview. Be prepared. Deliver expectations and explain responsibilities — it’s critical since it is normal for employees to have an emotional reaction to anything they view as a job threat.

Why automated testing?

If things are going well, why do we want to implement automated tests? The demand is to do more with less, which makes manual testing an impossible task, but introducing automated testing into an existing software development lifecycle can also be daunting. However, when implemented, automated testing is a valuable asset that shortens testing cycles and helps teams become more agile.

marketingautomation1011

image source: http://marketing-works.net/

(more…)

Appium Bootcamp – Chapter 5: Writing and Refactoring Your Tests

August 5th, 2014 by Bill McGee

appium_logoThis is the fifth post in a series called Appium Bootcamp by noted Selenium expert Dave Haeffner

Read:  Chapter 1 Chapter 2 | Chapter 3 | Chapter 4 | Chapter 5 | Chapter 6 | Chapter 7 | Chapter 8

Dave recently immersed himself in the open source Appium project and collaborated with leading Appium contributor Matthew Edwards to bring us this material. Appium Bootcamp is for those who are brand new to mobile test automation with Appium. No familiarity with Selenium is required, although it may be useful. This is the fifth of eight posts; two new posts will be released each week.

Now that we’ve identified some test actions in our apps, let’s put them to work by wiring them up in code.

We’ll start with the iOS app and then move onto Android. But first, we’ll need to do a quick bit of setup.

Quick Setup

Since we’re setting up our test code from scratch, we’ll need to make sure we have the necessary gems installed — and done so in a way that is repeatable (which will come in handy for other team members and for use with Continuous Integration).

In Ruby, this is easy to do with Bundler. With it you can specify a list of gems and their versions to install and update from for your project.

Install Bundler by running gem install bundler from the command-line and then create a file called Gemfile with the following contents:

# filename: Gemfile

source 'https://rubygems.org'

gem 'rspec', '~> 3.0.0'
gem 'appium_lib', '~> 4.0.0'
gem 'appium_console', '~> 1.0.1'

After creating the Gemfile run bundle install. This will make sure rspec (our testing framework), appium_lib (the Appium Ruby bindings), and appium_console (our interactive test console) are installed and ready for use in this directory.

Capabilities

In order to run our tests, we will need to specify the capabilities of our app. We can either do this in our test code, or we can leverage the appium.txt files we used for the Appium Console.

Let’s do the latter approach. But first, we’ll want to create two new folders; one for Android and another for iOS. Once they’re created, let’s place each of the appium.txt files into their respective folders.

├── Gemfile
├── Gemfile.lock
├── android
│   └── appium.txt
└── ios
    └── appium.txt

Be sure to update the app capability in your appium.txt files if you’re using a relative path.

Writing Your First Test

With our initial setup taken care of, let’s create our first test file (a.k.a. “spec” in RSpec). The test actions we identified in the previous post were focused on navigation in the app. So let’s call this spec file navigation_spec.rband place it in the ios folder.

├── Gemfile
├── Gemfile.lock
├── android
│   └── appium.txt
└── ios
    └── appium.txt
    └── navigation_spec.rb

Now let’s write our test to launch Appium for iOS and perform a simple navigation test.

In RSpec, describe denotes the beginning of a test file, whereas it denotes a test. So what we have is a test file with a single test in it.

In this test file, we are starting our Appium session before each test (e.g., before(:each)) and ending it after each test (e.g., after(:each)). More specifically, in before(:each), we are finding the path to the iOSappium.txt file and then loading it. After that we start the Appium session and promote the Appium commands so they will be available for use within our test. We then issue driver_quit in after(:each) to cleanly end the Appium session. This is equivalent to submitting an x command in the Appium console.

The commands in our test (it 'First cell' do) should look familiar from the last post. We’re finding the first cell, grabbing it’s title, click on the cell, and then looking to see if the title appeared on the inner screen.

After saving this file, let’s change directories into the ios folder (e.g., cd ios), and run the test (assuming your Appium Server is running — if not, load up the Appum GUI and click Launch) with rspec navigation_spec.rb. When it’s running, you will see the iOS simulator launch, load up the test app, click the first cell, and then close.

This is a good start, but we can clean this code up a bit by leveraging some simple page objects and a central configuration.

A Page Objects Primer

Automated tests can quickly become brittle and hard to maintain. This is largely due to the fact that we are testing functionality that will constantly change. In order to combat this, we can use page objects.

Page Objects are simple objects that model the behavior of an application. So rather than writing your tests directly against your app, you can write them against these objects. This will make your test code more reusable, maintainable, and easier to fix when the app changes.

You can learn more about page objects here and here.

Refactoring Your First Test

Let’s create a new directory called pages within our ios directory and create two new files in it: home.rb and inner_screen.rb. And while we’re at it, let’s create a new folder to store our test files (called spec — which is a folder RSpec will know to look for at run time) and move our navigation_spec.rb into it.

├── Gemfile
├── Gemfile.lock
├── android
│   └── appium.txt
└── ios
    ├── appium.txt
    ├── pages
    │   ├── home.rb
    │   └── inner_screen.rb
    └── spec
        ├── navigation_spec.rb

Let’s open up ios/pages/home.rb to create our first page object.

 

# filename: ios/pages/home.rb

module Pages
  module Home
    class << self

      def first_cell
        @found_cell = wait { text 2 }
        self
      end

      def title
        @found_cell.name.split(',').first
      end

      def click
        @found_cell.click
      end

    end
  end
end

module Kernel
  def home
    Pages::Home
  end
end

Since the Appium commands are getting promoted for use (instead of passing around a driver object), storing our page objects in a module is a cleaner approach (rather than keeping them in a class that we would need to instantiate).

To create the Home module we first wrap it in another module called Pages. This helps prevent any namespace collisions as well simplify the promotion of Appium methods.

In Home, we’ve created some simple static methods to mimic the behavior of the home screen (e.g., first_celltitleclick). By storing the found cell in an instance variable (e.g., @found_cell) and returning self, we will be able to chain these methods together in our test (e.g., first_cell.title). And in order to cleanly reference the page object in our test, we’ve made the home method available globally (which references this module).

Now let’s open up ios/pages/inner_screen.rb and create our second page object.

# filename: pages/inner_screen.rb

module Pages
  module InnerScreen
    class << self

      def has_text(text)
        wait { text_exact text }
      end

    end
  end
end

module Kernel
  def inner_screen
    Pages::InnerScreen
  end
end

This is the same structure as our previous page object. In it, we’re performing an exact text search.

Let’s go ahead and update our test to use these page objects.

# filename: ios/spec/navigation_spec.rb

require 'appium_lib'
require_relative '../pages/home'
require_relative '../pages/inner_screen'

describe 'Home Screen Navigation' do

  before(:each) do
    appium_txt = File.join(Dir.pwd, 'appium.txt')
    caps = Appium.load_appium_txt file: appium_txt
    Appium::Driver.new(caps).start_driver
    Appium.promote_appium_methods RSpec::Core::ExampleGroup
    Appium.promote_singleton_appium_methods Pages
  end

  after(:each) do
    driver_quit
  end

  it 'First cell' do
    cell_title = home.first_cell.title
    home.first_cell.click
    inner_screen.has_text cell_title
  end

end

We first require the page objects (note the use of require_relative at the top of the file). We then promote the Appium methods to our page objects (e.g., Appium.promote_singleton_appium_methods Pages). Lastly, we update our test.

Now when we run our test from within the ios directory (e.g., cd ios then rspec) then it will run just the same as it did before.

Now the test is more readable and in better shape. But there is still some refactoring to do to round things out. Let’s pull our test setup out of this test file and into a central config that we will be able to leverage for both iOS and Android.

Central Config

In RSpec, we can configure our test suite from a central location. This is typically done in a file called spec_helper.rb. Let’s create a folder called common in the root of our project and add a spec_helper.rb file to it.

├── Gemfile
├── Gemfile.lock
├── android
│   └── appium.txt
├── common
│   └── spec_helper.rb
└── ios
    ├── appium.txt
    ├── pages
    │   ├── home.rb
    │   └── inner_screen.rb
    └── spec
        ├── navigation_spec.rb

Let’s open up common/spec_helper.rb, add our test setup to it, and polish it up.

 

# filename: common/spec_helper.rb

require 'rspec'
require 'appium_lib'

def setup_driver
  return if $driver
  caps = Appium.load_appium_txt file: File.join(Dir.pwd, 'appium.txt')
  Appium::Driver.new caps
end

def promote_methods
  Appium.promote_singleton_appium_methods Pages
  Appium.promote_appium_methods RSpec::Core::ExampleGroup
end

setup_driver
promote_methods

RSpec.configure do |config|

  config.before(:each) do
    $driver.start_driver
  end

  config.after(:each) do
    driver_quit
  end

end

After requiring our requisite libraries, we’ve created a couple of methods that get executed when the file is loaded. One is to setup (but not start) Appium and another is to promote the methods to our page objects and tests. This approach is taken to make sure that only one instance of Appium is loaded at any one time.

We then configure our test actions so they run before and after each test. In them we are starting an Appium session and then ending it.

In order to use this central config, we will need to require it (and remove the unnecessary bits) in our test.

# filename: ios/spec/navigation_spec.rb

require_relative '../pages/home'
require_relative '../pages/inner_screen'
require_relative '../../common/spec_helper'

describe 'Home Screen Navigation' do

  it 'First cell' do
    cell_title = home.first_cell.title
    home.first_cell.click
    inner_screen.has_text cell_title
  end

end

Note the order of the require_relative statements — they are important. We need to load our page objects before we can load our spec_helper, or else the test won’t run.

If we run the tests from within the ios directory with rspec, we can see everything execute just like it did before.

Now that we have iOS covered, let’s wire up an Android test, some page objects, and make sure our test code to supports both devices.

Including Android

It’s worth noting that in your real world apps you may be able to have a single set of tests and segmented page objects to help make things run seamlessly behind the scenes for both devices. And while the behavior in our Android test app is similar to our iOS test app, it’s design is different enough that we’ll need to create a separate test and page objects.

Let’s start by creating spec and pages folders within the android directory and then creating page objects in pages (e.g., home.rb and inner_screen.rb) and a test file in spec (e.g., navigation_spec.rb).

├── Gemfile
├── Gemfile.lock
├── android
│   ├── appium.txt
│   ├── pages
│   │   ├── home.rb
│   │   └── inner_screen.rb
│   └── spec
│       ├── navigation_spec.rb
├── common
│   └── spec_helper.rb
└── ios
    ├── appium.txt
    ├── pages
    │   ├── home.rb
    │   └── inner_screen.rb
    └── spec
        ├── navigation_spec.rb

Now let’s open and populate our page objects and test file.

module Pages
  module Home
    class << self

      def first_cell
        @found_cell = wait { text 2 }
        self
      end

      def click
        @found_cell.click
      end

    end
  end
end

module Kernel
  def home
    Pages::Home
  end
end

This page object is similar to the iOS one except there’s no title search (since we won’t be needing it).

module Pages
  module InnerScreen
    class << self

      def has_text(text)
        wait { find_exact text }
      end

    end
  end
end

module Kernel
  def inner_screen
    Pages::InnerScreen
  end
end

In this page object we’re performing a search for an element by text (similar to the iOS example), but using find_exact instead of text_exact because of how the app is designed (we need to perform a broader search that will search across multiple attributes, not just the text attribute).

Now let’s wire up our test.

require_relative '../pages/home'
require_relative '../pages/inner_screen'
require_relative '../../common/spec_helper'

describe 'Home Screen Navigation' do

  it 'First cell' do
    home.first_cell.click
    inner_screen.has_text 'Accessibility Node Provider'
  end

end

Now if we cd into the android directory and run our test with rspec it should launch the Android emulator, load the app, click the first cell, and then end the session. The emulator will remain open, but that’s something we’ll address in a future post.

One More Thing

If we use the console with the code that we have right now, we won’t be able to reference the page objects we’ve created — which will be a bit of a pain if we want to reference them when debugging test failures. Let’s fix that.

Let’s create a new file in our android/spec and ios/spec directories called requires.rb. We’ll move our require statements out of our test files and into these files instead.

├── Gemfile
├── Gemfile.lock
├── android
│   ├── appium.txt
│   ├── pages
│   │   ├── home.rb
│   │   └── inner_screen.rb
│   └── spec
│       ├── navigation_spec.rb
│       └── requires.rb
├── common
│   └── spec_helper.rb
└── ios
    ├── appium.txt
    ├── pages
    │   ├── home.rb
    │   └── inner_screen.rb
    └── spec
        ├── navigation_spec.rb
        └── requires.rb

Here’s what one of them should look like:

# filename: ios/spec/requires.rb

# require the ios pages
require_relative '../pages/home'
require_relative '../pages/inner_screen'

# setup rspec
require_relative '../../common/spec_helper'

Next, we’ll want to update our tests to use this file.

require_relative 'requires'

describe 'Home Screen Navigation' do

  it 'First cell' do
    cell_title = home.first_cell.title
    home.first_cell.click
    inner_screen.has_text cell_title
  end

end
# filename: android/spec/navigation_spec.rb

require_relative 'requires'

describe 'Home Screen Navigation' do

  it 'First cell' do
    home.first_cell.click
    inner_screen.has_text 'Accessibility Node Provider'
  end

end

Now that we have a central requires.rb for each device, we can tell the Appium Console to use it. To do that, we’ll need to add some additional info to our appium.txt files.

 

# filename: ios/appium.txt

[caps]
deviceName = "iPhone Simulator"
platformName = "ios"
app = "../../../apps/UICatalog.app.zip"

[appium_lib]
require = ["./spec/requires.rb"]
# filename: android/appium.txt

[caps]
platformName = "android"
app = "../../../apps/api.apk"
avd = "training"
deviceName = "Android"

[appium_lib]
require = ["./spec/requires.rb"]

This new require value is only used by the Appium Console. Now if we run arc from either the ios or android directories, we’ll be able to access the page objects just like in our tests.

And if we run our tests from either directory, they will still work as directed.

Outro

Now that we have our tests, page objects, and central configuration all sorted, it’s time to look at wrapping our test execution and make it so we can run our tests in the cloud.

Read:  Chapter 1 Chapter 2 | Chapter 3 | Chapter 4 | Chapter 5 | Chapter 6 | Chapter 7 | Chapter 8

About Dave Haeffner: Dave is a recent Appium convert and the author of Elemental Selenium (a free, once weekly Selenium tip newsletter that is read by thousands of testing professionals) as well as The Selenium Guidebook (a step-by-step guide on how to use Selenium Successfully). He is also the creator and maintainer of ChemistryKit (an open-source Selenium framework). He has helped numerous companies successfully implement automated acceptance testing; including The Motley Fool, ManTech International, Sittercity, and Animoto. He is a founder and co-organizer of the Selenium Hangout and has spoken at numerous conferences and meetups about acceptance testing.

Follow Dave on Twitter – @tourdedave

Appium Bootcamp – Chapter 4: Your First Test

July 31st, 2014 by Bill McGee

appium_logoThis is the fourth post in a series called Appium Bootcamp by noted Selenium expert Dave Haeffner

Read:  Chapter 1 Chapter 2 | Chapter 3 | Chapter 4 | Chapter 5 | Chapter 6 | Chapter 7 | Chapter 8

Dave recently immersed himself in the open source Appium project and collaborated with leading Appium contributor Matthew Edwards to bring us this material. Appium Bootcamp is for those who are brand new to mobile test automation with Appium. No familiarity with Selenium is required, although it may be useful. This is the fourth of eight posts; two new posts will be released each week.

There are a good deal of similarities between Selenium and Appium tests. We will be using similar actions (like click) along with some kind of wait mechanism (e.g., an explicit wait) to make our tests more resilient. There will also be an assertion used to determine if our actions were successful or not.

In order to put these concepts to work, let’s consider the basic structure of the test apps we’ve been working with. They are straightforward in that they both have text elements that, when clicked, take you to a dedicated page for that element (e.g., Accessibility triggers the Accessibility page). Let’s step through our first set of test actions (in the console) that we’ll use to automate this behavior; verifying that each element brings us to the correct page.

Let’s dig in with some examples.

An iOS Example

The behavior of our app can be easily mapped to test actions by first using a text match to find the element we want, and clicking it. We can then make sure we are in the right place by performing another text match (this time an exact text match). When we wire this up to our test framework, this match will be responsible for passing or failing the test. More on that in the next post.

text('Buttons, Various uses of UIButton').click
text_exact 'Buttons'

The only problem with this approach is that it is not resilient. The global wait for each test action (a.k.a. an implicit wait) is set to 0 seconds by default. So if there is any delay in the app, the test action will not complete and throw an element not found exception instead.

To overcome these timing problems we can employ an explicit wait around our test actions (both the click and the exact text match). This is simple enough to do with the wait command.

wait { text('Buttons, Various uses of UIButton').click }
wait { text_exact 'Buttons' }

These test actions are resilient now, but they’re inflexible since we were using statically coded values. Let’s fix that by using dynamic values instead.

cell_1 = wait { text 2 }
cell_title = cell_1.name.split(',').first

wait { cell_1.click }
wait { text_exact cell_title }

Now we’re finding the first text by it’s index. Index 2 contains the first element (a.k.a. a cell), whereas index 1 is the table header. After that, we’re extracting the name and dynamically finding the title. Now our test will continue to work if there are any text changes.

This is good, but now let’s expand things to cover the rest of the app.

cell_names = tags('UIATableCell').map { |cell| cell.name }

cell_names.each do |name|
  wait { text_exact(name).click }
  wait { text_exact name.split(',').first }
  wait { back }
end

We first grab the names of each clickable cell, storing them in a collection. We then iterate through the collection, finding each element by name, clicking it, performing an exact match on the resulting page, and then going back to the main screen. This is repeated until each cell is verified.

This works for cells that are off the screen (e.g., out of view) since Appium will scroll them into view before taking an action against them.

An Android Example

Things are pretty similar to the iOS example. We perform a text match, click action, and exact text match.

text('Accessibility').click
text_exact 'Accessibility Node Provider'

We then make things resilient by wrapping them in an explicit wait.

wait { text('Accessibility').click }
wait { text_exact 'Accessibility Node Provider' }

We then make our selection more flexible by upgrading to dynamic values.

cell_1 = wait { text 2 }

wait { cell_1.click }
wait { find_exact 'Accessibility Node Provider' }

We then expand things to exercise the whole app by collecting all of the clickable elements and iterating through them.

cell_names = tags('android.widget.TextView').map { |cell| cell.name }

cell_names[1..-1].each do |cell_name|
  wait { scroll_to_exact(cell_name).click }
  wait_true { ! exists { find_exact cell_name } }
  wait { back }
  wait { find_exact('Accessibility'); find_exact('Animation')  }
end

A few things to note.

The first item in the cell_names collection is a header. To discard it, we use cell_name[1..-1] which basically says start with the second item in the collection (e.g., [1) and continue (e.g., ..) all the way until the end (e.g.,-1]).

In order to interact with cells that are off the screen, we will need to use the scroll_to_exact command, and perform a click against that (instead of a text match).

Since each sub-screen doesn’t have many unique attributes for us to verify against, we can at the very least verify that we’re no longer on the home screen. After that, we verify that we are brought back to the home screen.

Outro

Now that we have our test actions sussed out, we’re ready to commit them to code and plug them into a test runner.

Read:  Chapter 1 Chapter 2 | Chapter 3 | Chapter 4 | Chapter 5 | Chapter 6 | Chapter 7 | Chapter 8

About Dave Haeffner: Dave is a recent Appium convert and the author of Elemental Selenium (a free, once weekly Selenium tip newsletter that is read by thousands of testing professionals) as well as The Selenium Guidebook (a step-by-step guide on how to use Selenium Successfully). He is also the creator and maintainer of ChemistryKit (an open-source Selenium framework). He has helped numerous companies successfully implement automated acceptance testing; including The Motley Fool, ManTech International, Sittercity, and Animoto. He is a founder and co-organizer of the Selenium Hangout and has spoken at numerous conferences and meetups about acceptance testing.

Follow Dave on Twitter – @tourdedave

 

Appium Bootcamp – Chapter 3: Interrogating Your App

July 29th, 2014 by Bill McGee

appium_logoThis is the third post in a series called Appium Bootcamp by noted Selenium expert Dave Haeffner

Read:  Chapter 1 Chapter 2 | Chapter 3 | Chapter 4 | Chapter 5 | Chapter 6 | Chapter 7 | Chapter 8

Dave recently immersed himself in the open source Appium project and collaborated with leading Appium contributor Matthew Edwards to bring us this material. Appium Bootcamp is for those who are brand new to mobile test automation with Appium. No familiarity with Selenium is required, although it may be useful. This is the third of eight posts; a new post will be released each week.

Writing automated scripts to drive an app in Appium is very similar to how it’s done in Selenium. We first need to choose a locator, and use it to find an element. We can then perform an action against that element.

In Appium, there are two approaches to interrogate an app to find the best locators to work with. Through the Appium Console, or through an Inspector (e.g., Appium Inspector, uiautomatorviewer, or selendroid inspector).

Let’s step through how to use each of them to decompose and understand your app.

Using the Appium Console

Assuming you’ve followed along with the last two posts, you should have everything setup and ready to run.

Go ahead and startup your Appium server (by clicking Launch in the Appium GUI) and start the Appium Ruby Console (by running arc in a terminal window that is in the same directory as your appium.txt file). After it loads you will see an emulator window of your app that you can interact with as well as an interactive prompt for issuing commands to Appium.

The interactive prompt is where we’ll want to focus. It offers a host of readily available commands to quickly give us insight into the elements that make up the user interface of the app. This will help us easily identify the correct locators to automate our test actions against.

The first command you’ll want to know about is page. It gives you access to every element in the app. If you run it by itself, it will output all of the elements in the app, which can be a bit unwieldy. Alternatively you can specify additional arguments along with it. This will filter the output down to just a subset of elements. From there, there is more information available that you can use to further refine your results.

Let’s step through some examples of that and more for both iOS and Android.

An iOS Example

To get a quick birds eye view of our iOS app structure, let’s get a list of the various element classes available. With the page_class command we can do just that.

[1] pry(main)> page_class
get /source
13x UIAStaticText
12x UIATableCell
4x UIAElement
2x UIAWindow
1x UIATableView
1x UIANavigationBar
1x UIAStatusBar
1x UIAApplication

UIAStaticText and all of the others are the specific class names for types of elements in iOS. You can see reference documentation for UIAStaticText here. If you want to see the others, go here.

With the page command we can specify a class name and see all of the elements for that type. When specifying the element class name, we can either specify it as a string, or a symbol (e.g., 'UIAStaticText' or:UIAStaticText).

[2] pry(main)> page :UIAStaticText
get /context
post /execute
{
    :script => "UIATarget.localTarget().frontMostApp().windows()[0].getTree()"
}
UIAStaticText
   name, label, value: UICatalog
UIAStaticText
   name, label: Buttons, Various uses of UIButton
   id: ButtonsTitle   => Buttons
       ButtonsExplain => Various uses of UIButton
UIAStaticText
   name, label: Controls, Various uses of UIControl
   id: ControlsExplain => Various uses of UIControl
       ControlsTitle   => Controls
UIAStaticText
   name, label: TextFields, Uses of UITextField
   id: TextFieldExplain => Uses of UITextField
       TextFieldTitle   => TextFields
...

Note the get and post (just after we issue the command but before the element list). It is the network traffic that is happening behind the scenes to get us this information from Appium. The response to post /execute has a script string. In it we can see which window this element lives in (e.g., windows()[0]).

This is important because iOS has the concept of windows, and some elements may not appear in the console output even if they’re visible to the user in the app. In that case, you could list the elements in other pages (e.g.,page window: 1). 0 is generally the elements for your app, whereas 1 is where the system UI lives. This will come in handy when dealing with alerts.

Finding Elements

Within each element of the list, notice their properties — things like namelabelvalue, and id. This is the kind of information we will want to reference in order interact with the app.

Let’s take the first element for example.

UIAStaticText
   name, label, value: UICatalog

In order to find this element and interact with it, we can can search for it with a couple of different commands: findtext, or text_exact.

> find('UICatalog')
...
#
> text('UICatalog')
...
#
> text_exact('UICatalog')
...
#

We’ll know that we successfully found an element when we see a Selenium::WebDriver::Element object returned.

It’s worth noting that in the underlying gem that enables this REPL functionality, if we end our command with a semi-colon it will not show us the return object.

> find('UICatalog')
# displays returned value

> find('UICatalog');
# returned value not displayed

To verify that we have the element we expect, let’s access the name attribute for it.

> find('UICatalog').name
...
"UICatalog"

Finding Elements by ID

A better approach to find an element would be to reference its id, since it is less likely to change than the text of the element.

UIAStaticText
   name, label: Buttons, Various uses of UIButton
   id: ButtonsTitle   => Buttons
       ButtonsExplain => Various uses of UIButton

On this element, there are some IDs we can reference. To find it using these IDs we can use the id command. And to confirm that it’s the element we expect, we can ask it for its name attribute.

> id('ButtonsTitle').name
...
"Buttons, Various uses of UIButton"

For a more thorough walk through and explanation of these commands (and some additional ones) go here. For a full list of available commands go here.

An Android Example

To get a quick birds eye view of our Android app structure, let’s get a list of the various element classes available. With the page_class command we can do just that.

[1] pry(main)> page_class
get /source
12x android.widget.TextView
1x android.view.View
1x android.widget.ListView
1x android.widget.FrameLayout
1x hierarchy

android.widget.TextView and all of the others are the specific class names for types of elements in Android. You can see reference documentation for TextView here. If you want to see the others, simply do a Google search for the full class name.

With the page command we can specify a class name and see all of the elements for that type. When specifying the element class name, we can specify it as a string (e.g., 'android.widget.TextView').

[2] pry(main)> page 'android.widget.TextView'
get /source
post /appium/app/strings

android.widget.TextView (0)
  text: API Demos
  id: android:id/action_bar_title
  strings.xml: activity_sample_code

android.widget.TextView (1)
  text, desc: Accessibility
  id: android:id/text1

android.widget.TextView (2)
  text, desc: Animation
  id: android:id/text1
...

Note the get and post (just after we issue the command but before the element list). It is the network traffic that is happening behind the scenes to get us this information from Appium. get /source is to download the source code for the current view and post /appium/app/strings gets the app’s strings. These app strings will come in handy soon, since they will be used for some of the IDs on our app’s elements; which will help us locate them more easily.

Finding Elements

Within each element of the list, notice their properties — things like text and id. This is the kind of information we will want to reference in order interact with the app.

Let’s take the first element for example.

android.widget.TextView (0)
  text: API Demos
  id: android:id/action_bar_title
  strings.xml: activity_sample_code

In order to find that element and interact with it, we can search for it by text or by id.

> text('API Demos')
...
#
> id('android:id/action_bar_title')
...
#

We’ll know that we successfully found an element when we see a Selenium::WebDriver::Element object returned.

It’s worth noting that in the underlying gem that enables this REPL functionality, if we end our command with a semi-colon it will not show us the return object.

> text('API Demos')
# displays returned value

> text('API Demos');
# returned value not displayed

To verify we’ve found the element we expect, let’s access the name attribute for it.

> text('API Demos').name
...
"API Demos"

Finding Elements by ID

A better approach to find an element would be to reference its ID, since it is less likely to change than the text of the element.

In Android, there are a two types of IDs you can search with — a resource ID, and strings.xml. Resource IDs are best. But strings.xml are a good runner-up.

android.widget.TextView (10)
  text, desc: Text
  id: android:id/text1
  strings.xml: autocomplete_3_button_7

This element has one of each. Let’s search using each with the id command.

# resource ID
> id('android:id/text1')
...
#

# strings.xml
> id('autocomplete_3_button_7')
...
#

You can see a more thorough walk through of these commands here. For a full list of available commands go here.

Ending the session

In order to end the console session, input the x command. This will cleanly quit things for you. If a session is not ended properly, then Appium will think it’s still in progress and block all future sessions from working. If that happens, then you need to restart the Appium server by clicking Stop and then Launch in the Appium GUI.

x only works within the console. In our test scripts, we will use driver.quit to kill the session.

Using An Inspector

With the Appium Ruby Console up and running, we also have access to the Appium Inspector. This is another great way to interrogate our app to find locators. Simply click the magnifying glass in the top-right hand corner of the Appium GUI (next to the Launch button) to open it. It will load in a new window.

Once it opens, you should see panes listing the elements in your app. Click on an item in the left-most pane to drill down into the elements within it. When you do, you should see the screenshot on the right-hand side of the window auto-update with a red highlight around the newly targeted element.

You can keep doing this until you find the specific element you want to target. The properties of the element will be outputted in the Details box on the bottom right-hand corner of the window.

It’s worth noting that while the inspector works well for iOS, there are some problem areas with it in Android at the moment. To that end, the Appium team encourages the use of uiautomatorviewer (which is an inspector tool provided by Google that provides similar functionality to the Appium inspector tool). For more info on how to set that up, read this.

For older Android devices and apps with webviews, you can use the selendroid inspector. For more information on, go here.

There’s loads more functionality available in the inspector, but it’s outside the scope of this post. For more info I encourage you to play around with it and see what you can find out for yourself.

Outro

Now that we know how to locate elements in our app, we are ready to learn about automating some simple actions and putting them to use in our first test.

Read:  Chapter 1 Chapter 2 | Chapter 3 | Chapter 4 | Chapter 5 | Chapter 6 | Chapter 7 | Chapter 8

About Dave Haeffner: Dave is a recent Appium convert and the author of Elemental Selenium (a free, once weekly Selenium tip newsletter that is read by thousands of testing professionals) as well as The Selenium Guidebook (a step-by-step guide on how to use Selenium Successfully). He is also the creator and maintainer of ChemistryKit (an open-source Selenium framework). He has helped numerous companies successfully implement automated acceptance testing; including The Motley Fool, ManTech International, Sittercity, and Animoto. He is a founder and co-organizer of the Selenium Hangout and has spoken at numerous conferences and meetups about acceptance testing.

Follow Dave on Twitter – @tourdedave

Guest Post: Bridging the Test Divide – Beyond Testing For A Release

June 16th, 2014 by Bill McGee

This is the second of a three part series by Matthew Heusser, software delivery consultant and writer. 

When I start to think about testing, I think about it in two broad strokes: new feature testing and release-testing. New feature testing tries to find problems with something new and specific, while release-testing happens after “code complete”, to make sure the whole system works together, that a change here didn’t break something there.

Release-testing (which some call regression testing) slows down the pace of release and delays feedback from our customer. Release-testing also increases cycle time – the time from when we begin work on a feature until it hits production. Over time, as our software become more complex, the amount of testing we want to do during release testing goes up.

Meanwhile, teams want to ship more often, to tighten the feedback loop.

Today I am going to talk about making release testing go away – or at least drastically reducing it.

It all starts during that tutorial in Spain I wrote about last time.

Two Worlds

The frequency of release for the people in my tutorial was very diverse, but two groups really struck me — the telecom that had a four-month test-release cycle, and the Latvian software team with the capability to deploy to production every single day.

That means arriving at the office the morning, looking at automated test runs, and making a decision to deploy.

There is a ‘formula’ to make this possible. It sounds simple and easy:

  • Automate a large number of checks on every build
  • Automate deploy to production
  • Continuously monitor traffic and logs for errors
  • Build the capability to rollback on failure

That transforms the role of test from doing the “testing we always do” to looking at the risk for a given release, lining it up against several different test strategies, and balancing risk, opportunity, reward, and time invested in release-testing?

The trick is to stop looking at the software as a big box, but instead to see it as a set of components. The classic set of components are large pieces of infrastructure (the configuration of the web server, the connections to the database, search, login, payment) and the things that sit on top of that – product reviews, comments, static html pages, and so on. Develop at least two de-ploy strategies — one for audited and mission-critical systems (essential infrastructure, etc) and another for components and add-ons.

We’ve been doing it for years in large IT organizations, where different systems have different release cycles; the trick is to split up existing systems, so you can recognize and make low-risk changes easier.

This isn’t something I dreamed up; both Zappos and Etsy have to pass PCI audits for financial services, while Zappos is part of Amazon and publicly traded. Both of these organizations have a sophisticated test-deploy process for parts of the application that touch money, and a simpler process for lower-risk changes.

So split off the system into different components that can be tested in isolation. Review the changes (perhaps down to the code level) to consider the impact of the change, and test the appropriate amount.

This can free up developers to make many tiny changes per day as long as those changes are low risk. Bigger changes along a theme can be batched together to save testing time — and might mean we can deploy with still considerably less testing than a ‘full’ site retest.

But How Do We Test It?

A few years ago, the ideal vision of getting away from manual, documented test cases was a single ‘test it’ button combined with a thumbs up or down at the end of an “automated test run.”

If the risk is different for each release, and we are uncomfortable with our automation, then we actually want to run different tests for each release — exactly what thinking testers (indeed, anyone on the team) can do with exploratory testing.

So let the computers provide some automated checks, all the time. Each morning, maybe every half an hour, we get a report, look at the changes, and decide what is the right thing for this re-lease. That might mean full-time exploratory testing of major features for a day or two, it might be emailing the team and asking everyone to spend a half hour testing in production.

This result is grown up software testing, varying the test approach to balance risk with cost.

The first step that I talked about today is separating components and developing a strategy that changes the test effort based on which parts were changed. If the risk is minimal, then deploy it every day. Hey, deploy it every hour.

This formula is not magic. Companies that try it find engineering challenges. The first build/deploy system they write tends to become hard to maintain over time. Done wrong continuous testing creates systematic and organizational risk.

It’s also a hard sell. So let’s talk about ways to change the system to shrink the release-test cycle, deploy more often, and reduce risk. The small improvements we make will stand on their own, not threaten anyway — and allow us to stop at any time and declare victory!

A Component Strategy

that_badWhen a company like etsy.com says that new programmers commit and push code to production the first day, do they really mean modifications to payment processing, search, or display for all products?

Of course not.

Instead, programmers follow a well-written set of directions to … wait for it … add the new user to the static HTML ‘about us’ page that lists all the employees, along with an image. If this change generates a bug, that will probably result in an X over an image the new hire forgot to upload, or maybe, at worst, break a div tag so the page mis-renders.

A bad commit on day one looks like this – not a bungled financial transaction in production.

How much testing should we have for that? Should we retest the whole site?

Let’s say we design the push to production so the ‘push’ only copies HTML and image files to the webserver. The server is never ‘down’, and serves complete pages. After the switch, the new page appears. Do we really need to give it the full monty, the week-long burn down of all that is good and right in testing? Couldn’t the developer try it on a local machine, push to stag-ing, try again, and “just push it?”

Questions on how?

More to come.

By Matthew Heusser – matt.heusser@gmail.com for Sauce Labs

Stay tuned next week for the third part of this mini series! You can follow Matt on Twitter at @mheusser.

Have an idea for a blog post, webinar, or more? We want to hear from you! Submit topic ideas (or questions!) here.