Mobile Test Automation + Sauce Labs + Behat
Sauce Labs announced support for iOS and Andriod devices in August, allowing the Behat community to enjoy the benefits of Apple and Android Sauce. Mobile automation testing is more challenging than web applications and desktop applications testing because mobile apps are new in the market and don’t have proper industry standards yet. Mobile applications tend to lack proper automation tools and infrastructures for a variety of the mobile devices out there.
One of the major issue in mobile automation is setting up an environment for mobile devices, which usually requires a high level of QA and infrastructure skills. Sauce Labs solved most of the these problems by providing cloud support for mobile automated testing, so this post will explain some great new features of Behat 2.4 (part 1 of this blog series) and its integration with Apple Sauce and Android Sauce (part 2).
Let’s start with the Behat installtion.
Remove previous Behat installation [Pear]
In the last post about Behat, we installed Behat with ‘pear’ packages. The Behat version installed with pear packages is no longer supported so be sure to read the Behat installation guide for further information. Now, let’s uninstall the old version of Behat and install a new one. Open your terminal and type in the following commands.
Check your existing installation with this:
$ which behat
If you installed Behat with the pear package, you will see something like this:
$ which behat
/usr/local/pear/bin/behat
Remove this Behat vesion by running:
$ sudo pear uninstall behat/behat
uninstall ok: channel://pear.behat.org/behat-2.3.5
$ sudo pear uninstall behat/gherkin
uninstall ok: channel://pear.behat.org/gherkin-2.1.1
Remove ‘Mink‘ too, as we are going to use ‘MinkExtension’:
$ sudo pear uninstall behat/mink
uninstall ok: channel://pear.behat.org/mink-1.3.3
Behat 2.4 installation [Composer]
Now that we have removed the old version of Behat, we won’t find anything in binary when you execute the ‘behat’ command:
$ behat
-bash: /usr/local/pear/bin/behat: No such file or directory
Now create a new Behat installation directory. Here we have ‘/opt/behat’ directory with all the permissions.
$ sudo mkdir /opt/behat
$ cd /opt/behat/
$ sudo chmod -R 777 /opt/behat/
Composer.json
Create the ‘composer.json’ file inside ‘/opt/behat’ and add all the third-party libraries, packages, dependencies into it like this:
$ sudo vi composer.json
Add the following dependencies in the json file (you can add more as per your project requirements). You can find more packages on the Packagist website:
{
"require": {
"behat/behat": "2.4.*@stable",
"behat/mink": "1.4@stable",
"behat/mink-goutte-driver": "*",
"behat/mink-selenium-driver": "*",
"behat/mink-selenium2-driver": "*",
"behat/mink-extension": "*",
"sauce/sausage": ">=0.5",
"sauce/connect": ">=3.0"
}, "minimum-stability": "dev", "config": { "bin-dir": "bin/" } }
Note that we have included packages for Behat, Mink, and Mink drivers such as Selenium & WebDriver. We have also included the MinkExtension and Sauce Connect packages in order to use them in the projects. MinkExtension creates Mink instances in each sub-context or it could be used as a subcontext on its own. You can read more about it on the MinkExtension documentation page. We will demonstrate how to use MinkExtension and Sauce Connect later in this post.
The next step is to download composer.phar file consisting of all the above packages. Let’s do that with the following commands:
$ curl http://getcomposer.org/installer | php
$ curl http://getcomposer.org/installer | php
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 11038 0 11038 0 0 88692 0 --:--:-- --:--:-- --:--:-- 307k
#!/usr/bin/env php
All settings correct for using Composer
Downloading...
Composer successfully installed to: /opt/behat/composer.phar
Use it: php composer.phar
This will download ‘composer.phar’ file, but let’s confirm with this:
$ ls
composer.json composer.phar
Now install it using the following command:
$ php composer.phar install
This will download all the dependencies. You can see it in the ‘Behat’ directory. Once it has finished downloading, your directory should look like this:
$ ls
bin composer.json composer.lock composer.phar vendor
It will also create a composer.lock file that will contain the list of installed packages and their associated versions and vendor/autoload.php file that can be used to autoload all the dependencies in your project. You should now be able to use any class from your installed packages. Now, you are ready to run Behat using ‘./bin/behat’ command.
$ ./bin/behat
Don’t forget to make ‘./bin’ directory executable so that we can execute all the binary from outside:
$ cd /opt/behat
$ sudo chmod -R 777 bin/
Access Behat Locally
You can now access Behat locally from where you downloaded composer. OR you can make it global.
$ ./bin/behat
Check the Behat version:
$ ./bin/behat --version
Behat version DEV
Make Behat Global
Now, let’s create a global symlink to access Behat from anywhere.
$ sudo ln -s /opt/behat/bin/behat /usr/local/bin/behat
Make Sauce Connect Global
You can also run Behat and sauce_config, sauce_connect from anywhere. It completes the Behat2.4 installation process.
$ sudo ln -s /opt/behat/bin/sauce_config /usr/local/bin/sauce_config
$ sudo ln -s /opt/behat/bin/sauce_connect /usr/local/bin/sauce_connect
Note: You may need to restart your terminal session to get the newest version.
$ which behat
/usr/local/bin/behat
If you had any trouble with the previous commands, watch the video demo of the Behat installation with Composer on Youtube.
Creating Our Behat Project
Now, let’s create our first Behat project by running the following commands.
$ sudo mkdir SaucyBehat
$ sudo chmod -R 777 SaucyBehat
$ cd SaucyBehat
$behat --init
+d features - place your *.feature files here
+d features/bootstrap - place bootstrap scripts and static files here
+f features/bootstrap/FeatureContext.php - place your feature related code here
Behat has already created a ‘features’ directory to write feature files. We need to implement the step definitions in auto-generated ‘feature/bootstrap/FeatureContext.php’ file. Let’s first create a feature file to check Sauce Labs features:
$ sudo vi Sauce.feature
Add the following feature:
Feature: SauceLabs Documentation
In order to learn SauceLabs features
As a Software Tester
I need to see resources
@javascript
Scenario: Access feaures
Given I am on "/"
When I follow "Features"
And I saw page loaded
Then I should be on "/features"
And I should see "Check out our features."
We also need to create a configuration file called ‘behat.yml’ in the project root directory. It should use MinkExtension. Let’s create ‘default’ profile to run tests locally:
default:
context:
class: 'FeatureContext'
extensions:
Behat\MinkExtension\Extension:
base_url: 'http://saucelabs.com'
javascript_session: 'selenium2'
goutte: ~
selenium2:
We have now created a ‘default’ profile, which uses the Context ‘FeatureContext.php’ and the Extension called ‘MinkExtension. Mink Extension provides different drivers such as Selenium, Selenium2 [WebDriver] etc. We have used ‘Selenium2′ driver to run tests. Read more about MinkExtention here or watch it on GitHub.
Now, we need to update ‘feature/bootstrap/FeatureContext.php’ to use ‘MinkExtension’. In order to do this, we need to extend the file to ‘Behat\MinkExtension\Context\MinkContext’ instead of ‘BehatContext’. First we need to delete all the code from unnessesary code from FeatureContext File. Now, it should look like this:
/**
* Features context.
*/
class FeatureContext extends Behat\MinkExtension\Context\MinkContext
{
}
We are set to run the Behat feature locally. Now we need to download the Selenium Server and run it in the background.
$ /path/to/seleniumjar/ java -jar selenium-server-standalone-2.25.0.jar
Let’s run ‘Behat’ from the root of the project. (Remember you always have to run ‘behat’ from root project directory unless you provided PATH in ‘behat.yml).
$ behat
You can see test running in the browser. It will complain about the ‘UnDefined’ step, which Mink didn’t understand. We need to copy that code snippet and implement it in the FeatureContext.php file using clean MinkApi’s. Here is the implemented step to get the feature working. Now, FeatureContext.php file will look like this:
/**
* Features context.
*/
class FeatureContext extends Behat\MinkExtension\Context\MinkContext
{
/** * @Given /^I saw page loaded$/ */
public function iSawPageLoaded()
{
$this->getSession()->wait("5000", "document.getElementById('pricing-table')");
}
}
Now if you run the Behat command again, you will see that the tests passed like this:
Feature: SauceLabs Documentation
In order to learn SauceLabs features
As a Software Tester
I need see resource
@javascript
Scenario: Access feaures # features/Sauce.feature:7
Given I am on "/" # FeatureContext::visit()
When I follow "Features" # FeatureContext::clickLink()
And I saw page loaded # FeatureContext::iSawPageLoaded()
Then I should be on "/features"# FeatureContext::assertPageAddress()
And I should see "Check out our features."# FeatureContext::assertPageContainsText()
1 scenarios (1 passed)
5 steps (5 passed)
0m35.164s
And that’s it! In part 1 of this tutorial, we covered, Behat, Sauce Connect, Mink, installing Selenium with Composer and running Behat tests in local browsers. In part 2, we will learn how to run Behat tests on Mobile devices using Sauce Labs.
Even so, automating browser tests seemed like magic. It was mesmerizing to watch the tests running. A never-tiring invisible hand filling in form fields and clicking buttons. (Ok, maybe I’m just easily mesmerized).
The claim of Tddium to completely lift the testing burden – unit and Selenium – into the cloud, and run both in parallel blew me away. To the extent that I was dubious as to how well it would work. But the answer? Very well indeed! Now add in Tddium’s Github integration and Continuous Integration support and…. wow… just….wow.
