If you’re a fan of RSpec for writing concise, readable tests in Ruby, you might be interested to know that RSpec tests can now easily automate iOS and Android mobile apps using Appium, an open source mobile test automation tool. Appium is compatible with Selenium WebDriver client libraries, so an Appium test is similar to a Selenium test.
To start, fork and clone Appium from https://github.com/appium/appium, and follow the installation instructions to set up Appium in your environment.
Install Appium’s dependencies and build the sample apps by running the following from the Appium working directory:
$ ./reset.sh --ios
Once the sample projects have have been built, you can then start Appium by running the following:
$ grunt appium
Once Appium is running, get a new terminal and go to the sample-code/examples/rspec directory.
First, install the test dependencies with bundler:
$ bundle install
Then, run the example RSpec tests:
$ rspec simple_test.rb
Now that you’re set up and able to run example tests, it’s time to learn how they work. Let’s look at a simple example of a test that clicks a button and interacts with an alert box in an iOS app. First, we open a “describe” block, and set up a “before” block to initialize the Selenium client we’ll use to talk to Appium:
describe "sample app" do
before :all do
@driver = Selenium::WebDriver.for(
:remote,
:desired_capabilities => {'browserName' => 'iOS',
'platform' => 'Mac',
'version' => '6.0',
'app' => '/path/to/TestApp.app'}
:url => "http://127.0.0.1:4723/wd/hub")
end
The “desired_capabilities” parameter here specifies the platform (iOS 6.0) and the app we want to test. Next, we’ll add an “after” block to quit the Appium session at the end of each test:
after :all do
@driver.quit
end
And finally, we’ll open an “it” block and write the core of our test. This test finds all the ‘button’ elements on the page, selects the second one (index 1), and clicks it. That action will bring up an alert in the app, which we switch to so that we can examine its text field:
it "should handle alerts" do
els = @driver.find_elements(:tag_name, 'button')
els[1].click
a = @driver.switch_to.alert
a.text.should eq("Cool title")
a.accept
end
end
And those are the basics! There are more examples to look at in Appium’s simple_test.rb. Please let us know if you have any questions or comments about running Appium tests with RSpec.
Appium is an open source test automation tool which allows you to easily write functional tests that automate iOS and Android mobile apps. One big advantage Appium has over other mobile test automation tools is that Appium tests can be written in any language that has a Selenium client library, including Python, Ruby, Node.js, and, perhaps most interesting to mobile developers, Objective-C and Java.
To start, fork and clone Appium from https://github.com/appium/appium, and follow the installation instructions to set up Appium in your environment.
Build the sample projects by running the following from the command line:
grunt buildApp:TestApp grunt buildApp:UICatalog
Once the sample projects have have been built, you can then start Appium by running the following:
grunt appium
Change the working directory to the sample-code/examples/java/junit directory, and run the tests by executing:
mvn test
Or run a single test by executing:
mvn -Dtest=com.saucelabs.appium.SimpleTest test
A Java Appium test is much the same as a Selenium test…you create a RemoteWebDriver instance by specifying some DesiredCapabilities, eg.
@Before
public void setUp() throws Exception {
// set up appium against a local application
File appDir = new File(System.getProperty("user.dir"), "../../../apps/TestApp/build/Release-iphonesimulator");
File app = new File(appDir, "TestApp.app");
DesiredCapabilities capabilities = new DesiredCapabilities();
capabilities.setCapability(CapabilityType.BROWSER_NAME, "iOS");
capabilities.setCapability(CapabilityType.VERSION, "6.0");
capabilities.setCapability(CapabilityType.PLATFORM, "Mac");
//tell Appium where the location of the app is
capabilities.setCapability("app", app.getAbsolutePath());
//create a RemoteWebDriver, the default port for Appium is 4723
driver = new RemoteWebDriver(new URL("http://127.0.0.1:4723/wd/hub"), capabilities);
}
Tests can also be written to be run against Sauce Labs. To do this, just update your tests to reference a zip file containing your app, and update your RemoteWebDriver instance to point to ondemand.saucelabs.com, eg:
@Before
public void setUp() throws Exception {
String sauceUserName = "YOUR_SAUCE_USERNAME";
String sauceAccessKey = "YOUR_SAUCE_ACCESS_KEY";
DesiredCapabilities capabilities = new DesiredCapabilities();
capabilities.setCapability(CapabilityType.BROWSER_NAME, "iOS 6.0");
capabilities.setCapability("device", "iPhone Simulator");
capabilities.setCapability(CapabilityType.PLATFORM, "Mac 10.8");
//zip file containing your app to be tested
capabilities.setCapability("app", "http://appium.s3.amazonaws.com/TestApp6.0.app.zip");
driver = new RemoteWebDriver(new URL(MessageFormat.format("http://{0}:{1}@ondemand.saucelabs.com:80/wd/hub", sauceUserName, sauceAccessKey)),
capabilities);
}
The tests themselves are written just like regular Selenium tests, eg.
@Test
public void example() throws Exception {
// find an element by tag name
WebElement button = driver.findElement(By.tagName("button"));
button.click();
// get the value of the element
WebElement texts = driver.findElement(By.tagName("staticText"));
assertEquals(texts.getText(), "some expected value");
}
The Java sample tests for Appium include the following classes:
This is a quick tutorial to get you started running automated tests completely in the cloud as part of your development process using Travis CI and real browsers in Sauce Labs.
As a guinea pig during this blog post, I used a small, open source Python project of my own called cssify. You can find the cssify build results on Travis and even watch running jobs or videos of the browsers in action in the project’s Open Sauce page.
Steps involved:
Set up your tests to run on Travis CI
Configure Travis CI to start Sauce Connect
Configure your Selenium tests to run on Sauce Labs
Provide additional details to Sauce
What is Travis CI?
Travis CI is a hosted continuous integration service for the open source community. It is integrated with GitHub and offers first class support for:
C, C++, Clojure, Erlang, Go, Groovy, Haskell, Java, JavaScript (with Node.js), Perl, PHP, Python, Ruby and Scala.
Their CI environment provides multiple runtimes (e.g. Node.js or PHP versions), data stores and so on. Because of this, hosting your project on Travis means you can effortlessly test web applications against multiple runtimes and data stores without even having all of them installed locally.
Setup Your Tests to Run on Travis CI
There’s multiple tutorials out there to get started using Travis CI. One of my favorite things about Travis is how easy it is to configure and get started.
Check out the official Travis CI docs to get started in your own programming language.
For running your Selenium tests, remember to install the right dependencies, including the Selenium client bindings, like I did in my project. Also, make sure you make Travis start your webapp before the tests start. Your application needs to be serving pages to be able to test it using our browsers :)
Configure Travis CI to Start Sauce Connect
Sauce Connect creates a magical rainbow tunnel that will allow the Sauce browsers driven by your Selenium tests to access the Web Application that your Travis build starts. Once a Sauce Connect tunnel is running, browsers in our cloud can use it to hit localhost, internal IPs or even to just proxy the public internet through the right location.
I’ve created a small Bash script that takes care of downloading, starting and waiting for Connect to be ready. To use Sauce Connect in your build, all you need to do is curl and run this script in your .travis.yml before_script section as I did in my project:
Configure Your Selenium Tests to Run on Sauce Labs
Once your builds are happily running and Sauce Connect is started automatically by Travis, the next step is having your Selenium scripts run in the Sauce Labs cloud and hit the Application Under Test (AUT) through the tunnel.
To start running real browsers in the Sauce Cloud, make your scripts use Remote WebDriver and point the command executor to localhost:4445 (Sauce Connect will listen in that port and securely and efficiently relay to Sauce). You can find an example of this on the cssify codebase.
Now for these browsers to be able to hit the AUT when opening localhost, you need to specify an additional Desired Capability, tunnel-identifier.
The value this capability should be set to is stored in the environment variable TRAVIS_JOB_NUMBER. You can find an example in the cssify codebase for this as well.
In addition to running your tests in real browsers using the cloud, there’s additional details you can provide to speed up debugging and navigating test results.
In the cssify tests you can find some examples Desired Capabilities that Sauce uses and that you can pull from Travis CI’s environment variables.
You’re set! You are now running your tests using Travis CI, they start a Sauce tunnel on their own and drive real browsers in the cloud. Time to start thinking about running these in parallel to speed up your builds now!
Today I’m going to show you how you can use Selenium Builder with GitHub, Travis, and Sauce Labs to make a continuous integration solution for your website that’s all on the Internet, nothing stored locally. You can either follow along with the demo video below or read on for the written tutorial.
The great thing about this setup is that if you put your site code and your tests into the same GitHub repository, then whenever you update the site code, and whenever you update the tests, Travis will rerun your Selenium tests on Sauce OnDemand – and then send you an email about whether or not they still work!
So you can get a full Selenium CI solution by putting together these components. And because they’re all components, if you want to use something else in part – for example Jenkins instead of Travis – you can construct your own solution.
For now I’m going to show you this set of four things that work really well together. The great thing about this is that you store everything on GitHub, so all your tests are stored in the cloud, nothing is stored on some local machine somewhere. You can record and edit tests from any computer that runs Firefox. You can access your tests, edit them, & play them back locally. And all of the actual execution of the testing – the CI – is handled by Travis and Sauce. So you actually don’t need to build any of your own testing infrastructure!
OK! So how do you actually do this?
The first thing you need to do is install the plugins. You go into Selenium Builder, you go into Plugins, and you just click install on GitHub integration and Sauce. Restart Builder.
Next up, you want to record some tests. Let’s just say for now we’re testing this website.
Let’s record a verification.
Now we’re going to save this test. In the File menu, save the test to GitHub.
Because this is the first time we’re using it, I’m going to have to enter my details.
Now, as you can see, we can access our repositories. And I’ve set up this little repo here as an example one.
So let’s save this test in here, as JSON, which is the standard format Builder works in. This means we can later open it again and edit it.
Let’s do another test.
Now we save that one as well. Just record as many tests as you want and put them all into this place.
Now we’ve done that, the next thing we need to do is configure the Selenium interpreter, which is actually a little node.js program that takes these JSON scripts and uses Webdriver to play them back.
Let’s clone the repo to disk so we can add the config files we need.
Create a new text file. I’ve just pasted in a bunch of things, which are actually reasonably straightforward.
The config is a JSON file that contains two things: the settings that you want to run the tests (in this case, we want to run them on Firefox) and if you want to run them on Sauce Labs’ infrastructure. You have to tell Sauce your username and access key before this can happen, but it’s not a good idea to commit your Sauce access key to a GitHub repository.
So instead, we’re going to put the username and access key into environment variables. As long as we run the interpreter with the SAUCE_USERNAME and SAUCE_ACCESS_KEY environment variables set to the right values, it will work fine. And how we set those variables, we’ll get to in a moment, as that will be in the Travis configuration.
Here we have the tests we actually want to run. This config format supports glob syntax, so you don’t have to list each test individually. Just put in the path to the directory with an asterisk at the end, and it will find the tests. And it will ignore anything that’s not a JSON file.
That’s the entirety of the config, so let’s save that in our repo.
And commit and push it.
Now the next thing we need to do is to tell Travis that we exist, and that we would like to use its services.
You can log yourself in…
…using your GitHub account.
And then you go in here…
…and you enable Travis integration for this particular repo.
OK. Now the final step is to configure Travis. We need to tell Travis what running an integration test for this particular project actually means. That’s done by putting a file into the top level of the repository, which tells Travis what to do. It’s called .travis.yml.
So again, I’ve just pasted in something.
Now we tell it to install the Selenium interpreter, and then run it with the configuration we already made. And the one thing we’re lacking here is the environment variable for the Sauce access key. Setting variables in Travis is easy if you don’t mind them being public, which you probably don’t mind for your Sauce username.
Now this is the one tricky bit in this tutorial. You need to encrypt your access key to let Travis know what it is without revealing it to the world. The way to do that is to first install a Ruby gem.
Once that’s installed, make sure you’re in the repository you want to encrypt something for and log in to Travis.
Then you can use this command to encrypt an environment variable.
What comes out is a blob of encrypted data…
…which you can paste into the .travis.yml config file like this. The encrypted data is tied to your git credentials and this particular repo, so it should be safe from snooping or theft. Your access key should also be safe. If it does ever become compromised, you can also generate a new one. That’s what I’ll be doing in a few minutes.
Now you can save this file…
…commit it, and push it. And of course this means Travis now goes “Oh, this has been pushed to! OK, is there a .travis.yml? Yeah, there is.”
Let’s actually go onto Travis. We can see it’s in the queue right now.
Ok, good, it’s running the tests.
You can see the details of each test being run.
But you don’t need to know any of the details, because in the end, we get an email saying, “Hey! It’s succeeded, everything’s fine!”
So I’ll just show you what happens if you break things. You can edit these JSON scripts with a text editor, which is pretty easy.
Let’s go in here and change this verify to something that’s not actually the case.
Save, commit, and push, and Travis runs again.
And now we get told that the build is broken! One of our tests failed!
So let’s fix the test up again.
Save, commit, push.
And shortly after, Travis tells us that we’ve fixed the build. Excellent.
That’s all really – you can use these four components to make a continuous integration system for your site. If you have any questions, drop Adam or me an email.
You’re probably wasting two things when you’re testing. Your time… because the other thing is your extra CPUs. Any time you’re waiting on a relatively slow resource, your CPU is just sitting there, twiddling its silicon thumbs. If you’re only using one CPU core at a time, the other cores are doing much the same. Unfortunately for web developers who do things involving CRUD operations, slow resources includes databases. Most unfortunately for you, Dear Reader (and us, Dear… Us), they also include the browsers you’re integrating with for Selenium testing.
This is what waiting for browsers and io should be measured with
One of the best ways to get more test for your tokens is to run more tests at once. If you’ve got several tests going, even if some are waiting for one slow resource, the others can use the CPU. The more tests you can run at once, the shorter your test cycles will be, especially if you could, say, spin up more then one copy of the slow resource to test with (Psst: I’m talking about Sauce Labs Parallelization).
I’ve been looking for ways to make parallelization easier when testing with Ruby, and I stumbled across the Parallel Tests gem. It’s actively developed, has some nice documentation and integrates with rspec, rspec-rails and test:unit. Their benchmarks show that using the gem, the Rails ActionPack test suite time was cut in half, from 88 seconds to 44, with just 4 test runners. This, conveniently, is the number of parallel tests you can run with a Mild plan.
Checkmate. Wait. Snap. Game Set Match?
So for many of you, there’s already a possibility of making your tests take half the time. Which means you can run twice as many. Which means a much faster turnaround time for TDD, BDD, and release testing.
This is a guest blog post by Sarah Goff-Dupont, Product Marketing Manager for Atlassian Bamboo.
Fast feedback on code changes has been a bit of an obsession ever since the Agile2012 conference got me thinking about how important this is for agile teams in particular–partly because turbo-charged feedback is a natural work-in-progress limiter. How so? Getting results back to devs so quickly that they don’t have time to start in on another issue keeps everyone focused on the task at hand. That, in turn, fosters an efficient development process. And efficiency, in my never-humble opinion, is what agile development is all about.
(Caption: Faster feedback could save this poor imaginary developer from juggling 3+ coding tasks!)
Running automated tests in parallel is a great way to reduce build times and get results to developers and QA engineers faster. Today I’m going to dig back into my long-lost career as a test automation engineer and share with you the technique we used for running tests in parallel. ‘Cuz it was super-slick and I loved it. I’ll use Atlassian Bamboo (which integrates nicely with Sauce Labs, btw) as my example CI tool as we walk through it.
TestNG Groups FTW!
If you’ve never used TestNG before, you might think that’s a typo above and that I meant “TestING”. (Google’s autocomplete certainly thinks it’s a typo, so hey: you’re in good company.) But no. TestNG is an automated testing utility, much like JUnit. In fact, you can use it in combination with JUnit, WebDriver, Selenium, etc. And the reason you’d want to is Groups. By sorting your test suite into groups, then creating separate Jobs in Bamboo to run each group, you can speed up your testing stages by 100% –and likely, better.
TestNG is designed for Java tests, but can also be used for Ruby projects. And it’s not the only framework to offer groups. Proboscis for Python and PHPUnit also have them. JUnit has “categories”, which largely serve the same purpose, but are rather clunky and make TestNG’s groups look like a bacon-wrapped Ferrari by comparison (sleek, fast, and yummy).
Implementing Groups in Your Test Code
Like a handful of other testing frameworks, TestNG takes it’s cues from an @Test annotation. Inside that, you can place a number of attributes, one of which is Groups. Group names are both dynamic and arbitrary –small sentence, big implications. First, you can create a new group on the fly just by assigning a test to it. No need to declare and define the group in a separate file. The act of assigning a test to a group will also create that group, if it doesn’t already exist. (Now, that’s a double-edged sword because it also means you can end up with a group called “smoke_test” and another called “smoke_tets” if you’re careless.)
Second, you can devise whatever system (or systems) of meaning you like for your groups. Slice them along functional area of the application, by level of the code stack, by whether you currently expect the test to pass, etc. And tests can belong to multiple groups. So a single test might conceivably belong to the “user_authentication”, “API”, “smoke_test” and “in_development” groups. TestNG even supports groups of groups. I haven’t played with that personally, but it’s pretty bad-ass and their documentation on it is helpful.
(Caption: Quick pro-tip: TestNG doesn’t like it when you use a hyphen in group names. Use the underscore instead.)
Of course, if your application’s super-structure doesn’t know about TestNG, you’ll get all sorts of errors and your builder won’t know what do with all those groups at runtime. So be sure to declare TestNG as a dependency for your project or module.
Using Groups in Bamboo Builds
Back in Bamboo, set up Jobs based on the groups you’ve defined in your test code. If you’re executing tests via Maven, Gradle or Ant, this is pretty trivial. In the example shown here, I pass a group name into Maven as an argument. When it’s time to run the tests, TestNG will scour my project, pull out all the tests belonging to that group, and run only those. Doesn’t matter if the tests for that group are located in different classes or packages. As long as they’re in the same project or module, TestNG will find them.
Note that because each Job has it’s own working build directory, you’ll need to pull the test code into each Job. A source code checkout Task is generally the simplest way to do that. If your code base is large, you can take advantage of the option to check out from a sub-directory in your repo and only pull down that part of the tree. Another option is pulling down all the test code in the very first Stage of your Plan, then zipping it up and passing that as an artifact to downstream Jobs.
Agents of Change
It’s probably obvious, but splitting your test suite into 10 batches isn’t going to help you much if you only have 1 or 2 build agents. Of course we’d love if everyone bought a Bamboo license that supports lots of agents so they can really take advantage of parallelization. Maybe that’s feasible for you, and maybe not. Either way, be mindful of your agent count when considering how to split up your test suite so you’re not just spinning your wheels.
You can take this idea of parallelized tests about as far as you want. There’s no restriction as far as TestNG or Bamboo is concerned. Many of our own CI builds have over 20 test groups running simultaneously. Again: you’re limited only by the number of build agents you’re running.
(Caption: How do you run over 17,000 functional tests in less than an hour? Parallelization, baby!)
A Parallel Universe
As with most things in build engineering, there are several ways to accomplish parallelized testing. Tinkering with custom scripts or your suite.xml are two other common approaches. For big multi-module projects, it might make sense to split the suites/groups into separate repositories and run a Job for each repo. Regardless of which method you chose, you’ll get the most bang for your buck by focusing your parallelization efforts on integration-, API- and UI-level tests. Unit-level tests already run quite fast, so they’re unlikely to be the bottleneck in your builds. (Extra credit for the over-achievers: many testing frameworks allow you to run unit tests multi-threaded!)
If you’re new to test parallelization, congratulations on making it all the way through this blog! Check out our new automated testing resources page for more helpful tips & tricks.
—– Sarah Goff-Dupont is a former test automation engineer, and a fan of anything that makes life easier for the nerds. Like Continuous Integration and automation, for example. Because manual testing gets boring. Her heros include Margret Thatcher, MacGyver, and her mom.
We recently conducted a webinar that illustrated how you can get up and running with CloudBees and Sauce Labs with one click, using the Sauce Clickstarts for CloudBees.
There were a number of questions raised in the webinar that we didn’t get time to address, so we’ve included answers below. If you have any further questions, please include a comment below.
Q: If you define that you want to deploy to RUN@CloudBees in pre-build step , is it guaranteed that the Maven job will be run only and only if deployment is finished?
A: Yes, that’s correct.
Q: How do you satisfy that application is deployed before Sauce tests are run?
A: The CloudBees Deployer plugin will return the status of the deployment: the next test step won’t run until the deployment is successful.
Q: In the JBoss example, tests are run against an in-memory JBoss. Is it possible to run the tests against the RUN@CloudBees instance?
A: Yes, both configs are possible. In order to run against a different JBoss instance, you can specify your Jenkins build to use a different profile (which corresponds to a separate Arquillian container).
Q: Where is the sauce-jboss-clickstart and sauce-java-clickstart source code located?
A: https://github.com/CloudBees-community/sauce-jboss-clickstart and https://github.com/CloudBees-community/sauce-java-clickstart. Please feel free to fork!
Q: How are code changes moved to the cloud after changes are made locally?
A: What happens is you create a “cloud” repo with GitHub or CloudBees and then clone it for local development – then push changes back up to the cloud and the Jenkins jobs run automatically.
Q: So SSH is used to move code to the cloud after the build?
A: SSH is the default but there are other options
Q: Can we have two ClickStart applications active at the same time?
A: Yes, you can have multiple applications running at the same time.
Q: So the browser set in Jenkins config creates an environment variable that will override the default Sauce Labs SELENIUM_BROWSER?
A: Yes, the Sauce Jenkins plugin will set a series of environment variables that represent the selected browser(s). Further information on the environment variables set by the plugin is available from the Jenkins tutorial page.
Q: What are the browser requirements for the Sauce browser test screen to capture video?
A: You don’t need to have a certain browser for that. Your test will automatically be recorded and screenshots captured every time you run a test on Sauce.
Q: “Drone extension for arquillian” huh, what? :-)
A: I know, right? Take a look at our recent blog post for more information.http://sauceio.com/index.php/2012/12/sauce-extension-for-arquillian-drone/ :-)
Q: I read that it is possible to test applications deployed behind firewalls, can you explain how this works?
A: Yes, you can use Sauce Connect to test websites running on a local webserver. The Sauce Jenkins plugin can be configured to launch Sauce Connect prior to the running of your testss.
Q: Is it possible to select more than one browser/OS combination in the same job?
A: Yes, it is possible to set more than one browser – in this case, the Sauce plugin will set a SAUCE_ONDEMAND_BROWSERS envionment variable that contains the browser information (in JSON format).
Q: Do users of Sauce need to log in via the CloudBees environment first? Do users have to have a CloudBees account in order to use the Sauce service?
A: No, if you just wanted to use Sauce, you could sign up via http://saucelabs.com/signup. If you want to use it with Cloudbees, however, we recommend signing up through Cloudbees.
Q: What needs to be configured on the Jenkins build job to allow the Sauce test run links to appear in the Jenkins build job?
A: You will need to output the information to the stdout in the following format:
where ‘session id’ is the Selenium session id and ‘some job name’ is an identifier for the test under execution (usually the test name).
When the job has completed, the Sauce Jenkins plugin will parse the build output, and if it finds the above output, will update the Sauce Job to store the Jenkins build number.
Q: Is the Sauce plugin built to support Node.js/Ruby/.NET/PHP apps?
A: Yes! The Sauce plugin for Jenkins supports all languages. In the coming weeks, we will add more Clickstart examples that demonstrate some of the other languages and platforms supported by CloudBees.
Today we are excited to announce a new tool that’s been on our minds since we released our manual testing service last year. Sauce for Mac, our first native desktop app, lets you launch any of our 90+ browser/OS combos right from your Mac in a matter of seconds. (Don’t believe us? Download it now to see for yourself).
With Sauce for Mac, you can Command-Tab your way to testing bliss as you check how your app looks and behaves on browsers and simulators such as the iPhone 5, Internet Explorer 7, Firefox 14, and Chrome – all at the same time. There’s no infrastructure to setup or install, so you can say goodbye to virtual machine or any other expensive setup you’ve cobbled together for testing browsers and OSs not installed on your own laptap. Our Mild Plan comes with unlimited manual testing with Sauce for Mac (as well as from our website), and is only $12/month. And of course, Sauce for Mac comes equipped with all our standard features, including screenshots, bug reporting, videos of every test session, and standard debugging tools.
To start a session, type in your URL and pick the browser and OS you want to navigate to. If it’s your first time using Sauce, you can test on IE6 without an account. To access the full browser list, sign up for a free account, which comes with 30 test minutes per month on up to 2 browser tabs. (Paid accounts come with unlimited minutes and up to 5 browser tabs).
And that’s the extent of the setup process! You’re now plugged into the Sauce cloud, which instantly delivers pristine VMs each and every time you fire one up (we take security very seriously ‘round here). Now you’re able to:
Check for design or CSS issues against 90+ Browser/OS’s
Capture snapshots of issues with the click of a button
Test against local or firewalled sites
Debug using the standard browser developer tools
We hope you’ll give Sauce for Mac a try and let us know how it goes. And if you’re a non-Mac user who find this interesting, shoot us an email. We’d be glad to build this for PC folks too if there’s demand.
As part of the upcoming Sauce CloudBees webinar, we are going to demonstrate running a Clickstart, which will populate a CloudBees environment with a running Java EE application that is pre-configured to run tests using Sauce.
The sample project uses Arquillian, which greatly simplifies the integration testing of Java EE applications.
We’ve created a Sauce extension for Arquillian Drone, an add-on for Arquillian that makes running Selenium/WebDriver tests easy.
To include the Sauce extension within your project, include the following dependency in your Maven pom.xml file:
If you are running builds using the Sauce CI plugins, then the extension will read the appropriate environment variables set by the plugin, so you can just add this to your arquillian.xml file:
<extension qualifier="sauce-webdriver"/>
The Drone logic will handle the lifecycle of your WebDriver/DefaultSelenium instance, so your test logic can be as follows:
@RunWith(Arquillian.class)
public class AmazonTest {
@Drone
WebDriver driver;
@Test
public void amazonTitle() {
driver.get("http://www.amazon.com/");
assertEquals("Amazon.com: Online Shopping for Electronics, Apparel, Computers, Books, DVDs & more",
driver.getTitle());
}
}
That’s it! We end up with less code, and a centralised location for the configuration of the Java EE container and the WebDriver/SeleniumRC settings.
Please let us know if you have any feedback on the Sauce Drone extension,and make sure you tune into our upcoming CloudBees webinar!
This post was originally written by Alan Parkinson of Hindsight Software. One of Sauce’s authorized partners, Hindsight Software specializes in building tools and services around Selenium and test automation. It has been reprinted with his permission.
Automated functional tests provide valuable feedback to developers by notifying them when they have completed or broken functionality. The value of these tests can be maximised by providing the test results in the shortest possible time. The reason being the problem is likely to be fresh in the developer’s mind and quicker for them to fix.
A typical functional or UI test suite can take many hours to run because the tests can only be run sequentially. The main cause for the sequential tests is the dependence on data. Common practice has test cases clear the database of the application under test and populate it with the required data before it starts. This high level of database manipulation does not allow the tests to be run in parallel because each test will interfere with the database at different times, therefore corrupting data required used by other tests.
The common solution for long running test suites is to break them into batches and run each batch on a separate machine with its own instance of the application under test and its database. This is easy to achieve using spare hardware, Virtual Machine’s or Cloud computing resources. You don’t even have to write code for spinning up new machines, or access remote machines, as Continuous Integration (CI) servers like Jenkins and Bamboo provide functionality for running builds on multiple computers. This feature is better known as Slaves or Agents.
The majority of the popular testing tools have ways of grouping tests together and we can use this feature to create our “Batches” of tests. These features are known as “categories” in JUnit, “groups” in TestNG and “tags” in Cucumber, Lettuce and Behat.
This is a quick tutorial to get you started running automated tests completely in the cloud as part of your development process using 
Once your builds are happily running and Sauce Connect is started automatically by Travis, the next step is having your Selenium scripts run in the Sauce Labs cloud and hit the Application Under Test (AUT) through the tunnel.
