Posts Tagged ‘Continuous Integration’

Guest Post: All About Mobile Continuous Integration

April 20th, 2015 by Bill McGee
mobile_devices_continuous_integration

Image credit: Loris Grillet

CI’s not a new thing. Wikipedia says the phrase was first used back in 1994, way before modern mobile apps. Today it’s commonplace in many dev shops for developers to expect that their code is automatically tested when they commit and even automatically deployed to a staging environment.

For mobile developers though, it’s been a slow road to adopting many of these same practices.  In large part, this is because mobile brings with it a whole set of unique challenges that make implementation tough.  Nevertheless, tools have evolved a lot and mobile dev teams get a lot of value and goodness from having a solid CI system in place. Here are my top 3 reasons for using CI with the mobile products I work on: (more…)

Speed up Selenium with Application Mocks

April 8th, 2015 by Chris Riley

Let’s face it, we all know Selenium is slow. Not the Selenium scripts themselves, but the process of interacting with the operating system and browser drivers, in addition to your application’s own responsiveness. Luckily for us, there’s a great way to speed up functional testing in Selenium by combining a comprehensive parallel testing grid with application mocks. (more…)

Repost: Testing in a Real Browser with Sauce Labs + Travis CI

March 26th, 2015 by Bill McGee

This post comes from our friend Sam Saccone, who wrote a nice how-to on using Sauce Connect and Travis CI. Check out the original post on his blog.

I recently found myself implementing a basic set of JavaScript integration tests on an open source project. I quickly came to the realization that there is a serious lack of good documentation on how to get a basic browser test running on a Travis CI using Sauce Labs and Sauce Connect.

After stumbling across the barren desert of outdated doc pages, incorrect stackoverflow answers, and ancient google group posting, I present to you the spoils of my quest to the underbelly of the web.

Let’s approach this in the context of a real problem: Testing javascript in a real web browser. (more…)

DZone: Guide To Continuous Delivery [DOWNLOAD]

February 4th, 2015 by Bill McGee

DZone 2015 Guide to Continuous Delivery

We are very excited to be a research partner for DZone’s Guide to Continuous Delivery, a premium resource focused on continuous integration and DevOps management trends, strategies, and tools.

Readers of the guide will get an overview of continuous delivery practices and how continuous delivery affects many aspects of an organization. This guide includes:

  • Articles written by continuous integration and deployment experts – including Sauce labs
  • Detailed profiles of 35+ continuous delivery tools and solutions
  • “Continuous Delivery Maturity Checklist” that gauges where your continuous delivery skills rank
  • “Continuous Delivery: Visualized” infographic that details the tools developers use at every stage of the pipeline

DZone’s continuous delivery guide offers key insights into continuous integration and through a survey of 750+ developers and experts, allowing readers to learn trends from practitioners in the technology professional community. Additionally, the guide’s solutions directory compares various tools for continuous integration, application release automation, and configuration management to help readers wisely choose the solutions they need.

Download a free copy of the guide HERE.

About DZone

DZone provides expert research and learning communities for developers, tech professionals, and smart people everywhere.  DZone has been a trusted, global source of content for over 15 years.

Re-Blog: CI & CD With Docker, Beanstalk, CircleCI, Slack, & Gantree

December 10th, 2014 by Bill McGee

Bleacher-report-logo

This is a follow-up post to a series highlighting Bleacher Report’s continuous integration and delivery methodology by Felix Rodriguez. To find out how they were previously handling their stack, visit the first, second, and third posts from June 2014. 

There is definitely a huge Docker movement going on in the dev world right now and not many QA Engineers have gotten their hands dirty with the technology yet. What makes Docker so awesome is the ability to ship a container and almost guarantee its functionality. (more…)

HomeAway + Sauce Labs: Continuous Delivery Made Awesome [VIDEO]

October 7th, 2014 by Bill McGee

A few weeks back, we sat down with Carl Shaulis at HomeAway to learn how they leverage Sauce Labs‘ cloud in their testing process.

HomeAway, Inc. is the world’s leading online marketplace for the vacation rental industry. With over one million live vacation rental listings in 190 countries paired with a family of 50 websites and hundreds of applications thanks to a series of acquisitions, quality is their top priority. To get ahead of the curve, Carl’s team is trying to automate and use best practices like continuous integration as much as possible. We were also thrilled to hear that they’re using Appium, a cross-platform test automation framework powered by Sauce Labs, for their mobile testing.  Carl also says Sauce fits perfectly into their testing process.

Watch the video below to learn more about their path towards continuous delivery, and how Sauce fits into that story.

(more…)

Announcing CircleCI Integration on Sauce Labs

August 11th, 2014 by Bill McGee

Last week our friends at CircleCI showed us how to securely run cross-browser front-end tests on the Sauce Labs cloud using their hosted Continuous Integration service. We’ve long been advocates of good continuous integration practices and have developed a few plugins for some of the more common CI servers out there. We’re super excited to add CircleCI to our list and even more excited about how easy it is to get it going!

Continuous Integration in the Cloud

Continuous Integration, if you don’t already know, is the process of building your app frequently from a shared source code repository and running tests to make sure everything works. If your tests don’t pass and the build is not successful, the code that was checked in since the last good build is where the defects were introduced, and so problems are much easier to find and fix quickly.

Maintaining a local CI server can be a hassle. Anyone who’s spent any considerable time configuring Jenkins jobs with all it’s various plugins and tasks can tell you all about it. CircleCI, on the other hand, integrates directly with GitHub and can actually *infer* necessary settings directly from your code (if you’re following good development practices for that language and environment) and so many projects just magically build themselves on CircleCI without any additional configuration on your part. It’s like three clicks from zero to CI. Pretty sweet! If you do need to tweak or customize any settings, you can easily do so by describing those settings in a circle.yml file placed in your repo.

Running Tests on Sauce Labs Browsers

Sauce Labs is the world’s largest cross-browser client cloud. We host over 375 different configurations of operating systems and browsers so you can ensure that your app works on all the specific platforms and versions you need to support. These days that’s an ever-growing list! So it makes sense to run these tests with your continuous integration process so you know things work across the board and you don’t end up spending a bunch of time and trouble trying to hunt down bugs that were introduced much earlier in the development cycle.

Now, if your build deploys your code to a publicly accessible environment, CircleCI will simply execute your Selenium tests and you probably won’t need to configure anything, since Sauce Labs browsers will be able connect to that environment over the public network. However, if you want CircleCI to execute your tests locally in it’s build containers, you’ll need to use Sauce Connect.

Sauce Connect is a secure tunneling utility which allows you to execute tests behind firewalls via an encrypted connection between Sauce Labs and your testing environment. When you run Sauce Connect yourself, you typically do it from a command line or shell script and supply your Sauce Labs account credentials as parameters. With a CircleCI build, you’ll need to set it up in the circle.yml file so it can be launched as part of the build process and those tests can run inside the local container.

All that’s really involved here is telling the build task where to find Sauce Connect and how to start it up with your account credentials.

The first part is downloading and unpacking the Sauce Connect file, which you add as a custom dependency entry in your circle.yml:

	dependencies:
	  post:
		   - wget https://saucelabs.com/downloads/sc-latest-linux.tar.gz
		   - tar -xzf sc-latest-linux.tar.gz

The second part is to add your credentials, launch the tunnel, and check that it’s running before kicking off the tests. You’ll put these lines in the `test` section of circle.yml:

test:
     override:
        - ./bin/sc -u $SAUCE_USERNAME -k $SAUCE_ACCESS_KEY -f ~/sc_ready:
            background: true
            pwd: sc-*-linux
        # Wait for tunnel to be ready
        - while [ ! -e ~/sc_ready ]; do sleep 1; done

That’s all there is to it. You can find out the details here and see a full example on GitHub. And CircleCI has a nice little utility to help you add your credentials as environment variables so that they are not visible as plain text in the repo.

With CircleCI tackling all the necessary work involved in supporting your continuous integration process and Sauce Labs hosting the world’s most extensive cross-browser client cloud, you can be free of the costs and hassles of managing all these systems and grids and get back to focusing on the business of making great software.

Michael Sage, Principal Technology Evanglist, Sauce Labs

Michael Sage is a Principal Technology Evangelist at Sauce Labs. He helps software teams develop, deliver, and care for great apps. He’s lived through two or three technology revolutions and expects a few more to come, and finds the prospect incredibly exciting. A native of Philadelphia, he’s made San Francisco his home for over 25 years, but still can’t find a good hoagie there.

Recap: Continuous Testing in the Cloud [WEBINAR]

June 30th, 2014 by Bill McGee

Last week our own Michael Sage discussed the topic of  Continuous Testing in the Cloud for our latest webinar.

In it, he went over how to take advantage of cloud-hosted development resources to help increase release time and improve application quality as a best practice. He also talked about how to use Sauce Labs to securely execute your Selenium tests in parallel and reduce the time it takes to run your critical integration and acceptance tests.

Missed the webinar? You can listen to the recording here, or check out the slides below.

Re-Blog: JavaScript Multi Module Project – Continuous Integration

June 11th, 2014 by Bill McGee

lubos-krnacOur friend Lubos Krnac describes how to integrate Sauce with Protractor in a quest to implement continuous integration in his JavaScript multi module project with Grunt.

Below is a quote from his most recent blog post along side some code.

Read the rest of his post to get the full how-to here.

An important part of this setup is Protractor integration with Sauce Labs. Sauce Labs provides a Selenium server with WebDiver API for testing. Protractor uses Sauce Labs by default when you specify their credentials. Credentials are the only special configuration in test/protractor/protractorConf.js (bottom of the snippet). The other configuration was taken from the grunt-protractor-coverage example. I am using this Grunt plug-in for running Protractor tests and measuring code coverage.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
// A reference configuration file.
exports.config = {
  // ----- What tests to run -----
  //
  // Spec patterns are relative to the location of this config.
  specs: [
    'test/protractor/*Spec.js'
  ],
  // ----- Capabilities to be passed to the webdriver instance ----
  //
  // For a full list of available capabilities, see
  // and
  capabilities: {
    'browserName': 'chrome'
    //  'browserName': 'firefox'
    //  'browserName': 'phantomjs'
  },
  params: {
  },
  // ----- More information for your tests ----
  //
  // A base URL for your application under test. Calls to protractor.get()
  // with relative paths will be prepended with this.
  baseUrl: 'http://localhost:3000/',
  // Options to be passed to Jasmine-node.
  jasmineNodeOpts: {
    showColors: true, // Use colors in the command line report.
    isVerbose: true, // List all tests in the console
    includeStackTrace: true,
    defaultTimeoutInterval: 90000
  },
  
  sauceUser: process.env.SAUCE_USERNAME,
  sauceKey: process.env.SAUCE_ACCESS_KEY
};

You may ask “how can I use localhost in the configuration, when a remote selenium server is used for testing?” Good question. Sauce Labs provides a very useful feature called Sauce Connect. It is a tunnel that emulates access to your machine from a Selenium server. This is super useful when you need to bypass company firewall. It will be used later in main project CI configuration.

Have an idea for a blog post, webinar, or more? We want to hear from you! Submit topic ideas (or questions!) here.

Bleacher Report’s Continuous Integration & Delivery Methodology: Creating an Integration Testing Server

June 10th, 2014 by Bill McGee

Bleacher-report-logoThis is the second of a three part series highlighting Bleacher Report’s continuous integration and delivery methodology by Felix Rodriguez.  Read the first post here.

Last week we discussed how to continuously deliver the latest version of your application to a staging server using Elastic Beanstalk. This week we will be discussing how Bleacher Report continuously runs integration tests immediately after the new version of our app has been deployed.

When our deploy is complete, we use a gem called Slackr to post a message in our #deploys chat room. This is simple enough and just about any chat software can do this. We chose to use Slack because of the built-in integration functionality.

We created an outgoing webhook that submits any posts to our #deploys channel as a post to our Cukebot server. The Chukebot server searches the text, checks for a “completed a deploy” message, then parses the message as a Json object that includes the deploy_id, user, repo, environment, branch, and Github hash.

class Parser
  ##################################################
  ## Sample Input:
  # OGUXYCDI: Dan has completed a deploy of nikse/master-15551-the-web-frontpage-redux to stag_br5. Github Hash is 96dd307. Took 5 mins and 25 secs
  ##################################################
  def self.slack(params)
    text = (params["text"])
    params["deploy_id"] = text.match(/^(.*):/)[1]
    params["branch"] = text.match(/of\s(.*)\sto/)[1]
    params["repo"] = text.match(/to.*_(.*?)\d\./)[1]
    params["cluster"] = text.match(/to(.*?)_.*\d\./)[1]
    params["env"] = text.match(/to\s.*_.*?(\d)\./)[1]
    params["suite"] = set_suite(params["repo"]) 
    params["hash"] = text.match(/is\s(.*?)\./)[1]
    puts params.inspect
    return params
  end
end

Once parsed, we have all the information we need to submit and initiate a test suite run. A test suite and its contained tests are then recorded into our postgresql database.

Here is an example of what this suite would look like:

{
  id: 113,
  suite: "sanity",
  deploy_id: "FJBETJTY",
  status: "running",
  branch: "master",
  repo: "br",
  env: "4",
  all_passed: null,
  cluster: " stag",
  failure_log: null,
  last_hash: "0de4790"
}

Each test for that suite is stored in relation to the suite like so:

{
  id: 1151,
  name: "Live Blog - Has no 500s",
  url: "http://www.saucelabs.com/tests/20b9a64d66ad4f00b21bcab574783d73",
  session_id: "20b9a64d66ad4f00b21bcab574783d73",
  passed: true,
  suite_id: 113
},
{
  id: 1152,
  name: "Writer HQ - All Article Types Shown",
  url: "http://www.saucelabs.com/tests/4edbe941fdd8461ab6d6332ab8618208",
  session_id: "4edbe941fdd8461ab6d6332ab8618208",
  passed: true,
  suite_id: 113
}

This allows us to keep a record over time of every single test that was run and to which suite and deploy it belongs. We can get as granular as the exact code change using the Github hash and later include screenshots of the run. We also have a couple of different endpoints we can check for failed tests in a suite only, tests that have passed only, or the last test suite to run on an environment. We wanted to record everything in order to analyze our test data and create even more integrations.

This helps us automatically listen for those completed deploy messages we talked about earlier, as well as to have a way of tracking those tests runs later. After every test suite run we then post the permalink of the suite back into our #cukes chat room so that we have visibility across the company.

Another added benefit is that it allowed us to build a front end for non tech savvy people to initiate a test suite run on any environment.

Check it out for yourself; we just open sourced it.

Stay tuned next week for part two of this mini series! You can follow Felix on Twitter at .

Have an idea for a blog post, webinar, or more? We want to hear from you! Submit topic ideas (or questions!) here.