Posts Tagged ‘Continuous Delivery’

How Yahoo! Mail Transformed Its Functional Testing + Continuous Delivery Process [WEBINAR]

October 8th, 2014 by Bill McGee

Yahoo Mail 2 (1)

Learn how one Engineer helped transform internet giant Yahoo! Mail’s quality engineering process through functional testing and continuous delivery – and the role Sauce Labs plays in their process.

Be sure to sign up for our next webinar on Wednesday, October 22nd, 2014, at 11:00am PDT. You won’t want to miss this one! 

Click here to register.

(more…)

HomeAway + Sauce Labs: Continuous Delivery Made Awesome [VIDEO]

October 7th, 2014 by Bill McGee

A few weeks back, we sat down with Carl Shaulis at HomeAway to learn how they leverage Sauce Labs‘ cloud in their testing process.

HomeAway, Inc. is the world’s leading online marketplace for the vacation rental industry. With over one million live vacation rental listings in 190 countries paired with a family of 50 websites and hundreds of applications thanks to a series of acquisitions, quality is their top priority. To get ahead of the curve, Carl’s team is trying to automate and use best practices like continuous integration as much as possible. We were also thrilled to hear that they’re using Appium, a cross-platform test automation framework powered by Sauce Labs, for their mobile testing.  Carl also says Sauce fits perfectly into their testing process.

Watch the video below to learn more about their path towards continuous delivery, and how Sauce fits into that story.

(more…)

Recap: Continuous Testing in the Cloud [WEBINAR]

June 30th, 2014 by Bill McGee

Last week our own Michael Sage discussed the topic of  Continuous Testing in the Cloud for our latest webinar.

In it, he went over how to take advantage of cloud-hosted development resources to help increase release time and improve application quality as a best practice. He also talked about how to use Sauce Labs to securely execute your Selenium tests in parallel and reduce the time it takes to run your critical integration and acceptance tests.

Missed the webinar? You can listen to the recording here, or check out the slides below.

Bleacher Report’s Continuous Integration & Delivery Methodology: Creating an Integration Testing Server

June 10th, 2014 by Bill McGee

Bleacher-report-logoThis is the second of a three part series highlighting Bleacher Report’s continuous integration and delivery methodology by Felix Rodriguez.  Read the first post here.

Last week we discussed how to continuously deliver the latest version of your application to a staging server using Elastic Beanstalk. This week we will be discussing how Bleacher Report continuously runs integration tests immediately after the new version of our app has been deployed.

When our deploy is complete, we use a gem called Slackr to post a message in our #deploys chat room. This is simple enough and just about any chat software can do this. We chose to use Slack because of the built-in integration functionality.

We created an outgoing webhook that submits any posts to our #deploys channel as a post to our Cukebot server. The Chukebot server searches the text, checks for a “completed a deploy” message, then parses the message as a Json object that includes the deploy_id, user, repo, environment, branch, and Github hash.

class Parser
  ##################################################
  ## Sample Input:
  # OGUXYCDI: Dan has completed a deploy of nikse/master-15551-the-web-frontpage-redux to stag_br5. Github Hash is 96dd307. Took 5 mins and 25 secs
  ##################################################
  def self.slack(params)
    text = (params["text"])
    params["deploy_id"] = text.match(/^(.*):/)[1]
    params["branch"] = text.match(/of\s(.*)\sto/)[1]
    params["repo"] = text.match(/to.*_(.*?)\d\./)[1]
    params["cluster"] = text.match(/to(.*?)_.*\d\./)[1]
    params["env"] = text.match(/to\s.*_.*?(\d)\./)[1]
    params["suite"] = set_suite(params["repo"]) 
    params["hash"] = text.match(/is\s(.*?)\./)[1]
    puts params.inspect
    return params
  end
end

Once parsed, we have all the information we need to submit and initiate a test suite run. A test suite and its contained tests are then recorded into our postgresql database.

Here is an example of what this suite would look like:

{
  id: 113,
  suite: "sanity",
  deploy_id: "FJBETJTY",
  status: "running",
  branch: "master",
  repo: "br",
  env: "4",
  all_passed: null,
  cluster: " stag",
  failure_log: null,
  last_hash: "0de4790"
}

Each test for that suite is stored in relation to the suite like so:

{
  id: 1151,
  name: "Live Blog - Has no 500s",
  url: "http://www.saucelabs.com/tests/20b9a64d66ad4f00b21bcab574783d73",
  session_id: "20b9a64d66ad4f00b21bcab574783d73",
  passed: true,
  suite_id: 113
},
{
  id: 1152,
  name: "Writer HQ - All Article Types Shown",
  url: "http://www.saucelabs.com/tests/4edbe941fdd8461ab6d6332ab8618208",
  session_id: "4edbe941fdd8461ab6d6332ab8618208",
  passed: true,
  suite_id: 113
}

This allows us to keep a record over time of every single test that was run and to which suite and deploy it belongs. We can get as granular as the exact code change using the Github hash and later include screenshots of the run. We also have a couple of different endpoints we can check for failed tests in a suite only, tests that have passed only, or the last test suite to run on an environment. We wanted to record everything in order to analyze our test data and create even more integrations.

This helps us automatically listen for those completed deploy messages we talked about earlier, as well as to have a way of tracking those tests runs later. After every test suite run we then post the permalink of the suite back into our #cukes chat room so that we have visibility across the company.

Another added benefit is that it allowed us to build a front end for non tech savvy people to initiate a test suite run on any environment.

Check it out for yourself; we just open sourced it.

Stay tuned next week for part two of this mini series! You can follow Felix on Twitter at .

Have an idea for a blog post, webinar, or more? We want to hear from you! Submit topic ideas (or questions!) here.