A Getting Started Guide to Setting Up Jenkins

September 7th, 2016 by Greg Sypolt

The goal of this getting started guide is to help teams get Jenkins continuous integration (CI) servers configured, and discover how to make a newly deployed CI infrastructure fully operational. Jenkins is a leading open source CI server. It is flexible, providing hundreds of plugins to support building, testing, and deployment, and is capable of automating any project. Jenkins CI infrastructure can be deployed to on-prem, in the cloud using configuration management tools, and third-party vendor. For the purpose of this article, let’s assume that our Jenkins CI servers are deployed in the cloud and focus on configuration of Jenkins’ web interface. I will walk through various processes and steps to set up a new Jenkins CI server for production.

Recommended best practices for CI architecture

Don’t jump head first into configuration and creating a pipeline without planning, designing and establishing standards for your CI architecture. Taking the time to think about infrastructure first will enable a stable and restorable infrastructure. Let’s review a few recommended best practices to consider in your future CI pipeline.

Backup CI Server (Fail Safe)

It may seem obvious. Recommend setting up a backup process for Jenkins configuration. Script a Jenkins job to use the thinBackup plugin or S3 plugin to send the Jenkins configuration to an Amazon S3 (cloud storage).

Pipeline Configuration

Here are recommendations to consider: set environment variables (i.e., hidden passwords, SSH keys, API keys, etc.); security—create generic reusable jobs, naming conventions (i.e., jobs and environment variables); keep jobs small—modulization, scalable infrastructure that allows for auto-scaling of Slave nodes. (more…)

Sauce Updates OnDemand Plugin for Jenkins Pipeline

August 10th, 2016 by Ken Drachnik

We have updated our OnDemand plugin for Jenkins with explicit support for Jenkins Pipeline. This update enables development and testing teams to easily create, manage, and run automated tests using scripts on Jenkins, resulting in faster testing.

The newly enhanced OnDemand plugin enables Sauce Labs users to skip the Jenkins UI and run tests directly via scripts. Previously, users had to work within the Jenkins UI to create testing tasks. While this is initially convenient, as automation grows and becomes more complex, the process of creating testing tasks becomes more complex, too. Now OnDemand enables testers to automate releases for complex and non-sequential tasks that previously required manual intervention.

Key benefits of the update include:

Create scripts to automate Jenkins operations
Save time by scripting test sequences that can automate testing, retry tests automatically and run processes in parallel. Developers and QA can more easily customize their Jenkins workflows and reporting results.

Automatically create test scripts for Jenkins
Use the Jenkins snippet generator to easily create Pipeline scripts to execute Sauce Commands via Jenkins. Users don’t need to know how to script in Pipeline – simply copy the code snippets to program workflows

The Sauce OnDemand Jenkins Plugin is available as a free integration. Download the plugin from Jenkins and read how to configure it on our docs page.

Happy Testing!

Testing and Continuous Integration: Making Everything Work Together

May 17th, 2016 by Hemant Jain

Continuous integration (CI) has emerged as one of the most efficient ways to develop code. But testing has not always been a major part of the CI conversation.

In some respects, that’s not surprising. Traditionally, CI has been all about speeding up the coding, building, and release process. Instead of having each programmer write code separately, integrate it manually, and then wait until the next daily or weekly build to see if the changes broke anything, CI lets developers code and compile on a virtually continuous basis. It also means developers and admins can work together seamlessly, since the programming and build processes are always in sync.

Unfortunately, the quality assurance team does not necessarily reap the same benefits. While CI assures that your app keeps building successfully as the code is continually updated, it doesn’t automatically test how new builds behave within different types of environments. An otherwise well-run CI operation might require app testing to be done separately, on a non-continuous basis, instead of building it into the rest of the process.

This poses real problems for an organization. Unless you add automated testing to your CI mix, you could end up with an app that users can download and install properly, but which suffers from critical usability issues in certain browsers or operating systems. Arguably, an app that installs successfully but frustrates users due to lack of testing is worse than one that doesn’t install at all. (more…)

JIRA is Just a Click Away with Our New Plugin

May 3rd, 2016 by Ken Drachnik

As more and more devs work in agile teams, they need tools to plan, track, and release software and many of them use JIRA. As a tracking tool, JIRA is amazing for collaboration and project planning. For many teams, JIRA is the place of record for everything in the software development lifecycle. We have found that many of our customers use JIRA and the #1 product ask was to integrate Sauce Labs’ test results with JIRA so it would be simple to track all the tests associated with a project in one place.

Let’s say you are running an automated or manual test on Sauce Labs and find a bug. You want to add it to JIRA so that someone on your team can take a look or so that it can be prioritized in the backlog. Historically, one would have to download all the Sauce assets, login to JIRA, create a ticket, and upload the assets again. This can be tedious when you’re running lots of tests.

With Sauce Labs for JIRA, this is all simplified and automated. With the click of a button you can now create a JIRA ticket directly from your Test Details page. The plugin gives you the option to upload the screenshots, logs, and video link to your tests, making it easy to share out among your team.

To download Sauce Labs for JIRA, visit the Atlassian Marketplace: https://marketplace.atlassian.com/plugins/sauce-jira-integration/cloud/overview.  To read more, visit our JIRA integration Docs page.

Happy Testing!

The Sauce Labs Ecosystems & Integrations Team

Recap: Easy Continuous Deployment You Can Trust (Webinar)

April 14th, 2016 by Bill McGee

Thanks to everyone who joined us for our recent webinar, “Easy Continuous Deployment You Can Trust”, featuring Solano Labs Founding Engineer, Brian Kendzior, and Sauce Labs Solutions Architect, Neil Manvar.

In their presentation, Brian and Neil demonstrated a continuous deployment release process that used Solano CI, Sauce Labs, and AWS CodePipeline. The release process included smoke, unit, integration and browser tests to guarantee issue-free deployments.

Brian and Neil also demonstrated how to:

  • Build your software release pipeline by connecting different steps into an automated workflow using AWS CodePipeline
  • Run your automated web tests on any browser and operating system combination using Sauce Labs
  • Auto-parallelize your test runs with a Continuous Integration server using Solano CI

Want to learn more about automated testing and Continuous Integration? Download our free report, “How to Get the Most out of Your CI/CD Workflow Using Automated Testing”. 

Access the recording HERE and view the slides below: 

Test Faster and Smarter by Testing in Production

April 14th, 2016 by Chris Riley

You may dread the term testing in production (TiP). The thought of potential loss of data, downtime, and a damaged reputation to organizations can be daunting. But things need not be that way. In fact, today, testing in production is used by some of the biggest organizations with much success. But can it become a reality for your team?

Accident or Intentional?

Testing in production is not a completely new concept. In fact, you’ve probably seen it in action more often than you imagine. Think of an app that you’ve released or one that you know of that was poorly tested. You likely spent the next few weeks firefighting, and got it to be functional faster than you thought possible. In this case, you were forced to test in production. But, what if you could hone the art of testing in production and use it to your advantage? What if you could spot and fix issues so much faster than you do today? What if you could influence development from start to end? What if you could do all this without risking the reputation of your team or organization? That’s the promise of TiP, and it’s worth a second glance. (more…)

Recap: Achieve True Continuous Integration with Sauce Labs and Microsoft Visual Studio Team Services (Webinar)

March 11th, 2016 by Bill McGee

Thanks to everyone who joined us for our recent webinar, “Achieve True Continuous Integration with Sauce Labs and Microsoft Visual Studio Team Services”, featuring Sauce Labs Product Manager Jack Moxon. In his presentation. Jack shows the new Sauce Labs plugin for Visual Studio Team Services (VSTS) and how to launch tests on Sauce Labs as part of their VSTS build, enabling Continuous Integration (CI).

Jack also shows how the plugin allows users to launch Sauce Connect – a tunnel that allows customers to securely test pre-production web and mobile apps in the cloud. The plugin also integrates Sauce Labs test results and detailed reports (like videos, screenshots and logs) back into Visual Studio and TFS to enable collaboration and expedite the debugging process.

The webinar covers:

  • How Sauce Labs and Visual Studio Team Services (VSTS) are integrated for Continuous Integration
  • Launching automated tests as a part of a VSTS build
  • How to test pre-production apps securely behind your firewall
  • Sharing detailed test results and reports to collaborate and debug faster

Want to learn more about the Sauce Labs plugin for VSTS? We’ll be exhibiting at Microsoft Build in San Francisco March 30 – April 1, 2016. The conference is sold out but you can add a reminder to your calendar to make sure you don’t miss the live stream.

Access the recording HERE and view the slides below: 

Recap: Continuous Testing Meets the Classroom: Testing Online, Interactive Curriculum at Code.org [Webinar]

February 12th, 2016 by Bill McGee

Thanks to everyone who joined us for our recent webinar, “Continuous Testing Meets the Classroom: Testing Online, Interactive Curriculum at Code.org”, featuring Brian Jordan from Code.org. In his presentation, Brian discussed how Code.org approaches testing throughout the product development cycle, given their unique testing challenges – developing interactive, game-like curriculum for just the types of browsers you’d expect to find in school computer labs – from Internet Explorer 9 to iPads across 40+ languages.

The presentation also showed:

  • What Code.org’s open source automated testing stack looks like
  • What visual testing with Applitools, cross-browser Selenium tests on Sauce Labs, and live-site monitoring with New Relic and Honeybadger look like in practice
  • About the Code.org “Bug Collection” —real live examples of bugs detected before they hit production

Missed the presentation, want to hear it again, or share with a colleague?

Access the recording HERE and view the slides below.

Want to learn more about automated testing and Continuous Integration? Download our free report, “How to Get the Most out of Your CI/CD Workflow Using Automated Testing”.

Try Making It Smaller – It’s The Agile Way

November 4th, 2015 by Greg Sypolt

Agile is a must for development shops. Agile is a mature, iterative, collaborative methodology that breaks the development process down into shorter sprints. At its core, Agile development is about small iterations, test automation and a continuous integration pipeline.

Waterfall Was Created For a Perfect World, But We Don’t Live In One

Agile is a reaction to the slower, sequential approach known as Waterfall. Where Waterfall requires upfront planning to ensure that all details are accounted for, with no room for surprises or changes, Agile accounts for the inevitability of change, adapting to the project as it unfolds.

“Imagine a waterfall on the cliff of a steep mountain. Once the water has flowed over the edge of the cliff and has begun its journey down the side of the mountain, it cannot turn back. It is the same with waterfall development.”  (Search Software Quality)

To understand the advantages of Agile, it’s important to first understand the more traditional Waterfall methodology:

  • It is a sequential design process: discover, plan, build, test, review
  • Each project is based on the extensive collection of clear documentation gathered at the beginning
  • The whole product is only tested at the end of the cycle
  • It doesn’t take into account a client’s evolving needs or leave any room for revision (more…)

Cattle, Not Pets – Use Automation Software To Provision Your Servers

October 6th, 2015 by The Sauce Labs Team

This guest post was written by Julian Dunn, Product Manager at Chef Software (@julian_dunn)

In Greg Sypolt’s earlier post on immutable infrastructure, he outlined practices for treating your servers more like cattle and not pets. It’s better to rebuild things if you can rather than spending calories on manually debugging and fixing long-lived servers. (Of course, you will inevitably have a few pets in your infrastructure; for instance, database servers that actually hold customer data that would be inconvenient to constantly rebuild.)

Automation software like Chef can help with the provisioning and automated setup of new cattle in a consistent way without having to resort to shell scripting or other ugly, hard-to-understand tools. Chef can also help you with maintaining consistency among the few pet servers that you do have.

One trend we’ve seen in the pets-to-cattle migration is the externalization of interface testing tools from long-lived, always-on test driver clusters to ephemeral instances or containers to drive UI tests, record the results, report them to a central dashboard and then destroy themselves. These cloud-native testing systems like Sauce Labs also provide a far richer set of functionality, like video recording and playback of the UI test, than can be achieved with artisanally-crafted, on-premise solutions. In other words, why mess around with having to configure and maintain Xvfb, Selenium, Webdriver and so on, yourself, for a boatload of platforms and platform versions when you can just use Sauce Labs?

(more…)