Avoiding the UI: Why and How to Run Tests With Scripts

October 4th, 2016 by Michael Churchman

There’s no doubt about it: a user interface (whether it’s graphic or text-only) can be very nice, at least when you need to make decisions in real time or enter data on the spot.

But when you know exactly what you’re going to do and how you’re going to perform each step, and you have a set of tasks that you’re likely to perform more than once or twice, any kind of user interface can slow you down, get in the way, and eventually become a maddening, time-wasting annoyance. This is nowhere more true than with testing, where complex and highly repetitive tasks are the norm, and there is nothing to be gained by waiting to enter an occasional command or piece of data.

This article explains the value of test scripts, and the circumstances under which you should consider using them to perform software tests. (more…)

Software Testing Tools for Your QA Team

September 27th, 2016 by Chris Riley

Ashley Hunsberger, Greg Sypolt and Chris Riley contributed to this post.

Software testing tools are a vital resource for every successful QA team. But with so many tools and testing frameworks out there – from Selenium and Protractor to Espresso and Xcode – how do you choose which are best? How should your toolset vary depending on whether you do desktop testing, mobile testing, or both? And how do you make the most of software testing tools?

Below are answers to these questions from the panelists of a recent Sauce Labs webinar focused on software testing and QA. The webinar was hosted by Chris Riley, with Ashley Hunsberger and Greg Sypolt serving as panelists. You can also find their recommendations on software testing tools below.

Which tools has Greg used for test automation at Gannett?

Greg: Here’s an inventory of testing frameworks and tooling used across Gannett products (technology alignment): (more…)

A Getting Started Guide to Setting Up Jenkins

September 7th, 2016 by Greg Sypolt

The goal of this getting started guide is to help teams get Jenkins continuous integration (CI) servers configured, and discover how to make a newly deployed CI infrastructure fully operational. Jenkins is a leading open source CI server. It is flexible, providing hundreds of plugins to support building, testing, and deployment, and is capable of automating any project. Jenkins CI infrastructure can be deployed to on-prem, in the cloud using configuration management tools, and third-party vendor. For the purpose of this article, let’s assume that our Jenkins CI servers are deployed in the cloud and focus on configuration of Jenkins’ web interface. I will walk through various processes and steps to set up a new Jenkins CI server for production.

Recommended best practices for CI architecture

Don’t jump head first into configuration and creating a pipeline without planning, designing and establishing standards for your CI architecture. Taking the time to think about infrastructure first will enable a stable and restorable infrastructure. Let’s review a few recommended best practices to consider in your future CI pipeline.

Backup CI Server (Fail Safe)

It may seem obvious. Recommend setting up a backup process for Jenkins configuration. Script a Jenkins job to use the thinBackup plugin or S3 plugin to send the Jenkins configuration to an Amazon S3 (cloud storage).

Pipeline Configuration

Here are recommendations to consider: set environment variables (i.e., hidden passwords, SSH keys, API keys, etc.); security—create generic reusable jobs, naming conventions (i.e., jobs and environment variables); keep jobs small—modulization, scalable infrastructure that allows for auto-scaling of Slave nodes. (more…)

How to Avoid Thread.Sleep in Test Automation

August 30th, 2016 by Sahas Subramanian

Several years back, I wanted to understand if it’s really necessary to use thread.sleep in test automation code. Here is what stuck in my mind: Thread.Sleep(n) means blocking the current thread for the number of time slices that occur within “n” milliseconds.

There are a few things to consider before using thread.sleep in your test automation code. They include:

  1. The thread time slice varies from OS to OS and processor to processor. This means that once your thread goes to sleep, it’s going to wait for its turn to come back into the scheduler. So it’s highly likely that the thread.sleep time means more than what you really intended. For example, with thread.sleep(1000), you intended 1,000 milliseconds, but it could potentially sleep for more than 1,000 milliseconds too as it waits for its turn in the scheduler.
  2. Each thread has its own use of CPU and virtual memory. That means our thread acquired its resources (say 1MB virtual memory, some CPU cycles and some context switching cycles), and is simply not using them.
  3. If it’s a foreground thread, it’s preventing the application from exiting as well.
  4. What about Thread.Sleep(0)? This tells the processor that I’m ready to lose the remainder of my time slice, and you may pick up the next thread from the queue. On one hand, it feels good and efficient. But on the other hand, you are also telling the processor that you are better at scheduling than the processor. (I don’t know the side effects this can cause, but I’m not smarter than the processor.)


We are talking about milliseconds when dealing with thread.sleep, but it’s important to understand that the more we use thread.sleep, the more we defer solving the real problem, which is how to handle async in modern web apps. (more…)

Building a Test Automation Strategy

August 23rd, 2016 by Chris Riley

Ashley Hunsberger, Greg Sypolt and Chris Riley contributed to this post.

Bringing test automation into your organization is not as easy as writing and running a Selenium script. It involves first getting buy-in, building a team, establishing a strategy, and picking the right tools. During the Q&A portion of a recent webinar hosted by Chris Riley, Ashley Hunsberger, and Greg Sypolt, the presenters realized that these aspects of introducing test automation are well known, but not well understood. In our first post of the series we discussed getting buy-in. Below, in the second post, we discuss how to build a test automation strategy.

Getting started with test automation is easy. If you have a technically minded QA team, you can usually create your test script, sign up for a test cloud, and run the script in just a few hours. But keeping a test automation environment going for the long term is not as easy as any of us would like to believe. QA teams are generally better at building strategy than any other. And when it comes time to build a test automation environment, strategy is a key first element to both getting started and keeping it going. (more…)

Options for Validating HTTP Codes in Selenium

August 18th, 2016 by Alex Entrekin

There is no way to directly get HTTP status codes in the WebDriver API (see the lengthy discussion in issue #141). But that doesn’t mean you have to leave Selenium or go without any status codes in your test scripts.

In fact, some of the supported methods – proxies and tightly coupled headless browsers – should make you feel at home if you’ve transitioned from Selenium RC, or are comfortable with traffic sniffing proxies.

Headless Scriptable WebKits that Play Nicely with Selenium

If you don’t want to invest the time configuring a proxy, consider setting up WebDriver to handle headless testers like HTMLUnitDriver, Jasmine or PhantomJS.HTMLUnitDriver is well-supported in Selenium, but its Javascript engine (Rhino) is not used by most browsers. However, if you know you won’t be testing any pages with Javascript, then it’s no concern.

But if you are, use GhostDriver to enable PhantomJS as WebDriver’s backend. (more…)

Getting Buy-In For Test Automation

August 16th, 2016 by Chris Riley

Ashley Hunsberger, Greg Sypolt and Chris Riley contributed to this post.

Bringing test automation into your organization is not as easy as writing and running a Selenium script. It involves getting buy-in, building a team, establishing a strategy and picking the right tools. During the Q&A portion of a recent webinar hosted by Chris Riley, Ashley Hunsberger, and Greg Sypolt, the presentation team realized that while each aspect is well known, it is not well understood. And that is why in this four-part blog post series, we are going to address each one. This post, the first of the series, discusses getting buy-in.

The need and high-level direction for test automation is usually driven by those who do. In an ideal scenario, both the doers and the decision-makers come to the conclusion together that faster, more repeatable testing is more than just a cost-saver. It supports the initiatives of moving modern development practices forward.

But this is not the reality. Those who know what is possible need to sell the idea from the bottom up. For QA, there are many challenges to doing this: (1) QA teams often do not have their own budget; (2) you are not just convincing development leadership, you are convincing your peers in development and IT as well; (3) all aspects are not in your control (such as continuous integration and test infrastructure). So let’s answer questions asked during the webinar to give some insight on how to approach buy-in. (more…)

Recap: Automation Best Practices (Webinar)

July 14th, 2016 by Bill McGee

Thanks to everyone who joined us for our recent webinar, “Automation Best Practices, featuring Sauce Labs’ Automation Specialist Leo Laskin.

In this webinar, Leo discussed the value of open source resources in testing and also shared his personal experience in moving from manual to automated testing, the lessons he has learned, and the steps he took to build a powerful, international test coding army.

The presentation covered:

  • The good, bad and ugly when writing and testing code through automation
  • Challenges faced, including navigating around XPath and development best practices
  • Popular, key automation practices attendees can use to build their own coding armies

Interested in learning more about automated testing using Selenium? Download a free copy of Dave Haeffner’s Getting Started With Selenium.

Access the recording HERE and view the slides below: 

Headless Browser Testing with CasperJS

June 30th, 2016 by Greg Sypolt

I must admit, the first time I heard about headless browser testing, I had zero knowledge of the technology. As I started to learn more about headless browser testing and compared it with Selenium, it quickly came to my attention that both are different, and both have different objectives. There is no rivalry or battle; both testing frameworks serve a purpose in your delivery chain.

A headless browser is a web browser without a graphical user interface. Headless browsers provide automated control of a web page in an environment similar to popular web browsers, but are executed via a command line interface or using network communication.1)Headless browser – Wikipedia, the free encyclopedia.” 2015. 1 Jun. 2016

Selenium is a portable software testing framework for web applications. Selenium also provides a record/playback tool for authoring tests without learning a test scripting language.2)Selenium (software) – Wikipedia, the free encyclopedia.” 2011. 1 Jun. 2016

Let’s take a closer look at CasperJS (headless browser testing framework) and how we can add another testing layer to our delivery chain. (more…)

References   [ + ]

Kickstart Your Automation Efforts

June 28th, 2016 by Joe Nolan

Gaining traction on your new automation efforts can be a challenge, especially when your team is new to the art. Teams can stall due to lack of time, no overall direction, or knowledge paralysis. But you can solve this roadblock by temporarily bringing on a developer.

I recently wrote about problems with QA teams adopting automation in my blog post “Why is Manual QA so Prevalent?” Shortly after writing that post, I joined a new team, and quickly discovered multiple issues. We needed help.

The Problems

I inherited a team that had been rooted in manual testing and was in the process of adapting automation practices. There are normally 1-2 engineers per scrum team, but they had recently become shorthanded due to a couple of promotions out of the team.

The team began to stall in its automation efforts due to: (more…)