Environment-Agnostic Testing and Test Data Management for End-to-End Test Stability

July 28th, 2016 by Sahas Subramanian

In the Design Patterns for Scalable Test Automation webinar we discussed the importance of adapting proper patterns for the scaling and maintaining of E-E tests. A couple of additional important aspects for End-to-End (E-E) test stability are:

  • Environment-agnostic tests – Tests should be independent, self-contained units, and should run against any environment without code change, and with no dependency on anything else (apart from the runner)
  • Test data – How to prevent tests failing because expected data wasn’t available in the system

In the context of a web app (not legacy, thick-client applications), let’s take a look at how to deal with these challenges.

Environment-agnostic Tests

E-E tests need environment-specific configuration information such as the URL, role, user name, password, etc. Needless to say, hardcoding these parts of the test is not a good practice. It makes updates and maintenance difficult. A better solution would be to tokenize, keep the key/value pair separate and use them as part of the test flow. Different technologies offer different tactics to handle the need. Read the rest of this entry »

JUnit Testing: Getting Started and Getting the Most out of It

July 26th, 2016 by Chris Tozzi

If you’re a Java developer, you probably know and love JUnit. It’s the go-to tool of choice for unit testing (and, as we will see below, other types of testing as well) for Java apps.

In fact, JUnit is so popular that it’s the most commonly included external library on Java projects on GitHub, according to a 2013 analysis. No other Java testing framework comes close in popularity to JUnit.

But while JUnit is widely used, are all of the projects that deploy it getting the most out of it? Probably not. Here’s a look at what you should be doing to use JUnit to maximal effect.

JUnit Basics

First, though, let’s go over the basics of JUnit, just in case you haven’t used it before.


JUnit supports any platform on which Java runs, and it’s pretty simple to install. Simply grab the junit.jar and hamcrest-core.jar files from GitHub and place them in your test class path. Read the rest of this entry »

The Sauce Journey – “Forming, Storming, Norming, Performing”

July 21st, 2016 by Joe Alfaro

A few years ago, while working elsewhere, I came upon a scene of two engineers literally screaming at each other over the top of their cubicle walls about some aspect of a project. “Oh good,” I thought, “they’ve reached the storming stage, things can only get better from here.”

As I talked about in my previous post, forming Scrum teams leads to emergent behavior on the part of individuals as they adjust to the new regime. The same is true of small teams; once formed, the way in which individuals interact with each other tends to undergo a sequence of changes as well. The behavioral scientist Bruce Tuckman labeled these stages as Forming-Storming-Norming-Performing. As unpleasant as the transitions from stage to stage might be, all teams must progress through them in order to reach the point where they are truly self-managing.

In the forming stage, teams cohere in relation to external influences, like goals and tasks, but tend to remain focused on themselves as individuals. This is reinforced in the way that we typically constitute technical teams, where each person is recruited for their individual technical strengths. Forming is typically a stage that is driven by intellectual and analytical considerations, since it focuses on defining the project, identifying tasks, and assigning team members who can fulfill them. Read the rest of this entry »

How Does PhantomJS Fit Into Your Cloud Testing Strategy?

July 19th, 2016 by Greg Sypolt

PhantomJS is a lightweight headless test runner which is perfect for command-line-based testing. What is PhantomJS? It allows us to access the browser’s DOM API. After all, PhantomJS is still a browser without the GUI skin. It is suitable for local development, version control pre-commit hooks testing, and as part of your continuous integration pipeline testing.

The headless test runner WILL NOT be a replacement for Selenium functional testing. The idea behind a command-line base testing suite is that it will provide fast feedback for a deployed web application without spinning up a browser. It is critical to set a standard where developers naturally execute static code analysis, unit tests, server spec tests, and PhantomJS tests in a local development environment before creating a pull request. The next section will help you understand why a well-defined cloud testing strategy will reduce web application bugs in production.

Cloud Testing Strategy

A major component of cloud platform solutions is that all of the code goes through the gauntlet of testing before merging a pull request into the master branch, and before pushing changes to production. We need to prove that the code deploys the web application correctly before we start using it in the cloud staging and production environment. It’s much easier and cheaper to catch and troubleshoot issues locally, as opposed to having production servers go down or affect the user experience. Read the rest of this entry »

Recap: Automation Best Practices (Webinar)

July 14th, 2016 by Bill McGee

Thanks to everyone who joined us for our recent webinar, “Automation Best Practices, featuring Sauce Labs’ Automation Specialist Leo Laskin.

In this webinar, Leo discussed the value of open source resources in testing and also shared his personal experience in moving from manual to automated testing, the lessons he has learned, and the steps he took to build a powerful, international test coding army.

The presentation covered:

  • The good, bad and ugly when writing and testing code through automation
  • Challenges faced, including navigating around XPath and development best practices
  • Popular, key automation practices attendees can use to build their own coding armies

Interested in learning more about automated testing using Selenium? Download a free copy of Dave Haeffner’s Getting Started With Selenium.

Access the recording HERE and view the slides below: 

Quality Assurance and Software Testing: A Brief History

July 12th, 2016 by Chris Tozzi

Developers have been testing software since they first started building software following World War II. And quality assurance as a whole has a history that stretches back much further than that, of course.

But have you ever wondered how we got from the early days of programming – when developers relied on ad hoc methods for finding bugs in their code – to the modern world of Selenium and cloud-based testing?

Keep reading for a (brief and totally non-exhaustive) history of quality assurance and software testing.

The Origins of Quality Assurance

I could start by describing quality assurance processes in preindustrial societies, long before anyone had ever heard of software. But that would actually require writing a book.

So I’ll just quickly note some things that are probably obvious if you think about them, but that you might take for granted. Before the Industrial Revolution and the advent of modern capitalism, the calculus of quality assurance was a bit different than it is today. Markets were usually monopolized by guilds. Without free market competition, assuring quality wasn’t necessarily important for keeping customers happy. And in the absence of strong governments, attempts by the state to prevent defects in products tended to be rare or ineffectual. Read the rest of this entry »

Testing for IoT Development

July 7th, 2016 by Chris Tozzi

Like “the cloud” before it, the Internet of Things (IoT) is fast becoming one of the hottest new trends. Like it or not, there’s a good chance you’ll soon be developing IoT apps.

By extension, you’ll also probably have to develop a plan for testing IoT apps. That may sound intimidating if you’ve never done IoT tests before.

But it doesn’t have to be. Here’s an overview of the special considerations to bear in mind when planning for the IoT-centric future and the software tests that will come with it.

The IoT is Not a Single Thing

First, though, let’s be clear about what we mean by IoT.

To understand IoT testing, you have to understand that there is no single IoT. Instead, there are many different kinds of IoT devices, environments and apps.

Software running on an IoT-enabled traffic light will require very different sorts of tests from an IoT app on your smartwatch.

That means that the IoT testing landscape will be much more diverse than what we’re used to today, and so will the toolset that will have to accompany it. Read the rest of this entry »

Headless Browser Testing with CasperJS

June 30th, 2016 by Greg Sypolt

I must admit, the first time I heard about headless browser testing, I had zero knowledge of the technology. As I started to learn more about headless browser testing and compared it with Selenium, it quickly came to my attention that both are different, and both have different objectives. There is no rivalry or battle; both testing frameworks serve a purpose in your delivery chain.

A headless browser is a web browser without a graphical user interface. Headless browsers provide automated control of a web page in an environment similar to popular web browsers, but are executed via a command line interface or using network communication.1)Headless browser – Wikipedia, the free encyclopedia.” 2015. 1 Jun. 2016

Selenium is a portable software testing framework for web applications. Selenium also provides a record/playback tool for authoring tests without learning a test scripting language.2)Selenium (software) – Wikipedia, the free encyclopedia.” 2011. 1 Jun. 2016

Let’s take a closer look at CasperJS (headless browser testing framework) and how we can add another testing layer to our delivery chain. Read the rest of this entry »

References   [ + ]

Kickstart Your Automation Efforts

June 28th, 2016 by Joe Nolan

Gaining traction on your new automation efforts can be a challenge, especially when your team is new to the art. Teams can stall due to lack of time, no overall direction, or knowledge paralysis. But you can solve this roadblock by temporarily bringing on a developer.

I recently wrote about problems with QA teams adopting automation in my blog post “Why is Manual QA so Prevalent?” Shortly after writing that post, I joined a new team, and quickly discovered multiple issues. We needed help.

The Problems

I inherited a team that had been rooted in manual testing and was in the process of adapting automation practices. There are normally 1-2 engineers per scrum team, but they had recently become shorthanded due to a couple of promotions out of the team.

The team began to stall in its automation efforts due to: Read the rest of this entry »

Q & A : Design Patterns for Scalable Test Automation

June 24th, 2016 by The Sauce Labs Team

Thanks to everyone who joined the webinar given by Sahas Subramanian, “Design Patterns for Scalable Test Automation with Selenium and WebdriverIO”. There were a number of great questions that were posed prior to and during the session, and we asked Sahas to consolidate some of these questions and answer them in this follow-up post. Disclaimer: opinions shared below are Sahas’ and not those of his employer or Sauce Labs.

If you missed the webinar, you can find the video, slides and link to a related blog post here. Should you have any additional questions, send a tweet to @Sahaswaranamam.

Q: How can you best handle security authentication pop-ups from specific browsers? What are the best ways to switch between tabs and to close tabs?

A: Use the getCurrentTabId API to get the handle of the current window. Once you have the pop-up window handle, you could close it using browser.close(popUpHandle)

Reference: http://webdriver.io/api/window/getCurrentTabId.html

Q: How should I handle SOAP/SOAPUI testing?

A: Generally speaking, Selenium and Webdriver are appropriate for UI testing. If your intention is to test the APIs, I would suggest using tools like JMeter and/or Taurus. Reference: http://gettaurus.org

Q: How do I create my own wrapper? (How can I check for page title?)

A: Check out http://webdriver.io/api/protocol/title.html Read the rest of this entry »