Dealing With Test Log Data

October 26th, 2016 by Michael Churchman

Test logs. What are they good for? What can you do with them? What should you do with them? These aren’t always easy questions to answer, but in this post, we’ll take a look at what’s possible and what’s advisable when it comes to testing log data.

What are Test Logs Good For?

What are test logs good for? Or are they good for anything at all?

Let’s start with an even more basic question: What is (and what isn’t) a testing log? A testing log is not simply test output. Minimal pass/fail output may log the results of testing, but a true testing log should do more than that. At the very least, it should log the basics of the testing process itself, including test files used, specific test steps as they are performed, and any output messages or flags, with timestamps for each of these items. Ideally, it should also log key processes and variables indicating the state of the system before, during, and after the test. Read the rest of this entry »

Sauce Labs Donates Appium Copyright to JS Foundation

October 17th, 2016 by Jonathan Lipps

Today, Sauce Labs is proud to be a part of a significant moment in the JavaScript community and ecosystem, with the official launch of the JS Foundation. The JS Foundation exists to foster innovation and collaboration in the world of JavaScript, and aims to be an umbrella for the wide range of projects that are either written in JavaScript or otherwise participate in the JavaScript ecosystem. Sauce Labs has joined the JS Foundation as a founding member in part because we believe that exciting things are happening in the JS world, around testing and beyond, and want to support the growth of this awesome community.

But this is not all—today Sauce Labs is very happy to announce that we have donated Appium, the current standard in mobile automation around the world, to the JS Foundation! Written in Node.js, Appium is a clear example of the breadth of the JS ecosystem and how JS can power an enormous range of projects useful not only to JS developers but to a much wider audience. Appium has always been an open source project (available under the Apache 2 license), and developed with full transparency to its users. The Appium project is extremely proud of its large contributorship, most of which has no relationship with Sauce Labs. At Sauce, we felt it was time to make a legal reality out of what has been a reality for the project since its inception, and we couldn’t be more excited to hand Appium’s copyright over to the JS Foundation for long-term stewardship.

Rest assured, this does not signal a reduction of interest in Appium from Sauce Labs; to the contrary, it’s an acknowledgement that the Appium project belongs to the community, not just Sauce Labs. Our Open Source team will continue to focus on Appium’s development as one of its main priorities. Our hope is that in the coming months, Appium’s move to a non-profit foundation will open up new opportunities for contributors and sponsors to be more involved in Appium’s roadmap and position in the ecosystem.

At Sauce, we care about testing and DevOps practices in all language communities, and that is part of why we built our test cloud around the vision of Selenium and Appium’s language-agnostic design. Stay tuned for how we are continuing to be involved with Python, Ruby, and others. But today we celebrate a wonderful development for the JavaScript community with the introduction of the JS Foundation, and are ecstatic to support it and Appium moving forward.

Happy testing, and happy JavaScripting!

Avoiding the UI: Why and How to Run Tests With Scripts

October 4th, 2016 by Michael Churchman

There’s no doubt about it: a user interface (whether it’s graphic or text-only) can be very nice, at least when you need to make decisions in real time or enter data on the spot.

But when you know exactly what you’re going to do and how you’re going to perform each step, and you have a set of tasks that you’re likely to perform more than once or twice, any kind of user interface can slow you down, get in the way, and eventually become a maddening, time-wasting annoyance. This is nowhere more true than with testing, where complex and highly repetitive tasks are the norm, and there is nothing to be gained by waiting to enter an occasional command or piece of data.

This article explains the value of test scripts, and the circumstances under which you should consider using them to perform software tests. Read the rest of this entry »

Thinking Outside the Box: #30DaysOfTesting

September 29th, 2016 by Ashley Hunsberger

Do you ever find yourself stuck in a QA rut? I will be the first to admit that I have been there. You probably know the signs. You can predict to the minute what you’ll be doing that day (or week, sprint, or month). You start to go on autopilot, not really thinking of different ways to test, or worse – not even thinking of the testing that’s truly best for the product.

Sometimes you just need to mix it up, think outside the box, and do something different… but how?

One good answer is to complete the Ministry of Testing’s #30DaysOfTesting Challenge

The challenge is a series of activities for software testers that are designed to be performed over a thirty-day period. The test launched in July, but it is a self-guided challenge, which you can perform during any period, according to your own schedule.

A different kind of challenge

The general idea is to do a different testing activity each day over the course of a month – all very different from each other. (Full disclosure: I am terrible at actually doing things like this once a day, much less sharing it, but that’s beside the point). The end goal? Improving yourself as a tester (while having a bit of fun)!

While completing some challenges on the list, I had to really learn what I was doing. Mind maps aren’t actually my forte, and I still have yet to tackle an “out by one” error. Other challenges forced me to step outside my comfort zone (like inviting a non-tester to an event, in which case I actually got a UI Architect to apply to speak at a test conference!)

Image Source:

Tips for Success

As with anything worth doing, this list can be a challenge just to complete! Here are some tips that worked for me:

Make time for it! Just schedule 30 minutes to an hour each morning and do something from the list. This helped me warm up for the day in my regular work, and got my creative juices flowing.

Don’t sweat it if you can’t do something EVERY. SINGLE. DAY. I took the pressure off of myself to do each thing on the given day, and instead picked something that interested me at the moment. While it’s great if you can perform the challenge within a month, you shouldn’t rake yourself over the coals if it takes longer. Completing the challenges and learning from them is more important than sticking to a rigid schedule.

Why are you doing this? Before beginning the challenge, set a personal goal – and not just because someone said to. What are you looking to accomplish for yourself?

Most importantly, have fun with it! Some days, as my kids would say, “I just don’t wanna!” Take testing out of it. Do a task from a different perspective! I completed one challenge in a way that had ABSOLUTELY NOTHING to do with testing (yes, shameless picture of my dog).

Image Source:

Get Results

I’ll admit that at first all I could think of were those exercise promos (you know what I’m talking about – “In just 20 minutes a day, you too can look like this…”). But as I started doing these exercises challenges, I actually found myself improving (and identifying the areas where I so badly need to improve).

The best things about the challenges: I started to remember back to when I entered the QA field more than a decade ago, and had FUN with testing. And more importantly, I found I was constantly learning and I continue to want to know more. My list of things to research grows each day, and I can’t wait to sign on in the morning to pick a new challenge.

Ashley Hunsberger is a Quality Architect at Blackboard, Inc. and co-founder of Quality Element. She’s passionate about making an impact in education and loves coaching team members in product and client-focused quality practices. Most recently, she has focused on test strategy implementation and training, development process efficiencies, and preaching Test Driven Development to anyone that will listen. In her downtime, she loves to travel, read, quilt, hike, and spend time with her family.

Software Testing Tools for Your QA Team

September 27th, 2016 by Chris Riley

Ashley Hunsberger, Greg Sypolt and Chris Riley contributed to this post.

Software testing tools are a vital resource for every successful QA team. But with so many tools and testing frameworks out there – from Selenium and Protractor to Espresso and Xcode – how do you choose which are best? How should your toolset vary depending on whether you do desktop testing, mobile testing, or both? And how do you make the most of software testing tools?

Below are answers to these questions from the panelists of a recent Sauce Labs webinar focused on software testing and QA. The webinar was hosted by Chris Riley, with Ashley Hunsberger and Greg Sypolt serving as panelists. You can also find their recommendations on software testing tools below.

Which tools has Greg used for test automation at Gannett?

Greg: Here’s an inventory of testing frameworks and tooling used across Gannett products (technology alignment): Read the rest of this entry »

Team Building and Quality Assurance

September 22nd, 2016 by Chris Riley

Ashley Hunsberger, Greg Sypolt and Chris Riley contributed to this post.

How do you build an effective team of quality assurance (QA) engineers? Where do you look to recruit the best QA professionals? How should you integrate your QA team within the rest of your organization? These and other questions related to the topic of team building came up during a recent webinar hosted by Chris Riley, Ashley Hunsberger, and Greg Sypolt. Here are links to the first and second post in this series.

Team building is an essential part of an effective QA operation, of course. Your testing strategy and execution are only as strong as the people who design and conduct them. And in today’s IT world, where DevOps is changing the way different types of engineers interact, and the supply of good QA professionals outstrips demand, having a well-planned and executed approach to team building is crucial.

Below are some of the most important team-building questions that came up during the webinar, along with answers from participants.

One thing I have noticed lately is that QA engineers are getting more respect. With automation, their skillset looks more like those of a developer. Do you agree? Read the rest of this entry »

The Importance of Eliminating Network Hops

September 20th, 2016 by Greg Sypolt

Are you experiencing slower execution times while running Selenium scripts in the Selenium cloud network? Too many network hops will add latency and slow down your test execution. Plus, every additional network hop adds cost to your execution. One way to optimize Selenium execution performance is to eliminate as many network hops as possible.

“A hop is one portion of the path between source and destination. Data packets pass through bridges, routers and gateways on the way. Each time packets are passed to the next device, a hop occurs.”1)Hop (networking) – Wikipedia, the free encyclopedia.” 2011. 26 Jan. 2016

The Sauce Labs team is working on a new tool that may be interesting to you. The solution will eliminate a network hop, making running tests in parallel on multiple browsers automatic and simple.

Why Hops Matter

In my first time using a cloud-based solution for Selenium, I was both thrilled and impressed by the possibilities. A cloud-based provider such as Sauce Labs maintains and uses configuration management tools to spin up and tear down virtual machines with various OS platforms and browsers. I understand the value in this type of service. The amount of time and resources needed to build and maintain infrastructure is expensive. Been there, done that! Read the rest of this entry »

References   [ + ]

1. Hop (networking) – Wikipedia, the free encyclopedia.” 2011. 26 Jan. 2016

Making Your App Testable

September 16th, 2016 by Dan Cuellar

When writing test automation, one of the most important factors for determining the amount of time and resources you will consume (and ultimately the success or failure of the endeavor) is the testability of your application. By testability, I’m referring to how the app interacts with UI (and other) automation frameworks, the ease by which a test script can setup the scenarios you wish to test, and how you make your tests safe for concurrency.

Making Elements Accessible

Let’s address the matter of controlling your application with automation frameworks first. Given that you are reading this blog, it is likely you are using either Selenium or Appium as your automation framework, so this blog post will only address these frameworks.

Web Content (Selenium)

Let’s start with Selenium. In order for your web app to be easily controlled by Selenium, you need to think ahead as to how you will identify important pieces of the DOM when constructing tests. Selenium provides many means by which you can do this, called “Locator Strategies”. Some are better than others. You should consider which of these you will be using in your tests as you develop the user interface. Ideally, each element would have an ID attribute applied directly to the tag for any element that the test will exercise.

Sometimes, for one reason or another, you may not be able to use an ID. If this is the case, the next recommended technique is to use a CSS selector. If your web app is developed with good principles, such as BEM (Block Element Modifier), it is likely easier to automate as it should have a relatively short globally unique CSS selector. If it does not, I would not recommend adding a CSS class just for automation, as it isn’t the purpose of the styling language to do automation. Rather, I would suggest that you use an HTML  data-* attribute. You can use the CSS selector locator to grab an element by one of these attributes, but the good news is you aren’t adding unnecessary classes to your CSS, and the purpose of the HTML attribute will be much clearer. Read the rest of this entry »

Selenium 3 Is Coming!

September 12th, 2016 by Simon Stewart

Selenium 3 is coming! I’m here to tell you about what’s changed, and what impact this will have on your testing.


  • WebDriver users will just find bug fixes and a drop-in replacement for 2.x.
  • Selenium Grid users will also find bug fixes and a simple update.
  • The WebDriver APIs are now the only APIs actively supported by the Selenium project.
  • The Selenium RC APIs have been moved to a “legacy” package.
  • The original code powering Selenium RC has been replaced with something backed by WebDriver, which is also contained in the “legacy” package.
  • By a quirk of timing, Mozilla have made changes to Firefox that mean that from Firefox 48 you must use their geckodriver to use that browser, regardless of whether you’re using Selenium 2 or 3.

In more depth:

When we released Selenium 2.0 in 2011, we introduced the new WebDriver APIs, and encouraged everyone to start moving to them. If you’re using the WebDriver APIs, then Selenium 3.0 is a simple drop-in upgrade. We’ve not changed any of the WebDriver APIs, and the code is essentially the same as the last 2.x release. If you’re using Selenium Grid, the same applies: in most cases, you can just drop in the new JAR (or update your maven dependency to 3.0.0), and you’re done.

At the same time as the Selenium project is shipping Selenium 3.0, Mozilla is changing the internals of Firefox in a way that makes it more stable and secure, but which also makes the community-provided Firefox Driver no longer work. As such, if you use Firefox for your testing, you’ll need to use the geckodriver, which is an executable similar to the chromedriver and MS’s edgedriver. You’ll need to start using geckodriver even if you’re using Selenium 2 — the change is in the browser, not Selenium. Read the rest of this entry »

How Often Should You Parallel Test?

September 9th, 2016 by Michael Churchman

How often should you parallel test? If that sounds like a trick question, maybe it is. In this post, we’ll let you in on the “trick” part of the question, and then we’ll talk about what really matters when it comes to when and how often you should parallel test.

Parallel Testing – What is it?

First, the trick. It lies in what parallel testing is, and more to the point, what it isn’t.

What is parallel testing? The term “parallel testing” is generic and rather broad, but it typically refers to automated systems for simultaneously testing multiple applications or components, with each application or subcomponent being tested on a different computer. It is sometimes also used to describe automated testing of a single application or component on multiple platforms. The test computers can be individual hardware units, or more typically, separate virtual machines. In all cases, however, the combination of automation and multiple test systems makes it possible to run many more tests than would be practical with serial testing, and it reduces the time required for testing to a fraction of that required for the equivalent serial tests. The key points to keep in mind about parallel testing are that: Read the rest of this entry »