An Interview With QA Managers

November 23rd, 2015 by Joe Nolan

Whenever I go to a QA conference, I’m struck by just how many managers relate how the training is well and good, but they can’t get their companies to implement it. The problem is usually a resource constraint, company culture, or lack of management buy-in.

I wanted to understand this a little more, so using my role as founder of the DC Software QA and Testing Meetup, I reached out to my members and found two QA managers interested in discussing their teams.

About the Interviewees

Brian works for a federal government contractor on a development team consisting of 30 software and 5 QA engineers. The QA team has a manual testing background, with 2 of them having 3 years of UI automation experience. They all have a beginner to intermediate level of understanding of Agile processes, and largely worked in a Waterfall SDLC prior to their recent Agile adoption project. 3 of the members are aligned to Agile development teams while the other 2 are functioning in an extended fashion.

Sue works on a smaller team composed of both in-house and 3rd party development and QA team members, giving a total of 8 developers and 4 QA (plus a business analyst acting as QA). The QA team is able to conduct both front- and back-end tests for a Web-based product that serves a mix of commercial and government customers. The team members are dedicated to projects and cross-trained to back each other up.

Both managers are passionate, long term QA managers. They have worked with a variety of companies and projects and are both hands-on managers. Read the rest of this entry »

Merging Business Process With Continuous Delivery

November 20th, 2015 by Ashley Hunsberger

One of the biggest hurdles in getting to Continuous Delivery and truly being Agile does not lie within the development team itself. Change requires a mindset that all people (managers and executives too) must adopt. Just as it takes a village to raise a child, it also takes a (corporate) village to raise a new culture. The movement away from Waterfall to Agile and Continuous cannot be handled just by one person.

Does the following situation sound familiar? Someone asks you for an estimate for the Level of Effort (LOE) of a feature. Not just a t-shirt size, but in days — something to be delivered sometime that year. You don’t have much time to really dive into it (it could even be during the same meeting that you first hear about the new feature), but you have to know all the details up front. So you give a number that you aren’t supposed to be held to because of all sorts of caveats you listed. But, someone remembers that number. And since someone said you are Agile (and Continuous Delivery), now you have to not only meet that estimate you gave up front, you are now tied to an Agile sprint with no room for error. This goes against everything you say you are!

Read the rest of this entry »

Sauce Adds Automated Testing to Microsoft Visual Studio Online

November 18th, 2015 by Bill McGee

Sauce for VSO

Today we announced Sauce for Visual Studio Online at Microsoft Connect();, an integration that brings the goodness of Sauce Labs’ automated testing to Microsoft’s cloud-based continuous delivery platform.

If you are using VSO, you can now seamlessly and easily authenticate and launch tests on Sauce Labs as a part of your VSO build process. The plugin will also let you launch Sauce Connect™ to securely test pre-production apps and other assets that are behind a corporate firewall, and will also integrate Sauce Labs test results and data (like videos, screenshots, and logs) back into VSO.

You can learn more at the Visual Studio Marketplace, as well as on our revamped Docs section. Happy testing!


Check out the New Sauce Labs Docs

November 16th, 2015 by Ken Drachnik

As a company with its roots in the open source community, we understand how important information is in the world of modern software. From the code to the ReadMe, all the way up through user forums and training classes, each is a source of information that explains how something works, how to use it, and how to accomplish what you need to get done.

In your everyday use of Sauce Labs, probably no information is more important than the main product documentation, also known as “The Manual.” For the past several months we’ve been working on a complete overhaul of the manual so it’s easier to find information, and to make sure that the information it contains is of the highest quality. The result is our new Product Documentation wiki. Along with features like labels and a re-organized table of contents to help you find what you need faster, our wiki also has features like commenting so you can interact with the content it contains. Like good software, good documentation should be developed in response to the needs of its users, and we hope you will let us know what your needs are.

Our new wiki is the first of many new information development efforts to help you get the most out of using Sauce Labs. We’re also working on on-boarding and advanced training materials, refactoring our knowledge base at, and launching community forums. If you have any comments on our new information development efforts, or have ideas for documentation topics, training materials, knowledge base articles, or any other kind of information that would be particularly useful to you, please feel free to reach out to me directly at

Developing Consistency

November 16th, 2015 by Tom Overton

“Well, it works fine on my machine.”

That’s a phrase I loathe hearing when engineers are doing code review and an issue is discovered. Left to their own devices, engineers will approach setup of their development environments in whatever manner each one deems most efficient. Very frequently this creates two critical issues, a lack of consistency between the environments of the developers, and by consequence, disparities between development and production. These issues are fundamental. Quality is compromised when the application or the environments behave differently for each engineer. This is further complicated when there is a QA team. How were their environments built? How can they be confident they have written the appropriate test cases before release to production?

You want none of this…

Image source:,,

Image source:,,

The obvious benefit of ensuring consistent environments is improved code quality, but another great benefit is reduced finger-pointing and improved team morale. When setting up a team’s development environment, the first consideration to make is where to locate it. Should it be local or remote?

Read the rest of this entry »

The Importance of Mobile Testing

November 12th, 2015 by Greg Sypolt

It is a digital world, and all Generations have embraced the mobile revolution. It’s no surprise that millennials are leading the way. I saw a statistic recently that 85% of Millennials owns a smartphone. Next time you are waiting in line, take a look around.  Statistics don’t lie.



The mobile revolution has only just begun. The importance of having a mobile testing strategy is critical to our app releases.

What makes a great mobile app?

There are multiple elements that can influence a great (or bad) mobile app. The team works so hard developing an app (from idea, planning, writing code, and testing), the last thing you want to hear is negative feedback instantly after releasing the app. As long as the focus is on simplicity, understanding your users (audience), and the performance of your app, you are heading in the right direction. Read the rest of this entry »

Using Metrics to Discover Testing Holes

November 10th, 2015 by Joe Nolan

How do you know if you are wasting your time? What tests are effective and which ones are done because “That’s the way we do things”?

Tracking bug metrics can bring testing holes to light.

You know that feature you worked on, you were so proud of it. It was all fresh and shiny like a red Porsche:

Image source:

Image source:

And when it finally hits the app store, your ratings end up looking like this:

Image Source:

Image Source:

You ask yourself: How did it happen? Why are the customers finding so many bugs? What are we missing?

It sounds like you’ve got some holes in your testing!

The Proof is in the Pudding (or Data)

Read the rest of this entry »

Evangelizing Quality

November 6th, 2015 by Ashley Hunsberger

When I hear the word evangelist, all I can think about are those people on TV, preaching away. Not necessarily my cup of tea. As defined, an evangelist is someone who seeks to convert others to what he or she believes, especially by public preaching. I’ve never considered myself an evangelist — it goes against my general keep-to-myself nature. (Public speaking. Blech.) But I want to share what I know, and what I’ve learned, and I hope that others see the value in it. So, in the Church of Quality, I guess that makes me an evangelist. And we need more.

How do you get people to think more about quality?

It usually starts with a bad experience. Whether a customer issue came in, or you just had a very bad day testing, something negative tends to trigger a look at the quality of not just the product, but also the processes that broke. You need people to understand WHY quality is important, and why they should care. Read the rest of this entry »

Try Making It Smaller – It’s The Agile Way

November 4th, 2015 by Greg Sypolt

Agile is a must for development shops. Agile is a mature, iterative, collaborative methodology that breaks the development process down into shorter sprints. At its core, Agile development is about small iterations, test automation and a continuous integration pipeline.

Waterfall Was Created For a Perfect World, But We Don’t Live In One

Agile is a reaction to the slower, sequential approach known as Waterfall. Where Waterfall requires upfront planning to ensure that all details are accounted for, with no room for surprises or changes, Agile accounts for the inevitability of change, adapting to the project as it unfolds.

“Imagine a waterfall on the cliff of a steep mountain. Once the water has flowed over the edge of the cliff and has begun its journey down the side of the mountain, it cannot turn back. It is the same with waterfall development.”  (Search Software Quality)

To understand the advantages of Agile, it’s important to first understand the more traditional Waterfall methodology:

  • It is a sequential design process: discover, plan, build, test, review
  • Each project is based on the extensive collection of clear documentation gathered at the beginning
  • The whole product is only tested at the end of the cycle
  • It doesn’t take into account a client’s evolving needs or leave any room for revision Read the rest of this entry »

Boo! Your Scariest Testing Nightmares

October 29th, 2015 by The Sauce Labs Team

In the spirit of Halloween, this past month testing experts from around the globe answered the call to join in a “crowd-sourced” article highlighting their most frightening testing experience. These are real-life examples of testing gone wrong—leading to all sorts of mayhem in some cases. We allowed these testers to hide their identity, and to keep their organizations anonymous, to solicit the most behind-the-scenes responses.

In no particular order here are the top five Testing Nightmares, as told by the testers themselves:

The Importance Of Making Sure Your Hosting Company Gets Paid
By Ivar (from a medium sized company/ now acquired)

The company I was working for at the time had bet its existence on signing an enterprise contract that it had already been in negotiations for more than year with. We were building a digital asset management system that controlled the licensing and delivery of assets to corporate devices. (Think ‘private app store’.) I had written backup tooling that wasn’t ready for use but was included, unused, in the branch that was being deployed. Ten minutes after the final deployment (in preparation for client review!) *all* of our assets disappeared. Nothing new in the deployment was related to asset storage except my code. There was an eight-figure contract on the line and we had no backups. Fueled by the mother of adrenaline rushes, I reviewed my work and remained convinced that my code was not the cause of the loss. I was already into investigating other possible causes when another developer informed our team that after calling our asset host, he discovered that we had neglected to update our billing credit card. The host had turned off access to all of our assets ‘to get our attention’. They succeeded, but at oh what a price… Read the rest of this entry »