An Interview With QA Managers

November 23rd, 2015 by Joe Nolan

Whenever I go to a QA conference, I’m struck by just how many managers relate how the training is well and good, but they can’t get their companies to implement it. The problem is usually a resource constraint, company culture, or lack of management buy-in.

I wanted to understand this a little more, so using my role as founder of the DC Software QA and Testing Meetup, I reached out to my members and found two QA managers interested in discussing their teams.

About the Interviewees

Brian works for a federal government contractor on a development team consisting of 30 software and 5 QA engineers. The QA team has a manual testing background, with 2 of them having 3 years of UI automation experience. They all have a beginner to intermediate level of understanding of Agile processes, and largely worked in a Waterfall SDLC prior to their recent Agile adoption project. 3 of the members are aligned to Agile development teams while the other 2 are functioning in an extended fashion.

Sue works on a smaller team composed of both in-house and 3rd party development and QA team members, giving a total of 8 developers and 4 QA (plus a business analyst acting as QA). The QA team is able to conduct both front- and back-end tests for a Web-based product that serves a mix of commercial and government customers. The team members are dedicated to projects and cross-trained to back each other up.

Both managers are passionate, long term QA managers. They have worked with a variety of companies and projects and are both hands-on managers. (more…)

Merging Business Process With Continuous Delivery

November 20th, 2015 by Ashley Hunsberger

One of the biggest hurdles in getting to Continuous Delivery and truly being Agile does not lie within the development team itself. Change requires a mindset that all people (managers and executives too) must adopt. Just as it takes a village to raise a child, it also takes a (corporate) village to raise a new culture. The movement away from Waterfall to Agile and Continuous cannot be handled just by one person.

Does the following situation sound familiar? Someone asks you for an estimate for the Level of Effort (LOE) of a feature. Not just a t-shirt size, but in days — something to be delivered sometime that year. You don’t have much time to really dive into it (it could even be during the same meeting that you first hear about the new feature), but you have to know all the details up front. So you give a number that you aren’t supposed to be held to because of all sorts of caveats you listed. But, someone remembers that number. And since someone said you are Agile (and Continuous Delivery), now you have to not only meet that estimate you gave up front, you are now tied to an Agile sprint with no room for error. This goes against everything you say you are!


Developing Consistency

November 16th, 2015 by Tom Overton

“Well, it works fine on my machine.”

That’s a phrase I loathe hearing when engineers are doing code review and an issue is discovered. Left to their own devices, engineers will approach setup of their development environments in whatever manner each one deems most efficient. Very frequently this creates two critical issues, a lack of consistency between the environments of the developers, and by consequence, disparities between development and production. These issues are fundamental. Quality is compromised when the application or the environments behave differently for each engineer. This is further complicated when there is a QA team. How were their environments built? How can they be confident they have written the appropriate test cases before release to production?

You want none of this…

Image source:,,

Image source:,,

The obvious benefit of ensuring consistent environments is improved code quality, but another great benefit is reduced finger-pointing and improved team morale. When setting up a team’s development environment, the first consideration to make is where to locate it. Should it be local or remote?


The Importance of Mobile Testing

November 12th, 2015 by Greg Sypolt

It is a digital world, and all Generations have embraced the mobile revolution. It’s no surprise that millennials are leading the way. I saw a statistic recently that 85% of Millennials owns a smartphone. Next time you are waiting in line, take a look around.  Statistics don’t lie.



The mobile revolution has only just begun. The importance of having a mobile testing strategy is critical to our app releases.

What makes a great mobile app?

There are multiple elements that can influence a great (or bad) mobile app. The team works so hard developing an app (from idea, planning, writing code, and testing), the last thing you want to hear is negative feedback instantly after releasing the app. As long as the focus is on simplicity, understanding your users (audience), and the performance of your app, you are heading in the right direction. (more…)

Using Metrics to Discover Testing Holes

November 10th, 2015 by Joe Nolan

How do you know if you are wasting your time? What tests are effective and which ones are done because “That’s the way we do things”?

Tracking bug metrics can bring testing holes to light.

You know that feature you worked on, you were so proud of it. It was all fresh and shiny like a red Porsche:

Image source:

Image source:

And when it finally hits the app store, your ratings end up looking like this:

Image Source:

Image Source:

You ask yourself: How did it happen? Why are the customers finding so many bugs? What are we missing?

It sounds like you’ve got some holes in your testing!

The Proof is in the Pudding (or Data)


Evangelizing Quality

November 6th, 2015 by Ashley Hunsberger

When I hear the word evangelist, all I can think about are those people on TV, preaching away. Not necessarily my cup of tea. As defined, an evangelist is someone who seeks to convert others to what he or she believes, especially by public preaching. I’ve never considered myself an evangelist — it goes against my general keep-to-myself nature. (Public speaking. Blech.) But I want to share what I know, and what I’ve learned, and I hope that others see the value in it. So, in the Church of Quality, I guess that makes me an evangelist. And we need more.

How do you get people to think more about quality?

It usually starts with a bad experience. Whether a customer issue came in, or you just had a very bad day testing, something negative tends to trigger a look at the quality of not just the product, but also the processes that broke. You need people to understand WHY quality is important, and why they should care. (more…)

Try Making It Smaller – It’s The Agile Way

November 4th, 2015 by Greg Sypolt

Agile is a must for development shops. Agile is a mature, iterative, collaborative methodology that breaks the development process down into shorter sprints. At its core, Agile development is about small iterations, test automation and a continuous integration pipeline.

Waterfall Was Created For a Perfect World, But We Don’t Live In One

Agile is a reaction to the slower, sequential approach known as Waterfall. Where Waterfall requires upfront planning to ensure that all details are accounted for, with no room for surprises or changes, Agile accounts for the inevitability of change, adapting to the project as it unfolds.

“Imagine a waterfall on the cliff of a steep mountain. Once the water has flowed over the edge of the cliff and has begun its journey down the side of the mountain, it cannot turn back. It is the same with waterfall development.”  (Search Software Quality)

To understand the advantages of Agile, it’s important to first understand the more traditional Waterfall methodology:

  • It is a sequential design process: discover, plan, build, test, review
  • Each project is based on the extensive collection of clear documentation gathered at the beginning
  • The whole product is only tested at the end of the cycle
  • It doesn’t take into account a client’s evolving needs or leave any room for revision (more…)

Boo! Your Scariest Testing Nightmares

October 29th, 2015 by The Sauce Labs Team

In the spirit of Halloween, this past month testing experts from around the globe answered the call to join in a “crowd-sourced” article highlighting their most frightening testing experience. These are real-life examples of testing gone wrong—leading to all sorts of mayhem in some cases. We allowed these testers to hide their identity, and to keep their organizations anonymous, to solicit the most behind-the-scenes responses.

In no particular order here are the top five Testing Nightmares, as told by the testers themselves:

The Importance Of Making Sure Your Hosting Company Gets Paid
By Ivar (from a medium sized company/ now acquired)

The company I was working for at the time had bet its existence on signing an enterprise contract that it had already been in negotiations for more than year with. We were building a digital asset management system that controlled the licensing and delivery of assets to corporate devices. (Think ‘private app store’.) I had written backup tooling that wasn’t ready for use but was included, unused, in the branch that was being deployed. Ten minutes after the final deployment (in preparation for client review!) *all* of our assets disappeared. Nothing new in the deployment was related to asset storage except my code. There was an eight-figure contract on the line and we had no backups. Fueled by the mother of adrenaline rushes, I reviewed my work and remained convinced that my code was not the cause of the loss. I was already into investigating other possible causes when another developer informed our team that after calling our asset host, he discovered that we had neglected to update our billing credit card. The host had turned off access to all of our assets ‘to get our attention’. They succeeded, but at oh what a price… (more…)

Team Capacity Planning

October 28th, 2015 by Joe Nolan

“Can your QA team handle this?”

When I hear this question, the gears in my head begin to spin. What day is it? Who is available? What is on our plate? Do I expect my team to call in sick after the upcoming team happy hour tomorrow?

Whether you are doing sprint planning or an impromptu project, if you lead a team, then you have to be comfortable with Capacity Planning!

Level of Effort (LOE) is Not Duration or Capacity

Let’s say you are reviewing Task 1 with your analyst:

Lead: How long will this task take you to do?
Analyst: Two days.

Many people stop here. They have successfully obtained the Level of Effort (LOE) which they can plug into their plan. But the conversation should continue:

Lead: Are there any dependencies to complete this task?
Analyst: Yes. When I am halfway done I will need the other analyst to complete Task 2 before I can proceed.
Lead: I see she will be taking about a day to complete Task 2. Given that, it looks like it will be 3 days to complete the work from start to finish.

Now you have the Duration. The remaining question is: Do you have the Capacity? (more…)

What Does Quality Strategy Mean?

October 26th, 2015 by Greg Sypolt

To consistently deliver quality software that meets clients’ expectations, your organization should consider defining a quality strategy. What does quality strategy mean? It’s a set of quality activities that will be carried out with every sprint and release cycle, and everyone approves. An effective quality strategy blueprint includes all facets of testing (automation, manual, reviews, and acceptance criteria to name a few), which help reduce risk and tighten release cycles.
Invest in Great User Stories

Start right, end right! Everyone needs to recognize one of the most important elements of building better software with quality — great user stories. For those not doing Agile and who may not know what a user story is, it represents a small slice of a requirement which benefits the user or customer that can be delivered in an iteration.

Well-formed user stories will meet the criteria of Bill Wake’s INVEST acronym: (more…)