Posts Tagged ‘Quality Assurance’

Using Metrics to Discover Testing Holes

November 10th, 2015 by Joe Nolan

How do you know if you are wasting your time? What tests are effective and which ones are done because “That’s the way we do things”?

Tracking bug metrics can bring testing holes to light.

You know that feature you worked on, you were so proud of it. It was all fresh and shiny like a red Porsche:

Image source:

Image source:

And when it finally hits the app store, your ratings end up looking like this:

Image Source:

Image Source:

You ask yourself: How did it happen? Why are the customers finding so many bugs? What are we missing?

It sounds like you’ve got some holes in your testing!

The Proof is in the Pudding (or Data)


Evangelizing Quality

November 6th, 2015 by Ashley Hunsberger

When I hear the word evangelist, all I can think about are those people on TV, preaching away. Not necessarily my cup of tea. As defined, an evangelist is someone who seeks to convert others to what he or she believes, especially by public preaching. I’ve never considered myself an evangelist — it goes against my general keep-to-myself nature. (Public speaking. Blech.) But I want to share what I know, and what I’ve learned, and I hope that others see the value in it. So, in the Church of Quality, I guess that makes me an evangelist. And we need more.

How do you get people to think more about quality?

It usually starts with a bad experience. Whether a customer issue came in, or you just had a very bad day testing, something negative tends to trigger a look at the quality of not just the product, but also the processes that broke. You need people to understand WHY quality is important, and why they should care. (more…)

Should You Have a Dedicated Automation Team Within Your QA Department?

September 1st, 2015 by Israel Felix

If you’ve led or managed QA teams that have included an automation test team, you’ve probably been in a situation where you had to decide whether you should keep them on board. Normally the decision needs to be made when there is a change in leadership, wherein the new management comes with a mandate to consolidate groups and reduce costs. This situation also tends to arise when working with startups or small companies when they are ready to put together or augment their QA teams. So should you have a dedicated automation team?

Typically, there are two camps with regards to dedicated automation teams. There are those who believe that we should have dedicated automation teams, and those who believe that QA engineers should handle manual testing and automation testing. From my experience working in QA within both small and large companies, I almost always prefer to have a dedicated automation team. However, there are a few scenarios where having a QA team that takes on both roles might make sense.

Time to Market

For automation to be done right, it needs to be a full-time job. From developing the framework and creating the libraries and scripts for different platforms to executing and debugging failures — it will all simply consume too much of an engineer’s time and compromise the actual testing and release date. As you already know, time to market and keeping a release on schedule is top priority, so testing needs to get done, no matter what. (more…)

Why Manual Testing Helps Your Release

August 11th, 2015 by Ashley Hunsberger

Will we ever truly be at 100% automation?  I hope not. Of course automation is critical in implementing Continuous Integration and Delivery, but there are just some things that you can’t leave to a machine. Human evaluation is important.

In a world where we are looking to release faster and faster, why would we want manual testing?  Let’s take a look at some of the things you may want to do that automation can’t, and how manual evaluation helps us deliver the right product.

The human aspect

Several years ago, our UX team kept asking, “Is it delightful?”  I’ve worked on many features that I truly felt would make for a better experience in education.  There are several that, frankly, I just couldn’t stand. We just weren’t building the right product sometimes; even if all tests passed, and there were no bugs — if I didn’t like using it, I found myself asking, “How would users feel?” I have to say, I’m fascinated by the human factor and evoking feelings (for better or worse) when testing software.

As a consumer of software, sometimes I find myself thinking, Wow, did ANYONE look at this? (For example, I’m on the Board of Directors for my Home Owner’s Association, and the software we use to track documents, get assessments, and so forth just makes me want to cry).

I’ll be honest – sometimes it is difficult to see how usable something is until there is something to use it for. Hopefully, though, we can spot this early as we define acceptance criteria and are evaluating the workflows, specs, wireframes, or prototypes.

Be like Lewis and Clark

The obvious manual testing activity that should be at the top of everyone’s list is exploratory testing. In an ideal world, the features themselves are completely automated, and development is done when all tests pass. This is fantastic, but what if that were it?  Lewis and Clark had specific goals, and reported what they found along the way. Exploratory testing is similar to me: I start with a charter (or a goal) from a user perspective, and report what I find along the way. Perhaps it is a feature that works fine in unit and integration tests, but once I start looking at it myself in an end to end workflow, I think of other scenarios that perhaps we didn’t think of when writing scripts. (more…)

Guest Post: Functional Testing in 2016 – Forecast

April 15th, 2015 by Chris Riley

Massive changes in the development world are good and extreme for devs, but quality assurance (QA) teams are impacted as much, if not more. They may be taking on more tasks, looking at new tools, and thinking about new ways to execute their growing test suites. And looking forward, QA in the future looks much different than it does today. It is moving so fast that the changes – both good and bad – will be even more obvious by next year. Here is what QA looks like in 2016. (more…)