Will we ever truly be at 100% automation? I hope not. Of course automation is critical in implementing Continuous Integration and Delivery, but there are just some things that you can’t leave to a machine. Human evaluation is important.
In a world where we are looking to release faster and faster, why would we want manual testing? Let’s take a look at some of the things you may want to do that automation can’t, and how manual evaluation helps us deliver the right product.
The human aspect
Several years ago, our UX team kept asking, “Is it delightful?” I’ve worked on many features that I truly felt would make for a better experience in education. There are several that, frankly, I just couldn’t stand. We just weren’t building the right product sometimes; even if all tests passed, and there were no bugs — if I didn’t like using it, I found myself asking, “How would users feel?” I have to say, I’m fascinated by the human factor and evoking feelings (for better or worse) when testing software.
As a consumer of software, sometimes I find myself thinking, Wow, did ANYONE look at this? (For example, I’m on the Board of Directors for my Home Owner’s Association, and the software we use to track documents, get assessments, and so forth just makes me want to cry).
I’ll be honest – sometimes it is difficult to see how usable something is until there is something to use it for. Hopefully, though, we can spot this early as we define acceptance criteria and are evaluating the workflows, specs, wireframes, or prototypes.
Be like Lewis and Clark
The obvious manual testing activity that should be at the top of everyone’s list is exploratory testing. In an ideal world, the features themselves are completely automated, and development is done when all tests pass. This is fantastic, but what if that were it? Lewis and Clark had specific goals, and reported what they found along the way. Exploratory testing is similar to me: I start with a charter (or a goal) from a user perspective, and report what I find along the way. Perhaps it is a feature that works fine in unit and integration tests, but once I start looking at it myself in an end to end workflow, I think of other scenarios that perhaps we didn’t think of when writing scripts. (more…)