Q & A : Design Patterns for Scalable Test Automation

June 24th, 2016 by The Sauce Labs Team

Thanks to everyone who joined the webinar given by Sahas Subramanian, “Design Patterns for Scalable Test Automation with Selenium and WebdriverIO”. There were a number of great questions that were posed prior to and during the session, and we asked Sahas to consolidate some of these questions and answer them in this follow-up post. Disclaimer: opinions shared below are Sahas’ and not those of his employer or Sauce Labs.

If you missed the webinar, you can find the video, slides and link to a related blog post here. Should you have any additional questions, send a tweet to @Sahaswaranamam.

Q: How can you best handle security authentication pop-ups from specific browsers? What are the best ways to switch between tabs and to close tabs?

A: Use the getCurrentTabId API to get the handle of the current window. Once you have the pop-up window handle, you could close it using browser.close(popUpHandle)

Reference: http://webdriver.io/api/window/getCurrentTabId.html

Q: How should I handle SOAP/SOAPUI testing?

A: Generally speaking, Selenium and Webdriver are appropriate for UI testing. If your intention is to test the APIs, I would suggest using tools like JMeter and/or Taurus. Reference: http://gettaurus.org

Q: How do I create my own wrapper? (How can I check for page title?)

A: Check out http://webdriver.io/api/protocol/title.html (more…)

Catching Bugs Too Late

June 23rd, 2016 by Ashley Hunsberger

Putting quality first is critical. Teams must take ownership of quality, but to do so they have to create an environment that allows them to build quality in, instead of testing it out much further down the road to delivery. Finding bugs late is too costly if you aren’t yet to the point of being able to prevent them (implementing BDD). Ensure you can find them early.

Staying green is hard work!

I’ve seen many things change this year. My daughter began kindergarten. (How did THAT happen so fast?!) I also began blogging, and our department is trying to shift from Waterfall to Agile and Continuous Delivery, with teams shifting to own quality instead of tossing code over to QA… all great changes. But one thing has remained the same. We were still finding bugs late.

I’ve written many times about the importance of quality first. But how did our team take action on that? First, we HAD to have automation. Purely manual testing was just not going to cut it anymore. Don’t get me wrong, I still very much value human-based testing. But frankly, it can catch things too late. So, enter our automated tests. We began with what we called our pre-commit tests. These must be run — you guessed it — before you commit code! Yes, they are slower than unit or integration tests. But they take around 7-8 minutes (allowing time to go grab some coffee, stretch, whatever). They are our most critical features and workflows. Aside from running locally before committing, they are also scheduled and running many times over during the day with all the commits going on. Once we established that set of tests, we began our work on more user acceptance tests – still hero workflows, but trying to keep in mind the fine line between useful UI tests and too many tests (think of the testing pyramid). (more…)

Patterns and Coding Practices for Stable End-to-End GUI Tests

June 16th, 2016 by Sahas Subramanian

We all know the importance of the Test Automation Pyramid and why it makes sense to align various automations in this way. Given that guiding principle, end-to-end GUI tests sit at the top, with a considerably small number of tests compared to other types (Unit, Integration, Service tests), and they are useful to verify business workflows. In the book Agile Testing: A Practical Guide for Testers and Agile Teams, the authors explain the testing quadrants, the GUI tests’ fit in the grand scheme of things, how to rationalize intention, and be smart about overall Quality strategy.

sahas_fig1

http://www.exampler.com/old-blog/2003/08/21/#agile-testing-project-1

The intention of the E-E GUI tests is to verify “whether we build the right thing from the business perspective.”

(more…)

Accelerate Multi-browser Testing Using Sauce Labs and Webdriver.io

June 7th, 2016 by Sahas Subramanian

There are a lot of webdriver-based testing frameworks out there. Webdriver.io is a relatively new cool kid on the block. It has enough to differentiate itself and helps us to focus on creating reliable GUI tests. A few highlights on what this framework comes with:

  1. Out-of-the-box ES 2015 support – leverage all the goodness that ES6/ES2015 offers.
  2. Out-of-the-box promises support – This framework uses Q.js internally, and every single command represents a promise. The second command will be executed only after the first command promise is resolved.
  3. Out-of-the-box Sauce Labs runner service – to run tests on Sauce Labs
  4. Out-of-the-box parallel test runner (wdio) – to run tests across multiple OS/Browser mixes in parallel
  5. A rich set of APIs makes the test code short and concise

 

These advantages help organizations focus on their core responsibility — to automate their verification process quickly instead of developing and maintaining custom libraries to support automation efforts (more productivity — we all love it).

(more…)

Two Approaches to Test Automation Architectures

May 23rd, 2016 by Chris Riley

I’ve yet to see two development environments that are alike. But even if there is no cookie cutter approach to software delivery, there are standard approaches, and methodologies that are consistent throughout modern software development and that frame nearly all environments.

Because there is a big move in software testing to go from purely manual testing (a non-technical process) to a fully automated deeply technical one, how QA processes are set up, and how it fits into the overall delivery chain is very important. Let’s take a look at the two most common architectures for test automation, and why they may or may not be the best approach. (more…)

Security and Testing

May 19th, 2016 by Michael Churchman

Is your test environment secure? Do you know who has access to your test data, your source code, your design specifications?

There was a time, back in the days of stand-alone test systems and networks that were strictly local-area, when those questions would have been easy to answer. A co-worker or two might have been looking over your shoulder, but that would have been about it.

These days, however, applications are exposed to the public web, and such questions can have serious implications for your software’s security and your company’s bottom line. Software and IT companies may still have physical locations, but much of the development and testing is done off-site, by employees, contractors, and services that transfer data over the Internet, such as cloud-based testing solutions. That is why picking one who cares about your application security is important. Lets look at the risks, then at what a good solution looks like. (more…)

Testing and Continuous Integration: Making Everything Work Together

May 17th, 2016 by Hemant Jain

Continuous integration (CI) has emerged as one of the most efficient ways to develop code. But testing has not always been a major part of the CI conversation.

In some respects, that’s not surprising. Traditionally, CI has been all about speeding up the coding, building, and release process. Instead of having each programmer write code separately, integrate it manually, and then wait until the next daily or weekly build to see if the changes broke anything, CI lets developers code and compile on a virtually continuous basis. It also means developers and admins can work together seamlessly, since the programming and build processes are always in sync.

Unfortunately, the quality assurance team does not necessarily reap the same benefits. While CI assures that your app keeps building successfully as the code is continually updated, it doesn’t automatically test how new builds behave within different types of environments. An otherwise well-run CI operation might require app testing to be done separately, on a non-continuous basis, instead of building it into the rest of the process.

This poses real problems for an organization. Unless you add automated testing to your CI mix, you could end up with an app that users can download and install properly, but which suffers from critical usability issues in certain browsers or operating systems. Arguably, an app that installs successfully but frustrates users due to lack of testing is worse than one that doesn’t install at all. (more…)

Capture Network Traffic with Automation Scripts

May 12th, 2016 by Greg Sypolt

When learning about the ability to capture network traffic by using my existing Selenium scripts or the headless test framework – PhantomJS scripts, I was excited. A whole new set of tests is about to be added to the continuous integration (CI) pipeline. We often come across requirements when we need to capture and analyze browser network traffic in real time to find HTTP status of the page, examine the headers, validate parameters, do performance analysis, and more. Just another testing strategy to protect the end-user experience when they are using your web application in real time.

What is network traffic?

It includes all the network requests the browser makes when retrieving and loading the Javascript, CSS, image files, and more for a single web page. In lay terms, it is the communication between the browser and server when it is loading a web page.

Why do we need to inspect web application network traffic?

The inspection of network traffic paints a picture of a web page’s condition. The painting starts by going to a web page which triggers all the HTTP requests and responses that need to be collected in real time. To finish, analyze and measure activity across all identified pages of your application. (more…)

Help Wanted – The Pivotal Role QA Can Play in Leading the DevOps Charge

May 9th, 2016 by Chris Riley

Faster, more frequent releases at a higher quality. That is all DevOps is. That’s not hard to understand. What is hard to accept, however, is how much organizations are neglecting the latter part of this equation. Not only does a lack of focus on quality slow down releases in the long term, it does not fit with the overall goal of DevOps.

DevOps for existing development organizations is hard to implement. Entrenched development shops not only do not have the option to stop everything and start over, they are also slipstreaming new processes and technologies into an existing process, and an already-established delivery pipeline. I have seen many organizations for which the transition has worked out great, and other times where it has fallen flat. But I’ve very rarely seen an environment where QA drives change. (more…)

Why is Manual QA Still So Prevalent?

April 28th, 2016 by Joe Nolan

This past week I casually heard comments alluding to the imminent death of the QA Analyst or Manual Tester. (To be clear, I am not referring to the QA Automation Engineer, who builds test automation.)

Not only does the function not seem to be going away, recruiters are still out their hunting testers down. Out of curiosity I did my own review of randomly selected job posts from Monster and Indeed for average QA positions, and discovered that there are still a lot of jobs available for manual testers.

With the importance of catching bugs early, and the ability to automate all testing, why do companies and projects resist the investment in CI and test automation? I will explore the reasons why now, and whether the resistance is good or bad. (more…)