Patterns and Coding Practices for Stable End-to-End GUI Tests

June 16th, 2016 by Sahas Subramanian

We all know the importance of the Test Automation Pyramid and why it makes sense to align various automations in this way. Given that guiding principle, end-to-end GUI tests sit at the top, with a considerably small number of tests compared to other types (Unit, Integration, Service tests), and they are useful to verify business workflows. In the book Agile Testing: A Practical Guide for Testers and Agile Teams, the authors explain the testing quadrants, the GUI tests’ fit in the grand scheme of things, how to rationalize intention, and be smart about overall Quality strategy.

sahas_fig1

http://www.exampler.com/old-blog/2003/08/21/#agile-testing-project-1

The intention of the E-E GUI tests is to verify “whether we build the right thing from the business perspective.”

(more…)

Accelerate Multi-browser Testing Using Sauce Labs and Webdriver.io

June 7th, 2016 by Sahas Subramanian

There are a lot of webdriver-based testing frameworks out there. Webdriver.io is a relatively new cool kid on the block. It has enough to differentiate itself and helps us to focus on creating reliable GUI tests. A few highlights on what this framework comes with:

  1. Out-of-the-box ES 2015 support – leverage all the goodness that ES6/ES2015 offers.
  2. Out-of-the-box promises support – This framework uses Q.js internally, and every single command represents a promise. The second command will be executed only after the first command promise is resolved.
  3. Out-of-the-box Sauce Labs runner service – to run tests on Sauce Labs
  4. Out-of-the-box parallel test runner (wdio) – to run tests across multiple OS/Browser mixes in parallel
  5. A rich set of APIs makes the test code short and concise

 

These advantages help organizations focus on their core responsibility — to automate their verification process quickly instead of developing and maintaining custom libraries to support automation efforts (more productivity — we all love it).

(more…)

Two Approaches to Test Automation Architectures

May 23rd, 2016 by Chris Riley

I’ve yet to see two development environments that are alike. But even if there is no cookie cutter approach to software delivery, there are standard approaches, and methodologies that are consistent throughout modern software development and that frame nearly all environments.

Because there is a big move in software testing to go from purely manual testing (a non-technical process) to a fully automated deeply technical one, how QA processes are set up, and how it fits into the overall delivery chain is very important. Let’s take a look at the two most common architectures for test automation, and why they may or may not be the best approach. (more…)

Recap: Transitioning from Traditional to Modern Testing Practices (Webinar)

May 13th, 2016 by Bill McGee

Thanks to everyone who joined us for our recent webinar, “Practical Tips on How to Transition from Traditional to Modern Testing Practices”, featuring DevOps analyst Chris Riley and test engineering experts Ashley Hunsberger and Greg Sypolt.

Chris, Ashley and Greg discussed the benefits of automated testing — accelerated delivery, improved quality, and reduced costs — and the transformation required of all organizations moving from manual testing to automated testing. They also addressed the following questions:

  • How do you build an automation strategy from the bottom up?
  • How do you begin to introduce automated testing to your organization?
  • What do you do when things go wrong?
  • How do you measure success?
  • What are the types of tools you should have in your tool chain?

Want to learn more about automated testing and Continuous Integration? Download our free report, “How to Get the Most out of Your CI/CD Workflow Using Automated Testing”. 

Access the recording HERE and view the slides below: 

Capture Network Traffic with Automation Scripts

May 12th, 2016 by Greg Sypolt

When learning about the ability to capture network traffic by using my existing Selenium scripts or the headless test framework – PhantomJS scripts, I was excited. A whole new set of tests is about to be added to the continuous integration (CI) pipeline. We often come across requirements when we need to capture and analyze browser network traffic in real time to find HTTP status of the page, examine the headers, validate parameters, do performance analysis, and more. Just another testing strategy to protect the end-user experience when they are using your web application in real time.

What is network traffic?

It includes all the network requests the browser makes when retrieving and loading the Javascript, CSS, image files, and more for a single web page. In lay terms, it is the communication between the browser and server when it is loading a web page.

Why do we need to inspect web application network traffic?

The inspection of network traffic paints a picture of a web page’s condition. The painting starts by going to a web page which triggers all the HTTP requests and responses that need to be collected in real time. To finish, analyze and measure activity across all identified pages of your application. (more…)

Help Wanted – The Pivotal Role QA Can Play in Leading the DevOps Charge

May 9th, 2016 by Chris Riley

Faster, more frequent releases at a higher quality. That is all DevOps is. That’s not hard to understand. What is hard to accept, however, is how much organizations are neglecting the latter part of this equation. Not only does a lack of focus on quality slow down releases in the long term, it does not fit with the overall goal of DevOps.

DevOps for existing development organizations is hard to implement. Entrenched development shops not only do not have the option to stop everything and start over, they are also slipstreaming new processes and technologies into an existing process, and an already-established delivery pipeline. I have seen many organizations for which the transition has worked out great, and other times where it has fallen flat. But I’ve very rarely seen an environment where QA drives change. (more…)

Recap: Easy Continuous Deployment You Can Trust (Webinar)

April 14th, 2016 by Bill McGee

Thanks to everyone who joined us for our recent webinar, “Easy Continuous Deployment You Can Trust”, featuring Solano Labs Founding Engineer, Brian Kendzior, and Sauce Labs Solutions Architect, Neil Manvar.

In their presentation, Brian and Neil demonstrated a continuous deployment release process that used Solano CI, Sauce Labs, and AWS CodePipeline. The release process included smoke, unit, integration and browser tests to guarantee issue-free deployments.

Brian and Neil also demonstrated how to:

  • Build your software release pipeline by connecting different steps into an automated workflow using AWS CodePipeline
  • Run your automated web tests on any browser and operating system combination using Sauce Labs
  • Auto-parallelize your test runs with a Continuous Integration server using Solano CI

Want to learn more about automated testing and Continuous Integration? Download our free report, “How to Get the Most out of Your CI/CD Workflow Using Automated Testing”. 

Access the recording HERE and view the slides below: 

Get Started with Protractor Testing for AngularJS

April 6th, 2016 by Greg Sypolt

How do you test your AngularJS applications? With Protractor. Protractor is an end-to-end testing framework for AngularJS applications. This getting started guide is for new software developers in test who are interested in learning about Protractor testing. By following this getting started guide, you’ll understand how to build a firm foundation and learn fundamental Protractor testing for AngularJS applications.

Build a solid foundation

To build better software tests, you’ll need a solid base of the technologies behind your application, which teaches you the principles that will be essential when coding, executing and debugging your tests. Let’s focus on the correct learning path for Protractor testing. This is the age of JavaScript (JS) applications and frameworks. JavaScript is the foundation and critical learning block needed to be successful with Protractor testing, and what follows is your roadmap to learn JavaScript, NodeJS, and AngularJS for Protractor testing. (more…)

test.allTheThings() – Announcing General Availability of Real Device Cloud

March 31st, 2016 by Ken Drachnik

After months of beta testing with customers, we are announcing general availability for our Real Device Cloud today.  With the ability to test websites, native apps, and hybrid apps on both iOS and Android devices, we enable enterprise customers to easily scale their CI/CD testing needs on the most popular devices. By providing large numbers of each device type, our customers will not have to wait in a queue for their tests to run on a real device, thus speeding up the pace of testing, especially when running concurrent parallelized tests.  In conjunction with our mobile emulators and simulators, we now provide the most comprehensive automated mobile testing platform in the market, covering both web and mobile app testing needs across a variety of device types. After running your tests, view the results on our test details pages, with videos of each test, screen shots, logs and commands so you can resolve issues quickly and iterate on your app more frequently.  

To request a quote for real devices, contact your account executive.

Mobile Testing on Sauce Labs Includes:

  • Instant Availability – Get access to the most popular iOS or Android devices with no waiting, no queues and no reservations system.
  • Android emulators and iOS simulators covering over 140 device-OS combinations.
  • Massive Concurrency – Run your tests in parallel to dramatically reduce your total test time.
  • Integrate with your CI tool of choice – automate all your tests using the top CI tools like Jenkins, Bamboo, Microsoft VSTS, Travis, Team City or Circle CI.
  • Test native apps, hybrid apps, and mobile web –  all on the same platform.
  • Security – test with Sauce Connect, allowing your codebase to stay behind your firewall while utilizing Sauce’s extensive collection of OS platforms, browsers, and devices.
  • Pinpoint issues quickly with full reporting – instant access to videos, screenshots and all the data for your tests so you can analyse your results quickly.
  • Enterprise management features – account and team management lets you provision test resources as needed and SSO integration means you don’t have to go to IT to add another tester to your account.
  • Professional Services and Training – we have professional consultants and partners to help you get started with Appium and Selenium or if you’re already proficient, our experts can help your team become super users.

 

For more information, visit our Automated Mobile Testing Page

Fragile Automation

March 28th, 2016 by Ashley Hunsberger

User Interface (UI) Testing.

The idea is simple — automate some UI tests to ensure your application is still behaving as expected. Usually your first set of tests — running green, no doubt — make you all cheer and pat yourselves on the back. Then you open up the framework to more people. Despite the reviews (so many reviews), the failures start to come, and they don’t stop. Or they run green and then fail and then run green again. And then fail again. So why are they so unstable? Is it bad scripts? Environment issues? Sometimes you just don’t know, and you think you are going to lose your mind. Let’s take a look at some common and potential issues you may be facing.

Architecture, Environments and Settings

Is your infrastructure designed for stability? Are you using on-premise or cloud instances? What may have saved you a dollar upfront could cost you many more down the road – so your testing environment is important.

Understand if your tests require particular system settings. Tests failing because of unwanted server variables is a waste of everyone’s time. We found that out the hard way a long time ago. You may need to have some isolated tests that cannot be run on the same server so the majority of your other tests can pass. (Or maybe decide how important the test really is.)

Or let’s say your testing frameworks are stable, but what about the tools or libraries you are importing? Are you pinning your stack to a version of these tools or libraries? A new version can completely break everything. (more…)