Two Approaches to Test Automation Architectures

May 23rd, 2016 by Chris Riley

I’ve yet to see two development environments that are alike. But even if there is no cookie cutter approach to software delivery, there are standard approaches, and methodologies that are consistent throughout modern software development and that frame nearly all environments.

Because there is a big move in software testing to go from purely manual testing (a non-technical process) to a fully automated deeply technical one, how QA processes are set up, and how it fits into the overall delivery chain is very important. Let’s take a look at the two most common architectures for test automation, and why they may or may not be the best approach. Read the rest of this entry »

Security and Testing

May 19th, 2016 by Michael Churchman

Is your test environment secure? Do you know who has access to your test data, your source code, your design specifications?

There was a time, back in the days of stand-alone test systems and networks that were strictly local-area, when those questions would have been easy to answer. A co-worker or two might have been looking over your shoulder, but that would have been about it.

These days, however, applications are exposed to the public web, and such questions can have serious implications for your software’s security and your company’s bottom line. Software and IT companies may still have physical locations, but much of the development and testing is done off-site, by employees, contractors, and services that transfer data over the Internet, such as cloud-based testing solutions. That is why picking one who cares about your application security is important. Lets look at the risks, then at what a good solution looks like. Read the rest of this entry »

Testing and Continuous Integration: Making Everything Work Together

May 17th, 2016 by Hemant Jain

Continuous integration (CI) has emerged as one of the most efficient ways to develop code. But testing has not always been a major part of the CI conversation.

In some respects, that’s not surprising. Traditionally, CI has been all about speeding up the coding, building, and release process. Instead of having each programmer write code separately, integrate it manually, and then wait until the next daily or weekly build to see if the changes broke anything, CI lets developers code and compile on a virtually continuous basis. It also means developers and admins can work together seamlessly, since the programming and build processes are always in sync.

Unfortunately, the quality assurance team does not necessarily reap the same benefits. While CI assures that your app keeps building successfully as the code is continually updated, it doesn’t automatically test how new builds behave within different types of environments. An otherwise well-run CI operation might require app testing to be done separately, on a non-continuous basis, instead of building it into the rest of the process.

This poses real problems for an organization. Unless you add automated testing to your CI mix, you could end up with an app that users can download and install properly, but which suffers from critical usability issues in certain browsers or operating systems. Arguably, an app that installs successfully but frustrates users due to lack of testing is worse than one that doesn’t install at all. Read the rest of this entry »

Recap: Transitioning from Traditional to Modern Testing Practices (Webinar)

May 13th, 2016 by Bill McGee

Thanks to everyone who joined us for our recent webinar, “Practical Tips on How to Transition from Traditional to Modern Testing Practices”, featuring DevOps analyst Chris Riley and test engineering experts Ashley Hunsberger and Greg Sypolt.

Chris, Ashley and Greg discussed the benefits of automated testing — accelerated delivery, improved quality, and reduced costs — and the transformation required of all organizations moving from manual testing to automated testing. They also addressed the following questions:

  • How do you build an automation strategy from the bottom up?
  • How do you begin to introduce automated testing to your organization?
  • What do you do when things go wrong?
  • How do you measure success?
  • What are the types of tools you should have in your tool chain?

Want to learn more about automated testing and Continuous Integration? Download our free report, “How to Get the Most out of Your CI/CD Workflow Using Automated Testing”. 

Access the recording HERE and view the slides below: 

Capture Network Traffic with Automation Scripts

May 12th, 2016 by Greg Sypolt

When learning about the ability to capture network traffic by using my existing Selenium scripts or the headless test framework – PhantomJS scripts, I was excited. A whole new set of tests is about to be added to the continuous integration (CI) pipeline. We often come across requirements when we need to capture and analyze browser network traffic in real time to find HTTP status of the page, examine the headers, validate parameters, do performance analysis, and more. Just another testing strategy to protect the end-user experience when they are using your web application in real time.

What is network traffic?

It includes all the network requests the browser makes when retrieving and loading the Javascript, CSS, image files, and more for a single web page. In lay terms, it is the communication between the browser and server when it is loading a web page.

Why do we need to inspect web application network traffic?

The inspection of network traffic paints a picture of a web page’s condition. The painting starts by going to a web page which triggers all the HTTP requests and responses that need to be collected in real time. To finish, analyze and measure activity across all identified pages of your application. Read the rest of this entry »

Help Wanted – The Pivotal Role QA Can Play in Leading the DevOps Charge

May 9th, 2016 by Chris Riley

Faster, more frequent releases at a higher quality. That is all DevOps is. That’s not hard to understand. What is hard to accept, however, is how much organizations are neglecting the latter part of this equation. Not only does a lack of focus on quality slow down releases in the long term, it does not fit with the overall goal of DevOps.

DevOps for existing development organizations is hard to implement. Entrenched development shops not only do not have the option to stop everything and start over, they are also slipstreaming new processes and technologies into an existing process, and an already-established delivery pipeline. I have seen many organizations for which the transition has worked out great, and other times where it has fallen flat. But I’ve very rarely seen an environment where QA drives change. Read the rest of this entry »

JIRA is Just a Click Away with Our New Plugin

May 3rd, 2016 by Ken Drachnik

As more and more devs work in agile teams, they need tools to plan, track, and release software and many of them use JIRA. As a tracking tool, JIRA is amazing for collaboration and project planning. For many teams, JIRA is the place of record for everything in the software development lifecycle. We have found that many of our customers use JIRA and the #1 product ask was to integrate Sauce Labs’ test results with JIRA so it would be simple to track all the tests associated with a project in one place.

Let’s say you are running an automated or manual test on Sauce Labs and find a bug. You want to add it to JIRA so that someone on your team can take a look or so that it can be prioritized in the backlog. Historically, one would have to download all the Sauce assets, login to JIRA, create a ticket, and upload the assets again. This can be tedious when you’re running lots of tests.

With Sauce Labs for JIRA, this is all simplified and automated. With the click of a button you can now create a JIRA ticket directly from your Test Details page. The plugin gives you the option to upload the screenshots, logs, and video link to your tests, making it easy to share out among your team.

To download Sauce Labs for JIRA, visit the Atlassian Marketplace: https://marketplace.atlassian.com/plugins/sauce-jira-integration/cloud/overview.  To read more, visit our JIRA integration Docs page.

Happy Testing!

The Sauce Labs Ecosystems & Integrations Team

Sauce Labs Named a Contender Among Mobile Test Automation Tools

April 28th, 2016 by Bill McGee

We are happy to announce a new industry citation for Sauce Labs: our inclusion in “The Forrester Wave™: Mobile Front-End Test Automation Tools, Q2, 2016.” The Wave is an influential report by leading research firm Forrester Research, and Sauce Labs was one of 10 companies selected for inclusion.

This Wave is based on an evaluation of 40 criteria across 10 vendors that considers how each provider measures up to help application development and delivery. Sauce Labs was among vendors cited as a “Contender” in the report. On describing Sauce, the report states:

Sauce Labs is the open source champion, offering comprehensive target app types … Sauce Labs successfully embraces the bring-your-own-tools (BYOT) and pick-your-favorite-language approaches to mobile app testing, which should make most developers happy. Focusing on automation and CI, it offers a robust, cloud-only testing solution that includes virtual machines for testing web and mobile applications as well as real devices. It satisfies security-conscious environments through a secure connection back into the data center using Sauce Connect.

More than anything, we are beyond excited that our Real Device Cloud (RDC) was included in such a significant report. Especially as we know Forrester went through an exhaustive evaluation of Sauce Labs that looked at factors including product fit, customer success, and Forrester client demand.

We believe this report provides Sauce Labs with huge visibility to potential new users, as many North America Fortune 1000 companies are Forrester clients. We are excited to be a part of the growing Mobile testing market, we are encouraged by the recognition of our strategy and roadmap, and we are thrilled with the momentum the Sauce Labs Real Device Cloud has gained in a short amount of time.

Want to learn more about automated mobile testing? Download our free report, “Automated Mobile Testing Requires Both Real Devices and Emulators“. 

Why is Manual QA Still So Prevalent?

April 28th, 2016 by Joe Nolan

This past week I casually heard comments alluding to the imminent death of the QA Analyst or Manual Tester. (To be clear, I am not referring to the QA Automation Engineer, who builds test automation.)

Not only does the function not seem to be going away, recruiters are still out their hunting testers down. Out of curiosity I did my own review of randomly selected job posts from Monster and Indeed for average QA positions, and discovered that there are still a lot of jobs available for manual testers.

With the importance of catching bugs early, and the ability to automate all testing, why do companies and projects resist the investment in CI and test automation? I will explore the reasons why now, and whether the resistance is good or bad. Read the rest of this entry »

Appium 1.5.2 Released on Sauce

April 26th, 2016 by Isaac Murchie

We are pleased to announce the release of Appium 1.5.2 through npm and on Sauce Labs! This is a bug fix release that deals with many issues with 1.5.1, including a major bug where large Android applications would cause Appium to run out of memory and die.

General

  • deprecated --command-timeout. Use newCommandTimeout desired capability instead
  • ensure implicit wait can be set through timeout method
  • add better logging for EPIPE errors

iOS

  • make sure ipa files are handled correctly for installing on real devices
  • ensure that existing SafariLauncher on device is used instead of rebuilding and reinstalling
  • fix issues with getting webview contexts on real devices
  • add full timeout support through timeout method
  • make sure Xpath searches respect implicit wait timeout
  • make sure bare Instruments process arguments are accepted

Android

  • fix failure when apk file is too large
  • re-implement setting geolocation so it does not use Telnet
  • add support for Chromium browser
  • fix issues with flick
  • fix bug where touch action release would throw an error
  • fix bug in later Android SDK version where noticing a newly started avd would fail
  • implement autoWebviewTimeout