Sauce Labs Named a Contender Among Mobile Test Automation Tools

April 28th, 2016 by Bill McGee

We are happy to announce a new industry citation for Sauce Labs: our inclusion in “The Forrester Wave™: Mobile Front-End Test Automation Tools, Q2, 2016.” The Wave is an influential report by leading research firm Forrester Research, and Sauce Labs was one of 10 companies selected for inclusion.

This Wave is based on an evaluation of 40 criteria across 10 vendors that considers how each provider measures up to help application development and delivery. Sauce Labs was among vendors cited as a “Contender” in the report. On describing Sauce, the report states:

Sauce Labs is the open source champion, offering comprehensive target app types … Sauce Labs successfully embraces the bring-your-own-tools (BYOT) and pick-your-favorite-language approaches to mobile app testing, which should make most developers happy. Focusing on automation and CI, it offers a robust, cloud-only testing solution that includes virtual machines for testing web and mobile applications as well as real devices. It satisfies security-conscious environments through a secure connection back into the data center using Sauce Connect.

More than anything, we are beyond excited that our Real Device Cloud (RDC) was included in such a significant report. Especially as we know Forrester went through an exhaustive evaluation of Sauce Labs that looked at factors including product fit, customer success, and Forrester client demand.

We believe this report provides Sauce Labs with huge visibility to potential new users, as many North America Fortune 1000 companies are Forrester clients. We are excited to be a part of the growing Mobile testing market, we are encouraged by the recognition of our strategy and roadmap, and we are thrilled with the momentum the Sauce Labs Real Device Cloud has gained in a short amount of time.

Want to learn more about automated mobile testing? Download our free report, “Automated Mobile Testing Requires Both Real Devices and Emulators“. 

Why is Manual QA Still So Prevalent?

April 28th, 2016 by Joe Nolan

This past week I casually heard comments alluding to the imminent death of the QA Analyst or Manual Tester. (To be clear, I am not referring to the QA Automation Engineer, who builds test automation.)

Not only does the function not seem to be going away, recruiters are still out their hunting testers down. Out of curiosity I did my own review of randomly selected job posts from Monster and Indeed for average QA positions, and discovered that there are still a lot of jobs available for manual testers.

With the importance of catching bugs early, and the ability to automate all testing, why do companies and projects resist the investment in CI and test automation? I will explore the reasons why now, and whether the resistance is good or bad. Read the rest of this entry »

Appium 1.5.2 Released on Sauce

April 26th, 2016 by Isaac Murchie

We are pleased to announce the release of Appium 1.5.2 through npm and on Sauce Labs! This is a bug fix release that deals with many issues with 1.5.1, including a major bug where large Android applications would cause Appium to run out of memory and die.

General

  • deprecated --command-timeout. Use newCommandTimeout desired capability instead
  • ensure implicit wait can be set through timeout method
  • add better logging for EPIPE errors

iOS

  • make sure ipa files are handled correctly for installing on real devices
  • ensure that existing SafariLauncher on device is used instead of rebuilding and reinstalling
  • fix issues with getting webview contexts on real devices
  • add full timeout support through timeout method
  • make sure Xpath searches respect implicit wait timeout
  • make sure bare Instruments process arguments are accepted

Android

  • fix failure when apk file is too large
  • re-implement setting geolocation so it does not use Telnet
  • add support for Chromium browser
  • fix issues with flick
  • fix bug where touch action release would throw an error
  • fix bug in later Android SDK version where noticing a newly started avd would fail
  • implement autoWebviewTimeout

The Sauce Journey – From Star to Scrum

April 22nd, 2016 by Joe Alfaro

When I first started at Sauce, one group within the Engineering organization was called *Dev, referred to in conversation as “StarDev.” *Dev was so named because they were the wildcard development group – some of their work touched on this part of the infrastructure, or this part of the product, or this part of the Web interface, and so on. As a result of this interdisciplinary approach, much of the product and operational development fell under their purview, but it was difficult to tell exactly what it was they were responsible for. What was even more difficult was that when it came time to undertake actual projects, there was no clear delineation of what role *Dev needed to play in their development.

When you see the emergence of a group like *Dev, it’s a clear sign that your engineering organization has reached an evolutionary stage common to startups. That’s the stage where you go from a handful of people sitting in a room together hacking out code, to trying to formalize responsibility for parts of the product while also engaging other stakeholders within the company, like Product Management. The tendency in these moments is to think in terms of silos – this group is responsible for operations, this one for the front-end interface, this one for backend infrastructure, this one for testing, and so on. The problem with this approach is that it almost always results in organizational mutations like *Dev. because the nature of today’s software and technologies doesn’t recognize such neat distributions of responsibility. This is especially true of PaaS, IaaS, and other cloud-based offerings, that are themselves hybrid service/product offerings. I saw this phenomena occur when I took the reins at Lynda.com. Although the team had grown to more than 30 engineers, they could only work on one project at a time. The whole team “swarmed” on a given feature, since nobody owned any one piece of the service. The first change that I made was to organize the team into smaller scrum teams that each owned a feature from end to end. This simple reorganization unblocked the log jam that had been created by the swarm mentality, and allowed teams to work on multiple projects at once. Read the rest of this entry »

Mobile vs. Web: Which is Harder to Test?

April 19th, 2016 by Joe Nolan

Have you ever worked on a web-based test team and switched to a mobile team and wondered if your life is about to get easier or harder? There are significant differences between testing mobile vs. web, and yes, one is MUCH harder than the other. Want to guess which one? Read on and see if you guessed correctly.

Let’s Compare

The table below shows the different facets of testing and where its execution is most challenging. Read the rest of this entry »

Recap: Easy Continuous Deployment You Can Trust (Webinar)

April 14th, 2016 by Bill McGee

Thanks to everyone who joined us for our recent webinar, “Easy Continuous Deployment You Can Trust”, featuring Solano Labs Founding Engineer, Brian Kendzior, and Sauce Labs Solutions Architect, Neil Manvar.

In their presentation, Brian and Neil demonstrated a continuous deployment release process that used Solano CI, Sauce Labs, and AWS CodePipeline. The release process included smoke, unit, integration and browser tests to guarantee issue-free deployments.

Brian and Neil also demonstrated how to:

  • Build your software release pipeline by connecting different steps into an automated workflow using AWS CodePipeline
  • Run your automated web tests on any browser and operating system combination using Sauce Labs
  • Auto-parallelize your test runs with a Continuous Integration server using Solano CI

Want to learn more about automated testing and Continuous Integration? Download our free report, “How to Get the Most out of Your CI/CD Workflow Using Automated Testing”. 

Access the recording HERE and view the slides below: 

Test Faster and Smarter by Testing in Production

April 14th, 2016 by Chris Riley

You may dread the term testing in production (TiP). The thought of potential loss of data, downtime, and a damaged reputation to organizations can be daunting. But things need not be that way. In fact, today, testing in production is used by some of the biggest organizations with much success. But can it become a reality for your team?

Accident or Intentional?

Testing in production is not a completely new concept. In fact, you’ve probably seen it in action more often than you imagine. Think of an app that you’ve released or one that you know of that was poorly tested. You likely spent the next few weeks firefighting, and got it to be functional faster than you thought possible. In this case, you were forced to test in production. But, what if you could hone the art of testing in production and use it to your advantage? What if you could spot and fix issues so much faster than you do today? What if you could influence development from start to end? What if you could do all this without risking the reputation of your team or organization? That’s the promise of TiP, and it’s worth a second glance. Read the rest of this entry »

Waiting for Green

April 12th, 2016 by Ashley Hunsberger

Every now and then, you may encounter a time when you need to stabilize your automated UI tests (for myself, that time is now). Although you don’t want to add to a framework that you are stabilizing, you probably don’t want to halt development on new features. (Warning — telling your leadership team no one is allowed to add more tests until everything goes green might not go over well.)

What do you do in the meantime? The answer is simple, and I look to some practices in Behavior Driven Development (BDD) as a guide – build a test skeleton into your current framework.

The First Rule of Stabilization: Don’t Create a Manual Test Suite

While you may temporarily need to revert to manual execution, it does not necessarily mean you’ll want to go back to a manual test suite for a couple of reasons: Read the rest of this entry »

Announcing Sauce Connect Launcher

April 8th, 2016 by Ken Drachnik

We’re proud to announce the release of Sauce Connect Launcher!

Sauce Connect is a secure tunnel which lets users run tests against applications behind the firewall or on localhost. You can use Sauce Connect to run both manual and automated tests.

Sauce Connect Launcher (SCL) is a Firefox add-on which allows you to run a Sauce Connect tunnel directly from Firefox. With SCL you don’t have to download Sauce Connect or go to the command line to install and run Sauce Connect. A common use case for this tool would be to use the Firefox add-on to launch a Sauce Connect tunnel and then run a manual test from Firefox on localhost.

To use the tool, simply download the add-on from the Firefox marketplace. When prompted, input your Sauce username and access key (these can be found on your User Settings page).

Once this is done, to launch a test simply go to Tools > Web Developer > Launch SC.

Sauce Connect Launcher

 

Now start your tunnel. You can stop the tunnel by going to Tools > Web Developer > Shutdown Sauce Connect.

Once this tunnel is established, you can run tests on local host.

Happy Testing!

Get Started with Protractor Testing for AngularJS

April 6th, 2016 by Greg Sypolt

How do you test your AngularJS applications? With Protractor. Protractor is an end-to-end testing framework for AngularJS applications. This getting started guide is for new software developers in test who are interested in learning about Protractor testing. By following this getting started guide, you’ll understand how to build a firm foundation and learn fundamental Protractor testing for AngularJS applications.

Build a solid foundation

To build better software tests, you’ll need a solid base of the technologies behind your application, which teaches you the principles that will be essential when coding, executing and debugging your tests. Let’s focus on the correct learning path for Protractor testing. This is the age of JavaScript (JS) applications and frameworks. JavaScript is the foundation and critical learning block needed to be successful with Protractor testing, and what follows is your roadmap to learn JavaScript, NodeJS, and AngularJS for Protractor testing. Read the rest of this entry »