[Webinar] What DevOps Is – and Why You Should Care

July 8th, 2015 by Bill McGee

DevOps has become the newest buzzword. You’ve probably seen or heard the term DevOps once, twice, a thousand times. But even the biggest supporters of DevOps will admit that the concept is creating as much noise and confusion as converts.

The practice of DevOps is not new, yet in the last two years it has seemingly dominated chatter within software development. But what does DevOps really mean? And what is the impact of DevOps on QA teams, if any at all?

Join us as DevOps Analyst Chris Riley shares the meaning and history of DevOps, his perspective on the movement, and his ideas about its future. He will share the knowledge that he has gathered from tools vendors and practitioners, all to help you navigate the sea of DevOps conversations to maximize the movement to your advantage.

This webinar will cover:

  • The difference between the practice of DevOps and the movement
  • What the future of DevOps holds
  • The intersection of DevOps and QA

Join us for this presentation next Tuesday, July 14 at 11am PDT/2pm EDT. There will be a Q&A with Chris afterwards.

Click HERE to register today.

Want to read more about DevOps and Continuous Integration? Download this free GigaOm white paper.

Appium + Sauce Labs Bootcamp: Chapter 3, Working with Hybrid Apps and Mobile Web

July 6th, 2015 by Isaac Murchie

Mobile applications can be purely native, or web applications running in mobile browsers, or a hybrid of the two, with a web application running in a particular view or set of views within a native application. Appium is capable of automating all three types of applications, by providing different “contexts” in which commands will be interpreted.


A context specifies how the server interprets commands, and which commands are available to the user. Appium currently supports two contexts: native and webview. Both of these are handled by different parts of the system, and may even proxy commands to another framework (such as webviews on Android, which are actually served by a managed ChromeDriver instance). It is important to know what context you are in, in order to know how you can automate an application.

Native contexts

Native contexts refer to native applications, and to those parts of hybrid apps that are running native views. Commands sent to Appium in the native context execute against the device vendor’s automation API, giving access to views and elements through name, accessibility id, etc. As well, in this context commands to interact directly with the device, to do operations such as changing the wifi connect or setting the location, can be used. These very powerful operations are not available within the context of a webview.

In addition to native and hybrid applications, the native context can be accessed in a mobile web app, in order to have some of the methods only available there. In this case it is important to understand that the commands are not running against the web application running in the browser, but rather are interacting with the device and the browser itself. Read the rest of this entry »

It worked on my machine – Communication Trap

July 1st, 2015 by Ashley Hunsberger

“I don’t see it on my machine!” said every developer ever. Every QA professional I have talked to in my career has heard this at least once.

But why?

Have we asked what’s in a bug?

The answer can either be what gets your team on the road to efficiency, or it can become a kink in the delivery hose. Let’s discuss how your QA can help the team deliver faster by providing a consistent language to keep everyone on target.

Don’t Let The Bad Bugs Bite…

Over the last decade, I have seen issues that have almost no noted content in them (seriously, some have just declared something to the tune of “This feature is… not working”). Then there are tickets that are the golden standard, that have all the information you could possibly want (and probably some with more than you need that turn out to be a few bugs in themselves).

But what happens when you don’t have a common way to report a ticket, and why is it important?

I just came across an issue recently that seemed to have some steps to reproduce, but the setup was not included. Try as I might, I could not replicate the bug. The only way that I could come close to the reported result did not match the steps provided, and I could only guess that the setup I created was what the reporter had done. I will let you guess how long this issue took. Hint: It wasn’t a few hours.

Or perhaps you have an offshore team. I’ve seen many, many instances when someone reports a bug that just doesn’t have enough information in it. If the engineer cannot figure out exactly what the issue is, and has to place it on hold, back to the reporter, the engineer waits another night while the person on the other side of the world hopefully notices the ticket is back in his or her queue for more details. That is another full day that the bug exists, delaying when the root cause can be identified and the issue fixed. Read the rest of this entry »

Recap: Test Automation Newbie? Robot Framework Will Save the Day! [Webinar]

June 26th, 2015 by Bill McGee

Thanks to everyone who joined us for our recent webinar, “Test Automation Newbie? Robot Framework Will Save the Day“, featuring Bryan Lamb (Founder, RobotFrameworkTutorial.com) and Chris Broesamle (Solutions Engineer, Sauce Labs), The webinar demonstrated how you can use Robot Framework, an open source, generic framework to create continuous automated regression tests for web, batch, API, or database testing. Topics covered included:

  • Where to find Robot Framework
  • How to install it
  • How to create automated test cases using plain English keywords
  • How to integrate automated tests into a Jenkins build
  • How to run test cases locally or on the Sauce Labs platform

Missed the presentation, want to hear it again, or share with a colleague?

Access the recording HERE and view the slides below.

Want to read more about using automated testing to get more out of your CI/CD workflow? Download this free white paper.

Guest post: Proving that an application is as broken as intended

June 25th, 2015 by Björn Kimminich

Typically you want to use end-to-end (e2e) tests to prove that everything works as intended in a realistic environment. In the Juice Shop application that idea is changed to the contrary. Here the main purpose of the e2e test suite is to prove that the application is as broken as intended!

Juice Shop: Broken beyond hope – but on purpose!

“WTF?” you might ask, and rightfully so. Juice Shop is a special kind of application. It is an intentionally insecure Javascript web application designed to be used during security trainings, classes, workshops or awareness demos. It contains over 25 vulnerabilities that an aspiring hacker can exploit in order to fulfill challenges that are tracked on a scoreboard.

The job of the e2e test suite is twofold:

  1. It ensures that the overall functionality (e.g., logging in, placing products in the basket, submitting an order, etc.) of the application is working. This is the above mentioned typical use case for e2e tests.
  2. It performs attacks on the application that should solve all the existing challenges. This includes SQL Injections, Cross-Site Scripting) attacks, business logic error exploits and many more.

When does Juice Shop pass its e2e test suite? When it is working fine for the average nice user and all challenges are solvable, so an attacker can get a 100% on the scoreboard! Read the rest of this entry »

Immutable Infrastructure

June 23rd, 2015 by Greg Sypolt

Server hugging is a disease

In the dark days before immutable servers, people clung to servers and treated them as untouchable gold. These people still exist, and hang onto their servers instead of moving into the Cloud. They are server huggers. What does the term “server hugger” mean? Its the desire to “touch” servers, “reboot” them on a regular basis, constantly upgrading really old software and hardware needed, etc. Before anyone can help cure the problem, server huggers have to admit it. They struggle to let go and embrace the Cloud. The thought of trashing servers and disposable components is absolutely frightening to server huggers, especially the (unfounded) fear of losing control. Server virtualization and Cloud computing will make these types extinct in the near future. Let’s start unshackling your servers!

Unshackle your servers

It’s all within reach. Start the transition today! Before the unshackling can begin, you need to build a Cloud architecture strategy for predictability, scalability, and automated recovery. The next step is huge: getting buy-in from development and operation teams. It starts by presenting your strategy and demonstrating the importance of virtual servers and containers to make it possible. Let’s start setting your dedicated servers free!

Start small and expand slowly! Always look for ways to simplify your infrastructure — build, measure, and learn. Some people like to learn by jumping directly into writing their own code and others may seek experts. Choose your own adventure! You are making giant leaps towards immutable servers. Do not treat virtual machines in a static way.

Make infrastructure part of the application

What is an immutable infrastructure? It is made up of immutable components that are replaced at every build and deployment, with changes made only by modifying a versioned definition, rather than being updated in place. So when building infrastructure as part of the application, the following three words immediately come to mind: Read the rest of this entry »

[Webinar] Test Automation Newbie? Robot Framework Will Save the Day!

June 19th, 2015 by Bill McGee

You’re a tester on an Agile development team. You’re drowning in regression tests and limiting your team’s velocity. You know you need to automate those tests, ideally on multiple browsers. You know you can’t do it without a test automation framework.

If you’re responsible for creating diverse, scalable automated tests but don’t have the time, budget, or a skilled-enough team to create yet another custom test automation framework, then you need to know about Robot Framework!

In this webinar, Bryan Lamb (Founder, RobotFrameworkTutorial.com) and Chris Broesamle (Solutions Engineer, Sauce Labs) will reveal how you can use this powerful, free, open source, generic framework to create continuous automated regression tests for web, batch, API, or database testing. With the simplicity of Robot Framework, in conjunction with Sauce Labs, you can improve your test coverage and time to delivery of your applications.

You’ll get a clear introduction to Robot Framework including:

  • Where to find it
  • How to install it
  • How to create automated test cases using plain English keywords
  • How to integrate your automated tests into a Jenkins build
  • How to run your test cases locally or on the Sauce Labs platform

Join us for this presentation next Tuesday, June 23 at 11am PDT/2pm EDT. There will be a Q&A with Byran and Chris afterwards.

Click HERE to register today.

Want to read more about using automated testing to get more out of your CI/CD workflow? Download this free white paper.

Moving QA Upstream

June 17th, 2015 by Ashley Hunsberger

Every few years, new development and release methods appear. Lately, most of us are looking at continuous integration and continuous delivery. But what about those teams that are trying to transition to faster releases? That’s going to be a huge hurdle if you save your testing for the end, before release (and in my experience, that has almost always resulted in pushing the release date). There is an important shift that needs to be made in order to be successful: bring in QA early.

Moving Upstream – From the Field

I recently participated in a sprint retrospective. During our ‘lessons learned’ meeting, an engineer said, “I really wish we had had these conversations with QA earlier… they really knew the feature.” In my perfect world, the referenced discussion would have happened before any development had begun. Unfortunately, sometimes factors are out of our control (like the number of resources — I have discovered that I cannot, in fact, clone myself). The end result of this meeting was that the engineer realized that some unexpected new work and some revisions to code he had already written would need to be done. Extra scope aside, the engineer brought up a great point: the tester knows the system/feature, and really does know what questions to ask.

When you bring QA in earlier and discuss a feature up front with the engineer and designer — before any work is done — all questions and assumptions are out on the table. This pre-mortem allows the team to size up possible issues, and prepare for feature testing with better expectations. Together, you eliminate the guesswork that comes from working on your own, which will likely be different depending on whether you are a developer or a tester. When QA and devs work together, engineers know precisely what they have to develop in order for the feature to pass and meet the acceptance criteria, and when it is time to audit, designers should not be surprised. Read the rest of this entry »

Appium + Sauce Labs Bootcamp: Chapter 2, Touch Actions

June 15th, 2015 by Isaac Murchie

One aspect of mobile devices that needs to be automated in order to fully test applications, whether native, hybrid, or web, is utilizing gestures to interact with elements. In Appium this is done through the Touch Action and Multi Touch APIs. These two APIs come from an early draft of the WebDriver W3C Specification, and are an attempt to atomize the individual actions that make up complex actions. That is to say, it provides the building blocks for any particular gesture that might be of interest.

The specification has changed recently and the current implementation will be deprecated in favor of an implementation of the latest specification. That said, the following API will remain for some time within Appium, even as the new API is rapidly adopted in the server.

Touch Actions

The Touch Action API provides the basis of all gestures that can be automated in Appium. At its core is the ability to chain together _ad hoc_ individual actions, which will then be applied to an element in the application on the device. The basic actions that can be used are:

  • press
  • longPress
  • tap
  • moveTo
  • wait
  • release
  • cancel
  • perform

Of these, the last deserves special mention. The action perform actually sends the chain of actions to the server. Before calling perform, the client is simply recording the actions in a local data structure, but nothing is done to the application under test. Once perform is called, the actions are wrapped up in JSON and sent to the server where they are actually performed! Read the rest of this entry »

The All New Sauce Labs UI

June 11th, 2015 by Ken Drachnik

New UI screen shotAs Sauce continues to grow its functionality, we try to be thoughtful in how we make changes to our UI to accommodate new features. Today, we are proud to announce an entirely new graphical UI that enhances Continuous Integration testing workflows, available immediately. Our new UI provides better insights into software builds and test processes. It’s all of our work here at Sauce to help users speed up their application testing to bring their new software applications to market faster.

Here’s a look at what’s new:

New Dashboard
Our dashboard now offers a summary view on the status your builds and tests across your organization. It automatically groups tests into builds so you and your team can concentrate on the status of desired builds rather than on individual tests, helping you to quickly identify blocking issues.

Updated Tests and Tunnels Pages
Now you can click on a build to receive complete details on all tests within it, as well as highlight the ones that have failed. You can also view the status of each individual test, Sauce Connect™ tunnels.

Team Management and User Details
We know it can be difficult to track usage of third party tools across your organization and ensure they’re being effectively used. That’s why Sauce now offers some new enterprise management features that make it easier to manage permissions and access to testing specific resources. Now you can benefit from enhanced reporting, giving your organization insight into individual usage via 60-day graphs.

New Archives Page
This feature for the first time offers a searchable archives page to show all of your account activity over the past 90 days. Your team can easily view the history of their work to help them spot trends in application tests using custom queries or existing filters.

Single Sign On (SSO)
And finally, we know that provisioning new accounts for individuals can be a time consuming process. Sauce now supports SSO, enabling IT managers to “just-in-time” provision new user accounts with centralized account management and access control.

We are excited about today’s new updates and hope you are too. To learn more about what’s new, you can view a Webinar explaining the new UI.

Want to try it? Sign up for a free 14 day trial.