Posts Tagged ‘automated testing’

Sauce Updates OnDemand Plugin for Jenkins Pipeline

August 10th, 2016 by Ken Drachnik

We have updated our OnDemand plugin for Jenkins with explicit support for Jenkins Pipeline. This update enables development and testing teams to easily create, manage, and run automated tests using scripts on Jenkins, resulting in faster testing.

The newly enhanced OnDemand plugin enables Sauce Labs users to skip the Jenkins UI and run tests directly via scripts. Previously, users had to work within the Jenkins UI to create testing tasks. While this is initially convenient, as automation grows and becomes more complex, the process of creating testing tasks becomes more complex, too. Now OnDemand enables testers to automate releases for complex and non-sequential tasks that previously required manual intervention.

Key benefits of the update include:

Create scripts to automate Jenkins operations
Save time by scripting test sequences that can automate testing, retry tests automatically and run processes in parallel. Developers and QA can more easily customize their Jenkins workflows and reporting results.

Automatically create test scripts for Jenkins
Use the Jenkins snippet generator to easily create Pipeline scripts to execute Sauce Commands via Jenkins. Users don’t need to know how to script in Pipeline – simply copy the code snippets to program workflows

The Sauce OnDemand Jenkins Plugin is available as a free integration. Download the plugin from Jenkins and read how to configure it on our docs page.

Happy Testing!

Implied Testing

March 30th, 2016 by Joe Nolan

Implied Testing is a way to write a test that indicates other parts of your workflow are working as you try to accomplish a goal. Make use of Implied Testing to minimize the amount of documentation and testing artifacts on a project.

According to the Manifesto for Agile Software Development, we should favor working software over comprehensive documentation. While this sounds good in theory, all too often test teams are asked to produce documentation explaining what they plan to test (in detail). The concept of Implied Testing will help save a lot of writing as it will eliminate duplication, and streamline the tests that feed into automation, allowing for simpler, more re-usable scripts.

Why Do We Write Detailed Tests?

In an ideal Agile world we limit our test documentation and focus on automated tests, with manual smoke and exploratory tests to supplement them. Our Acceptance Criteria in the stories should guide the tests necessary for the story to be complete. Unfortunately, circumstances can require an extensive amount of test documentation artifacts to be produced. Why? (more…)

Fragile Automation

March 28th, 2016 by Ashley Hunsberger

User Interface (UI) Testing.

The idea is simple — automate some UI tests to ensure your application is still behaving as expected. Usually your first set of tests — running green, no doubt — make you all cheer and pat yourselves on the back. Then you open up the framework to more people. Despite the reviews (so many reviews), the failures start to come, and they don’t stop. Or they run green and then fail and then run green again. And then fail again. So why are they so unstable? Is it bad scripts? Environment issues? Sometimes you just don’t know, and you think you are going to lose your mind. Let’s take a look at some common and potential issues you may be facing.

Architecture, Environments and Settings

Is your infrastructure designed for stability? Are you using on-premise or cloud instances? What may have saved you a dollar upfront could cost you many more down the road – so your testing environment is important.

Understand if your tests require particular system settings. Tests failing because of unwanted server variables is a waste of everyone’s time. We found that out the hard way a long time ago. You may need to have some isolated tests that cannot be run on the same server so the majority of your other tests can pass. (Or maybe decide how important the test really is.)

Or let’s say your testing frameworks are stable, but what about the tools or libraries you are importing? Are you pinning your stack to a version of these tools or libraries? A new version can completely break everything. (more…)

3 Simple Strategies to Get Started With Automation

March 24th, 2016 by Joe Nolan

If your test automation team’s directive is to automate X amount of tests, and you have no strategy as to which tests they should focus on, you are wasting your time. Before you begin writing your first line of automation code, make sure you have a strategy in place. Otherwise, you will have a ton of ineffective tests to maintain.

Don’t Choose a Random Goal

How many times have you been told that the goal of the team is to have X amount of test coverage? This is an arbitrary value picked out of the sky. What is it based on? If a UI automation team were to cover 80% of the stories in a sprint, they would never get done in time.

We all know how fragile UI automation is! How many times will a designer make a change that directly affects the UI and breaks the test? This is almost manageable during a sprint while you are working closely together, but how about when the product is sent to be translated to another language? The translator inevitably comes back with suggestions to allow for phrases more common and translatable. Bugs might be entered and UI changes made by a maintenance team with no heads up to the automation team, and Bam! — You have broken tests that need to be investigated. (more…)

Should You Have a Dedicated Automation Team Within Your QA Department?

September 1st, 2015 by Israel Felix

If you’ve led or managed QA teams that have included an automation test team, you’ve probably been in a situation where you had to decide whether you should keep them on board. Normally the decision needs to be made when there is a change in leadership, wherein the new management comes with a mandate to consolidate groups and reduce costs. This situation also tends to arise when working with startups or small companies when they are ready to put together or augment their QA teams. So should you have a dedicated automation team?

Typically, there are two camps with regards to dedicated automation teams. There are those who believe that we should have dedicated automation teams, and those who believe that QA engineers should handle manual testing and automation testing. From my experience working in QA within both small and large companies, I almost always prefer to have a dedicated automation team. However, there are a few scenarios where having a QA team that takes on both roles might make sense.

Time to Market

For automation to be done right, it needs to be a full-time job. From developing the framework and creating the libraries and scripts for different platforms to executing and debugging failures — it will all simply consume too much of an engineer’s time and compromise the actual testing and release date. As you already know, time to market and keeping a release on schedule is top priority, so testing needs to get done, no matter what. (more…)

Getting the Existing Team On Board with Automation (Scripts)

August 27th, 2015 by Greg Sypolt

Introduction

In an attempt to do more with less, organizations want to test their software adequately, as quickly as possible. Businesses are demanding quick turnaround, pushing new features and bug fixes to production within days, along with quality. Everyone knows manual testing is labor-intensive and error-prone, and does not support the same kind of quality checks that are possible through the use of an automated test. Nobody likes change, but it’s time to educate your team on the importance of onboard automated testing.

The only way to make sense out of change is to plunge into it, move with it, and join the dance.  – Alan Watts

Everyone has had a job interview at some point in their lives, right? It is important to be prepared! The first few minutes of an interview are a make or break moment. Why? Because first impressions can have long-lasting effects. Never underestimate the power of first impressions. The same principle applies when onboarding automation to an existing manual testing team. Your initial presentation to your team or organization should be treated like a job interview. Be prepared. Deliver expectations and explain responsibilities — it’s critical since it is normal for employees to have an emotional reaction to anything they view as a job threat.

Why automated testing?

If things are going well, why do we want to implement automated tests? The demand is to do more with less, which makes manual testing an impossible task, but introducing automated testing into an existing software development lifecycle can also be daunting. However, when implemented, automated testing is a valuable asset that shortens testing cycles and helps teams become more agile.

marketingautomation1011

image source: http://marketing-works.net/

(more…)

[WEBINAR] Making the Transition from Manual to Automated Testing

August 19th, 2015 by Bill McGee

If you have been manually testing your Web application, making the transition to test automation can seem daunting. Where do you start, how do you determine what tests to automate and what tools do you use? These are questions that are at the top of the list for every first timer. Smoothing your transition to building test automation can be made easier with a little preparation and the right tools.

In this webinar we will discuss the key differences between manual and automated testing and provide tips to help you prepare for the transition. We will also show you how QA analysts that are not programmers can build test automation quickly and very easily using the eureQa Testing Platform, and run these tests across browsers and devices on the Sauce Labs’ Cloud.

This webinar will cover:

  • Preparing to automate cross browser/cross device tests – what to expect and what to look out for?
  • Planning the test automation – what, when, who and how
  • Maintaining your test automation – Keeping up with application changes
  • Getting value from your automation – Managing test results
  • Demo of eureQa Testing Platform showing how you can build test automation and run it on multiple browsers and devices on Sauce Labs

Join us for this presentation next Tuesday, August 25th at 11am PDT/2pm EDT. There will be a live Q&A following the presentation

Click HERE to register today.

Want to learn more about automated testing? Download this free white paper, “How To Get The Most Out Of Your CI/CD Workflow Using Automated Testing“.

The Benefits of Parallel Testing

August 6th, 2015 by Greg Sypolt

Running in slow motion?

Are you running, but can’t make your feet move as fast as you want them to? This is a common feeling among beginners, as well as experienced automation developers. As your regression suite grows, it takes longer to run tests, and soon you have a problem because your regression suite is running longer and longer. There are a few approaches to smarter testing: reduce the number of tests, only run the tests applicable to the change, or optimize the test execution.

Marching toward continuous integration

As your software development team marches toward Continuous Integration (CI), the process involves a lot of automated testing. Even automated testing consumes precious time, so you need to ensure your automated tests are designed to scale. If a particular change is going to cause one or more tests to fail, the team needs to know about it as quickly as possible. Developing scripts to be lean and independent allows for fast feedback to developers.

Let’s unleash the power of parallelization

Use parallelization to speed up slow automated UI tests, and set a standard to develop lean and independent starts. A single cucumber scenario can easily take minutes to run. When you have a lot of scenarios, they can quickly compound your suite and take several minutes or hours to complete. No one wants slow automated tests — tests so slow that they only run a couple of times per day. Everyone expects automated tests to be launched for every build and send feedback within minutes, not hours. Set a standard for every build: the test execution must complete and send feedback within 10 minutes. (more…)

Test Automation KPIs

June 3rd, 2015 by Greg Sypolt

One of the interesting things about automation is that it frees you up from time-intensive manual testing, allowing you to spend time on strategic elements—because if you do not spend time on strategy, your capabilities as a team will not grow. And part of that growth means focusing on valuable metrics— metrics that will help you learn, and improve your processes.

Once you have processes in place, the next crucial step is to invest in automation. Automation helps you work faster, and makes your work consistent, traceable, and shareable, which is also imperative. All this comes only after establishing the right KPIs (key performance indicators).

Automation: Deliver Faster, from Months to Minutes

Ask yourself this question: without CI (continuous integration), how long would it take your organization to deploy a change that involves just one line of code? For instance, say your organization sets an objective to deploy a change in production within 30 minutes. To achieve this objective, everyone has to agree on the tools and processes that are needed for an easy button approach (aka continuous integration).

Let’s review the roles, team responsibilities, and the CI process. (more…)

Beyond the Release: Continuous Integration The Transforms Organizations [Webinar]

May 20th, 2015 by Bill McGee

Continuous Integration is not simply about automated releases, it is also about knowing about how your software delivery pipeline works – its weak points and how it is performing over time, critical data points to make sure your CI system is healthy and sustainable.

In our next webinar, Chris Riley (DevOps Analyst), Andy Pemberton (CloudBees) and Abhijit Pendyal (Sauce Labs) will show you how Jenkins and Sauce Labs can work together to build a comprehensive CI tool set to help you release your desktop apps faster, at a higher quality and with more visibility.

This webinar will cover:

  • How CI goes beyond releases and into pipeline optimization
  • The challenges in adopting CI and the importance of getting it right
  • Thought leaders’ insights into future possibilities for CI

Join us for this presentation next Wednesday, May 27 at 11am PDT/2pm EDT. There will be a Q&A with both Chris, Andy and Abhijit at the end of the presentation.

Click HERE to register today.

Want to read more about Continuous Delivery? Download the 2015 Guide to Continuous Delivery Research Spotlight.