Getting the Existing Team On Board with Automation (Scripts)

August 27th, 2015 by Greg Sypolt

Introduction

In an attempt to do more with less, organizations want to test their software adequately, as quickly as possible. Businesses are demanding quick turnaround, pushing new features and bug fixes to production within days, along with quality. Everyone knows manual testing is labor-intensive and error-prone, and does not support the same kind of quality checks that are possible through the use of an automated test. Nobody likes change, but it’s time to educate your team on the importance of onboard automated testing.

The only way to make sense out of change is to plunge into it, move with it, and join the dance.  – Alan Watts

Everyone has had a job interview at some point in their lives, right? It is important to be prepared! The first few minutes of an interview are a make or break moment. Why? Because first impressions can have long-lasting effects. Never underestimate the power of first impressions. The same principle applies when onboarding automation to an existing manual testing team. Your initial presentation to your team or organization should be treated like a job interview. Be prepared. Deliver expectations and explain responsibilities — it’s critical since it is normal for employees to have an emotional reaction to anything they view as a job threat.

Why automated testing?

If things are going well, why do we want to implement automated tests? The demand is to do more with less, which makes manual testing an impossible task, but introducing automated testing into an existing software development lifecycle can also be daunting. However, when implemented, automated testing is a valuable asset that shortens testing cycles and helps teams become more agile.

marketingautomation1011

image source: http://marketing-works.net/

(more…)

Continuous Integration – The Heart of DevOps

August 26th, 2015 by Chris Riley

Missed our earlier webinar, “Beyond the Release – CI That Transforms“? Check out the recap below.

In this webinar, we discussed the power of CI and possible considerations for the future. One of the more interesting aspects of the webinar was seeing the modern pipeline with the new Sauce CI Dashboard alongside Cloudbees’ automation templates. We also conducted a small CI survey at the beginning of the webinar, and ended with a Q&A.

The Power of CI

Continuous Delivery and Deployment (CD) steal the show in DevOps conversations, but the reality is that Delivery and Deployment are not for every organization, nor are they already widely adopted. In order to move on to delivery and deployment, organizations must get Continuous Integration (CI) right — unless they were built from day one with the DevOps framework, and did not have to fit the processes into existing environments.

The reason CI is so powerful is that it allows you to dip your toe into the modern delivery pipeline without the risk or complexity of building out delivery and deployment all at once – and with the potential of failure. You can consider CI as the on-boarding for DevOps. And from a process and tool standpoint, CI is nearly identical to delivery and deployment, it means that once you get it right you can easily move on.

Webinar – CI Survey Results

I’m pleased to say the results of the Sauce Labs CI survey were almost exactly what I expected, served with a side of surprise. For me, the most interesting aspect of the survey results is how they appear to be in conflict with the perceived high CI adoption and success rates already existing in the market. Let’s look at the results among 500+ attendees:

What Types of Automated Tests do you run?

  • Unit 28%
  • Functional 40%
  • Integration 27%
  • None 6%

6% of the attendees are not running automated tests at all! This was astonishing to me. I expected 1% at most, especially given the audience, because they are already familiar with automation. At a minimum, I would expect all companies to automate Unit testing. However there is a high likelihood that what this 6% is saying is that they have automated tests, just initiated manually.

The results also showed that many are running functional tests. This is great! However, only 27% are running integration tests. This is troubling because compared to the reported 45% who state they are doing CI already, the lack of integration testing would seem to contradict that statement. I suspect that this is a definition problem, where some may define CI as being simply a shared testing environment, and not really the CI process as described in the webinar. (more…)

Reducing Regression Execution Times

August 13th, 2015 by Israel Felix

We all know the saying “time is money.” QA managers are constantly under pressure not only to deliver high-quality software products, but also to do so within time constraints.

Regression testing is a vital component in any software development life cycle to ensure that no new errors are introduced as a result of new features or the correction of existing bugs. Every time we modify existing source codes, new and existing test cases need to be executed on different configurations such as operating systems and platforms. Testing all of these permutations manually is simply not cost- or time-effective, and is also inconsistent. Automated regression addresses these challenges. But as we increase feature coverage and permutation testing, companies increase their execution times to a level that is no longer acceptable for the delivery of high-quality products within a tight schedule. Here are a few ways to improve execution time:

1. Introduce Continuous Integration (CI) Tools:

When going from a few scripts to a few thousand scripts, you begin to notice some growing pains. I frequently notice engineers manually executing script batches one at a time in a serialized manner and monitoring them as they execute. This consumes both time and resources. This becomes even more challenging when regression runs overnight or during weekends, with no one available to troubleshoot. As your automation grows, you need to have an infrastructure in place that allows you to scale and be able to run regressions unattended.

I recommend you use a Continuous Integration (CI) tool to manage automated regression executions, such as CircleCI or Jenkins. Tools like CircleCI or Jenkins can help you bring up virtual machines, start regressions, handle a more dynamic queuing mechanism, monitor regressions, and warn you if something goes wrong that requires manual intervention. It can also help you through a recovery mechanism that can be triggered if something becomes non-operational.

2. Use CI Tools not Only to Run Scripts, but also to Automate All Manual Steps!

When it is time to execute regression, there are a series of steps that need to be executed before a script can be run, such as:

  • Loading the new software to be tested
  • Updating your scripts with the latest version
  • Configuring servers
  • Executing scripts
  • Posting results
  • Communicating failure details

Very frequently we see customers using CI only to run scripts, relying on a manual process to execute the remaining steps, which is indeed a very time-consuming process.

I recommend minimizing manual intervention, and trying to achieve an end-to-end automated process using a CI tool that allows you to monitor and orchestrate the different tasks. (more…)

Recap: Simplify CI with the Updated Jenkins Sauce OnDemand Plugin (Webinar)

August 5th, 2015 by Bill McGee

Thanks to everyone who joined us for our recent webinar, “Simplify CI with the Updated Jenkins Sauce OnDemand Plugin”, featuring Neil Manvar and Jack Moxon from Sauce Labs. In this presentation, Neil and Jack gave an overview of the updated Jenkins SauceOn Demand plugin and how it can help streamline builds, simplify Sauce Connect setup, and troubleshoot tests faster. The webinar included a tour of the new features and demonstrated how the plugin works in Jenkins, as well as:

  • Learning about how the new Jenkins Plugin can optimize CI workflows
  • Seeing how the updated plugin simplifies Sauce Connect setup
  • How to speed your testing by running tests across different browsers and OS combinations simultaneously

Missed the presentation, want to hear it again, or share with a colleague?

Access the recording HERE and view the slides below.  Neil’s Github repo.

Hungry for more about CI? Download our DevOps is Crippled Without Continuous Integration white paper.

 

Updated Jenkins CI Plugin for Sauce

July 29th, 2015 by Ken Drachnik

Because enterprises are under extreme internal and external pressure to deliver better software, faster, development teams are transitioning to new software delivery processes such as Continuous Integration (CI) and Continuous Delivery (CD). Both practices have proven their value in accelerating software production beyond traditional approaches, yet achieving continuous testing to support these processes is often the most difficult hurdle. Our updated Jenkins Plugin offers developers the combined power of the Sauce platform with Jenkins CI, the world’s leading CI tool, to further simplify and accelerate the development process.

The latest release of the Jenkins CI Plugin offers a set of rich features to help developers better utilize their CI systems to increase developer productivity so they can release better software, faster.

Here’s a summary of what’s new:

Updated UI
The latest release of the Sauce OnDemand Jenkins plugin includes an updated UI with features to make it easier to run tests faster in parallel and understand the results.

Updated Browser Selection Tool
This makes it easier to speed testing by running tests in parallel. The new UI makes it very simple to select multiple platforms and browsers at one time. (more…)

[Webinar] Managing Continuous Delivery of Mobile Apps – for the Enterprise

July 23rd, 2015 by Bill McGee

Today, almost all organizations have mobile apps and for some, the mobile app is their only way of interacting with customers. With this increasing emphasis on mobile, the pressure to routinely update mobile apps means embracing Continuous Integration (CI) and Continuous Delivery (CD) methodologies.

Enabling CI / CD in your mobile development process means understanding the different solutions, overcoming unique challenges and ensuring the right ownership of the processes. In this webinar, join Harshal Vora from InfoStretch and Abhijit Pendyal from Sauce Labs to learn the steps required to enable Continuous Delivery of Mobile Application Platforms.

This webinar will cover:

  • Value of CI/CD in Mobile Development
  • CI/CD Architecture for Mobile Application Platforms
  • CI/CD Case Study – Requirements, Challenges and End Results
  • Demo – Jenkins / Code Update / Build Mobile App / Run automated tests using Sauce Labs

Join us for this presentation next Wednesday, July 29 at 11am PDT/2pm EDT. There will be a Q&A with Harshal and Abhijit afterwards.

Click HERE to register today.

Want to learn more about making Continuous Integration (CI) a part of your mobile development process? Download this free white paper, “Why Continuous Integration Should be Part of Your Mobile Development Process“.

Recap: Beyond the Release – CI That Transforms Organizations [Webinar]

May 29th, 2015 by Bill McGee

Thanks to everyone who joined us for our last webinar, “Beyond the Release: CI That Transforms Organizations“, featuring Chris Riley (DevOps Analyst), Andy Pemberton (CloudBees), and Abhijit Pendyal (Sauce Labs). The webinar demonstrated how Jenkins and Sauce Labs can work together to build a comprehensive CI tool set to help dev teams release desktop apps faster, at a higher quality and with more visibility. Topics covered included:

  • How CI goes beyond releases and into pipeline optimization
  • The challenges in adopting CI and the importance of getting it right
  • Thought leaders’ insights into future possibilities for CI

Missed the presentation, want to hear it again, or share with a colleague?

Watch the recording HERE and view the slides below.

Want to read more about Continuous Delivery? Download the 2015 Guide to Continuous Delivery Research Spotlight.

Beyond the Release: Continuous Integration The Transforms Organizations [Webinar]

May 20th, 2015 by Bill McGee

Continuous Integration is not simply about automated releases, it is also about knowing about how your software delivery pipeline works – its weak points and how it is performing over time, critical data points to make sure your CI system is healthy and sustainable.

In our next webinar, Chris Riley (DevOps Analyst), Andy Pemberton (CloudBees) and Abhijit Pendyal (Sauce Labs) will show you how Jenkins and Sauce Labs can work together to build a comprehensive CI tool set to help you release your desktop apps faster, at a higher quality and with more visibility.

This webinar will cover:

  • How CI goes beyond releases and into pipeline optimization
  • The challenges in adopting CI and the importance of getting it right
  • Thought leaders’ insights into future possibilities for CI

Join us for this presentation next Wednesday, May 27 at 11am PDT/2pm EDT. There will be a Q&A with both Chris, Andy and Abhijit at the end of the presentation.

Click HERE to register today.

Want to read more about Continuous Delivery? Download the 2015 Guide to Continuous Delivery Research Spotlight.

Not Just Faster Releases; Better Understanding

May 7th, 2015 by Chris Riley

This is a guest post by Chris Riley, a technologist who has spent 12 years helping organizations transition from traditional development practices to a modern set of culture, processes and tooling.

In DevOps there are several processes and tools that go beyond their face value. System Analytics, Application Performance Monitoring, and Continuous Integration (CI) go far beyond their core abilities. In particular, CI not only changes the speed and quality of releasing code, it improves communication and finds bugs in the software development process itself.

One of my biggest messages for companies moving to modern software delivery is the idea of being deliberate. Being deliberate means picking a culture, process, and tools that focus on results and how to attain them. Too often tools, especially those that are open source, are adopted on a whim without much forethought. This is the converse of being deliberate; allowing the tools to define the process and culture for you.

When organizations follow the ‘deliberate’ approach, they naturally get to a place where they move faster and what they have built is sustainable. A huge component of getting to this place is CI. No DevOps shop can survive without a continuous integration process. It allows front-end developers to participate in quality, find bugs before QA, test new functionality faster than ever. (more…)

Video: “Eliminate Rollbacks” Talk by Neil Manvar

May 6th, 2015 by Bill McGee

Missed Velocity Conf SF? We’ve got you covered. Check out this talk by Neil Manvar, Sauce Lab’s new Professional Services Lead. View “Eliminate Rollbacks” below and let us know what you think in the comments.