Recap: Continuous Testing Meets the Classroom: Testing Online, Interactive Curriculum at Code.org [Webinar]

February 12th, 2016 by Bill McGee

Thanks to everyone who joined us for our recent webinar, “Continuous Testing Meets the Classroom: Testing Online, Interactive Curriculum at Code.org”, featuring Brian Jordan from Code.org. In his presentation, Brian discussed how Code.org approaches testing throughout the product development cycle, given their unique testing challenges – developing interactive, game-like curriculum for just the types of browsers you’d expect to find in school computer labs – from Internet Explorer 9 to iPads across 40+ languages.

The presentation also showed:

  • What Code.org’s open source automated testing stack looks like
  • What visual testing with Applitools, cross-browser Selenium tests on Sauce Labs, and live-site monitoring with New Relic and Honeybadger look like in practice
  • About the Code.org “Bug Collection” —real live examples of bugs detected before they hit production

Missed the presentation, want to hear it again, or share with a colleague?

Access the recording HERE and view the slides below.

Want to learn more about automated testing and Continuous Integration? Download our free report, “How to Get the Most out of Your CI/CD Workflow Using Automated Testing”.

Selenium Design Patterns

February 11th, 2016 by Greg Sypolt

In software engineering, a design pattern is a general reusable solution to a commonly occurring problem in software design. A design pattern is not a finished design that can be transformed directly into code. It is a description or template for how to solve a problem that can be used in many different situations.1)Software design pattern – Wikipedia, the free encyclopedia” 2011. 18 Jan. 2016

Why are design patterns so important for Selenium development? It can speed up development and reduce the maintenance impact. Using design patterns in test automation development is not necessary, but a seasoned automation engineer understands the importance. If you are new to test automation, I highly recommend learning the best way to write automated tests, and this article is a great start. Read the rest of this entry »

References   [ + ]

Top 5 Quality Disasters (or Misses) of 2015

February 8th, 2016 by Ashley Hunsberger

2015 was quite the year for quality in almost every industry. Here are some defects (some disastrous, some just funny) that really caught my attention over the last year, and a few lessons we can learn from them as we develop our own test strategies such as data usage, environments, security, and more in our day-to-day work.

#1 – Social Media: Facebook tells me I’ve known you since before I was born!

Image Source: http://ti.me/1mp25FH

Image Source: http://ti.me/1mp25FH

This was probably one of the funnier, and definitely not critical, bugs of the year. (Let’s face it — no one actually got hurt, and I think we all chuckled a little when we saw it pop up in our news feeds!) While no one actually _confirmed_ what happened, most theories surround the Unix Epoch (January 1, 1970, which with time zone adjustments could be sometime on December 31, 1969), so 46 years from the day we started seeing our nearly golden friendships appear. Microsoft engineer Mark Davis offered his hypothesis (see http://ti.me/1mp25FH).

What can we learn from this? Let’s say you’re adding in a feature that has to deal with dates. Always consider testing not just with fresh data, but also data that was present prior to the feature being added. I must admit that in my own testing we hate dealing with date/time stamps as it usually involves manipulating data in unrealistic scenarios, but test we must. Make sure you are doing some time travel with your data — it helps get you closer to what your customers will have! Read the rest of this entry »

Announcing New REST API Rate Limits

February 3rd, 2016 by Yaroslav Borets

Sauce Labs is introducing a new rate limiting on our REST endpoints in order to ensure a great experience for all of our customers. In addition to the recent limits placed on the number of requests per second we will be implementing further restrictions with dedicated hourly request limits for each endpoint. The new restrictions will limit the access to all endpoints to 10 reqs/s or 3500 reqs/hour if the user is logged in and 2 reqs/minute if user is logged out.  The limits will be tracked on a per account basis for both logged in and logged out users.

The new limits will go in the effect on Tuesday, March 1st 2016. We strongly encourage customers who use the REST API to modify their code to be able to gracefully handle a new set of restrictions. Please refer to the code samples below on how to prepare for the new limits as well as the headers to use.

The addition of more restrictive rate limits will be handled in a multi-stage process as follows:

  1. Starting February 1st , customers can opt-in to the new rate limits in order to test how their code handles rate limiting. The opt-in capability will be provided via a new header.
  2. On March 1st, the new rate limits will be in place by default, but customers can opt out using a dedicated header.
  3. Finally, in the beginning of April the new rate limits will be in place, and customers will no longer be able to opt out.

Read the rest of this entry »

Paired Testing: Two Is Better Than One

January 27th, 2016 by Ashley Hunsberger

Paired programming brings two developers together to produce higher quality code compared to those same two engineers coding separately. Just as paired programming has someone writing code while another person reviews the code as it is being written, paired testing has someone doing the testing while another person takes notes, asks questions, and spots/reports bugs. I’ve personally found paired testing to foster creativity, maintain focus, provide a new way to teach others, and help release better software in general. Two testers are better than one.

Pick a partner!

It’s probably important to note that not all people actually like to pair up. Let’s face it, many of us work in a world of introverts. Some people just don’t like to talk, or share their personal space. That said, there are a few good pairings you can look at: Read the rest of this entry »

Changing Development Culture to Become Quality Focused

January 25th, 2016 by Joe Nolan

How many project teams have you worked on where the accepted culture was to rely on the QA members to bear the load for quality? As the leader of a QA meetup, I still constantly hear stories from my members about developers’ assumptions that it is QA’s responsibility to find bugs. Not only is this attitude demoralizing for QA, it is also not in the best interest of the team. How can a team change development culture to one that is quality focused for the entire team?

From the Top Down

The answer seems obvious — Management needs to declare that all team members will have a hand in quality. This truly is critical to a successful culture change! Besides empowering dev managers and Scrum Masters to direct team activities (such as enforcing unit tests), it will give QA members the confidence to push the team as well. Read the rest of this entry »

Consider Your Application’s Home: Designing for Resiliency

January 21st, 2016 by Eric Jeanes

I am a firm believer in taking a cross-discipline-based approach to technology — taking something learned in one subject area and applying it to a problem in our everyday work. The political philosopher John Rawls, in his seminal work A Theory of Justice, provides a construct that (surprisingly) has a place in specific stages of application development.

When building systems, we are constantly held back to some degree by technical debt (the time wasted by repetitive tasks, and the bug fixes required to keep systems up and running). Not only is this time costly, but it is also typically less interesting than designing and constructing new systems (naturally). Read the rest of this entry »

A Functional Tester Looks at Performance

January 15th, 2016 by Ashley Hunsberger

Even if you aren’t directly responsible for performance, it is important to consider it under the umbrella of quality. As a tester, how do you move forward and help drive performance quality (especially when you are new to this area, like me)? What are the ramifications of not considering performance within QA? Let’s take a look at what performance is, the questions QA can ask during design and implementation, some of the types of testing that can be done, and making performance part of your acceptance criteria (and, therefore, part of your Definition of Done).

What is software performance, and why is it important?

As an end user, I think of performance as just how fast or stable something is. If I click on something, does it take forever to load in a website? Does my app crash every time I try to open it or submit something? Do I give up and find a better solution to meet my needs? Of course we want a feature to work, but do we think about the system holistically?

Read the rest of this entry »

Planning Quality Architecture for 2020

January 11th, 2016 by Greg Sypolt

I was inspired by Denali Lumma (@denalilumma) when she delivered a glimpse of the future in her talk about 2020 testing at the Selenium 2015 conference. The session was an excellent introduction that compared many scenarios of the minority testing elite versus the more common development team. The elite companies consider infrastructure FIRST, and the majority thinks about infrastructure LAST. It got my wheels turning regarding the future of software development. I don’t have all the answers right now, but I want to be part of the movement to plan and build architecture with quality. A few words come to mind when thinking about quality architecture — automation, scalability, recoverability, and analytics.

Build a culture

When building a culture, avoid too much control. You want a culture that embraces freedom, responsibility, and accountability. Why is building a culture like this important? It allows passionate employees to innovate and find big-time solutions. You can’t plan for innovation. It naturally happens. When you give passionate employees an inch, they’ll take a mile. The future team culture needs to push the envelope and step outside their comfort zone. Read the rest of this entry »

Re-Energize Your QA Career With Automation and DevOps

January 7th, 2016 by Joe Nolan

It’s time for you to stop being content with the status quo and re-energize your QA career with Automation and DevOps — otherwise, you might find yourself fading away like Marty McFly! I’m talking to YOU, manual tester! And YOU, QA manager! Oh, and YOU TOO, automation engineer! Every one of you who has a vested interest in your career growth needs to familiarize yourself with automation and DevOps tools.

Of Course You Need to Understand Automation

Let’s face it: In this day and age of software development, speed is the key to survival. In order to achieve clean builds, Continuous Integration, Continuous Delivery, and Agile development, manual testing just ain’t gonna cut it.

Everyone with the QA title needs to continuously build on their skill set, just like a developer. Even if you aren’t actively writing automation code, you still need to understand the capabilities and benefits of each type of automated test, especially the ones written by your development team. The team is relying on your expertise to guide them with acceptance criteria for stories, while bringing QA concepts to the table. Read the rest of this entry »