Posts Tagged ‘selenium’

Re-Blog: Building Markdown-Based Developer Docs

August 13th, 2014 by Amber Kaplan

Sauce Labs developer Chris Wren and his team have been working tirelessly to improve our documentation system, so we thought we’d share what they’ve been up to. Says Chris:

“Recently at Sauce Labs we decided to retool our documentation system. This decision came after accumulating docs in a number of template systems and repos which were difficult to standardize and maintain. The result of this effort was a new markdown-based docs site available at docs.saucelabs.com.”

For all the details, be sure to check out Chris’ post – just click the image below to view.

Building markdown-based developer docs

Want to contribute to Sauce Labs’ documentation? In the spirit of open source, we’ve housed them in GitHub. Submit away.

Updates Coming to Default Selenium and Chrome Versions on Sauce (August 2)

July 28th, 2014 by Santiago Suarez Ordoñez

On Saturday, August 2nd, we will update our Selenium and Chrome default versions to meet current, stable implementations. This update affects users that run automated Selenium tests on Sauce.

Default versions of Selenium and Chrome are used only for tests that don’t have a specified browser version. Users who choose to assign Selenium and Chrome versions to their tests will remain unaffected.

Below you’ll find more details about the updates.

Selenium

Currently the default Selenium version is 2.30.0. Following the update on August 2, the new default Selenium version will be 2.42.2. We advise you to test the new version (2.42.2) in advance using the following desired capability:

"selenium-version": "2.42.2"



If you run into any issues with the new default, note you can continue using the previous version (2.30.0) after Saturday by making the test request the selenium-version desired capability referred to below:

"selenium-version": "2.30.0"


Chrome

Currently the default Chrome versions are Chrome 27 and Chromedriver 1. Following the update on August 2, the new default Chrome versions will be Chrome 35 and Chromedriver 2.10. We advise you to test the new versions (Chrome 35, Chromedriver 2) in advance using the following desired capabilities:

"browserName": "chrome"
"version": "35"



By requesting Chrome 35, Chromedriver 2.10 will be used automatically.


If you run into any issues with the new default, you can continue using the previous versions (Chrome 27, Chromedriver 1) after Saturday by making the test request the “version” desired capabilities referred to below:

"browserName": "chrome"
"version": "27"


Troubleshooting Issues

If you see any issues after moving your tests to these new versions, we suggest checking for known issues on https://code.google.com/p/selenium/issues/list or contacting the Chromedriver and Selenium user groups.

Happy testing!

Campus Explorer Reduces Testing Time From 72 Hours to 72 Minutes Using Sauce Labs

July 18th, 2014 by Amber Kaplan

We sat down with Senior QA Manager Sage Rimal to hear how they use Sauce Labs at Campus Explorer.  Sage shared how they’ve automated their tests on Sauce, and have since reduced their testing time from 72 hours to 72 minutes.

Watch the video below to get the latest!

Want to share your story? We want to hear from you! Submit a request here.

Jason Huggins Takes a Leave of Absence at Sauce Labs to Work on HealthCare.gov 2.0

June 24th, 2014 by Amber Kaplan

nerds healthcare

Last year Sauce Labs cofounder Jason Huggins was part of the tech surge team of advisors who worked to fix the site and home of “Obamacare” through the power of test automation. He’s decided to take another leave of absence from Sauce to help the Ad Hoc and the Marketplace 2.0 teams to further this great cause.

Here’s what Jason had to say about the state of the government and automation:

Long term, government needs to get better at software development, including test automation. For now, the best way to fight that fight is by example. If the Marketplace 2.0 team can help the government avoid last year’s drama when open enrollment begins again in October, then we’ll have a good story to share with other departments and agencies. But if we get a repeat of last year, then it’ll be a lot harder to convince government officials to change their ways.

-Jason Huggins, June 23, 2014

Read Jason’s inspiring, original post here. You can follow his adventures on Twitter at @hugs.

While we’ll miss him around the office, we’re confident this is a worthy cause.

Guest Post: Test Lessons at ExpoQA

June 6th, 2014 by Amber Kaplan

This is the first of a three part series by Matthew Heusser, software delivery consultant and writer. 

Every now and again and opportunity comes along that you just can’t refuse. Mine was to teach the one-day version of my class, lean software testing, in Madrid, Spain, then again the following week in Estonia. Instead of coming back to the United States, I’ll be staying in Europe, with a few days in Scotland and a TestRetreat in the Netherlands.

And a lot of time on airplanes.

The folks at Sauce Labs thought I might like to take notes and type a little on the plane, to share my stories with you.

The first major hit in Madrid is the culture shock; this was my first conference where English was not the primary language. The sessions were split between English and Spanish, with translators in a booth making sure all talks were available in all languages.

The Testing Divide

Right now, in testing, I am interested in two major categories: The day to day work of testing new features and also the work of release-testing after code complete. I call this release testing a ‘cadence’, and, across the board, I see companies trying to compress the cadence.

My second major surprise in Madrid is how wide the gap is —and I believe it is getting wider —between legacy teams that have not modernized and teams starting from scratch today. One tester reported a four-month cycle for testing. Another team, relying heavily on Cucumber and Selenium, were able to release every day.

Of course, things weren’t that simple. The Lithuanian team used a variety of techniques I can talk about in another post to reduce risk, something like devOps, which I can talk about in another post. The point here is the divide between the two worlds.

Large cadences slow down delivery. They slow it down a lot; think of the difference between machine farming in the early 20th century and the plow and horse of the 19th.

In farming, the Amish managed to survive by maintaining a simple life, with no cars, car insurance, gasoline, or even electricity to pay for. In software, organizations that have a decades-long head start: banks, insurance companies, and pension funds, may be able to survive without modernization.

I just can’t imagine it will be much fun.

Batches, Queues and Throughput

Like many other conferences, the first day of ExpoQA is tutorial day, and I taught the one-day version of my course on lean software testing. I expected to learn a little about course delivery, but not a lot —so the learning hit me like a ton a bricks.

The course covers the seven wastes of ‘lean’, along with methods to improve the flow of the team – for example, decreasing the size of the measured work, or ‘batch size’. Agile software development gets us this for free, moving from ‘projects’ to sprints, and within sprints, stories.

In the early afternoon we use dice and cards to simulate a software team that has equally weighted capacity between analysis, dev, test and operations —but high variability in work size. This slows down delivery. The fix is to reduce the variation, but it is not part of the project, so what the teams tend to do is to build up queues of work, so any role never runs out of work.

What this actually does is run up the work in progress inventory – the amount of work sitting around, waiting to be done. In the simulation I don’t penalize teams for this, but on real software projects, ‘holding’ work created multitasking, handoffs, and restarts, all of which slow down delivery.

My lesson: Things that are invisible look free —and my simulation is far from perfect.

After my tutorial it is time for a conference day – kicked off by Dr. Stuart Reid, presenting on the new ISO standard for software testing. Looking at the schedule, I see a familiar name; Mais Tawfik, who I met at WOPR20.Mais is an independent performance consultant; today she is presenting on “shades of performance testing.”

Performance Test Types

Starting with the idea that performance testing has three main measurements: Speed, Scalability, and Stability, Mais explains that there are different types of performance tests, from front-end performance (javascript, waterfalls of HTTP requests, page loading and rendering) to back-end (database, webserver), and also synthetic monitoring – creating known-value transactions continuously in production to see how long they take. She also talks about application usage patterns – how testing is tailored to the type of user, and how each new release might have new and different risks based on changes introduced. That means you might tailor the performance testing to the release.

At the end of her talk, Mais lists several scenarios and asks the audience what type of performance test would blend efficiency and effectiveness. For example, if a release is entirely database changes, and time is constrained, you might not execute your full performance testing suite/scripts, but instead focus on rerunning and timing the database performance. If the focus on changes is the front end, you might focus on how long it takes the user interface to load and display.

When Mais asks if people in the organization do performance testing or manage it, only a handful of people raise their hands. When she asks who has heard of FireBug, even less raise their hand.

Which makes me wonder if the audience is only doing functional testing. If they are, who does the performance testing? And do they not automate, or do they all use Internet Explorer?

The talk is translated; it is possible that more people know these tools, it was just that the translator was ‘behind’ and they did not know to raise their hands in time.

Here’s hoping!

Time For A Panel

At the end of the day I am invited to sit on a panel to discuss the present (and future) of testing, with Dr. Reid, Dorothy Graham, Derk-Jan De Grood, Celestina Bianco and Delores Ornia. The questions include, in no particular order:

  •             Will testers have to learn to code?
  •             How do we convince management of the important of QA and get included in projects?
  •             What is the future of testing? Will testers be out of a job?
  •             What can we do about the dearth of testing education in the world today?

For the problem with the lack of education, Dorothy Graham points to Dr. Reid and his standards effort as a possible input for university education.

When it is my turn, I bring up ISTQB The International Software Testing Qualifications Board. – if ISTQB is so successful (“300,000 testers can’t be wrong?”) then why is the last question relevant? Stefaan Luckermans, the moderator, replied that with 2.9 Million testers in the world, the certification had only reached 10%, and that’s fair, I suppose. Still, I’m not excited about the quality of testers that ISTQB turns out.

The thing I did not get to say, because of time, that I want to do is point out that ISTQB is, after all, just a response to a market demand for a 2-3 day training certification. What can a trainer really do in 2-3 days? At most, maybe, teach a single technical tool, turn the lightbulb of thinking on, or define a few terms. ISTQB defines a few terms, and it takes a few days.

The pursuit of excellent testing?

That’s the game of a lifetime.

By Matthew Heusser – matt.heusser@gmail.com for Sauce Labs

Stay tuned next week for part two of this mini series! You can follow Matt on Twitter at @mheusser.

Have an idea for a blog post, webinar, or more? We want to hear from you! Submit topic ideas (or questions!) here.

Ask a Selenium Expert: How to Handle Exceptions

May 29th, 2014 by Amber Kaplan

selenium testing & sauceThis is the final follow-up question in a series of 8 from Selenium expert Dave Haeffner. Read up on the firstsecondthirdfourthfifthsixth, and seventh.

During our recent webinar, “Selenium Bootcamp,” Dave discussed  how to build out a well factored, maintainable, resilient, and parallelized suite of tests that run locally, on a Continuous Integration system, and in the cloud. The question below is number 6 of 8 in a mini series of follow-up questions.

8. What’s the best way to handle StaleElementReferenceExceptions? Is this ok just to squelch them?

Yes, for the most part, it’s okay to rescue these exceptions. You can see an approach on how to do that here.

That wraps up our follow-up session from Dave Haeffner! Get more info on Selenium with Dave’s book, The Selenium Guidebook, or follow him on Twitter or Github.

Have an idea for a blog post, webinar, or more? We want to hear from you! Submit topic ideas (or questions!) here.

Ask a Selenium Expert: Should I Test Load with Selenium?

May 22nd, 2014 by Amber Kaplan

selenium testing & sauceThis is part 7 of 8 in a mini series of follow-up Q&A’s from Selenium expert Dave Haeffner. Read up on the firstsecondthirdfourthfifth, and sixth.

During our recent webinar, “Selenium Bootcamp,” Dave discussed  how to build out a well factored, maintainable, resilient, and parallelized suite of tests that run locally, on a Continuous Integration system, and in the cloud. The question below is number 6 of 8 in a mini series of follow-up questions.

7. ­If I want to test load using Selenium, will I have to run the same test multiple times in one “script” or can I instruct Selenium to run it multiple times?

While you could use Selenium to test load, it’s not the best tool for the job since it’s pretty expensive to achieve this outcome. There are better tools suited for the job — like JMeter or Gattling. That being said, there are some companies that specialize in Selenium-based load testing. You can find some of them on the ‘Commercial Support’ section of the Selenium HQ Support page.

Alternatively, you could try a more home grown approach like I outline in this write-up.

-Dave Haeffner, April 9, 2014

Can’t wait to see the rest of the Q&A? Read the whole post here.  Get more info on Selenium with Dave’s book, The Selenium Guidebook, or follow him on Twitter or Github.

Have an idea for a blog post, webinar, or more? We want to hear from you! Submit topic ideas (or questions!) here.

Ask a Selenium Expert: is Random Execution Order Necessary?

May 14th, 2014 by Amber Kaplan

This is part 6 of 8 in a mini series of follow-up Q&A’s from Selenium expert Dave Haeffner. Read up on the firstsecondthirdfourth, and fifth.

During our recent webinar, “Selenium Bootcamp,” Dave discussed  how to build out a well factored, maintainable, resilient, and parallelized suite of tests that run locally, on a Continuous Integration system, and in the cloud. The question below is number 6 of 8 in a mini series of follow-up questions.

6. Since each test you showed performs its own setup and teardown, is random execution order really necessary?

Yes. Random execution order is necessary to help identify (and prevent) cross-test dependencies. It also has the added benefit of exercisizing the application under test in a random order.

-Dave Haeffner, April 9, 2014

Can’t wait to see the rest of the Q&A? Read the whole post here.  Get more info on Selenium with Dave’s book, The Selenium Guidebook, or follow him on Twitter or Github.

Have an idea for a blog post, webinar, or more? We want to hear from you! Submit topic ideas (or questions!) here.

Ask a Selenium Expert: Selenium Grids, Scaling, and Parallelization

May 7th, 2014 by Amber Kaplan

selenium testing & sauceThis is part 5 of 8 in a mini series of follow-up Q&A’s from Selenium expert Dave Haeffner. Read up on the firstsecondthird, and fourth.

Dave discussed  how to build out a well factored, maintainable, resilient, and parallelized suite of tests that run locally, on a Continuous Integration system, and in the cloud in our recent webinar, “Selenium Bootcamp“.

Following the webinar, there were several follow-up questions. Dave’s agreed to respond to 8. Below you’ll find the fourth Q&A. Stay tuned next Wednesday for the next question.

5. ­Is Selenium Grid still relevant/useful for parallelization? ­

Selenium Grid is a great option for scaling your test infrastructure if you’re okay with handling the overhead of spinning up/maintaining a bunch of machines. It, by itself, will not give you parallelization. That is to say, it can handle however many connections you throw at it, but you will still need to find a way to execute your tests in parallel. You can learn more about Selenium Grid on it’s project main page.

-Dave Haeffner, April 9, 2014

Can’t wait to see the rest of the Q&A? Read the whole post here.  Get more info on Selenium with Dave’s book, The Selenium Guidebook, or follow him on Twitter or Github.

Have an idea for a blog post, webinar, or more? We want to hear from you! Submit topic ideas (or questions!) here.

Ask a Selenium Expert: How to Run One Test on Multiple Browsers

April 30th, 2014 by Amber Kaplan

selenium testing & sauceThis is part 4 of 8 in a mini series of follow-up Q&A’s from Selenium expert Dave Haeffner. You can read up on the firstsecond, and third.

Dave discussed  how to build out a well factored, maintainable, resilient, and parallelized suite of tests that run locally, on a Continuous Integration system, and in the cloud in our recent webinar, “Selenium Bootcamp“.

Following the webinar, there were several follow-up questions. Dave’s agreed to respond to 8. Below you’ll find the fourth Q&A. Stay tuned next Wednesday for the next question.

4. ­How can I specify to Sauce Labs to run 1 test on multiple browsers?

You can see an example of one way to accomplish it in these write-ups:

-Dave Haeffner, April 9, 2014

Can’t wait to see the rest of the Q&A? Read the whole post here.  Get more info on Selenium with Dave’s book, The Selenium Guidebook, or follow him on Twitter or Github.

Have an idea for a blog post, webinar, or more? We want to hear from you! Submit topic ideas (or questions!) here.