Ask a Selenium Expert: How to Handle Exceptions

May 29th, 2014 by Amber Kaplan

selenium testing & sauceThis is the final follow-up question in a series of 8 from Selenium expert Dave Haeffner. Read up on the firstsecondthirdfourthfifthsixth, and seventh.

During our recent webinar, “Selenium Bootcamp,” Dave discussed  how to build out a well factored, maintainable, resilient, and parallelized suite of tests that run locally, on a Continuous Integration system, and in the cloud. The question below is number 6 of 8 in a mini series of follow-up questions.

8. What’s the best way to handle StaleElementReferenceExceptions? Is this ok just to squelch them?

Yes, for the most part, it’s okay to rescue these exceptions. You can see an approach on how to do that here.

That wraps up our follow-up session from Dave Haeffner! Get more info on Selenium with Dave’s book, The Selenium Guidebook, or follow him on Twitter or Github.

Have an idea for a blog post, webinar, or more? We want to hear from you! Submit topic ideas (or questions!) here.

Ask a Selenium Expert: Should I Test Load with Selenium?

May 22nd, 2014 by Amber Kaplan

selenium testing & sauceThis is part 7 of 8 in a mini series of follow-up Q&A’s from Selenium expert Dave Haeffner. Read up on the firstsecondthirdfourthfifth, and sixth.

During our recent webinar, “Selenium Bootcamp,” Dave discussed  how to build out a well factored, maintainable, resilient, and parallelized suite of tests that run locally, on a Continuous Integration system, and in the cloud. The question below is number 6 of 8 in a mini series of follow-up questions.

7. ­If I want to test load using Selenium, will I have to run the same test multiple times in one “script” or can I instruct Selenium to run it multiple times?

While you could use Selenium to test load, it’s not the best tool for the job since it’s pretty expensive to achieve this outcome. There are better tools suited for the job — like JMeter or Gattling. That being said, there are some companies that specialize in Selenium-based load testing. You can find some of them on the ‘Commercial Support’ section of the Selenium HQ Support page.

Alternatively, you could try a more home grown approach like I outline in this write-up.

-Dave Haeffner, April 9, 2014

Can’t wait to see the rest of the Q&A? Read the whole post here.  Get more info on Selenium with Dave’s book, The Selenium Guidebook, or follow him on Twitter or Github.

Have an idea for a blog post, webinar, or more? We want to hear from you! Submit topic ideas (or questions!) here.

Guest Post: Open Sauce Enables Plone to Focus on Robot Framework

May 16th, 2014 by Amber Kaplan

Robot Framework Our friends in the Plone community recently took to Open Sauce for their testing needs to save time. The results have been stellar; with the time saved they’re able to focus on improving Robot Framework, according to their release manager Eric Steele.

Check out the rest of what they have to say below.

When I took over as release manager for the Plone CMS project, we ran our test suite nightly, but that only covered our Python code and some simple form submissions. The entire JavaScript layer remained largely untested, save a few click-arounds by hand before each release. The suspicion that some critical feature might have broken in a browser combination we hadn’t tried kept me up at night. As I began preaching the need for continuous integration and in-browser testing, it was surprising to find a whole team’s-worth of people excited to obsess over running tests, improving coverage, and collecting a fleet of VMs to run the few Selenium tests we’d put together at that point. The latter proved to be our undoing; we spent more time managing our testing infrastructure than we did doing actual testing.

Thankfully, Sauce Labs’ Open Sauce came along to save us.

Open Sauce has freed up my testing team to do far more interesting things. We’ve put quite a bit of effort into helping Robot Framework grow. Robot’s Behavior-Driven Development abstraction seems to fit everyones’ heads a bit better and allows us to easily alter tests based on which features are active. Asko Soukka, previously featured on this blog, became Plone’s Person of the Year for 2013 based on the work he put into extending Robot Framework for our community.

Asko has created a set of Robot keywords to enable automated screenshots for our Sphinx documentation. This allows our documentation to show the Plone user interface in the same language as the document. Groups deploying Plone sites can regenerate our end-user documentation with screenshots featuring their own design customizations. It’s a huge win; users see examples that look exactly like their own site. Finally, in a bit of pure mad science, Asko has piped those image generation scripts through a text-to-speech program to create fully-automated screencasts.

The Plone community is currently at work on the upcoming release of Plone 5. With its new widgets layer and responsive design, there are so many new ways that bugs could creep into our system. Happily, that’s not the case. I get a nightly report full of screenshots of Plone in action across browser, device, and screen size. Basic accessibility gotchas are quickly caught. Content editing and management features are automatically tested on both desktop and mobile. Open Sauce allows us to focus on getting things done and done correctly. Finally, I can sleep soundly — or at least find something else to worry over. -Eric Steele, Release Manager, Plone.org. Read Eric’s blog here or follow him on Twitter.

Do you have a topic you’d like to share with our community? We’d love to hear from you! Submit topics here, feel free to leave a comment, or tweet at us any time.

[Live Panel] Mobile Testing with Appium

May 15th, 2014 by Amber Kaplan

2014-04-24_Appium_SM_Images_Facebook_851x315px

As we know from the survey we conducted early this year, mobile testing is incredibly prominent in the QA world. To support this growing field, we’ve heavily invested in the open source project Appium; the leading cross-platform framework for testing hybrid and native mobile applications. Used by organizations large and small, including Expedia and Gilt Groupe, Appium provides a language-agnostic means to automatically test mobile functionality without modifying an app in any way. 

To help celebrate the release of the  latest battle-tested version 1.0 [Orion], we’re thrilled to announce our next webinar: “Live Panel: Appium Core Committers Answer Your Questions” with leading contributors to the project, Jonathan Lipps, Matthew Edwards, and Dan Cuellar. Join us for this panel discussion on Thursday, May 29, from 11AM-12PM PDT.

Questions to the panelists may include:

  • Appium’s philosophy
  • An overview of the 1.0 release
  • The roadmap ahead
  • Best practices for automating mobile tests

A live Q&A will follow. All attendees are encouraged to participate.

Register here today – you won’t want to miss this one!

Ask a Selenium Expert: is Random Execution Order Necessary?

May 14th, 2014 by Amber Kaplan

This is part 6 of 8 in a mini series of follow-up Q&A’s from Selenium expert Dave Haeffner. Read up on the firstsecondthirdfourth, and fifth.

During our recent webinar, “Selenium Bootcamp,” Dave discussed  how to build out a well factored, maintainable, resilient, and parallelized suite of tests that run locally, on a Continuous Integration system, and in the cloud. The question below is number 6 of 8 in a mini series of follow-up questions.

6. Since each test you showed performs its own setup and teardown, is random execution order really necessary?

Yes. Random execution order is necessary to help identify (and prevent) cross-test dependencies. It also has the added benefit of exercisizing the application under test in a random order.

-Dave Haeffner, April 9, 2014

Can’t wait to see the rest of the Q&A? Read the whole post here.  Get more info on Selenium with Dave’s book, The Selenium Guidebook, or follow him on Twitter or Github.

Have an idea for a blog post, webinar, or more? We want to hear from you! Submit topic ideas (or questions!) here.

Appium 1.0 [Orion] and Appium Inspector on OSX Mavericks Demo [VIDEO]

May 12th, 2014 by Amber Kaplan

appium_logo_final

‘Appy Monday!

Our friend Shantikant Jagtap just released a comprehensive post highlighting the Appium 1.0 [Orion] launch, new Appium features, how to download Appium, and how to use it with the OSX app on Mavericks and Appium Inspector.

Be sure to check out the post here and watch the video demo below, which highlights how to use the Appium OSX app, plus how to record a script using Appium Inspector.

Have an idea for a blog post, webinar, or more? We want to hear from you! Submit topic ideas (or questions!) here.

DevNetwork Recognizes Sauce Labs as a Leader in App Testing and Automation Technology [INFOGRAPHIC]

May 8th, 2014 by Amber Kaplan

Thanks to DevNetwork for including Sauce Labs in their infographic, “Leadership Map of Developer Technology” as a leader in App Testing and Automation Technology! You can see where Sauce fits into the ecosystem in the infographic below.

Dev Network

DevNetwork, producers of APIWorld, DataWeek, and DeveloperWeek, organized over 120 companies into 9 core groups of technologies across 34 categories in this version one edition of the developer technology industry.

“The emergence of a new ecosystem of cloud tools and technologies for developers means that almost every company no longer needs to feel locked into one technology stack or platform,” said Geoff Domoracki, CEO of DevNetwork.  “Because there are hundreds of technologies offering platform-as-a-service, infrastructure-as-a-service, database-as-a-service, even dev ops or back-end processing via API we saw the value to the community that we outline a comprehensive view of the best developer technologies in this infographic.”

About DevNetwork

DevNetwork is a San Francisco based technology conference company with over 6,500 participants across its conferences and events per year. These include APIWorld 2014, the nation’s largest API conference that’s 100% focused on developers, as well as DeveloperWeek 2015, which has grown to be San Francisco’s largest week-long tech event series. ContactDevNetwork to get your company involved in its media and events.

Ask a Selenium Expert: Selenium Grids, Scaling, and Parallelization

May 7th, 2014 by Amber Kaplan

selenium testing & sauceThis is part 5 of 8 in a mini series of follow-up Q&A’s from Selenium expert Dave Haeffner. Read up on the firstsecondthird, and fourth.

Dave discussed  how to build out a well factored, maintainable, resilient, and parallelized suite of tests that run locally, on a Continuous Integration system, and in the cloud in our recent webinar, “Selenium Bootcamp“.

Following the webinar, there were several follow-up questions. Dave’s agreed to respond to 8. Below you’ll find the fourth Q&A. Stay tuned next Wednesday for the next question.

5. ­Is Selenium Grid still relevant/useful for parallelization? ­

Selenium Grid is a great option for scaling your test infrastructure if you’re okay with handling the overhead of spinning up/maintaining a bunch of machines. It, by itself, will not give you parallelization. That is to say, it can handle however many connections you throw at it, but you will still need to find a way to execute your tests in parallel. You can learn more about Selenium Grid on it’s project main page.

-Dave Haeffner, April 9, 2014

Can’t wait to see the rest of the Q&A? Read the whole post here.  Get more info on Selenium with Dave’s book, The Selenium Guidebook, or follow him on Twitter or Github.

Have an idea for a blog post, webinar, or more? We want to hear from you! Submit topic ideas (or questions!) here.

The State of Mobile Testing in 2014 (INFOGRAPHIC)

May 6th, 2014 by lauren nguyen

In early 2014, Sauce Labs conducted a survey to learn more about the state of mobile testing. We’ve highlighted our findings in the fun infographic below.

Mobile_Infographic_5-30-14b (1)

We don’t know about you, but some of our favorite take-aways from those surveyed are:

  • 95% are performing mobile tests in order to solve common developer and QA issues.
  • 81% are performing functional tests.
  • 78% believe it’s valuable to test on a combination of simulator/emulators and real devices.
  • 81% support up to 10 devices total in their matrixes.

With the recent launch of the cross-platform framework Appium 1.0, we’re willing to bet mobile testing will continue to grow in importance as we welcome 2015.

Join us in celebrating mobile and the growth of mobile testing and automation!

Keep testing awesome,

Sauce Labs

Oh, Appy Day! Appium 1.0 is Here

May 2nd, 2014 by Jonathan Lipps

Today we’re excited to celebrate the Appium team’s imminent release of Appium 1.0. Appium 1.0, in addition to providing new features, increased stability, and Appium’s very own client libraries, stands as a milestone in the history of this amazing open source project (which was recognized as one of the most promising new open source projects of 2013). After a year and a half of hard work from a dedicated team of over 100 contributors, we felt it was time to make the statement that Appium is ready to come out of its initial development phases. Companies like Expedia, LinkedIn, Lookout, Opera, Pandora, StubHub and Zoosk have decided to test their mobile apps and websites with Appium’s flexible, cross-platform interface, and it’s time for Appium to make its debut on a bigger stage.

In addition to the various stability and usability improvements we’ve made over the last several months, here’s what you have to look forward to in Appium 1.0:

  • A brand new set of Appium client libraries, in Java, Ruby, JavaScript, Python, PHP, and C#. These libraries make it easy to take advantage of all the new API methods we’ve added to the WebDriver spec.
  • Updated documentation and website. We invite you to enjoy (and contribute to!) our new docs and website.
  • New Desired Capabilities API. We worked hand-in-hand with the leaders of the Selenium/WebDriver project to draft a new mobile automation spec, and we’re proud to announce that Appium is the first project to support this spec.
  • Full XML/XPath support. Appium now returns meaningful XML responses as app source, instead of JSON. In addition, we now have full XPath query support over this XML source, so that it will be much easier to find the elements you need, no matter how complex the query.
  • Android hybrid support. On Android 4.4+ devices, you can now use Appium for your hybrid apps without falling back to the bundled Selendroid.
  • New locator strategies. We have added the `ios uiautomation` and `android uiatomator` locator strategies so you can find elements on those platforms using a syntax straight from the UIAutomation or UiAutomator APIs.
  • Better Windows environment support. Appium plays much more nicely with Windows, when you’re using Windows to run your Android Appium tests.
  • More platform support. In the past few months we’ve added support for iOS 7.1 and Android 4.4.
  • Updated Appium.app / Appium.exe. The GUI has undergone a total overhaul. Happy app inspecting!

We hope you’ll agree that we’ve come a long way in the last year and a half. If you’re already a heavy Appium user, get ready to upgrade your tests for use with 1.0! And if you’ve never given Appium a chance, we hope you’ll take this opportunity to get started with mobile automation. And at Sauce Labs, we’re extremely proud to support Appium 1.0 immediately after its release at the launch party we are hosting on Friday night.

Join us in celebrating the growth of mobile automation made awesome!