Translating Web App Functional Testing To Mobile

April 27th, 2015 by Greg Sypolt

Technological advancements and the explosion of devices across multiple platforms means hardware and software developers have a much more difficult job. They have to keep up with the demand to develop and roll out new products. One of the most significant issues is accounting for the differences in system response, when responding to mobile traffic rather than to internet traffic. (more…)

Best Practices in Mobile CI [WEBINAR]

April 22nd, 2015 by Bill McGee

Modern organizations today feel immense pressure to deliver better software faster, and this is no different in the mobile space. The best practice of Continuous Integration for web dev has been embraced for years as it is a proven mechanism that accelerates production cycles. However,mobile developers have been slow to adopt CI, despite needing a quick go-to-market plan.

In large part, this is because mobile brings with it a set of unique challenges that make implementation tough. Nevertheless, tools have evolved and mobile development teams now have many options to choose from to implement a solid mobile CI system.

In our next webinar, Kevin Rohling (Emberlight, Ship.io) and Kristian Meier (Sauce Labs) will cover best practices in implementing a mobile CI system and demonstrate how you can easily build, test, and deploy mobile apps.

This webinar will cover: (more…)

Guest Post: All About Mobile Continuous Integration

April 20th, 2015 by Bill McGee
mobile_devices_continuous_integration

Image credit: Loris Grillet

CI’s not a new thing. Wikipedia says the phrase was first used back in 1994, way before modern mobile apps. Today it’s commonplace in many dev shops for developers to expect that their code is automatically tested when they commit and even automatically deployed to a staging environment.

For mobile developers though, it’s been a slow road to adopting many of these same practices.  In large part, this is because mobile brings with it a whole set of unique challenges that make implementation tough.  Nevertheless, tools have evolved a lot and mobile dev teams get a lot of value and goodness from having a solid CI system in place. Here are my top 3 reasons for using CI with the mobile products I work on: (more…)

Repost: Lessons Learned from Automating iOS Apps – What To Do When Tests Require Camera Roll Resources?

March 18th, 2015 by Bill McGee

This post comes from our friend Jay Sirju at Animoto, who is leveraging Sauce, Appium, and CI methodologies to automate mobile testing for their iOS application.  Check out the original post on their blog.

Here at Animoto, the mobile application development team had spent some time over the past year investigating and implementing CI methodologies into the development cycle of the Animoto Video Maker application for iOS. A major part of this initiative involved creating automated test cases that would run at various times and circumstances.

First a bit of background. When we started this, we had already implemented a good amount of automation for the Animoto website. We had chosen to use Selenium and ran our automated tests against various browsers using Sauce Labs. We decided to extend our existing infrastructure to support running automated tests using theAppium library against Sauce Labs.   For those unfamiliar with mobile testing on Sauce Labs, they use the iOS and Android Simulators to run tests. I know, not ideal, but we can get to that another time.

The Problem

For anyone who has ever launched a fresh iOS Simulator (before Xcode 6), the OS is in it’s factory state. The Animoto Video Maker App transforms your pictures, video, and text, into professional looking videos… see where I’m going?

Screen-Shot-2015-02-18-at-11.31.04-AM

A lot of user flows depend on having some photos in the camera roll. A factory-fresh simulator without any photos means there are man flows we can’t automate. Unfortunately, at the time of writing, Sauce Labs does not have a way to upload assets to populate the Camera Roll, and the simulators are reset after executing each test case. Starting with XCode 6, the Camera Roll does have some images out of the box, but what was really needed were meaningful pictures and videos. So what is needed is a way to populate the Camera Roll while working within the constraints of running tests in Sauce Labs. Well, the app already reads images and pictures from Camera Roll, what about writing to it as well?

Adding or Altering Configuration Profile

Before we get to actually populating the Camera Roll, we need a mechanism to ensure that this logic is performed only when the intent is to run automation. Out of the box, Xcode provides 3 build configurations (Debug, Release, and Distribution). We can edit these configurations, add new ones, or even delete unnecessary ones. In this case, we can simply add a Test configuration to the mix. Once we did that, we were able to change various build settings to help create hooks for app automation. We can start out by adding a Preprocessor Macro for the Test build configuration, so that we can tell the pre-processor when to compile test hooks into the build.

animoto 2

Okay, now we can do some fun stuff with the build. For the sake of brevity, let’s focus specifically on the original issue: Getting pictures and videos into the Camera Roll.

Change Test Configuration Profile Settings

First things first – how does one get pictures and videos up to Sauce Labs? We can simply add them to the iOS project, but that would increase the size of the application bundle regardless of which build configuration is being used. Definitely not ideal. A better choice would be to store them somewhere externally and copy them to the application bundle when the Test configuration is used. This can be done by running a script when building the project.

[code language=”ruby”]if [ ${CONFIGURATION} == "Test" ]; then
cp -r ${PICTURE_AND_VIDEO_LOCATION}/ ${BUILT_PRODUCTS_DIR}/${PRODUCT_NAME}.app
fi[/code]

Populating the Camera Roll

Now we have an application bundle that contains a bunch of sample pictures and videos. This is great because when the application gets uploaded to Sauce Labs for testing, so do all the sample data.   The following code example assumes all the sample images are in a folder within the application bundle named ‘TestImages’:

[code language=”ruby”]+ (void) populateCameraRoll
{
NSString* absolutePicturePath = [[NSBundle mainBundle] pathForResource:@"TestImages" ofType:nil];
NSArray* pictureList = [[NSFileManager defaultManager] contentsOfDirectoryAtPath:absolutePicturePath error:nil];

for(int i = 0; i < [pictureList count]; i++)
{
NSString* absolutePictureFilePath =[ NSString stringWithFormat:@"/%@/%@", absolutePicturePath,[pictureList objectAtIndex: i]];

NSData *jpeg = [NSData dataWithContentsOfFile:absolutePictureFilePath];

UIImage *image = [UIImage imageNamed:absolutePictureFilePath];

CGImageSourceRef source = CGImageSourceCreateWithData((__bridge CFDataRef)jpeg, NULL);
CFDictionaryRef imageMetaDataRef = CGImageSourceCopyPropertiesAtIndex(source,0,NULL);
NSDictionary *imageMetadata = CFBridgingRelease(imageMetaDataRef);
CFRelease(source);

if (image != nil)
{
ALAssetsLibrary* library = [[ALAssetsLibrary alloc] init];

dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_HIGH, 0), ^{
dispatch_semaphore_t sema = dispatch_semaphore_create(0);

[library writeImageToSavedPhotosAlbum:[image CGImage] metadata:imageMetadata completionBlock:^(NSURL *assetURL, NSError *error)
{
dispatch_semaphore_signal(sema);
}];
dispatch_semaphore_wait(sema, DISPATCH_TIME_FOREVER);
});
}
}
}
@end[/code]

The above code makes a bunch of assumptions. First, it only works for images. Another folder consisting of sample videos can be created, and the using similar logic and the writeImageToSavedPhotosAlbum method. Next, calling writeImageToSavedPhotosAlbum can indeed fail. For the sake of keeping this example code more readable, error handling was excluded. Retry logic should be included if an error is returned.

Finally, you may have noticed the use of a semaphore in the example code. Writing images to the Camera Roll is actually an asynchronous call, meaning that the call returns immediately while a separate thread processes writing the image data to the Camera Roll. The writeImageToSavedPhotosAlbum method can fail if there are too many threads trying to write image data simultaneously. The semaphore is used ensure that images are written to the Camera Roll sequentially. This makes using the writeImageToSavedPhotosAlbum method much more stable.

Okay, so now that is left is to call the method when running the Test configuration. This can easily be done using the Preprocessor Macro setting that was above mentioned.

[code language=”ruby”]#if TEST
[ANTestClass populateCameraRoll];
#endif[/code]

It is recommended to call the method somewhere deterministic (ie -a button tap). Simply populating the Camera Roll at start-up may mess up launching Apples Instrumentation library because of the Alert displayed when the app accesses the Camera Roll for the first time. This lesson was learned the hard way.

This opens up the ability to add more automated test hooks into the application under test, but as a word of warning; the more hooks added, the more the test configuration of the app diverges from what is being released to customers.

 

Have an idea for a blog post, webinar, or more? We want to hear from you! Submit topic ideas (or questions!) here.

Appium 1.3.6 Available on Sauce

March 4th, 2015 by The Sauce Labs Team

Appium logo w- tagline {final}-01The Appium team is pleased to announce that version 1.3.6 is now available on Sauce.  The new build includes changes from 1.3.5, which was previously unavailable for Sauce users.

iOS
– fix for a bug when driver.get() never returns for page with alert.
– iOS 8.2 support.
– fixed safari startup crashes.
– ensure Appium drops into the right continuation cb when selecting hybrid contexts.

Android
– fix XPath regression where Appium failed to recognize non-ASCII characters.
– fix regression where Appium failed to set ADB’s path during Chromedriver tests.
– now finds the location of adb earlier.
– ensure encoding stream in Bootstrap.jar closes correctly.
– add workaround for issue where UiAUtomator fails to find visible elements.
– fixed undefined member error for the release object.
– add a delete key test.

Selendroid
– upgrade to Selendroid 0.13.0.

Vote To Help Appium Win A DeveloperWeek Award

January 9th, 2015 by Bill McGee

Appium logo w- tagline {final}-01Crowd voting for DeveloperWeek awards is open and will continue through January 15th. Appium was thrown into to the mix as a mobile developer tool of choice, so we’d love your help with voting! Appium is an open source, cross-platform mobile automated testing tool that was developed by Sauce Labs and a thriving community of open source contributors.

Voting is easy. Just follow these four steps:

  1. Register HERE to vote;
  2. Then, visit the Appium nomination page HERE;
  3. Click “Endorse” to vote for Appium;
  4. Share with your network.  Here’s some suggested copy:
    • Twitter: Help  win a award! Click here to vote until 1/15: http://ow.ly/H5fog 
    • LinkedIn: Help Appium, sponsored by Sauce Labs, win a DeveloperWeek award! Click here to learn how to vote until January 15th: http://ow.ly/H5fxJ
    • Google Plus: Help #Appium, sponsored by #SauceLabs, win a #DeveloperWeek award! Click here to learn how to vote until January 15th: http://ow.ly/H5fiu #mobile
    • Facebook: Help #Appium, sponsored by #SauceLabs, win a #DeveloperWeek award! Click here to learn how to vote until January 15th: http://ow.ly/H5ftV #mobile

.

We Automated 12 Android Phones To Sing ‘Appy Holidays To You [VIDEO]

December 23rd, 2014 by Bill McGee

With Holiday Season and the close of the fiscal year approaching, we brought in all of our remote workers to spend some time together. We had the obligatory corporate holiday party, a late-night LAN party, and countless, invaluable meetings that laid the foundation for software development projects in the new year.

Our VP of Engineering also arranged our first hackathon, in which we submitted ideas for programming projects that were tangentially related to our business. To participate, we were broken into teams and powered through our ideas and execution over two days.

Some of our developers have been interacting with Android phones, but haven’t had a chance to play with them. Thus, I ended up on a team of three with the goal of getting multiple Android devices to sing in harmony.

We’re happy to share the result of our efforts with this cheerful holiday video.

See the original video here. Follow the conversation on Reddit here and here and on HackerNews here.

So, how does it work?

We went with a pretty quick-and-dirty approach, seeing as this was a two-day event. Having multiple devices sync musical notes in realtime was definitely out of the question, so we opted for seeing if we could coordinate the phones to begin playing their individuals parts of a song at the same time.

Android devices are finicky and difficult to coordinate, many of their operations take varying amounts of time, even on identical devices. Simply pushing a song to each of them and telling them to play, results in some phones playing within a second while others can be up to four seconds behind.

Here’s what we did:

@classam wrote a python script which takes any MIDI file and distributes its separate instrumental parts onto an arbitrary number of tracks. It’s pretty neat: specify just two tracks, and half the parts of the song will be played on one track, and half on the other. If you specify more tracks than available parts, it copies the more significant parts onto the leftover tracks so every phone will feel like it’s a valued member of the choir.

I wrote an Android app which runs on the phones. The app runs on each phone and listens on a socket. When it gets the ‘GO’ command over the USB cable, it plays the music file it’s been given. It also displays a musical visualization of the sound it’s playing so you can match up the phones to the sounds you hear.

@etmillin wrote a javascript program which finds all the Android devices connected to a computer, gets a connection to the song App running on each one, and sends the ‘GO’ command to all of them at once.

When you glue our three pieces together, the following happens:

  • A MIDI song gets broken into parts, one for each phone
  • The parts of the song get uploaded, one to each phone
  • The song-playing app gets started on all the phones
  • The ‘GO’ command gets sent to all the phones at once

Now they all play together!

Other hackathon projects included experimenting with different types of network messaging solutions, devOps management tools, ECMAscript7, contributing to open source projects like Travis CI, and building hardware which displays the state of our continuous integration build.

Be warned, we didn’t polish the code for this blog post but you can find it on github: https://github.com/classam/5rat

-Jonah Stiennon, Ecosystems, Sauce Labs

Want to work at Sauce Labs? Submit your resume! https://saucelabs.com/careers

Test Automation Link Round-Up

November 13th, 2014 by Bill McGee
The Internet was kind to us last week. Here’s a quick round-up of some pieces on automated testing, Appium and other mobile test tools, and Sauce Labs. See below for snippets and links to the full articles.

 

Mobile testing tools are experiencing a growth spurt. New products and services emerge nearly every day, and the time and energy needed to evaluate them may make all the free trials in the world seem not so free. This article looks at several mobile testing tools, listing their benefits, features and tradeoffs to help testers and IT managers make smarter investment decisions. 
Click HERE to read more.

 

This guide provides a compendium of Selenium WebDriver resources to help beginners or advanced users learn about the Browser Test Automation framework. The guide also includes various other platforms and tools that allow you to build out a Test Automation Framework.
Click HERE to read more.

 

Use of automated testing software is gaining popularity for many reasons. Automated software testing enables an organization to leverage technology to perform mundane repetitive tasks efficiently, saving both time and cost. Another side effect of automating your test is that It gives you more time to create tests, increasing overall test coverage. However, these benefits come at a cost. Data shows that about 80% of Automation initiatives fail for various reasons. To ensure that the test automation is successful and offers maximum ROI, one should follow certain best practices. 
Click HERE to read more.
Have an idea for a blog post, webinar, or more? We want to hear from you! Submit topic ideas (or questions!) here.

Re-Blog: Testing JavaScript on Various Platforms with Karma and Sauce Labs

November 6th, 2014 by Bill McGee

Thanks to Ben Ripkins for this great blog post on codecentric!

See an excerpt below.

The web can be a brutal environment with various combinations of browsers and operating systems (platforms). It is quite likely that your continuous integration setup only covers a small portion of your users’ platforms. The unfortunate truth is that testing on these platforms is necessary to accommodate for compliance differences and partial support of standards and technologies like HTML, JavaScript, CSS and network protocols.

Sauce Labs is a service that can be used to test web applications by simulating user behaviour or executing JavaScript unit tests. Sauce Labs eases testing by supporting 442 different platform combinations and additionally recording the test execution. This means that you can actually see what a user might have seen and therefore trace errors easily.

For the purpose of this blog post we will cover the following setup:

  • Sources and tests written using CommonJS modules. Modules are transpiled for the web using Browserify.
  • Mocha as our test framework.
  • Local test execution in the headless PhantomJS browser.
  • Testing against the following platforms in our CI environment:
    • Chrome 35 on Windows 7,
    • Firefox 30 on an arbitrary operating system,
    • iPhone 7.1 on OS X 10.9 and
    • Internet Explorer on Windows 8.1.

(more…)

Sharecare Scales Mobile Automated Testing With Sauce Labs [VIDEO]

September 10th, 2014 by Bill McGee

We took a whirlwind trip to New Haven, CT to sit down with Daniel Gempesaw, Software Testing Architect at Sharecare. Sharecare is a wellness platform founded in part by Dr. Oz and Oprah.

We learned that after Sharecare started automating their testing process while using Sauce, time spent for each deploy fell from 7 days to 1 day. With more free time, the team is able to focus on scaling their mobile testing with Appium and employing new best practices such as mobile CI.

Watch this video to learn how they scaled their mobile testing with Sauce Labs.

(more…)