What makes quality testing for mobile devices challenging?

With the advent of responsive Web design, most people can browse on their mobile devices and see a website that appears very similar to their desktop browser. Typically, a mobile site looks simpler than its desktop counterpart, giving the appearance that testing mobile applications is easier than testing of other software. However, there are certain challenges that every mobile app quality tester faces that make this type of testing special and difficult in its own way.

Quality Testing for Mobile Devices

Variety of Devices

In a recent study on the diversity of hand-held devices, it is estimated that, by the end of 2013, there were more mobile devices than people in the world. In 2013, over 12,000 different mobile devices were in use by consumers. This year, it is estimated that the number will rise to well over 20,000. As you can imagine, hand-held devices differ greatly in their features, screen sizes, resolution, hardware capabilities, and input methods (for example, touch, slide, QWERTY).

Diversity in Platforms and OS

The market also features a wide variety of mobile operating systems. The major ones include Android, iOS, Windows Phone, Symbian, and BlackBerry. Naturally, each operating system has its own benefits and limitations. That’s why testing even one website with so many devices running on different platforms poses a great challenge for any tester.

Mobile Network Operators

According to Wikipedia, in the United States alone, there are over 75 wireless network operators. Some of these networks use older technology, such as CDMA or GSM. Each network operator utilizes a different kind of network infrastructure, which can limit the flow of information. Network performance can also have a powerful impact on the user experience with mobile browsing.

Evolution of OS Versions

If you look at the difference between Android’s 1.0 and 4.4 Kit Kat versions or iOS versions such as iPhone OS 1.0 and iOS 7.1, you’ll see that it’s really big. Version control is yet another great challenge for mobile app publishers and testers.

Fragmentation of the Green Robot

Android Device Fragmentation

Image of Android device fragmentation courtesy of OpenSignal.com

Earlier, I mentioned the variety of mobile operating systems on the market, but even within individual operating systems, there is a great deal of diversity. Nowhere is this more evident than in the mobile OS by Google: Android. The website OpenSignal.com provides a report each year about the fragmentation of Android. Some of the highlights of this report include the following:

  • There are hundreds of manufacturers of phones that carry the Anrdoid OS, the largest being Samsung.
  • Samsung is also the largest brand to carry Android phones, with 43% of the mobile market.
  • There are currently 19 different versions of the Android OS, the newest being 4.4, Kit Kat.
  • There is a clear correlation between fragmentation and GDP per capita.
  • The proliferation of sensors in Android devices allows for increasingly sophisticated apps and networks.
  • There are almost 100 different physical screen sizes for Android devices vs. just four for iOS.

One Solution: Automation

Since most devices vary in input methods, keystrokes, display properties, and menu structure, a single test script won’t function on all devices. Because of this, it is important to leverage the power of automation. With this powerful tool, testers can, to a certain extent, automate testing practices, known as smoke testing or regression testing. A variety of mobile testing applications are available that replicate human interaction to make life easier for the tester.

As you can see, quality testing for mobile devices is no walk in the park. Fortunately, compared to the PC industry, people tend to upgrade their mobile devices more frequently. This makes it easier to target the newer models, operating systems, and mobile browsers.

Share with others!Share on Facebook0Tweet about this on TwitterShare on LinkedIn0Share on Google+9Share on Reddit0Email this to someone

Kenny Withers

I am a blogger, strategist and speaker who works with companies to optimize their online personal and company presence, brand, internet marketing and social media marketing. This blog covers topics related to Social Media, Digital Marketing, Blogging, Twitter, Facebook, YouTube, LinkedIn, Search Engine Optimization (SEO), Search Engine Marketing (SEM), Brand Marketing, and Content Marketing.

1 Response

  1. Bobby Gindehall says:

    Saved your site in my bookmarks =). I’m shocked that your web pages are not all over the web! Anyway, thanks for the great read.