Some people might opt to test every mobile device against every Android version (for a total of nine runs) — but that’s overdoing it, according to BrowserStack blogger Hylke de Jong in a series of articles on testing for fragmentation.
“Don’t get too hung up by device names, either. If, for whatever reason, one isn’t available, move onto a mobile device with the same screen resolution (not size), since that’s our differentiator,” he says.
The BrowserStack blog series takes a look at market data on devices, platforms, browsers and so on in use today, how that diversity comes into play during software development and testing—and what the more than two million developers on BrowserStack do to account for it.
Jong talks about building an ideal test strategy across browsers and devices.
“Everyone who has been involved with testing apps or web applications on mobile browsers knows the trouble different platforms and browsers can cause,” he says. “Things that work perfectly fine in one mobile browser, break down suddenly and spectacularly in another one (I’m looking at you, IE11).”
The first step, he notes, is to understand which mobile devices and versions of Android, for example, are used most by customers.
“Screen size might sound like an obvious thing to look at, but actually isn’t a great differentiator. Your app will look the same on a 5in, 5.5in or 6in screen, as long as they all use the same resolution (e.g. 1920×1080),” he says.
“Therefore, it’s better to look at screen resolution of the mobile phones.”
For web testing, more factors come into play: different operating systems, different browsers, a wider array of resolutions, etc. Take a hard look at the extent of fragmentation, and combine it with analytics data to see where the differentiators are, he suggests.
“For instance, is the latest version of Chrome really that different on Windows and macOS, or is browser behavior the same regardless of operating system?”