All Android developers should have at least 1 “old” device running OS 2.3.3 and a current “popular” device. Ideally, one should also have a current device that is considered “maxed out” on specs. A company should additionally have the latest “Google” device (currently the Nexus series), and an HTC, Sony, and Samsung device. (These manufacturers are mentioned because of popularity and/or significant differences not found when developing on other devices.) Additionally, OS 4.2, 4.3, and 4.4, though minor OS increments, offer differences that should be considered.
Though development for iPhone/iPad is more forgiving given the fewer configurations, it still offers challenges. For example, if you are developing on a Mac running OS X Mavericks with a version of Xcode above 5.0 for a product that still needs to support iOS 5.x, you will need a physical device because the iOS 5.x simulator isn’t available for that development configuration.
If testing mobile websites, the configurations can be endless.
At Apps World 2014, Perfecto Mobile (http://www.perfectomobile.com) introduced me to mobile cloud device testing. Their product offers access to real devices (not emulators or simulators) connected to actual carriers physically hosted at one of their sites around the world.
The concept of mobile cloud device testing allows the ability to test on a multitude of configurations of devices, locations/timezones, carriers, and operating systems.
Beyond access to multiple devices, Perfecto Mobile offers automation testing across these platforms via scripts written in Java. I wasn’t able to personally delve as far as I wanted into these automation tests, the recording feature, or the object mapper before my trial ran out, but the demo at Apps World gave me the impression it behaves similar to Xcode’s Automation instrument but expanded to all devices. The scripts enable your team to target certain device configurations and automatically launch, execute the given tests, clean and close the devices, and export the test results to your team. I wish I could say more because it looked really promising but without actual usage, I can only mention what I viewed during the demo.
It’s impossible to cover every configuration during native Android application development, but after a release, for all platforms, if your product is experiencing issues and a crash report doesn’t reveal enough, mobile cloud device testing offers the a real option for true coverage.
Below is a listing of some features of interest Perfecto Mobile offers:
- Access to real devices connected to actual carriers (not emulators or simulators) physically hosted at one of Perfecto’s sites around the world. Since these are real devices, you can dial numbers, make calls, send text messages, and install apps.
- UI for devices available displays availability, manufacturer, model, os + version, location, network, phone number, device id, firmware, resolution.
- Ability to open multiple devices at the same time.
- Requests for devices and configurations not available are responded to in real-time.
- Ability to take screenshots and record sessions to save and/or share results with others.
- Ability to share the device screen in real-time with your team.
- Ability to collect information for a device such as battery level, CPU, memory, and network activity.
- Export of device logs.
- Beta of MobileCloud for Jenkins plug-in that allows running scripts on actual cloud devices after a build so you can see reports on a single device after a build (multiple devices is not available yet).