The basics of mobile application testing - where to start?

Jul 19, 2022, Mobile

Marcin Urban QA Specialist

Have you ever stopped using a mobile application with an unexpected incoming call? Or had a phone with a low screen resolution and couldn’t see the whole text while using the app? Or maybe you were driving through the wilderness without internet connection - how did the application behave then? All the above-mentioned situations - as well as many others - should be foreseen and verified by the mobile application tester at the testing stage during software development.

In this article, I’ll discuss the specifics of mobile application testing, with particular emphasis on the basic issues that are extremely important for a tester who wants to develop in this type of testing.

What should a mobile application tester know?

The most important thing is that there are many differences between mobile application testing compared to web app ones. That’s why you need to familiarize yourself with their characteristics. Therefore, a mobile tester should know:

ways to download / install the mobile application for a given platform – if you’re new to mobile testing, here’s a bit of a warning: for tests other than the verification of the production version of the mobile application, it will not be a GooglePlay store, AppStore or AppGalery. You will more often come across other tools for distributing mobile apps such as Bitrise, Firebase / Apptester, building the mobile application yourself based on the code or directly installing the .apk file on the device.
types, number and distribution of operating system versions – currently the main ones are iOS, Android and HMS. Others have a low market share, have ceased to be supported, or are used only in specific markets, industries, etc.
selling device types and existing versions of OS in a given area and quantitative statistics of amount used devices there
different screen sizes and resolutions
different methods of data entry

Types of mobile apps

We already have an introduction and we know how to download / install mobile apps on a mobile device. Now, let’s focus on the types of applications we distinguish:

Native mobile apps

Native apps are created using platform-specific programming languages (Android – Kotlin, Java, iOS – Objective-C, Cel C and Swift). They usually provide better performance compared to other solutions by taking full advantage of the sensors and functions of the platform. However, the cost of their development is usually higher due to the fact that new functions must be implemented separately in each code base, i.e. for each platform separately.

Browser-based web apps

A mobile app is accessed via a browser. It looks like a responsive website (RWD), and the functionalities are usually very limited and dependent on the browser capabilities.

Ways to build browser-based applications:

Mobile versions of websites and applications. Most often, a mobile browser refers to the mobile version of the application, e.g. olx.pl, redirecting to m.olx.pl. They are often “stripped” of the functions available, for example, on a computer.
Adaptive / responsive web apps adjust the design according to the size or device of the user. Different devices have other application view, e.g. a website displayed with several columns on the computer, with two on the tablet and one on the smartphone. Compared to mobile versions of a website, responsive versions of a website should offer the same content and functionality for different devices, although there are some minor exceptions.
Progressive web applications create shortcuts to certain web pages on the screen. Any platform which use standard browser can be used to run that type of app. Progressive web app can work offline, it is fast and can use some features of user device.

Hybrid apps

They provide the possibility of using the device’s potential better than browser-based applications. They enable access to native phone functions, such as the camera, which are limited or impossible to access via the web app. However, hybrid mobile applications do not use the full potential of the device like native ones. One example is the offline mode, which is not 100% maintained in the case of hybrid mobile apps. On the other hand, the advantage of a hybrid mobile application is certainly the fact that it can work on many platforms from one source of code. It can be said that a hybrid mobile app is a compromise between the universality of the solution, the costs of production and maintenance, users’ habits to native solutions, use of resources and efficiency, combining native and web solutions.

Check out the 10 useful Chrome plugins for testers

Problematic issues that may occur during mobile app testing efforts

As I mentioned at the beginning, testing mobile devices is different from testing web apps. Here’s a list of problematic issues that may occur during mobile application testing, affecting their scope:

Mobile devices are diverse, as are mobile application distribution channels, types of apps themselves, and ways of their use by users.
There are individual system settings of users that allow for relatively large personalization of selected elements, e.g. font size, color brightness, etc.
The market of mobile devices and the most frequently used software versions or overlays is diverse – it is worth using statistics on user preferences in a given region for which the application is created, e.g. the most frequently used brands, operating systems, etc.
Hardware / resource differences across devices, UI designs, screen sizes.
Diversity of development tools required / supported / provided by platforms / producers.
Availability of physical devices, emulators, simulators or the use of external test laboratories.
The introduction of new technologies, devices and / or platforms during the application lifecycle.
Installing and updating the application itself, storing data and settings.
The range of the mobile Internet network, its quality, changes of transmitters, location.
Quick feedback from users resulting from errors, publication of feedback on the application.
Time and process of publishing the application on the given stores.
Maintaining the state of the logged in user or correct logging out of the user after a given time.
Support for animation on various platforms.

Programming platforms and selected tools that can support mobile application testing

We use programming platforms to create mobile applications. In the case of Android, we use Android Studio, while in iOS – Xcode. Programming platforms often contain tools that are helpful not only in typical developer tasks, but can also support application development and mobile testing. They have many useful built-in tools, such as taking screenshots, extracting device / application logs, sending random events and notifications to the device, monitoring various parameters such as: memory and CPU usage, http traffic tracking, viewing the layout implemented in the application and creating virtual devices. These tools include Android Virtual Device (AVD) Manager, Android Debug Bridge (ADB), Android Device Monitor, Instruments for iOS, Layout inspector and Logcat.

Some interesting tools found outside of programming platforms that can facilitate mobile testing are also:

Flipper – a platform for debugging iOS, Android and React Native apps, with many plugins for logs, error reports, database browsing, network traffic and layout.
Charles proxy – a tool for viewing, monitoring and editing network traffic also from mobile devices.
Postman / Insomnia / Soap UI – in the context of testing mobile applications, these programs are useful for verifying data returned to the app from the server.

Is that all? Of course not. The spectrum of tools useful in mobile application testing is very large, and specific tools are selected based on the current needs and the characteristics of the project. It is also worth noting that the development tools – hidden by default on the Android platform, available for iOS from the xCode level – are very useful and it is worth taking a closer look at them. However, this is an extensive and broad issue which needs a separate article. Due to the fact that the current post is about the beginnings of mobile app testing, I also do not include tools useful in mobile test automation.

Mobile application testing types

According to the ISTQB syllabus, there are a great number of types of mobile application testing. In my opinion, these are the most important ones:

Testing user interface

There are many different screen sizes, diagonals, resolutions in pixels per inch and dots per inch on the market. Using the information about the most frequently used ones, and taking into account their diversity, it is worth checking:

scaling interface elements / images of different resolutions and whether they do not overlap,
if there is no problem with the selection of displayed options,
whether the arrangement of elements is correct after changing the screen orientation, the application is still functional, the previously entered data do not disappear, etc.
changing the text size in the system settings,
so-called multitouch tests, single touch, short press and long press – if the application supports these functionalities and if they work properly.

Testing device input sensors in the context of application, connectivity

Mobile devices are equipped with many sensors such as GPS, thermometer, light sensor, accelerometer, etc.

In this test category, it is worth checking if the application works correctly for the sensors used, e.g. movement in different planes, different light intensity, sound, response to changes in settings, displaying the exact location when using Wi-Fi, cellular data, after loss and restoration of network connections and cellular data, network changes, whether the application uses biometric recognition correctly, NFC payments.

Interrupt testing

Modern smartphones have many functions performed in real time when we use the application. It is worth checking how the application behaves in typical interrupts, such as:

during incoming and outgoing calls / messages,
in the absence of network / internet access,
when connecting the charger,
during “Do not disturb” mode, etc.,
after turning off the smartphone or updating the operating system while the application is running.

Notifications testing

Verification includes correct operation of notifications in various situations (application in the foreground, in the background, closed, in “Do not disturb” mode, running on low battery) and checking whether the notification relates to a specific place in the application to which it redirects a user after selecting it.

Testing the behavior in the system settings application

Testing whether applications adhere to the settings for sound level, brightness, dark mode, text size, power saving, languages, do not disturb mode, etc. on the smartphone.

Installation testing

Installations from stores, by copying the file to a mobile device and installing from it. In addition to the regular installation / uninstallation / update, you can check e.g. canceling / interrupting / resuming, as well as verifying whether, e.g. after updating, the application data remains and it is possible to continue using it.

Stress / performance testing

Application speed tests should be performed in case of: high CPU load, little phone disk space, high memory usage, battery load, and many interactions with the application.

Usability, accessibility, input and functionality testing

With particular emphasis on verifying that the application:

meets the business assumptions,
allows the user to make mistakes,
is intuitive and consistent,
displays correctly data with e.g. enlarged font size,
allows many social groups to use it, such as disabled, left-handed, etc.
works with the most popular keyboards,
supports gestures, captures images, photos, scans codes, etc.

Post-release verification testing

Verification of the correctness of the release by downloading, installing and – if possible – verifying the application in the production environment.

We don't have the right number of physical devices? - simulators and emulators

The terms simulator and emulator are often misused interchangeably. Simulators mimic the system in which they run the application. Mobile apps tested on simulators run in the simulator, they do not run on the real device, so you cannot check functions such as battery usage, interrupts, etc.

Mobile apps compiled for deployment and testing on the emulator are compiled into real bytecode which can also be used by the actual device. So the emulator will run an exact copy of the operating system. The emulator models the hardware, it uses the same environment as the physical hardware, so it takes into account RAM, CPU, battery performance, etc. For device emulation you can use, for example, Android Studio, and Xcode for simulation.

Mobile app testing - is it the right job for you?

I hope that the knowledge I shared has helped you clarify and organize some of the issues related to the beginnings of working in the field of mobile application testing. It was mostly about industry or technical issues, but you’re probably asking yourself a question: ‘Is it really for me?’. Let’s try to answer it by reviewing the soft features / skills that are useful in the professional mobile app testing process.

Communicativeness, clarity in asking questions and cooperation

Testers communicate with team members on a daily basis. They must do it clearly and precisely. Try to imagine a programmer’s face receiving a message like, “I go to the mobile application, select a tab and it doesn’t work.” Exactly … that’s why the clarity of asking questions, description, the ability to briefly but readily describe the error is very important.

Coaching and assertiveness

A tester should promote the qualitative thinking model among the entire project team, suggest process and application improvements, make developers aware that they are responsible for their code and should also test it. Apart from that, he should be able to substantively and assertively conduct talks with other members of the project team or other stakeholders of the software being developed.

Creativity and curiosity

Mobile application testers need to be creative and curious about the behavior of the application in different situations. Possible functional use cases, the amount of equipment that can be used, or the way of using the device according to one’s own needs, create a great number of possible options. Creativity helps you find more defects that, when corrected, result in better software quality.

Analytical thinking

Testers need to understand and analyze the technical aspects of the solution and be able to apply them in mobile testing. They will help you understand the mobile app and know when something is not working as it should.

Other

The remaining features, the influence of which does not need to be further described and explained, include: awareness of continuous learning, critical thinking, effective time management and the ability to prioritize tasks.

So, answering the question: ‘Is it really for me?’, remember that nobody is born with a ready-made package of skills, knowledge and predispositions, and acquiring competences requires time, willingness, commitment and hard work. Nobody is perfect and not everyone has all of the above-mentioned features, but that doesn’t mean they aren’t good testers. Some features can be overcompensated with others. It is also worth remembering that both technical and soft skills can be worked on and developed along with experience. Good luck! 😊