This study was performed to review the most frequently used questionnaires for evaluating satisfaction, usability, acceptance, and quality outcomes of mHealth services. Usability is the most evaluated outcome in the mHealth studies. SUS, PSSUQ, and CSUQ were the top three most used questionnaires for evaluating the usability of mHealth services, respectively. The two most used questionnaires for evaluating the quality of mHealth applications were MARS and uMARS. In addition, TAM and UTAUT were the most used questionnaires for measuring the user acceptance of mHealth services. The three most used questionnaires for evaluating user satisfaction were NPS, CSQ, and GEQ.
Usability evaluation questionnaires
The present study showed that SUS questionnaire had been used much more than similar questionnaires such as PSSUQ and CSUQ in evaluating the usability of mHealth services. SUS is a general questionnaire that is used for evaluating the usability of electronic systems such as mobile devices. Compared with other questionnaires such as CSUQ, SUS is a quicker tool for judging the perceived usability of systems because it has fewer items with less scale pointing. This questionnaire also includes a question regarding the satisfaction of the user with the digital solution. The satisfaction evaluation questionnaires focus on tools that evaluate only this outcome, but it is also contained in the usability outcome [18, 19]. Because of these features and its reproducibility, reliability, and validity, researchers and evaluators of mHealth services have frequently used the SUS questionnaire. Another study that reviewed the most used questionnaires for evaluating telemedicine services also showed that SUS is the most used general questionnaire after the Telehealth Usability Questionnaire (TUQ), which is a specific questionnaire for evaluating the usability of telemedicine systems [35].
Although MAUQ was specifically designed for evaluating the usability of mHealth applications and considered both interactive and standalone mHealth applications [17], it was rarely used in the studies that were included in our review. This lack of use might be due to the fact that MAUQ was introduced 2 years ago, and researchers are less familiar with this questionnaire. It is recommended that researchers and evaluators of mHealth services use such questionnaires that were specifically designed for evaluating these services.
Quality evaluation questionnaires
MARS and its user version (uMARS) were the most used questionnaires for assessing the quality of mHealth applications. To use MARS for evaluating mHealth applications, users should be professional in mHealth. Because of this limitation, uMARS was designed to be administered by end-users without special expertise. The importance of the quality and reliability of information and content provided in mHealth applications and the impact that this content has on people's health led to the design of MARS [12]. MARS prompted researchers to look at another consequence of mHealth, which significantly impacts the practical and safe use of mHealth applications. This issue has led to the use of these questionnaires in many studies.
Acceptance evaluation questionnaires
This study revealed that TAM and UTAUT were the most used questionnaires for measuring mHealth acceptance. These questionnaires were derived from two models with the same name. Generally, TAM and UTAUT are the most used acceptance models in health informatics because of their simplicity [36]. Both models focus on the usefulness and easy use of technology. Since UTAUT derives from eight models such as TAM, it evaluates two additional factors, including social environment and organizational infrastructure, that may impact the adoption of the new technology [36]. However, since TAM and UTAUT have not been developed in healthcare settings, different emotional, organizational, and cultural factors that may influence technology acceptance in healthcare settings are not covered by these two questionnaires [23, 30]. Therefore, researchers in health informatics would better design the acceptance questionnaire based on the objective systems.
Satisfaction evaluation questionnaires
The present research revealed that NPS is the most widely used tool for measuring the satisfaction of m-Health users. NPS is a very small tool for evaluating client satisfaction. This tool only has one question [25]. The fact that this scale has only one item has probably contributed to its wide use. It should be taken into account that a single question cannot identify the various factors that affect user satisfaction with a service. After NPS, CSQ and GEQ were the most used questionnaires in reviewed articles. CSQ has two characteristics that may affect its usage. The first one is that it considers the quality of different aspects, such as procedure, environment, staff, service, and outcome. The second characteristic is that with this comprehensiveness, this questionnaire has only eight items [29]. Studies that used mobile-based games to provide mHealth services used GEQ [37, 38] because it is a specific, comprehensive, and practical questionnaire that measures game user satisfaction [34]. Melin et al. presented a questionnaire for assessing the satisfaction of mHealth applications users [39]. However, none of the papers included in our study used this questionnaire because this is a new tool, and researchers are less familiar with it. It is recommended that researchers in mHealth, use this specific questionnaire in their future studies.
Evaluation outcomes
Most of the included papers evaluated the usability of mHealth services using a questionnaire. Usability is a critical issue that affects willingness to use a system. Therefore, it is essential to evaluate this outcome in different phases of system development. The questionnaire is the most used method for evaluating the usability outcome of a mobile application because of its simpleness in terms of accomplishment and data analysis [17]. A review study also showed that the usability of mHealth applications is mostly assessed using a questionnaire [40]. Another study revealed that questionnaires were mostly used for evaluating the user satisfaction outcome of telemedicine [35]. The differences between the results of our research and those of this study may be due to the fact that mHealth services are mostly presented with an application; therefore, evaluating the user interface of the application is very important and should be considered for effective use [40].
Limitations
To the best of our knowledge, this is the first study that reviewed the most used questionnaires for evaluating the satisfaction, usability, acceptance, and quality outcomes of mHealth services. Nevertheless, this study has some limitations. We only searched the PubMed database to retrieve relevant papers. Also, we restricted our search to the Title/Abstract field. Moreover, we excluded review papers and only included the app review studies that use MARS. These limitations may have led to the missing of some papers from our study.