Mobile Technology for Community Health in Ghana: what happens when technical functionality threatens the effectiveness of digital health programs?
© The Author(s). 2017
Received: 19 June 2016
Accepted: 16 February 2017
Published: 14 March 2017
Despite the growing use of technology in the health sector, little evidence is available on the technological performance of mobile health programs nor on the willingness of target users to utilize these technologies as intended (behavioral performance). In this case study of the Mobile Technology for Health (MOTECH) program in Ghana, we assess the platform’s effectiveness in delivering messages, along with user response across sites in five districts from 2011 to 2014.
MOTECH is comprised of “Client Data Application" (CDA) which allows providers to digitize and track service delivery information for women and infants and “Mobile Midwife” (MM) which sends automated educational voice messages to the mobile phones of pregnant and postpartum women. Using a naturalist study design, we draw upon system generated data to evaluate message delivery, client engagement, and provider responsiveness to MOTECH over time and by level of facility.
A total of 7,370 women were enrolled in MM during pregnancy and 14,867 women were enrolled postpa1rtum. While providers were able to register and upload patient-level health information using CDA, the majority of these uploads occurred in Community-based facilities versus Health Centers. For MM, 25% or less of expected messages were received by pregnant women, despite the majority (>77%) owning a private mobile phone. While over 80% of messages received by pregnant women were listened to, postpartum rates of listening declined over time. Only 25% of pregnant women received and listened to at least 1 first trimester message. By 6–12 months postpartum, less than 6% of enrolled women were exposed to at least one message.
Caution should be exercised in assuming that digital health programs perform as intended. Evaluations should measure the technological, behavioral, health systems, and/or community factors which may lead to breaks in the impact pathway and influence findings on effectiveness. The MOTECH platform’s technological limitations in ‘pushing’ out voice messages highlights the need for more timely use of data to mitigate delivery challenges and improve exposure to health information. Alternative message delivery channels (USSD or SMS) could improve the platform’s ability to deliver messages but may not be appropriate for illiterate users.
KeywordsDigital health mHealth Maternal and child health Frontline health workers IVR messaging
Mobile phones are the leading form of communication worldwide . Their widespread and increasing use, particularly in low and middle income countries where the disease burden is highest, has led to growing calls to harness the potential of mobile and wireless technology to improve health and health care delivery. Mobile-health (mHealth)—defined as the use of mobile and wireless technology for health —aims to improve health outcomes by addressing critical health systems constraints to service delivery, coverage, and utilization .
Throughout the last decade, over 600 mHealth pilot strategies and programs have been introduced globally . Despite the proliferation of mHealth programs, evidence on their effectiveness is limited [5–7] and only modest gains have been observed for interventions aiming to improve provider diagnosis and management, and/or increased user demand through messaging using automated voice or SMS . While calls to address broader evidence gaps in linking digital technologies to outcome and impact level health indicators are emerging, little to no attention has been paid to improving program monitoring, including measuring critical processes on the technological and behavioral performance of the program.
The technological performance aims to determine if the technology platforms performs as intended, whether in aiding program implementation or generating data critical to monitoring and evaluation. Depending on program design, technology platforms may aim to deliver health information messages, alerts and reminders, and/or support the collection of data, including patient-level information on the timing and content of services provided. Variability in how the technology platforms are designed and maintained, including decisions on which data elements are collected, with what frequency, by whom, and for what purpose, greatly influences the consistent delivery of messages and routine monitoring of programs.
The behavioral performance of a program is characterised by the interaction of users, including health care providers and patients, with the technology and program delivery strategy in a dynamic and evolving health systems and community context. For mobile technologies aiming to improve supply side dimensions of service delivery, providers must be willing and able to utilize the technology; a factor which is influenced by perceptions on usability, feasibility and acceptability within a given health systems context . For mHealth solutions which aim to improve demand for and utilization of services through messaging apps, users must not only receive messages (technological performance) which may require negotiating use of the phone if shared (community/household factors) but then decide to open, listen to and/or read them once received (behavioral performance). Whether the technology corresponds to a change in health behaviour and practices requires a complex negotiation of social norms/practices, household and community dynamics, as well as overcoming financial and physical barriers to care.
Collective consideration of the technological and behavioral performance of an mHealth program is essential for accurately measuring program effectiveness and protecting against ‘program drift’—defined as the deviation from intended design in real-world delivery of programs resulting in decreased benefit for patients . In the absence of digital technologies, including mobile phones, data derived from (paper-based) special surveys or health information systems may be used to measure effectiveness, including coverage, or inform process evaluations aimed at assessing fidelity and implementation quality, amongst other factors . The use of mobile and other digital health strategies have the potential to generate data in real-time on essential processes as well as patient careseeking and health status, enhancing not only program implementation and monitoring but evidence needed for summative evaluations.
Study context and population
Ghana is home to a population of almost 25 million disbursed across 216 districts in 10 administrative regions . Over 40% of Ghanaians are under the age of 15 and over half live in urban areas . While child mortality rates have declined by 50% since 1990, for every 1,000 live births 62 children die; 47% within the first 28 days of life . Similar declines in maternal mortality have been observed over the last three decades, however, the lifetime risk of maternal death remains high at 1 in 66, and for every 100,000 live births an estimated 380 women die . Observed declines in mortality have been driven by concurrent increases in the utilization of critical MNCH services, yet gaps in continuity of care during pregnancy, delivery, and postpartum persist. Despite near universal attendance of at least 1 antenatal care (ANC) session and high utilization of 4 or more ANC visits (84%), nearly 30% of deliveries are not attended by a skilled birth attendant, and nearly 20% do not receive postnatal care (PNC) .
Field level implementation of the MOTECH program launched in August of 2010 in Kassena-Nankana West (KNW) district in the Upper Each Region of Ghana, and replicated in 2011 in Awutu Senya District in the Central Region. In 2012, with added funding from USAID and the Bill and Melinda Gates Foundation (BMGF), MOTECH was expanded into new districts: Gomoa West in Central region, Dangme East in Greater Accra region, South Tongu in the Volta region. Administrative division of two districts occurred during implementation; effectively splitting Awutu Senya into Awutu Senya East and West and Dangme East into Ada East and West and raising the total number of program districts to seven across four regions.
Figure 1 provides an overview of the MOTECH program and its two inter-related components: Client Data App (supply side) and Mobile Midwife (demand side).
The Client Data App consists of simplified digital and paper registers consolidating information previously collected in over a dozen paper based registers in Health Centers and Community-based Health Planning and Services (CHPS) facilities. The Client Data App is a supply side intervention which allows facility and community based providers to record all care provided into five simplified paper registers, and digitize clinical care information pertaining to MNCH and other essential care to better track and deliver care to women and infants. Frontline health workers record patient care data onto mobile devices and upload these data to the server (Additional file 1: Figure S1). Data collected by health workers using Client Data App are uploaded into MOTECH’s central database and cross-checked against GHS guidelines on routine care needs for pregnant women, infants, and lactating mothers to trigger a system of alerts about upcoming and missed care sent to both clients in Mobile Midwife program and health workers . These alerts are sent as weekly short message service (SMS) lists containing Mobile Midwife clients with their IDs and care they require. Field teams, composed of Grameen Foundation and district health data staff, routinely monitored and scored health facilities on the accuracy and completion of clinical information uploaded into the MOTECH server compared to care captured in the paper-based registers. Facilities in MOTECH districts that attained 85% completion and accuracy ratings for reporting for three consecutive months were eligible to be ‘automated.’ Automated facilities received auto-generated monthly reports containing aggregate care. This threshold for automation was determined in collaboration with GHS, and based on government standards for data quality.
Mobile Midwife is a demand side component of the MOTECH program which aims to improve client knowledge and awareness of key health information during pregnancy and postpartum period, with the goal of stimulating best practices and encouraging timely and appropriate service utilization. Mobile Midwife is comprised of a maximum of 88 stage-based educational messages and care alerts for pregnant and postpartum women timed to their gestational age or age of their infants, respectively. Educational content was developed based on global and national MNCH guidelines and varied slightly between districts in order to debunk local cultural beliefs and practices. Clients received messages as automated voice recordings in local languages or SMS messages at a day and time of their choosing every week. The voice messaging service was persistent. A weekly message was resent to a registered phone number if a client listened to the call for less than 30 s, either because they missed the call, the device was powered off, there were call congestion issues during message delivery, or if the mobile network was temporarily down. In this case, the system was designed to send that message multiple times over the subsequent two hours, and once every other day until the next message is due.
Mobile Midwife and Client Data App are underpinned by a larger technology infrastructure which includes platform support and server hosting. The MOTECH platform is based on a modular, extensible open-source software comprised of a core platform and several modules, which were developed in 2009 and upgraded in 2012. Specific details on system architecture and evolution over time are available elsewhere, including security aspects and available modules [13, 14]. The software development responsibilities were split between two teams: the server-side components were designed by a group from the University of Southern Maine, and the mobile phone components were designed by a young company based in Ghana . Following the initial development of the overall system architecture, the platform and server hosting were maintained at the Grameen Foundation offices in Accra, Ghana.
Program start-up and implementation
Description of MOTECH program activities and inputs
Description and Inputs
National leadership meetings held among central program leadership, regional and district health management teams
Content development, provider training materials
GF staff worked with GHS Family Health Division and Health Promotion Unit at National level, and with District counterparts to develop messaging content, training materials, and marketing materials
• Voice message program national negotiations and ongoing partnership with Telecommunications Companies
• Infrastructure establishment to manage call system
Platform support and server hosting
Central, Regional, and District staff time allocated to develop the program for each district
Established MOTECH National Steering Committee
Data on health system and telecommunication infrastructure compiled by DHMT for central database
District capacity building
Establish partnerships with Regional and District Health Management Teams
• Establish Regional MOTECH sub-Steering Committee for Greater Accra and Volta regions
• Establish district Technical Working groups
• Placement of 1 full time GF staff in each district and concurrent appointment of one DHMT staff member to serve as “District MOTECH Coordinator” and lead day to day operations. Both staff work under direction of District Director of Health.
Standardization, translation, and testing of Voice health messaging content
Phone purchases for facilities
Customer service referral system for technical or programmatic issues
• Orientation for leadership
• Nurse training on Simplified Registers and technical data entry application,
• Sub-District orientation on program
District Launch events, Durbars and other marketing
• Regional steering committee meeting for program planning in district;
• Technical steering committees in the district meet monthly for 4 months to direct start up
Cost to maintain and use existing vehicles without new capital cost purchase
Central Office space
Airtime for voice messages, Nurse data upload cost
Personnel & Benefits
Grameen, Regional, and District staff time allocated to initiating the program for each district
Platform support, server hosting, system modification to absorb call capacity
DHMT included program tasks in current workflow
Routine data entry application for use in monitoring
Refresher training, Training of new hires
Equipment & Materials
Replacement/resupply of phones and Simplified Registers
Field Office Maintenance
GHS District Office space
Central Office space— Grameen Foundation
Personnel & Benefits
Central, Regional, and District staff time allocated to maintaining the program
Airtime for voice messages, Nurse data upload cost
Data platform, IT technical assistance
Following start-up activities, health facilities began patient registration and enrollment into MOTECH. Providers were encouraged to enter and upload patient level information on a daily basis to facilitate the sending of alerts and reminders. In two districts (Awutu Senya East and Awutu Senya West), high performing facilities were automated and thus given the opportunity to receive monthly feedback reports from Grameen Foundation, summarizing service delivery statistics. To support facility-based activities routine district level meetings were held and supervision visits made to support facility level providers. Where possible, DHMT members sought to include program tasks in their current workflow. However, the Grameen Foundation provided additional program support including device replacement, refresher training, and routine data entry, management and feedback.
Study design and measurement
Technological performance was assessed by exploring the MOTECH platform’s effectiveness in ‘pushing’ out messages across the continuum of care and by content area to registered women. Technological indicators included the proportion of messages ‘pushed out’ out of the total number of messages expected to be ‘pushed’ (Additional file 1: Figure S1 and Additional file 2: Table S1). Behavioral performance was assessed by capturing trends in provider and client engagement with Client Data App and MM, respectively. To measure provider engagement with Client Data App and ascertain the broader feasibility of recording patient-level health information, we assessed the frequency and volume of data uploads over time and by level of facility. To explore client engagement, we identified the proportion of and characteristics of message recipients who listened to at least 50% the length of each message received (defined as ‘active listeners’).
Data collection extraction and analysis
Data from Mobile Midwife and Client Data App were obtained from MOTECH server housed at Grameen Foundation offices in Accra, Ghana. Data elements specific to Client Data App included health facility data uploads, while data from Mobile Midwife included data on health information messages, including messages sent and accessed by pregnant women and mothers of infants. Messaging access was defined by the number of clients who ‘listened’ to stage-based health information messages successfully ‘pushed out.’ Since sharing of phones and phone numbers is common practice in Ghana, many clients were listed under the same phone numbers. For the purposes of this analysis, we extracted details of ‘listening’ for unique phone numbers to facilitate client level analysis. We further excluded registered women who did not have their own or have access to a mobile phone in their household and insteadhad to rely on a community phone to call into the system to retrieve their weekly messages. As part of the Client Data App, facility based providers are required to upload data on the careseeking practices of women registered into MM. Data on the number of uploads per month were analyzed using proportions and frequencies. We used confidence intervals at the 95% level to assess statistical differences in rates of active listening across thematic content areas. Data were analysed using Stata 13.2 for five districts: Awutu Senya West, Awutu Senya East, Gomoa West, Ada East and Ada West. The timeline for implementation varied across sites, and spanned from July 2011 to September 2014 overall.
Client data application
Enrollment into mobile midwife
Characteristics of women enrolled into Mobile Midwife from October 2011 to September 2014 in five districts
April 2013-Sept 2014
Awutu Senya East
Awutu Senya West
July 2011-Sept 2014
Gestational age at enrolment
Number of previous children
2 or more
Age at enrollment
Less than 6 weeks
7 weeks-6 months postpartum
6-12 months postpartum
During the postpartum period, 14,867 women were newly enrolled: 24% within 6 weeks of delivery, 50% between 7 weeks and 6 months postpartum, and 26% between 6 and 12 months postpartum. An estimated 77% of those enrolled postpartum owned a personal mobile phone. Enrollment trends and participant characteristics were similar across geographic areas for postpartum women.
User engagement with messages
Program effectiveness, technological and behavioral performance
(Technological + Behavioral performance)
Number of women expected to receive at least one message
Mean of the proportion of messages that each woman listened to overall (including messages not received)
Number of women expected to receive at least one message
Mean of the proportion of eligible messages pushed to each woman
Number of women who received at least one message
Mean of the proportion of messages that each woman listened to
Messages across the continuum of care
Birth – 6 weeks postpartum
7 weeks - 6 months postpartum
6-12 months postpartum
Messages by thematic areas
Postpartum family planning
Infant care and developmental milestones
Pregnancy care, danger signs
Postpartum care, danger signs
Infant feeding/Nutrition, anemia
Immunizations, hygiene and infection control
To estimate the overall effectiveness of the program, we estimated overall exposure to messaging content by assessing the proportion of messages that each woman listened to out of the total they were expected to receive including those not received (Table 3). Study findings suggest that exposure to messaging content declined from 25% in the 1st trimester to 6% at 6–12 months postpartum. This decline occurred as the number of eligible MOTECH users increased from 1,618 in the 1st trimester to 22,237 at 6–12 months postpartum. Across thematic areas, the mean proportion of women exposed varied from 19% for pregnancy care danger signs to 8% for infant care and development milestones.
Case study characteristics for investigating the technological and behavioral performance of MOTECH in Ghana
Despite the growing use of technology in the health sector, little evidence is available on the technological performance of mobile health programs nor on the willingness of target users to utilize these technologies as intended (behavioral performance). In this case study of the Mobile Technology for Health (MOTECH) program in Ghana, we assess the platform’s effectiveness in delivering messages, along with user response across sites in five districts from 2011–2014.
1. Determine what proportion of expected messages are successfully ‘pushed’ out of the MOTECH platform; and
2. Describe differences in rates of active listening among users across study sites, during pregnancy and postpartum, and across thematic content areas.
The technological and behavioral performance of the MOTECH program in Ghana
System generated data on patient uploads, registration, message delivery and user engagement
Proportions and frequencies; Confidence intervals at the 95% level to assess statistical differences in rates of active listening across thematic content areas.
• A total of 7,370 women were enrolled in MM during pregnancy and 14,867 women were enrolled postpartum.
• While providers were able to register and upload patient-level health information using CDA, the majority of these uploads occurred in Community-based facilities versus Health Centers where RMNCH client loads are higher.
• For MM, 25% or less of expected messages were received by pregnant women, despite the majority (>77%) owning a private mobile phone.
• While over 80% of messages received by pregnant women were listened to, postpartum rates of listening declined over time.
• Only 25% of pregnant women received and listened to at least 1 first trimester message.
• By 6–12 months postpartum, less than 6% of enrolled women were exposed to at least one message.
• While data on the number of individuals enrolled into MOTECH are presented, the true denominator from which these individuals are drawn remains unknown. Further details on the characteristics of pregnant and postpartum women not enrolled into Mobile Midwife are also not available. Research at a household level is recommended to better measure the population level coverage and sustained engagement in the program.
• Client level analyses of active listening were restricted to unique phone numbers and women with their own phones or access to a phone in their household. Given the common practice of sharing phones and phone numbers, this may have introduced selection bias and led to an overestimate of active listening status.
• While we sought to extract data on reminders sent to clients and nurses, these efforts were unsuccessful. Future mHealth platforms should consider ways to improve routine data extraction and use of data to allow for continuous program monitoring.
• Finally, efforts to explore linkages between messaging exposure and careseeking were mired by missing data. Future programs should consider ways to improve the accuracy and completeness in data reporting as well as linkages with existing health information systems which collect data on service utilization and reported practices.
MOTECH’s Client Data App was designed to produce automated reminders and alerts, searchable lists of clients needing care, and in two districts (Awutu Senya East and West) where the program had been implemented without encountering prolonged programmatic or political issues, CHPS facilities received automated monthly aggregate reports for upwards reporting. Findings suggest that health care providers were able to upload health information on pregnant and postpartum women enrolled into MOTECH and that the majority of these uploads occurred in frontline CHPS facilities. Variability in the volume and frequency of uploads were observed across all sites, particularly within the first and last quarters of implementation. Facilities that received automated summary reports tended to have more consistent data uploads, however, the usability and reliability of the data in these reports depended heavily on health workers’ mastering clinical data entry and uploading at a high standard of accuracy and consistency. In practice, this proved difficult for the higher-client-volume health centers and hospitals, where overburdened midwives providing much of antenatal, delivery and early postnatal care, and siloed treatment areas meant digitizing patient level data was a low priority. Moreover, the small size of feature phones, specifically their screen and keypad size, likely contributed to inability to community health nurses and other staff to maintain consistent data uploads. Because of these challenges, none of the district health centers achieved the 85% accuracy and completeness level required to allow meaningful use of automated reports . In contrast, at the community level, health facilities were better able to attain accuracy and completeness level targets as demonstrated in Awutu Senya, where all the facilities achieved the accuracy threshold and nurses even went on to train others in the system. Overall, a mobile-based client data system like MOTECH’s Client Data App can be of real value, particularly at the community-level among health workers that are manageable patient volumes. At the facility-level, maintaining data accuracy and completeness is likely to hinge upon added human resources/support given that certain cadres of health workers are currently overburdened with patient volumes, as well as larger mobile devices for data entry. Strong district leadership, consistent monitoring of data quality, and reliable network access were all essential optimizing the use of Client Data App .
Mobile Midwife sought to improve knowledge, awareness, and utilization of MNCH services through exposure to health information messages, and care reminders and alerts. In contrast to other mHealth programs which have used SMS delivery channels  to push out information messages, the majority of MOTECH participants opted to receive information in local languages through voice delivery channels. Delivery through this modality may improve access and awareness among women with limited to no literacy—an estimated 30% of all Ghanians. Despite this potential, and message delivery system’s ability to be persistent when calls are listened to for less than 30 s, quarterly fluctuations were observed in the proportion of expected messages pushed out of the system and overall, only two-thirds of expected messages were delivered. The precise factors underpinning system failures in message delivery are unknown due to the way the system logs the call failures . However, they are likely due to one or more of the following reasons: inconsistent network connectivity, call congestion in the E1 phone lines as more clients were enrolled in the program, client phones’ being powered off because they were not charged or unable to be charged because of inconsistent access to the power grid, and, finally, technical problems within the MOTECH system .
Message delivery challenges occurred despite extensive efforts to conduct performance testing and upgrade the software and devices used. In 2012, Grameen Foundation significantly revised the open-source MOTECH software platform to expand the range of features and make it more robust for larger-scale deployments by multiple organizations internationally. At this time, performance/stress testing was conducted by sampling 20,000 subscribers and sending messages out to them all at the same time. While findings from the performance testing suggested that the system performed well and was capable of sending 120 messages per minute for SMS and data communications, voice calls were identified as a bottleneck. In particular, the program was constrained by the availability of E1 phone lines to deliver voice messages. Services drew upon two active lines available in Ghana, which together supported 60 simultaneous calls. As a way to accommodate a growing number of clients enrolling into the program, outbound calls were distributed more evenly over the course of a day, and eventually a “queue” system was introduced in the software to ensure that the maximum number of simultaneous calls were not exceeded. Grameen Foundation also housed a customer support center with staff capable to dealing with low tech issues in both Client Data App and Mobile Midwife, and a streamlined process of moving complicated software issues to the tech/IT department. In the first half of the program period, staff received inquires from clients and health workers not just on registration, data uploads, and message delivery, but also on MNCH-related care, something Grameen Foundation staff were not trained to handle. As a result of this, as well as commitment to build district-level capacity to manage Mobile Midwife, responsibilities of client feedback and inquiries were transferred to the DHMTs, and the customer support center only dealt with technological issues with Mobile Midwife raised by the districts. Given that nearly all MOTECH enrollees opted to receive automated voice over SMS messages—a factor which when considered with the number of available phone lines and passive monitoring of message delivery, may explain MOTECH-system related delivery challenges. Future programs should more carefully weigh the options on messaging format given to clients with the feasibility of message delivery based on consideration of the number of available phone lines, message size, and number. Systems for the continuous extraction and assessment of data and simple dashboards tracking message delivery should additionally be instituted from the program’s inception to identify trends in the proportion of expected messages received over time by delivery channels (SMS, voice, USSD), and content area in real-time. Unfortunately, our evaluation occurred retrospectively in 2015 and not in time enough to yield meaningful changes in the MOTECH program or platform.
Beyond the MOTECH platform’s technical performance in message delivery, we sought to derive a measure of overall ‘exposure’ by determining the proportion of women who actively listened to the messages received. To measure the dose of exposure, we defined ‘active listening’ as the proportion of women who listened to at least 50% of content of 50% of messages. While we explored alternative thresholds, we opted for the 50% threshold based on the assumption that individuals would have the capacity to glean the bulk of topical content in that window of time. Future analyses may wish to assume more stringent thresholds of 100% and additionally interview clients to identify whether messages were shared with family members and/or listened to multiple times. Findings on active listening suggested that overall willingness to listen to messages exceeded 60% for all topical areas. However, rates of active listening were observed to decline during the postpartum window which may suggest that there is a ‘novelty’ effect associated with mobile messaging and/or that alternative content formats are needed to rekindle interest (e.g. quizzes, narrative stories, etc.). Future programs may also need to consider an abbreviated number of messages better targeted to address critical gaps in practices and/or careseeking.
To explore the effectiveness of exposure to messaging we initially sought to evaluate the careseeking patterns of women enrolled into Mobile Midwife, including the timing and utilization of services across the continuum of care. However, these data have not been presented because a large proportion of data were missing, and/or unable to be linked to the data on listening. This is part due to the fact that clients move between public and private facilies for MNCH care, and the difficulty of maintaining data uploads in high patient volume facilities, such as health centers and hospitals, where much of that critical care is provided. Future efforts to evaluate the effectiveness of health information messages in bolstering careseeking, should address this critical evidence gap by assessing the effects of specific bundles of messaging based on enrollment during pregnancy or postpartum, by thematic area and across the continuum of care.
This is first analysis of its kind to examine large volumes of back-end data to draw conclusions about the effectiveness of a large-scale mHealth program. Study findings aim to encourage transparency in presenting data on digital health platforms and foster wider dialogue on how to optimize these systems to improve the accessibility and use of data to inform implementation and evidence on their effectiveness. The increased availability of data as part of mHealth programs offers much promise for improving and measuring the exposure of target users to critical program content and allowing for the routine use of patient-level health information at multiple levels of the health system. Study findings highlight importance of real-time program monitoring to protect against ‘program drift’ and ensure the functionality of the underlying technical platform as well as the continued engagement of target users. Efforts to scale-up and sustain the use of mobile phones and other digital devices to improve service delivery in health facilities will need to ensure the adequacy of human resources along with strong district leadership, consistent monitoring of data quality, and reliable network access. Challenges in the delivery of health information messages to pregnant and postpartum women limited the program’s reach and exposure. While over 80% of messages received by pregnant women were listened to, declines in postpartum rates of active listening highlight the need for further research to identify the factors underpinning user engagement over time.
Comparison with other studies
Calls to improve the rigor of mHealth evidence generation and reporting are emerging . Findings from a systematic review of mHealth solutions on coverage and use of ANC, PNC, and childhood immunizations in low- and middle-income countries suggests that there is some evidence of effectiveness at changing behavior . Among specific studies, two RCTs—one in Zanzibar and one in Kenya— which explored the effectiveness of text message reminders and education delivered to pregnant women’s mobile phones, found evidence of statistically significant increases in ANC in their intervention groups relative to their control groups [16, 19]. In Zanzibar, program activities represent one deployment of the Mobile Alliance for Maternal Action (MAMA) project which has additional sites in Bangladesh, Malawi, India, and South Africa and like MOTECH aims to improve MNCH outcomes through the delivery of timed health information messages and appointment reminders. While evaluation activities are still underway for many MAMA deployments, published findings from Zanzibar have emphasized summative findings only; suggesting not only increases in the proportion of women receiving the recommended four ANC visits during pregnancy as well as a trend towards improved quality of care4 . In Kenya, a similar initiative of health information messages and service prompts corresponded to an increase in adherence to recommended ANC visits . In both settings, the absence of details on the technological platforms which serve as the backbone of these programs greatly limits understanding of the factors influencing effects observed, their comparability with alternative resource uses, and generalizability to other settings. While the word count limitations of peer review manuscripts may hinder efforts to report the details of all evaluation activities, robust evaluations should consider the technological performance of the program as a key component of its impact pathway and in particular, its ability to perform as intended. As findings from MOTECH demonstrate, the technological performance of a program is subject to change over time and as scale increases. The failure to monitor this may impact exposure to the program and in turn findings on its effectiveness.
Elsewhere in the literature, a similar pattern of mHealth program evaluations providing limited details on the technological performance of mHealth programs is emerging. Evaluation findings of Text2Floss—a 7-day text messaging intervention which sought to improve oral health behavior and knowledge in the US— suggest that the program led to improvements in knowledge and oral health outcomes . While the characteristics of the automated, multichannel, two-way SMS gateway text messaging service are described, evaluation findings do not provide details on the performance of the technology platform, including messages sent and received out of those expected. Elsewhere in the US, a report on the lessons learnt from a randomized controlled trial of an mHealth smoking cessation program does define the technology functionality by the “pathing of participants from one stage to the next” and report recording these data by the software program, however, no additional data or details are presented . Similarly in Australia, findings from a process evaluation of the TXT2BFIT mHealth program designed to improve weight management in young adults (18–35 years) presented details on the number of text messages delivered and replied to from MessageMedia® web platform; suggesting high rates (98%) of message delivery . However, broader details on the platform were not well detailed . Beyond summative evaluations, qualitative studies have sought to explore user perceptions of mobile tools and/or health information messaging programs in the US [23, 24], New Zealand , and India [26–29]. However, none of these present quantitiative data on the technological performance of the program; instead focusing on reported perceptions of the technology, including feasibility, usability and/or acceptability.
There are several limitations to our analyses (Table 4). Implementation of the MOTECH program relies on an initial period of client registration by health workers. While data on the number of individuals enrolled into MOTECH are presented, the true denominator from which these individuals are drawn remains unknown. Similarly, study findings are limited by the absence of details on pregnant and postpartum women not enrolled into Mobile Midwife. Research at a household level is recommended to better measure the population level coverage and sustained engagement in the program. Client level analyses of active listening were restricted to unique phone numbers and women with their own phones or access to a phone in their household. Given the common practice of sharing phones and phone numbers, this may have introduced selection bias and led to an overestimate of active listening status. Health worker use of the Client Data App is limited in this analysis to absolute numbers on the numbers of patients seeking care uploaded into the system. Details on the frequency and timing of these uploads coupled with the completeness of records is missing. Given the challenges reported with the Mobile Midwife message delivery platforms, it is feasible that similar challenges were present in the uploading of patient level information by health workers.
Future research is recommended to more closely track the flow of data across recipients and levels of the health system over time, particularly within and across districts. While we sought to extract data on reminders sent to clients and nurses, these efforts were unsuccessful. Future mHealth platforms should consider ways to improve routine data extraction and use of data to allow for continuous program monitoring. Finally, with regard to message delivery platforms, considerable challenges were noted in the preceding sections. Beyond these, added details on the length of each message, in addition to the number and format, are needed to formulate recommendations on the optimal data package appropriate for use in a program. Since messaging programs can be costly, it is further recommended that linkages between messaging exposure and careseeking be assessed and the value for money of this component weighed against alternative resource uses.
Digital health programs should not assume that messaging programs perform as intended both with regard to technical functionality and the behavioral responses of end-users. Evaluations of mobile health programs should measure technological, behavioral, health systems, and/or community factors which may lead to breaks in the impact pathway and ultimately, influence findings on program delivery and effectiveness.
Early deployments in 2010 relied on a Nokia 1680c-2 feature phone. These were upgraded to the Nokia 2330c-2 and Nokia C1-01, and later to the Nokia Asha 200 to enhance functionality and usability through features such as dual SIM capacity and a larger screen and full keyboard that made it easier to enter data and reduced entry errors. To replace aging Nokia 2330c-2 s and to test a smartphone (Android) version of the Client Data AppClient Data AppClient Data AppClient Data AppClient Data Application, Samsung Galaxy S4 phones were used in the Upper East region.
Facility-level health workers include Community Health Nurses, Midwives, data entry staff, and health extension workers working at CHPS, Health Centers and Hospitals
Initially, this service also provided support for clients enrolled in Mobile Midwife, however, midway through the program, monitoring and feedback from clients were transferred to the districts as part of program capacity building.
Measured by increasing the identification and referral of antepartum complications, increases in the number of women receiving preventive health services and attending ANC late in pregnancy.
Community health nurse
Community-based health planning and services
Ghana health services
Mobile alliance for maternal action
Maternal, newborn, and child health
Mobile technology for community health
Short messaging service
We are grateful to the support provided by Ghana Health Services throughout the implementation of MOTECH. Funding for MOTECH was provided by the Bill and Melinda Gates Foundation, USAID’s Saving Lives at Birth, and the United Nations Foundation. Funding to support the publication of this manuscript was provided by the Bill and Melinda Gates Foundation. Without their continued support to dissemination of results in all forms, analyses such as these would not be possible. The data contained within this manuscript were generously collected by health care providers with support from District officials in Ada East and West, Awutu Senya East and West, and Gomoa West—we thank them for their participation in MOTECH. The fourth author’s work on this manuscript was supported by the National Institute of Mental Health (K01MH107310). Additional support for the JHSPH team came from the World Health Organization’s Department of Reproductive Health and Research under the United Nations Innovation Working Group Catalytic mHealth Grant Mechanism, managed by the United Nations Foundation with funding from the Norwegian Agency for Development Cooperation (NORAD).
MOTECH in Ghana was a partnership of the Ghana Health Service, Grameen Foundation, and the Columbia University Mailman School of Public Health. The initiative was cofinanced by the Bill & Melinda Gates Foundation; Saving Lives at Birth: A Grand Challenge for Development (SLAB), a multinational partnership of the United States Agency for International Development (USAID), the Norwegian Ministry of Foreign Affairs, Grand Challenges Canada, the UK Department for International Development (DFID), the Korea International Cooperation Agency (KOICA), and the Bill & Melinda Gates Foundation; and the United Nations Foundation. Technical support to the Monitoring and Evaluation of MOTECH was provided by Johns Hopkins Bloomberg School of Public Health with support from the World Health Organization’s Department of Reproductive Health and Research and United Nations Foundation, under the Every Women, Every Child Innovations Working Group mHealth Catalytic Grant Mechanism.
Availability of data and materials
The datasets supporting the conclusions of this article are available upon individual request from the Grameen Foundation Ghana. Please contact the corresponding author for additional queries.
Evaluation efforts were conducted retrospectively by the Grameen Foundation and Johns Hopkins Bloomberg School of Public Health and with support from the World Health Organization. KR is the overall study PI. AM and AEL conceived the idea for this analysis with KR, DH and DM. Data were provided by DH and analyzed by DM and AEL with inputs from AM, LJ, AL, and GM. AEL wrote the first draft of this manuscript with inputs from DM, KR, AM, LJ, AL, and GM. All authors approved the final manuscript.
The authors declare that they have no competing interests.
Consent for publication
Ethics approval and consent to participate
Given the secondary nature of this analyses, JHU faculty leading this analysis were considered exempt by the institutional review board at the Johns Hopkins Bloomberg School of Public Health in Baltimore, Maryland, U.S.A. The Ghanaian Ministry of Health in Accra, Ghana also approved the study’s activities as part of routine monitoring and evaluation. Users’ registered into MOTECH consented to the collection of patient level health information at the time of registration into the program, however, these data are not presented.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
- ICT. The world in 2014: ICT facts and figures. In: International Telecommunication Union, ICT Data and Statistics Division. Geneva: Telecommunication Development Bureau; 2014.Google Scholar
- WHO. eHealth Tools and Services: Needs of the Member States. Report of the WHO Global Observatory for eHealth. Geneva: World Health Organization; 2006.Google Scholar
- Mehl G, et al. In: Marsch L, Lord S, Dallery J, editors. Harnessing mHealth in Low-Resource Settings to Overcome Health System Constraints and Achieve Universal Access to Healthcare in Healthcare and Technology: Using science-based innovations to transform practice. New York, New York: Oxford University Press; 2015.Google Scholar
- Cargo, M., South Africa mHealth Landscape. GSMA. 2013. http://www.gsma.com/mobilefordevelopment/programme/mhealth/south-africa-mhealth-landscape.
- Kallander K, et al. Mobile health (mHealth) approaches and lessons for increased performance and retention of community health workers in low—and middle-income countries: a review. J Med Internet Res. 2013;15(1):e17.View ArticlePubMedPubMed CentralGoogle Scholar
- Free C, et al. The effectiveness of mobile-health technologies to improve health care service delivery processes: a systematic review and meta-analysis. PLoS Med. 2013;10(1):e1001363.View ArticlePubMedPubMed CentralGoogle Scholar
- Free C, et al. The effectiveness of mobile-health technology-based health behaviour change or disease management interventions for health care consumers: a systematic review. PLoS Med. 2013;10(1):e1001362.View ArticlePubMedPubMed CentralGoogle Scholar
- Rothstein JD, et al. Qualitative Assessment of the Feasibility, Usability, and Acceptability of a Mobile Client Data App for Community-Based Maternal, Neonatal, and Child Care in Rural Ghana. Int J Telemed Appl. 2016;2016:2515420.PubMedPubMed CentralGoogle Scholar
- Chambers, D., R. Glasgow, and K. Stange, The dynamic sustainability framework: addressing the paradox of sustainment amid ongoing change. Implement Sci, 2013. 8(117). http://implementationscience.biomedcentral.com/articles/10.1186/1748-5908-8-117.
- Moore G, et al. Process evaluation in complex public health intervention studies: the need for guidance. J Epidemiol Community Health. 2014;68(2):101–2.View ArticlePubMedGoogle Scholar
- Service, G.S., G.H. Service, and I. International. Ghana Demographic and Health Survey 2014, GSS, GHS. Rockville, Maryland: ICF International; 2015.Google Scholar
- A decade of tracking progress for maternal, newborn and child survival: the 2015 report for Ghana, in Countdown to 2015: Maternal, newborn and child survival. 2015.
- Foundation, G. Using mobile technology to strengthen maternal, newborn, and child health: a case study of MOTECH's five years in rural Ghana. Washington, DC: Grameen Foundation; 2015.Google Scholar
- MOTECH. MOTECH core architecture [cited 2016 January 15]; Available from: http://docs.motechproject.org/en/latest/architecture/core_architecture.html.
- Crowe S, et al. The case study approach. BMC Med Res Methodol. 2011;11:100.View ArticlePubMedPubMed CentralGoogle Scholar
- Lund S, et al. Mobile phones improve antenatal care attendance in Zanzibar: a cluster randomized controlled trial. BMC Pregnancy Childbirth. 2014;14:29.View ArticlePubMedPubMed CentralGoogle Scholar
- Agarwal S, et al. Guidelines for reporting of health interventions using mobile phones: mobile health (mHealth) evidence reporting and assessment (mERA) checklist. BMJ. 2016;352:i1174.View ArticlePubMedGoogle Scholar
- Watterson, J., J. Walsh, and I. Madeka, Using mHealth to Improve Usage of Antenatal Care, Postnatal Care, and Immunization: A Systematic Review of the Literature. BioMed Research International, 2015. 2015
- Fedha T. Impact of Mobile Telephone on Maternal Health Service Care: A Case of Njoro Division. Open J Prev Med. 2014;4:365–76.View ArticleGoogle Scholar
- Hashemian TS, Kritz-Silverstein D, Baker R. Text2Floss: the feasibility and acceptability of a text messaging intervention to improve oral health behavior and knowledge. J Public Health Dent. 2015;75(1):34–41.View ArticlePubMedGoogle Scholar
- Ybarra ML, et al. Process evaluation of a mHealth program: lessons learned from Stop My Smoking USA, a text messaging-based smoking cessation program for young adults. Patient Educ Couns. 2014;97(2):239–43.View ArticlePubMedPubMed CentralGoogle Scholar
- Partridge SR, et al. Process evaluation of TXT2BFiT: a multi-component mHealth randomised controlled trial to prevent weight gain in young adults. Int J Behav Nutr Phys Act. 2016;13(1):7.View ArticlePubMedPubMed CentralGoogle Scholar
- Asgary R, et al. Perceptions, Attitudes, and Experience Regarding mHealth Among Homeless People in New York City Shelters. J Health Commun. 2015;12:1473–80.View ArticleGoogle Scholar
- Cannuscio CC, et al. A behavioral economics intervention to encourage epinephrine-carrying among food-allergic adults: a randomized controlled trial. Ann Allergy Asthma Immunol. 2015;115(3):234–40. e1.View ArticlePubMedGoogle Scholar
- Kool B, et al. Perceptions of adult trauma patients on the acceptability of text messaging as an aid to reduce harmful drinking behaviours. BMC Res Notes. 2014;7:4.View ArticlePubMedPubMed CentralGoogle Scholar
- Nguyen L, et al. Perceptions of data processes in mobile-based versus paper-based health information systems for maternal, newborn and child health: a qualitative study in Andhra Pradesh. India BMJ Innov. 2015;1:1–7.View ArticleGoogle Scholar
- Ganapathy K, et al. m-Health: A Critical Analysis of Awareness, Perception, and Attitude of Healthcare Among Providers in Himachal Pradesh, North India. Telemed J E Health. 2016;22(8):675–88.View ArticlePubMedGoogle Scholar
- Sneha P, et al. A pilot study to assess perceptions of using SMS as a medium for health information in a rural setting. Technol Health Care. 2014;22(1):1–11.Google Scholar
- Smith, S., et al., Potential for the use of mHealth in the management of cardiovascular disease in Kerala: a qualitative study. BMJ Open. 2015;55(11):e009367. doi:10.1136/bmjopen-2015-009367.