Skip to main content

Advertisement

We're creating a new version of this page. See preview

  • Research article
  • Open Access
  • Open Peer Review

Development and evaluation of a mobile application for case management of small and sick newborns in Bangladesh

  • 1,
  • 2,
  • 3,
  • 1,
  • 2,
  • 2,
  • 1,
  • 4,
  • 5,
  • 5,
  • 2,
  • 6,
  • 7 and
  • 1Email authorView ORCID ID profile
BMC Medical Informatics and Decision Making201919:116

https://doi.org/10.1186/s12911-019-0835-7

  • Received: 29 May 2018
  • Accepted: 6 June 2019
  • Published:
Open Peer Review reports

Abstract

Background

In low-income settings, community health workers (CHWs) are frequently the first point of contact for newborns. Mobile technology may aid health workers in classifying illness and providing referral and management guidance for newborn care. This study evaluates the potential for mobile health technology to improve diagnosis and case management of newborns in Bangladesh.

Methods

A mobile application based on Bangladesh’s Comprehensive Newborn Care Package national guidelines (mCNCP) was developed to aid CHWs in identifying and managing small and sick infants. After a 2-day training, CHWs assessed newborns at Sylhet Osmani Medical College Hospital and in the Projahnmo research site (Sylhet, Bangladesh) using either mCNCP or a comparable paper form (pCNCP), similar to standard IMCI-formatted paper forms. CHWs were randomized to conduct a block of ~ 6 newborn assessments starting with either mCNCP or pCNCP, then switched to the alternate method. Physicians using mCNCP served as gold standard assessors. CHW performance with mCNCP and pCNCP were compared using chi-squared tests of independence for equality of proportions, and logistic regressions clustered by CHW.

Results

Two hundred seven total CHW assessments were completed on 101 enrolled infants. mCNCP assessments were more often fully completed and completed faster than pCNCP assessments (100% vs 23.8%, p < 0.001; 17.5 vs 23.6 min; p < 0.001). mCNCP facilitated calculations of respiratory rate, temperature, and gestational age. CHWs using mCNCP were more likely to identify small newborns (Odds Ratio (OR): 20.8, Confidence Interval (CI): (7.1, 60.8), p < 0.001), and to correctly classify 7 out of 16 newborn conditions evaluated, including severe weight loss (OR: 13.1, CI: (4.6, 37.5), p < 0.001), poor movement (OR: 6.6, CI: (2.3, 19.3), p = 0.001), hypothermia (OR: 14.9, CI: (2.7, 82.2), p = 0.002), and feeding intolerance (OR: 2.1, CI: (1.3, 3.3), p = 0.003). CHWs with mCNCP were more likely to provide counseling as needed on 4 out of 7 case management recommendations evaluated, including kangaroo mother care.

Conclusions

CHWs in rural Bangladesh with limited experience using tablets successfully used a mobile application for neonatal assessment after a two-day training. mCNCP may aid frontline health workers in Bangladesh to improve completion of neonatal assessment, classification of illnesses, and adherence to neonatal management guidelines.

Keywords

  • m-Health
  • Newborn care
  • Newborn case management
  • Newborn assessment
  • User-centered design
  • Integrated Management of Childhood Illnesses
  • mCNCP
  • Comprehensive Newborn Care Package
  • Community health worker
  • Bangladesh clinical guidelines
  • Newborn danger signs

Background

Identification and management of neonatal illnesses are challenging in low- and middle- income countries (LMICs), where human resources are limited and health systems may be weak. Over half of births in LMICs occur at home, and community-based health workers (CHWs) are often the first to assess these newborns [1]. CHWs typically have multiple tasks with inadequate time to complete the newborn assessment, and may have limited clinical knowledge for complex calculations or decision-making. Identifying and managing preterm (< 37 weeks of gestation) and small-for-gestational age (SGA; < 10% birth weight for gestational age) infants, who have increased risks for morbidity and mortality, present their own challenges in LMICs [24]. Gestational age calculation requires a last menstrual period (LMP), that is often missing or requires computation. SGA determination requires a weight-for-age percentile calculation or standard chart reference [5, 6]. Furthermore, small and preterm infants have special clinical needs, including extra warmth and feeding support [7], and may benefit from earlier referral and more frequent follow-up visits to support their survival and development [8, 9]. Clinical decisions about case-management (such as when to refer or specific antibiotic dosing) are often dependent on infant size, gestational age, or postnatal age, and can be cumbersome for CHWs.

These aforementioned challenges are present in Bangladesh, where 62% of births occur outside the home [10], and CHWs are usually the first to assess newborns within the first days of life [10]. The neonatal mortality rate is high at 28/1000 live births [10]. The majority of neonatal deaths occur among preterm or SGA infants [11], and prevalence of preterm birth is 22.3% and SGA is 39.6% [12, 13]. Many of these deaths can be prevented with interventions if newborn danger signs are identified and managed early [3]. In 2013, Bangladesh’s Ministry of Health and Family Welfare (MOHFW) introduced the Comprehensive Newborn Care Package (CNCP) national training and implementation guidelines for newborn care [14, 15], with an emphasis to reduce neonatal mortality through evidence-based interventions, such as chlorhexidine application for umbilical cord care and kangaroo mother care [16, 17]. The MOHFW has made a commitment to national scale-up of CNCP guidelines by 2022 [18].

Evaluations of the World Health Organization’s Integrated Management of Childhood Illnesses (IMCI) have revealed that health workers’ newborn assessments with paper forms are frequently incomplete and miss children requiring referral for treatment [19, 20]. Mobile technology, on the other hand, has demonstrated facilitation of exam completion, and adherence to referral and treatment guidelines in LMICs [19]. As part of the United States Agency for International Development’s (USAID) Saving Lives at Birth program, Brigham and Women’s Hospital’s Global Newborn Health Lab partnered with the Johns Hopkins University’s research partnership in Bangladesh (Projahnmo Study Group) and Saving Newborn Lives (Save the Children-Bangladesh) team to design and test a mobile application (app) to aid frontline health workers in the case management of small and sick newborns. The mCNCP (mobile-Comprehensive Newborn Care Package) phone and tablet-based app was designed using concepts of user-centered design to ensure acceptability and ease of navigation. The app features pictures of common newborn conditions from the CNCP training materials and built-in clinical decision support intended to improve identification and guide management and referral of infants needing care. It also includes algorithms to identify and provide special management advice for small and premature infants, based on CNCP and the American Academy of Pediatrics’ Essential Care for Small Babies curriculum [8, 16]. We hypothesized that CHWs using mCNCP would evaluate newborns more completely and efficiently, and would more accurately identify newborn danger signs, identify small and preterm babies, and provide correct management advice, compared to CHWs using standard paper-based forms (pCNCP). Our aims in this study were to test this hypothesis and to assess user satisfaction and preferences to aid future improvements to mCNCP’s development.

Methods

The intervention: development and functions of mCNCP

Prior to the development of the app, we conducted key informant interviews with 26 neonatal health stakeholders in Bangladesh, including leaders of non-governmental organizations and professional health societies (n = 6) and health workers (n = 20), to inform the functionalities of the mCNCP job aid. Participants emphasized that mCNCP should be accessible to CHWs with low technology literacy, include multi-media to increase user understandability, have offline data entry capabilities for field work, and use Android operating systems. We partnered with Saving Newborn Lives to ensure the app integrated the CNCP guidelines and harmonized with their mobile health (m-Health) training materials which they were concurrently developing [14].

mCNCP was developed over the course of six months in an XLS Form, which is converted to Xform standard and HTML by Ona (ona.io), a data-secure platform. The app was designed to guide the health worker through the neonatal assessment, including vital signs, basic newborn examination, and assessment for danger signs. mCNCP has built-in algorithms for classifying small babies (i.e., premature and/or low birth weight), neonatal illnesses (such as sepsis and jaundice), and auto-calculation of weight-loss and gestational age. The app has data entry checks (e.g., answer range constraints, required double entry, and instructions to check responses) and requires questions to be answered before advancing. mCNCP automatically saves responses, generates a summary of findings, provides case management advice and follow-up dates, and uploads all assessments to an online dashboard to display summary charts across infants. Media attachments include visual references from CNCP training materials and a one-minute timer to assist in respiratory rate count. Figure 1 shows examples of the mCNCP’s interface and the summary of findings page generated from a newborn assessment.
Fig. 1
Fig. 1

mCNCP Mobile Application Interface (English Translation). Screenshots of the mCNCP mobile application interface (with English-translated instructions), using an Android tablet or phone. Interface images demonstrate prompts for health workers to measure vital signs, assess for conditions and danger signs, and manage conditions. a Prompt for health worker to provide date and time of birth using a calendar and drop-down time feature, and a table to translate month names from English to Bangla; b Prompt for health worker to measure and record infant’s temperature; c Prompt for health worker to assess infant for severe chest indrawing; d Prompt for health worker to assess infant’s movement and tone; e Generated summary of feeding assessment and feeding advice based on health workers’ responses; f Generated summary of infant’s danger signs, conditions, and referral management advice. At the time of the study, mCNCP pages were displayed in the Bangla language. Most images in the mCNCP version used in the study were taken from Bangladesh’s Comprehensive Newborn Care Package (CNCP) guidelines for newborn assessment, that were developed in collaboration with the American Academy of Pediatrics using open source images from Essential Care for Small Babies (ECSB; Provider Guide), Essential Care for Every Baby (ECEB; Parent Guide), and Helping Babies Breathe (HBB; Provider Guide)

Throughout mCNCP’s development, we utilized user-centered design processes, and received iterative feedback from end-users (n = 12 CHWs, 5 physicians, and 3 paramedics). The feedback was used to improve its utility, effectiveness, and efficiency. Health workers were consulted in focus groups before the initial coding of mCNCP, and to test two iterations of the app before the study. Research staff, physicians, and paramedics continuously and iteratively tested mCNCP for correctness of referral pathways, calculations, and aesthetic features in the development phase. Bangla translations were reviewed to ensure understandability for the average CHW’s reading level and medical knowledge. User feedback led to the inclusion of calendar (date and time) functions, a one-minute timer for respiratory rate, and summary pages for diagnosis and management recommendations.

The app was accessed on Android tablets through ODKCollect, a free and open-source data collection platform. A paper-based form (pCNCP) was created for the study, with the same assessment questions and order as mCNCP. The same skip logic was in both methods, with pCNCP providing written instructions to the CHW and mCNCP automatically skipping inapplicable assessment sections based on previous responses. Based on the standard format for IMCI assessment, pCNCP had checkboxes next to conditions, and spaces to write case management recommendations, as seen in Additional file 1: Figure S1.

Training

All CHWs had 3–5 years of previous experience with neonatal care, newborn examination, and IMCI protocols. Twelve CHWs were trained by a study physician with expertise in IMCI and CNCP in three parts (i.e., sessions) over two days. CHWs were first trained in the national CNCP guidelines and provided a booklet with detailed descriptions of the guidelines. The first training session included the CNCP guidelines, identification of small babies (i.e., < 2500 g, < 37 weeks, or < 74 mm foot length), small baby care (i.e., referral thresholds, special advice for thermal care and feeding), and initial introduction to the mCNCP app. Two months later, the second and third training sessions were held in two small groups consisting of 6 CHWs each. The second session was a refresher training on CNCP guidelines, where CHWs received detailed instructions on how to navigate both mCNCP and pCNCP; this included how to respond to questions using either method, how to edit responses, use the calendar function, and properly save completed assessments with mCNCP. On the same day, the third session reviewed small baby care guidelines (including the various referral thresholds based on infant size, age, and danger signs), as well as supervised practice of pCNCP calculations and testing mCNCP on newborns case scenarios. CHWs were given case scenarios with varying combinations of danger signs, mothers’ LMP dates, birth weights and current weights, and temperatures. They were then asked to calculate gestational ages based on LMP and percent weight loss or gain with calculators to practice using pCNCP. CHWs were provided feedback and further instruction when errors occurred on either method during this session. Physicians and paramedics trained in CNCP guidelines acted as gold standard assessors, and were taught how to navigate and save responses on mCNCP. Study recruitment began the afternoons after the third training session.

Study setting and participant recruitment

Study participants were recruited from two different settings in Sylhet, Bangladesh between March and April 2017. Newborns were eligible if < 28 days old, and not deemed severely ill by their attending physician. The first and primary site of participant recruitment was Sylhet Osmani Medical College Hospital (SOMCH). Ninety-two newborns were enrolled from the pediatric and obstetric wards over a 4-day period. Mothers of recently delivered or admitted newborns were approached consecutively for consent to participate. The second site of participant recruitment was in a community-based setting in the rural Projahnmo research site located in the Zakiganj sub-district (upazilla) in Sylhet district, northeastern Bangladesh. This site was chosen in order to test the feasibility and usability of the app in a community-based setting where CHWs typically assess newborns. Eighteen eligible newborns were consecutively enrolled over a two-week period in the field site.

Sample size

A sample size of 79 assessments per method was determined to be required to detect a 20 percentage-point difference in the proportion of fully completed neonatal assessment, from 60% in the pCNCP group to 80% in the mCNCP group, with an 80% power and alpha of 5%. This was based on our initial estimate of the proportion of CHWs that would complete assessments, and was supported by previously published data from a similar study in Tanzania [19].

Data collection

A total of 12 trained female CHWs participated in the study. In SOMCH, for a given study day, 6 CHWs and 3 gold standard assessors (2 physicians and 1 paramedic) conducted assessments on approximately 12 recruited infants each. Randomization occurred at the CHW level, whereby CHWs were randomly assigned CHW identification (ID) numbers, and even or odd ID numbers determined whether their first method of assessment was mCNCP or pCNCP. After 6 newborn assessments, CHWs then alternated to the other method for the remaining 6 neonatal assessments, such that all CHWs used both methods in a particular day. All CHW assessments were compared against a gold standard assessor's assessment by a physician or paramedic using mCNCP, conducted on the same infant during the same shift. Iterative changes were made to mCNCP at the end of each day to improve the smoothness of the mCNCP’s operations but did not affect its content.

Process indicator measurement and definitions

Efficiency of methods was measured by time in minutes to complete exams, recorded by mCNCP, or calculated from the start and end times written by CHWs for pCNCP. Completeness was defined as CHWs having recorded answers for all conditions, and at least one generated or written care recommendation per assessment. Accuracy of identifying newborn danger signs and correctness of care management advice were determined by comparing each CHWs’ infant assessments against the gold standard assessment for the same infant. Correct classification of danger signs was defined as correctly identifying an infant as having or not having each condition, and correct counseling was defined as providing case management advice when required, as determined by gold standard assessors.

We developed Likert scale questions and surveys for acceptability, functionality, simplicity, and preference for method of assessments to inform further iterations of mCNCP’s design and development (Additional file 2: Figure S2). Survey themes were inspired in part by the Norman Nielson Group’s five components of usability, were administered after the first two days of enrollment, and completed by all 12 CHWs [21]. Focus groups with CHWs and gold standard assessors were conducted at the end of each of the four study days at SOMCH. Discussion themes included interface design, understandability, functionality, future uses, and acceptability of mCNCP by CHWs and the patient community. Key points were summarized from notes and audio recording transcriptions.

Statistical analysis

Statistical analyses were performed using Stata 14.1 (StataCorp, College Station, Texas) [22]. Average times to complete assessments by each method were compared with two-sided t-tests for differences in means. Proportions were calculated for the completeness of assessments, correct classification of each condition, and correct counseling by each method. The gold standard assessment was used as the reference for correctness. Chi-squared tests were used to compare the significance of differences between proportions. To estimate the odds ratios and 95% confidence intervals of correct classification for each condition comparing mCNCP versus pCNCP, generalized estimating equations (GEE) using a binomial family, logit link, and robust variance were used in order to account for the clustered nature of the data by CHW [23]. Likert scale responses on CHW satisfaction for each method of assessment were compared with two-sided t-tests.

Results

CHW characteristics

On average, CHWs were 24.8 years old (± 3.9), had an 11th-grade education (10.8 ± 1.3) and 5 years of CHW work experience (5.1 ± 2.4). While all CHWs reported owning a mobile phone, only 2 (16.7%) reported owning a smart phone and 1 (8.3%) had ever used a tablet.

Participant study flow

Out of 110 infants enrolled, 9 were excluded from analysis due to problems saving the mCNCP assessment (n = 1), missing gold standard physician assessment (n = 3), or gold standard assessments missing a CHW assessment (n = 5) (Additional file 3: Figure S3). One hundred one unique infants were included in the completeness analysis for most conditions, resulting in 207 assessments at SOMCH (n = 83 infants) and in the field (n = 18 infants), using either mCNCP (n = 103) or pCNCP (n = 104). Each CHW completed an average of 11.4 (SD 8.0) assessments included in analysis. Some infants were examined more than once by either of the methods. Assessments from the first day of study enrollment were excluded from correctness analysis as the pCNCP forms had only checkmarks next to each condition following the style of standard IMCI-formatted paper forms. In order to differentiate between an unchecked condition indicating the CHW assessed for the danger sign and determined it a non-issue, as opposed to the CHW did not assess the danger sign and therefore the assessment was incomplete, we added pCNCP response choices of “yes” or “no” for each danger sign. Analysis of correct classification of newborn danger signs by each method of assessment was thus performed on the 170 assessments (n = 84 pCNCP; n = 86 mCNCP) on 82 unique infants from the study days after this edit to ensure completeness in the paper form was made.

Efficiency of assessment

mCNCP assessments were completed an average of 6.1 min faster than pCNCP assessments (95% confidence interval, CI: − 8.4, − 3.8; p < 0.001). Assessments took an average of 17.5 (± 10.8) and 23.6 (± 11.6) minutes to complete by CHWs using mCNCP and pCNCP, respectively.

Completeness of neonatal assessment

Neonatal assessments by CHWs were significantly more likely to be fully completed with use of mCNCP (Table 1). Using pCNCP, 23.8% completed the entire assessment for all danger signs (and recorded their findings), compared to 100% of mCNCP CHW assessments (p < 0.001). CHWs often neglected to complete or record their findings on pCNCP, or to note whether danger signs were present or absent. They recorded respiratory rates and temperatures, performed calculations of weight-loss and infant postnatal age, and provided specific recommendations for care or referrals less often when using pCNCP. Most notably, mCNCP facilitated gestational age calculations compared to pCNCP (100% vs. 20.7%, p < 0.001), assisting identification of preterm infants. The only instance where mCNCP did not markedly improve completeness over the pCNCP was in recording infant weight if available.
Table 1

Comparison of Completeness of Neonatal Assessments by CHWs using Paper Forms (pCNCP) and mobile-CNCP (mCNCP)

Completeness of Exams

Paper Form: pCNCP

Mobile App: mCNCP

% Point Difference (mCNCP – pCNCP)

Chi Square Test P-Value

Full Assessment Completeda

20/84 (23.8%)

86/86 (100%)

76.2

< 0.001

Any Care Management Recommendation Givenb*

34/104 (32.7%)

103/103 (100%)

67.3

< 0.001

Respiratory Rate Recorded

47/104 (45.2%)

103/103 (100%)

54.8

< 0.001

Temperature Recorded

62/104 (59.6%)

103/103 (100%)

40.4

< 0.001

Weight Recordedc

101/104 (97.1%)

102/103 (99.0%)

1.9

0.32

Weight-loss Calculation Completedd*

99/104 (95.2%)

103/103 (100%)

4.8

0.024

Gestational Age Calculated (if LMP recorded)*

6/29 (20.7%)

34/34 (100%)

79.3

< 0.001

Postnatal Age Calculatede*

77/104 (74.0%)

103/103 (100%)

26.0

< 0.001

aAll danger signs were assessed, with weight (either birth weight or current weight), temperature, and respiratory rate recorded, and at least one management recommendation given. The version of the IMCI-style paper form on the first day of study enrollment had checkmarks next to newborn conditions, but did not have response choices of “yes” or “no” for each danger sign. This option was included for the remaining days of the study, and thus the first day of assessments was not included in the completeness analysis. b At least one care recommendation or referral was given. c Either birth weight or current weight recorded. d A calculated percent weight loss or gain was recorded (whether correct or incorrect). eInfant’s postnatal age was calculated (whether correct or incorrect).; *mCNCP’s built-in algorithms performed these calculations and generated recommendations

Participant characteristics

Prevalence of each newborn danger sign and characteristics of participating infants in each method category (pCNCP or mCNCP) are summarized in Table 2, determined by gold standard assessors’ expert classification and CHW classification. The demographics and disease profiles were similar between the comparison and intervention groups by expert classification. Characteristics determined by CHW classifications for each method, however, elucidate when CHWs under- or over-identified exiting danger signs. Using pCNCP, CHWs missed all cases of low birth-weight, and under-estimated small babies and cases of infected umbilicus. Conversely, CHWs with pCNCP over-estimated cases of extremely low weight, poor movement, fever, hypothermia, and breastfeeding issues compared to expert classification. CHWs systematically under-estimated fast-breathing infants by both methods, and over-estimated cases of skin pustules, jaundice, eye infection, and feeding intolerance, by both methods, compared to experts.
Table 2

Characteristics of Newborns Observed in Comparison (Paper form: pCNCP) and Intervention Group (Mobile App: mCNCP)

 

Expert classification of pCNCP cohort

Expert classification of mCNCP cohort

P-Value†

CHW classification of pCNCP cohort

CHW classification of mCNCP cohort

P-Value†

Female

26/84 (31.0%)

29/86 (33.7%)

0.70

26/82‡ (31.7%)

32/86 (37.2%)

0.45

Infant post-natal age in days (mean)

6.5 (6.7 SD)

6.5 (6.7 SD)

0.98

6.6 (7.2 SD)

5.6 (6.2 SD)

0.90

Gestational age at birth in weeks (mean)

40.1 (3.2 SD)

40.0 (2.6 SD)

0.89

39.6 (2.2 SD)

39.4 (2.2 SD)

0.86

Low birth-weighta*

41/84 (48.8%)

45/86 (52.3%)

0.65

0/84 (0%)

45/86 (52.3%)

< 0.001

Smallb*

41/84 (48.8%)

45/86 (52.3%)

0.65

9/84 (10.7%)

46/86 (53.5%)

< 0.001

Birth-weight or Current Weight < 1500 g*

6/84 (7.1%)

8/86 (9.3%)

0.61

18/84 (21.4%)

7/86 (8.1%)

0.014

Severe Weight Lossc*

21/84 (25.0%)

21/86 (24.4%)

0.93

14/84 (16.7%)

20/86 (23.3%)

0.28

Moderate Weight Lossd*

18/84 (21.4%)

18/86 (20.9%)

0.94

22/84 (26.2%)

17/86 (19.8%)

0.32

Poor Movement: Unconscious or moves ONLY when Stimulated

3/84 (3.6%)

3/86 (3.5%)

0.98

34/84 (40.5%)

4/86 (4.7%)

< 0.001

History of Convulsions

19/84 (22.6%)

19/86 (22.1%)

0.93

23/84 (27.4%)

25/86 (29.1%)

0.81

Poor Feeding: Unable to feed OR Stopped feeding well

52/84 (61.9%)

52/86 (60.5%)

0.85

35/84 (41.7%)

47/86 (54.7%)

0.09

Severe Chest In-drawing

3/84 (3.6%)

5/86 (5.8%)

0.49

9/84 (10.7%)

17/86 (19.8%)

0.10

Fast Breathinge*

18/84 (21.4%)

15/86 (17.4%)

0.51

9/84 (10.7%)

2/86 (2.3%)

0.026

Umbilicus: red or pus

13/84 (15.5%)

13/86 (15.1%)

0.95

4/84 (4.8%)

18/86 (20.9%)

0.002

Skin Pustules

0/84 (0%)

0/86 (0%)

2/84 (2.4%)

6/86 (7.0%)

0.16

Jaundice on Soles or Body

9/84 (10.7%)

10/86 (11.6%)

0.85

18/84 (21.4%)

23/86 (26.7%)

0.42

Eye infection

0/84 (0%)

0/86 (0%)

2/84 (2.4%)

5/86 (5.8%)

0.26

Feverf*

13/84 (15.5%)

13/86 (15.1%)

0.95

19/84 (22.6%)

10/86 (11.6%)

0.06

Hypothermia g*

5/84 (6.0%)

5/86 (5.8%)

0.97

23/84 (27.4%)

5/86 (5.8%)

< 0.001

Feeding Intoleranceh

7/84 (8.3%)

6/86 (7.0%)

0.74

22/84 (26.2%)

12/86 (14.0%)

0.046

Problems with breastfeedingi

32/84 (38.1%)

32/86 (37.2%)

0.91

62/84 (73.8%)

32/86 (37.2%)

< 0.001

aBirth-weight < 2500 g; bBirth-weight < 2500 g or foot- length < 74 mm; cWeight loss > 10% for small infant or > 15% for non-small infant; dWeight loss is 8–10% for small infant or 10–15% for non-small infant; eRespiratory rate > 60 breaths per min; fTemperature > 38 °C or 100.4 °F; gTemperature < 35.5 °C or 95.9 °F; hInfant has at least one condition: chokes, turns blue or pale when feeding, vomits frequently, has distended or tender abdomen, or bloody stools.; iInfant has problems with at least one: waking easily for feeds, breastfeeding for 10+ minutes for per side, sleeping comfortably between feeds, having 5+ wet diapers per day, mother’s breasts haven’t softened.; N: infant participants.; *For mCNCP, these danger signs were identified by the built-in algorithms based on guideline thresholds and CHW assessment responses.; †P-values are from two-sided t-tests for differences in means for infant post-natal age and gestational age at birth, and from chi-squared tests of proportions for all other variables. ‡2 CHWs using pCNCP did not record the infants’ sex.; Of the 170 assessments performed on 82 unique infants, 84 assessments were conducted by CHWs using pCNCP and 86 assessments were conducted by CHWs using mCNCP. Expert classifications were gold standard assessors’ determinations of danger signs (present or not present) using mCNCP. This table shows that the baseline characteristics of infants in the study are similar when comparing proportions of newborn danger signs in the intervention (mCNCP) and comparison (pCNCP) cohorts as determined by the gold standard experts (physicians or paramedics). When comparing the classification of infants in the assessment method cohorts as determined by the CHWs, however, the CHWs tended to over- or under-estimate conditions, and the baseline characteristics thus appear different across the intervention and comparison groups

Accuracy of neonatal assessment danger sign classifications

Table 3 shows comparisons of correctly classified neonatal conditions, as determined by gold standard assessors, using the mCNCP and pCNCP. Using mCNCP, CHWs were 20.8 times more likely to correctly classify small babies by weight or foot-size (OR: 20.8, (CI: 7.9–48.1), p < 0.001), with a 39.4 percentage-point difference from pCNCP classification (p < 0.001). CHWs using mCNCP were 13.1 and 13.8 times more likely to correctly classify infants with severe or moderate weight loss (p < 0.001), respectively, and significantly more likely to correctly classify danger signs of poor movement, fever, hypothermia, and feeding intolerance. Jaundice was the only condition significantly more likely to be correctly classified by CHWs using pCNCP by chi-squared tests; this may have been due to mCNCP’s use of pictures featuring very yellow, jaundiced babies, which could have anchored CHWs to a higher degree of jaundice than necessary for referral. No significant differences were found for classifying history of convulsions, poor feeding, severe chest in-drawing, fast breathing, infected umbilicus, skin pustules, or eye infection. Identifying feeding intolerance and breast-feeding problems were significantly more likely to be identified by mCNCP only by either chi-squared tests of proportions or odds ratio, but not by both analyses.
Table 3

Comparison of Correct Classification of Newborn Conditions by CHWs using Paper Forms (pCNCP) and mobile-CNCP (mCNCP)

Danger Signs

Correctly Classified by pCNCP

Correctly Classified by mCNCP

% Pt Diff (mCNCP - pCNCP)

Chi Square Test P-Value

OR

CI for OR

Logit P-Value

Low birth-weighta*

43/84 (51.2%)

82/86 (95.4%)

44.2

< 0.001

19.5

(7.9, 48.1)

< 0.001

Smallb*

48/84 (57.1%)

83/86 (96.5%)

39.4

< 0.001

20.8

(7.1, 60.8)

< 0.001

Birth-weight or Current Weight < 1500 g*

70/84 (83.3%)

85/86 (98.8%)

15.5

< 0.001

17.0

(1.8, 160.7)

0.013

Severe Weight Lossc*

57/84 (67.9%)

83/86 (96.5%)

28.6

< 0.001

13.1

(4.6, 37.5)

< 0.001

Moderate Weight Lossd*

56/84 (66.7%)

83/86 (96.5%)

29.8

< 0.001

13.8

(7.3, 26.1)

< 0.001

Poor Movement: Unconscious or moves ONLY when Stimulated

53/84 (63.1%)

79/86 (91.9%)

28.8

< 0.001

6.6

(2.3, 19.3)

0.001

History of Convulsions

68/84 (81.0%)

70/86 (81.4%)

0.4

0.94

1.0

(0.3, 3.3)

0.96

Poor Feeding: Unable to feed OR Stopped feeding well

59/84 (70.2%)

65/86 (75.6%)

5.4

0.43

1.3

(0.5, 3.2)

0.56

Severe Chest In-drawing

74/84 (88.1%)

68/86 (79.1%)

−9.0

0.11

0.5

(0.2, 1.5)

0.22

Fast Breathinge*

71/84 (84.5%)

73/86 (84.9%)

0.4

0.95

1.0

(0.5, 2.0)

0.94

Umbilicus: red or pus

69/84 (82.1%)

61/86 (70.9%)

−11.2

0.09

0.5

(0.2, 1.6)

0.25

Skin Pustules

82/84 (97.6%)

80/86 (93.0%)

−4.6

0.16

0.3

(0.1, 1.5)

0.15

Jaundice on Soles or Body

69/84 (82.1%)

57/86 (66.3%)

−15.8

0.018

0.4

(0.2, 1.1)

0.08

Eye infection

82/84 (97.6%)

81/86 (94.2%)

−3.4

0.26

0.4

(0.1, 2.2)

0.29

Feverf*

68/84 (81.0%)

83/86 (96.5%)

15.5

0.001

6.5

(1.3, 31.9)

0.021

Hypothermia g*

62/84 (73.8%)

84/86 (97.7%)

23.9

< 0.001

14.9

(2.7, 82.2)

0.002

Feeding Intoleranceh

63/84 (75.0%)

74/86 (86.1%)

11.1

0.07

2.1

(1.3, 3.3)

0.003

Problems with breastfeedingi

46/84 (54.8%)

62/86 (72.1%)

17.3

0.019

2.1

(0.9, 4.9)

0.07

aBirth-weight < 2500 g; bBirth-weight < 2500 g or foot- length < 74 mm; cWeight loss > 10% for small infant or > 15% for non-small infant; dWeight loss is 8–10% for small infant or 10–15% for non-small infant; eRespiratory rate > 60 breaths per min; fTemperature > 38 °C or 100.4 °F; gTemperature < 35.5 °C or 95.9 °F; hInfant has at least one condition: chokes, turns blue or pale when feeding, vomits frequently, has distended or tender abdomen, or bloody stools.; iInfant has problems with at least one: waking easily for feeds, breastfeeding for 10+ minutes for per side, sleeping comfortably between feeds, having 5+ wet diapers per day, mother’s breasts haven’t softened.; OR Odds ratio, CI Confidence interval; *For mCNCP, these danger signs were identified by the built-in algorithms based on guideline thresholds and CHW assessment responses.; Of the 170 assessments performed on 82 unique infants, 84 assessments were conducted by CHWs using pCNCP and 86 assessments were conducted by CHWs using mCNCP. Expert classifcations were gold standard assessors' determiniations of danger signs (present or not present) using mCNCP. CHWs' assessments were considered correct if their classification of individual danger signs were the same as the gold standard assessment of the newborn's condition

Using mCNCP, CHWs more often provided correct counseling to infants requiring case management, including advising mothers on the need to provide kangaroo mother care, cup-feed, and increase feeding frequency (Additional file 4: Table S1). By chi-squared tests of proportions only, significant differences were found in mCNCP aiding correct counseling for infants requiring referral and special advice for breastfeeding small infants, over pCNCP (91.4% vs 73.8%; p = 0.003).

End-user satisfaction

CHWs significantly preferred mCNCP over pCNCP, and most CHWs thought mCNCP aided ease of understanding instructions and making referral decisions, improved confidence to decide when to refer infants, reduced entry errors, and was the desirable method for future newborn assessments (Additional file 5: Table S2). Results from other sections of the survey can be found in Additional file 6: Tables S3-S5. The majority of CHWs felt confident enough to use mCNCP in the field on their own after the study and believed the app would assist them in newborn assessments and visitation (Additional file 6: Table S3). All CHWs reported feeling more comfortable with and preferred using the app, finding it faster and more accurate in providing recommendations than pCNCP (Additional file 7: Table S4). Additional file 8: Table S5 displays CHWs thoughts on mCNCP’s functionality and understandability, and can be used to inform some of the features of mCNCP for future iterations of the app.

In focus groups, CHWs voiced the desire to use mobile technology in their daily work routine, as they found the built-in calculation tools, one-minute timer, and the generated summary page to be helpful in guiding referrals and counseling. Physicians had informed us that an abbreviated version of the mCNCP would be useful for facility settings in Bangladesh due to the high patient volume and insufficient doctor to patient ratio. To improve the mCNCP’s functionality, CHWs suggested adding a global positioning system (GPS) functionality to aid referral to the nearest facility, considering alternatives for the calendar drop-down function, and discussing with the platform’s developers about the occasional freezing during the automatic saving time-points. A summary of their feedback can be found in Fig. 2.
Fig. 2
Fig. 2

Summary of CHW Focus Group Findings on mCNCP User-Satisfaction and Preferences. Key findings across post-user-testing focus groups with community health workers (CHWs). Topics include positive feedback, identified areas for improvement, and perceptions on the feasibility of mCNCP scale-up

Discussion

A mobile application developed in accordance with Bangladeshi newborn care guidelines, mCNCP, aided CHWs to assess and manage common newborn conditions and illnesses. CHWs using mCNCP assessed and managed newborns more efficiently, completely, and correctly compared to those using traditional paper-based forms. Of note, the app aided CHWs in identifying and managing preterm and low birth weight babies, who carry high risks of neonatal morbidity and mortality. CHWs were also more likely to provide appropriate guidance to parents related to feeding and body temperature regulation (e.g., kangaroo mother care). At the end of the study, CHWs using mCNCP reported high levels of satisfaction and confidence in using the technology.

mCNCP aided frontline health workers in a LMIC to more accurately identify neonatal danger signs and risk factors, particularly prematurity and fetal growth restriction (SGA). Prematurity and fetal growth restriction are causes or risk factors for 80% of neonatal deaths [11]. However, these high-risk infants are particularly challenging to identify in LMICs due to limited data, capacity, and resources for antenatal and neonatal care. mCNCP facilitated the auto-calculation of gestational age from mothers’ LMP dates and had built-in algorithms to identify low birth-weight infants based on birth-weight or foot size. CHWs using mCNCP also had enhanced ability to classify conditions of severe or moderate weight loss (by auto-calculating percent weight change), fever or hypothermia (by auto-calculating temperature thresholds), poor movement, and feeding intolerance. The accurate and timely identification of these high-risk conditions and danger signs is the first step required to determine the need for referral and delivery of life-saving interventions, such as respiratory support or antibiotics.

Another strength of mCNCP was its ability to aid health workers in more complex clinical decision-making, by auto-generating referral and management advice based on multiple inputs, thresholds, and calculations. mCNCP’s auto-generation of follow-up visit dates could facilitate timely check-ups to monitor an infant’s health status. Traditional IMCI paper forms use a simple checklist-based approach, and referral is recommended with the presence of any single danger sign [24]. However, for small infants, there are additional considerations and calculations (such as percent weight loss) and more complicated decisions, with lower thresholds for referral [8]. CHWs using mCNCP more often provided correct management advice for small infants requiring increased feeding frequency, cup-feeding, or thermal care, and identified infants requiring referral.

Additionally, m-Health tools may help increase the completion of assessments and reduce data recording errors. mCNCP required CHWs to answer every question before advancing and thus ensured the full completion of the neonatal assessment. The app’s answer constraints, data entry checks, and prompts to review responses may have also contributed to the elimination of errors and improvement upon accurately diagnosing infant danger signs. Similarly, a study in Tanzania reported that an IMCI mobile application significantly improved adherence to guidelines, completeness of assessments, and accuracy of classification of conditions compared to use of traditional paper forms [19]. Studies in China comparing smartphone applications to pen-and-paper forms had similar results in reducing data recording errors [25], and finding the paper method to be more error-prone [26].

CHWs preferred using mCNCP over pCNCP, stating it was understandable, functional, easy to use, and faster to complete, and voiced a desire to use mobile applications in their daily work. Given their previous inexperience with smart phones or tablets, this feedback is encouraging for the acceptance, capabilities, and potential health implications of future m-Health interventions and indicates low-level health workers are capable of learning the technology quickly. As the devices become increasingly affordable, countries can consider the potential for scalability of m-Health solutions operated by CHWs.

mCNCP is uniquely situated as a neonatal data collection tool for frontline health workers, created with user-centered design and iterative processes. Many m-Health interventions involve text message correspondence between health care providers and pregnant women or mothers to increase maternal and newborn service utilization, including two in Bangladesh: MAMA and JiVitA [27]. To our knowledge, at the time we did not identify other m-Health solutions in Bangladesh that provide decision-making support to maternal or newborn health providers [26, 28]. Through user-centered design, we have addressed some of the challenges past m-Health interventions have faced, including providing detailed care advice and addressing low literacy levels of our users, including technology and medical literacy [26]. Incorporating features such as GPS, video and audio instructions, and the ability to take and store pictures of patients, per stakeholders’ suggestions, could be beneficial for tracking patient outcomes and improving diagnoses [26]. A Bangladesh study found that mobile applications with a feature to allow for real-time access to patient medical data led to more timely triage and initial treatment of patients, supporting statements from our key stakeholders [26]. Integration of mCNCP with the existing health management information system in Bangladesh may be beneficial to the MOHFW and health administrators to monitor and track the epidemiology of neonatal diseases.

Limitations

In this evaluation, we did not assess the impact of the mCNCP on provider to parent communication, care-seeking behaviors, or health outcomes. Previous studies have demonstrated the ability of m-Health tools to improve communication of diagnoses between providers and patients, and parents’ recall of case management advice [29]. Mothers’ health-seeking behaviors and healthcare utilization throughout pregnancy, delivery, and the newborn period have also improved with the use of m-Health systems and devices [26, 27, 30]. Studying the impact of mCNCP on patient behavior would further the understanding of its implications on improving patient health. We experienced some limitations in developing mCNCP for our intended end-users, and pose lessons learned that could benefit future m-Health tools and programs. Ona, mCNCP’s platform, while easy to operate and program, had limits to functionality and appearance at the time of the app’s development. Some CHWs found mCNCP’s drop-down calendar function difficult to use, and we were unable to use the luni-solar Bengali calendar with this feature. As previously mentioned, the initial pCNCP paper forms in our study were based on the standard IMCI format, which did not allow for accurate assessment of whether the person completing the form assessed for all danger signs or skipped the section. This issue was rectified for our paper forms part-way through our study. We experienced some data loss at the beginning of the study, losing 2 mCNCP assessments from the first day of patient enrollment at SOMCH and 1 from the first day of field-based assessments, indicating a brief user learning period is needed to ensure all mCNCP forms are saved properly. Furthermore, for future iterations of mCNCP, data needs to be fed back into Health Management Information Systems for monitoring purposes, which we did not have the opportunity to develop for this study.

Conclusions

This study demonstrates that CHWs with little to no experience using mobile tablets can use and operate a mobile application for newborn assessments after a two-day training with user-testing and supervision. mCNCP serves as a decision-aid tool, which has shown to improve completeness and efficiency of newborn assessments, classification of illness, and adherence to clinical guidelines compared to the standard method of paper forms. User-centered design ensured the usability, functionality, and acceptability of the mobile application by CHWs, who voiced their satisfaction with and preference for mCNCP as a method of assessment. The impact of m-Health tools such as mCNCP on health outcomes should be rigorously evaluated. Furthermore, future efforts should be made to develop systems to integrate the results and findings of such tools into Health Information Management Systems in order to guide regional and national level monitoring and planning.

Abbreviations

App(s): 

Application(s)

CHW: 

Community health worker

CNCP: 

Comprehensive Newborn Care Package

GPS: 

Global positioning system

ID: 

Identification

IMCI: 

Integrated Management of Childhood Illnesses

LMIC: 

Low- or middle-income country

LMP: 

Last menstrual period

mCNCP: 

mobile Comprehensive Newborn Care Package

MOHFW: 

Ministry of Health and Family Welfare

pCNCP: 

standard paper-based form comparable to mCNCP

SGA: 

Small-for-gestational age

SOMCH: 

Sylhet Osmani Medical College Hospital

Declarations

Acknowledgements

We thank Wahida Saraj who provided valuable insight on Save the Children’s m-Health training materials for CNCP, Alain Labrique who provided guidance in early stages of the project, Karima Ladhani who assisted on developing mCNCP’s early versions, and Lian Folger who assisted manuscript editing.

Funding

The study was funded by USAID’s Saving Lives at Birth Program, Grant Number: AID-OAA- F-15-00023, with additional funding provided by Brigham and Women’s Hospital Department of Pediatric Newborn Medicine.

Authors’ contributions

All authors were involved in the design, implementation, or analysis and data interpretation. ACL conceived the study, obtained funding, developed clinical algorithms for the app, and supervised the app development, data collection, and analysis. LES contributed to study design and data collection, and led the coding and development of the mCNCP app, data analysis, and manuscript writing. MR led translations of paper forms, training of health workers, and focus group discussions and interpretation. RW led development of early mCNCP mobile app prototypes. AB, SR1, and SA managed and coordinated Bangladesh field implementation and study activities. SR2, TAN, CH, and HD participated in development of the app, providing critical inputs on the clinical algorithms. RW and HD led initial stakeholder interviews. TAN, NNB and IFJ participated in focus group analysis. All authors read, reviewed, provided critical inputs, and approved the final manuscript.

Ethics approval and consent to participate

Ethics approval was received from Partners Human Research Committee (Protocol #: 2015P002122) and the National Research Ethics Committee of Bangladesh Medical Research Council. We obtained written, informed consent from parents of infants in the study and focus group participants.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Authors’ Affiliations

(1)
Department of Pediatric Newborn Medicine, Brigham and Women’s Hospital, Suite BB-502, 75 Francis Street, Boston, MA 02115, USA
(2)
Projahnmo Research Group, Johns Hopkins University- Bangladesh, “Abanti”, Flat #5B, House #37, Road #27, Banani, Dhaka, 1213, Bangladesh
(3)
Maternal and Child Health Division, International Centre for Diarrhoeal Disease Research, Bangladesh, 68, Shaheed Tajuddin Ahmed Sarani, Mokakhali, Dhaka, 1212, Bangladesh
(4)
Health Section, United Nations Children’s Fund, Rangpur Field Office, Rabdhaballav Road, Rangpur, 5400, Bangladesh
(5)
Harvard Medical School, 25 Shattuck St, Boston, MA 02115, USA
(6)
Save the Children: Bangladesh, House CWN (A) 35, Road 43, Dhaka, 1212, Bangladesh
(7)
Johns Hopkins Bloomberg School of Public Health, 615 N Wolfe St, Baltimore, MD 21205, USA

References

  1. UNICEF. The state of the world's children 2009: Maternal and newborn health. 2008. https://www.unicef.org/sowc09/docs/SOWC09-FullReport-EN.pdf.
  2. Blencowe H, et al. National, regional, and worldwide estimates of preterm birth rates in the year 2010 with time trends since 1990 for selected countries: a systematic analysis and implications. Lancet. 2012;379(9832):2162–72.Google Scholar
  3. WHO. Fact sheet: Preterm birth. https://www.who.int/en/news-room/fact-sheets/detail/preterm-birth. Accessed 25 June 2018.
  4. Bhutta ZA, et al. Can available interventions end preventable deaths in mothers, newborn babies, and stillbirths, and at what cost? Lancet. 2014;(9940):347–70.Google Scholar
  5. WHO Multicentre Growth Reference Study Group. WHO Child Growth Standards: Length/height-for-age, weight-for-age, weight-for-length, weight-for-height and body mass index-for-age: Methods and development. Geneva: World Health Organization; 2006. https://www.who.int/childgrowth/standards/technical_report/en/.
  6. Buck Louis GM, et al. Racial/ethnic standards for fetal growth: the NICHD fetal growth studies. Am J Obstet Gynecol. 2016;213(4):449.e1–449.e.41.Google Scholar
  7. WHO. What kind of care do preterm babies need? https://www.who.int/features/qa/preterm_baby_care/en/. Accessed 25 June 2018.
  8. American Academy of Pediatrics. Helping babies survive: Essential care for small babies. https://www.aap.org/en-us/advocacy-and-policy/aap-health-initiatives/helping-babies-survive/Pages/Essential-Care-Small-Babies.aspx. Accessed 25 June 2018.
  9. Terhune M. Essential care for every baby–a new curriculum from the AAP. Global Health Link. 2015;(38). http://www.ucdenver.edu/academics/colleges/PublicHealth/research/centers/globalhealth/about/Documents/Newsletter%202015/GlobalHealthLink38.pdf.
  10. National Institute of Population Research and Training (NIPORT), Mitra and Associates, and the DHS Program ICF International. Bangladesh Demographic and Health Survey. 2014;(2016). https://dhsprogram.com/pubs/pdf/FR311/FR311.pdf.
  11. Lawn JE, et al. Every newborn: progress, priorities, and potential beyond survival. Lancet. 2014;384(9938):189–205.Google Scholar
  12. Shah R, et al. Incidence and risk factors of preterm birth in a rural Bangladeshi cohort. BMC Pediatr. 2014;14:112.Google Scholar
  13. Lee AC, et al. National and regional estimates of term and preterm babies born small for gestational age in 138 low-income and middle-income countries in 2010. Lancet. Glob Health. 2013;1(1):e26–e36.Google Scholar
  14. Save the Children International. Comprehensive newborn care package (CNCP) launched in Bangladesh to improve newborn survival. 2015. https://everyone.savethechildren.net/articles/comprehensive-newborn-care-package-cncp-launched-bangladesh-improve-newborn-survival. Accessed 25 June 2018.
  15. Star Health Report. Improving newborn survival. The Daily Star. 2015. https://www.thedailystar.net/health/improving-newborn-survival-75585. Accessed 25 June 2018.
  16. Mannan I. Background and objective of development and demonstration of Comprehensive Newborn Care Package (CNCP). Bangladesh Neonatal Forum. 2014. http://www.bnfbd.com/presentation-gallery/2.CNCP%20Background%20Ishtiaq%20Mannan_processed.pdf. Accessed 25 June 2018.
  17. Bangladesh Ministry of Health and Family Welfare. Development and demonstration of Comprehensive Newborn Care Package in a district learning laboratory (Kushtia). National Newborn Health Bulletin. 2016. https://www.mcsprogram.org/wp-content/uploads/2018/06/Bangladesh-KAP-Summary-Sheet.pdf. Accessed 25 June 2018.
  18. Directorate General of Health Services, Bangladesh Ministry of Health and Family Welfare. 4th Health, Population and nutrition sector programme (4th HPNSP) operational plan (OP): Maternal neonatal child and adolescent health (MNC&AH), January 2017–June 2022. 2017. http://www.dghs.gov.bd/images/docs/OP/2018/MNC_AH.pdf. Accessed 25 June 2018.
  19. Mitchell M, Hedt-Gauthier BL, Msellemu D, Nkaka M, Lesh N. Using electronic technology to improve clinical care– results from a before-after cluster trial to evaluate assessment and classification of sick children according to Integrated Management of Childhood Illness (IMCI) protocol in Tanzania. BMC Med Inform Decis Mak. 2013;13:95.Google Scholar
  20. Arifeen SE, et al. Effect of the Integrated Management of Childhood Illness strategy on childhood mortality and nutrition in a rural area in Bangladesh: a cluster randomised trial. Lancet. 2009;374(9687):393–403.Google Scholar
  21. Nielsen Norman Group. Usability 101: Introduction to usability. 2012. https://www.nngroup.com/articles/usability-101-introduction-to-usability/. Accessed 25 June 2018.
  22. StataCorp. Stata Statstical Software: Release 14. College Station: StataCorp LP. 2015.Google Scholar
  23. Zeger SL, Liang KY. Longitudinal data analysis for discrete and continuous outcomes. Biometrics. 1986;42(1):121–30.Google Scholar
  24. WHO. Integrated Management of Childhood Illness chart booklet. 2014. https://apps.who.int/iris/bitstream/handle/10665/104772/9789241506823_Chartbook_eng.pdf;jsessionid=9BD45EB8E573944B2F4067A67A99C1EA?sequence=16.
  25. Zhang S, et al. Smartphone versus pen-and-paper data collection of infant feeding practices in rural China. J Med Internet Res. 2012;14(5):e119.Google Scholar
  26. Amoakoh-Coleman M, et al. Effectiveness of mHealth interventions targeting health care workers to improve pregnancy outcomes in low- and middle-income countries: a systematic review. J Med Internet Res. 2016;18(8):e226.Google Scholar
  27. Sondaal SF, et al. Assessing the effect of mHealth interventions in improving maternal and neonatal Care in low- and Middle-Income Countries: a systematic review. PLoS One. 2016;11(5):e0154664.Google Scholar
  28. Khan NUZ, et al. Experience of using mHealth to link village doctors with physicians: lessons from Chakaria, Bangladesh. BMC Med Inform Decis Mak. 2015;15:62.Google Scholar
  29. Perri-Moore S, et al. Using an eIMCI-derived decision support protocol to improve provider-caretaker communication for treatment of children under 5 in Tanzania. Glob Health Commun. 2015;1(1):41–7.Google Scholar
  30. Lee SH, et al. Effectiveness of mHealth interventions for maternal, newborn and child health in low- and middle-income countries: Systematic review and meta-analysis. J Glob Health. 2016;6(1):010401.Google Scholar

Copyright

© The Author(s). 2019

Advertisement