Predicting asthma control deterioration in children

Asthma is the most common pediatric chronic disease [1, 2] and the most frequent reason for preventable pediatric hospitalization [3]. Asthma affects 7.1 million American children [4, 5], accounts for one third of pediatric emergency department (ED) visits [6], and incurs an annual total direct healthcare cost around 9.3 billion dollars [1]. In 2009, 640,000 ED visits, 157,000 hospitalizations, and 185 deaths [4] were due to pediatric asthma. Poor asthma control in children is associated with decreased quality of life [7], increased school absenteeism with work loss for parents [8], and a high hospital readmission rate [9]. Despite its impact, asthma remains a poorly controlled disease [10]. Effective interventions to improve and maintain asthma control are needed.

Asthma control on a patient fluctuates frequently over time due to multiple factors [11–13]. An asthma exacerbation is often preceded by a critical period of decreased asthma control [14]. The critical period often goes unrecognized by patients, caregivers, and physicians [10, 15–18], resulting in missed opportunities for taking preventive interventions such as education and medication prescription and adjustment [15, 17].

Using predictive models can facilitate recognition of impending loss of asthma control before significant symptoms emerge. While many predictive models for diagnosing and treating asthma exist [19], little has been done for predicting asthma control deterioration. Existing models focus on predicting asthma exacerbations, which often represent a late manifestation of persisting loss of asthma control, and have low sensitivities and low positive predictive values [20–24].

In the past, our group developed and validated an asthma control monitoring tool, the Asthma Symptom Tracker (AST) [25]. This self-monitoring tool was designed to assess a child’s asthma control level on a weekly basis. The objective of this study was to develop a model for predicting asthma control deterioration one week ahead, by using scores from previously completed AST assessments [25] in conjunction with patient attributes and environmental variables.

The original study [25] provided 2912 weekly assessments of asthma control on 210 asthmatic children 2 to 18 years old. After excluding baseline assessments and 30 patients with only one AST assessment, 2617 AST assessments from 180 patients were available for predictive modeling. Table 2 shows patient demographics and baseline clinical characteristics. The percentage of uncontrolled asthma in the AST assessments was 23.5 % overall. As shown in Fig. 2, this percentage was significantly higher in the first week after hospitalization (50 %), then stabilized near 19 % during the remaining follow-up assessments.

Variable	Category	N
Sex	male	110
	female	70
Age (in years)	2–5	112
	6–10	46
	11–14	19
	15–18	3
Race	Native American	3
	Asian	5
	black	4
	Hispanic	26
	Pacific islander	6
	white	118
	other	14
	unknown	4
State of residence	Idaho	1
	Nevada	4
	Utah	172
	Wyoming	3
Chronic asthma severity level	intermittent	35
	persistent	144
	unknown	1
Insurance category	Medicaid	71
	private	101
	self-paid	8
Exposure to secondhand smoke	yes	35
	no	116
	unknown	29
Presence of any comorbidity	yes	2
	no	178
Previous asthma admission	yes	35
	no	145

The multiboost with decision stumps classifier had the best performance, with a sensitivity of 73.8 % vs. 74.5 %, an accuracy of 71.8 % vs. 74.4 %, a specificity of 71.4 % vs. 74.4 %, an AUC of 0.761 vs. 0.757, a PPV of 37.1 % vs. 50.7 %, and a NPV of 92.4 % vs. 89.2 %, when measured by the two evaluation approaches, respectively. The support vector machine and deep learning classifiers performed similarly. The naive Bayes, random forest, and k-nearest neighbor classifiers performed less well, particularly with respect to sensitivity.

We also used additional independent variables, beyond the component scores of the patient’s AST assessment, collected for this study to improve the advanced predictive models’ performance. These variables included the AST assessment one week prior to the prediction date, the patient attributes, and the environmental variables described above. None of these variables improved the models’ performance (detailed results are not shown). Thus, our best models used only the five individual component scores of the patient’s AST assessment on the prediction date as independent variables, making these models easy to use in practice.

The objective of our study was to develop and test new predictive models for asthma control deterioration using a combination of patient demographic information, clinical information, and environmental variables. Using the multiboost with decision stumps classifier, we were able to successfully predict asthma control deterioration one week ahead with reasonable accuracy, demonstrating the feasibility of predictive modeling. However, performance needs to achieve a higher level of accuracy (e.g., >80 %) and PPV, while maintaining high sensitivity and specificity, before such models can be used to support real-time clinical decision making. If accuracy goals can be met, such a model could be integrated into electronic asthma self-monitoring systems, such as the electronic-Asthma Tracker (e-AT) [43], to provide prediction-based decision support and personalized early warnings of potential asthma control deterioration for asthmatic children. In this case, all independent variables used in the model need to be collected by the electronic asthma self-monitoring system. After the user enters his/her current AST assessment into the system, the system will use the model to predict the user’s asthma control level one week later. If the user is predicted to experience asthma control deterioration, the system will display a personalized warning message to the user.

Although not perfect, our results are encouraging, particularly as the first work on predicting a child’s asthma control deterioration one week ahead. In comparison, despite years of work, existing models on predicting asthma exacerbations have low sensitivities (typically <60 %) and low PPVs (typically <27 %) [20–24], much below those achieved by our best model.

Despite bringing significant burden to patients and the healthcare system, asthma continues to be a poorly controlled disease [10]. Poor asthma control is associated with frequent asthma exacerbations [44]. However, an asthma exacerbation is usually preceded by a critical period of asthma control deterioration [14]. This provides opportunity for interventions if early evidence of deterioration can be identified. Physicians, caregivers, and patients all tend to overestimate the level of asthma control, particularly in children [10, 15–18], resulting in poor recognition of deterioration until an acute exacerbation occurs. One way to identify risk of asthma control deterioration is to develop predictive models. To date, predictive models for deteriorating asthma control have focused on asthma exacerbation, often a late manifestation of loss of asthma control [44].

Using environmental variables, patient attributes, and the patient’s daily peak expiratory flow rate (PEFR) in the previous few days, Lee et al. [45] built a model to predict an asthma exacerbation. Lee’s model, however, did not predict asthma control deteriorations preceding an exacerbation, and thus cannot be used to support early intervention to prevent clinical deterioration. In addition, although monitoring PEFR is commonly used to identify early signs of asthma control deterioration, it has several limitations including: (1) the measurement is labor intensive, impacting compliance [43]; (2) PEFR is effort dependent [46] with low reproducibility [47–49]; (3) PEFR primarily assesses large airway airflow and underestimates airflow limitations in medium and small airways [49, 50]; (4) PEFR goals are usually based on the patient’s best PEFR [49], which may differ from the predicted or desired goals [43]; and (5) PEFR goals increase with age and height [51] and must be re-calculated periodically, which is often overlooked [43].

In our AST, asthma control assessments are based on the Asthma Control Test questionnaire adapted for weekly assessment of asthma control status. Thus, we avoid using the more difficult and less accessible forced expiratory volume in 1 second (FEV), forced vital capacity (FVC), and PEFR, and their limitations for use in children.

In our study, the multiboost with decision stumps, support vector machine, and deep learning classifiers performed similarly and achieved reasonable accuracy, sensitivity, specificity, AUC, and NPV. All of these three classifiers could predict a child’s asthma control deterioration one week ahead with reasonable accuracy. The naive Bayes, random forest, and k-nearest neighbor classifiers performed less well, particularly with respect to sensitivity.

The AST assessment reflects the patient’s asthma control level over the past week. Successive patient AST assessments are highly correlated with each other. Also, adding the AST assessment one week prior to the prediction date does not improve the models’ performance. We would expect that obtaining AST assessments on a patient more than once per week will not increase prediction accuracy, as information contained in additional AST assessments has already been included in the AST assessments on the prediction date and one week prior to the prediction date.

Our study has several limitations. First, the patients were recruited during hospitalization for asthma exacerbation. Each year, only ~1.6 % of asthmatic children are hospitalized [4]. As is typical with predictive modeling, our models’ performance is affected by the percentage of uncontrolled asthma in AST assessments. The percentage may be lower in patients not hospitalized than in patients hospitalized. A model’s performance usually degrades as the percentage of uncontrolled asthma in AST assessments decreases. It remains to be seen how our models will perform on patients not hospitalized. Second, we had a small sample size and were limited by the number of patient attributes and environmental variables. Collecting additional AST assessments and patient attributes can potentially improve the models’ performance. Such attributes might include information on allergies, parental asthma [20], healthcare access, the number of prescribing providers [21], viral infection severity [52], compliance with asthma controller medications, and other known predictors of asthma control such as pet exposure [53]. Third, our sample is relatively homogenous. For instance, 66 % of the patients are white. The small sample size limits our capability to (a) detect the association between a variable that is relatively homogenous and the asthma control level, and (b) conduct subgroup analysis to determine whether prediction accuracy differs among various patient subgroups (e.g., by race or by chronic asthma severity level). Fourth, our environmental variable data came from regional monitoring stations and may not reflect a patient’s actual exposures [45, 54, 55]. Accurate measurement of environmental exposures would benefit from using a personal exposure monitor [46, 55–57] and may help increase the models’ performance. We did find environmental variables correlated with an asthmatic child’s level of asthma control, but the correlation was relatively weak [58, 59]. By including the environmental variables, but not the patient’s AST score, in the predictive models, our best model achieved a low sensitivity of 41.7 % and a low AUC of 0.593.

To better understand our predictive models’ performance, we used two evaluation methods simultaneously. These two methods address different situations. When a choice among multiple predictive models needs to be made, these two evaluation methods can provide insights into which model is most suitable for the clinical situation and desired outcome. The patient’s AST score on the prediction date reflects the patient’s asthma control level, and hence can be an approximate surrogate for the environmental variables in the previous seven days assuming they have a non-trivial influence on asthma control. We felt that this, and the high correlation between successive patient AST assessments, rendered our modeling attempts less successful than desired. It is likely that AST assessments have an overpowering influence on the prediction in comparison to environmental variables and other patient attributes, making their relative contributions insignificant. This effect was compounded by our small sample size.

As with any intervention relying on patient-reported data, our technology’s utility hinges on patient adherence to continuous reporting of their data. If patients cannot obtain benefits from efforts expended on reporting their data, adherence is likely to wane. The functionality of predicting asthma control deterioration, once done accurately and incorporated into the e-AT, will provide direct benefits to patients and may help improve patient adherence.

We have several goals in mind for future work. First, we would like to improve the models’ performance. This will be accomplished by: (1) increasing the sample size to improve the capability to detect the association between a variable and the asthma control level, (2) obtaining additional patient attributes among the known predictors of asthma control, (3) collecting additional environmental variables, such as pet exposure at the patient’s home, (4) collecting patient-specific environmental variables with portable monitors rather than estimating from regional monitoring stations, and (5) integrating patient pharmacogenomics information relating to medication metabolism.

Second, we would like to investigate how our models will perform in the ambulatory setting with non-hospitalized patients.

Third, we would like to probe the possibility of making an earlier prediction. In general, the earlier and the more accurate the prediction, the more useful the prediction will be for clinical decision making.

Fourth, we would like to extend our predictive models to incorporate intervention information, in a way similar to that in interrupted time series models or intervention models [60]. Our current models consider no intervention information. However, once prediction-based warnings start to be provided to an asthmatic child, the child may be given a preventive intervention. The intervention will impact the child’s asthma control level in the future and thus needs to be considered in the predictive model.

Our best models predicted with reasonable accuracy a child’s asthma control level one week ahead. With improvements in accuracy, the models can be integrated into electronic asthma self-monitoring systems to provide real-time decision support and personalized early warnings on potential asthma control deterioration for asthmatic children. This will allow implementing preventive actions to reduce asthma exacerbations, improve clinical outcomes, increase quality of life, and reduce healthcare cost.

The study was reviewed and approved by the Institutional Review Boards of the University of Utah and Intermountain Healthcare.