Data source for case-load simulation
The data used to generate the simulated case-load was collected at six HF-clinics in Germany and Spain as part of the MyHeart heart failure management observational study [22] Patients were included in this study if they had chronic HF with elevated levels of the N-terminal prohormone of the brain natriuretic peptide (NT-proBNP ≥ 500 pg/ml), were taking at least 40 mg/day of furosemide or equivalent and were in New York Heart Association (NYHA) functional class II, III or IV.
Participating patients were required to; (a) answer two daily symptom questionnaires using a personal digital assistant, once in the morning and once in the evening and (b) take daily measurements of weight, blood pressure and trans-thoracic impedance (TTI) using a wearable vest. The study was purely observational and no treatment decisions were made based on the data logged. In total, 91 of the patients enrolled in the MyHeart study had sufficient compliance with the measurement system for their data to be included in this simulation; of which 70% were men with a mean (±standard deviation) age of 63 (±12) years, body mass index of 29 (±6) kg/m2 and left ventricular ejection fraction of 31 (±12) %. On average these patients were monitored for 10 months.
Thoracic fluid information from monitoring of trans-thoracic impedance
TTI measurements are sensitive to the amount of fluids in the tissue as fluids increases the conductivity of the tissue [23, 24]. However, uptake of this technology has been slow due to cumbersome prototype technologies and difficulties in interpreting the results since electrical resistance does not directly translate into a lung water assessment [25]. Recent analysis of the MyHeart trial data suggests that non-invasive TTI is more sensitive to impending deterioration than standard measures of fluid accumulation such as weight [16].
Different methods of deterioration detection: alert algorithms used in simulation
Three different alert algorithms to prompt patient reviews was used in this simulation experiment; a rule-of thumb algorithm using weight (weight-RoT) based on the ESC guideline, and two advanced algorithms: a trend algorithm using weight (weight-MACD) and a trend algorithm using trans-thoracic impedance (impedance-CUSUM). These are described in more detail below. Whilst the weight-MACD and impedance-CUSUM have been shown to be more effective in the detection of impending deterioration [13, 16], they may not be as easy to interpret as simple differences, which could paradoxically result in longer review times and worse decisions.
Weight-RoT
Monitoring weight changes caused by fluid retention is routinely recommended for HF management and different algorithms for alerting caregivers to potential worsening HF have been suggested [1, 13, 26]. Typically, the difference between the current and past weight measurements is used to decide whether the patient needs reviewing and/or whether changes to their management made. An increase of 2 kg or more in the past 3 days was used for this algorithm, as recommended in the ESC guidelines [1].
Weight-MACD
Trend detection algorithms have been suggested to improve detection of fluid retention by removing much of the inherent variability found in difference measures and instead look at the trend of change over longer time periods [13, 14]. For this study, the moving average convergence divergence (MACD) trend algorithm was used. This algorithm looks at the differences between two exponentially weighed averages with different time-horizons; one over a long-period and one over a short one. The use of such an algorithm has been shown to improve specificity in detecting worsening HF [13]. The parameter settings for the MACD algorithm were chosen in order to optimise the sensitivity and specificity of detecting impending hospitalisation for worsening HF. The alert threshold was set to be 0.54, which is slightly lower than that defined previously using the MyHeart data [16], and will therefore generate slightly more alerts.
Impedance-CUSUM
Another trend-detection method from process control is to use cumulative sums to detect changes. For this study, the cumulative sum control chart (CUSUM) was used which compares the deviation from a moving average normalised for standard deviation to establish whether a change has occurred [27]. This method has been successfully applied to intra-thoracic impedance and pulmonary pressures [27, 28]. Again the parameter settings for the CUSUM algorithm were optimised, with the alert threshold set to −7, slightly lower than reported previously using the MyHeart dataset [16].
Data preparation
The three algorithms described above were applied to the MyHeart data to create alerts indicating possible deterioration of heart failure. For each of the algorithms, the measurement data was segmented into 28-day episodes that ended in an alert. This process generated 556, 314 and 287 episodes using the weight-RoT, weight-MACD and impedance-CUSUM algorithms respectively. In patients with alerts occurring on consecutive days, we removed every 2nd and 3rd episode to minimise the chances of showing the same alert period within a given case-load simulation. This procedure generated 303, 147 and 134 episodes to be reviewed in the weight-RoT, weight-MACD and impedance-CUSUM arms respectively, these together with the alerts belonging to cases decompensated are presented in Fig. 1.
The resulting alert episodes were divided randomly into 15 groups (or case-load snapshots) and each episode assigned randomly to a fictitious patient name. A proportional amount of episodes containing no alerts within a 28-day windows, were selected from the remaining data and combined with the alert episodes to create 15 virtual case-loads of patients in each of the study arms. The patient episodes without alerts serve no real function within the study other than to create the impression of a real case-load; some patients having alerts and some not.
Simulated telemonitoring station
The simulated HTM data and resultant alerts was presented to the participants using a simple interactive graphical user interface (GUI) that has the feel of a real HTM system. The resultant GUI was designed to be simple yet understandable and capture the telemonitoring process steps given in Fig. 2. The final GUI design was arrived at iteratively. Firstly, a non-functional GUI prototype based loosely on the system used by HTM nurses at Castle Hill Hospital (UK) was shown to a HTM nurse for review and comment.
Recommendations from this nurse were then incorporated into the next design iteration and the GUI was loaded with some test data. This was then shown to a second nurse (ACG) for further review and comment on the functionality of the GUI design. The final GUI design, after incorporating the suggestions of both nurses, with the addition of some additional adjustments, is shown in Fig. 3.
During a run of the simulation, the case-load of patients is displayed in the panel to the left (labelled A on Fig. 3). Those patients that have generated an alert are highlighted in red at the top of this list and will need to be reviewed by the HTM nurse/clinician. When a patient is selected the daily measurement data will be displayed in the main panel (B). For participants randomised to either of the weight algorithm arms, this will display weight data for the preceding 28 days by default; with red circles indicating alerts that have been raised in the past three days. Similarly, if they are randomised to the trans-thoracic impedance arm, they will see the trans-thoracic impedance data by default. By clicking on the buttons (C) above panel B, the nurse/clinician can review the other daily measurement data (i.e. blood pressure, heart rate and weight). In the weight algorithm arms, the trans-thoracic impedance data remains hidden. The values of the most recent HTM measurements (D) are shown below panel B on the right-hand-side. To the left of this (E), basic patient information is provided such as age, sex, NYHA class, HF aetiology and co-morbidities.
Once the data have been reviewed by the nurse/clinician they can then rate how important they felt the alert was using a five point Likert scale (i.e. l = low to 5 = high) (F). They then indicate what action they would take in response to the alert. If they considered the alert to be of no concern, they can click No Action and then move on to the next patient. If they do have concern, they can opt to Call Patient (G).
Simulated call
Since the data used in this simulation was retrospective, no actual calls can be made. To emulate the type of information a call might generate, the data from the comprehensive symptom questionnaires collected during the MyHeart trial was used to create an overview of a patient’s symptoms during the previous 5 days, see Fig. 4. The patients in MyHeart were asked to rate their general mood (How was your mood today?) and wellbeing (How was your general well-being today?) on a 5-point Likert-scale. Responses for the past 5 days were combined with reference lines for the average mood and wellbeing over the preceding two weeks (labelled A on Fig. 4). Multiple choice questions covering symptom levels (Did you have to go to the toilet? (at night); Did you have any coughing last night?; Did you experience breathlessness last night?; Did you experience any chest pain? (at night); Did you feel shortness of breath at rest today?; Did you experience palpitations today?) are color-coded according to their frequency i.e. none = white, once = yellow, twice or more = red (labelled B and C). Questions having only a single answer (Do you feel more ankle or leg swelling than yesterday?; Have you changed your medication?; Did you tolerate exercise better today than yesterday?, Did you feel light-headedness when getting up this morning?) were marked with a cross for yes or empty circles for no (labelled D).
Logging of decision
Having “called” the patient, the nurse/clinician can either click Go back to patient data (E) to further review the monitoring data or click Select action (F) and indicate what action they recommend from the following options;
-
1.
High level of concern, e.g. send a community nurse to the patient
-
2.
Raised level of concern, e.g. increase the dosage of diuretics
-
3.
Beginning level of concern, e.g. close follow-up in the following days
-
4.
Low level of concern: no further action.
However, if the participant feels that these suggested responses do not adequately capture the preferred action they had an option to comment further.
The time spent reviewing each patient is logged automatically by the simulator. Once a case-load of alerts has been reviewed the participant can click on Finished monitoring and the next case-load will be presented or they have the option to pause the simulation. The complete experiment for each participant was expected to last 2–3 h.
Study design
We recruited healthcare professionals (nurses and doctors) in the United Kingdom with experience of managing patients with heart failure using HTM. Participants were recruited via nursing, heart failure and telehealth networks known to the authors, including Twitter, Linked-In and other social media sites.
Potential participants were asked to complete an online questionnaire (using Survey Monkey) to establish their level of experience of managing patients with HF and use of HTM systems. Participants with experience of both HF management and HTM were then randomised to receive alerts generated using one of the described algorithms. They were sent a self-extracting application package containing the simulation program together with instructions on how to install it, how to conduct the experiment and their randomization code. Participants randomised to the trans-thoracic impedance arm also received a short educational video to explain the measurement technology and how to interpret the impendence signal, since only a few participants would’ve been familiar with this measurement modality.
Each participant was then presented with 15 case-loads to review using the simulation system. The order in which the cases were presented to the participants was identical within each study arm.
Statistical considerations: study size, simulation time and analysis
This was a pilot study and with the difference in amount of alerts between the trend algorithms and the rule-of-thumb method we anticipated that only a small number, eight participants in each arm, would be adequate to show differences (see Additional file 1: Appendix 1 for the statistical argument), therefore we aimed to recruit approximately 10 participants in each arm.
Participants were randomised following a randomised string in the order they agreed to take part in the study with dropouts after agreement appended at the end of the string. The primary hypothesis was that the reduction in alerts by applying a trend algorithm or a novel marker will lower the total time spent in the simulated interface (i.e. weight and impedance trend alerts vs. standard alerts). A Mann–Whitney U-test was used with a significance level of 0.05. Secondary analysis included: the suggested interventions (e.g. call GP, review medication, etc.) and the rated importance of the alerts. Since not all participants are monitored during the experiment they might leave the simulation, e.g. to get a cup of tea or coffee, without pausing. This would result in excessively long review times for specific alerts. We therefore limited the maximum time to review an alert to five minutes, thus any review time lasting longer was marked as lasting five minutes. Similarly, sometimes participants might miss to review an alert which would lower their total review time. In these cases we imputed the average review time for that alert.