Skip to main content

Advertisement

We're creating a new version of this page. See preview

  • Research article
  • Open Access
  • Open Peer Review

The past, present and future of opioid withdrawal assessment: a scoping review of scales and technologies

BMC Medical Informatics and Decision Making201919:113

https://doi.org/10.1186/s12911-019-0834-8

  • Received: 6 February 2019
  • Accepted: 6 June 2019
  • Published:

The Correction to this article has been published in BMC Medical Informatics and Decision Making 2019 19:126

Open Peer Review reports

Abstract

Background

A common challenge with all opioid use disorder treatment paths is withdrawal management. When withdrawal symptoms are not effectively monitored and managed, they lead to relapse which often leads to deadly overdose. A prerequisite for effective opioid withdrawal management is early identification and assessment of withdrawal symptoms.

Objective

The objective of this research was to describe the type and content of opioid withdrawal monitoring methods, including surveys, scales and technology, to identify gaps in research and practice that could inform the design and development of novel withdrawal management technologies.

Methods

A scoping review of literature was conducted. PubMed, EMBASE and Google Scholar were searched using a combination of search terms.

Results

Withdrawal scales are the main method of assessing and quantifying opioid withdrawal intensity. The search yielded 18 different opioid withdrawal scales used within the last 80 years. While traditional opioid withdrawal scales for patient monitoring are commonly used, most scales rely heavily on patients’ self-report and frequent observations, and generally suffer from lack of consensus on the criteria used for evaluation, mode of administration, type of reporting (e.g., scales used), frequency of administration, and assessment window.

Conclusions

It is timely to investigate how opioid withdrawal scales can be complemented or replaced with reliable monitoring technologies. Use of noninvasive wearable sensors to continuously monitor physiologic changes associated with opioid withdrawal represents a potential to extend monitoring outside clinical setting.

Keywords

  • Opioid use disorder
  • Surveys
  • Scales
  • Questionnaires
  • Wearable sensor
  • Physiological monitoring

Background

Opioids are natural (e.g., morphine), semi-synthetic (e.g., oxycodone) or synthetic (e.g., tramadol and methadone) narcotics primarily used to treat acute and chronic pain [1]. They are often used recreationally due to their euphoric, tranquilizing, and sedative qualities [2]. Misuse and abuse of opioids and associated overdose, referred to as opioid use disorder (OUD) [3], is a serious public health issue [4] and has been declared a public health emergency in United States [5]. Deaths due to opioid overdose continue to rise. For example, the number of opioid overdose deaths was 5 times higher in 2016 than in 1999, accounting for about 115 deaths per day [6].

OUD has significant societal impacts, such as escalating direct and indirect healthcare costs [79]. Florence et al. [8] estimated the total economic burden of prescription opioid overdose, abuse, and dependence for the year 2013 to be $78.5 billion. They found that more than a third of this amount ($28.9 billion) resulted from increased healthcare and substance abuse treatment costs. Indirect costs include lost workplace productivity [7], and criminal justice costs [7, 9]. In addition, the observed decline in men’s labor force participation from 1999 to 2015 has been associated with increased opioid prescriptions [10]. The opioid crisis is taking a toll on families, especially children, forcing an unprecedented number to enter foster care due to parental substance use [11].

Several treatment paths exist for OUD including opioid maintenance treatment, detoxification (detox) [2, 12], and medication-assisted treatment [13, 14]. Opioid maintenance treatment, a well-established first line approach to handle OUD [15], assesses suitability of opioid users who wish to start treatment, and provides legal substitutes. The detox process involves administration of medication in a controlled and medically supervised environment to reduce the severity of withdrawal symptoms that occur when people stop using opioids [16]. For opioid users who have expressed an informed choice, detox may be completed within a community-based program in up to 12 weeks or as inpatient in up to 28 days [16, 17]. Agonists, substances that fully bind to and stimulate opioid receptors (e.g., methadone, levomethadyl acetate) and partial agonists (e.g., Buprenorphine) medications are frequently used for both maintenance and detox purposes [12]. Medication-assisted treatment is an evidence-based treatment for OUD that combines behavioral therapy and approved medications including methadone, naltrexone, and buprenorphine [13, 14]. There is evidence suggesting that medication-assisted treatment is more effective than opioid maintenance treatment, and detox [13].

A common challenge with all OUD treatment paths is withdrawal management. Opioid users experience challenging and often severe withdrawal symptoms when they abruptly discontinue or reduce opioid intake. Intensity of opioid withdrawal symptoms depends on, among others, the type of opioid used, duration of usage, underlying medical conditions, and family history [18, 19]. Time frame for opioid withdrawal symptoms may be classified into early, peak, and late phases. The early phase is characterized by acute withdrawal symptoms such as lacrimation, yawning, and rhinorrhea. Symptoms including gastrointestinal symptoms, gooseflesh and aching reach their maximum intensity in the peak phase [19], and tail off in the late phase [18]. Detox that is not followed up with evidence-based treatment leads to a state of dysphoria which may expose patients to a high risk of relapse, and, even more tragically, a high risk of opioid overdose and death [20].

A prerequisite for effective opioid withdrawal management is early identification and assessment of withdrawal symptoms. One common assessment method for inpatient facilities (e.g., rehabilitation centers) is patient observation. This method involves frequent patient observations (in some cases every 1–2 h; [21]) and depending on the legal substitute used for treatment, clinicians may be required to take vital signs every 1.5 to 2 h with patient both sitting and standing [22]. Over the years, several withdrawal scales have been developed and used by clinicians to complement patient observation and to aid in assessing opioid withdrawal in outpatient cases. Most scales require patients to self-report their immediate or recent (e.g., in the past 24 h) symptoms experienced. Many signs and symptoms may be lost during the hours of the day when clinicians are not observing patients or when patients are not self-reporting. While efforts have aimed at improving the sensitivity and specificity of opioid withdrawal detection [23], the withdrawal scales have been associated with several limitations such as recall bias, distortion, and imprecision as well as the burden, both cognitive and psychological, due to recurring self-reporting (by patients) or observations (by caregivers) [24].

While opioid overdose deaths have increased tremendously over the past two decades such trend has been much more stable for chronic diseases, such as heart disease (Fig. 1). In fact, the downward trends in fatalities of other chronic diseases might be associated, in part, with advances in remote monitoring tools and techniques which may suggests that opioid withdrawal and relapse have lagged behind. For example, conditions such as diabetes, obesity, hypertension,
Fig. 1
Fig. 1

Percent change from baseline year 2000 of opioid deaths (8407 deaths in 2000) vs. heart disease deaths (728,796 deaths in 2000) from 2000 to 2016. For each category, percent change for each year was calculated as [(number of deaths in that year - number of deaths in year 2000)/number of deaths in year 2000] × 100. Data obtained from [6]

and chronic heart diseases have leveraged and benefited from advancements in smartphone and wearable sensor technologies (e.g., Jakicic et al. [25] monitored obesity using a multi-sensor device worn on the upper arm; Pedone, Chiurco, Scarlata, and Incalzi [26] monitored chronic obstructive pulmonary disease with a multi-sensor). In comparison, technologies to address and combat the opioid epidemic are few and are limited to diversion or monitoring overuse/misuse of prescription opioids (e.g., Electronic Prescribing for Controlled Substances, and CancelRx) with choice of services that would benefit patients [27]. Furthermore, a relatively small number of studies [2830] have explored the use of biosensors to detect physiological changes associated with opioid intake in real-time.

Our initial steps to address this gap was to investigate available opioid withdrawal assessment methods to uncover opportunities for innovative technologies for withdrawal management. To the best of our knowledge, no study has critically reviewed and synthesized the commonly used opioid withdrawal scales. Additionally, there has not been any review of technological methods for opioid withdrawal management. The objective of this research was to review and document different opioid withdrawal monitoring methods, including surveys, scales and technology, to identify gaps in research and practice that could inform design and development of novel withdrawal management technologies.

Methods

A scoping review of literature was conducted. We searched PubMed, Embase, and Google Scholar for articles published until April 19, 2019 using a combination of search terms (Table 1). We included studies if they (1) were written in English, (2) took place in an outpatient or inpatient setting, medication-assisted or opioid treatment program, rehabilitation center or were experiments (3) employed opioid withdrawal scale(s), wearable sensors, mHealth, or other technologies as predictor of opioid withdrawal severity (4) were peer-reviewed. In addition, we adopted the SPICE (setting, perspective, intervention, comparison, evaluation) framework, listed in Table 2. We excluded studies published in a language other than English, animal studies, alcohol or non-opiate studies, literature reviews, and non-peer-reviewed publications.
Table 1

Search terms used in the scoping review

Term

Search terms combination

Predictor

(“opi* withdrawal scale*” OR “biosensor*” OR “wearable sensor*” OR “opioid use disorder” OR “substance withdrawal syndrome” OR “opi* withdrawal syndrome” OR “withdrawal symptom*” OR “prescribed opioid use” OR “mHealth” OR “smart sensing tech*”)

Population

AND (“clinician*” OR “opioid patient*” OR “opioid addict*” OR “opioid user*” OR “caregiver*” OR “physician*” OR “doctor*” OR “surgeon*” OR “healthcare professional” OR “observer” OR “experimenter”)

Setting

AND (“inpatient” OR “outpatient” OR “opioid treatment cent*” OR “opioid treatment program*” OR “opiate treatment program*” OR “treatment cent*” OR “rehab* cent*” OR “medication-assisted treatment” OR “medication assisted treatment” OR “medication assisted treatment program*” OR “medication-assisted treatment program*” OR “medication-assisted treatment therapy” OR “medication assisted treatment therapy” OR “drug addiction cent”, OR “experiment”)

Evaluation

AND (“point score” OR “score” OR “severity of opi* withdrawal” OR “opi* withdrawal severity” OR “severity score” OR “severity of withdrawal” OR “withdrawal severity” OR “severity score” OR “score for severity” OR “withdrawal severity score” OR “withdrawal score for severity” OR “opi* withdrawal severity score” OR “opi* withdrawal score for severity”)

Exclusionary

NOT “animal studies” NOT “rat” NOT “mice” NOT “alcohol” NOT “non-opiate” NOT “literature review”

Table 2

SPICE criteria for inclusion of studies

Parameter

Inclusion criteria

Setting

Outpatient, inpatient, opioid treatment program, opioid treatment center, rehabilitation center, medication-assisted treatment program, medication-assisted treatment center, medication-assisted therapy, drug addiction centers, experiment

Population

Opioid patients, opioid users, opioid addicts, opioid dependents, caregivers, clinicians, doctors, surgeons, physicians, healthcare professionals, observer

Intervention

Opiate withdrawal scales, opioid withdrawal scales, biosensors, wearable sensors, monitoring, physiological monitoring, mHealth

Comparison

N/A

Evaluation

Determine severity of withdrawal

Selection and triage

Article selection was carried out in two stages. In the first stage, two reviewers independently reviewed titles and abstracts against the inclusion and exclusion criteria using a web-based tool for systematic and scoping reviews called Rayyan [31]. The decision to fully review an article was made when both reviewers agreed to include the abstract. The reviewers resolved disagreements regarding article eligibility by discussing with a third reviewer.

In the second stage, the full-text articles were reviewed to determine eligibility. Furthermore, backward and forward reference search were conducted on all full-text articles that met the study selection criteria. Our initial search yielded 312 unique articles from which 104 articles were included after screening the titles and abstracts. Out of these 104 articles, 21 met the inclusion criteria and were included in the final review (see Fig. 2).
Fig. 2
Fig. 2

Process of searching and selecting articles included in the scoping review

Data extraction and analysis

Two reviewers independently read the full text of each article identified for inclusion in the review to extract pertinent data using a data extraction form. The SPICE framework was used to choose elements in the data extraction form. From each article, reviewers independently abstracted the following: setting (e.g., rehabilitation center), population (e.g., opioid user), intervention (e.g., scales, technology), and evaluation (e.g., severity of withdrawal). Reviewers transferred abstracted data to a detailed Excel spreadsheet. Authors met and organized the information extracted from articles into type of withdrawal assessment method, symptoms monitored, and temporal window covered. For the survey-based methods, authors also extracted information on scale name, mode of scale/survey administration and rating criteria employed. Date of scale development was inferred from when the study was first published.

Results

Type of assessments

Of the 21 articles that were included in the final review, 18 (86%) articles assessed opioid withdrawal using scales and surveys. Our search did not yield any empirical studies employing wearable sensors or mobile health (mHealth) apps to monitor opioid withdrawal symptoms. The remaining 3 (14%) articles employed technology to detect physiological changes during opioid intake.

Scales and surveys to monitor opioid withdrawal

The review revealed that opioid withdrawal scales are the main method of assessing and quantifying opioid withdrawal intensity. These scales employ a combination of observable behaviors (signs), patients’ self-reports (symptoms) and/or physiological measures. Overall, this search identified 18 different scales used over the past 80 years (see Fig. 3 for a visual timeline), dating back to Kolb and Himmelsbach’s [32] scale and the most recent being Clinical Opiate Withdrawal Scale (COWS) [19]. Table 3 provides a summary of the scales.
Fig. 3
Fig. 3

Timeline of opioid withdrawal scales development

Table 3

Summary of opioid withdrawal scales (1938–2018)

Scale name

Mode of administration

Scale items

Rating criteria

Temporal window covered

Citation

Kolb and Himmelsbach Scale

Clinician-administered

14

Points-based

24 h

[33]

Himmelsbach Scale

Clinician-administered

14

Points-based

Hourly or 24 h

[34]

SDQ1 or Fraser Scale

Clinician-administered

Patient-reported

6 (clinician)

4 (patient)

Weighted 5-point scale

15, 30, 45, 60, 90, 120 min post-drug

[45]

ARCI2

Patient-reported

550

True or False

Immediate feeling

[46]

OPW3

Patient-reported

29

5-point scale

Immediate feeling

[47]

WOW4

Clinician-administered

84

True/False

Immediate feeling

[35]

SOW5

Clinician-administered

80

True/False

Immediate feeling

[35]

Short Form ARCI

Patient-reported

49

True/False

Immediate feeling

[48]

Wang Scale

Clinician-administered

10

Points-based

10, 20, 30 min post injection

[37]

Judson Scale

Clinician-administered and Patient-reported

10 (clinician)

7 (patient)

4-point scale

Prior to zero time, and 10, 20, 30 min post injection

[38]

OWS6

Patient-reported

32

4-point scale

24 h

[39]

SOWS7

Patient-reported

16

5-point scale

Immediate feeling

[40]

OOWS8

Clinician-administered

13

Present/Absent

10-min observation period

[40]

ARSW9

Patient-reported

16

10-point scale

24 h

[42]

CINA10

Clinician-administered

13

Point-based

5, 10, 15 mins post injection

[49]

SOWS-Gossop

Patient-reported

10

4 -point scale

24 h

[43]

SOWQ11

Patient-reported

20

Anchored 100 mm analogue scale

24 h

[44]

COWS12

Clinician-administered

11

Weighted scale

2-min observation period

[19]

1Single Dose Opiate Questionnaire

2Addiction Research Center Inventory

3Opiate Withdrawal Subjective Experience Scale

4Weak Opiate Withdrawal Scale

5Strong Opiate Withdrawal Scale

6Opiate Withdrawal Scale

7Subjective Opiate Withdrawal Scale

8Objective Opiate Withdrawal Scale

9Adjective Rating Scale for Withdrawal

10Clinical Institute Narcotic Assessment

11Subjective Opiate Withdrawal Questionnaire

12Clinical Opiate Withdrawal Scale

Most of the scales used today are modeled after the Himmelsbach’s [33] scale, which assesses withdrawal syndrome intensity based on changes in observable behaviors (yawning, lacrimation, rhinorrhea, perspiration, tremor, restlessness, emesis, gooseflesh) and physiological measures (hyperpnea, systolic blood pressure, rectal temperature, weight). Using this scale, the total opioid withdrawal syndrome intensity score is computed as the sum of the points scored by each item on the scale.

The scales developed in the 1960s – Single Dose Opiate Questionnaire (SDQ), Addiction Research Center Inventory (ARCI), Opiate Withdrawal Subjective Experience Scale (OPW), focused on subjective effects of opioid withdrawal. To further study these effects, Haertzen, Meketon, and Hooks [34] developed two clinician-administered questionnaires – the 84-item Weak Opiate Withdrawal Scale (WOW) and the 80-item Strong Opiate Withdrawal Scale (SOW) to measure less intense and more intense withdrawal respectively.

With the emergence of methadone maintenance programs in the 1970s, physicians had a harder time establishing diagnosis of opiate dependence. Some patients mastered the art of going from one methadone program to the next, mimicking symptomatology of acute withdrawal to get a large initial dose of methadone. This widespread behavior caused a need for an objective scale to determine late-stage acute opiate withdrawal and withdrawal severity [35]. Researchers (e.g., [36]) developed protocols to precipitate withdrawal in order to assess the level of physical dependence on opioids. Using these protocols, the Wang scale [37], Judson scale [36], Opiate Withdrawal Scale (OWS) [38], Subjective Opiate Withdrawal Scale (SOWS) [39], and Objective Opiate Withdrawal Scale (OOWS) [39] sought to control feigned responses and improve on sensitivity and specificity for detecting withdrawal.

A number of patient-reported outcome instruments have been developed to measure acute symptoms of opioid withdrawal symptoms. These instruments developed to measure withdrawal include OWS [38], SOWS [39], Adjective Rating Scale for Withdrawal (ARSW) [40], Short Opiate Withdrawal Scale (SOWS)-Gossop [41, 42], and Subjective Opiate Withdrawal Questionnaire (SOWQ) [43].

SOWS-Gossop, Clinical Institute Narcotic Assessment (CINA) and Clinical Opiate Withdrawal Scale (COWS) are the most widely used instruments to evaluate opiate withdrawal symptoms in the reviewed articles [19, 22]. SOWS-Gossop [42] is a 10-item, patient-reported scale on which each item is rated as 0 (none), 1 (mild), 2 (moderate), or 3 (severe). It is a short version of the 32-item OWS [38]. CINA is a validated 13-item clinician-administered scale that measures opioid withdrawal signs and symptoms [44]. COWS [19] is a widely used 11-item clinician-administered instrument mostly due to its time-efficiency (i.e., can be completed within 2 min).

Mode of administration

Nine out of 18 available scales are clinician-administered only, 8 are patient-reported only, and 2 are both clinician-administered and patient-reported. Clinician-administered questionnaires are completed using traditional paper and pencil while clinicians talk face-to-face to patients and observe for opioid withdrawal signs. Patient self-administered questionnaires (patient-reported) require patients to complete questionnaires by hand and return them to the clinician. Scales that use both clinician-administered and patient-reported questionnaires (e.g., SDQ) concurrently but independently require both patients and clinicians to use traditional paper and pencil to complete questionnaires. None of the reviewed literature reported on which mode of administration is preferred or provides utility.

Number of scale items

The scales vary in the number of scale items (i.e., signs and/or symptoms) included, ranging from as many as 550 scale items (e.g., ARCI) to as few as 10 items (e.g., SOWS-Gossop). For example, the 550 items on the ARCI subjectively measure drug-induced effects as well as effects associated with personality and psychiatric disorders. The short-form of the ARCI has 49 items that allows for quick administration, and repeated measurements. On the other hand, OWS [38] is a 32-item patient-reported checklist of opiate withdrawal symptoms, with SOWS-Gossop [42] being a 10-item winnowed version.

Scale items are made up of signs and/or symptoms. Whereas a symptom is a manifestation of a disease apparent to a patient alone, a sign is a manifestation of a disease that a clinician and/or patient can perceive [48]. Thus, while signs are usually observed by the clinician, symptoms are self-reported by patients. Lacrimation and yawning were the most frequent signs, appearing on 15 out of the 18 scales. Weight loss, shivering, heart rate, hypernea, temperature change and blood pressure were the least frequent signs, appearing on 2 out of 18 scales (see Fig. 4a). Joint and/or muscle aches was the most frequent symptom, appearing on 11 out of 18 scales (see Fig. 4b). The least frequent scales were pleasant sick and feverish. One of the most widely used scales, COWS, assesses both signs and symptoms. Items include anxiety or irritability, gastrointestinal upset, restlessness, bone or joint aches, sweating, rhinorrhea, tremor, gooseflesh, yawning, pupil size, and pulse rate.
Fig. 4
Fig. 4

a Signs versus number of scales they appeared on; b Symptoms versus number of scales they appeared on

Rating criteria

The scales differ in their use of rating criteria. For each item on a scale, various response categories are used to rate its clinical severity. Across all scales, ratings for items are summed up to create a total score, with a higher score indicating greater severity. Five (ARCI, WOW, SOW, Short Form ARCI, OOWS) of the scales used binary rating systems (“true/false” or “present/absent”). Three out of these 5 scales are clinician administered and the remaining 2 are patient-reported. Here, the total score for each scale is the sum of “true” or “present” items. Only 1 scale, SOWQ, employed a 20-item anchored analog scale. Patients rate each item on a 100 mm analogue scale anchored on both ends by pairs of pleasant and unpleasant feelings. The total score is the sum of analog scores for all 20 items [43].

While most scales use Likert scale ratings, there is no agreement on the type of scale with four (Judson Scale, OWS, SOWS-Gossop out of 18 scales use a 4-point rating, 2 (OPW, SOW) out of 18 use a 5-point rating, 3 (Kolb and Himmelsbach Scale, Himmelsbach Scale, Wang Scale) out of 18 use point-based rating, and 1 (ARSW) out of 18 scales uses a 10-point rating system. One scale, SDQ, uses a weighted 5-point rating system, for which physiological measures are given highest weights than behavioral signs, followed by patient reported symptoms. Many of the studies did not provide the rationale behind choice of points or weights for items on the scales.

The scales also differ in the value of total score. For example, whereas scores for the 11-item CINA range from 0 to 31 and higher scores are associated with more severe withdrawal syndrome, scores for the 11-item COWS range from 0 to 47 with specific ranges for the level of severity (e.g., scores from 5 to 12 are considered mild, scores from 13 to 24 are considered moderate, scores from 25 to 26 are considered moderately severe and scores more than 36 are considered severe withdrawal).

Temporal window covered

Temporal window covered differ among the scales depending, in part, on the nature of the legal substitute used (e.g., methadone or naloxone) to precipitate withdrawal, mode of drug administration (e.g., sublingual or intramuscular), and mode of scale administration. Five out of 18 scales, mostly patient-administered, have 24-h post-drug temporal window. Six scales require addicts to report their “immediate feeling”. The remaining scales varied markedly in temporal window covered, ranging from a 2-min observation period for COWS to 10, 20, 30 min post injection for the Wang scale.

Technologies to monitor opioid intake

Three studies employed wearable biosensors to detect physiological changes during opioid intake. In all three studies, the biosensors were attached to the wrist as a band and the common biomarkers that were monitored included electrodermal activity (EDA), skin temperature, and whole-body acceleration. The temporal windows covered were before, during, and after opioid administration.

The first of these studies, a preliminary observational one, employed a wristband biosensor (Q sensor™, Affectiva, Waltham, MA) to continuously measure electrodermal activity (EDA), skin temperature, and physical movements of 4 patients before, during, and after cocaine and opioid administration. They found that patients with extensive opioid use demonstrated little to no change in EDA while low-moderate users of opioid showed rise in EDA and decrease in skin temperature [28]. In a follow up study, Carreiro et al. [29] employed the same wristband biosensor to continuously measure EDA, skin temperature, and physical movements of 30 patients before, during, and after naloxone administration. They found that although EDA did not show significant difference post administration compared to baseline, participants’ average skin temperature was significantly higher after naloxone administration. Overall movement was found to have decreased significantly after drug administration [29]. In the most recent study, Mahmud et al. [30] collected EDA, skin temperature and accelerometer data from 30 participants who wore the same wristband to develop an automated real-time system that detects opioid intake with 99% accuracy.

Discussion

The scoping review of different opioid withdrawal monitoring methods revealed greater research efforts and emphasis on scales/surveys to monitor symptoms compared to technologies, which focused solely on symptoms associated with opioid uptake. While limited, advancements in non-invasive and wearable sensor technology could potentially serve as efficient complementary solutions in managing withdrawal symptoms either in inpatient or remote settings.

Challenges with existing monitoring methods

Our results show that opioid withdrawal scales are and continue to be the main instrument used to assess and quantify opioid withdrawal symptoms. Over the years, scales have been developed either to assess the degree of physical dependence (before buprenorphine or methadone induction) and/or to compare the efficacy of opioid withdrawal treatments [19]. However, it is very difficult to compare these scales because of their wide heterogeneity, especially, with respect to the number of scale items, rating criteria, and temporal window covered.

CINA, COWS, and SOWS-Gossop appear to be the most widely used in research and clinical settings [19, 22]. In particular, COWS, designed to be administered by a clinician, is known to be more practical and relatively easy to use [19, 22]. However, with respect to the other scales there is no evidence suggesting the usability, practicality or lack thereof. In addition, while there are overlap and similarities between the scales, evidence of cross-validation is largely absent. Future work is needed to investigate context-dependent factors contributing to adoption and efficacy of different scales in different settings.

Monitoring of opioid withdrawal with scales can be conceptualized as sampling of patients’ behavior or experience over time. However, not all signs and symptoms may be captured by questionnaires or retrospective reports of behavior [50]. Reliance on global assessments or retrospective reports prevents clinicians from accurately understanding and characterizing dynamic behavioral changes over time and across situations in both real-world and clinical settings [51].

Another issue is that opioid withdrawal scales that require subjective patients’ reports ask for recall or summary information over long periods of time. Patients’ contexts and mental states at the time of recall often bias their memory retrieval process [50]. This exposes these scales to recall bias [51] which may affect the quality and reliability of the data provided. Indeed, opioid withdrawal scales may be considered as patient-reported outcome measures (PROMs) [52]. A patient-reported outcome is any information on the outcome of healthcare directly reported by a patient without interpretation or modification by a clinician [53]. Administering PROMs that measure subjective symptoms, like opioid withdrawal scales, requires consideration of issues about data collection – data source (self vs. proxy), mode of administration (self vs. interviewer) and method of administration (paper-and-pencil vs. electronic), and scoring [54]. These issues have limitations which complicate scoring and analysis of opioid withdrawal scales response data [54]. Scales that require direct response may be limited by patients’ cognitive or communication abilities. Those that require a proxy to respond about a patient’s experiences may not accurately represent the patient’s subjective experiences [54]. Scales that require patients to self-administer and record responses have potential for missing data, whereas those that require an interviewer to read questions and record responses suffer from interviewer costs and interviewer bias [55]. Scales that are administered using paper-and-pencil are time consuming and are prone to data entry errors [56]. Patient self-administered PROMs using technology minimizes data entry errors, can provide immediate scoring and are amenable to incorporation within electronic health records [57]. Scores for most PROMs, including opioid withdrawal scales, are based on the classical test theory, and are computed as a linear combination of all items on a particular PROM [58, 59]. For a score on an opioid withdrawal scale to be considered valid, all items must be used, making it test-dependent [54, 58, 59].

The findings suggest that some opioid withdrawal scales may impose high length, complexity and frequency of administration to users. Lengthy questionnaires have been identified as a general obstacle in clinical practice [60] and may lead to increased respondent burden and response fatigue, leading to reduced completion, and reduced data quality [61]. While more recent scales are relatively short, they still require physical and cognitive efforts to understand the questions, recall information from memory, evaluate the connection between the retrieved information and the question, and communicate their responses [62]. Furthermore, patient-reported scales demand that patients do not have visual impairments and are able to read and write in the language of the questionnaire. Since some scales require several observations and measurements per day, there is a need to support clinical decision-making by facilitating the comparison between measurements (e.g., to determine trends over time). No evidence was found in the reviewed literature suggesting that this need is currently fulfilled.

Potential technological solutions

The current method of monitoring opioid withdrawal using scales is challenging outside of clinic or research environments. An opioid monitoring method that accurately monitors withdrawal symptoms as they occur in real time would provide several distinct advantages including the ability to obtain environmental and behavioral contexts surrounding withdrawal symptoms [28], additional source of information does not rely on self-reports and does not suffer from recall bias, and the potential for advanced data analytics, trend analysis, and decision support.

Using noninvasive wearable sensors, including temperature, accelerometer, electrodermal activity, and photo plethysmography sensors, to continuously monitor physiologic changes associated with opioid withdrawal represents a potential to extend monitoring outside clinical setting [63]. Sensors have shown promise to measure symptoms such as tremor, joint/muscle aches [64], gooseflesh [65], and anxiety [66]. Unlike opioid withdrawal scales, wearable technologies have the advantage of using biosensors to continuously measure and record physiologic changes in multiple contexts [67]. Continuous biosensing may provide a way to decrease opioid withdrawal observation time, and possibly allow for remote monitoring [68].

Even though remote monitoring technologies have shown promise in the management of chronic diseases (e.g., [60, 61]), application of such technologies in opioid withdrawal monitoring has been very limited. Short- and long-term physiological symptoms associated with opiate withdrawal timelines have been established [18]. This implies that wearable smart sensing technologies may be used to effectively and reliably identify, evaluate, and communicate physiological responses associated with opioid withdrawal and may inform clinical decision support tools and self-management technologies, with minimal burden to patients or care providers. Additionally, by retroactively examining patient-specific data on withdrawal symptoms and future doses, information can be provided back to clinical researchers to derive better intervention strategies.

While sensor-enabled remote monitoring tools to assess signs and symptoms of opioid withdrawal shows potential, several important challenges remain to be investigated. Most importantly, patient engagement and compliance have consistently shown to be problematic in other domains utilizing such tools and technologies [69, 70], particularly diabetes [71]. Work is needed to understand if such tools and technologies will be used on a sustainable manner and if not, investigate contextual factors contributing to such behaviors.

Limitations

There are some limitations in the study that warrant discussion. The scoping review utilized relatively fewer, albeit relevant, number of databases to identify potentially eligible studies. Despite this limitation, we found saturation in the types and descriptions of subjective methods for opioid withdrawal monitoring (e.g., surveys and scales). The limited findings pertaining to technology-based methods may likely be due to the lagging nature of technology development for opioid withdrawal and/or uptake monitoring. It is also possible that technology design processes are not effectively captured and disseminated using traditional peer-reviewed analysis approach. Indeed, to capture emerging trends in technologies for opioid withdrawal monitoring, non-empirical investigations like technology landscape analysis of commercial technologies (e.g., mHealth apps) may be more appropriate [72].

Conclusions

While traditional opioid withdrawal scales for patient monitoring are commonly used, most scales rely heavily on patients’ self-report and frequent observations, and generally suffer from lack of consensus on the criteria used for evaluation, mode of administration, type of reporting (e.g., scales used), frequency of administration, and assessment window. Therefore, it is timely to investigate how such scales can be complemented or replaced with reliable monitoring technologies. Smart sensing technologies that provide real-time information on the physiological and psychological status of patients have shown promise in addressing a similar need for other conditions such as diabetes, cardiovascular diseases, and mental health and may significantly, and proactively, improve treatment care of opioid patients by keeping clinicians aware of patient status.

Work is in progress to identify viable technologies to assess a wide range of withdrawal symptoms more objectively. This includes a comprehensive systematic review and analysis of past, present and future trends in wearable sensor technologies. Our long-term goal is to develop a technological framework using a human-centered design process that is non-invasive, reliable, and proactive to detect and manage opiate withdrawal symptoms by utilizing an array of sensors that can be employed to detect temporal and spectral patterns of physiological and psychological responses associated with short- and long-term withdrawal symptoms.

Notes

Abbreviations

ARCI: 

Addiction research center inventory

ARSW: 

Adjective rating scale for withdrawal

CINA: 

Clinical institute narcotic assessment

COWS: 

Clinical opiate withdrawal scale

Detox: 

Detoxification

EDA: 

Electrodermal activity

OOWS: 

Objective opiate withdrawal scale

OPW: 

Opiate withdrawal subjective experience scale

OUD: 

Opioid use disorder

OWS: 

Opiate withdrawal scale

PROMs: 

Patient-reported outcome measures

SDQ: 

Single dose opiate questionnaire

SOW: 

Strong opiate withdrawal scale

SOWQ: 

Subjective opiate withdrawal questionnaire

SOWS: 

Subjective opiate withdrawal scale

SOWS-Gossop: 

Short opiate withdrawal scale-Gossop

SPICE: 

Setting, perspective, intervention, comparison, evaluation

WOW: 

Weak opiate withdrawal scale

Declarations

Acknowledgments

The open access publishing fees for this article have been covered by the Texas A&M University Open Access to Knowledge Fund (OAKFund), supported by the University Libraries and the Office of the Vice President for Research.

Funding

This independent work was supported by the Texas A&M Triads for Transformation. The funding body was not involved in any aspects of the study design, analysis, or dissemination.

Authors’ contributions

JKN, FS, RKM, ME contributed to the conception, design, and conduct of the study. All authors contributed to the preparation, review, and approval of the final manuscript and the decision to submit the manuscript for publication.

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Authors’ Affiliations

(1)
Industrial and Systems Engineering Department, Texas A and M University, College Station, TX 77843, USA

References

  1. Monwell B, Bülow P, Gerdner A. Type of opioid dependence among patients seeking opioid substitution treatment: are there differences in background and severity of problems? Subst Abuse Treat Prev Policy. 2016;11(1):23.PubMedPubMed CentralGoogle Scholar
  2. Schuckit MA. Treatment of opioid-use disorders. N Engl J Med. 2016;375(4):357–68.PubMedGoogle Scholar
  3. American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders (fifth edition). Washington, DC: American Psychiatric Association; 2013.Google Scholar
  4. Campbell CI, Bahorik AL, VanVeldhuisen P, Weisner C, Rubinstein AL, Ray GT. Use of a prescription opioid registry to examine opioid misuse and overdose in an integrated health system. Prev Med. 2018;110:31–7.PubMedPubMed CentralGoogle Scholar
  5. Pravetoni M, Robinson C, Schmiel SE, Mueller DL. Alum adjuvant is more effective than MF59 in promoting early germinal center formation in response to peptide-protein conjugates and enhancing efficacy of candidate vaccines against opioid abuse in adult and old mice. J Immunol. 2018;200(1 Supplement):181.20.Google Scholar
  6. Centers for Disease Control and Prevention, & National Center for Health Statistics. (2017). Underlying Cause of Death 1999–2015 on CDC WONDER Online Database, released 2017. Data are from the Multiple Cause of Death Files, 1999–2016, as compiled from data provided by the 57 vital statistics jurisdictions through the Vital Statistics Cooperative Program. Accessed at http://wonder.cdc.gov/ucd-icd10.html on August 1, 2018.
  7. Birnbaum HG, White AG, Schiller M, Waldman T, Cleveland JM, Roland CL. Societal costs of prescription opioid abuse, dependence, and misuse in the United States. Pain Med. 2011;12(4):657–67.PubMedGoogle Scholar
  8. Florence CS, Zhou C, Luo F, Xu L. The economic burden of prescription opioid overdose, abuse, and dependence in the United States, 2013. Med Care. 2016;54(10):901–6.PubMedPubMed CentralGoogle Scholar
  9. McAdam-Marx C, Roland CL, Cleveland J, Oderda GM. Costs of opioid abuse and misuse determined from a Medicaid database. J Pain Palliat Care Pharmacother. 2010;24(1):5–18.PubMedGoogle Scholar
  10. Krueger AB. Where have all the workers gone?: an inquiry into the decline of the U.S. labor force participation rate. Brook Pap Econ Act. 2017;2017(2):1–87.Google Scholar
  11. Rivlin, A. M. and Matthew, D. B. (2017). Unfinished business: bipartisan help for child victims of the opioid crisis. Retrieved September 17, 2018, from https://www.brookings.edu/opinions/unfinished-business-bipartisan-help-for-child-victims-of-the-opioid-crisis/.Google Scholar
  12. Stotts AL, Dodrill CL, Kosten TR. Opioid dependence treatment: options in pharmacotherapy. Expert Opin Pharmacother. 2009;10(11):1727–40.PubMedPubMed CentralGoogle Scholar
  13. Fullerton CA, Kim M, Thomas CP, Lyman DR, Montejano LB, Dougherty RH, et al. Medication-assisted treatment with methadone: assessing the evidence. Psychiatr Serv. 2014;65(2):146–57.PubMedGoogle Scholar
  14. Maglione MA, Raaen L, Chen C, Azhar G, Shahidinia N, Shen M, et al. Effects of medication assisted treatment (MAT) for opioid use disorder on functional outcomes: a systematic review. J Subst Abus Treat. 2018;89:28–51.Google Scholar
  15. Soyka M, Strehle J, Rehm J, Bühringer G, Wittchen H-U. Six-year outcome of opioid maintenance treatment in heroin-dependent patients: results from a naturalistic study in a nationally representative sample. Eur Addict Res. 2017;23(2):97–105.PubMedGoogle Scholar
  16. Diaper AM, Law FD, Melichar JK. Pharmacological strategies for detoxification. Br J Clin Pharmacol. 2014;77(2):302–14.PubMedPubMed CentralGoogle Scholar
  17. Pilling S, Strang J, Gerada C. Psychosocial interventions and opioid detoxification for drug misuse: summary of NICE guidance. BMJ. 2007;335(7612):203–5.PubMedPubMed CentralGoogle Scholar
  18. Farrell M. Opiate withdrawal. Addiction. 1994;89(11):1471–5.PubMedGoogle Scholar
  19. Wesson DR, Ling W. The clinical opiate withdrawal scale (COWS). J Psychoactive Drugs. 2003;35(2):253–9.PubMedGoogle Scholar
  20. Nunes EV, Gordon M, Friedmann PD, Fishman MJ, Lee JD, Chen DT, et al. Relapse to opioid use disorder after inpatient treatment: protective effect of injection naltrexone. J Subst Abus Treat. 2018;85:49–55.Google Scholar
  21. American Society of Addiction Medicine: ASAM National Practice Guideline for the use of medications in the treatment of addiction involving opioid use. 2015. ASAM website. https://www.asam.org/docs/default-source/practice-support/guidelines-and-consensus-docs/asam-national-practice-guideline-supplement.pdf. Accessed 20 Sept 2018.
  22. Sigmon SC, Bisaga A, Nunes EV, O'Connor PG, Kosten T, Woody G. Opioid detoxification and naltrexone induction strategies: recommendations for clinical practice. Am J Drug Alcohol Abuse. 2012;38(3):187–19.PubMedPubMed CentralGoogle Scholar
  23. Tompkins DA, Bigelow GE, Harrison JA, Johnson RE, Fudala PJ, Strain EC. Concurrent validation of the clinical opiate withdrawal scale (COWS) and single-item indices against the clinical institute narcotic assessment (CINA) opioid withdrawal instrument. Drug Alcohol Depend. 2009;105(1–2):154–9.PubMedPubMed CentralGoogle Scholar
  24. Fishman SM, Wilsey B, Yang J, Reisfield GM, Bandman TB, Borsook D. Adherence monitoring and drug surveillance in chronic opioid therapy. J Pain Symptom Manag. 2000;20(4):293–307.Google Scholar
  25. Jakicic JM, Davis KK, Rogers RJ, King WC, Marcus MD, Helsel D, et al. Effect of wearable technology combined with a lifestyle intervention on long-term weight loss: the IDEA randomized clinical trial. Jama. 2016;316(11):1161–71.PubMedPubMed CentralGoogle Scholar
  26. Pedone C, Chiurco D, Scarlata S, Incalzi RA. Efficacy of multiparametric telemonitoring on respiratory outcomes in elderly people with COPD: a randomized controlled trial. BMC Health Serv Res. 2013;13(1):82.PubMedPubMed CentralGoogle Scholar
  27. Uhrig P. Changing the course of the opioid epidemic: the power and promise of proven technology; 2018.Google Scholar
  28. Carreiro S, Smelson D, Ranney M, Horvath KJ, Picard RW, Boudreaux ED, et al. Real-time mobile detection of drug use with wearable biosensors: a pilot study. J Med Toxicol. 2015;11(1):73–9.PubMedGoogle Scholar
  29. Carreiro S, Wittbold K, Indic P, Fang H, Zhang J, Boyer EW. Wearable biosensors to detect physiologic change during opioid use. J Med Toxicol. 2016;12(3):255–62.PubMedPubMed CentralGoogle Scholar
  30. Mahmud MS, Fang H, Wang H, Carreiro S, Boyer E. Automatic detection of opioid intake using wearable biosensor. In: 2018 international conference on computing, networking and communications (ICNC): IEEE; 2018. p. 784–8.Google Scholar
  31. Ouzzani M, Hammady H, Fedorowicz Z, Elmagarmid A. Rayyan—a web and mobile app for systematic reviews. Syst Rev. 2016;5(1):210.PubMedPubMed CentralGoogle Scholar
  32. Kolb L, Himmelsbach CK. Clinical studies of drug addiction, III: a critical review of the withdrawal treatments with method of evaluating abstinence syndromes. Am J Psychiatr. 1938;94(4):759–99.Google Scholar
  33. Himmelsbach CK. The morphine abstinence syndrome, its nature and treatment. Ann Intern Med. 1941;15(5):829–39.Google Scholar
  34. Haertzen CA, Meketon MJ, Hooks NT. Subjective experiences produced by the withdrawal of opiates. Br J Addict. 1970;65:245–55.Google Scholar
  35. Wang RI, Wiesen RL, Lamid S, Roh BL. Rating the presence and severity of opiate dependence. Clin Pharmacol Ther. 1974;16(4):653–8.PubMedGoogle Scholar
  36. Judson BA, Himmelberger DU, Goldstein A. The naloxone test for opiate dependence. Clin Pharmacol Ther. 1980;27(4):492–501.PubMedGoogle Scholar
  37. Blachly PH. Naloxone for diagnosis in methadone programs. Jama. 1973;224(3):334–5.PubMedGoogle Scholar
  38. Bradley BP, Gossop M, Phillips GT, Legarda JJ. The development of an opiate withdrawal scale (OWS). Br J Addict. 1987;82:1139–42.PubMedGoogle Scholar
  39. Handelsman L, Cochrane KJ, Aronson MJ, Ness R, Rubinstein KJ, Kanof PD. Two new rating scales for opiate withdrawal. Am J Drug Alcohol Abuse. 1987;13(3):293–308.PubMedGoogle Scholar
  40. Bickel WK, Stitzer ML, Bigelow GE, Liebson IA, Jasinski DR, Johnson RE. Buprenorphine: dose-related blockade of opioid challenge effects in opioid dependent humans. J Pharmacol Exp Ther. 1988;247(1):47–53.PubMedGoogle Scholar
  41. Vernon MK, Reinders S, Mannix S, Gullo K, Gorodetzky CW, Clinch T. Psychometric evaluation of the 10-item short opiate withdrawal scale-Gossop (SOWS-Gossop) in patients undergoing opioid detoxification. Addict Behav. 2016;60:109–16.PubMedGoogle Scholar
  42. Gossop M. The development of a short opiate withdrawal scale (SOWS). Addict Behav. 1990;15:487–90.PubMedGoogle Scholar
  43. Loimer N, Linzmayer L, Grünberger J. Comparison between observer assessment and self rating of withdrawal distress during opiate detoxification. Drug Alcohol Depend. 1991;28(3):265–8.PubMedGoogle Scholar
  44. Peachey JE, Lei H. Assessment of opioid dependence with naloxone. Br J Addict. 1988;83(2):193–201.PubMedGoogle Scholar
  45. Haertzen CA, Harris EH, Belleville RE. Development of the addiction research center inventory (ARCI): selection of items that are sensitive to the effects of various drugs. Psychopharmacology. 1963;4(3):155–66.Google Scholar
  46. Haertzen CA, Meketon MJ. Opiate withdrawal as measured by the addiction research center inventory (ARCI). Dis Nerv Syst. 1968;29(7):450–5.PubMedGoogle Scholar
  47. Martin WR, Sloan JW, Sapira JD, Jasinski DR. Physiologic, subjective, and behavioral effects of amphetamine, methamphetamine, ephedrine, phenmetrazine, and methylphenidate in man. Clin Pharmacol Ther. 1971;12(2part1):245–58.PubMedGoogle Scholar
  48. King LS. Signs and symptoms. JAMA. 1968;206(5):1063–5.PubMedGoogle Scholar
  49. Fraser HF, Van Horn GD, Martin WR, Wolbach AB, Isbell H. Methods for evaluating addiction liability. (A) “attitude of opiate addicts toward opiate-like drugs, (B) a short-term “direct” addiction test”. J Pharmacol Exp Ther. 1961;133:371–4.PubMedGoogle Scholar
  50. Shiffman S, Stone AA, Hufford MR. Ecological momentary assessment. Annu Rev Clin Psychol. 2008;4:1–32.PubMedGoogle Scholar
  51. Prince SA, Reid RD, Bernick J, Clarke AE, Reed JL. Single versus multi-item self-assessment of sedentary behaviour: a comparison with objectively measured sedentary time in nurses. J Sci Med Sport. 2018;21(9):925–9.PubMedGoogle Scholar
  52. Bansal D, Bhagat A, Schifano F, Gudala K. Role of patient-reported outcomes and other efficacy endpoints in the drug approval process in Europe (2008–2012). J Epidemiol Glob Health. 2015;5(4):385–95.PubMedGoogle Scholar
  53. Black N. Patient reported outcome measures could help transform healthcare. Bmj. 2013;346:f167.PubMedGoogle Scholar
  54. Cella DF, Hahn EA, Jensen SE, Butt Z, Nowinski CJ, Rothrock N, Lohr KN. Patient-reported outcomes in performance measurement (Vol. 97). Research Triangle Park, NC: RTI Press; 2015.Google Scholar
  55. Mercieca-Bebber R, Palmer MJ, Brundage M, Calvert M, Stockler MR, King MT. Design, implementation and reporting strategies to reduce the instance and impact of missing patient-reported outcome (PRO) data: a systematic review. BMJ Open. 2016;6(6):e010938.PubMedPubMed CentralGoogle Scholar
  56. MacKenzie SB, Podsakoff PM. Common method bias in marketing: causes, mechanisms, and procedural remedies. Journal of retailing. 2012;88(4):542–55.Google Scholar
  57. Coons SJ, Gwaltney CJ, Hays RD, Lundy JJ, Sloan JA, Revicki DA, et al. Recommendations on evidence needed to support measurement equivalence between electronic and paper-based patient-reported outcome (PRO) measures: ISPOR ePRO good research practices task force report. Value Health. 2009;12(4):419–29.PubMedGoogle Scholar
  58. DeVellis RF. Classical test theory. Med Care. 2006;1:S50–9.PubMedGoogle Scholar
  59. Martinez-Martin P. Composite rating scales. J Neurol Sci. 2010;289(1–2):7–11.PubMedGoogle Scholar
  60. Mark TL, Johnson G, Fortner B, Ryan K. The benefits and challenges of using computer-assisted symptom assessments in oncology clinics: results of a qualitative assessment. Technol Cancer Res Treat. 2008;7(5):401–5.PubMedGoogle Scholar
  61. Rolstad S, Adler J, Rydén A. Response burden and questionnaire length: is shorter better? A review and meta-analysis. Value Health. 2011;14(8):1101–8.PubMedGoogle Scholar
  62. Bowling A. Mode of questionnaire administration can have serious effects on data quality. J Public Health. 2005;27(3):281–91.Google Scholar
  63. Chintha KK, Indic P, Chapman B, Boyer EW, Carreiro S. Wearable biosensors to evaluate recurrent opioid toxicity after naloxone administration: a Hilbert transform approach. Proc Annu Hawaii Int Conf Syst Sci. 2018;2018:3247–52.PubMedPubMed CentralGoogle Scholar
  64. Treister R, Kliger M, Zuckerman G, Aryeh IG, Eisenberg E. Differentiating between heat pain intensities: the combined effect of multiple autonomic parameters. PAIN®. 2012;153(9):1807–14.Google Scholar
  65. Benedek M, Kaernbach C. Physiological correlates and emotional specificity of human piloerection. Biol Psychol. 2011;86(3):320–9.PubMedPubMed CentralGoogle Scholar
  66. Rosebrock LE, Hoxha D, Norris C, Cacioppo JT, Gollan JK. Skin conductance and subjective arousal in anxiety, depression, and comorbidity. J Psychophysiol. 2016;31(4):145–57.Google Scholar
  67. Krinsley JS, Chase JG, Gunst J, Martensson J, Schultz MJ, Raccone FS, Wernerman J, Bohe J, De Block C, Desaive T, Kalfon P, Preiser J. Continuous glucose monitoring in the ICU: clinical considerations and consensus. Crit Care. 2017;21:197.PubMedPubMed CentralGoogle Scholar
  68. Steinhubl SR, Waalen J, Edwards AM, Ariniello LM, Mehta RR, Ebner GS, et al. Effect of a home-based wearable continuous ECG monitoring patch on detection of undiagnosed atrial fibrillation: the mSToPS randomized clinical trial. JAMA. 2018;320(2):146–55.PubMedGoogle Scholar
  69. Bertz JW, Epstein DH, Preston KL. Combining ecological momentary assessment with objective, ambulatory measures of behavior and physiology in substance-use research. Addict Behav. 2018;83:5–17.PubMedGoogle Scholar
  70. Schubart JR, Stuckey HL, Ganeshamoorthy A, Sciamanna CN. Chronic health conditions and internet behavioral interventions: a review of factors to enhance user engagement. Comput Inform Nurs. 2011;29(2):81–92.PubMedPubMed CentralGoogle Scholar
  71. Nelson LA, Coston TD, Cherrington AL, Osborn CY. Patterns of user engagement with mobile-and web-delivered self-care interventions for adults with T2DM: a review of the literature. Curr Diab Rep. 2016;16(7):66.PubMedPubMed CentralGoogle Scholar
  72. Guderian CC. Identifying emerging technologies with smart patent indicators: the example of smart houses. Int J Innov Technol Manag. 2019;1:1950040.Google Scholar

Copyright

© The Author(s). 2019
corrected publication 2019

Advertisement