- Research article
- Open Access
- Open Peer Review
Adherence to standardized assessments through a complexity-based model for categorizing rehabilitation©: design and implementation in an acute hospital
BMC Medical Informatics and Decision Makingvolume 18, Article number: 21 (2018)
The use of measurement instruments has become a major issue in physical therapy, but their use in daily practice is rare. The aim of this paper is to describe adherence to standardized assessments by physical therapists using a complexity-based model for categorizing rehabilitation (CMCR) at the Clínica Alemana of Santiago, an acute hospital in Chile.
This retrospective cohort study used 145,968 participant records that were stored in the inpatient database between July 2011 and December 2015. Adherence to the CMCR by 31 physical therapists working with intensive care unit (ICU) and non-ICU inpatients was assessed every quarter using the electronic patient records (EPR). This instrument (CMCR) linked clinical functional assessment to the degree of severity, thereby setting a score used to categorize patients as low, medium and high complexity. 96,400 instances of inpatient care where the physician recommended physical therapy were categorized. This was from a total of 145,968 instances of inpatient care recorded throughout the duration of the study (17 quarters). Trends in adherence were analyzed using a Prais-Winsten regression (a first-order autoregressive model). The trends were compared using a repeated measures ANOVA for mixed models with a significance level of 0.05. The use of the CMCR was included as one of the organization’s quality indicators associated with the hospital’s accreditation processes.
Adherence increased by 1.48% every quarter (p = 0.005) for both ICU and non-ICU patients. On average, adherence with ICU patients was 16.98% greater than with non-ICU patients. Although adherence was always greater with ICU patients, the rate of increase with non-ICU patients was significantly greater: 1.62% (p = 0.007) vs. 1.28% (p = 0.003), respectively.
The CMCR facilitated adherence to standardized assessments by physical therapists working with ICU and non-ICU inpatients in an acute hospital, while linking this instrument to the organization’s quality management process proved to be an effective strategy for the duration of this study (17 quarters).
The use of assessment tools in clinical practice is important for both clinical rehabilitation professionals and patients. It is also essential when it comes to showing results . The use of assessment scales contributes to improving clinical reasoning, objectifying and quantifying the interventions, which is essential for a good clinical practice [2,3,4].
Despite these benefits, the standardized use of clinical assessment scales in rehabilitation remains a challenge. In their study, Jette et al. interviewed 1000 physical therapists, all members of the American Physical Therapy Association (APTA). That study revealed that only 48% of physical therapists used assessment scales during their daily clinical practice, as they felt it improved communication with patients as well as care planning. The remaining 52% indicated that they did not use assessment scales with their patients . A number of papers have described the barriers preventing rehabilitation professionals from using routine assessment tools. These include lack of time allocated for the application, lack of knowledge of the available scales, lack of clarity to interpret the results, and even problems inherent to the healthcare provider [6,7,8,9]. The literature notes the need for rehabilitation professionals to assemble a core set of assessment instruments, which can be easily applied, and where the results obtained can be clearly interpreted. To the best of our knowledge, there is no clinical model in Chile that allows the application of assessment scales in physical therapy to be standardized. The literature has shown that continuous training and systematic feedback on the benefits of using assessment scales improves adherence among rehabilitation professionals .
As a consequence of the national and international accreditation processes in the Clínica Alemana’s Rehabilitation Service, a lack of records was detected, which led to the need to implement an innovation strategy to increase the use of standardized and systematic assessments with inpatients. In view of this need, the complexity-based model for categorizing rehabilitation (CMCR) was designed. It was meant to facilitate the use of standardized clinical assessments by physical therapists.
The objective of this research was to describe the adherence of physical therapists to the use of standardized assessment by means of a CMCR at an acute hospital.
This retrospective cohort study used 145,968 participant records that were stored in the inpatient database between July 2011 and December 2015. All of the participants were inpatients at the Clínica Alemana of Santiago. They were prescribed physical therapy by the physician and then categorized by the physical therapist, using standardized assessments via the CMCR.
The CMCR was developed using a matrix of clinical variables. To design the matrix, the rehabilitation teams were grouped by medical specialty (Neurorehabilitation, Adult ICU, Pulmonary Rehabilitation, Pediatric Rehabilitation, Motor Rehabilitation, Neonatology, Orthopedic Surgery, Geriatrics, Oncology and Cardiac Rehabilitation). Subsequently, based on their clinical experience and available evidence, the teams were then asked to select and share standardized clinical assessments to be used with an inpatient.
The matrix has two axes: the vertical axis provides the functional clinical assessment to be applied during the patient’s rehabilitation, while the horizontal axis shows its 3 severity levels. The lowest severity level is column X, where each cell has a score of 1 point. The sum of X (Σx) provides the total number of points with respect to the variables chosen. Table 1 shows 4 clinical variables chosen, so Σx is 4. Then, the next severity level is column Y, where each cell has a score of 2 points: as there are 4 variables, the sum of Y (Σy) is 8. Finally, the highest severity level is column Z, where each cell has a score of 3 points, so the sum of Z (Σz) is 12 (Table 1). The lowest complexity is obtained from the sum of column X, Total Score (TS) = Σx, medium complexity is obtained from the range of Σx + 1 ≤ TS ≤ Σy, and the highest complexity, from the range obtained of Σy + 1 ≤ TS ≤ Σz (Table 1). Each complexity is associated with a therapeutic load (number of sessions) in order to take the patient from the highest to the lowest complexity level within the shortest possible time. In the interests of the aforementioned categorization, the professionals were asked to fill out the instrument within 15–30 min, depending on the scales available in each matrix.
The management of the same complexities will vary according to the clinical variable that determines the issues in the patient; therefore, we can have patients of the same complexity being attended with different protocols. For example, in the patients categorized with the ICU mobility matrix, the medium complexity could be a result of the change the state of consciousness, which is why the protocols to apply will be focused on this area. Otherwise, the medium complexity of the mobility could be due to the change in the functional mobility, for a lack of motor skill, and the protocols that would be applied would be in this area, different from the previous one.
The re-categorization process will depend on where the patient is hospitalized and on the agreed protocols in each unit. For example, if the patient is in the ICU, they are categorized every day, once a day, or when the patient’s condition changes. An example of this is in the cases of patients categorized in the morning and then a spontaneous ventilation test is performed, and when extubated, their ventilatory support condition changes, which again requires that they be re-categorized. In the case of patients undergoing neurorehabilitation, they are categorized once a week (every Wednesday) by applying a battery of evaluation scales that reveal their progress and determine the therapeutic load.
As an example, the matrix for categorizing Motor Rehabilitation among ICU patients is shown in Table 2 [11, 12], and the matrix for categorizing Adult Pulmonary Rehabilitation for patients with Invasive Mechanical Ventilation is shown in Table 3. These matrices were selected from among the various other matrices developed. 96,400 instances of inpatient care where the physician recommended physical therapy were categorized. This was from a total of 145,968 instances of inpatient care that were recorded throughout the duration of the study (17 quarters). These records were stored in a database in order to be analyzed on a quarterly basis. The use of this data was approved by the Scientific Ethics Committee of the Faculty of Medicine, Clinica Alemana of Santiago-Universidad del Desarrollo, who authorized the exemption requiring participants’ consent as the data was anonymized.
The CMCR was rolled out via the hospital’s electronic patient records (EPR), in March 2011 through a coordinated effort in order to standardize the use of the model in clinical practice and optimize the data collection process. The results were then stored in the inpatient database and analyzed quarterly. Adherence to the CMCR was also used as a quality indicator as part of the hospital’s accreditation process (Fig. 1).
Meetings were held every semester with each of the corresponding teams to show their adherence level and design strategies for improvement. Strategies refer to re-evaluating if the chosen clinical variables were the correct ones for the clinical situation of the patients. The way in which the data were recorded was also evaluated and variations were made according to feedback from the teams, and finally in the quarterly team meetings, the need for retraining in the application of a specific scale that made use of the full categorization model was determined. Additionally, individual interviews were conducted to analyze individual-level variables that may have influenced adherence by healthcare professionals.
Throughout this time, the different matrices were put to the test by the rehabilitation teams and any necessary adjustments were made. The percentage use of the CMCR with ICU and non-ICU inpatients was recorded quarterly. The trend in these percentages between July 2011 and December 2015 was then analyzed.
Trends in adherence were analyzed using a Prais-Winsten regression (a first-order autoregressive model) [13, 14]. The trends were compared using a repeated measures ANOVA for mixed models [15, 16], with a significance level of 0.05. The data was processed using STATA 13.0.
First, Fig. 1 shows the overall level of adherence per quarter (between July 2011 and December 2015). Adherence increased each quarter by an average of 1.48% (p = 0.005). Autocorrelation was around 30%, i.e., adherence in a given quarter had a 30% impact on the following quarter.
Second, Fig. 1 shows the results for the units with the highest and lowest levels of adherence to the model (ICU and non-ICU, respectively). Initial adherence with ICU patients was 63%, and increased by an average of 1.28% per quarter (p = 0.003). Meanwhile, initial adherence with non-ICU inpatients was 54%, with a quarterly increase of 1.62% (p = 0.007).
The results of this study show that physical therapists adhere to the use of standardized clinical assessments by means of a CMCR. Therapists working in the ICU showed earlier adherence than those not working in the ICU; however, the increase in non-ICU units was greater over time.
When interpreting the results, we noted that adherence sustained over time may be due to physical therapists being able to comparatively assess the advantage of having a model that standardized, arranged and promoted objective decision-making through the use of systematic and agreed assessment scales. The fact that the categorizations contain specific clinical variables carefully chosen by the teams is meant to reduce variability in clinical practice and ensure that the professionals observe the same, most important determinants affecting patients’ clinical behavior without their subjective criteria having a negative influence. In addition, we incorporated a cross-sectional nomenclature (high, medium and low complexity), which made the clinical evolution of a patient from high to low complexity easier to observe, and gave a therapeutic load (number of sessions) associated with the patient’s functional needs, showing intervention-related clinical improvement. Another implication of the CMCR is related to the allocation of human resources; when the demand for services increases, we prioritize care for the patients categorized as high and medium complexity over the patients of low complexity, since the clinical variables incorporated into this complexity are less severe, which makes it possible for us to optimize the human resources and redistribute them without putting the patients at risk. Finally, its simple design made its daily use easier.
The characteristics mentioned for the CMCR overcame the barriers described by the literature for the use of assessment scales by physical therapists in clinical practice .
The data analysis was performed using a database that contains all the care services provided by physical therapists over 17 quarters. Adherence was assessed quarterly, and the percentage of the instances of inpatient care categorized from the total number of instances of inpatient care performed per quarter was obtained. There is no individual patient follow-up; the result shows the overall adherence of all the care provided over a period of time.
Sustained adherence to the study during the 17-month follow-up was also relevant. Most studies about sustainability published in the literature show its decrease after a year of any kind of implementation . We believe the successful sustainability shown in this study is due to two important strategies: first, auditing and feedback to professionals, and second, linking the CMCR as a quality indicator in the national and international accreditation processes in which our institution participates. This was meant to align the implementation strategies with the hospital’s organizational goals, a concept referred to by Scheck McAlearney as “organizational coherence” . This is clearly visible when analyzing the peaks in adherence over the 5-year period of the study as they coincide with the accreditation processes conducted within the organization (Fig. 2).
When analyzing adherence with ICU and non-ICU inpatients separately, we noted a higher level of initial adherence with ICU patients. This behavior is in line with other studies that have suggested that patients managed in closed intensive care units by well-trained staff enjoy more positive results that non-ICU inpatients . Therefore, the ICU would appear to be a favorable environment for implementing new clinical tools.
The implication here is that there is likely a greater concentration of innovators in intensive care units who bring new ideas to the system and who, in turn, become leaders in disseminating innovation [21,22,23,24].
The goal in recent years has been to incorporate the most up-to-date clinical care, integral, interdisciplinary and holistic vision required for this kind of intervention effectively. Locally in Chile, the challenge is much greater: not only do we have to install the rehabilitation model in regular clinical care, but we also have to set it up from the start of the acute health care intervention (ICU, intermediate), and we must generate models capable of standardizing evaluations, interventions, protocols, outcomes, goals and follow-up. Despite the difficulties this can imply, the movement towards innovation is more likely to become a reality if the vision is shared and consistent with the institutional culture and organization policy. From this point of view, the CMCR can be a concrete alternative for achieving this goal.
The main limitation of this study is the lack of evidence regarding the implementation of clinical models involving standardized assessments in rehabilitation. Another limitation of the work is that other rehabilitation groups were not analyzed because the greatest percentage of professionals (90%) are physical therapists, and it was decided that the implementation should begin with the most representative rehabilitation group. Thus, no study was made of the individual variables that influence the CMCR in this group of professionals, but rather their use through the clinical records. Our article focuses mainly on the behavior of the physical therapists with respect to the use of the CMCR as a clinical tool, since for successful implementation and to see its impact on patients, we needed to ensure that the greatest percentage of professionals was using it. We hope in future investigations to be able to show the collected data of the impact on the patients.
The CMCR facilitated adherence to standardized assessments by physical therapists in an acute hospital. Furthermore, the strategies adopted during the implementation of the model and for its long-term sustainability were successful throughout the duration of this study (17 quarters). Adherence continuously increased over time with both ICU and non-ICU inpatients, with the increase for ICU patients being slightly greater.
The results suggest that intensive care units provide a favorable environment for the use of standardized assessment scales. Linking an innovation to the organization’s internal processes, such as Quality Accreditation, should also be considered as a fundamental element when designing such a tool.
American Physical Therapy Association
Complexity-based Model for Categorizing Rehabilitation
Electronic patient records
Intensive care united
Bland MD, Sturmoski A, Whitson M, Harris H, Connor LT, Fucetola R, Edmiaston J, Huskey T, Carter A, Kramper M, Corbetta M, Lang CE. Clinician Adherence to a Standardized Assessment Battery Across Settings and Disciplines in a Poststroke Rehabilitation Population. Arch Phys Med Rehabil. 2013;94(6):1048–53.e1.
Walsh TS, Salisbury LG, Boyd J, Ramsay P, Merriweather J, Huby G, Forbes J, Rattray JZ, Griffith DM, Mackenzie SJ, Hull A, Lewis S, Murray GD. A randomised controlled trial evaluating a rehabilitation complex intervention for patients following intensive care discharge: the RECOVER study. BMJ Open. 2012;2(4):e001475.
Panella M, Marchisio S, Barbieri A, Di Stanislao F. A cluster randomized trial to assess the impact of clinical pathways for patients with stroke: rationale and design of the clinical pathways for effective and appropriate care study [NCT00673491]. BMC Health Serv Res. 2008;8(1):223.
Grube MM, Dohle C, Djouchadar D, Rech P, Bienek K, Dietz-Fricke U, et al. Evidence-based quality indicators for stroke rehabilitation. Stroke. 2012;43(1):142–6.
Jette DU, Halbert J, Iverson C, Miceli E, Shah P. Use of standardized outcome measures in physical therapist practice: perceptions and applications. Phys Ther. 2009;89(2):125–35.
Parry SM, Granger CL, Berney S, Jones J, Beach L, El-Ansary D, Koopman R, Denehy L. Assessment of impairment and activity limitations in the critically ill: a systematic review of measurement instruments and their clinimetric properties. Intensive Care Med. 2015;41(5):744–62.
Duncan EA, Murray J. The barriers and facilitators to routine outcome measurement by allied health professionals in practice: a systematic review. BMC Health Serv Res. 2012;12(1):96.
Swinkels RA, van Peppen RP, Wittink H, Custers JW, Beurskens AJ. Current use and barriers and facilitators for implementation of standardised measures in physical therapy in the Netherlands. BMC Musculoskelet Disord. 2011;12(1):106.
Van Peppen RP, Maissan FJ, Van Genderen FR, Van Dolder R, Van Meeteren NL. Outcome measures in physiotherapy management of patients with stroke: a survey into self-reported use, and barriers to and facilitators for use. Physiother Res Int. 2008;13(4):255–70.
Jamtvedt G, Young JM, Kristoffersen DT, O’Brien MA, Oxman AD. Does telling people what they have been doing change what they do? A systematic review of the effects of audit and feedback. Qual Saf Health Care. 2006;15:433–6.
De Jonghe B, Bastuji-Garin S, Sharshar T, Outin H, Brochard L. Does ICU-acquired paresis lengthen weaning from mechanical ventilation? Intensive Care Med. 2004;30:1117–21.
Ali NA, O'Brien JM Jr, Hoffmann SP, Phillips G, Garland A, Finley JC, Almoosa K, Hejal R, Wolf KM, Lemeshow S, Connors AF Jr, Marsh CB. Midwest critical care consortium. Acquired weakness, handgrip strength, and mortality in critically ill patients. Am J Respir Crit Care Med. 2008;178:261–8.
Judge GG, Griffiths WE, Hill RC, Lütkepohl H, Lee TC. The theory and practice of econometrics. 2nd ed. New York: Wiley; 1985.
White H. A heteroskedasticity-consistent covariance matrix estimator and a direct test for heteroskedasticity. Econometrica. 1980;48:817–38.
Andrews M, Schank T, Upward R. Practical Fixed-effects estimation methods for the three-way errorcomponents model. Stata J. 2006;6:461–81.
Baltagi BH, Song SH, Jung BC. The unbalanced nested error component regression model. J Econ. 2001;101:357–81.
Stevens JG, Beurskens AJ. Implementation of measurement instruments in physical therapist practice: development of a tailored strategy. Phys Ther. 2010;90(6):953–61.
Ament SM, de Groot JJ, Maessen JM, Dirksen CD, van der Weijden T, Kleijnen J. Sustainability of professionals’ adherence to clinical practice guidelines in medical care: a systematic review. BMJ Open. 2015;5(12):e008073.
McAlearney AS, Terris D, Hardacre J, Spurgeon P, Brown C, Baumgart A, Nyström ME. Organizational coherence in health care organizations: conceptual guidance to facilitate quality improvement and organizational change. Q Manage Health Care. 2013;22(2):86–99.
Curtis JR, Cook DJ, Wall RJ, Angus DC, Bion J, Kacmarek R, Kane-Gill SL, Kirchhoff KT, Levy M, Mitchell PH, Moreno R, Pronovost P, Puntillo K. Intensive care unit quality improvement: a “how-to” guide for the interdisciplinary team. Crit Care Med. 2006;34(1):211–8.
Perez M, Terrón M. La teoría de la difusión de la innovación y su aplicación al estudio de la adopción de recursos electrónicos por los investigadores en la universidad de Extremadura. Rev Esp Doc Cient. 2004;27(3):308–329.16.
Leape LL, Rogers G, Hanna D, Griswold P, Federico F, Fenn CA, Bates DW, Kirle L, Clarridge BR. Developing and implementing new safe practices: voluntary adoption through statewide collaboratives. Qual Saf Health Care. 2006;15:289–95.
Scott SD, Plotnikoff RC, Karunamuni N, Bize R, Rodgers W. Factors influencing the adoption of an innovation: an examination of the uptake of the Canadian heart health kit (HHK). Implement Sci. 2008;3(1):41.
Waltz TJ, Powell BJ, Matthieu MM, Damschroder LJ, Chinman MJ, Smith JL, Proctor EK, Kirchner JE. Use of concept mapping to characterize relationships among implementation strategies and assess their feasibility and importance: results from the expert recommendations for implementing change (ERIC) study. Implement Sci. 2015;10:109.
We wish to thank Dr. C. Krebs for his collaboration and unconditional support, essential for developing the model. We are also grateful to the members of our hospital’s Physical Medicine and Rehabilitation Service, without whom this project would not have been possible.
The authors declare that they have not received financial aid of any kind.
Availability of data and materials
All data generated or analyzed during this study are included in this published article and its Additional file 1.
Ethics approval and consent to participate
With respect to the ethical responsibilities established by the work: it followed all the norms the institution establishes for patient care and data confidentiality. The use of this data was approved by the Scientific Ethics Committee of the Faculty of Medicine, Clínica Alemana de Santiago-Universidad del Desarrollo, who authorized the exemption of requiring participants’ consent as the data was anonymized.
Consent for publication
The authors declare they have no conflict of interest with the development of the work, which was always oriented toward improving the care processes for our patients.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Database CMCR. (XLSX 5317 kb)