Substantial differences existed between manual review and automated classification methods, with PPR identifying many more readmissions as potentially preventable. This may have occurred because PPR uses a sole criterion to identify potential preventability: clinical relatedness to the index admission. In contrast, manual review classified as non-preventable many readmissions that were clinically related to the index stay. For example, a 75-year-old man was admitted twice within 30 days for exacerbation of chronic obstructive pulmonary disease. Reviewers found that his follow-up care and transition care plan were appropriate. The patient and his physician felt that the readmission could not have been prevented by Kaiser Permanente, and the reviewers agreed.
To a lesser extent, manual review also identified potentially preventable readmissions that PPR did not identify. For example, a 54-year-old woman was first admitted for partial thickness burns and then readmitted with a digestive system diagnosis. Reviewers found that, had she received appropriate referrals and post-discharge follow up, the readmission may potentially have been prevented. This assessment is consistent with recent research suggesting that, immediately after discharge, patients may be at generalized elevated risk and need additional support to manage ongoing health conditions . PPR did not identify this case as potentially preventable.
Manual review, a subjective process, might have resulted in misclassifications. Manual review processes including more than one reviewer are associated with an increase in the proportion of readmissions identified as preventable . Our manual review process used a nurse reviewer/physician team to assess preventability and identified 47% of readmissions as potentially preventable, nearly double the reported median . It is unlikely that between-methods differences resulted from underestimation of preventability on manual review.
A strength of our report is that we used both methods among the same cases, controlling for variables that have made it difficult to compare methods of measuring preventability in the past, such as patient population and quality of hospital care . Several limitations deserve mention. Reviewers were affiliated with (physicians) or employed by (nurses) Kaiser Permanente, which might have affected their assessment; however, they had not provided care for cases they reviewed. Our assessment took place in an integrated care setting with comprehensive EHR capabilities, and the generalizability of our findings to other settings is unknown. PPR is designed to assess potential preventability over thousands of cases; our analysis may have been too small to assess its accuracy. A different automated classification system may have generated different results, although studies using administrative data alone yield preventability estimates of 55% to 77.1%, much higher than the median for manual review of less than 22% .
Few validation reports of PPR exist to which we can compare our results. PPR identified 6.2% of 30-day readmissions among pediatric patients as potentially preventable and excluded some diagnoses amenable to quality improvement or of uncertain preventability; the authors concluded that caution was warranted when applying the tool to pediatric populations . In preliminary findings from an ongoing study at the U.S. Department of Veterans Affairs (VA), PPR identified just over half of researcher-identified pneumonia readmissions . Another VA study found that PPR and the Centers for Medicare & Medicaid Services (CMS) all-cause readmission measure were moderately correlated; when the variable of potential preventability was removed from the analysis, correlation increased .
Our objective was to determine whether PPR could replace manual review as a method for identifying preventable readmissions to support our ultimate goal of identifying system gaps that contributed to them. The significant discrepancy between results precludes that option; PPR classification agreed with manual review only slightly better than half the time. It would overlook 15% of preventable readmissions and direct most of our organizational attention to readmissions that were not potentially preventable.
The developers of PPR recommend that it be used as a screening tool to identify types of patients and providers with higher than expected readmission rates as a means of focusing subsequent manual review on those patients who have the greatest likelihood of having a preventable readmission. We did not assess the use of PPR across settings and cannot comment on its ability to measure relative performance across facilities. However, in a recent comparison of PPR and the CMS all-cause readmission measure, PPR hospital profiles would have generated different payment penalties for 30% of hospitals .
The true number of potentially preventable readmissions remains unknown, and the choice of method greatly influences the proportion identified as potentially preventable. However, meaningful identification of preventability, which pinpoints missed opportunities leading to avoidable readmissions and forms the basis for quality improvement efforts, depends on the review of primary data [2, 8, 18]. Future research is required to identify and test ways to refine the PPR to increase its concordance with manual review. For example, studies with larger samples may identify subsets of readmissions in which sensitivity and specificity are improved. For instance, one of our additional analyses suggests that PPR sensitivity may vary with the timing of readmissions; further research is required to confirm this finding. Research is also required to establish the effectiveness of using automated classification and manual review in combination to identify potentially preventable readmissions and quality improvement opportunities to address them.