This article has Open Peer Review reports available.
Logistic regression model can reduce unnecessary artificial liver support in hepatitis B virus-associated acute-on-chronic liver failure: decision curve analysis
- Gang Qin†1, 2Email author,
- Zhao-Lian Bian†1,
- Yi Shen2,
- Lei Zhang3,
- Xiao-Hong Zhu1,
- Yan-Mei Liu2 and
- Jian-Guo Shao1Email author
© The Author(s). 2016
Received: 14 November 2015
Accepted: 29 May 2016
Published: 4 June 2016
Several models have been proposed to predict the short-term outcome of acute-on-chronic liver failure (ACLF) after treatment. We aimed to determine whether better decisions for artificial liver support system (ALSS) treatment could be made with a model than without, through decision curve analysis (DCA).
The medical profiles of a cohort of 232 patients with hepatitis B virus (HBV)-associated ACLF were retrospectively analyzed to explore the role of plasma prothrombin activity (PTA), model for end-stage liver disease (MELD) and logistic regression model (LRM) in identifying patients who could benefit from ALSS. The accuracy and reliability of PTA, MELD and LRM were evaluated with previously reported cutoffs. DCA was performed to evaluate the clinical role of these models in predicting the treatment outcome.
With the cut-off value of 0.2, LRM had sensitivity of 92.6 %, specificity of 42.3 % and an area under the receiving operating characteristic curve (AUC) of 0.68, which showed superior discrimination over PTA and MELD. DCA revealed that the LRM-guided ALSS treatment was superior over other strategies including “treating all” and MELD-guided therapy, for the midrange threshold probabilities of 16 to 64 %.
The use of LRM-guided ALSS treatment could increase both the accuracy and efficiency of this procedure, allowing the avoidance of unnecessary ALSS.
KeywordsModel for end-stage liver disease Logistic regression model Acute-on-chronic liver failure Hepatitis B virus Decision curve analysis
Acute-on-chronic liver failure (ACLF) is characterized by severe jaundice, coagulopathy, hepatic encephalopathy (HE), and high morbidity and mortality. Hepatitis B virus (HBV) infection is the commonest cause of ACLF in the Asian region . Currently, the treatment of HBV-associated ACLF (HBV-ACLF) is based on two main regimes – antiviral treatment and artificial liver support system (ALSS) in China . Antiviral therapy with nucleos(t)ide analogues has been proven to significantly improve prognosis and survival rate in HBV-ACLF patients . However, the use of the ALSS has shown a variable range of safety and efficiency in several clinical trials [4–8]. In the last decade, several models have been proposed to predict the short-term outcome of HBV-ACLF after treatment. For instance, antiviral treatment can significantly decrease the 3-month mortality only in patients with the model for end-stage liver disease (MELD) score below 30, compared with those with higher MELD scores . Another study suggested that ACLF patients with lower MELD scores showed significantly improved prognosis compared with those with higher MELD scores . Therefore, the variable results of the ALSS treatment in terms of cost-effectiveness might be explained by the lack of guidance of such predictive models. An ideal prediction model may provide objective information about whether future treatment is likely to result in a favorable outcome or survival.
Recently, some studies revealed higher diagnostic accuracy for predictive models which combined liver dysfunction with etiological factor (e.g. HBV). In particular, Zheng et al., in a population of 452 patients with diagnosis of HBV-ACLF, established the logistic regression model (LRM) score, with an area under the receiving operating characteristic curve (AUC) of 0.844. With the cutoff of 0.2, a sensitivity of 86.7 and specificity of 75.5 % were reported . LRM has shown promising results for prognosis prediction in HBV-ACLF ever since its introduction into clinical application. Yang et al. compared the predictive performance of MELD with that of LRM in a population of 273 HBV-ACLF patients. In ACLF patients with liver cirrhosis (LC), the AUC of LRM (0.851) was comparable with that of MELD (0.840). Yet, in patients with non-cirrhotic ACLF, the AUC of LRM (0.897) was significantly higher than that of MELD (0.758) .
In a previous study, we reported that ALSS could improve short- and long-term prognosis in patients with HBV-ACLF . Here we reanalyzed the data in order to determine whether better decisions for ALSS treatment could be made with a model (e.g. MELD or LRM) than without, through decision curve analysis.
From January 2003 to December 2007, all patients admitted to our hospital with the diagnosis of HBV-ACLF according to the Chinese guidelines [14, 15] were screened. Eligible patients were enrolled with the following criteria: (i) aged between 18 and 70 years; (ii) presumptively diagnosed as hepatitis B surface antigen (HBsAg) carrier, chronic hepatitis B (CHB) or HBV-related liver cirrhosis (HBC); (iii) progressive hyperbilirubinemia, with serum total bilirubin (TBil) ≥10 mg/dL; (iv) coagulopathy with plasma prothrombin activity (PTA) ≤40 % or international normalized ratio (INR) >1.5; (v) within 4 weeks from symptom onset complicated by ascites and/or HE. ACLF is further classified into early stage (30 % < PTA ≤ 40 %) and middle-to-end stage (PTA ≤ 30 %) . Some patients were excluded according to the following criteria: acute HBV infection, super-infection with other viruses, chronic liver failure, coexistence of hepatocellular carcinoma (HCC), severe gastrointestinal bleeding, pregnancy, or liver transplant recipients.
The primary endpoint was 3-month survival. The secondary endpoint was survival at 5 years after diagnosis of ACLF.
The medical profiles of the patients were retrospectively analyzed in the present study. Our study protocol was approved by the institutional review board (IRB) of Nantong Third People’s Hospital, Nantong University and conducted in accord with the ethical guidelines of the Declaration of Helsinki 1975. Patient consents were waived by the same IRB.
MELD score (range from 6 to 40) was calculated according to the standard formula . MELD = 11.2 × ln(INR) + 9.6 × ln[creatinine (mg/dL)] + 3.8 × ln[TBil (mg/dL)] + 6.4 (constant for liver disease etiology). Laboratory values of bilirubin, INR or creatinine less than 1 were rounded off to 1, in order to avoid negative scores. Creatinine greater than 4 mg/dL or with renal replacement therapy was capped at 4 mg/dL. In addition, the factor for etiology of liver disease was not used.
LRM score was calculated in accordance with the original reference . LRM =−1.343 + 0.772 × HE + 2.279 × HRS + 0.85 × LC + 1.026 × HBeAg − 2.117 × PTA/age. For HE, hepatorenal syndrome (HRS), LC and hepatitis B e antigen (HBeAg), yes/positive = 1 while no/negative = 0. The score was rounded to the nearest tenth.
The patients with HBV-ACLF were assigned randomly to groups either given ALSS combined with standard medical therapy (SMT) or only SMT. The randomization was conducted by the Department of Epidemiology and Medical Statistics, Nantong University based on the SAS module, as we described in a previous study .
The accuracy of PTA, MELD or LRM was evaluated separately. For each model, the AUC, sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), as well as diagnostic odds ratio (DOR), with the 95 % confidence intervals (CIs) were evaluated.
All statistical analysis, including the DCA and plots implementation, were done using Stata version 13 (StataCorp, TX, USA). DCA analysis was performed using the code found at https://www.mskcc.org/departments/epidemiologybiostatistics/health-outcomes/decision-curve-analysis-01 according to its tutorials.
A total of 283 patients diagnosed with HBV-ACLF were screened for inclusion in this study cohort. Among these, 51 patients were excluded from this study: five for age over 70 years, eight for super-infection with hepatitis E virus, two for coexistence of HCC, seven for liver transplantation, 29 for incomplete data. Thus the final cohort compromised 232 patients with complete medical records.
Demographic, clinical and laboratory features of the study patients
Number of patients
178 (76.7 %)/54 (23.3 %)
46.1 ± 10.5 (45; 21–69)
142 (61.2 %)
HBV DNA (lg copies/mL)
4.1 ± 2.5
22.2 ± 9.2
0.93 ± 0.74
27.1 ± 16.8
4.2 ± 2.2
32.1 ± 5.0
112 (48.3 %)
194 (83.6 %)
152 (65.5 %)
64 (27.6 %)
37 (16.0 %)
29.0 ± 5.4
−0.6 ± 1.4
104 (44.8 %)
Predictive performance of the models
Performance the models to predict 3-month outcome with the recommended cutoffs
Decision curve analysis
Relationship between True ALSS Treatment and Result of a LRM-guided ALSS Treatment
N = 232
LRM < 0.2
Net Benefit for ALSS for All ACLF Patients or According to LRM, Using a Threshold of Pt
Advantage of LRM-guided ALSS
ALSS for All
Reduction in Avoidable ALSS per 100 Patients
Artificial liver support system was first applied to treat acute liver failure in 1970s with the attempt to replace certain detoxification function of the liver. On the one hand, the therapy cost is as expensive as nearly $US2500 for each session of ALSS in China. The incidence of adverse events (i.e. bleeding, hypotension, infection, coagulopathy, and catheter-related events) were reported [5, 6, 8, 13, 20]. On the other hand, several clinical trials and systemic reviews suggested that ALSS could reduce mortality in ACLF patients compared with standard medical therapy [21–24]. Therefore, ALSS has been recommended as one important method for the treatment of ACLF [1, 15, 25, 26].
It has been extensively debated whether to treat all ACLF patients or to treat selected patients. Some efforts have been made toward the identification of factors or models for predicting the prognosis after ALSS. For instance, many factors, including HE, PTA, bilirubin, creatinine, sodium, preexisting cirrhosis and age, were found as independent predictors for the short-term survival rate in ACLF . Several models such as MELD and LRM, with recommended cutoff values, have been proposed to predict the survival outcomes of ACLF patients [11, 27]. Since their introduction into clinical practice, the MELD and LRM scores have been tested in quite a few studies. Yet wide range of sensitivity and specificity has been reported in predicting mortality of ACLF patients [9, 10, 12, 28]. Despite the well-known utility of MELD in allocating donor livers , the clinical utility of MELD and other models for other treatments remains unclear.
Although discrimination and calibration are essential aspects of a prediction model, they do not evaluate clinical usefulness such as the ability to make better decisions with a model than without. Decision curve analysis is a method for assessing the benefits of a model through a range of patient preferences in accepting risk of overtreatment and undertreatment to facilitate decision making [29, 30]. The hypothesis in our study was that we may make better decisions for ALSS treatment with a model (e.g. MELD or LRM) than without. For a prediction model aiming to guide therapeutic decisions, a cut-off is required for the decision threshold. Right at the threshold, the likelihood of benefit, e.g. improved survival as a result of ALSS treatment, exactly balances the likelihood of harm, e.g. adverse events and expensive costs. However, as empirical evidence for the relative weight of benefits and harms is often lacking, it is always not easy to define a threshold.
In this study, we applied the DCA to evaluate the cost/benefit ratio of one single marker (i.e. PTA) and two models (i.e. MELD and LRM). The utility of PTA alone resulted in no more net benefit gain than random ALSS assignment. Using the MELD or LRM scores, some number of unnecessary ALSS treatment could be avoided at the cost that only a small proportion of patients with HBV-ACLF being advised not to undergo ALSS treatment. Starting from the cutoff of 16 %, the net benefit gain of LRM-guided strategy starts to be remarkable. The DCA results showed that for patients with threshold probabilities between 0 and 16 %, relatively preferring for empirical therapy, the net benefit is the greatest if all patients are treated. Across this range of threshold probabilities, patients tend to be more concerned about missing a timely treatment than about receiving unnecessary one. For the midrange threshold probabilities between 16 and 64 %, the LRM-guided ALSS therapy is superior to other strategies, including the MELD score. For higher thresholds (>64 %) at which patients appear be more concerned about unnecessary treatment than missed one, the option to not treat is preferred and none of the predictive models has value.
Admittedly, there are some limitations in our study. First, we applied the decision curve analysis theory retrospectively with our cohort. Second, our findings were based on a small sample size. Finally, the present methodology may be appropriate for point decision making but not necessarily for decisions which reoccur over time, because the scores may frequently change in the natural history of ACLF.
Our findings indicate that the use of LRM-guided ALSS treatment could increase both the accuracy and efficiency of this procedure. Promising results from studies on the novel LRM score for ACLF prognosis may lead to better accuracy when predicting post-treatment outcomes in the near future, allowing the avoidance of unnecessary ALSS.
ACLF, acute-on-chronic liver failure; ALSS, artificial liver support system; AUC, area under the receiving operating characteristic curve; CHB, chronic hepatitis B; CI, confidence interval; DCA, decision curve analysis; DOR, diagnostic odds ratio; HBC, HBV-related cirrhosis; HBeAg, hepatitis B e antigen; HBsAg, hepatitis B surface antigen; HBV, hepatitis B virus; HCC, hepatocellular carcinoma; HE, hepatic encephalopathy; HRS, hepatorenal syndrome; INR, international normalized ratio; IRB, institutional review board; LC, liver cirrhosis; LRM, logistic regression model; MELD, model for end-stage liver disease; NPV, negative predictive value; PPV, positive predictive value; Pt, threshold probability; PTA, prothrombin activity; SBP, spontaneous bacterial peritonitis; SMT, standard medical therapy; TBil, total bilirubin
We would like to thank Prof. Lu-Jun Wang from Nantong Third People’s Hospital, Nantong University for patient health record, Mr Xu-Lin Wang and Mr Sheng Zhang from School of Public Health, Nantong University for data acquisition.
This study was supported in part by grant number 81370520 from National Natural Science Foundation of China (NSFC), by the Grant for Clinical Research number BE2015655 from the Department of Science and Technology, Jiangsu Province, China, by grant number BK2012653 from the Natural Science Foundation of Jiangsu Province, China, by the Young Investigator Grant number Q201208 from the Department of Health, Jiangsu Province, China, and by grant number MS12015004 from the Nantong Science and Technology Bureau, Jiangsu Province, China. The funding agreement ensured the authors’ independence in designing the study, interpreting the data, writing, and publishing the report.
Availability of data and materials
Our dataset consists of patient data provided by Nantong Third People’s Hospital, Nantong University can only be shared with its written consent.
GQ conceived the research idea and study design and prepared the manuscript. Z-LB carried out the field study and did data analysis. YS did data analysis. LZ provided valuable discussion and support. X-HZ carried out the field study and interpreted the results. Y-ML carried out the field study and performed the statistical analysis. J-GS participated in the study design and coordination and helped to refine the manuscript. All authors read and approved the final manuscript.
The authors declare that they have no competing interests.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
- Sarin SK, Kedarisetty CK, Abbas Z, Amarapurkar D, Bihari C, Chan AC. Acute-on-chronic liver failure: consensus recommendations of the Asian Pacific Association for the study of the liver (APASL). Hepatol Int. 2014;8:453–71.View ArticlePubMedGoogle Scholar
- Li LJ, Zhang YM, Liu XL, Du WB, Huang JR, Yang Q, Xu XW, Chen YM. Artificial liver support system in China: a review over the last 30 years. Ther Apher Dial. 2006;10(2):160–7.View ArticlePubMedGoogle Scholar
- Yu S, Jianqin H, Wei W, Jianrong H, Yida Y, Jifang S, Liang Y, Zhi C, Hongyu J. The efficacy and safety of nucleos(t)ide analogues in the treatment of HBV-related acute-on-chronic liver failure: a meta-analysis. Ann Hepatol. 2013;12(3):364–72.PubMedGoogle Scholar
- Mitzner SR, Stange J, Klammt S, Risler T, Erley CM, Bader BD, Berger ED, Lauchart W, Peszynski P, Freytag J, et al. Improvement of hepatorenal syndrome with extracorporeal albumin dialysis MARS: results of a prospective, randomized, controlled clinical trial. Liver Transpl. 2000;6(3):277–86.View ArticlePubMedGoogle Scholar
- Heemann U, Treichel U, Loock J, Philipp T, Gerken G, Malago M, Klammt S, Loehr M, Liebe S, Mitzner S, et al. Albumin dialysis in cirrhosis with superimposed acute liver injury: a prospective, controlled study. Hepatology. 2002;36(4 Pt 1):949–58.View ArticlePubMedGoogle Scholar
- Hassanein TI, Tofteng F, Brown Jr RS, McGuire B, Lynch P, Mehta R, Larsen FS, Gornbein J, Stange J, Blei AT. Randomized controlled study of extracorporeal albumin dialysis for hepatic encephalopathy in advanced cirrhosis. Hepatology. 2007;46(6):1853–62.View ArticlePubMedGoogle Scholar
- Kribben A, Gerken G, Haag S, Herget-Rosenthal S, Treichel U, Betz C, Sarrazin C, Hoste E, Van Vlierberghe H, Escorsell A, et al. Effects of fractionated plasma separation and adsorption on survival in patients with acute-on-chronic liver failure. Gastroenterology. 2012;142(4):782–789.e783.View ArticlePubMedGoogle Scholar
- Banares R, Nevens F, Larsen FS, Jalan R, Albillos A, Dollinger M, Saliba F, Sauerbruch T, Klammt S, Ockenga J, et al. Extracorporeal albumin dialysis with the molecular adsorbent recirculating system in acute-on-chronic liver failure: the RELIEF trial. Hepatology. 2013;57(3):1153–62.View ArticlePubMedGoogle Scholar
- Sun LJ, Yu JW, Zhao YH, Kang P, Li SC. Influential factors of prognosis in lamivudine treatment for patients with acute-on-chronic hepatitis B liver failure. J Gastroenterol Hepatol. 2010;25(3):583–90.View ArticlePubMedGoogle Scholar
- Xia Q, Dai X, Huang J, Xu X, Yang Q, Liu X, Chen Y, Li L. A single-center experience of non-bioartificial liver support systems among Chinese patients with liver failure. Int J Artif Organs. 2014;37(6):442–54.View ArticlePubMedGoogle Scholar
- Zheng MH, Shi KQ, Fan YC, Li H, Ye C, Chen QQ, Chen YP. A model to determine 3-month mortality risk in patients with acute-on-chronic hepatitis B liver failure. Clin Gastroenterol Hepatol. 2011;9(4):351–6. e353.View ArticlePubMedGoogle Scholar
- Yang WB, Chen EQ, Bi HX, Bai L, Chen XB, Feng P, Tang H. Different models in predicting the short-term prognosis of patients with hepatitis B virus-related acute-on-chronic liver failure. Ann Hepatol. 2012;11(3):311–9.PubMedGoogle Scholar
- Qin G, Shao JG, Wang B, Shen Y, Zheng J, Liu XJ, Zhang YY, Liu YM, Qin Y, Wang LJ. Artificial liver support system improves short- and long-term outcomes of patients with HBV-associated acute-on-chronic liver failure: a single-center experience. Medicine. 2014;93(28):e338.View ArticlePubMedPubMed CentralGoogle Scholar
- Si CW, Zhuang H. Prevention and cure project of viral hepatitis. Zhonghua Gan Zang Bing Za Zhi. 2000;8(6):324–9.Google Scholar
- Li LJ. Diagnostic and treatment guidelines for Iiver failure. Zhonghua Gan Zang Bing Za Zhi. 2006;14(9):643–6.Google Scholar
- Kim WR, Biggins SW, Kremers WK, Wiesner RH, Kamath PS, Benson JT, Edwards E, Therneau TM. Hyponatremia and mortality among patients on the liver-transplant waiting list. N Engl J Med. 2008;359(10):1018–26.View ArticlePubMedPubMed CentralGoogle Scholar
- Vickers AJ, Elkin EB. Decision curve analysis: a novel method for evaluating prediction models. Med Decis Mak. 2006;26(6):565–74.View ArticleGoogle Scholar
- Vickers AJ, Kattan MW, Daniel S. Method for evaluating prediction models that apply the results of randomized trials to individual patients. Trials. 2007;8:14.View ArticlePubMedPubMed CentralGoogle Scholar
- Vickers AJ, Cronin AM, Elkin EB, Gonen M. Extensions to decision curve analysis, a novel method for evaluating diagnostic tests, prediction models and molecular markers. BMC Med Inform Decis Mak. 2008;8:53.View ArticlePubMedPubMed CentralGoogle Scholar
- Wigg AJ, McCormick R, Wundke R, Woodman RJ. Efficacy of a chronic disease management model for patients with chronic liver failure. Clin Gastroenterol Hepatol. 2013;11(7):850–8. e851-854.View ArticlePubMedGoogle Scholar
- Liu JP, Gluud LL, Als-Nielsen B, Gluud C. Artificial and bioartificial support systems for liver failure. Cochrane Database Syst Rev. 2004;1:CD003628.PubMedGoogle Scholar
- Shen Y, Wang XL, Wang B, Shao JG, Liu YM, Qin Y, Wang LJ, Qin G. Survival benefits with artificial liver support system for acute-on-chronic liver failure: a time series-based meta-analysis. Medicine. 2016;95(3):e2506.View ArticlePubMedGoogle Scholar
- Kjaergard LL, Liu J, Als-Nielsen B, Gluud C. Artificial and bioartificial support systems for acute and acute-on-chronic liver failure: a systematic review. JAMA. 2003;289(2):217–22.View ArticlePubMedGoogle Scholar
- Stadlbauer V, Krisper P, Aigner R, Haditsch B, Jung A, Lackner C, Stauber RE. Effect of extracorporeal liver support by MARS and prometheus on serum cytokines in acute-on-chronic liver failure. Crit Care. 2006;10(6):R169.View ArticlePubMedPubMed CentralGoogle Scholar
- Li LJ. Diagnostic and treatment guidelines for Iiver failure. Zhonghua Gan Zang Bing Za Zhi. 2012;21(3):177–83.Google Scholar
- Hessel FP, Bramlage P, Wasem J, Mitzner SR. Cost-effectiveness of the artificial liver support system MARS in patients with acute-on-chronic liver failure. Eur J Gastroenterol Hepatol. 2010;22(2):213–20.View ArticlePubMedGoogle Scholar
- Wiesner R, Edwards E, Freeman R, Harper A, Kim R, Kamath P, Kremers W, Lake J, Howard T, Merion RM, et al. Model for end-stage liver disease (MELD) and allocation of donor livers. Gastroenterology. 2003;124(1):91–6.View ArticlePubMedGoogle Scholar
- Shen Y, Liu YM, Wang B, Zhu YG, Wang YY, Wang XL, Ji JL, Shao JG, Qin Y, Qin G. External validation and comparison of six prognostic models in a prospective cohort of HBV-ACLF in China. Ann Hepatol. 2016;15(2):236–45.PubMedGoogle Scholar
- Steyerberg EW, Vergouwe Y. Towards better clinical prediction models: seven steps for development and an ABCD for validation. Eur Heart J. 2014;35(29):1925–31.View ArticlePubMedPubMed CentralGoogle Scholar
- Fitzgerald M, Saville BR, Lewis RJ. Decision curve analysis. JAMA. 2015;313(4):409–10.View ArticlePubMedGoogle Scholar