- Research article
- Open Access
- Open Peer Review
Adapting a Markov Monte Carlo simulation model for forecasting the number of Coronary Artery Revascularisation Procedures in an era of rapidly changing technology and policy
BMC Medical Informatics and Decision Making volume 8, Article number: 27 (2008)
Treatments for coronary heart disease (CHD) have evolved rapidly over the last 15 years with considerable change in the number and effectiveness of both medical and surgical treatments. This period has seen the rapid development and uptake of statin drugs and coronary artery revascularization procedures (CARPs) that include Coronary Artery Bypass Graft procedures (CABGs) and Percutaneous Coronary Interventions (PCIs). It is difficult in an era of such rapid change to accurately forecast requirements for treatment services such as CARPs. In a previous paper we have described and outlined the use of a Markov Monte Carlo simulation model for analyzing and predicting the requirements for CARPs for the population of Western Australia (Mannan et al, 2007). In this paper, we expand on the use of this model for forecasting CARPs in Western Australia with a focus on the lack of adequate performance of the (standard) model for forecasting CARPs in a period during the mid 1990s when there were considerable changes to CARP technology and implementation policy and an exploration and demonstration of how the standard model may be adapted to achieve better performance.
Selected key CARP event model probabilities are modified based on information relating to changes in the effectiveness of CARPs from clinical trial evidence and an awareness of trends in policy and practice of CARPs. These modified model probabilities and the ones obtained by standard methods are used as inputs in our Markov simulation model.
The projected numbers of CARPs in the population of Western Australia over 1995–99 only improve marginally when modifications to model probabilities are made to incorporate an increase in effectiveness of PCI procedures. However, the projected numbers improve substantially when, in addition, further modifications are incorporated that relate to the increased probability of a PCI procedure and the reduced probability of a CABG procedure stemming from changed CARP preference following the introduction of PCI operations involving stents.
There is often knowledge and sometimes quantitative evidence of the expected impacts of changes in surgical practice and procedure effectiveness and these may be used to improve forecasts of future requirements for CARPs in a population.
Coronary heart disease (CHD) still remains the largest single cause of death and a major cause of morbidity in most developed countries . Although rates of myocardial infarction have fallen in most developed countries, hospital admission rates for CHD have continued to rise [2, 3]. The treatment of CHD has meanwhile evolved, first with the widespread uptake of effective short and long-term medical treatment  and then with greater use of coronary artery revascularisation procedures (CARPs) [3, 4] which started with coronary artery bypass grafting (CABG) during the 1970s and evolved with percutaneous coronary interventions (PCI) with the introduction and rapid use of percutaneous transluminal coronary angioplasty in the early 1980s, coronary artery stents in 1995 and more recently drug eluting stents . The use of stents has contributed to a decline in the risk of a repeat CARP following PCI  while drug-eluting stents may lead to further reductions in these risks [6, 7]. Based on several randomized controlled trials, CARPs are more effective than medical treatment in reducing mortality and morbidity from CHD [8, 9].
There is great variation in rates of CARPs within and between populations with comparable rates of CHD [10, 11]. Because of this uncertainty it is important to develop probabilistic methods that model the requirements for CARPs at the population level by capturing movement of individuals from one CHD/CARP state to another based on the epidemiology and natural history of CHD. This can be modelled by using Markov Monte Carlo simulation.
This study is facilitated by the availability of a unique Western Australian health information system which allows identification and linkage of hospital admission and deaths records pertaining to single individuals, thereby providing a complete event history for CHD patients . In a previous paper we have described and outlined the use of a Markov Monte Carlo simulation model for predicting CHD incidence and requirements for CARPs for the Western Australian population . In this paper, we expand on the use of this model for forecasting CARPs in Western Australia with a focus on how the standard model may be adapted to achieve better forecasting performance if anticipated changes are incorporated. The 1995 Western Australian population cohort is used to illustrate this approach with comparison of the predicted and actual numbers of CARPs performed over the period 1995 to 1999.
Statewide datasets comprising (linked) cardiovascular hospital admissions for all persons who had a cardiovascular-related admission for the period 1980 to 2001 together with all deaths and population census information were used to establish CHD/CARP event history for population cohorts and estimate (population) probabilities of certain CHD and CARP events and deaths .
Under the Markov Monte Carlo simulation approach, for the cohort comprising the population of Western Australia aged 35–79 years at the beginning of 1995, model simulations are conducted to predict the number of CARPs (and other CHD events) that occur within the cohort over the next 5 years. Under our Markov model the cohort is split into initial history state groups (using their known CHD/CARP history from the linked health information system data back to 1980). The history states are (1) a history of having a PCI some time in the past; (2) a history of having a CABG some time in the past; (3) a history of having a CHD admission but no CABG or PCI some time in the past; and (4) no history of CHD admission, CABG or PCI in the past. In the context of forecasting, population transition probabilities that have been estimated for past years (eg 1992 to 1994) are extrapolated into future years for which forecasts are required (eg 1995 to 1999). These extrapolated probabilities are used within the model to simulate events among cohort members into the future (and taking into account the ageing of the cohort). At the end of the first cycle people could have moved to a different disease history state or moved into one of two possible death (absorbing) states, namely (5) death from CHD or (6) death from causes other than CHD. Within a cycle, our model allows for a CHD admission without a CARP, up to two admissions with CABG or PCI in addition to possible death (Figure 1). A more detailed description of the model and the estimation of population transition probabilities have been provided elsewhere .
For extrapolating transition probabilities two standard methods were used. These are (for each age, sex and history group) using the mean of the probabilities for the last three years (the mean method) and using linear extrapolation of transition probabilities based on the (weighted) annual changes over the last three years with twice as much weight assigned to the most recent annual change (the linear method). The mean extrapolation method is expected to perform well in situations when the (absolute) level of the probabilities remains fairly constant into the future and the linear extrapolation method is expected to perform well in situations when the probabilities are changing and are expected to continue to change at approximately the same rate. Modified extrapolation methods are obtained by modifying selected key probabilities of CARP events and adopting the standard extrapolation probabilities for the rest of the model probabilities.
The forecast number of events is based on the average of 100 replicate simulations of the entire cohort . The accuracy of the total forecast number of CABGs and PCIs is evaluated using the percent over- or under- compared to the total actual number. The accuracy of forecasts across age and sex groups is evaluated by calculating a goodness-of-fit (GOF) observed-predicted chi-squared statistic. The smaller the GOF statistic the better are the model forecasts. We used SAS version 9.0  for calculating the simulation inputs and TreeAge Data Pro 2005 for Markov simulation modelling .
No ethical approval was required for the project as the data for this project were already provided to the researchers by the Data Linkage Unit of the WA Department of Health.
Table 1 shows the number and history distribution of the WA 1995 population cohort. Overall 93.1% of men and 96.1% of women were in the 'no history of CHD' state but this varied from 99.2% in 35–39 year old men to 76.2% in 75–79 year old men and from 99.6% in 35–39 year women to 84.6% in 75–79 year old women.
Tables 2 and 3 show the actual and forecast number of CABGs and PCIs respectively for the standard models. The results show that the mean method overestimates the total actual number of CABGs by 20.5 percent and the linear method by 15.6 percent. The GOF values are reasonably close for the two methods. The forecast numbers of PCIs from the mean method under-estimates the actual number of PCIs by 15.3 percent and the linear method over-estimates by 17.7 percent. The GOF value is considerably larger for the mean method.
The poor performance of these standard methods for forecasting the requirements of CARPs over 1995–99 may be due to the fact that the effectiveness of PCIs increased after 1994 when stents were first introduced in WA and as a result the probability of requiring another CARP within the same year was reduced. Within the model these are represented by risk of a CABG or another PCI in the same year following a PCI procedure. For both males and females, the trends in these probabilities for the largest history group 'no history of CHD' are shown in Figures 2a, 2b, 3a and 3b. It is clear that the trend over the period 1992 to 1994 did not continue into 1995–1999 and hence the standard extrapolation methods would not be accurate for these probabilities.
Another explanation for the poor performance of the standard methods was that there was an increasing trend in PCI rates during the early 1990s as the number of facilities performing PCIs increased and this continued until about 1995 . This, together with the increasing effectiveness of PCIs meant that many more PCIs but fewer CABGs were being performed. Within the model these are captured by risk of a CABG and risk of a PCI. For both sexes, the trends in these risks for the 'no CHD history' group are shown in Figures 4a, 4b, 5a and 5b. It is clear that the risk of a CABG and that of a PCI gradually declined and increased respectively after 1994 for most age groups 50–54 and above (the trends for younger age groups are not shown as these risks are low), for both sexes.
In order to incorporate modifications to the standard extrapolation methods relating to the increased effectiveness of PCI procedures (ie the decreased risk of requiring a second CARP within the same year) we have examined the clinical trial evidence for the improved effectiveness of PCIs with stents. The results of a meta-analysis of 29 RCTs by Brophy et al.  indicate that risk of a repeat PCI would have declined by 41 percent in Western Australia from 1995 due to the introduction of stents. The observed risks for the Western Australian population showed that the risk of a CABG after a PCI also declined by 41 percent and thus we also reduce this risk accordingly.
To quantify the changed trends for PCI and CABG rates we use evidence from a population based observational study in Western Australia which showed that for males risk of a PCI increased by about 25 percent and risk of a CABG declined by about 15 percent when stents were introduced in WA in 1995 . For females, there were similar shifts but the magnitude was about half that in males.
To accommodate these changed trends we have modified the standard extrapolated probabilities for these events by the amounts indicated by the evidence. Forecasts based on the following modified models have been obtained and the results are shown in Tables 2 and 3.
Model 1: the annual risk of a repeat PCI declines by 41 percent while the other risks are mean extrapolated,
Model 2: the annual risk of a repeat PCI and risk of a CABG given PCI decline by 41 percent while the other risks are mean extrapolated,
Model 3: the annual risk of a repeat PCI declines by 41 percent while the other risks are linearly extrapolated,
Model 4: the annual risk of a repeat PCI and risk of a CABG given PCI decline by 41 percent while the other risks are linearly extrapolated, and
Model 5: the annual risk of a repeat PCI and risk of a CABG given PCI decline by 41 percent and risk of a PCI increases by 25 percent for males or 12.5 percent for females with a corresponding decline in risk of a CABG by 15 percent for males or 7.5 percent for females while the other risks are mean extrapolated.
The results show that the forecast numbers of PCIs under Model 1 become closer to the respective actual numbers compared to standard mean extrapolation although the discrepancies are still generally quite large. Also, this modification does not improve the forecasts for CABGs. The forecasts for numbers of CABGs and PCIs under Models 2, 3 and 4 improve only marginally. There is over-estimation in the actual number of CABGs by 21.9, 18.9, 15.4 and 9.5 percents respectively under Models 1, 2, 3 and 4. There is under-estimation in the actual number of PCIs by 14.1 and 18.4 percents respectively under Models 1 and 2, and over-estimation by 11.1 and 11.3 percents under Models 3 and 4.
As the forecasts under Model 3 which use linear extrapolations seriously overestimated the numbers of PCIs the linear method was not further considered. Model 5, which also incorporates modifications to risk of a PCI and risk of a CABG has considerably better forecasts. There is now only 6.1 percent over-estimation of number of CABGs under Model 5 while for PCIs there is an under-estimation by only 3 percent.
In this paper we have examined forecasts for 1995–1999 based on extrapolation of trends from 1992–94. This was done because actual event numbers were known for these years and to illustrate the need to modify the standard methods in a period like this when there were rapid changes in CARP surgical technology and practice. However, the limitations of the retrospective nature of this study should be acknowledged. We believe that the modifications could have been anticipated in 1995 when knowledge of increased PCI effectiveness was becoming available and the changing preference for PCI over CABG was common knowledge among cardiologists and cardio-thoracic surgeons. However, quantifying the impacts of these expected changes in 1995 would have been more difficult than in our illustration which used evidence published after 1995.
Our CHD/CARP model does not directly incorporate the effects of risk factors such as blood pressure, cholesterol, obesity and smoking on event risks. Whilst cohort-based models that incorporate risk factors have been proposed (eg, Sesso et al. ) it is not practical to incorporate risk factors directly into our model because it has 96 different risks (for each age and sex group) and thus 96 risk factor models would be required. Further, as our model simulates individual pathways, such an approach would also require risk factor data on the entire population of Western Australia. However, our model can still indirectly incorporate changes in trends in risk factors by appropriate modification of model probabilities that relate to risk of certain CHD events (eg for people currently with no history of CHD).
Using the approach of modifying selected and relevant model event probabilities, our model is flexible enough to explore the effect on the future requirements of CARPs of a variety of changing circumstances. In addition to advances in surgical procedures and changing preference for surgical procedures as we have illustrated, other scenarios such as changes in the effectiveness of, and access to, medical treatments may be investigated. For example, randomized controlled trials have shown that there were significant reductions in requirements for CARPs among higher risk individuals who were given antiplatelet therapy versus controls . An example of a possible change in health policy related to treating CHD patients could be the availability of low cost statins . Since the cost of the Australian Pharmaceutical Benefit Scheme which subsidizes the cost of statin drugs has been increasing rapidly, the government may change its current policy and only provide subsidy to people at higher risk of CHD. The effect of these changes relating to medical treatments could be investigated using our CHD/CARP model through modifications to selected and relevant model probabilities.
The CHD/CARP model can also be extended and used to determine the cost of future requirements for CARPs and to explore the most cost-effective strategies for treatment of CHD. To achieve this, indicative cost estimates are needed for each of the alternative CARP treatments for each year. For more description of cost effectiveness analysis one can refer to some studies which have evaluated economic impact of drug-eluting stents [20, 21], and cost effectiveness of CHD prevention through CHD risk reduction strategies .
It is clear from the model simulation results that forecasting events in an era of rapid changes is fraught with difficulties and standard extrapolation methods are not likely to perform well in such situations. However, there is often knowledge and sometimes quantitative evidence of the expected impacts of changes in procedure effectiveness and practice and these may be used to modify the standard forecasting methods to achieve better forecasts as we have illustrated. The simulation results implied that modifications that related to the changing relative use of a PCI to a CABG were more important for correctly projecting the numbers of CARPs than modifications that related only to the increased effectiveness of PCI procedures.
World Health Organization: World health report: reducing risks, promoting healthy life. 2002, Geneva: WHO
Kovar MG, Collins JH, Dolozier J, Havlik R, Pokras R: Trends in the availability and Use of Medical Care for Coronary Heart Disease and Related Diseases. Trends in Coronary Heart Disease Mortality. The Influence of Medical Care. Edited by: Millicent W, Higgins RVL. 1988, Oxford, Oxford University Press
Gillam R, Gillum B, Francis C: Coronary revascularization and coronary angiography in the United States: trends in soradial differences. Journal of the American College of Cardiology. 1997, 29: 1557-62. 10.1016/S0735-1097(97)00089-2.
Martin CA, Hobbs MS, Armstrong BK, De Klerk NH: Trends in the incidence of myocardial infarction in Western Australia between 1971 and 1982. American Journal of Epidemiology. 1989, 129: 655-68.
Hobbs MST, McCaul KA, Knuiman MW, Rankin JM, Gilfillan I: Trends in coronary artery revascularization procedures in Western Australia, 1980–2001. Heart. 2004, 90 (9): 1036-1041. 10.1136/hrt.2003.022160.
Morice MC, Serruys PW, Sousa JE, Fajadet J, Ban Hayashi E, Perin M, Colombo A, Schuler G, Barragan P, Guagliumi G, Molnar F, Falotico R: Randomized Study with the Sirolimus-Coated Bx Velocity Balloon-Expandable Stent in the Treatment of Patients with de Novo Native Coronary Artery Lesions. New England Journal of Medicine. 2002, 346 (23): 1773-80. 10.1056/NEJMoa012843.
Grube E, Silber S, Hauptmann KE, Mueller R, Buellesfeld L, Gerckens U, Russell ME: TAXUS I: six- and twelve-month results from a randomized, double-blind trial on a slow-release paclitaxel-eluting stent for de novo coronary lesions. Circulation. 2003, 107 (1): 38-42. 10.1161/01.CIR.0000047700.58683.A1.
Yusuf S, Zucker D, Peduzzi P, Fisher LD, Takaro T, Kennedy JW: Effect of coronary artery bypass graft surgery on survival: overview of 10-year results from randomised trials by the Coronary Artery Bypass Graft Surgery Trialists Collaboration [see comments] [published erratum appears in Lancet 1994 Nov 19, 344(8934):1446]. Lancet. 1994, 344 (8922): 563-70. 10.1016/S0140-6736(94)91963-1.
RITA-2: Coronary angioplasty versus medical therapy for angina: the second Randomised Intervention Treatment of Angina (RITA-2) trial. RITA-2 trial participants. Lancet. 1997, 350 (9076): 461-8. 10.1016/S0140-6736(97)07298-X. [see comments]
Anderson GM, Grumbach K, Luft HS, Roos LL, Mustard C, Brook R: Use of coronary artery bypass surgery in the United States and Canada. Influence of age and income. JAMA. 1993, 269 (13): 1661-6. 10.1001/jama.269.13.1661.
Higginson LA, Naylor CD: Rates of cardiac catheterization, coronary angioplasty and coronary artery bypass surgery in Canada. Canadian Journal of Cardiology. 1997, 13 (Suppl D): 47D-52D.
Holman CD, Bass AJ, Rouse IL, Hobbs MS: Population-based linkage of health records in Western Australia: development of a health services research linked database, Australia and New Zealand. Journal of Public Health. 1999, 23: 453-9.
Mannan HR, Knuiman MW, Hobbs M: A Markov simulation model for analysing and forecasting the number of coronary artery revascularization procedures in Western Australia. Annals of Epidemiology. 2007, 17 (12): 964-975. 10.1016/j.annepidem.2007.05.016.
SAS, version 9.0. SAS institute: Cary, NC
Treeage Pro: Treeage Pro User's Manual. 2005, Treeage software Inc., 1075 Main Street, Williamstown, MA 01267
Brophy JM, Belisle P, Joseph L: Evidence of use of coronary stents: A hierarchical Bayesian meta-analysis. Annals of Internal Medicine. 2003, 138: 777-786.
Sesso HD, Chen RS, L'Italien GJ, Lapuerta P, Lee WC, Glynn RJ: Blood Pressure Lowering and Life Expectancy Based on a Markov Model of Cardiovascular Events. Hypertension. 2003, 42: 885-10.1161/01.HYP.0000097602.67134.4D.
Antiplatelet Trialists' Collaboration: Collaborative overview of randomised trials of antiplatelet therapy–II: Maintenance of vascular graft or arterial patency by antiplatelet therapy. Antiplatelet Trialists' Collaboration [see comments]. British Medical Journal. 1994, 308 (6922): 159-68.
Lipid Management Guidelines – 2001: National Heart Foundation of Australia and Cardiac Society of Australia and New Zealand. Medical Journal of Australia. 2001, 175: S57-85.
Shrive FM, Manns BJ, Galbraith Knudtson ML, Ghali WA: Economic evaluation of sirolimus-eluting stents. Canadian Medical Association Journal. 2005, 172 (3): 345-351. 10.1503/cmaj.1041062.
Kong DF, Eisenstein EL, Sketch MH, Zidar JP, Ryan TJ, Harrington RA, Newman MF, Smith PK, Mark DB, Califf RM: Economic impact of drug-eluting stents on hospital systems: A disease-state model. American Heart Journal. 2204, 147 (3): 449-456. 10.1016/j.ahj.2003.11.005.
Whitfield M, Gillett M, Holmes M, Ogden E: Predicting the impact of population level risk reduction in cardio-vascular disease and stroke on acute hospital admission rates over a 5 year period – a pilot study. Public Health. 2006, 120 (12): 1140-1148. 10.1016/j.puhe.2006.10.012.
The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6947/8/27/prepub
The first author would like to acknowledge International Postgraduate Research Scholarship (IPRS) for providing him with support for this research project which formed his Doctoral dissertation at the University of Western Australia. The data for this project were provided by the Data Linkage Unit of the WA Department of Health. The initial part of this project was funded by a grant from the National Health and Medical Research Council of Australia (Project grant no. 139100).
The authors declare that they have no competing interests.
HRM did all aspects of this research work as part of his doctoral dissertation including model design, development, implementation, and analysis and also wrote the draft of this paper. MK supervised the research project giving valuable inputs in the model design and concept and finally helped draft the paper. MH was involved as a co-supervisor.
About this article
Cite this article
Mannan, H.R., Knuiman, M. & Hobbs, M. Adapting a Markov Monte Carlo simulation model for forecasting the number of Coronary Artery Revascularisation Procedures in an era of rapidly changing technology and policy. BMC Med Inform Decis Mak 8, 27 (2008). https://doi.org/10.1186/1472-6947-8-27
- Coronary Heart Disease
- Percutaneous Coronary Intervention
- Coronary Artery Bypass Grafting
- Percutaneous Coronary Intervention
- Coronary Heart Disease Event