Skip to main content

Online Cost-Effectiveness ANalysis (OCEAN): a user-friendly interface to conduct cost-effectiveness analyses for cervical cancer

Abstract

Background

Most cost-effectiveness analyses in the context of cervical cancer prevention involve the use of mathematical models to simulate HPV infection, cervical disease and prevention strategies. However, it is common for professionals who would need to perform these analyses to not be familiar with the models. This work introduces the Online Cost-Effectiveness ANalysis tool, featuring an easy-to-use web interface providing health professionals, researchers and decision makers involved in cervical cancer prevention programmes with a useful instrument to conduct complex cost-effectiveness analyses, which are becoming an essential tool as an approach for supporting decision-making that involves important trade-offs.

Results

The users can run cost-effectiveness evaluations of cervical cancer prevention strategies without deep knowledge of the underlying mathematical model or any programming language, obtaining the most relevant costs and health outcomes in a user-friendly format. The results provided by the tool are consistent with the existing literature.

Conclusions

Having such a tool will be an asset to the cervical cancer prevention community, providing researchers with an easy-to-use instrument to conduct cost-effectiveness analyses.

Peer Review reports

Background

Health care resources are scarce and therefore, their efficient allocation is a priority for policy-makers. To guide the process of decision making in this context, cost-effectiveness analysis is an essential tool. When the available prevention strategies are multiple and potentially synergic, as occurs in the context of cervical cancer, cost-effectiveness analysis becomes critical to ensure the optimal allocation of resources [1].

Cervical cancer and other cancer and noncancer diseases are caused by or related to human papillomavirus (HPV), a common sexually transmitted infection [2]. In fact, it is estimated that more than 80% of sexually active men and women will acquire an HPV infection by age 45 years [3]. HPV infections are asymptomatic in most cases, although some can lead to the formation of cervical abnormalities called cervical intraepithelial neoplasia (CIN), which can lead to cervical cancer. This cancer is one of the most prevalent cancers among women worldwide, especially in less developed countries [4]. Cervical cancer can be prevented by means of screening to find precancerous lesions (secondary prevention) -so they can be treated- or by HPV vaccination (primary prevention) to prevent infection from some of the most frequent high-risk types. Several countries are implementing a combined prevention strategy including both vaccination and screening, which is recommended on the basis of current evidence, although details may vary. The World Health Organization (WHO) recommends that countries should implement HPV vaccination in preadolescent girls if it is affordable, cost-effective, and sustainable and that achieve the highest possible coverage [5]. However, there are still many challenges faced by cervical cancer prevention programmes [6]. Currently, most cost-effectiveness analyses focused on the evaluation of cervical cancer prevention strategies are conducted by means of mathematical models that simulate the natural history of HPV and cervical cancer. These complex models must integrate data from various sources and should be effectively calibrated to health targets. The tool presented in this work is based on a Markov model that simulates the natural history of HPV infection and subsequent cervical disease. The model computes relevant health and economic outcomes as cases averted, life expectancy (LE) from 11 years, reduction in the lifetime risk of CC, life years saved, quality-adjusted life years (QALYs), net health benefits, and lifetime costs. In this paper we present OCEAN, a new easy to use web application that allows for cost-effectiveness analyses based on a prespecified mathematical model to be run. This model is described in detail in [7]⁠. The tool and a video tutorial illustrating its usage are available at https://iconcologia.shinyapps.io/HECR-OCEAN/. Despite the tool’s ease of use and design, thought to be usable for inexperienced users, the authors are available to provide guidance in case it is needed.

Implementation

The tool is based on a previously described Markov model [7]. The details of this model are also available as supplementary material.

The OCEAN tool

The tool provides an easy-to-use web interface to conduct cost- effectiveness analyses. The sample data available from the OCEAN tool, and in particular the yearly regression and progression transition probabilities between health states were extracted from a literature review [8,9,10,11,12]. The web interface is based on the shiny package [13] for R [14]. Its main screen is divided into two panels, one focused on the calibration process and the second on the cost-effectiveness analyses.

Calibration process

The goal of the calibration part is to provide the user with a reliable transition probability matrix adjusted to the specific setting, to feed the Markov model that will be used in the cost-effectiveness analyses. The calibration panel requires the following inputs:

  • Number of simulations: Number of simulated cohorts.

  • Number of simulations to keep: Number of simulated cohorts that will be kept and used in the cost-effectiveness analyses. Among all simulated cohorts, the best ones (the matrices producing the outcomes that minimize the differences with respect to target HPV infection prevalence and CC incidence). All these simulated cohorts will be used to obtain the cost-effectiveness outputs if the Uncertainty Level (see the inputs for the cost-effectiveness analysis in the next subsection) is not zero, and stored each in one sheet on the same file. This allows the tool to generate several estimates for each cost-effectiveness output (one per matrix) and therefore to generate and display confidence intervals.

  • Percentage of change: Maximum allowed difference between two equivalent transition probability matrices.

  • Mortality: Checkbox that should be ticked if mortality data are available and it is wanted to be used in the calibration process.

  • Transition probability matrix: Starting values for the transition probability matrix. A general transition probability matrix will be used if no user-specific matrix is provided.

  • Incidence file: Registered cervical cancer incidence for the specific setting, without any medical intervention (rate × 100,000 women). It is uploaded in a spreadsheet file and will be used as a target in the calibration process.

  • Prevalence file: Registered HPV16/18 infection prevalence for the specific setting. It is uploaded in an Excel file and will be used as a target in the calibration process.

  • Mortality file: Registered mortality due to cervical cancer for the specific setting (rate × 100,000 women). It is uploaded in an Excel file and will be used as a target in the calibration process if mortality is checked.

An Excel file including all calibrated matrices can be downloaded and used for the cost-effectiveness analyses. The default transition probability matrix is calibrated to Spanish data, but the users could upload their own HPV prevalence, incidence and mortality data and calibrate the transition probabilities matrix to their own settings.

Cost-effectiveness analysis

The cost-effectiveness panel of the tool requires the following inputs:

  • Number of different scenarios: The number of scenarios to be analyzed in the current session. Note that each scenario may include none (no intervention or natural history), one or more prevention strategies (screening alone, vaccination alone or vaccination followed by screening).

  • Matrix of transition probabilities between different health states without medical intervention: Transition probability matrix obtained in the previous panel or uploaded by the user. Additionally, the included default file can be used.

  • Medical costs and utility coefficients: An Excel file containing the treatment direct medical and nonmedical costs, indirect costs and utility coefficients. An example can be downloaded from the tool.

  • Discount rate: Discount rate to be applied to health and costs (undiscounted results can be obtained by setting the discount rate to 0).

  • Uncertainty level: It is known that the results of cost-effectiveness analyses are affected by a certain degree of uncertainty at different levels. They can be reflected in the OCEAN tool by using more than one transition probability matrix to feed the Markov model. If a file with several sheets is used and the uncertainty level is set to 0, an averaged matrix will be used, and only point estimates will be reported. If the uncertainty level is set to a value α between 0 and 50, all the matrices are used, and the outcomes obtained from each one are recorded. Then, the mean and percentiles \( \frac{\alpha }{2} \) and \( 1-\frac{\alpha }{2} \) for each outcome are calculated and reported.

The considered prevention strategies that can be selected for each scenario (alone or combined) include the following:

  • Natural history: No prevention strategy is considered in this scenario, which reproduces the natural history of HPV infection and cervical cancer. If this option is chosen, the other options disappear.

  • Screening: Several screening scenarios can be defined, depending on test (cytology, HPV DNA test or visual inspection), frequency (every 1–10 years), targeted ages, and switch age from cytology to HPV testing. The screening coverage, positive predictive value, sensitivity and costs are read from the Excel file loaded as “Scenario-specific values”. The structure of this file can be explored by downloading the example file from the tool. Screening may be organized (all women are screened with the selected frequency) or opportunistic (the screening period is variable). To set an opportunistic screening scenario, an additional Excel file specifying the proportion of women screened each period is required.

  • Vaccination: Preadolescent girls are successfully vaccinated at the age of 11 years with one, two or three doses of the vaccine against HPV types 16 and 18. Efficacy and coverage are set by the user and vaccination costs are read from the Excel file loaded as “Scenario-specific values”. The structure of this file can be explored by downloading the example file from the tool. Currently, only bivalent vaccine is considered but it is planned that quadrivalent and nonavalent vaccines will be available soon as well.

An Excel file with all generated results can be downloaded, including information about the inputs used to generate those particular results.

Results

This section reproduces the cost-effectiveness analysis reported in [7] using the OCEAN tool. The goal is to mimic some of the cervical cancer prevention strategies available in Spain, particularly comparing conventional cytology to HPV testing, with and without vaccination. All the input files used in this section are available as supplementary materials. First, we can use the calibration part of the tool to check whether the input matrix we use fits the Spanish registered data in a reasonable manner (Tables S2 and S3 show the considered HPV infection prevalence and cervical cancer incidence). Through the graphs provided by the calibration part of the OCEAN tool, it can be seen whether the input matrix fits the targeted values. The output calibrated matrices are also useful to introduce some random noise (determined by the percentage of change and the number of simulations to keep), allowing us to incorporate some uncertainty into the final outcomes, providing the researcher with a more realistic picture of the cost-effectiveness results. The output matrices from the calibration part can be used as input for the cost-effectiveness analysis part, where the details on the different prevention strategies considered are set. The output from the tool shows the user that the input matrix is well calibrated (Fig. 1).

Fig. 1
figure1

Example of the calibration panel and outputs for the age-specific cervical cancer incidence and the age-specific HPV16/18 prevalence compared to the observed data. The output is based on the 5 best-fitting (the matrices producing the outcomes that minimize the differences with respect to target HPV infection prevalence and CC incidence) simulations out of 10 demanded with a 10% change

Once a -or multiple- calibrated transition probability matrix that feeds the Markov model has been generated in the calibration part, it is time for the cost-effectiveness analysis. In this example, we will consider the following prevention strategies:

  • Natural history: The first scenario considers no prevention strategy.

  • Vaccination alone: In this scenario, preadolescent girls are successfully vaccinated at the age of 11 years with three doses of the vaccine against HPV types 16 and 18. The analysis was carried out assuming favorable vaccine with 70% efficacy and a lifelong duration of vaccine immunity to prevent cervical lesions caused by HPV 16 and 18 among uninfected women. No cross-protection against other high-risk HPV types was assumed. The uptake is set to 70%.

  • Screening alone: Assuming that screening with conventional cytology starts at 25 years of age and continues until age 65, this strategy considers that primary organized HPV DNA testing is performed in women older than 30 years of age with cytology triage for positive women. Women are screened in a 5-year period. For women younger than 30 years of age, cytology is the reference test. The sensitivity and specificity of HPV DNA testing to detect CIN2+ are 90.5 and 93.0%, respectively, and 90.5 and 91.9% for cytology after a positive HPV test [15]. Screening coverage is 70%. On the basis of a study carried out in Spain, we assume that the sensitivity and specificity of cytology to detect CIN2+ are 38.2 and 97.8%, respectively [15].

  • Combined vaccination and screening: In this scenario, we implement vaccination in girls aged 11 years, followed by screening according to the parameters and assumptions described previously for vaccination and screening alone.

For all the scenarios, a 3% discount rate for health outcomes and costs and a 5% uncertainty level are applied. The unitary costs used in this example are reproduced in Table S4. Figure 2 is a screenshot from the tool with the proper inputs corresponding to each described scenario.

Fig. 2
figure2

Example of inputs and parameters for four different scenarios: no intervention or natural history, vaccination alone, screening alone and combined vaccination and screening

The output tables reported by the tool can be seen in Fig. 3 and are available as supplementary material in the same format in which can be downloaded from the tool (Table S5).

Fig. 3
figure3

Example of output tables as reported by the tool for four different scenarios

The results tables from Fig. 3 show that the health and cost outcomes are very similar to those reported in [7]. A figure reproducing Fig. 3 from [7] is available as supplementary material (Fig. S2), also leading to the expected results.

Conclusions

Carrying out complex cost-effectiveness analyses can be challenging for many researchers involved in cervical cancer prevention and for public health decision makers. The OCEAN tool provides an easy-to-use interface, requiring only a working internet connection and fairly available input information to allow its users to compare the most common cervical cancer prevention strategies worldwide using their specific parameters. There is a lack of robust, rigorous, accessible tools allowing users to conduct reliable cost-effectiveness analyses, as these analyses may play an important role in deciding public health policies in cervical cancer. We consider that the outputs of the OCEAN tool are of potential usefulness to the public health and cervical cancer prevention communities, but with some constraints.

In general, it is known that the mathematical models used in cost-effectiveness analyses are subject to uncertainty at different levels [16], and therefore, the results provided by the tool (even setting the uncertainty level over zero) must be taken with caution. In particular, the calibration process of the OCEAN tool does not include optimization routines, although they have been recently recommended in the literature [1] due to time limitations on the on-line interface.

Assessing the impact of uncertainty in the results, either through statistical analysis or through sensitivity analysis, is recommended by recent guidelines on health economic evaluation [17, 18]. The OCEAN tool facilitates the task of handling uncertainty through deterministic and probabilistic sensitivity analyses [19,20,21], so the implications on the cost-effectiveness analysis can be easily examined.

Several extensions of the software are currently planned. For instance, other vaccines (quadrivalent and nonavalent) will be included, and the vaccination administration period will be extended to other ages, including catch-up vaccination and male vaccination. Using a static underlying model also has some relevant limitations such as not capturing herd immunity benefits due to HPV vaccination, although this Markov model can handle complex screening strategies and it is known that static models improve the transparency and robustness of the results compared to dynamic models [22]. To overcome this issue, more sophisticated models based on microsimulation will be included in the near future.

Other HPV-related diseases (genital warts, recurrent respiratory papillomatosis and other cancer locations such as the vulva, vagina, penis, anus and oropharynx) will also be considered in upcoming versions of the tool.

Availability and requirements

The OCEAN tool is accessible with only a working internet connection. Sample input files are available from the tool website and are ready to be used.

Availability of data and materials

The example used in this article can be reproduced by using sample input files from the tool. Additional tables and figures are available as supplementary material.

References

  1. 1.

    Moriña D, de Sanjosé S, Diaz M. Impact of model calibration on cost-effectiveness analysis of cervical cancer prevention. Sci Rep [Internet]. 2017;7:17208 Available from: http://www.ncbi.nlm.nih.gov/pubmed/29222509, [cited 2018 Feb 27].

    Article  Google Scholar 

  2. 2.

    Bosch FX, Lorincz A, Muñoz N, Meijer CJLM, Shah KV. The causal relation between human papillomavirus and cervical cancer. J Clin Pathol [Internet]. 2002;55:244–65 Available from: http://www.ncbi.nlm.nih.gov/pubmed/11919208, [cited 2018 Feb 27].

    CAS  Article  Google Scholar 

  3. 3.

    Chesson HW, Dunne EF, Hariri S, Markowitz LE. The Estimated Lifetime Probability of Acquiring Human Papillomavirus in the United States. Sex Transm Dis [Internet]. 2014;41:660–664. Available from: http://www.ncbi.nlm.nih.gov/pubmed/25299412, [cited 2018 mar 13].

  4. 4.

    Ferlay J, Soerjomataram I, Dikshit R, Eser S, Mathers C, Rebelo M, et al. Cancer incidence and mortality worldwide: Sources, methods and major patterns in GLOBOCAN 2012. Int J Cancer [Internet]. 2015;136:E359–86 Available from: http://www.ncbi.nlm.nih.gov/pubmed/25220842, [cited 2018 Feb 27].

    CAS  Article  Google Scholar 

  5. 5.

    WHO. Human papillomavirus vaccines: WHO position paper. Biol Int. 2009;37:338–44 Available from: http://www.ncbi.nlm.nih.gov/pubmed/19525124, [cited 2018 Feb 27].

    CAS  Google Scholar 

  6. 6.

    Ekwunife OI, O’Mahony JF, Gerber Grote A, Mosch C, Paeck T, Lhachimi SK. Challenges in Cost-Effectiveness Analysis Modelling of HPV Vaccines in Low- and Middle-Income Countries: A Systematic Review and Practice Recommendations. Pharm Int. 2017;35:65–82 Available from: http://link.springer.com/10.1007/s40273-016-0451-7, [cited 2018 Feb 27].

    Google Scholar 

  7. 7.

    Georgalis L, De Sanjosé S, Esnaola M, Bosch FX, Diaz M, To C. Present and future of cervical cancer prevention in Spain: a cost-effectiveness analysis. Eur J Cancer Prev. 2015;00:0–0.

    Google Scholar 

  8. 8.

    Myers ER, McCrory DC, Nanda K, Bastian L, Matchar DB. Mathematical model for the natural history of human papillomavirus infection and cervical carcinogenesis. Am J Epidemiol [Internet]. 2000;151:1158–71 Available from: http://www.ncbi.nlm.nih.gov/pubmed/10905528, [cited 2018 Feb 27].

    CAS  Article  Google Scholar 

  9. 9.

    Canfell K, Barnabas R, Patnick J, Beral V. The predicted effect of changes in cervical screening practice in the UK: results from a modelling study. Br J Cancer [Internet]. 2004;91:530–6 Available from: http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=2409838&tool=pmcentrez&rendertype=abstract%5Cnhttp://www.ncbi.nlm.nih.gov/pubmed/15266332%5Cnhttp://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=PMC2409838.

    CAS  Article  Google Scholar 

  10. 10.

    Insinga RP, Glass AG, Myers ER, Rush BB. Abnormal Outcomes Following Cervical Cancer Screening: Event Duration and Health Utility Loss. Med. Decis. Mak. [Internet], vol. 27. Los Angeles: Sage PublicationsSage CA; 2007. p. 414–22. Available from: http://mdm.sagepub.com/cgi/doi/10.1177/0272989X07302128, [cited 2017 Feb 3].

    Google Scholar 

  11. 11.

    Kulasingam SL, Benard S, Barnabas RV, Largeron N, Myers ER. Adding a quadrivalent human papillomavirus vaccine to the UK cervical cancer screening programme: a cost-effectiveness analysis. Cost Eff Resour Alloc [Internet]. 2008;6:4 Available from: http://www.ncbi.nlm.nih.gov/pubmed/18279515, [cited 2018 Feb 27].

    Article  Google Scholar 

  12. 12.

    Kohli M, Lawrence D, Haig J, Anonychuk A, Demarteau N. Modeling the impact of the difference in cross-protection data between a human papillomavirus (HPV)-16/18 AS04-adjuvanted vaccine and a human papillomavirus (HPV)-6/11/16/18 vaccine in Canada. BMC Public Health [Internet]. BMC Public Health. 2012;12:872 Available from: http://www.biomedcentral.com/1471-2458/12/872.

    Article  Google Scholar 

  13. 13.

    Chang W, Cheng J, Allaire JJ, Xie Y, McPherson J. shiny: Web Application Framework for R [Internet]. 2017. Available from: https://cran.r-project.org/package=shiny.

    Google Scholar 

  14. 14.

    R Core Team. R. A Language and Environment for Statistical Computing [Internet]. Vienna; 2017. Available from: https://www.r-project.org/.

  15. 15.

    Ibáñez R, Autonell J, Sardà M, Crespo N, Pique P, Pascual A, et al. Protecting the underscreened women in developed countries: the value of HPV test. BMC Cancer [Internet]. 2014;14:574 Available from: http://bmccancer.biomedcentral.com/articles/10.1186/1471-2407-14-574, [cited 2018 Feb 27].

    Article  Google Scholar 

  16. 16.

    Drummond M, Stoddart G. Methods for the economic evaluation of health care programmes [Internet]: Oxford University Press; 1997. Available from: https://books.google.es/books/about/Methods_for_the_Economic_Evaluation_of_H.html?id=sz1RnwEACAAJ&redir_esc=y, [cited 2018 Feb 27].

  17. 17.

    Kim S-Y, Russell LB, Sinha A. Handling Parameter Uncertainty in Cost-Effectiveness Models Simply and Responsibly. Med Decis Mak [Internet]. 2015;35:567–9 Available from: http://www.ncbi.nlm.nih.gov/pubmed/26280060, [cited 2018 Feb 27].

    Article  Google Scholar 

  18. 18.

    Brisson M, Edmunds WJ. Impact of model, methodological, and parameter uncertainty in the economic analysis of vaccination programs. Med Decis Making [Internet]. 2006;26:434–46 Available from: http://journals.sagepub.com/doi/10.1177/0272989X06290485, [cited 2018 Feb 27].

    CAS  Article  Google Scholar 

  19. 19.

    Stout NK, Knudsen AB, Kong CY, McMahon PM, Gazelle GS. Calibration methods used in cancer simulation models and suggested reporting guidelines. Pharm Int. 2009;27:–533, 45 Available from: http://link.springer.com/10.2165/11314830-000000000-00000, [cited 2018 Feb 27].

  20. 20.

    Kim JJ, Kuntz KM, Stout NK, Mahmud S, Villa LL, Franco EL, et al. Multiparameter calibration of a natural history model of cervical cancer. Am J Epidemiol. 2007;166:137–50.

    Article  Google Scholar 

  21. 21.

    Van de Velde N, Brisson M, Boily M-C. Modeling human papillomavirus vaccine effectiveness: quantifying the impact of parameter uncertainty. Am J Epidemiol [Internet]. 2007;165:762–75 Available from: http://www.ncbi.nlm.nih.gov/pubmed/17276976.

    Article  Google Scholar 

  22. 22.

    Westra TA, Rozenbaum MH, Rogoza RM, Nijman HW, Daemen T, Postma MJ, et al. Until which age should women be vaccinated against HPV infection? Recommendation based on cost-effectiveness analyses. J Infect Dis. 2011;204:377–84 Available from: https://www.ncbi.nlm.nih.gov/pubmed/21742836.

    Article  Google Scholar 

Download references

Acknowledgments

Not applicable.

Funding

This work was supported by RecerCaixa (2015ACUP00129) and by Instituto de Salud Carlos III FIS - PI16/01254 (CECO: 17FIS012). The research leading to these results received funding from the Instituto de Salud Carlos III-ISCIII (Spanish Government) co-funded by FEDER funds/European Regional Development Fund (ERDF) - a way to build Europe (RD12/0036/0056, PIE16/00049, PI14/01219, PI11/02090 and PI16/01254) and received funding from the Agència de Gestió d’Ajuts Universitaris i de Recerca (2017SGR1718) and the European Commission 7th Framework Programme COHEAHR (Health-F3–2013-603019). David Moriña acknowledges financial support from the Spanish Ministry of Economy and Competitiveness, through the María de Maeztu programme for Units of Excellence in R&D (MDM-2014-0445) and from Fundación Santander Universidades. This work was partially funded by grant RTI2018–096072-B-I00 from the Spanish Ministry of Science, Innovation and Universities.

Author information

Affiliations

Authors

Contributions

DM, PP and MD developed the underlying model. DM, PP and JIMG designed and built the web interface. All authors discussed the results and contributed to the final manuscript. The author(s) read and approved the final manuscript.

Corresponding author

Correspondence to David Moriña.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

None of the authors have any conflicts of interest.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Moriña, D., Martí, J.I., Puig, P. et al. Online Cost-Effectiveness ANalysis (OCEAN): a user-friendly interface to conduct cost-effectiveness analyses for cervical cancer. BMC Med Inform Decis Mak 20, 211 (2020). https://doi.org/10.1186/s12911-020-01232-9

Download citation

Keywords

  • Cervical cancer screening
  • Markov chain model
  • Cost-effectiveness
  • Online tool
  • cancer prevention
  • Decision making