Skip to main content

Table 1 Evidence generation within the justifiable healthcare model

From: The concept of justifiable healthcare and how big data can help us to achieve it

Step

Tool/method

Epistemologic aim

Pitfalls/problems

Role for big data/AI

Step 1: Evidence based medicine

Generate hypothesis

Physiology

Pathophysiology

Anatomy

Biology

Pathogenesis

Description:

Describe biological processes in detail

Causal Inference:

Enhance mechanistic understanding of basic biological processes and how they can be influenced

Information overload

Transportability/generalizability

Incorrect analytical approach

Pattern recognition

Classification

Clustering

Searching and Aggregating all already existing knowledge

Prediction of mode of action of different molecular structures as derived from existing knowledge

Propose alternative structures based on existing molecular knowledge

 

Epidemiology

Description:

Describe population in detail for given conditions and outcomes: prevalence, incidence, risk factors, associated factors

Prediction:

Predict relevant outcomes based on a given set of covariates for a given population (association) or an individual from that population (risk prediction or prognosis)

Causal Inference:

Enhance understanding of causal effect association between a covariate and an outcome in a given population

Data quality

Transportability/generalizability

Incorrect analytical approach (remaining confounding and bias; incorrect causal inference)

Pattern recognition

Classification

Clustering

Searching and Aggregating all existing knowledge

Prognostication

Trial emulation

Trial simulation

Causal inference techniques

Dynamic Decision Problems

Proof of concept

(small) phase 1 and 2 trials

Description: describe properties of population

Prediction: Predict relevant outcomes based on a given set of covariates for a given population (association) or an individual from that population (risk prediction or prognosis)

Causal inference:

Establish (near) causal effect association between a covariate and an outcome in a given population

Estimation of effect size of an intervention on a given outcome in a given patient population

Publication bias

Framing

Relevance of outcomes

Searching and Aggregating all existing knowledge toavoid unnecessary duplication or indicate gaps in knowledge

Prediction of mode of action of different molecular structures as derived from existing knowledge

Trial emulation

Trial simulation

Causal inference techniques

Dynamic Decision Problems

Efficacy

Randomized controlled trial

Causal Inference:

Estimation of effect size of an intervention on a given outcome in a given patient population

Information overload

Internal validity (classic bias)

Publication bias

Incorrect analytical approach

Framing

Relevance of outcomes

Rare side effects not captured in RCTs

Transportability/generalizability

For static decision problems only

Searching and Aggregating all existing knowledge to avoid unnecessary duplication or indicate gaps in knowledge

Causal inference techniques to determine transportability to other patient populations

Dynamic Decision Problems

 

Observational trial or registry

Description:

Describe associations between intervention and outcome

Describe effect modifiers of that association

Describe safety and association with side effects

Prediction:

Describe prognosis and evolution of conditions, and the modifiers

Causal Inference:

Estimation of effect size of an intervention on a given outcome in a given patient population in settings where RCTs are not possible or not ethical

(Unmeasured) Confounding

Publication bias

Framing

Relevance of outcomes

Transportability/generalizability

Trial emulation to explore residual confounding

Trial simulation

Effectiveness level 1

Systematic review of randomized controlled trials

Description:

Identify and aggregate available evidence

Judge quantity and quality of available evidence

Causal Inference:

Based on effect size estimators, mean changes in outcomes can be estimated at population level when the intervention is implemented

Based on subgroup analysis, change in outcome for subgroups can be estimated when the intervention is implemented

Information overload

Publication bias

Rare side effects not captured in RCTs

Searching and Aggregating all existing knowledge

Trial emulation to explore transportability/generalizibility

Pragmatic trial approaches

 

Systematic review of observational trials

Description:

Identify and aggregate available evidence

Judge quantity and quality of available evidence

Describe associations between intervention and outcomes

Describe safety and association with side effects

 

Searching and Aggregating all existing knowledge

Trial emulation to explore confounding

Step

Tool/method

Epistemologic aim

Pitfalls/problems

Big data

Step 2: Socio-economic evaluation

Health technology assessment

Incremental Cost Effectiveness Ratio

Estimate cost of the intervention per quality adjusted life year (QALY) as compared to an alternative intervention

 

Use wearables and online data collection to estimate QoL impact (patient reported outcomes/experience measures)

Searching and Aggregating all existing knowledge on this and alternative interventions

Detailed description and granularity of target population to help estimate impact in true society (transportability/generalizability)

 

Budget Impact

Estimate total cost on population level of the intervention

 

Detailed description and granularity of target population to help estimate impact in true society (transportability/generalizability)

 

Opportunity Cost

Estimate impact of implementation of this intervention on other interventions in the same or other comparable domain

 

Detailed description and granularity of target population to help estimate impact in true society (transportability/generalizability)

Searching and Aggregating all existing knowledge on this and alternative interventions

Societal evaluation

Quality assessment/improvement

Description:

Describe interventions and outcomes in different settings

Describe effect modifiers and case-mix in different settings

Prediction:

Identify subpopulations where certain interventions have different outcome

Causal Inference:

Estimation of effect size of an intervention on a real world population and conditions

Data collection is resource heavy(cost/labor force)

Missing data/Incomplete data

Cherry picking

Automatic online registration of relevant parameters to allow quality assessment in a sustainable, reliable, complete and cost-effective way

 

Pharmacovigilance (Side effects)

Describe safety and association with side effects

Rare side effects not captured in RCTs

Side effects might be different in subpopulations not represented in RCTs

Detailed description and granularity of target population to help estimate impact in true society (transportability/generalizability)

 

Patient reported outcome/experience measure (PROM/PREM)

Description:

Describe interventions and patient-relevant outcomes in different settings

Prediction:

Identify subpopulations where certain interventions have different outcome

Causal Inference:

Estimationw of effect size of an intervention on a real world population and conditions

RCTs not representative for real world practice and not using patient relevant outcomes

Differences in implementation might cause differences in outcomes

Automatic online registration of relevant parameters to allow quality assessment in a sustainable, reliable, complete and cost-effective way