Table 1 Aims, objectives and hypotheses, and outcomes

Phase One: Feasibility and Utility Analysis (Mixed Methods Anaylsis, Framework Method)

1.1) Establish the feasibility and utility of the data collection design

1.1.1) Evaluate compliance with the note-collection system

1.1.1) Evidence of feasibility of this data collection design within practice

1.1.2) Report any difficulties that arose in descriptive qualitative analysis process

1.1.2) Evidence of the collected data’s utility for qualitative insights into decision behaviour

1.2) Describe patterns of follow-up behaviour within the non-adherence data, establishing the feasibility and utility of AI2 to enable follow-up

1.2.1) Report on repeated follow-up decision behaviours in the data

1.2.1) Inductively derived descriptive codes, describing repeated patterns of decision behaviour

1.2.2) Use these codes to group notes into behavioural categories within the a priori framework of follow-up behaviours

1.2.2) Deductive categorization of and, therefore, generation of frequency data for code occurrence within categories of follow-up using a framework method approach

1.2.3) Describe frequency of behavioural patterns within different metadata derived categories of interest

1.2.3.1) Between-medication subtypes and follow-up status descriptive statistics

1.2.3.2) Within-patient, between-follow-up status descriptive statistics

Phase Two: Generation of Design Insights (Qualitative Analysis, Thematic Synthesis)

2) Explore emergent interaction behaviours with the non-adherence data beyond the categories of followed-up and not followed-up

2.1) Explore barriers to and facilitators for follow-up behaviours within AI²

2.1) Analytical themes going beyond the raw data and generating new categories for intervention and experimentation

Phase Three: Preliminary evaluation the impact of medication and patient-level characteristics on follow-up (Mixed Methods Analysis, Framework Method)

3) Addressing the problem of establishing—quantitatively—whether CDSS impacted clinician choice using data from Phase One

Hypothesis 1 (H1): The number of flagged patients followed-up will differ significantly between medication subtypes

Test(s): Chi-squared (χ²) test of homogeneity (Cramér’s v to indicate effect size) to confirm variance in distribution of follow-up status between-drugs. Pair-wise Fisher’s Exact tests of independence (χ² statistic and Cramér’s v to indicate effect size) to explore significance of difference between individual drug types

Assumptions, χ²:

a) Independence of observations

b) No more than 20% of cells have an expected frequency of < 5, no cell has an expected frequency < 1

c) χ² < critical value for the relevant degrees of freedom [88–90]

Assumptions, Fisher’s Exact Test:

d) Independence of observations

e) Fixed column totals, however, also appropriate where column totals are not fixed should cell sizes be too small for a χ² test [92]

Reported statistics: χ² statistic, expected counts per cell, actual counts per cell, p value, Cramér’s v

Hypothesis 2 (H2): The time taken by clinicians to action flags will differ significantly between medication subtypes

Test(s): It is anticipated that these data will not be normally distributed; this assumption will be tested with Shapiro–Wilk tests

Kruskal–Wallis H Test, η² for effect size

Assumptions:

a) Independence of observations

b) Cell size > 5

c) Continuous distribution [89]

Should the null hypothesis be rejected, a squared ranks test — exploring homo/heterogeneity of variances between samples will be conducted [89–91]

Reported statistics: H statistic, count per cell, p value, η² statistic

Hypothesis 3 (H3): There will be a significant difference in the time taken by clinicians to action flags between the two categories of follow-up

Test(s): It is anticipated that these data will not be normally distributed; this assumption will be tested with Shapiro–Wilk tests

Mann–Whitney U Test, η² for effect size

Assumptions: As per the Kruskall-Wallis H Test

Reported statistics: U statistic, count per cell, p value, η² statistic

Hypothesis 4 (H4): In patients with mixed follow-up status on their flags, a monotonic time × event relationship will exist — with follow-up more likely to occur in this group as the number of flagged non-adherence events increases

Test(s): Time × event (Cox proportional hazards) regression, log–log plots

Assumptions:

1) Non-informative censoring; that is, individuals not participating in the study would have the same probability of experiencing follow-up as those in the study should they have participated

2) Hazard functions remain proportional (eg., if an individual—at baseline—is less likely to be follow-ed up than another individual, this should not change over time). Tested with log–log plots

Reported statistics: Coefficient, standard error, hazard ratio, 95% CI, p value, log–log plots [93]