BMC Medical Informatics and Decision Making BioMed Central Study protocol Protocol for the Quick Clinical study: a randomised controlled trial

Background: Online information retrieval systems have the potential to improve patient care but there are few comparative studies of the impact of online evidence on clinicians' decision-making behaviour in routine clinical work.


Background
Several studies have shown that clinicians have many unanswered questions during clinical encounters which may impact on the quality and outcomes of decisions made. Depending on measurement techniques used to estimate the frequency of questions and the clinical setting, clinicians have anywhere between 1 and 6 questions per patient encounter and answers are pursed for only about 30% to 55% of questions [1][2][3][4]. Improving access to clinical evidence through the implementation of evidence retrieval systems is one way of supporting information seeking in routine care [5]. Easy access to up-to-date evidence at the point of care may be more effective than other interventions aimed at improving clinicians adoption of evidence-based practice such as CME, distribution of clinical guidelines, clinical detailing, etc.
The effect of implementing information retrieval technology in routine clinical settings and its impact on clinicians' decision-making has not been extensively investigated. A 2005 review found that on average one in three searches for information were associated with a positive impact on decision-making, however in 73% (19 of 26) of the studies examined in this review, the impact was self-reported by clinicians using the system [6]. Assessments of evidence retrieval systems have relied principally on post study surveys. Only a small number have directly measured use in clinical settings and in most cases patterns of searching and clinical usefulness of information retrieved could not be identified.
There are few comparative studies of online evidence retrieval technology in routine clinical settings. A Finnish study found no difference in compliance with recommendations between computerised and paper-based clinical guidelines among newly qualified physicians [7]. In another study involving fourth year medical students, participants reported that access to information on a handheld computer encouraged use of information, improved learning and increased confidence in decision-making [8]. However, the effect of information retrieval technology on clinicians decision-making process and impact upon patient outcomes has not been assessed.
We are conducting a randomised controlled trial (RCT) to investigate the effectiveness of an information retrieval system called Quick Clinical (QC) in general practice [9]. The feasibility of the QC system was demonstrated in a previous clinical trial, detailed results of this study are reported in a separate paper [10]. Overall, 193 clinicians used the online evidence system to conduct on average 8.7 searches per month. The majority of these searches were conducted from consulting rooms (81%) during office hours. The most frequent searches conducted related to questions about diagnosis (40%) and treatment (35%).
Search subjects included a broad spectrum of diseases, including common conditions such as asthma, diabetes and hypertension. In this paper we describe the design and proposed analysis of an RCT to evaluate the effectiveness of using QC in routine general practice consultations.

Study aims
The aim of this trial is to assess the effectiveness of an online evidence retrieval system (QC) in improving clinical decision-making processes in general practice.
The specific hypotheses to be tested in this study are that: 1. Online evidence systems are clinically acceptable and will be used by clinicians in 'real world' general practice settings.
2. Online evidence systems are effective in changing clinical decision-making behaviour and result in measurable improvements in evidence-based prescribing decisions.

Study design and setting
A randomized controlled parallel design is used to evaluate the effectiveness of QC.

Intervention
Participants are randomised to receive access to QC in their practice for 12 months. The system is available from a standard web browser e.g. Microsoft Internet Explorer, Firefox etc. (Figure 1). Each user has a personal username and password and completes an online tutorial about how to use the system. QC is based on the generic use of search filters specifically designed around the information needs of general practitioners (GP). Five types of search filters or "profiles" are available (disease aetiology, diagnosis, treatment, prescribing and patient education). Up to four keywords can be used to describe a clinical question. Clinicians search for information firstly by selecting a search filter to match their question type (e.g. diagnosis, treatment etc.) and then entering keywords to describe the query. For example, a clinician who encounters a 32-year old woman with a fourth presentation of pelvic pain in the last 6 months with ultrasound and swabs for infection all negative, may have a question regarding the social, psychological as well as biological causes of pelvic pain. The clinician could select the 'etiology' profile and enter 'pelvic pain', 'pathology' and 'psychosocial' as keywords (Figure 1). The search filters retrieve evidence from resources including Therapeutic Guidelines (a summarised evidence resource developed by Australian experts); the Merck Manual; Health Insite (a government funded health database); PubMed; MIMS (a pharmaceutical database); and a collection of relevant Australian guidelines identified by a panel of clinical experts. Users can also search each of these resources individually.

Outcomes
The primary outcome measures are clinician acceptance and use of QC and the resulting change in decision-making behaviour. The data from our pilot study of QC involving 227 GPs showed that online evidence was sought in relation to both frequent and rare clinical conditions across a broad range of clinical areas [10]. This suggests that we are likely to see access to evidence having a broad impact across common and rare clinical decisions. The RCT will therefore examine prescribing patterns related to frequently prescribed medications where there has been a recent significant shift in recommendations regarding their use based upon new evidence. Examining the effect on management decisions in clinical areas where there is shifting evidence enables a hypothesis driven approach to determine the impact of online evidence on clinician behaviour.
The RCT will therefore focus on the changes in clinicians' prescribing patterns in Australian National Health priority areas identified by a panel of experts including chief investigators, associate investigators and general practitioners (Table 1). Non-pharmacological clinical management in these priority areas will also be examined e.g. referral for podiatry (diabetes management), spirometry (asthma), HbA1c testing (diabetes), cognitive behavioural therapy (CBT) for depression, dieticians for lipid disorder. In addition to changes in priority areas, prescribing patterns in response to new evidence of the effectiveness of new or existing treatments identified through reviews of literature and expert input will also be assessed. Secondary outcome Quick Clinical user interface Figure 1 Quick Clinical user interface.
measures to determine impact on decision-making behaviour include self-reported changes in diagnosis, patient education, prescriptions, investigations and referrals.

Ethical considerations
Ethical approval for the study was obtained from the University of New South Wales's Human Research Ethics Committee and subsequently ratified by Sydney University and the Royal Australian College of General Practitioners ethics committees.
The study is also recognised by the Royal Australian College of General Practitioners (RACGP) for its continuing medical education (CME) program. Education points are not directly linked to participants' use of QC, but to trial completion.

Identification of eligible general practitioners
GPs who had a computer with Internet access in their consulting rooms and used electronic prescribing are recruited via a call for volunteers advertised in journals, newsletters and a clinician list-server. An overview of the trial process is shown in Figure 3. Volunteers are asked to register via a web page and to complete a short survey to determine their eligibility to participate in the study. The online registration gathers baseline data about the demographics of the sample including practice characteristics and computer use [see Additional file 1]. Eligibility and exclusion criteria are listed in Table 2. To participate in the study clinicians must be registered to practice in Australia; have a computer with Internet connectivity; and prescribe electronically. GPs who are planning to retire or move to another practice within 12 months or participating in other concurrent trials involving electronic extraction of prescription data are excluded from the study.
Eligible GPs are randomly allocated to either the control or intervention group, each clinician is given a unique study identification code and the web address of QC is sent to eligible participants one week prior to commencement of the trial. Participants visiting the QC web site are given detailed information about the trial along with the participant information statement and the consent form. A revocation of consent form is also available online. Clinicians willing to participate complete the online consent form and a pre-trial survey which assesses information seeking habits and attitudes to information seeking during consultations [see Additional file 2]. A psychometric instrument used to classify clinicians' response styles to new information as seekers, receptives, traditionalists, or pragmatists is also administered as part of the pre-trial survey [see Additional file 3] [11].

Data collection
The timing and content of study assessments are summarised in Table 5. Frequency and purpose of system use of QC are determined from automatically generated computer logs used to record details of each search including the search filter chosen, keywords entered, data sources accessed, user IP (Internet Protocol) address, date, time and duration of searches [see Additional file 4]. Users are asked to identify if the IP address is their home or consulting room. An online feedback facility within the QC user interface is used to record clinicians comments in context of specific searches that generate the feedback (Figure 2).
Participants in both the control and intervention group provide prescribing data in electronic format [see Additional file 5]. The data extraction process excludes all potentially identifying information such as patient name, address, gender, postcode, telephone number, Medicare  number, progress notes, images (e.g. X-rays) and any data fields that could identify a patient, such as free text fields. Data for the study period (i.e. 12 months retrospective data plus 12 months of the intervention period) is electronically extracted from prescribing software using secure means without any intrusion on clinicians' work.

Inclusion criteria • general practitioners registered to practice in Australia • use a computer with Internet access in consulting rooms • use electronic prescribing Exclusion criteria
• concurrent participation in other clinical trials involving electronic extraction of prescription data • planning to retire or move within the next 12 months The prescription data extraction software and installation instructions are sent by email and also are made available online. The first data extraction (i.e. retrospective prescriptions data from the previous 12 months) is performed after installation of the extraction software at the start of the study. The second extraction is scheduled at 6 months and the final at 12 months. An uninstall program will remove all extraction software, created files and changes to profiles in clinicians' prescribing software. At the end of 12 months all participants complete a post-trial survey [see Additional file 6]. Participants attitudes to information seeking and responses to the psychometric instrument used in the pre-trial survey are repeated post-trial.

Randomisation, blinding and allocation concealment
Eligible GPs are stratified by gender and randomised to the intervention or control group using a computer generated random number sequence in randomly assigned blocks (max. block size 8) and online evidence allocation ratio of 1:1. For each level of strata, a random sequence of study group assignments is generated using a computerized random-number generator [12]. Each level of strata are numbered consecutively. Each doctor receives the next consecutive number in his or her stratum.
Since QC is a behavioural intervention it is not possible to blind participants to the intervention. However investigators are blinded to group assignment. The randomisation sequence is generated and clinicians are randomly assigned to two groups by a researcher from the Evaluation team of the Centre for Health Informatics. This individual is not directly involved in the study nor its analysis.
The group assignment is revealed to clinicians online only after they consent to participate in the study following completion of the pre-trial survey. To minimise contami-Overview of trial process Figure 3 Overview of trial process. Enrollment nation of control participants who might work closely with practitioners who are part of the intervention group, only one clinician is eligible to participate in the trial from each practice. In cases where there are multiple registrations from a single practice, only the first registrant is eligible to participate. Participants in the intervention group are asked not to share their QC access details with their colleagues.

Communication with participants
Email will be the primary channel to communicate with participants for information and reminders about: 1. start of the trial (followed up with 2 fortnightly reminders) 2. instructions for installing and using prescribing data extraction tool 3. post-trial survey

Subject withdrawal
Participants who formally withdraw from the study are contacted by telephone to determine reason for withdrawal. Similarly participants who do not use QC nor pro-vide prescribing data are contacted to determine reasons for not completing the study.

Sample size considerations
Sampling unit is the GP and the unit of analysis is also the GP. We provide the following example to demonstrate that the RCT sample size will be more than sufficient for detecting statistically significant prescription changes in medication areas where there is a shift in evidence. The example used to verify the design is measuring a 10% increase in use of preventer medications (glucocorticoid inhaled medications) in Asthma. Currently on average, 53% of GP encounters for asthma lead to the prescription of a preventer medication [13]. We expect that access to QC will promote the use of preventer medication by GPs who use the intervention. We expect that in the intervention group -at least 58.4% of GP encounters for asthma lead to the prescription of a preventer medication.
As the prescriptions belonging to the same GP will tend to be correlated, the sample size calculations estimated the average number of condition-specific prescriptions per GP and adjustments were made for intracluster correlation factor estimated to be 0.02 [14].
The number of GPs required in each arm to detect a 10% change in average prescription rate per 100 condition-specific encounters per GP are listed in Table 4[15]. Given  that data collection in the trial will run for a minimum of 12 months 100 GPs in each arm will be sufficient to detect a 10% change in prescriptions (p = 0.05, power = 0.80).

Statistical analyses
All registered participants will have completed the pretrial survey. The subset of the intervention group analysed for primary outcome will have completed the post-trial survey and provided prescribing data. Participants in intervention group who encounter technical difficulties in using QC will be excluded from this analysis (i.e. those who had the opportunity to use QC but did not do so will be included). All of the control group except those who did not provide prescribing data will be included in the primary analysis. As summarised in Table 5 the intervention and control groups will be compared for baseline differences in prescription patterns, case-mix and profile of participants. Using an intention to treat analysis adjusted data will be analysed to determine differences in prescribing patterns and non-pharmacological management in identified priority areas, and areas in which there have been recent changes in response to new evidence using a one-tailed binomial test for proportions. Self-reported changes in diagnosis, patient education, prescriptions, investigations, referrals will also be examined. Utilisation and outcome measures will be analysed relative to QC usage and patient load (e.g. by number of patients seen).

Analyses to determine impact of Quick Clinical on decision-making
1. Baseline comparisons between control and intervention groups (unadjusted data) a) proportion of prescriptions by ATC* categories and in priority areas (using 12 months retrospective data) b) case-mix (age-gender distribution of patients) c) participants' profile (gender, age, place of graduation, geographic distribution, computer skills) 2. Interim analyses at 6 months a) analyses of baseline prescribing data. Reports of interim analyses affecting the ongoing conduct of the trial and interpretation of final results 3. Using an intention to treat analysis (i.e. irrespective of patterns of QC use in the intervention group) we will include all participants in the primary analyses using adjusted data to determine: a) broad differences in proportion of prescriptions by ATC classification b) specific differences in proportion of prescriptions by priority area (e.g. Antibiotics prescribed for URTI) c) differences in prescription patterns in response to new evidence of the effectiveness of new or existing treatments d) differences in non-pharmacological treatments e) analysis of prescription changes relative to search categories *Anatomical Therapeutic Chemical classification system