Skip to main content
Download PDF

Implementation of multiple-domain covering computerized decision support systems in primary care: a focus group study on perceived barriers

BMC Medical Informatics and Decision Making volume 15, Article number: 82 (2015) Cite this article

Abstract

Background

Despite the widespread availability of computerized decision support systems (CDSSs) in various healthcare settings, evidence on their uptake and effectiveness is still limited. Most barrier studies focus on CDSSs that are aimed at a limited number of decision points within selected small-scale academic settings. The aim of this study was to identify the perceived barriers to using large-scale implemented CDSSs covering multiple disease areas in primary care.

Methods

Three focus group sessions were conducted in which 24 primary care practitioners (PCPs) participated (general practitioners, general practitioners in training and practice nurses), varying from 7 to 9 per session. In each focus group, barriers to using CDSSs were discussed using a semi-structured literature-based topic list. Focus group discussions were audio-taped and transcribed verbatim. Two researchers independently performed thematic content analysis using the software program Atlas.ti 7.0.

Results

Three groups of barriers emerged, related to 1) the users’ knowledge of the system, 2) the users’ evaluation of features of the system (source and content, format/lay out, and functionality), and 3) the interaction of the system with external factors (patient-related and environmental factors). Commonly perceived barriers were insufficient knowledge of the CDSS, irrelevant alerts, too high intensity of alerts, a lack of flexibility and learning capacity of the CDSS, a negative effect on patient communication, and the additional time and work it requires to use the CDSS.

Conclusions

Multiple types of barriers may hinder the use of large-scale implemented CDSSs covering multiple disease areas in primary care. Lack of knowledge of the system is an important barrier, emphasizing the importance of a proper introduction of the system to the target group. Furthermore, barriers related to a lack of integration into daily practice seem to be of primary concern, suggesting that increasing the system’s flexibility and learning capacity in order to be able to adapt the decision support to meet the varying needs of different users should be the main target of CDSS interventions.

Peer Review reports

Background

Over the past years there has been an increase in the availability of computerized decision support systems (CDSSs) in all areas of healthcare, including the primary care setting. CDSSs are information systems designed to optimize clinical decision making [1]. By matching characteristics of individual patients to a computerized medical knowledge base CDSSs can provide patient-specific recommendations to healthcare providers during patient consultations. In this way, they have the potential to improve quality of care [2, 3].

Despite the increased availability of CDSSs in various healthcare settings, the use of these systems in practice is still limited [4]. Not surprisingly, conclusive evidence on their effectiveness in improving quality of care also remains to be established [5]. Whereas some reviews have shown that CDSSs can improve medical practice, they do not always result in improvements [1, 6–10]. Moreover, the effects of CDSSs on patient outcomes have been less studied and results have been less favorable and less consistent [10–16].

To improve the use and effectiveness of CDSSs, insight is needed into users’ perceived barriers to using CDSSs in practice. Several studies have been conducted to identify the factors that physicians perceive as hindering implementation [4, 12, 17–21]. These studies indicate that patient-related factors (e.g. effects on patient communication) and environmental factors (e.g. organizational context) are considered as important barriers to implementing CDSSs in practice. The body of evidence on barriers and how to overcome them, however, is still limited [4].

Most barrier studies have focused on CDSSs that are aimed at a limited number of decision points e.g. [12, 22] rather than on multiple-domain covering CDSSs targeting multiple groups of users. In addition, these CDSSs have usually been tested in selected small-scale academic settings rather than being practice-driven and implemented at a large scale. Large scale implemented CDSSs in primary care in which several types of primary care practitioners (PCPs) work with a multiple-domain covering CDSS [23] may yield different types of barriers among their users.

The aim of this study was therefore to identify the perceived barriers to using large-scale implemented CDSSs, covering multiple disease areas in primary care. In addition, interventions to improve the use of CDSSs as suggested by the target group were identified. We included all types of PCPs that could potentially work with CDSSs in our study, rather than just general practitioners (GPs), as to maximize the generalizability of our findings. This paper focuses on the perceived barriers; results on suggested interventions will be described elsewhere.

Methods

Setting

In the Netherlands, there are approximately 5,000 general practices. Within these practices nearly 11,000 GPs are delivering care [24]. More than 1,700 medical doctors are in training to become GPs [24]. These GP trainees all have completed a 6-year master program in general medicine and work 4 days a week during an average period of three years in a group practice under supervision of an experienced GP. One day per week is focused on educational activities and group meetings in which daily problems are discussed and videotapes are sometimes presented [25]. Aside from GPs and GP trainees, between 3.700 and 4.700 practice nurses (PNs) work within 75 % of these general practices [26]. They are mainly responsible for providing basic care such as regular check-ups for patients with a chronic illness and completing their patient files. Together, these PCPs (GPs, GP trainees and PNs) are responsible for providing primary care in Dutch general practices. Currently, a total of seven major different electronic health record systems (EHRS) are used.

CDSS initiative in Dutch general practice: NHGDoc

Within the Dutch primary care setting there is one main CDSS initiative, which is called NHGDoc. NHGDoc is a CDSS initiated and developed in 2006 by ExpertDoc BV and currently owned by the Dutch College of General Practitioners (DCGP, NHG in Dutch). NHGDoc is integrated by web services within the electronic health record system (EHRS) and is based on the NHG guidelines, the prevailing guidelines for general practice in the Netherlands [27]. It provides GPs, GP trainees and PNs evidence-based and, on the basis of structured data in the EHRS, patient-specific advices during consultation in terms of patient data registration, drug prescription and management [28].

At the time of conducting the focus group study NHGDoc covered the following disease areas: Cardiovascular risk management, Asthma/COPD, Diabetes mellitus type II, Thyroid disorders, Viral hepatitis and other liver diseases, Atrial fibrillation and Subfertility, Gastro protection and Chronic renal failure. For each NHG guideline key recommendations have been selected based on relevance of disease burden, revision status of the guideline, and opportunity to translate or normalise the recommendation into if-then rules. This selection of key recommendations is approved by representative experts of the guideline committees. Subsequently, the selected key recommendations are digitized, thoroughly tested and deployed into the NHGDoc system. The total number of key recommendations/advices that could be shown per domain varies from 50–250 key recommendations/advices. However, the average number of key recommendations/ advices that is shown per patient encounter is 7.2, based on an average number of 2.3 domains/disease areas.

At the time of conducting this study, NHGDoc was integrated in 6 out of 7 major EHRSs being used in Dutch general practice, covering approximately two-third of all Dutch general practices. NHGDoc had been available for approx. 2.5 years for MicroHIS X users, 2 years for Promedico-ASP users and less than half a year for users of the other included EHRSs.

NHGDoc - Basic functions: alerts and feedback

When a PCP opens a patient file in the EHRS, anonymous patient data and medical performance data are sent to the NHGDoc server. The data are then compared to the digitized guideline recommendations and in case of a discrepancy between current and advised care, an alert will be sent back to the PCP. By default, the NHGDoc alert button is displayed in green, but turns into yellow when a discrepancy is detected (see Fig. 1). It is up to the PCP to open the NHGDoc alert or not. In the Promedico EHRS, the alert button could also be gray, in which case the user would need to manually request the alert.

Fig. 1
figure 1

Appearance of an NHGDoc alert button in an EHRS (MicroHIS X). Copyright: iSOFT NEDERLAND B.V., Mendelweg 32, 2333 CS Leiden, the Netherlands

Full size image

When the GP (trainee) or PN clicks on the yellow NHGDoc alert button, an alert window appears (see Fig. 2). For each relevant domain, the NHGDoc alert includes up to three types of patient-specific advices: on patient data registration, on management and on drug prescription. The alert is sensitive to the specific patient case (based on patient-specific ICPC (International Classification of Primary Care) codes, NHG Lab codes (codes used by the Dutch College of General Practitioners for laboratory and other diagnostic tests and results) and ATC (Anatomical Therapeutic Chemical Classification System) codes), and generates feedback by showing the recommendation(s) for which discrepancies were found as compared to the guideline recommendations. At the bottom of the alert, the patient profile (characteristics of patient file) consisting of the data on which the advices are based, is shown.

Fig. 2
figure 2

Example of an NHGDoc alert. Copyright: ExpertDoc B.V., Veerkade 8d, 3016 DE Rotterdam, the Netherlands

Full size image

By clicking on the Feedback link at the top right corner, users have the option to ask or provide feedback to/from ExpertDoc (the organization that has developed and maintains NHGDoc) about the received alert. ExpertDoc also informs the NHG about the received user feedback.

NHGDoc - Personalization functions: Tailoring My NHGDoc to PCPs’ specific needs

Aside from the basic functions, NHGDoc allows the user to adapt the decision support to meet their personal preferences in two different ways.

Alert settings

Users can adjust the preferences of the alerts to match their personal needs. They can choose to switch alerts on and off on demand at several levels: the system (NHGDoc as a whole – only in the EHRS MicroHIS X), the modules (NHGDoc domains), the types of alerts (patient data registration, management, drug prescription), and the patients. NHGDoc can also be used as an educational tool for specific domains, for example, a practice can choose a temporary switch-on of a module to generate input for small-group peer review and quality improvement activities.

Reporting settings

Users also have the option to request specific reports with respect to the number and types of alerts they have received per domain within a specific period of time (per year, per month, per week or per day).

Study design

We used a qualitative study design including three 1.5-h focus groups. Focus groups have proven to be useful as a method of providing in-depth information and for exploring cognitions and motivations underlying behavior [29–32]. The focus group sessions therefore enabled us to identify the perceived barriers to using CDSSs in primary care among the target groups of users.

The focus group study, of which the results are presented in this paper, is part of a larger evaluation study on the effectiveness of NHGDoc in improving quality of primary care [33]. The need for ethical approval for the NHGDoc evaluation study was waived by the research ethics committee of the Radboud university medical center.

This focus group study has been designed and reported (whenever applicable) in accordance with the RATS guidelines [34].

Selection of participants

To select participants we sent a direct email to all 233 practices that participated in the NHGDoc evaluation study [33]. All PCPs working within these practices (GPs, GP trainees and PNs) were invited to participate in the focus group study. After two weeks a first reminder was sent and after four weeks we sent a second reminder. Also, an invitation was sent by email to all medical doctors receiving training as a GP at the University Medical Centre in Utrecht, the medical centre located in the area in which the focus groups were to be conducted. Additional announcements were placed on relevant websites, in newsletters and through social media (i.e. Facebook, Twitter and LinkedIn). PCPs could register for one of the three focus groups organized. All participants received a gift voucher of €100, − and were offered reimbursement of their travel expenses.

Focus group sessions

The focus group sessions were conducted at the NHG, which is located in the center of the Netherlands. In each focus group session, the PCPs had a semi-structured discussion about their perceived barriers to using CDSSs such as NHGDoc in primary care. The sessions were moderated by ML and RBK (principal investigators of the NHGDoc evaluation study and experienced moderators of focus groups) and by a representative of the NHG.

Prior to the formal start of the discussion, one of the moderators gave instructions about the focus group session and explained that the responses of the participants will remain anonymous and that their names will not be mentioned in publications. In addition, the moderators’ independence towards NHGDoc, the main CDSS in Dutch general practice, was emphasized. Representatives of ExpertDoc, the organization that has commercial interests in NHGDoc, were deliberately not invited to the focus group sessions. Participants were asked for their approval to participate and gave permission to audio-tape the session.

A predefined topic list was used to structure the discussion. This list consisted of the following broad themes: the value of CDSSs in a primary care setting, CDSSs in an ideal world, experiences with using CDSSs with the example of NHGDoc, perceived advantages and disadvantages, and barriers to using them in practice. The three focus group sessions were audio-taped.

Data analysis and synthesis

The focus group sessions were transcribed verbatim. Two researchers (ML and JWW) independently studied the transcripts in Atlas.ti 7.0. They first independently created a code list consisting of the main barriers to using CDSSs. After studying the first half of the first transcript, the two code lists were compared and discrepancies were discussed until consensus was reached. This process was repeated after studying the second half of the first transcript. The remaining transcripts were categorized using the mutually agreed on code list.

Next, the code list was discussed and emerging themes were grouped by theory-based categories. As a basis we used the framework of Cabana [35], which presents barriers to using clinical practice guidelines, complemented with literature focusing on barriers to using CDSSs [4, 17, 18]. The information in each category of barriers was reflected on and interpreted jointly. This process resulted in the framework of barriers to using CDSSs presented in Table 2.

Results

Description of participants

Twenty-five PCPs registered for one of the three focus groups of which 24 participated (96 %), varying from 7 to 9 per session. Fifty-four percent of the participants were male (n = 13); their mean age was 47 years (see Table 1). Most of the participants worked as a GP (N = 15); followed by PNs (N = 5) and GP trainees (N = 4). The majority of the participants worked in a duo practice (n = 9) and used the EHRS MicroHIS X (n = 12).

From the group of GPs (N = 15), the majority was male (80 %), were aged 45 years or older (71 %) (mean age = 52) and worked in either a solo practice or a duo practice (both 43 %). Compared to the total population of Dutch GPs [24], GPs in our sample were more often male (80 % versus 57 %), were somewhat older (52 versus 48.5 years) and worked relatively less often in group practices (14.2 % versus 36 %). The GP trainees were predominantly female (75 %) and had a mean age of 30 years. From the group of PNs all (n = 5) were female; their mean age was 47 years.

Table 1 Characteristics of participants
Full size table

Perceived barriers

Three main types of barriers emerged: knowledge-related barriers, barriers related to the evaluation of the features of the CDSS (source and content, format/lay out, and functionality), and external barriers interacting with the CDSS (patient-related and environmental factors) (see Table 2).

Table 2 Framework of barriers to using CDSSs
Full size table

Knowledge-related barriers

Knowledge regarding the (specific functions of the) CDSS

Lack of knowledge regarding how the CDSS works appeared to be a barrier to using it. Both insufficient knowledge regarding the basic functions of the CDSS (i.e. alerts and feedback) as well as regarding options to adapt the decision support to personal preferences were mentioned as barriers among the PCPs (see Table 3). Also, most of the PCPs reported never to have received a formal introduction of the system and/or education or training on how to use the CDSS.

Table 3 Examples of perceived barriers related to knowledge regarding the (specific functions of the) CDSS
Full size table

Barriers related to the evaluation of the features of the CDSS

Source and content of the CDSS

PCPs mentioned that the reliability of the source of the content (the initiator of the alerts) was a barrier to using it. Some users mentioned that they questioned whether the pharmaceutical industry was involved in determining the therapeutic recommendations (see Table 4). The content of the decision support was also perceived as a barrier among the PCPs: they sometimes doubted the currentness and therefore the reliability of the content as they believed that it might take some time before a revised guideline is updated in the system. Also, the PCPs agreed that the alert content was not always consistent with the varying needs of different user groups (GPs, GP trainees and PNs), nor with the varying needs of individual users across time (see Table 4).

Table 4 Examples of perceived barriers related to the source and content of the CDSS
Full size table

Format/lay out of the CDSS content

The format or lay out of the content of the CDSS was also mentioned as a barrier (see Table 5). With respect to the notification method of the alert both too intrusive alerts (e.g. pop-ups in the middle of the screen) as well as uninformative alerts (e.g. just a small green button with the text alert on it) were considered as barriers. PCPs also mentioned the readability of the alert text, which they often considered too verbose.

Table 5 Examples of perceived barriers related to the format/layout of the CDSS content
Full size table

Functionality of the CDSS

The participants indicated that the functionality of the CDSS was also perceived as a barrier to using it (see Table 6). First of all, the responsiveness of the system was mentioned to be a problem with the loading of alerts sometimes taking too long. Also, the intensity of alerts (low threshold for triggering alerts) was mentioned to be a barrier. Participants felt that the frequency of alerts, particularly with respect to patient data registration, was too high. In addition, a perceived lack of flexibility of the system in terms of being able to adapt the content of the decision support to personal preferences was perceived as a barrier to using a CDSS. Also, a lack of learning capacity of the system was indicated to be a barrier with the system using only fixed rules rather than learning from the PCPs' use of the system and adjusting the content accordingly.

Table 6 Examples of perceived barriers related to the functionality of the CDSS
Full size table

External barriers interacting with the CDSS

Patient-related factors

Patient-related factors were also perceived as barriers to using CDSSs. Many PCPs mentioned that using CDSSs has a negative effect on patient communication during consultation and is considered as a barrier to using them. Also, the discrepancy between the patient’s reason for visiting the PCP and the content of the alert was a reason not to use it (see Table 7).

Table 7 Examples of perceived barriers related to patient factors
Full size table

Environmental factors

Environmental factors were also commonly reported as a barrier to using CDSSs. PCPs mentioned the limited time during consultation, which makes it difficult to use the CDSS, as well as the additional work it requires to use the CDSS (see Table 8). In addition, the PCPs reported that CDSSs were often not linked to other systems with follow-up actions such as the electronic prescribing system or test ordering forms, which makes it complex to use them. Moreover, some PCPs indicated that they had concerns regarding the misuse of data (patient data and medical practice) by third parties (i.e. health inspectorate or health insurance companies).

Table 8 Examples of perceived barriers related to environmental factors
Full size table

Discussion

In this focus group study we identified the barriers to using large-scale implemented CDSSs covering multiple domains of care in general practice as perceived by Dutch PCPs. We found that three groups of barriers may hinder implementation: knowledge-related barriers, barriers related to the evaluation of the features of the CDSS and external barriers interacting with the CDSS. Particularly, insufficient knowledge regarding the CDSS and barriers related to a lack of integration into daily practice such as too high intensity of alerts, irrelevant alerts, and the additional time it requires to use the CDSS were commonly perceived barriers. Aside from a proper introduction of the system to the target group, our results indicate that improving the flexibility and learning capacity of the systems and thereby increasing options to adapt the decision support to meet the varying needs of different users, should be the target of improved CDSS interventions.

Prior studies on barriers to using CDSSs found that particularly external factors (i.e. patient-related factors and environmental factors) were perceived as barriers among users [4, 17–21] Consistent with these studies we found that external barriers were indeed relevant. However, we also identified commonly perceived knowledge-related barriers. These may particularly apply to large-scale implemented CDSSs, as it is more difficult to introduce a system among a larger sample of the target group. We assume that participants in our sample were somewhat self-selected with respect to attitude to and experience with working with ICT systems. However, even among these early adopters the mere dissemination of an innovation did not seem to be sufficient, emphasizing the importance of complementing dissemination with other strategies [36]. A proper introduction of the CDSS among its target users therefore seems a first important step to improve CDSS usage.

Within the group of barriers related to the evaluation of the features of the CDSS itself, we found that a high intensity of alerts, irrelevant alerts and a lack of flexibility and learning capacity of the CDSS were commonly reported. These barriers are all related to a perceived mismatch of the content of the CDSS with the varying needs of different users or user groups. They may particularly apply in a primary care setting in which different types of PCPs work with the same CDSS covering multiple disease areas of care and multiple types of alerts. Information needs [37] and therefore the perceived usefulness or relevance of alerts vary greatly across professions and even within one group of healthcare providers [4]. Improving the flexibility and learning capacity of CDSSs and thereby increasing options to tailor the decision support to the varying needs of users seems therefore particularly important.

A perceived too high intensity or irrelevant alerts may result in ‘alert fatigue’ and thereby ignoring of alerts, a phenomenon which has been discussed before [9, 38]. Irrelevant alerts, alerts that are not serious or that are repeatedly shown are the most common reasons to override them [39] and it is estimated that as much as 96 % of the alerts are overridden [39–41]. One way to deal with this alert fatigue is to demand reasons from clinicians before they can over-ride or by pass the reminders [9]. Using these so called ‘highly-insistent alerts’, however, can also be frustrating to clinicians. PCPs in our study rather felt that increased user personalization options are needed to ensure that the advice presented is relevant and useful at that time and thus may prevent alert fatigue. Even though the personalization functions of NHGDoc are quite unique, users still felt that these could be increased, for example by being able to set the threshold for alerts themselves. In addition, adaptive systems that learn from their PCPs’ use of the system and adapt the content accordingly, could be helpful.

Consistent with other studies we found that external barriers were also important barriers [4, 17–21]. PCPs indicated that they often do not feel comfortable spending much time on the computer during consultation. To deal with this patient-related barrier, it may be helpful to adapt the decision support to the needs of patients and to involve patients in using the decision support system during consultation. A recent review [9] showed that systems that involve both practitioner and patient are at least more effective in terms of improving quality of care. Another way to deal with this barrier, that also partly addresses the perceived too limited time and extra work it requires to work with the CDSS, is to rearrange patient consultation. In the Dutch setting, as in most other settings, in principal there is no time scheduled for working with CDSSs. More preparation time before actual patient consultation, could be helpful in optimally using the CDSS, while at the same time focusing more at the patient during actual consultation. Whether one of these approaches is feasible and effective needs to be further explored.

Our study has several limitations. First, this focus group study represents the opinions of only a small sample of the target group, questioning the generalizability of our findings [42]. However, our sample corresponds quite well in terms of basic characteristics to the total population of PCPs. Where older GPs were initially somewhat overrepresented in our study, we complemented our sample by adding some GP trainees to the focus groups. Therefore, we assume to have described substantial variation in perceived barriers to using CDSSs in primary care. Moreover, the aim of this study was to identify all possible barriers qualitatively rather than quantifying their relative importance. The framework of barriers to using CDSSs that emerged can be used as input for quantitative studies assessing the relative importance of the barriers.

Results of our study can be useful in designing or adapting CDSSs that are tailored to the identified barriers of their specific user groups in primary care and thus are potentially more effective in improving quality of care than CDSSs using non-user-centered designs [43–45]. However, we explored perceived barriers rather than actual barriers, which follow from PCPs’ perceptions of the situation. These may not accurately reflect the (whole range of) barriers. Whether they are actual or perceived may directly affect the effectiveness of interventions to address these barriers. Complementing this type of research with more objective types of research seems therefore useful. Future trials should directly compare effects of characteristics of CDSSs [9], such as the format and lay-out of a CDSS or certain functionalities of CDSSs.

Conclusions

Multiple-domain covering CDSSs implemented at a large scale elicit a diverse range of barriers among the target group, of which some of them may be more prominent compared to studies focusing on specialized CDSSs within small-scale academic settings. Lack of knowledge about the system is an important barrier, emphasizing the importance of a proper introduction of the system, particularly when implemented at a larger scale. Additionally, barriers related to a perceived mismatch of the content of the CDSS with the varying needs of users or user groups may be particularly relevant in a primary care setting in which CDSSs cover multiple disease areas and target multiple groups of users. Designing flexible and adaptive systems that can be used to provide decision support tailored to the needs of different users and user groups should therefore be the main target of CDSS interventions.

Abbreviations

CDSS:

Computerized decision support system

PCP:

Primary care practitioner

GP:

General practitioner

GP trainee:

General practitioner in training

PN:

Practice nurse

DCGP:

Dutch College of General Practitioners (NHG in Dutch)

EHRS:

Electronic health record system

ICPC codes:

International Classification of Primary Care codes

NHG lab codes:

Codes used by the Nederlands Huisartsen Genootschap (DCGP in English) to classify laboratory and other diagnostic tests and results

ATC codes:

Anatomical Therapeutic Chemical Classification System codes

References

  1. Hunt D, Haynes R, Hanna S, Smith K. Effects of computer-based clinical decision support systems on physician performance and patient outcomes: a systematic review. JAMA. 1998;280:1339–46.

    Article  CAS  PubMed  Google Scholar 

  2. Committee on Quality of Health Care in America Institute of Medicine. To err is human: building a safer health system. Washington DC: National Academies Press; 2000.

  3. Committee on Quality of Health Care in America Institute of Medicine. Crossing the quality chasm: a new health system for the 21st century. Washington DC: National Academies Press; 2001.

  4. Kortteisto T, Komulainen J, Makela M, Kunnamo I, Kaila M. Clinical decision support must be useful, functional is not enough: a qualitative study of computer-based clinical decision support in primary care. BMC Health Serv Res. 2012;12:349.

    Article  PubMed  PubMed Central  Google Scholar 

  5. Chaudhry B, Wang J, Wu S, Maglione M, Mojica W, Roth E, et al. Systematic review: impact of health information technology on quality, efficiency, and costs of medical care. Ann Intern Med. 2006;144:742–52.

    Article  PubMed  Google Scholar 

  6. Johnston M, Langton K, Haynes R, Mathieu A. Effects of computer-based clinical decision support systems on clinician performance and patient outcome: a critical appraisal of research. Ann Intern Med. 1994;120:135–42.

    Article  CAS  PubMed  Google Scholar 

  7. Garg A, Adhikari N, McDonald H, Rosas-Arellano M, Devereaux P, Beyene J, et al. Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review. JAMA. 2005;293:1223–38.

    Article  CAS  PubMed  Google Scholar 

  8. Kawamoto K, Houlihan C, Balas E, Lobach D. Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success. BMJ. 2005;330:765.

    Article  PubMed  PubMed Central  Google Scholar 

  9. Roshanov PS, Fernandes N, Wilczynski JM, Hemens BJ, You JJ, Handler SM et al. Features of effective computerised clinical decision support systems: meta-regression of 162 randomised trials. BMJ. 2013;346. doi:10.1136/bmj.f657.

  10. Moja L, Kwag KH, Lytras T, Bertizzolo L, Brandt L, Pecoraro V, et al. Effectiveness of computerized decision support systems linked to electronic health records: a systematic review and meta-analysis. Am J Public Health. 2014;104:e12–22. doi:10.2105/ajph.2014.302164.

    Article  PubMed  PubMed Central  Google Scholar 

  11. Roshanov P, Misra S, Gerstein H, Garg A, Sebaldt R, Mackay J, et al. Computerized clinical decision support systems for chronic disease management: A decision-maker-researcher partnership systematic review. Implement Sci. 2011;6:92.

    Article  PubMed  PubMed Central  Google Scholar 

  12. Young J, Donahue M, Farquhar M, Simpson C, Rocker G. Using opioids to treat dyspnea in advanced COPD: Attitudes and experiences of family physicians and respiratory therapists. Can Fam Physician. 2012;58:e401–e7.

    PubMed  PubMed Central  Google Scholar 

  13. Nieuwlaat R, Connolly S, Mackay J, Weise-Kelly L, Navarro T, Wilczynski N, et al. Computerized clinical decision support systems for therapeutic drug monitoring and dosing: A decision-maker-researcher partnership systematic review. Implement Sci. 2011;6:90.

    Article  PubMed  PubMed Central  Google Scholar 

  14. Sahota N, Lloyd R, Ramakrishna A, Mackay J, Prorok J, Weise-Kelly L, et al. Computerized clinical decision support systems for acute care management: A decision-maker-researcher partnership systematic review of effects on process of care and patient outcomes. Implement Sci. 2011;6:91.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Souza N, Sebaldt R, Mackay J, Prorok J, Weise-Kelly L, Navarro T, et al. Computerized clinical decision support systems for primary preventive care: A decision-maker-researcher partnership systematic review of effects on process of care and patient outcomes. Implement Sci. 2011;6:87.

    Article  PubMed  PubMed Central  Google Scholar 

  16. Haynes R, Wilczynski N, Team CCDSSSR. Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: methods of a decision-maker-researcher partnership systematic review. Implement Sci. 2010;5:12.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Martens JD, van der Weijden T, Winkens RAG, Kester ADM, Geerts PJH, Evers SMAA, et al. Feasibility and acceptability of a computerised system with automated reminders for prescribing behaviour in primary care. Int J Med Inform. 2008;77:199–207.

    Article  CAS  PubMed  Google Scholar 

  18. Sittig D, Krall M, Dykstra R, Russell A, Chin H. A survey of factors affecting clinician acceptance of clinical decision support. BMC Med Inform Decis Mak. 2006;6:6.

    Article  PubMed  PubMed Central  Google Scholar 

  19. Varonen H, Kortteisto T, Kaila M. Group ftES. What may help or hinder the implementation of computerized decision support systems (CDSSs): a focus group study with physicians. Fam Pract. 2008;25:162–7. doi:10.1093/fampra/cmn020.

    Article  PubMed  Google Scholar 

  20. Rousseau N, McColl E, Newton J, Grimshaw J, Eccles M. Practice based, longitudinal, qualitative interview study of computerised evidence based guidelines in primary care. BMJ. 2003;326:314.

    Article  PubMed  PubMed Central  Google Scholar 

  21. Zheng K, Padman R, Johnson MP, Diamond HS. Understanding technology adoption in clinical care: Clinician adoption behavior of a point-of-care reminder system. Int J Med Inform. 2005;74:535–43.

    Article  PubMed  Google Scholar 

  22. De Vries A, Van der Wal M, Nieuwenhuis M, De Jong R, Van Dijk R, Jaarsma T, et al. Perceived barriers of heart failure nurses and cardiologists in using clinical decision support systems in the treatment of heart failure patients. BMC Med Inform Decis Mak. 2013;13:54.

    Article  PubMed  PubMed Central  Google Scholar 

  23. Bryan C, Boren SA. The use and effectiveness of electronic clinical decision support tools in the ambulatory/primary care setting: a systematic review of the literature. Inform Prim Care. 2008;16:79–91.

    PubMed  Google Scholar 

  24. Van Hassel DTP, Kenens RJ. Cijfers uit de registratie van huisartsen - peiling 2012. NIVEL: Utrecht; 2013.

    Google Scholar 

  25. Prins F, Sluijsmans DA, Kirschner P. Feedback for General Practitioners in Training: Quality, Styles, and Preferences. Adv Health Sci Educ Theory Pract. 2006;11:289–303. doi:10.1007/s10459-005-3250-z.

    Article  PubMed  Google Scholar 

  26. Heiligers PJM, Noordman J, Korevaar JC, Dorsman S, Hingstman L, Van Dulmen AM et al. Praktijkondersteuners in de huisartspraktijk (POH’s), klaar voor de toekomst? Utrecht: NIVEL; 2012.

  27. Nederlands Huisartsen Genootschap (NHG). www.nhg.org/ (2015). Accessed March 13, 2015.

  28. ExpertDoc. www.expertdoc.nl (2015). Accessed March 13, 2015.

  29. Krueger RA, Casey MA. Focus groups: a practical guide for applied research. Thousand Oaks, CA: Sage; 2000.

    Book  Google Scholar 

  30. Murphy E, Mattson B. Qualitative research and family practice: a marriage made in heaven? Fam Pract. 1992;9:85–91. doi:10.1093/fampra/9.1.85.

    Article  CAS  PubMed  Google Scholar 

  31. Mays N, Pope C. Qualitative research: rigour and qualitative research. BMJ. 1995;311:109–12.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  32. Mays N, Pope C. Qualitative research in health care: assessing quality in qualitative research. BMJ. 2000;320:50–2. doi:10.1136/bmj.320.7226.50.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  33. Lugtenberg M, Westert G, Pasveer D, van der Weijden T, Kool R. Evaluating the uptake and effects of the computerized decision support system NHGDoc on quality of primary care: protocol for a large-scale cluster randomized controlled trial. Implem Sci. 2014;9:145.

    Article  Google Scholar 

  34. Clark JP. How to peer review a qualitative manuscript. In: Godlee FJT, editor. Peer Review in Health Sciences. 2nd ed. London: BMJ Books; 2003. p. 219–35.

    Google Scholar 

  35. Cabana MD, Rand CS, Powe NR, Wu AW, Wilson MH, Abboud PA, et al. Why don't physicians follow clinical practice guidelines: a framework for improvement. JAMA. 1999;282:1458–65.

    Article  CAS  PubMed  Google Scholar 

  36. Grol R, Wensing M. Effective implementation: a model. In: Grol R, Wensing M, Eccles M, editors. Improving patient care: the implementation of change in clinical practice. Oxford: Elsevier; 2005.

    Google Scholar 

  37. Goossens A, Bossuyt PMM, de Haan RJ. Physicians and Nurses Focus on Different Aspects of Guidelines When Deciding Whether to Adopt Them: An Application of Conjoint Analysis. Med Decis Making. 2008;28:138–45. doi:10.1177/0272989x07308749.

    Article  PubMed  Google Scholar 

  38. Kesselheim AS, Cresswell K, Phansalkar S, Bates DW, Sheikh A. Clinical Decision Support Systems Could Be Modified To Reduce ‘Alert Fatigue’ While Still Minimizing The Risk Of Litigation. Health Affairs. 2011;30:2310–7. doi:10.1377/hlthaff.2010.1111.

    Article  PubMed  Google Scholar 

  39. Van der Sijs H, Aarts J, Vulto A, Berg M. Overriding of Drug Safety Alerts in Computerized Physician Order Entry. J Am Med Inform Ass. 2006;13:138–47. doi:10.1197/jamia.M1809.

    Article  Google Scholar 

  40. Isaac T, Weissman JS, Davis RB, Massagli M, Cyrulik A, Sands DZ. Overrides of medication alerts in ambulatory care. Arch Intern Med. 2009;169:305–11. doi:10.1001/archinternmed.2008.551.

  41. Weingart SN, Toth M, Sands DZ, Aronson MD, Davis RB, Phillips RS. Physicians' decisions to override computerized drug alerts in primary care. Arch Intern Med. 2003;163:2625–31. doi:10.1001/archinte.163.21.2625.

    Article  PubMed  Google Scholar 

  42. Sim J. Collecting and analysing qualitative data: issues raised by the focus group. J Adv Nurs. 1998;28:345–52. doi:10.1046/j.1365-2648.1998.00692.x.

    Article  CAS  PubMed  Google Scholar 

  43. Baker R, Camosso-Stefinovic J, Gillies C, Shaw E, Cheater F, Flottorp S, et al. Tailored interventions to overcome identified barriers to change: effects on professional practice and health care outcomes. 2010.

    Google Scholar 

  44. Bosch M, Van der Weijden T, Wensing M, Grol R. Tailoring quality improvement interventions to identified barriers: a multiple case analysis. J Eval Clin Pract. 2007;13:161–8.

    Article  PubMed  Google Scholar 

  45. Horsky J, Schiff GD, Johnston D, Mercincavage L, Bell D, Middleton B. Interface design principles for usable decision support: A targeted review of best practices for clinical prescribing interventions. J Biomed Inform. 2012;45:1202–16.

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

The authors thank Margriet Bouma of the Dutch College of General Practitioners (NHG) for facilitating and conducting the focus group sessions. We thank ExpertDoc BV, in particular Rien Dorland, for providing relevant information regarding the (functions of the) CDSS NHGDoc. In addition, we thank Gijs Hesselink for providing advice on qualitative data analyses.

We thank the NHG for providing financial support for this study. The funders had no role in study design, data collection and proposed analysis, preparation of the manuscript or decision to submit the manuscript for publication.

Author information

Authors and Affiliations

  1. Scientific Institute for Quality of Healthcare (IQ healthcare), Radboud university medical center, P.O. Box 9101, 6500 HB, Nijmegen, The Netherlands

    Marjolein Lugtenberg, Jan-Willem Weenink, Gert P. Westert & Rudolf B. Kool

  2. Scientific center for care and welfare (Tranzo), Tilburg School of Social and Behavioral Sciences, Tilburg University, P.O. Box 90153, 5000 LE, Tilburg, The Netherlands

    Marjolein Lugtenberg

  3. School for Public Health and Primary Care (CAPHRI), Department of General Practice, Maastricht University, P.O. Box 616, 6200 MD, Maastricht, The Netherlands

    Trudy van der Weijden

Authors
  1. Marjolein Lugtenberg
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jan-Willem Weenink
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Trudy van der Weijden
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Gert P. Westert
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Rudolf B. Kool
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Marjolein Lugtenberg.

Additional information

Competing interests

All authors declare that they have no competing interests.

Authors’ contributions

ML drafted and revised the manuscript, has been involved in designing and conducting the focus group study, and in analysing and interpreting the data. JWW participated in analysing the data and in revising the manuscript. TvdW and GPW have been involved in designing the study and revising the manuscript. RBK participated in conducting the focus group study and in revising the manuscript. All authors have read and approved the final manuscript.

Authors’ information

Not applicable.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and Permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lugtenberg, M., Weenink, JW., van der Weijden, T. et al. Implementation of multiple-domain covering computerized decision support systems in primary care: a focus group study on perceived barriers. BMC Med Inform Decis Mak 15, 82 (2015). https://doi.org/10.1186/s12911-015-0205-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12911-015-0205-z

Keywords

BMC Medical Informatics and Decision Making

ISSN: 1472-6947

Contact us