Skip to main content
  • Research article
  • Open access
  • Published:

Criteria for assessing the quality of clinical practice guidelines in paediatrics and neonatology: a mixed-method study

A Correction to this article was published on 05 October 2021

This article has been updated



Evidenced-based practice is a key component of quality care. This study aims to explore users’ expectations concerning paediatric local clinical practice guidelines.


A mixed method approach was applied, including material from quantitative questionnaire and semi-structured interviews. Data were analysed using descriptive statistics and qualitative content analysis. Data were analysed with constant comparative method. Qualitative data were parsed and categorized to identify themes related to decision-making.


A total of 83 physicians answered the survey (response rate 83%). 98% of the participants wanted protocols based on international guidelines, 80% expected a therapeutic content. 24 semi-structured interviews were conducted to understand implementation processes, barriers and facilitators. Qualitative analysis revealed 5 emerging themes: improvement of local clinical practice guidelines, patterns of usage, reasons for non-implementation, alternative sources and perspectives.


Some criteria should be considered for the redaction of local clinical practice guidelines: focus on therapeutic, ease of access, establish local clinical practice guidelines based on international guidelines adapted to the local setting, document references and include trainees such as residents in the redaction.

Peer Review reports


Evidence-based medicine and practice are increasingly required by professional associations [1]. In addition, introduction of quality approach in the hospital is now part of national strategies for healthcare. It aims at standardisation of practices to reduce the risk of error and the cost of care [2]. Several hospitals in the world already have ISO 9001:2000 certifications which consist in generic standards that define requirements and guidelines for quality management systems [3]. In this current context, more and more clinical guidelines are being developed [1].

Clinical practice guidelines (CPGs) are intended to improve patient management and care [4] with the primary goal of improving patient health [5]. CPGs help to ensure that patients and physicians are provided with current and robust empirical evidence to adequately inform their decisions and guide their clinical practices.

However, these guidelines need to be adjusted locally according to local preferences and resources, while maintaining high-quality evidence-based recommendations [6].

The ADAPTE Working Group (mainly oncologists) published in 2006 a literature review and proposed a protocol to implement and adapt clinical guidelines locally. The objective was the transposition of guidelines from one cultural and organizational setting to another (the authors call it “trans-contextual adaptation”). This high-quality protocol was developed to decrease workload and inefficient use of resources when adapting guidelines, as opposed to a guideline made from scratch [7].

In spite of all these efforts, recommendations described in guidelines are not necessarily followed [1], for example Grol et al. found in an observational study that only 61% of general practitioners followed guidelines for relevant decisions [8].

Many authors have questioned the factors involved in the implementation of guidelines, such as Francke et al. in a meta-analysis of twelve systematic reviews [1]. However, few studies have considered these issues in the case of local CPGs. The originality of our study is the use of a mixed methodology including a quantitative approach combined with a qualitative method, and also the fact that we directly questioned the point of view of the guidelines users.

The main objective of this study is to explore what physicians expect from local CPGs in order to identify a set of criteria to assist in the development of these guidelines for paediatric and neonatology departments and to achieve better implementation. The secondary objective is to determine if the characteristics of professionals are related to these expectations and uses of local CPGs.


Study context

This study was conducted in the University hospital of Rennes and in the local hospital of Fougères. The paediatric department of the University hospital proposes two pocketbooks of local CPGs to all paediatricians (residents and attending physicians including fellows) working in the institution and in the local hospital. Each local CPG is written by a physician and then is peer-reviewed (one or several peers). Management of these local CPGs depends on one physician and is different for each sub-department (emergency, neonatology and paediatric intensive care unit). These local CPGs are available in 3 formats: on paper, on a mobile health application incorporated into smartphones and on the hospital intranet for some guidelines. Their structure is dictated by the quality department of the hospital as follows: object and purpose, responsibilities, definitions, actions and methods (diagnosis, therapy), references.


This study used a mixed-methods design with a sequential design: quantitative data collection first and then qualitative study. This sequence allows to interpret results derived from large-sample data and to better understand the findings from the first phase [9]. We followed COREQ guidelines, a 32-item checklist for explicit and comprehensive reporting of qualitative studies [10].


There are one hundred paediatricians (residents and attending physicians) in the University hospital and the local hospital and they were all invited via e-mail to participate in the study. French medical residents complete 6 years in medical school and then choose a specialty, the residency lasts between 6 and 10 semesters depending on the specialties. Residents are certified between the middle and the end of residency. They work full time at a hospital and they have a relative autonomy: they can do prescriptions under the responsibility of an attending physician.

Paediatric surgeons, geneticists, paediatric anaesthesiologists and paediatricians who only work as consultants within our hospital system and, who do not use the local CPGs, were not included in the study.

Quantitative study


Themes identified from a literature review were used to develop our 16-item survey. All participants (n = 100) received this survey online or in hard copy and responded anonymously. Physicians are usually in high and constant demand and to favour a high response rate, we elaborated a quick survey that can be completed in a maximum of ten minutes. Items included a 5 point-Likert scale for answering. This part of the study was conducted between January and March 2019.

Statistical analysis

Demographic data including sex, status, role in local CPG development, assignment, semester of training (for residents only) were summarized and compared using the Fisher exact test.

Continuous variable (age and duration of use) and coded Likert scales (− 2: strongly disagree and 2: strongly agree) were expressed as medians with interquartile ranges [25p, 75p] and compared with the Wilcoxon rank sum test. We chose an odd-numbered Likert scale of 5 points in order to offer a neutral mid-point. This avoids a forced choice and therefore a random response of one of the two points framing the omitted mid-point [11].

To identify groups of similar questions, we used a hierarchical clustering analysis on the Euclidean distance matrix formed by participant responses to the Likert-type questions. This method uses an agglomerative approach based on dissimilarity (Euclidean distance) measure between each pair of answers. We supposed an equal distance between scale points. We measured within-cluster similarity using the average silhouette width. Then, we selected k = 15 clusters of questions because of their high internal consistency based on a Cronbach’s α above 0.7 [12].

P < 0.05 was considered statistically significant for all the analysis. Data were analysed using the software R Core Team® (2018).

Qualitative study

Setting and sample

Participants who had taken part in the survey were eligible to participate in the semi-structured interviews on a voluntary basis. The inclusion criteria for interview participants were based on achieving a maximum variation in our sample regarding age, professional experience and assignment. Researchers who conducted the interviews informed participants, with a letter and then face-to-face, about the aim of the study and their right to withdraw their participation at any time without giving any reason.

Data collection

Semi-structured interviews were conducted in French by two paediatric residents (ID and JD). Data collection spanned from May 2019 to February 2020. Each interviewer carried out repeated interviews with researchers specialised in qualitative method. These interviews were not included in the data collection. Sessions were semi-structured, with a pre-defined list of open-ended questions focused first on the current uses of the local CPGs and then on desired improvements, ease of use and open-ended suggestions. The discussion guide was developed by all authors (NG, ID, JD, NM, GC, PP and AB) after a review of the literature before starting the study. Interviews were conducted in person at a private office space located in the participants’ hospital workplace. Participants were contacted through email or face-to-face.

To ensure reliability, we used the same interview guide in every interview. Sessions were recorded with the consent of each participant and then transcribed verbatim and de-identified. The interviewers knew most of the participants prior to the study as they work in the same paediatric department but they had no hierarchical superiority with the responder and this was not a personal matter. The personal goals and reasons for the research were revealed to the participants.

All through the interview, the moderator summarised and reformulated the speech and presented them back to the participant to avoid researcher’s interpretation and respect the participants’ own points. After the session, participants completed a brief survey to gain their socio-demographic characteristics.

Data analysis

The analysis procedure was conducted by two persons (JD and NM) using an inductive approach to identify themes that emerged from the data. Each transcript was independently read several times to facilitate immersion in the data. Following this, the researchers used open coding process to summarise participants’ views by assigning words to quotes or paragraphs. The coding of the two researchers were then compared and in the event of any discrepancies or a disagreement, two other physicians (NG and AB) adjudicated. This method enhances the validity of the assigned categories and attempts to reduce researcher bias. A catalogue of themes and sub-themes was then created and presented in tabular form. Constant comparative analysis was used to assess overall saturation [13]. Authors collectively selected and presented verbatim quotes to illustrate the thematic findings in tabular form. We coded data from transcripts using the Saldaña method [14] and evaluated the frequency of each theme using the qualitative data management software NVivo® 12 Plus (QSR International). To ensure the reliability of the coding and analysis of the data, findings were discussed among the authors. Data were then analysed based on the participants’ status and assignment. Still using NVivo® 12 Plus (QSR International), we imported participants’ demographic characteristics and compared their responses according to these attributes.

In the last phase of the analysis, data from the quantitative survey and the interviews were interpreted together. The interviews were used to explore and complete the quantitative data or to find differences between the two datasets.

Ethical considerations

The study was approved by the local Ethics Committee (reference number 19.147). All participants gave their informed consent before participating. A physician was responsible for explaining the research project to potential participants and to send an email newsletter about the purpose and conduct of the research.


Phase 1: survey

A total of 83 physicians participated in the study (response rate 83%); 42.2% (n = 35) were residents and 57.8% (n = 48) were attending physicians. The median age was 36.29 years [25; 64] and 81.9% were women. Few participants were writer of local CPGs (7.23%). The median duration of use of local CPGs was 4 years [2; 11]. Characteristics of the research participants are summarised in Table 1. A Fisher test (p > 0.3) did not show any significant repartition between responders and non-responders for principal assignment of physicians.

Table 1 Characteristics of quantitative questionnaires participants

The majority of participants (97%) reported using local CPGs sometimes on hard copy, 42% sometimes on intranet and 48% sometimes on smartphones. The existing local CPGs were considered diagnostic-oriented for 34% of the physicians and therapeutic-oriented for 82% of the physicians. About a quarter of the participants (28%) expressed the wish for more local CPGs for diagnosis purposes, and almost half (53%) expressed this wish for therapeutic purposes. Almost all participants (98%) wanted references based on international guidelines and 69% explicit references in local CPGs. Answers were not polarised for the question “a good guideline should be based on local guidelines or usual practice in unit”. Few participants (18%) felt that a good guideline should be based on physician experience. This cluster of questions showed a high internal consistency (α = 0.8 [0.72; 0.88]). The main objective of a local CPG should be therapeutic according to 80% of the participants and 70% used local CPGs to write prescription. Only 12% of the participants used local CPGs to make a diagnosis. About three-quarters of respondents (77%) reported using only some parts of the guideline they found interesting and not the entire guideline. For the question “I do not (always) implement the local CPG” answers were not polarised. However, there was a high internal consistency (α = 0.73, 95% CI [0.62–0.85]).

Physicians’ awareness of the grade of evidence was found to be mixed and answers were not polarised for this question. Almost half of the respondents (47%) considered grade of evidence when using and/or reading local CPGs. There was a high internal consistency in answers for these two questions (α = 0.78–95% CI [0.69–0.87]).

The use of local CPGs was mainly reported in those working in the emergency ward (60%). A high percentage of participants were interested in receiving notification when local CPGs were updated (82%) and were in favour of having local CPGs reviewed and approved by both a resident and an attending physician (61.5%).

Physicians mostly (78%) read just the part of the local CPG they were interested in and only 28% read the whole local CPG (α = 0.5–95 CI [0.28–0.71]).

Concerning the clarity of information contained in local CPGs, almost all the responders (97%–95 CI [90–99%]) considered decision trees as the best method for relaying the information and 81% thought they were better than text (α = 0.48–95 CI [0.28–0.69]). With regard to the classification of local CPGs, 23% of participants would like to see them classified by degree of severity and urgency and 50% did not express an opinion on this point. Finally, more than half of physicians (69%) had already searched for a protocol that did not exist.

Hierarchical cluster analysis showed that 8 out of 15 clusters had a Cronbach’s alpha above 0,5 and that 4 clusters had this value above 0,7. Internal consistencies were stronger in the following clusters: local CPGs supported by local guidelines, medical unit habits and physician experience.

Phase 2: qualitative study

Twenty-four physicians participated in the interviews. Almost all of them (n = 22) were women, 19 used local CPGs very often and 5 used it maximum once a week or less. “Very often” was defined by responders themselves as "daily" or "several times a week". Table 2 shows the characteristics of participants. Each interview lasted between 12 and 24 min. Data saturation was obtained because we did not find a new theme after several analyses of the data. Analysis revealed five emergent themes: improvement of local CPGs, patterns of use, reasons for non-implementation, alternative sources and perspectives (Table 3). In the second part, we analysed qualitative data based on participants’ status and assignment.

Table 2 Characteristics of semi-structured interviews participants (n = 24)
Table 3 Emerging themes with subthemes and exemplar quotations

Participants’ status and assignment

We analysed associations between attitudes, such as reason for non-implementation, improvement and pattern of use, toward local CPGs and physicians’ characteristics. No major differences by status or assignment were found in this analyse. In terms of patterns of use, the local CPGs were mainly used for verification and prescription regardless of status or assignment (Fig. 1).

Fig. 1
figure 1

Verbatim analyse depending on the status and assignment


CPGs are designed to improve standardization and quality of care. CPGs allow evidence-based practice to integrate faster routine care [15]. In this study, we explored the expectations of physicians regarding paediatrics local CPGs. Quantitative and qualitative approaches showed that the characteristics of professionals were not related to the expectations and uses of local CPGs. Francke et al. published a meta-analysis to gain a better understanding of the factors affecting guidelines implementation. Age and/or experience were found to be determinants of guidelines use with younger or less experienced professionals being more likely to use guidelines than older and experienced professionals [1]. Our results are not in line with these findings as in our sample, CPGs were widely used by both residents and attending physicians. However, in the qualitative part of the study attending physicians also reported that they sometimes relied on their department common practices.

Participants also expressed a desire to have local CPGs based on international guidelines with high grade of evidence. This is reflected in literature that adherence to evidence-based guidelines appears to be higher for guidelines with a clear scientific basis [16]. In another study, it is mentioned that quality of practice guidelines is directly related to the quality of medical evidence supporting the recommendation [17]. The GRADE system details in four points how to determine the strength of a recommendation [18].

In our study, participants were mainly interested in treatment CPGs. Similar findings were found in a German study which explored perception of guidelines (therapeutic and diagnostic) in a group of primary care physicians. Therapy management was reported as one of the most important aims for this group [19].

Even if participants declared to be mainly satisfied with the available CPGs, improvement of the local CPGs was the major theme identified. The physicians interviewed indicated that local CPGs needed to be better adapted to the local structure and updated as often as possible. However, among physicians, the reported frequency at which local CPGs should be updated was very different. A study provided an empirical estimate of the rate at which guidelines become outdated and it was found that guidelines should be reassessed every 3 years [20]. A cohort of 134 published (National Institute for Health and Care Excellence (NICE) clinical guidelines was analysed and found that 86% of guidelines were still up to date 3 years after their publication and that the median life span from its publication to the date when a decision was made to update was 5 years [21]. The importance of keeping guidelines up-to-date and updating parts of it rather than re-writing the whole guideline was underlined in another study. An ongoing monitoring system should be created by guideline developers to implement a systematic updating procedure depending on the guideline’s topic and the organizational capacity to enable rapid adaptations to current events [22].

The present study also identified the interest in extending local CPG writers to trainees and especially residents to introduce a different perspective. Guidelines developed by target groups and experts have been previously found to increase the chances of successful implementation [1]. Another study suggested that guideline development groups should be well balanced in terms of the selected disciplines to avoid dominance of those who are likely to be perceived to have higher power status [23].

Physicians were found to primarily use local CPGs for verification when prescribing, especially for uncommon diseases. Previous research that has studied the use of guidelines and protocols in a different setting from our study (hospice care) found that their use was mainly for pharmacological treatment [24]. Audi et al., updated a list of the 100 of the most commonly prescribed drugs in England and emphasized the importance of using standardised prescription forms as a tool to improve prescribing skills [25].

The identified barriers to the implementation of local guidelines were the clinical context including situations that differ from the guideline and, again, the lack of updating. Bergman et al. [17] note that guidelines in general are not modified to meet patient and site-specific needs. By collecting local outcome data, physicians are able to customise guidelines through the collection and feedback of this data from their population.

Residents raised the issue of difficulty in understanding CPGs and of sometimes conflicting recommendations from their attending physician. A study mentioned that rewriting guidelines to increase behavioural specificity may be the simplest and most effective method to increase implementation, by providing clearer information about requirements and improving the sense of doing well [26]. Cabana et al. detailed barriers to CPGs adherence including: lack of awareness and familiarity with CPGs; low outcome expectation from CPGs; accessibility; clinical time pressure and limited resources to implement recommendations [15]. Guideline adherence seemed better when reviewed with local leaders. A study proposed a stepwise procedure to increase physicians’ compliance with guidelines by adapting the recommendations to the target setting and context of use, taking into account the organization of the health care system and cultural context [7].

Physicians interviewed in our study used local CPGs also as a pedagogical tool for teaching residents. This finding is in accordance with a study that evaluated the interest of a formulary in drug prescribing and particularly in a medical student population [27]. The results of this study showed that using a formulary increases the competence of medical students in rational prescribing.

Physicians would like to see outdated CPGs more frequently updated or removed. This is confirmed by the Appraisal of Guidelines for Research and Evaluation (AGREE II) collaboration that a procedure for updating guidelines should be provided for a better implementation [28]. This rigor is an ongoing challenge. Actually, lack of time was a barrier often expressed by the local CPGs writers interviewed in our study and it was a reason for delaying updates.

Ciquier et al., emphasized that when implementing CPGs, it should be anticipated that CPGs will need to be appraised, monitored and possibly revised, in order to ensure applicability [29].

Use of electronic format may overcome some of the implementation barriers noted with the use of notifications as well as patient-specific recommendation based upon the integrated data at the time of the patient visit. Physician use and compliance with CPGs is improved when a computerized format is provided [30]. When information is accessible with minimal need for searching, information integration is likely to proceed in a perception-like, holistic manner [31].

Limitations and strengths

Our study was limited in several areas. Some answers were polarised (extreme responses on Likert scales) and some questions had a low internal consistency making it impossible to explore the corresponding answers. We selected 15 clusters of questions using the hierarchical clustering analysis and 5 clusters contained only one question. We could not analyse internal consistency for these 5 clusters and maybe they have a validity anyway. We did not explore the semantic of the local CPGs as our interview guide did not focus on this aspect of the CPGs. The French National Authority for Health has established that clinical practice guidelines should preferably be in electronic formats and take into account modern technological tools. Access should be straight forward lists with bullet points and decision trees should be provided as well as links to access reports and other documents. Another limitation of our study was the study sample comprised only of physicians (paediatricians only). It is not a multidisciplinary study because it does not involve all potential users of the guidelines. However, literature showed that specialty does not appear to be a significant factor in guideline adherence and suggested it is not necessary to target specific groups during development of practice guidelines [32].

Despite these limitations, recurring themes were clearly established, and our population included a wide diversity of profiles. The response rate to the quantitative survey was high, about 80%. Moreover, high internal consistency validates the good quality of the quantitative survey and answers obtained. The majority of users declared using local CPGs very often so there is an interesting level of critical thinking from the panel.

The mixed approach found consistent results and this probably empowers the level of evidence of the study. Quantitative approach highlighted several inadequacies in local CPGs as evidenced by the high number of polarised answers. Qualitative analysis enabled us to gain a deeper understanding and to clarify the answers that were not polarised (response neutral on Likert scale).

The generalizability of our findings is limited by the response rate and the sampling from only two hospitals of the region. Because recruitment was on a voluntary basis it is possible that those who completed the survey had stronger feelings about the topic however it was noted that respondents had similar characteristics to non-respondents.


We explored physicians’ expectations concerning the paediatrics local CPGs.

Our findings highlight that there is still a lot to be done in improving implementation of guidelines, therefore, promoting better outcomes for patients. Using the above-mentioned literature, we can suggest some interesting approaches to consider when developing local CPGs and for increasing their implementation (See Table 4). It is necessary to focus on therapeutic information and to classify items by specialties in order to have a quick access to the searched guideline. When it comes to the redaction of the local CPGs, it is important to justify local CPGs with international guidelines, while adapting them to the local healthcare setting, as well as listing the sources used and promoting double validation of local CPGs with a trainee and an attending physician.

Table 4 Quality criteria for local good practice guidelines

Frequency of updates and classification of guidelines, enabling for an easy and quick search, remain a major challenge in improving guideline adhesion. Local CPGs developers should define a systematic updating procedure before the guideline dissemination. Electronic format and mobile application could be tools to improve these shortcomings as it would enable the use of automated notifications or a keyword search system to provide rapid access to information. Another suggestion would be the existence of a crosstalk between local guidelines and prescribing software with the possibility to alert an outdated prescription.

Availability of data and materials

The data supporting the findings are contained within the manuscript and its Additional files 1, 2, 3: Supplementary. All verbatims of the semi-structured interviews analysed during the current study are available from the corresponding author on reasonable request.

Change history



International Organization for Standardization


Clinical practice guidelines


Consolidated criteria for reporting qualitative research


  1. Francke AL, Smit MC, de Veer AJ, Mistiaen P. Factors influencing the implementation of clinical guidelines for health care professionals: a systematic meta-review. BMC Med Inf Decis Mak. 2008;8(1):38.

    Article  Google Scholar 

  2. Seferdjeli L, Terraneo F. Comprendre le travail de soins à l’hôpital. Recherche en soins infirmiers. 15 avr 2015;N° 120(1):6–22.

  3. Van den Heuvel J, Koning L, Bogers AJJC, Berg M, van Dijen MEM. An ISO 9001 quality management system in a hospital: Bureaucracy or just benefits? Int J Health Care QA. 2005;18(5):361–9.

    Article  Google Scholar 

  4. Recommandations HAS “Good practice guidelines method”.

  5. McCulloch KM, Vidyasagar D. Assessing adherence to standards for neonatal resuscitation taught throughout the perinatal referral area. A quality assurance activity. Pediatr Clin North Am. 1993;40(2):431–8.

    Article  CAS  Google Scholar 

  6. Eccles M, Clapp Z, Grimshaw J, Adams PC, Higgins B, Purves I, et al. North of England evidence based guidelines development project: methods of guideline development. BMJ. 1996;312(7033):760–2.

    Article  CAS  Google Scholar 

  7. Fervers B, Burgers JS, Haugh MC, Latreille J, Mlika-Cabanne N, Paquet L, et al. Adaptation of clinical guidelines: literature review and proposition for a framework and procedure. Int J Qual Health Care. 2006;18(3):167–76.

    Article  Google Scholar 

  8. Grol R, Dalhuijsen J, Thomas S, Veld C, Rutten G, Mokkink H. Attributes of clinical guidelines that influence use of guidelines in general practice: observational study. BMJ. 1998;317(7162):858–61.

    Article  CAS  Google Scholar 

  9. Small ML. How to conduct a mixed methods study: recent trends in a rapidly growing literature. Ann Rev Sociol. 2011;37(1):57–86.

    Article  Google Scholar 

  10. Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care. 2007;19(6):349–57.

    Article  Google Scholar 

  11. Alwin DF, Krosnick JA. The reliability of survey attitude measurement: the influence of question and respondent attributes. Sociol Methods Res. 1991;20(1):139–81.

    Article  Google Scholar 

  12. Thinsungnoena T, Kaoungkub N, Durongdumronchaib P, Kerdprasopb K, Kerdprasopb N. The clustering validity with silhouette and sum of squared errors. Learning. 2015;3(7)

  13. Côté L, Turgeon J. Appraising qualitative research articles in medicine and medical education. Med Teach. 2005;27(1):71–5.

    Article  Google Scholar 

  14. Saldaña J. The coding manual for qualitative researchers. London: Sage; 2015.

    Google Scholar 

  15. Cabana MD, Rand CS, Powe NR, Wu AW, Wilson MH, Abboud P-AC, et al. Why dont physicians follow clinical practice guidelines? A Framework for Improvement. JAMA. 1999;282(15):1458–65.

    Article  CAS  Google Scholar 

  16. Saillour-Glenisson F, Michel P. Individual and collective facilitators of and barriers to the use of clinical practice guidelines by physicians: a literature review. Revue d’épidémiologie et de santé publique. 2003;51(1 Pt 1):65–80.

    CAS  Google Scholar 

  17. Bergman DA. Evidence-based guidelines and critical pathways for quality improvement. Pediatrics. 1999;10.

  18. Guyatt, Gordon H., Andrew D. Oxman, Regina Kunz, Yngve Falck-Ytter, Gunn E. Vist, Alessandro Liberati, et Holger J. Schünemann. Going from evidence to recommendations. BMJ 336, no 7652 (8 mai 2008): 1049–51

  19. Tausch BD, Härter MC. Perceived effectiveness of diagnostic and therapeutic guidelines in primary care quality circles. Int J Qual Health Care. 2001;13(3):239–46.

    Article  CAS  Google Scholar 

  20. Shekelle PG, Ortiz E, Rhodes S, Morton SC, Eccles MP, Grimshaw JM, et al (2001) Validity of the agency for healthcare research and quality clinical practice guidelines: how quickly do guidelines become outdated? JAMA 286(12):1461–7.

  21. Alderson LJH, Alderson P, Tan T. Median life span of a cohort of National Institute for Health and Care Excellence clinical guidelines was about 60 months. J Clin Epidemiol. 2014;67(1):52–5.

    Article  Google Scholar 

  22. Becker M, Neugebauer EAM, Eikermann M. Partial updating of clinical practice guidelines often makes more sense than full updating: a systematic review on methods and the development of an updating procedure. J Clin Epidemiol. 2014;67(1):33–45.

    Article  Google Scholar 

  23. Pagliari C, Grimshaw J. Impact of group structure and process on multidisciplinary evidence-based guideline development: an observational study. J Eval Clin Pract. 2002;8(2):145–53.

    Article  Google Scholar 

  24. Johnson DC, Kassner CT, Kutner JS. Current use of guidelines, protocols, and care pathways for symptom management in hospice. Am J Hosp Palliat Care. 2004;21(1):51–7.

    Article  Google Scholar 

  25. Audi S, Burrage DR, Lonsdale DO, Pontefract S, Coleman JJ, Hitchings AW, et al. The ‘top 100’ drugs and classes in England: an updated ‘starter formulary’ for trainee prescribers. Br J Clin Pharmacol. 2018;84(11):2562–71.

    Article  Google Scholar 

  26. Michie S, Johnston M. Changing clinical behaviour by making guidelines specific. BMJ. 2004;328(7435):343–5.

    Article  Google Scholar 

  27. De Vries T, Daniels JMA, Mulder CW, Groot OA, Wewerinke L, Barnes KI, et al. Should medical students learn to develop a personal formulary? Eur J Clin Pharmacol. 2008;64(6):641–6.

    Article  Google Scholar 

  28. Terrace L. Development and validation of an international appraisal instrument for assessing the quality of clinical practice guidelines: the AGREE project. Qual Saf Health Care. 2003;12(1):18–23.

    Article  Google Scholar 

  29. Ciquier G, Azzi M, Hébert C, Watkins‐Martin K, Drapeau M. Assessing the quality of seven clinical practice guidelines from four professional regulatory bodies in Quebec: What’s the verdict? J Eval Clin Pract

  30. Trivedi MH, Kern JK, Marcee A, Grannemann B, Kleiber B, Bettinger T, et al. Development and implementation of computerized clinical guidelines: barriers and solutions. Methods Inf Med. 2002;41(05):435–42.

    Article  CAS  Google Scholar 

  31. Söllner A, Bröder A, Hilbig BE. Deliberation versus automaticity in decision making: Which presentation format features facilitate automatic decision making? Judgm Decis Mak. 2013;8(3):21.

    Google Scholar 

  32. Cook DA, Pencille LJ, Dupras DM, Linderbaum JA, Pankratz VS, Wilkinson JM. Practice variation and practice guidelines: Attitudes of generalist and specialist physicians, nurse practitioners, and physician assistants. PLoS ONE

Download references


We thank all the physicians at the University Hospital of Rennes and the Hospital of Fougères who participated in the study. We also thank Laurianne Schoenberger for English language editing.


No funding was received.

Author information

Authors and Affiliations



JD participated in the study design, collection and analysis of the data, the writing of the report. ID and NG participated in the study design, collection and analysis of the data. GC and PP participated in the study design and interpretation of the data. AB and NG participated in the study design and analysis of the data through software R Core Team. JD and NM participated in the analysis of the data through N Vivo software. JD, NM, NG, ID and AB had complete access to the study data that support the publication. All authors revised this article critically, approved the final manuscript and agreed to its being submitted for publication.

Corresponding authors

Correspondence to Joanna Dagard or Nadia Mazille-Orfanos.

Ethics declarations

Ethics approval and consent to participate

This study and the consent procedure have received ethics approval from the Rennes University Hospital Ethics Committee (reference number 18.21). Participation was voluntary, the consent was verbal but all participants answered via email or text message to confirm their participation. The participants had the right to withdraw at any time without prejudice. Participation was voluntary, the consent was verbal but all participants answered via email or text message to confirm their participation. The participants had the right to withdraw at any time without prejudice.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

The original version of this article was revised: the names of each the authors were originally published with their family names as their given names

Supplementary Information

Additional file 1.

Hierarchical clustering.

Additional file 2.


Additional file 3.

Consolidated database.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Dagard, J., Mazille-Orfanos, N., Georgi, N. et al. Criteria for assessing the quality of clinical practice guidelines in paediatrics and neonatology: a mixed-method study. BMC Med Inform Decis Mak 21, 269 (2021).

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: