This article has Open Peer Review reports available.
Analysis of clinical uncertainties by health professionals and patients: an example from mental health
© Lloyd et al; licensee BioMed Central Ltd. 2009
Received: 11 December 2008
Accepted: 10 July 2009
Published: 10 July 2009
The first step in practising Evidence Based Medicine (EBM) has been described as translating clinical uncertainty into a structured and focused clinical question that can be used to search the literature to ascertain or refute that uncertainty. In this study we focus on questions about treatments for schizophrenia posed by mental health professionals and patients to gain a deeper understanding about types of questions asked naturally, and whether they can be reformulated into structured and focused clinical questions.
From a survey of uncertainties about the treatment of schizophrenia we describe, categorise and analyse the type of questions asked by mental health professionals and patients about treatment uncertainties for schizophrenia. We explore the value of mapping from an unstructured to a structured framework, test inter-rater reliability for this task, develop a linguistic taxonomy, and cross tabulate that taxonomy with elements of a well structured clinical question.
Few of the 78 Patients and 161 clinicians spontaneously asked well structured queries about treatment uncertainties for schizophrenia. Uncertainties were most commonly about drug treatments (45.3% of clinicians and 41% of patients), psychological therapies (19.9% of clinicians and 9% of patients) or were unclassifiable.(11.8% of clinicians and 16.7% of patients). Few naturally asked questions could be classified using the well structured and focused clinical question format (i.e. PICO format). A simple linguistic taxonomy better described the types of questions people naturally ask.
People do not spontaneously ask well structured clinical questions. Other taxonomies may better capture the nature of questions. However, access to EBM resources is greatly facilitated by framing enquiries in the language of EBM, such as posing queries in PICO format. People do not naturally do this. It may be preferable to identify a way of searching the literature that more closely matches the way people naturally ask questions if access to information about treatments are to be made more broadly available.
When practising evidence-based medicine (EBM), the ability to translate clinical queries into structured and focused clinical questions has been described as the first step of evidence based practice[1, 2]. In this way, it is argued, that answers can be found where evidence exists from systematic reviews and clinical trials. If the clinical question cannot be answered by referring to the evidence base then an uncertainty has been identified. The well built clinical question is seen as the key to finding the research evidence and making evidence based decisions. Teachers of EBM contend that the well structured clinical question contains four elements within it. These are the Patient, the Intervention, the Comparison and the Outcome . Together these elements form the acronym PICO. It is argued that the PICO format makes it relatively straightforward to elicit and combine the appropriate terms needed to find the evidence when searching databases such as PubMed . The PICO framework has been adopted by the U.S. National Library of Medicine, which with its over 16 million citations forming MEDLINE, is the largest and most used web source of abstracts. However, the ability to ask PICO style questions, search for the literature effectively and interpret the search results is not intuitive and requires training . Previous studies have looked at how effectively physicians ask clinical questions . This issue is becoming applicable to the general public, which is increasingly turning to internet resources to address uncertainties they might have about treatments.
In this study we focus on questions about treatments for schizophrenia posed by mental health professionals and patients. The objective was to gain a deeper understanding about the types of the questions which are asked naturally and whether they could be reformulated in a structured PICO framework. Furthermore, we defined a question-taxonomy and evaluated whether certain categories within a taxonomy lend themselves better for formulating well defined (and unambiguous) queries. The ultimate goal is to then devise a system within which to channel a user to pose good queries.
A convenience sample of patients and clinicians recruited from a web-survey, a conference and members of a mental health charity were asked to write down in their own words an uncertainty or question they had about the effect of treatment for schizophrenia. Both the form and the web-survey gave a brief explanation of uncertainty and examples of types of treatment uncertainties No ethical issues were identified in this non experimental, observational study conducted on anonymous responses to a survey. In order to evaluate the presence and the accuracy of the PICO elements present in peoples' responses, two of us (AC and MT) independently rated the questions for the presences of PICO elements. Results were then compared, discrepancies identified and discussed to agree on definitions. We then assessed inter-rater agreement. Every question was analysed and searched for PICO elements (Patients, Intervention, Comparison and Outcome) and once an element was found, appropriate text corresponding to the element was highlighted. We then examined the questions for evidence of a linguistic taxonomy. Questions were clustered according to an emergent taxonomy and associations between these categories and PICO elements were explored. Definitions and examples of the linguistic taxonomy are given in the results section.
Respondents and their questions
Frequency of topics raised by clinicians and patients. Percentage in brackets refers to the within group frequency.
Clinician n (%)
Patient n (%)
Education & training
PICO elements in patient and clinician uncertainties
PICO elements identified as present in the uncertainties asked by patients and clinicians about schizophrenia.
Clinician n (%)
Clinician n (%)
Kappa (k) for Inter-Rater Reliability
(p < 0.05)
(p < 0.05)
(p < 0.05)
(p > 0.05)
(p < 0.05)
(p < 0.05)
(p < 0.05)
(p < 0.05)
Inter-rater agreement on the PICO elements contained in clinicians' questions and separately for patient questions as calculated by Cohen's Kappa coefficient  are also shown in table 2. Kappa coefficients show moderate agreement between the raters for clinician and patient questions.
Identifying a linguistic taxonomy
Linguistic analysis of elements naturally occurring within clinicians, and patients, questions about treatment uncertainties for schizophrenia
The desired answer is to state the similarities and differences between two or more concepts or events.
"Old versus. atypical antipsychotic"
The desired answer explains a mode of action, mechanism or process.
"How to access the best counselling support"
The desired answer is a particular concept or event.
What is the difference between depression and happiness?
The desired answer is a time point.
"When do they use E.C.T. treatment?"
The desired answer explains the cause of a concept or event.
"Why is E.C.T. banned in several countries?"
A query is categorized as a statement, if it is an opinion.
"Too much medication is not regularly reviewed"
Ill defined query
Not categorized in any of the previous response classes.
"Does ECT work?"
Comparing Syntactic classes with well structured clinical questions in PICO format
Relationship between PICO elements and the linguistic categories contained in clinicians' questions (n = 161) about treatment uncertainties for schizophrenia:
Number and Percentage (%) of questions in which PICO element is present
We asked mental health patients and clinicians to identify an uncertainty about the treatment of schizophrenia. Clinicians and patients asked similar types of questions. Few naturally asked structured and focused clinical questions in the PICO format. However, the team was often able to identify PICO elements present in patient and clinicians questions. In many questions those PICO elements were not present making questions hard to classify using the structured clinical question format from evidence based medicine. Other taxonomies may better capture the nature of questions people naturally ask, for example a linguistic taxonomy of the type described here. Access to EBM resources is greatly facilitated by framing enquiries in the language of EBM [9, 10]. People do not naturally do this. It may be preferable to identify a way of searching the literature that more closely matches the way people naturally ask questions if access to information about treatments are to be made more broadly available .
The main limitation of our study is that this is a secondary analysis of data obtained using a convenience sample from one disease area. Our findings may or may not generalise to other patient and clinician groups. However, if anything, we would expect those persons who took the time to reply to our survey to be among the more interested ones in evidence based practice than those who did not. Respondents were deliberately not asked to structure their responses using the PICO format so as to minimise response bias.
It is basic principle of EBM that translating clinical uncertainty into structured and focused clinical questions is a prelude to searching the literature. Much time and effort is spent teaching clinicians to ask questions in this PICO format. Our results support the view that people do not naturally ask questions in a well structured format. Broadening access to EBM resources will either require training everyone, both patients and clinicians, in EBM or providing them with ready digested answers to common uncertainties and queries or ideally by identifying a method of searching the literature which does not require the searcher to be trained in EBM [12, 13]. Such an approach would greatly broaden access to knowledge and information about health care. It remains to be seen whether PICO elements result in better outcomes compared to other question formulations. Natural language queries or specific typologies of question might be more accurate in retrieving relevant.
This study was funded by a grant from the Welsh Office of Research and Development and builds on a collaboration with the James Lind Alliance.
- Geddes J: Asking structured and focused clinical questions: essential first step of evidence based practice. Evidence-Based Mental Health. 1999, 2: 35-36. 10.1136/ebmh.2.2.35.View ArticleGoogle Scholar
- Richardson WS: Ask, and ye shall retrieve. Evidence Based Medicine. 1998, 3: 100-1.Google Scholar
- Richardson S: The well built clinical question. A key to evidence based decisions ACP J Club. 1995, 123: A12-3.PubMedGoogle Scholar
- Bergus GR, Emerson M: Family medicine residents do not ask better. Formulated clinical questions as they advance in their training. Fam Med. 2005, 37 (7): 486-90.PubMedGoogle Scholar
- Sackett DL, Straus SE, Richardson WS, Rosenberg W, Haynes RB: Evidence-based medicine. How to practice and teach EBM. 2000, New York, NY: Churchill Livingston, 2Google Scholar
- Cheng GYT: A study of clinical questions posed by hospital clinicians. J Med Libr Assoc. 2004, 92 (4): 445-458.PubMedPubMed CentralGoogle Scholar
- Lloyd K, Rose D, Fenton M: Identifying uncertainties about the effects of treatments for schizophrenia. Journal of Mental Health. 2006, 15 (3): 263-268. 10.1080/09638230600731586.View ArticleGoogle Scholar
- Cohen J: Weighted kappa. Nominal scale agreement with provision for scaled disagreement or partial credit. Psychol Bull. 1968, 70: 213-20.PubMedGoogle Scholar
- Ely JW, Osheroff JA, Ebell MH, Bergus GR, Levy BT, Chambliss ML, Evans ER: Analysis of questions asked by family physicians regarding patient care. West J Med. 2000, 172 (5): 315-319. 10.1136/ewjm.172.5.315.View ArticlePubMedPubMed CentralGoogle Scholar
- Huang X, Lin J, Demner-Fushman D: Evaluation of PICO as a knowledge representation for clinical questions. AMIA Annu Symp Proc. 2006, 359-63.Google Scholar
- Schardt C, Adams MB, Owens T, Keitz S, Fontelo P: Utilization of the PICO framework to improve searching PubMed for clinical questions. BMC Med Inform Decis Mak. 2007, 15 (7): 16-10.1186/1472-6947-7-16.View ArticleGoogle Scholar
- Haase A, Follmann M, Skipka G, Kirchner H: Developing search strategies for clinical practice guidelines in SUMSearch and Google Scholar and assessing their retrieval performance. BMC Med Res Methodol. 2007, 30 (7): 28-10.1186/1471-2288-7-28.View ArticleGoogle Scholar
- Shultz M: Comparing test searches in PubMed and Google Scholar. J Med Libr Assoc. 2007, 95 (4): 442-5. 10.3163/1536-5050.95.4.442.View ArticlePubMedPubMed CentralGoogle Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6947/9/34/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.