BMC Medical Informatics and Decision Making

Background: The first step in practising Evidence Based Medicine (EBM) has been described as translating clinical uncertainty into a structured and focused clinical question that can be used to search the literature to ascertain or refute that uncertainty. In this study we focus on questions about treatments for schizophrenia posed by mental health professionals and patients to gain a deeper understanding about types of questions asked naturally, and whether they can be reformulated into structured and focused clinical questions.


Background
When practising evidence-based medicine (EBM), the ability to translate clinical queries into structured and focused clinical questions has been described as the first step of evidence based practice [1,2]. In this way, it is argued, that answers can be found where evidence exists from systematic reviews and clinical trials. If the clinical question cannot be answered by referring to the evidence base then an uncertainty has been identified. The well built clinical question is seen as the key to finding the research evidence and making evidence based decisions. Teachers of EBM contend that the well structured clinical question contains four elements within it. These are the Patient, the Intervention, the Comparison and the Outcome [3]. Together these elements form the acronym PICO. It is argued that the PICO format makes it relatively straightforward to elicit and combine the appropriate terms needed to find the evidence when searching databases such as PubMed [4]. The PICO framework has been adopted by the U.S. National Library of Medicine, which with its over 16 million citations forming MEDLINE, is the largest and most used web source of abstracts. However, the ability to ask PICO style questions, search for the literature effectively and interpret the search results is not intuitive and requires training [5]. Previous studies have looked at how effectively physicians ask clinical questions [6]. This issue is becoming applicable to the general public, which is increasingly turning to internet resources to address uncertainties they might have about treatments.
In this study we focus on questions about treatments for schizophrenia posed by mental health professionals and patients. The objective was to gain a deeper understanding about the types of the questions which are asked naturally and whether they could be reformulated in a structured PICO framework. Furthermore, we defined a questiontaxonomy and evaluated whether certain categories within a taxonomy lend themselves better for formulating well defined (and unambiguous) queries. The ultimate goal is to then devise a system within which to channel a user to pose good queries.

Methods
A convenience sample of patients and clinicians recruited from a web-survey, a conference and members of a mental health charity were asked to write down in their own words an uncertainty or question they had about the effect of treatment for schizophrenia. Both the form and the web-survey gave a brief explanation of uncertainty and examples of types of treatment uncertainties No ethical issues were identified in this non experimental, observational study conducted on anonymous responses to a survey. In order to evaluate the presence and the accuracy of the PICO elements present in peoples' responses, two of us (AC and MT) independently rated the questions for the presences of PICO elements. Results were then compared, discrepancies identified and discussed to agree on definitions. We then assessed inter-rater agreement. Every question was analysed and searched for PICO elements (Patients, Intervention, Comparison and Outcome) and once an element was found, appropriate text corresponding to the element was highlighted. We then examined the questions for evidence of a linguistic taxonomy. Questions were clustered according to an emergent taxonomy and associations between these categories and PICO elements were explored. Definitions and examples of the linguistic taxonomy are given in the results section.

Respondents and their questions
489 questions were collected for the Database of Uncertainties about the Effect of Treatment (DUET) for mental health from various health professionals and patients [7]. For the purpose of this exercise a subset of 239 questions on schizophrenia was selected, 161 from clinicians and 78 from patients. All 239 questions about schizophrenia were grouped according to the topic of enquiry. Twelve topics were identified. There was a good deal of similarity between the topics selected by clinicians and patients. Both groups were most likely to ask questions about drug therapies, followed by psychological therapies, followed by questions or statements that were not classifiable (Table 1).

PICO elements in patient and clinician uncertainties
The most common PICO element identifiable in the questions asked by both patients and clinicians was an "Intervention" followed by an "Outcome" followed by a "Patient". The PICO element least likely to be present was a "Comparator" ( Table 2).
Inter-rater agreement on the PICO elements contained in clinicians' questions and separately for patient questions as calculated by Cohen's Kappa coefficient [8] are also shown in table 2. Kappa coefficients show moderate agreement between the raters for clinician and patient questions.

Identifying a linguistic taxonomy
To identify the structure and content of the questions asked by respondents' we carried out a syntactic analysis to derive a taxonomy of question types. This exercise was conducted for the clinician questions. Patient generated questions were omitted for two reasons. Firstly, there was low inter-rater agreement for identifying PICO elements in patient generated questions. Secondly, because there were smaller number for patients the assumptions under-

Response Class Definition Example
Comparison query The desired answer is to state the similarities and differences between two or more concepts or events.
"Old versus. atypical antipsychotic" How query The desired answer explains a mode of action, mechanism or process.
"How to access the best counselling support"

What query
The desired answer is a particular concept or event. What is the difference between depression and happiness?

When query
The desired answer is a time point. "When do they use E.C.T. treatment?"

Why Query
The desired answer explains the cause of a concept or event. "Why is E.C.T. banned in several countries?" Statement A query is categorized as a statement, if it is an opinion. "Too much medication is not regularly reviewed" Ill defined query Not categorized in any of the previous response classes. "Does ECT work?" lying the chi square test were not valid. Linguistic analysis of elements naturally occurring within clinicians' questions about treatment uncertainties for schizophrenia were classified into the following categories: Comparison queries, How queries, ill-defined queries, what queries, why queries, when queries and statements. These categories are summarised in Table 3 together with examples from each category. Table 4, shows associations between PICO elements and the linguistic categories we had generated. Naturally occurring questions that involved "what" type queries were the ones most likely to contain identifiable PICO elements (x 2 = 33.31 p < 0.0001). Strong associations were observed for I (x 2 = 27.52, p < 0.0001) and O (x 2 = 23.97 p = 0.001). The statement category of query was the least likely to contain PICO elements.

Discussion
We asked mental health patients and clinicians to identify an uncertainty about the treatment of schizophrenia. Clinicians and patients asked similar types of questions. Few naturally asked structured and focused clinical questions in the PICO format. However, the team was often able to identify PICO elements present in patient and clinicians questions. In many questions those PICO elements were not present making questions hard to classify using the structured clinical question format from evidence based medicine. Other taxonomies may better capture the nature of questions people naturally ask, for example a linguistic taxonomy of the type described here. Access to EBM resources is greatly facilitated by framing enquiries in the language of EBM [9,10]. People do not naturally do this. It may be preferable to identify a way of searching the literature that more closely matches the way people naturally ask questions if access to information about treatments are to be made more broadly available [11].
The main limitation of our study is that this is a secondary analysis of data obtained using a convenience sample from one disease area. Our findings may or may not generalise to other patient and clinician groups. However, if anything, we would expect those persons who took the time to reply to our survey to be among the more interested ones in evidence based practice than those who did not. Respondents were deliberately not asked to structure their responses using the PICO format so as to minimise response bias.

Conclusion
It is basic principle of EBM that translating clinical uncertainty into structured and focused clinical questions is a prelude to searching the literature. Much time and effort is spent teaching clinicians to ask questions in this PICO format. Our results support the view that people do not naturally ask questions in a well structured format. Broadening access to EBM resources will either require training everyone, both patients and clinicians, in EBM or providing them with ready digested answers to common uncertainties and queries or ideally by identifying a method of searching the literature which does not require the searcher to be trained in EBM [12,13]. Such an approach would greatly broaden access to knowledge and information about health care. It remains to be seen whether PICO elements result in better outcomes compared to other question formulations. Natural language queries or specific typologies of question might be more accurate in retrieving relevant.