Coupling computer-interpretable guidelines with a drug-database through a web-based system – The PRESGUID project
© Dufour et al; licensee BioMed Central Ltd. 2004
Received: 18 December 2003
Accepted: 02 March 2004
Published: 02 March 2004
Clinical Practice Guidelines (CPGs) available today are not extensively used due to lack of proper integration into clinical settings, knowledge-related information resources, and lack of decision support at the point of care in a particular clinical context.
The PRESGUID project (PREScription and GUIDelines) aims to improve the assistance provided by guidelines. The project proposes an online service enabling physicians to consult computerized CPGs linked to drug databases for easier integration into the healthcare process.
Computable CPGs are structured as decision trees and coded in XML format. Recommendations related to drug classes are tagged with ATC codes. We use a mapping module to enhance computerized guidelines coupling with a drug database, which contains detailed information about each usable specific medication. In this way, therapeutic recommendations are backed up with current and up-to-date information from the database.
Two authoritative CPGs, originally diffused as static textual documents, have been implemented to validate the computerization process and to illustrate the usefulness of the resulting automated CPGs and their coupling with a drug database. We discuss the advantages of this approach for practitioners and the implications for both guideline developers and drug database providers. Other CPGs will be implemented and evaluated in real conditions by clinicians working in different health institutions.
CPGs integrate generic recommendations for specific medical circumstances. They have been defined as "systematically developed statements to assist practitioner and patient decisions about appropriate health care for specific medical circumstances" . They are designed to compile the best medical knowledge in order to provide physicians with a practical decisional aid. Clinical guidelines aim to eliminate clinician errors and promote best medical practice.
Most CPGs are now easily accessible on specialized web sites. For instance, the National Guideline Clearinghouse. alone has almost 1,000 publicly accessible guidelines. The ANAES (Agence Nationale d'Accréditation et d'Evaluation en Santé: The agency tasked by the French government with the production of medical references and CPGs) provides CPGs in PDF format on its Web site . The wide diffusion of CPGs does not however solve the problem of their effective use in daily practice. Research on the lack of adherence to CPGs has not been fully carried out. Davis' paper  shows that the compliance with guidelines increases if it is developed with the direct participation of clinician users (50% increase in relative compliance). Attaching guidelines to patients' records is a second important factor (20 % increase).
The guidelines are usually issued as long sealed electronic narratives (i.e.static PDF or HTML documents) that are difficult to use at the point of care during time-limited medical consultations. Disseminating CPGs in such a textual format has proved inefficient. The practitioner has to read several pages in order to find the appropriate care recommendation for a specific clinical circumstance. Their practical aspect rather than their content is at fault. Guidelines would prove much more efficient if they were available in the healthcare setting, integrated in the health care information system, easily adaptable to given clinical situations/scenarios and able to avoid overloading physicians with non-essential information. Minimizing the time spent consulting CPGs is crucial when attempting to improve their usage in every day practice. Physicians need both timely and pragmatic information to provide patients with the most appropriate care, and computerized CPGs have proved valuable in this respect [5, 6].
Many authors report integration in the information system as a crucial point for the dissemination of CPGs. The heterogeneity of legacy schemata and local vocabularies is an important reason why software systems are not interoperable. The community effort to develop prescribing aid systems coupled with computerized CPGs and requiring Electronic Medical Records (EMRs) is considerable [7, 8]. CPG-EMR integrations are desirable but not always feasible, for the moment, when CPGs are to be extensively disseminated [9, 10]. EMRs are not always available and the data they contain may not be complete or structured well enough for computer interpretation. Few practitioners use EMRs able to manage structured and standardized data in France today . Their large number and their heterogeneity are obstacles to their proper interoperability (e.g., there are more than 120 medical practice software packages currently available on the French market and most of these include an EMR module). In other words producing CPGs today which are integrated in the physician private information system is very expensive and the results are not easily accessible.
However, more and more physicians are connected to the Internet and use it to search for the available CPGs . Internet becomes an efficient means to improve CPGs' impact on daily practice if these online CPGs actually provide practitioners with pragmatic assistance . Providing current and precise information on drugs prescription within the CPGs will help improve both drug usage and routine CPG compliance.
The PRESGUID project objectives
From a pragmatic point of view, very few French physicians, in private practice, currently use an electronic patient record at their place of work. Without discarding the fully-integrated approach for the future, we have chosen, in the PRESGUID project (PREScriptions and GUIDelines), a pragmatic approach for the first step of implementation. We assume that producing more precise customized and shortened advice for drug prescription than the textual formats generally used by health agencies, significantly enhances CPG dissemination and health care quality. The project aims to meet the information needs of the practicing physician by combining clinical guidelines with documentation of regularly updated scientific evidence and reliable documentation on drug descriptions.
The CPGs produced by ANAES usually recommend the most appropriate therapeutic drug classes given the clinical setting. However this help is not sufficient in the daily healthcare process. Physicians must also choose among the recommended classes the appropriate specific medication to be prescribed. Ideally, clinicians should know every drug available and keep up-to-date with current pharmacopoeia (new molecules, market recalls, contraindications, marketing authorizations, etc.). In addition, they should make their selection based on both medical and economical criteria (using different incentives, the government encourages the prescription of cheaper generic medications). Computerizing CPGs and coupling them with drug databases for more complete information and improved clinical adaptability represents a definite improvement as far as physicians are concerned.
Vidal® is currently commercializing such a database which is both authoritative and extensively used by French physicians (Vidal is equivalent to the PDR in the USA) . The online version gives access to medication monographs and allows checking for drug interactions among a list of drugs selected by the practitioner. Yet, it does not include any module guiding the users towards CPGs or definite medical references.
The PRESGUID project is designed as an online service enabling physicians to consult computerized CPGs linked with drug databases allowing a patient specific decision support. It will consequently improve CPG consultation and integration into the healthcare process. We use a pragmatic approach both to implement the textual CPGs from the ANAES into a computer-interpretable format and to enhance treatment recommendation by referring to the Vidal® drug database.
The purpose is to provide physicians with an interactive CPG consultation interface displaying recommendations that match relevant patient data and the clinical setting. PRESGUID provides a web-based guideline system that takes, as input, clinical data on a particular patient and returns, as output, the customized recommendation. Should the recommendations require prescribing drugs, the system will query the drug database and will display detailed information about the relevant specific medications.
The guideline task asks for data requirements, abstraction (patient characteristics, classes of medications) and modeling requirements. Computable CPG development is facilitated by the use of an architecture integrating modular components and including user-friendly authoring tools and execution facilities . The PRESGUID project is based on such a Web architecture including a CPG development and distribution platform as well as a drug database.
CPG model and graphic editor
Action steps are used in order to collect patient data, compute and assign data value (e.g., compute the body mass index from weight and height) and display messages or recommendations.
Decision steps enable evaluation of a condition composed of one or more criteria. These criteria concern the data values derived from effecting the previous action steps. We use classical comparative operators, logical operators and mathematical functions to produce the expressions to be evaluated.
Following the patient data model, guideline authors have various options to create DataItem instances according to their own needs for a computable CPG. At this stage, we do not want to impose a particular patient data classification because there is no standard clearly established and widely used for patient data representation and classification. However, use of standard classification is a key point to allow interoperability and coupling CPGs with EMRs. In order take this into account, DataItem and Attribute instances are identified by an "Id" attribute. Future implementation may use this "Id" in order to map patient data inferred in a guideline with targeted classifications (e.g. ICD-10, MeSH,...).
The main objective of the authoring tool is to help CPG authors easily create accurate CPGs. A major consideration was to ensure flexibility and comprehensiveness for the authors. Within the authoring tool, a CPG is presented as a decision tree handled with drag and drop operations. The tool also provides specific forms to structure each element of the guideline (steps, decision criteria, patient data which have to be inferred, recommendations, etc.). Clinician experts involved in developing computable CPGs are familiar with decision tree presentation often used to draw-up classical paper-based guidelines. It is therefore easier for them to take part in the authoring process.
Within the authoring tool, patient data and other elements of the CPG can be linked to didactic documents (bibliographic references, definition of medical concepts, additional multimedia documents, relevant web sites URLs...) which can be displayed while consulting the CPG. During the authoring process, the CPG as well as its linked documents are identified by a set of properties (authors, summary, usage, MeSH keywords...) and are indexed on the CPG server. Moreover, guideline recommendations are ranked by evidence grading as defined by the Oxford Center for EBM.).
CPG server and engine
Fully developed CPGs are stored and indexed on a web server. To improve accuracy and consistency of guidelines we have opted for a procedure in which the guidelines can be tested in real conditions before putting them at users' disposal. An 'approved' status is delivered by the specialist in charge of the tests once he has judged the results satisfactory: the CPG then becomes available for any clinicians who have access to the server.
The guideline engine carries out a CPG according to patient data values. It links the steps, infers the conditional criteria and initializes actions to be performed (patient data requests, data calculation, recommendation display...).
The user interface is a dynamically generated web page. Initially, the information presented in the user interface is in XML format. We use eXtensible StyLe sheets (XSL) to generate HTML in order to format information and create data entry forms and controls.
The Vidal® database provides information about those drugs that are available on the French market and it is accepted as a reference by all healthcare professionals. It is implemented in an Oracle® relational database and delivered with a set of APIs providing access to detailed drug-related data and documents (monographs, dosage, chemical composition, etc.). The Vidal® database uses both the ATC (Anatomic Therapeutic Classification) as well as its own classification which although not formally recognized is nevertheless extensively used by healthcare professionals throughout France.
CPG and drugs database coupling
Knowledge coupling between CPGs and the drug database is based on XML message exchanges. We have developed a software component matching the XML description of therapeutic classes recommended in CPGs with appropriate medications within the Vidal® database.
The CPG authoring tool provides specific forms that allow browsing drug classifications; selecting required classes; and finally automatic embedding of the DRUG_PRESCRIPTION tags within recommendation messages.
The results are included within the initial CPG XML document. Users may consult the different medications and select the one to prescribe by consulting the detailed monographs from the drug database. Typical uses of the PRESGUID CPGs are described in the next section.
We have computerized CPGs from the ANAES library. Two textual CPGs issued by the ANAES (hypertension management and diabetes mellitus treatment) have been coded and implemented to test and validate the computerization process (a demonstration version is available at http://cybertim.timone.univ-mrs.fr/presguid/). These CPGs have been computerised using our guideline-authoring tool by the conceiver of this tool. Furthermore, two GPs, not directly involved in the PRESGUID project, have used the authoring tool in order to propose a computerised version of five CPGs from ANAES (high blood pressure, diabetes mellitus, cerebrovascular accident, benign prostatic hypertrophy and cancer pain treatment). They have been trained in the use of this tool over four hours and they have help assistance if needed. Their work is not yet fully completed. But so far, it appears that the tool is user-friendly because it is mainly focused on structuring a decision tree with especially intuitive drag-and-drop operations. Only a dozen minutes or so is required to get an initial version of a functional algorithm. It can be executed immediately and improved easily step by step. When initial textual CPGs are ambiguous and/or incomplete, it of course takes more time and expert advice is necessary. For example, the hypertension CPG indicates that smokers have higher cardiovascular risk (the recommendation is risk dependent) but does not specify which patients must be considered as smokers (Current smokers? Former smokers? Passive smokers? Active smokers? Tobacco quantity?...). Similarly, it gives a recommendation for patients with renal dysfunction and another one for patients with cardiac dysfunction but no explicit advice about patients who have both diseases.
We realise that the initial CPGs should be improved dramatically. From this point of view, the guideline-editor tool does not solve the problems of ambiguities and incompleteness of the given guidelines but it highlights these crucial problems. It helps towards validating guidelines given by experts.
A typical consultation scenario involving the diabetes mellitus CPG is presented below.
A DRUG_PRESCRIPTION tag nested in a recommendation is presented as a hyperlink (Figure 7, right-hand side). It provides access to all the specific medications that can be used in the present context.
Key points to improve CPG effectiveness concern efficiency, usefulness, information content, user-friendly interface and CPG integration into the clinician workflow processes prior to delivering recommendations on time, at the point of care. Moreover, access to up-to-date information and guided processes also improve drug prescription safety . In order to achieve these goals, we propose computable and interactive CPGs that can be used on computers connected to our server. The recommendations provided are related to specific patient data collected by clinicians during medical consultations. Furthermore, recommendations regarding prescription drugs are coupled with a knowledge database widely used by clinicians. Within CPG consultation interface, practitioners have direct access to detailed information which will help them choose among the medications potentially appropriate for the clinical context.
We have paid special attention to the accessibility and the detail level of CPGs. CPG impact depends on the assumption that clinicians will access the CPGs and will apply them in patient-specific circumstances. They must be easy to consult, integrated in the care process, and able to produce patient-specific recommendations rapidly.
The computerization of CPGs needs a strict and detailed structuring of their contents. The variability of CPG types and their complexity sometimes lead developers to construct over-specific systems thus narrowing their field of application. The authoring and the maintenance of guideline is a difficult process, which involves a detailed review of all the alternatives by evaluating current scientific evidence, a consensus process among medical experts, a specification of optimal decision and finally a documentation of the recommendations. CPGs are often complex and difficult to maintain in a patient-specific computer-based form.
In PRESGUID the guideline editor is an easy to use authoring interface requiring minimal training. It facilitates local adaptation and editing of guidelines. It can be used to create any type of CPG structured as an algorithm. It provides physicians with a very simple CPG management tool.
The prescribing aid systems currently available help detect drug contraindications and/or interactions. In our approach to knowledge coupling with CPGs, we have attempted to enhance both practical and operational aspects. This approach goes along with other significant efforts made to take the whole treatment strategy into account within systems using both CPGs and drug-knowledge databases: PRODIGY  and ASTI  are known examples of these systems. CPG consultation in PRESGUID is similar to the "guided mode" and "interacting mode" respectively of the ASTI and PRODIGY systems. In both these systems, the CPG module is intended to be provided as a plug-in/extension of commercial EMRs (one specific EMR for ASTI, several EMRs for PRODIGY which have to be compliant with minimum standards edited by the National Health Service Information Authority of the United Kingdom). Such an approach has great benefits in improving CPG integration within clinical practice but narrows their usage for the GPs who have these specific EMRs. In France there are presently no edited standards with which EMR software providers are supposed to be compliant. Moreover, there are many EMR editing software companies sharing this market. As a consequence, developing computerised CPGs is still restricted to very few systems and de facto few GPs have experience in the use of computerised CPGs connected to their EMR in France. This evidence convinces us that we have first to find a way to providing access to computerised CPGs potentially accessible to a large number of GPs without the needs of specific systems. We are aware that it is a first step to improving CPG use, but we also believe that it is a necessary step to show to a large number of practitioners and medical software providers the potentialities of computerised CPGs. In our opinion, it is also a way to influence the authoritative textual CPGs developers: computerising a CPG formatted as a textual document will reveal its ambiguities, its lack of knowledge and will consequently improve its logical structure. This can be illustrated for example, by the fact that in the French ASTI project the hypertension management CPG from the Canadian Medical Association has been preferred to the French CPG from ANAES because the latter was less detailed and less complete. Thus, computerisation of CPGs is not just a solution to improve their use by practitioners, it is also, as a first step, a quality assessment procedure to test the guidelines, and ideally to improve their contents.
With the PRESGUID project, we designed a decision system based on patient data directly collected from clinical users that allows for easy switching between the CPGs and the clinical system (this is a key requirement as far as patient-specific decision support is concerned). This system can be used as a stand-alone CPG system whose main advantage is that it is ready for use whatever the structure of the health information system may be. The associated CPGs can be widely disseminated over the Internet and easily shared between care providers. Further, this system is interactive and physicians can easily modify patient data to observe the effects on the resulting recommendations. We believe that this feature reduces the 'black box' negative effect that some physicians may feel when using decision-aid systems. This feeling is one of the obstacles to the wider adoption of computerized CPGs . Data collection, presently assumed by the clinician user when he/she consults the guideline, could be completed or replaced by the use of a mapping module with an EMR. Linking CPGs and EMR can improve their daily use. However, standalone CPGs can also have positive results in helping clinicians to choose and memorise the appropriate medication for a given patient .
As showed by Wang et al.  sharing CPGs among institutions can be achieved by using a mapping process between a guideline model and generalized guideline execution tasks. Such evidence reinforces our conviction of the necessity of basing computable CPG developments on consensual models.
Our developments take advantage of the GLIF model that presents many similarities with most models currently in use . As we have mentioned above, we use our own syntax to format CPG with XML. We believe that this syntax could be enriched and modified using XSL transformations in order to be compliant with GLIF expression language if needed. However, such transformations have not been tested. Other researchers have also used this emerging model as well as the XML format in order to structure CPGs [24–26]. Suggestions were made to improve CPG modeling based on patient data representation [27, 28], logic specification [29, 30] and additional information . Nonetheless, taking drug prescription into account is also an important factor in improving CPG modeling and implementing. This research has potential implications both for CPG developers and providers of electronic drug databases. Defining consensual methods to match drug databases and computable CPGs will enhance the whole treatment strategy. We suggest using a 'drug prescription' element embedding the ATC code associated with recommended therapies. Classifications may be different according to the drug databases that are being targeted. However it is still preferable to use nationally or internationally recognized classifications facilitating CPG sharing and re-use.
On the whole, we must bear in mind that CPG shareability is not only a question of syntax used, of knowledge representation and of coupling functionalities with EMRs. CPG shareability also requires integration into the health care workflow and similar/uniform health care processes. We need further analyzes to find a model which can take into account all these aspects so as to be fully satisfactory when implemented in the real world.
PRESGUID is a decision support system based both on authoritative CPGs and on an authoritative drug database. The project also has potential implications for healthcare institutions because the Vidal® database is now available free of charge for institutions. Our developments are thus applicable to the information systems used in these institutions. The coupling with other online drug databases using ATC coding would be similar. To improve accessibility, the guideline server uses a standard vocabulary (MeSH) and a set of properties to reference the guidelines (subjects, authors, public concerned, date, etc.). Thanks to these properties, we can structure a catalogue of the available guidelines. Depending on his/her needs, each user can build his/her own CPG list. In order to offer an even better structured guideline catalogue we plan to use more properties according to Shiffman et al. .
An evaluation study is currently in progress. Evaluations of the user-interface of the PRESGUID system have already been carried out by independent evaluators who performed a standard usability inspection (heuristic evaluation). The recommendations from the usability study revealed some problems. These recommendations have already been taken into account in the re-engineering of the Human-Computer Interface. The current online demonstration version integrates these recommendations and reflects corrections in line with the criterions of Scapin and Bastien . Evaluation of PRESGUID by GPs is already planned. This evaluation will be performed using five CPGs issued by the ANAES. The study protocol planes to evaluate the use of the system by interviews and video-recordings of GPs using the system.
We can certainly improve the PRESGUID decision system by taking advantage of the existing functionality that checks for drug interactions. The next PRESGUID version will use the API modules provided in the Vidal® database to check for interactions between drugs recommended in the CPGs and those already administered to patients. Finally, we propose a method to disseminate the CPGs produced by the ANAES in a user-friendly format and subsequently improve their application in daily practice. Experience shows that computerisation has to be taken into account from the first stage of CPG development to improve their clarity and their comprehensiveness.
The PRESGUID project is supported by the French Ministry of Research. We are also grateful for the help given by partners in this project: Dr Vincent Bouvier, Dr Jean-François Forget and Dr Eric Jarrousse from the Vidal® company of Medimedia; Pr Regis Beuscart, Marie-Catherine Beuscart-Zephir, Dr Alain Duhamel, Nicolas Leroy and Dr Jean-Marie Renard from the C.E.R.I.M. (Centre d'Etudes et de Recherche en Informatique Médicale) of the Medical faculty of Lille; Pr Pierre Le Beux and Dr Christine Riou from the L.I.M. (Laboratoire d'Informatique Médicale) of the Medical faculty of Rennes.
- Field MJ, Lohr KN: Clinical Practice Guidelines: Directions for a New Program. IOM Report. 1990, Washington, D.C.: National Academy PressGoogle Scholar
- National Guideline Clearinghouse. [http://www.guideline.gov]
- Agence Nationale d'Accréditation et d'Evaluation en Santé. [http://www.anaes.fr]
- Davis DA, Thomson MA, Oxman AD, Haynes RB: Evidence for the effectiveness of CME. A review of 50 randomized controlled trials. JAMA. 1992, 268 (9): 1111-7. 10.1001/jama.268.9.1111.View ArticlePubMedGoogle Scholar
- Johnston ME, Langton KB, Haynes RB, Mathieu A: Effects of computer-based clinical decision support systems on clinician performance and patient outcome. A critical appraisal of research. Ann Intern Med. 1994, 120 (2): 135-42.View ArticlePubMedGoogle Scholar
- Durieux P, Ravaud P, Dosquet P, Durocher A: [Effectiveness of clinical guideline implementation strategies: systematic review of systematic reviews] fr. Gastroenterol Clin Biol. 2000, 24 (11): 1018-25.PubMedGoogle Scholar
- Purves IN, Sugden B, Booth N, Sowerby M: The PRODIGY project – the iterative development of the release one model. Proc AMIA Symp. 1999, 359-63.Google Scholar
- Seroussi B, Bouaud J, Dreau H, Falcoff H, Riou C, Joubert M, Simon C, Simon G, Venot A: ASTI: a guideline-based drug-ordering system for primary care. Medinfo. 2001, 10 (Pt 1): 528-32.Google Scholar
- Tierney WM, Overhage JM, McDonald CJ: Computerizing guidelines: factors for success. Proc AMIA Annu Fall Symp. 1996, 459-62.Google Scholar
- Mikulich VJ, Liu YC, Steinfeldt J, Schriger DL: Implementation of clinical guidelines through an electronic medical record: physician usage, satisfaction and assessment. Int J Med Inf. 2001, 63 (3): 169-78. 10.1016/S1386-5056(01)00177-0.View ArticleGoogle Scholar
- Loomis GA, Ries JS, Saywell RM, Thakker NR: If electronic medical records are so great, why aren't family physicians using them?. J Fam Pract. 2002, 51 (7): 636-41.PubMedGoogle Scholar
- Fox J, Bury J, Humber M, Rahmanzadeh A, Thomson R: Publets: Clinical Judgement On The Web?. Proc AMIA Symp. 2001, 179-83.Google Scholar
- Zvarova J, Peleska J, Hanzlicek P, Zvara K: Enhanced care of hypertensive patients using the Internet. Med Inform Internet Med. 2002, 27 (3): 161-8. 10.1080/1463923021000034651.View ArticlePubMedGoogle Scholar
- Vidal Web Site. [http://www.vidal.fr]
- De Clercq PA, Blom JA, Hasman A, Korsten HH: GASTON: an architecture for the acquisition and execution of clinical guideline-application tasks. Med Inform Internet Med. 2000, 25 (4): 247-63. 10.1080/146392300455558.View ArticlePubMedGoogle Scholar
- Dufour JC, Fieschi M, Fieschi D, Giorgi R, Gouvernet J: A Platform to Develop and to Improve Effectiveness of Online Computable Guidelines. Stud Health Technol Inform. 2003, 800-5.Google Scholar
- Peleg M, Boxwala AA, Ogunyemi O, Zeng Q, Tu S, Lacson R, Bernstam E, Ash N, Mork P, Ohno-Machado L, Shortliffe EH, Greenes RA: GLIF3: the evolution of a guideline representation format. Proc AMIA Symp. 2000, 645-9.Google Scholar
- Oxford Center for Evidence-Based Medicine. [http://www.cebm.net/]
- Bates DW, Leape LL, Cullen DJ, Laird N, Petersen LA, Teich JM, Burdick E, Hickey M, Kleefield S, Shea B, Vander Vliet M, Seger DL: Effect of computerized physician order entry and a team intervention on prevention of serious medication errors. JAMA. 1998, 280 (15): 1311-6. 10.1001/jama.280.15.1311.View ArticlePubMedGoogle Scholar
- Trivedi MH, Kern JK, Marcee A, Grannemann B, Kleiber B, Bettinger T, Altshuler KZ, McClelland A: Development and implementation of computerized clinical guidelines: barriers and solutions. Methods Inf Med. 2002, 41 (5): 435-42.PubMedGoogle Scholar
- Seroussi B, Bouaud J, Antoine E: ONCODOC: a successful experiment of computer-supported guideline development and implementation in the treatment of breast cancer. Artif Intell Med. 2001, 22 (1): 43-64. 10.1016/S0933-3657(00)00099-3.View ArticlePubMedGoogle Scholar
- Wang D, Peleg M, Bu D, Cantor M, Landesberg G, Lunenfeld E, Tu SW, Kaiser GE, Hripcsak G, Patel VL, Shortliffe EH: GESDOR – A Generic Execution Model for Sharing of Computer-interpretable Clinical Practice Guidelines. Proc AMIA Symp. 2003, 694-8.Google Scholar
- Peleg M, Tu S, Bury J, Ciccarese P, Fox J, Greenes RA, Hall R, Johnson PD, Jones N, Kumar A, Miksch S, Quaglini S, Seyfang A, Shortliffe EH, Stefanelli M: Comparing Computer-interpretable Guideline Models: A Case-study Approach. J Am Med Inform Assoc. 2003, 10 (1): 52-68. 10.1197/jamia.M1135.View ArticlePubMedPubMed CentralGoogle Scholar
- Hoelzer S, Schweiger RK, Boettcher HA, Tafazzoli AG, Dudeck J: Value of XML in the implementation of clinical practice guidelines – the issue of content retrieval and presentation. Med Inform Internet Med. 2001, 26 (2): 131-46. 10.1080/14639230110061788.View ArticlePubMedGoogle Scholar
- Rosille D, Laurent F, Burgana A: Modelling of a case-based retrieval system for oncology. Proc MIE. 2003, 565-70.Google Scholar
- Veselý A, Zvárová J, Peleška J, Tomeèková M: On Direct Comparing of Medical Guidelines with Electronic Health Record. Proc MIE. 2003, poster exhibitionGoogle Scholar
- Johnson PD, Tu SW, Musen MA, Purves I: A virtual medical record for guideline-based decision support. Proc AMIA Symp. 2001, 294-8.Google Scholar
- Wang D, Peleg M, Tu SW, Boxwala AA, Greenes RA, Patel VL, Shortliffe EH: Representation primitives, process models and patient data in computer-interpretable clinical practice guidelines: A literature review of guideline representation models. Int J Med Inf. 2002, 68 (1–3): 59-70. 10.1016/S1386-5056(02)00065-5.View ArticleGoogle Scholar
- Peleg M, Ogunyemi O, Tu S, Boxwala AA, Zeng Q, Greenes RA, Shortliffe EH: Using features of Arden Syntax with object-oriented medical data models for guideline modeling. Proc AMIA Symp. 2001, 523-7.Google Scholar
- Dubey AK, Chueh HC: An XML-based format for guideline interchange and execution. Proc AMIA Symp. 2000, 205-9.Google Scholar
- Shiffman RN, Karras BT, Agrawal A, Chen R, Marenco L, Nath S: GEM: a proposal for a more comprehensive guideline document model using XML. J Am Med Inform Assoc. 2000, 7 (5): 488-98.View ArticlePubMedPubMed CentralGoogle Scholar
- Scapin DL, Bastien JMC: Ergonomic criteria for evaluating the ergonomic quality of interactive systems. Behav Inf Technol. 1997, 16: 220-31. 10.1080/014492997119806.View ArticleGoogle Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6947/4/2/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article: verbatim copying and redistribution of this article are permitted in all media for any purpose, provided this notice is preserved along with the article's original URL.