Retrospective checking of compliance with practice guidelines for acute stroke care: a novel experiment using openEHR’s Guideline Definition Language

Choice of guidelines

We chose the clinical domain of acute stroke care to drive and test our approach. The knowledge we gathered about acute stroke care guidelines comprises knowledge from the European ‘guidelines for management of ischaemic stroke and transient ischaemic attack’[24] as well as updates to those based on ground-breaking new evidence[25, 26], knowledge of thrombolysis contraindications, where thrombolysis is a decisive treatment option within acute stroke care, and knowledge of calculating the National Institutes of Health Stroke Scale (NIHSS) score. A further motivation for the choice of working with thrombolysis contraindications is that contraindications for a certain treatment seem generally suitable for making retrospective compliance checks, as contraindications are typically formulated in more concise terms than recommendations in guidelines’ text. Thus we mainly focused on thrombolysis contraindications.

Modelling of guidelines

NA interviewed TPM, a stroke physician and researcher, extensively for obtaining an understanding of the European guidelines for acute stroke management. The interviews led to a generic (guideline language-independent) representation of those guidelines on the basis of openEHR semantic concepts, which we recently published[20]. The procedure described in[20] begins with clarifying misunderstandings a medical informatician might have about guideline recommendations, and then aims at creating a chronological order of the activities that happen within the guideline processes. The activities according to this methodology correspond to openEHR CARE_ENTRY classes, i.e. OBSERVATIONs, EVALUATIONs, INSTRUCTIONs or ACTIONs, or openEHR templates. The activities are connected by either guideline conditions (e.g. ‘NIHSS score > 30’), part-whole relationships (e.g. monitoring consists of blood pressure, temperature and oxygen saturation monitoring) or their mere chronology, i.e. activities not designated with the former two relationships happen after each other chronologically. Later activities are further to the right and earlier activities further to the left (e.g. CT scan after thrombolysis in the guidelines leads to CT scan being to the right of thrombolysis). The idea with such a representation is to facilitate a smooth transition to executing guidelines in openEHR-based systems, be guideline representation model-independent and provide a basis for guideline verification between knowledge engineers or medical informaticians and physicians. Figure 1 shows an extract from our generic representation of the European acute stroke management guidelines.

We also used interviews with TPM to clarify knowledge gaps and misunderstandings about thrombolysis contraindications, which was necessary to be able to reach computable expressions of the contraindications. This clarification need typically arises when it comes to representing temporal aspects or diagnoses stated in abstract terms that are usually only clear to clinicians.

A stroke clinical research team at Karolinska Institutet and Karolinska University Hospital in Stockholm, Sweden provided us with the list of thrombolysis contraindications, which are in line with the European stroke guidelines we used and recent updates to those.

The following section shows the thrombolysis contraindications, which we partly refined (see above). These are typical examples of criteria that can be used for retrospective non-compliance checking.

Computerisation of guidelines

The Guideline Definition Language (GDL) is a declarative formalism very recently authored and added to the openEHR specifications[21]. GDL allows combining openEHR archetypes with rules and adding different clinical terminologies as well as human languages. GDL is neutral to any reference terminology as locally defined (coded) terms instead of external terminology codes are used by GDL rules, enabling modification of the term codes without changing the rule definition[21]. GDL can be used intuitively if one is familiar with ADL, the Archetype Definition Language (cf.[22]).

Based on the representation in Figure 1 and our refined thrombolysis contraindication expressions, it was straightforward to extract the corresponding rules. We used GDL in order to create rules that are well connected to a standard EHR approach and standard clinical terminologies. The standard EHR approach consisted of openEHR archetypes and the openEHR reference information model (openEHR RM). Bindings to any terminologies of choice were possible through GDL. We used the Systematised Nomenclature of Medicine Clinical Terms (SNOMED CT)[27], the International Classification of Diseases in its 10^th version (ICD-10)[28] and Anatomical Therapeutic Chemical (ATC) codes[29].

The following illustrates the procedure we used to computerise the guidelines with GDL, using the thrombolysis contraindication ‘National Institutes of Health Stroke Scale (NIHSS) score higher than 25’ (cf. section 'Modelling of guidelines' above. What our GDL representation needs in this case is a data type from the openEHR reference information model (openEHR RM) that represents a count – DV_COUNT, and for that to be obtained in the knowledge context of the NIHSS score in the form of an archetype. So we compare the score DV_COUNT element from an appropriate NIHSS OBSERVATION archetype (we authored this archetype in this case because it was not available in the common repositories) with the threshold value of 25. This way we have accounted for the CDS input part. Our CDS output would in this case be setting a DV_BOOLEAN value of a thrombolysis contraindications EVALUATION archetype (also self-authored) to true in case the NIHSS score exceeds 25 and false if it does not. The corresponding GDL lines of code would be

when = <"$gt0008>25",…>

then = <"$gt0016=true",…>,

where gt0008 and gt0016 are codes that simply refer to the data elements mentioned above and had been defined earlier in the GDL code from the respective archetypes.

Similarly, the conditions and actions in Figure 1 can be represented using the appropriate GDL expressions and archetype data elements and in those examples, terminology bindings are also often present due to the existence of several diagnoses or conditions such as ‘stroke’ or ‘stroke mimic’. For example, a stroke diagnosis could be represented by both the SNOMED CT concept ID 230690007 and the ICD-10 code I64. In GDL, the existence of this diagnosis would be checked using codes as above (also codes to represent the particular diagnosis as opposed to the relevant diagnosis archetype data element), which would then be connected to the different terminologies in GDL’s terminology binding section, e.g.

when = <"$gt0003 is_a local::gt0102|stroke|",…>

[other code before the terminology binding section]

["ICD10"] = (TERM_BINDING) <

bindings = <

["gt0102"] = (BINDING) <

codes = <[ICD10::I64],…>

[rest of ICD-10 section]

["SNOMED-CT"] = (TERM_BINDING) <

bindings = <

["gt0102"] = (BINDING) <

codes = <[SNOMED-CT::230690007],…>.

The Results section goes into more detail regarding the produced GDL rules and connects them to the additional files we deliver together with this article.

Execution of guidelines on patient data

We reviewed one real patient case in order to get a good understanding of what realistic patient data look like, e.g. realistic blood glucose values or patient history notes. Data from the patient case then guided us in developing a Java program that generated random acute stroke care data, which we used to populate our experimental environment. We created 49 patient cases which had different features that could be tested for compliance.

We stored the patient cases using the Data Archetype Definition Language (dADL)[22] and ran the guidelines represented in GDL on these patient cases.

Manual validation of compliance results

In order to validate the automatic compliance results we got, we went through the 49 mock patient cases manually to check their compliance with the stroke guidelines we used, and compared the correct results of the manual inspection with the compliance results achieved by our computerised CDS logic.

Tools used

We utilised the open-sourced GDL Editor for authoring GDL files and the CDS Workbench for running them on patient cases, where the CDS Workbench also facilitated creation of patient data based on openEHR archetypes. Where available archetypes from the international archetype repository[30] were not sufficient, we used the open-sourced Ocean Archetype Editor to create new archetypes or modify existing ones.

Additional file1 has the URL of the GDL website, with the possibility to download installation files of the GDL Editor. Figure 2 shows the CDS Workbench.

The GDL Editor provides a certain level of terminology integration by harnessing the built-in terminology service. It is possible, for example, to search for a term by a string partial match and navigate hierarchies in ICD-10 and ATC. But in order to achieve a higher level of integration with more sophisticated terminology resources such as SNOMED CT, a full-fledged terminology service will have to be used. The GDL Editor does not currently support archetypes based on different information models than the openEHR RM.

Ethical considerations

Data from one patient case, which had been collected as part of usual care, were anonymised following the patient’s informed consent. The patient authorised publication of her/his anonymised data upon review of the present manuscript. We did not, however, use these anonymised data one to one, but based our virtual patient cases on them (cf. section ‘Execution of guidelines on patient data’ above).

Technical set-up summed up

In accordance with the methods we chose for computerising and applying guidelines, our technical environment consisted of

Methods and materials summed up

Compliance checking

We authored GDL files that contain the guideline knowledge and logic of our chosen acute stroke guidelines. Then we ran the GDL file related to thrombolysis contraindications on patient cases in dADL format and validated the results manually. This manual validation of the 49 mock patients confirmed the results of the automatic compliance checking. Figure 3 shows an example of the statistics we obtained automatically from the CDS Workbench, with different compliance criteria and their fulfillment.

Technological components of GDL applied to acute stroke

openEHR archetypes based on the openEHR reference information model

In[20] we identified and authored archetypes that are needed to capture the clinical knowledge in the acute stroke care setting, but did not list those archetypes there as the focus of that study was to highlight the methodology used and its implications. We append the list of those archetypes to this article in Additional file2. Given the compliance criteria we had put together, we chose the archetypes required to represent the compliance data from the list above.

Our archetypes are based on the openEHR reference information model (openEHR RM) and are of the CARE_ENTRY type, i.e. OBSERVATIONs, EVALUATIONs, INSTRUCTIONs and ACTIONs (cf.[16]).

We reused 14 archetypes. Additionally we authored a new OBSERVATION archetype to represent the National Institutes of Health Stroke Scale (NIHSS) and a new EVALUATION archetype that consists merely of openEHR DV_BOOLEAN (cf.[16]) data elements, in order to capture thrombolysis contraindications. Figure 4 shows a screenshot of this archetype from the Ocean Archetype Editor (a tool for authoring archetypes).

The list of archetypes we used, which are within Additional file3, is as follows:

• openEHR-EHR-ACTION.intravenous_fluid_administration (reused)

• openEHR-EHR-ACTION.procedure (reused)

• openEHR-EHR-EVALUATION.alert (reused)

• openEHR-EHR-EVALUATION.problem_diagnosis (extended by a data element called ‘Confidence’, which is a coded text that has the options ‘Suspicion’ and ‘Certainty’, and further modified to allow a more precise timestamp in the data element ‘Date of initial onset’)

• openEHR-EHR-EVALUATION.thrombolysis_contraindications (new)

• openEHR-EHR-INSTRUCTION.imaging (reused)

• openEHR-EHR-INSTRUCTION.medication (reused)

• openEHR-EHR-ITEM_TREE.gas_administration (reused)

• openEHR-EHR-ITEM_TREE.imaging (reused)

• openEHR-EHR-ITEM_TREE.medication (reused)

• openEHR-EHR-OBSERVATION.blood_pressure (reused)

• openEHR-EHR-OBSERVATION.body_temperature (reused)

• openEHR-EHR-OBSERVATION.exam (reused)

• openEHR-EHR-OBSERVATION.indirect_oximetry (reused)

• openEHR-EHR-OBSERVATION.lab_test-blood_glucose (reused)

• openEHR-EHR-OBSERVATION.nihss (new)

Guideline definition language rules

We used 16 GDL rules to represent thrombolysis contraindications, 58 GDL rules to represent the National Institutes of Health Stroke Scale (NIHSS) score calculation (setting the different values according to different conditions and calculating the total score) and 6 rules as a sample from the European stroke management guidelines. The three respective GDL files are in Additional file4.

As examples of the GDL elements we arrived at in our acute stroke management setting, Figure 5 shows an extract from the archetype binding part of GDL, Figure 6 an extract from the rules section of GDL and Figure 7 GDL terminology bindings.

The archetype binding part in Figure 5 constitutes assigning GDL-specific codes to particular archetype elements needed by the rules of the GDL file of concern, e.g. a GDL file could group together rules related to the same concept, like thrombolysis contraindications. So we import, for instance, the element with the path /data[at0001]/items[at0002.1], which is the diagnosis text, and it is assigned the GDL-specific code gt0003, which is used in the rest of the GDL file to make diagnosis evaluations.

The GDL rules in Figure 6 correspond to our descriptions in the Methods section and so does the terminology binding in Figure 7 (cf. section ‘Computerisation of guidelines’ under Methods).

Patient cases in data archetype definition language format

Our 49 patient cases corresponded to 49 dADL files. Figure 8 shows an extract from a dADL file where imaging findings of a patient are recorded. The highlighted parts show how the Findings data element of an imaging archetype is set with the SNOMED CT concept ID 50960005, corresponding to the textual value “Haemorrhage”, and how the Anatomical site data element of the same archetype is set with the SNOMED CT concept ID 12738006, corresponding to the textual value “Brain”.

Examples illustrating mechanisms of GDL applied to acute stroke

The following examples illustrate GDL mechanisms and rely upon criteria from the section 'Modelling of guidelines' in Methods. These examples and others are available in detail and for hands-on experiences using Additional file3 together with Additional file4, or using Additional file6.

Here the value of the data element Date of initial onset of openEHR RM type DV_DATE_TIME from the archetype Problem/Diagnosis is compared using the GDL < operator with a GDL variable called Current Date/Time to check whether stroke onset was within the past 4.5 hours or not. If not, the data element Time since stroke onset > 4.5 hours of openEHR RM type DV_BOOLEAN from the archetype Thrombolysis Contraindications is set to True.

Figure 9 shows the GDL Editor rule view for this rule, in addition to the GDL Editor dialogue that chose the relevant archetype element in the condition and the GDL Editor terminology binding view, where the SNOMED CT part for stroke is highlighted.

The condition here is that the value of the data element SpO2 of openEHR RM type DV_QUANTITY from the archetype Indirect oximetry is checked for being lower than 95% using the < operator. If it is lower, the following actions take place: the data element Gas of openEHR RM type DV_TEXT from the archetype Gas administration is set with the SNOMED CT code for oxygen; the data element Means of delivery of openEHR RM type DV_CODED_TEXT from the archetype Gas administration is set with the archetype-internal code at0006 representing Nasal canula.

GDL in relation to other guideline representation models

Additional file7 contains a table that uses a couple of selected features to put GDL and other guideline formalisms – languages that create computer-interpretable guidelines - in relation to each other.

This table draws on our own experiences, descriptions by some of the guideline formalisms’ creators[2–5, 7, 21, 31], the review by Isern and Moreno[10], chapter 13 of[8] and a scan of recent literature.

What’s new with this automated compliance checking?

Several research efforts have been published to demonstrate automated retrospective checking of compliance with guidelines, e.g. in[34] and[35]. What is novel in the present work is that it provides a solution based on openEHR, a semantic technology for facilitating semantic interoperability that has been contributing to international standards such as ISO 13606 and receiving attention in industry, academia and nation-wide or regional initiatives of different countries such as Sweden, Brazil and Russia.

Lessons learned

A reflection regarding availability of clinical knowledge using this technology is that GDL provides a straightforward way of obtaining clinical EHR data elements due to its openEHR-archetypes underpinning. For example, if a rule evaluated whether or not a blood glucose value has exceeded a certain threshold, then all that needed to be done was to access the necessary blood glucose data instance from its corresponding archetype and check its magnitude and unit against the threshold.

Also, GDL allows reuse of archetypes both for reading data values as a part of checking guideline conditions (CDS input) and for setting data values as actions to be taken upon fulfilling guideline conditions (CDS output). For example, if a rule required that an MRI imaging test be ordered (CDS output) if the Glasgow Coma Scale score reaches a certain value (CDS input), then GDL can use the OBSERVATION Glasgow Coma Scale archetype to check its score and the INSTRUCTION imaging archetype to record the imaging order entry.

Furthermore, GDL could be used to enhance openEHR archetypes by uncovering shortcomings they have, leading to a feedback loop from CDS to clinical models (cf. section ‘openEHR archetypes based on the openEHR reference information model’ in Results above).

Deciding whether a component of a GDL rule is going to be an archetype data element or GDL construct is not ambiguous in our experience. GDL constructs are only needed to compare data elements against each other or constant values, check the existence of data elements or set the values of data elements. So the data elements as such were always archetype elements and that was straightforward to derive.

Stroke limitation

Compared to guidelines from other clinical domains, acute stroke care guidelines are rather straightforward to transfer to computer-interpretable formats. The structure of recommendations in acute stroke care guidelines is often transferrable to simple rules that satisfy basic if-then statements. Guidelines from other clinical domains may have more complex demands on guideline modelling and could thus be more challenging to computerise. We are aware of this limitation, but think that it does not significantly compromise exploring the overall feasibility of combining a semantic EHR technology with rules for automatic guideline compliance checking. Still, there is a need to represent and execute guidelines from other clinical areas in order to further validate GDL’s usefulness for guideline-oriented CDS generally (cf. section ‘Future directions’ below).

Utilisation of reference information models and standard terminologies

The reviews of different guideline representation models in[9] and[10] emphasize the need for an effective way to work with patient data when modelling computer-interpretable guidelines. They state that there is a lack of a standard way to represent patient data and achieve integration with EHRs. The advantage of our proposal is that it uses the openEHR RM, one of the available reference information models that are meant to standardise data types and data structures in EHRs (there are others like ISO 13606’s reference information model and the HL7 RIM).

The recent study by Zhou et al.[11] shows the lack of ‘rule authoring environments’ that allow for shareable and standardised rules. The rule language for guidelines that we use, GDL, directly tackles that issue by creating rules that are shareable and standardised, as they rely on shareable clinical knowledge content models that tend to get standardised over time in the form of archetypes and use a reference information model for healthcare data in the form of the openEHR RM.

Using other information models such as ISO 13606’s or the HL7 RIM should work with GDL, as long as certain features are provided by the respective information model: data types have to allow set and get operations on them; it should be possible to compare data elements of the same data type; a data element has to be accessible through unique string paths. Having other information models than the openEHR RM as a basis for GDL has not been tested anywhere and there are currently no tools to support that. Archetype data paths, for instance, containing attribute names from openEHR RM classes would hinder executing guideline content directly from archetypes created using other information models. In such cases, mappings between the openEHR RM and other information models would be required.

Additionally, GDL and openEHR archetypes support the utilisation of terminologies such as SNOMED CT and ICD, which further increases shareability and standardisation. Terminology bindings can be vital in providing the different levels of data abstraction or granularity that guidelines often demand. Extensive hierarchies, such as SNOMED CT’s, can be of good help there. For example, it might be important for a guideline to know whether a certain diagnosed disease is a cardiovascular disease. Intra-terminological relations can make it easy to say that ‘Pathological condition X is a cardiovascular disease’, for instance. As mentioned above under the section ‘Tools used’ in Methods, current tools like the GDL Editor need to be developed further to take full advantage of such features.

Separation of data from rules leads to shareability

We achieved improved shareability of CDS rules through using archetypes, a reference information model and reference terminologies. GDL tackles the well-known curly braces problem mentioned in the background as it separates data – which come from archetypes – from rule logic – provided by GDL expressions.

Furthermore, GDL achieves a separation between rule logic and natural languages (e.g. English, Spanish, Chinese, Arabic, Portuguese…) as well as between rule logic and reference terminologies such as ICD.

This solution vs. manual compliance checking methodology

Typically manual checking of compliance with guidelines involves checking whether certain conditions evaluate to yes or no, true or false, fulfilled or not fulfilled, i.e. it involves dichotomous or Boolean logic. Checklists often provide the means of Boolean data collection and those data can then be analysed in various ways, as Luker and Grimmer-Somers also show[36]. This sort of compliance checking matches the compliance checking our method performs, where our computerised approach provides all the advantages of automatisation.

However, manual compliance checks can also be more sophisticated. They can involve the derivation of quality indicators and the aggregation of expert opinions as well as various clinical data to obtain compliance scores[37, 38]. We did not incorporate such checks into this study, but think that integration of various such metrics would be possible with the same set-up, yielding valuable insights for decision makers in healthcare that may not be possible with merely manual efforts. The inclusion of quality indicators, results of expert opinions and compliance scores will usually rely upon the integration of relatively simple mathematical formulae.

Rule maintainability and knowledge scalability

There is too little experience with GDL to be able to judge how easily knowledge captured in GDL rules can be maintained and extended. That is a matter that needs evaluation over time. The good news is that there are already some governance models for openEHR archetypes[30], which can possibly be applied to GDL knowledge artefacts too. Furthermore, if these governance models prove successful with archetypes, a part of the rule knowledge would already be maintained, since GDL relies on reusable openEHR archetypes.

Whichever the case, it will always take a considerable amount of time to reach a computable format of the guidelines from their published form, as guidelines tend to be published mostly as narrative text nowadays. Reaching a representation like the one shown in Figure 1, for example, will probably constitute the majority of the time and cost burden involved in maintaining GDL rules. This is due to the time-intensive process of communication between the medical informatician and physician needed for removing vagueness as well as concretizing temporal aspects.

Closely related studies

There are a couple of research groups that have been working within tightly related research areas to the one this work belongs to. Marcos and Martínez-Salvador[39] developed their own methodology for arriving at the archetypes needed for a certain guideline as well, where their methodology had less of a graphical character than ours in[20] but followed an iterative algorithm for searching archetype repositories and creating new archetypes based on guideline content. Marcos, Maldonado et al.[40] developed a mapping technique to reach interoperability between EHRs and CDS systems that used concepts from the guideline representation model PROforma as well as openEHR together. Their solution had a focus on database data mapping, which had also been the idea behind the knowledge-data ontological mapper (KDOM) developed by Peleg et al. for relational databases in particular[41].

Future directions

To test the usefulness of GDL for the expression of clinical guideline knowledge in a shareable manner, GDL has to be used to computerise various guidelines from several clinical areas. This would challenge GDL’s ability to represent guideline knowledge in general, which is the primary prerequisite for its success.

Also, GDL seems just as suitable to implement CDS rules to be used at the point of care as it is to implement CDS rules for retrospective compliance checking; the representation of rules in GDL does not differ for those two use cases. However, evaluation studies within clinical practice would be needed to verify at-the-point-of-care CDS through GDL.

When it comes to tools that facilitate working with GDL, there are two obvious demands that tool developers may find interesting to address in the future for purposes of semantic interoperability. The first is the provision of functionality for binding data elements to terms from standard terminologies, e.g. 1) implementing algorithms for pre-coordination and post-coordination using SNOMED CT’s extensive hierarchy, the former helping to find useful relationships (e.g. is-a relationships) between terms in order to avoid the need to manually identify all fitting term concepts, and the latter making it possible to aggregate terms for forming unavailable ones; 2) using lexical and context-based techniques to find suitable terms in a terminology to bind to archetype data elements, as Meizoso García et al. demonstrated with promising success[42]; 3) identifying interesting terms through visualisation techniques[43]. Secondly, current GDL editing and execution tooling does not support using other standard information models than the openEHR RM. Thus, efforts to advance existing GDL tools or create new ones may want to facilitate designing and running GDL content using, for example, the HL7 RIM or ISO 13606’s information model.

Our study fits the context set by Mandl and Kohane in 2012: ’The IT foundation required for health care is the core set of health data types, the formalization of health care workflows, and encoded knowledge (e.g. practice guidelines, decision-support tools and care plans)’[44].