We have demonstrated for the first time that it is possible to retrospectively check the compliance of thousands of real patient cases with guideline recommendations using a technology that is based on openEHR’s Guideline Definition Language and a patient data registry. We utilised the SITS registry for obtaining patient data. By doing so, we validated the results of a previous experiment that was limited to a small number of mock patient cases , and took an important step toward further implementation of such methods in electronic health record systems and patient registries.
One limitation of this study is related to tools supporting the use of GDL. The CDS Workbench did not have built-in functions to deal with missing data values, so we had to create extra GDL rules in the GDL Editor. Furthermore, we had to map the SITS registry database to an openEHR-specific CSV database, which constituted an additional work step.
The fact that GDL artefacts provided both guideline recommendations and a means of configuring the compliance checking implementation (e.g. to deal with missing values) presents a problem of different purposes being mixed together, and may thus make GDL rules less maintainable and less reusable in the long term.
Another limitation is that this technology is not suited to measure the statistical significance of the obtained results. Such calculations would typically have to be done by a separate application.
Mapping vs. interoperability
Our methodology could not be applied directly to the CSV file that was exported from the SITS registry and thus this work included a manual mapping step (see the section Limitations above), which can be seen to hamper interoperability. The target of the mapping still followed the structure of standardised archetypes, producing shareable mapped artefacts if another system wants to implement the same kind of openEHR-specific CSV database. Optimally, an openEHR-based system would provide the underlying structure of the registry in question.
Also, when dealing with legacy systems, some form of manual mapping will always be required if the original model is not used to represent clinical guidelines.
The nature of clinical practice guidelines and how GDL fits in
In our experience, clinical practice guidelines from different clinical domains (i.e. not only from acute stroke management) tend to contain their recommendations in a condensed form that is transferrable to computational formats of the if-then-else type. Furthermore, it is becoming increasingly common to release guidelines with accompanying flowcharts (e.g. guidelines released by NICE  and NGC ) that are equally computable with rule-conformant structures as they take place in the GDL implementation presented herein.
However, it is important to keep in mind that there are still certain clinical scenarios (e.g. complex chemotherapy regimens) that require more complex structures than GDL functionality can accommodate at this point. A complete clinical situation with all its socio-technical complexities also requires additions to pure rule-based logic such as workflow support and situation awareness.
Having said that, it is also important to be aware of the characteristics of a guideline’s recommendations when attempting to computerise those using GDL. While GDL’s functionality has been undergoing continuous improvements since the release of the language in 2013, it is helpful to keep in mind that the typical use cases today are those that can be represented using common mathematical functions that entail comparison of data values to each other or thresholds using mathematical operators, or involve setting values using standardised terminology codes or mathematical formulae.
Arguably one of the biggest strengths of GDL lies in its ability to solve the ‘curly braces problem’, i.e. the inefficient reliance on locally defined data models for retrieving data needed by guideline logic. GDL solves that by being based on archetypes, standard data models and standard terminologies. Furthermore, technical concerns such as runtime rules’ language syntax and mappings to local clinical models are not mixed with the clinical views of the rules which are meant to be used and verified by clinicians.
Implications for patient care
Being able to check guideline or protocol compliance on retrospective patient data can give managers or other stakeholders in health care an opportunity to see the performance of their respective organisations when it comes to following the latest evidence and state of the art in their fields. Also, it allows conducting clinical research that may investigate reasons for certain compliance patterns.
Interoperability, shareability and reusability
When the above benefits are based on interoperability specifications this adds the advantages that
data from several organisations, which follow a certain interoperability standard, can be investigated at the same time and
all organisations implementing a certain interoperability standard can share and maintain the same guideline knowledge models.
The execution of standardised models of clinical and guideline knowledge, like openEHR archetypes and GDL rules, does not have to be limited to retrospective checking of guideline compliance, but could enhance clinical decision making at the point of care. The same GDL rules used for compliance checking could, for instance, lead to notifying a clinician about a deviation from a guideline recommendation in a prescription order they have just placed.
openEHR-based clinical decision support
While this work focussed on retrospective functionality, there have long been aspirations and experiences when it comes to using openEHR technologies for providing point-of-care decision support.
González-Ferrer et al. report positive results about the adequacy of openEHR archetypes within the specification of decision support-specific clinical statements . Chen et al. demonstrate initial successes in deploying a regional clinical decision support system based on GDL . In a further effort, Xiao et al. use archetypes to develop decision support functionality in the clinical area of methadone-based therapy .
Meanwhile, when looking at a higher level picture of how different semantic components come together to achieve clinical decision support, the work herein fits into three of the layers recently published by the European SemanticHealthNet project: the openEHR archetypes fit into the layer of structured heterogeneous data (layer 1), and the GDL rules fit into the layers of semantic mediators (layer 3) and virtual homogeneous data (layer 4). Reaching a full clinical decision support system with sound semantics will entail addressing the remaining two layers, but some of that work falls outside the scope of interoperability specifications as such, e.g. when application developers realise the layer of applications based on their particular business use cases .
Implications for the development and maintainability of health records
In the long run, reusability, shareability and interoperability advantages are likely to materialise into economic advantages when developing and maintaining electronic health records as well. Having reusable clinical content models in the form of archetypes and reusable guideline knowledge models in the form of GDL rules will further foster effective software development and maintainability. At the same time, the separation of clinical models from technical models (two-level modelling), which is one of the core openEHR solution constituents, will further foster effective software engineering by harnessing the benefits of model-driven development. Studies into this have varied in their findings, where some had positive insights about the effects of model-driven development in openEHR-based solutions  and others discovered challenges that still need to be solved in real world implementations .
Directions for future work
Evaluating GDL technology at the point of care is an important next step, as it could also lead to a better understanding about features which may be missing from GDL’s functionality.
Also, introducing measures and procedures to achieve quality assurance of GDL rules that may interest a wider community could prove useful. One example of doing this would be reviewing GDL rules the way archetypes are reviewed today, which could lead to the advantage of clear categorisation of GDL rules into the clinical domains they belong to. Additionally, modifying GDL to address a better separation of concerns where GDL rules strictly deal with clinical guideline content and nothing else would also lead to GDL rules of higher shareability and maintainability value.
Furthermore, more comparisons are needed with similar research that has used other interoperability standards than openEHR or other computer-interpretable guideline formalisms than GDL such as PROforma , Asbru  or SAGE . Eventually, it would be desirable for more and more agreement to take place regarding a minimum set of necessary functions for computerising clinical practice guidelines for different purposes, which, in turn, could lead to more standardisation and potentially better patient care.