An ontology-based tool for modeling and documenting events in neurosurgery

Abstract

Background

Intraoperative neurophysiological monitoring (IOM) plays a pivotal role in enhancing patient safety during neurosurgical procedures. This vital technique involves the continuous measurement of evoked potentials to provide early warnings and ensure the preservation of critical neural structures. One of the primary challenges has been the effective documentation of IOM events with semantically enriched characterizations. This study aimed to address this challenge by developing an ontology-based tool.

Methods

We structured the development of the IOM Documentation Ontology (IOMDO) and the associated tool into three distinct phases. The initial phase focused on the ontology’s creation, drawing from the OBO (Open Biological and Biomedical Ontology) principles. The subsequent phase involved agile software development, a flexible approach to encapsulate the diverse requirements and swiftly produce a prototype. The last phase entailed practical evaluation within real-world documentation settings. This crucial stage enabled us to gather firsthand insights, assessing the tool’s functionality and efficacy. The observations made during this phase formed the basis for essential adjustments to ensure the tool’s productive utilization.

Results

The core entities of the ontology revolve around central aspects of IOM, including measurements characterized by timestamp, type, values, and location. Concepts and terms of several ontologies were integrated into IOMDO, e.g., the Foundation Model of Anatomy (FMA), the Human Phenotype Ontology (HPO) and the ontology for surgical process models (OntoSPM) related to general surgical terms. The software tool developed for extending the ontology and the associated knowledge base was built with JavaFX for the user-friendly frontend and Apache Jena for the robust backend. The tool’s evaluation involved test users who unanimously found the interface accessible and usable, even for those without extensive technical expertise.

Conclusions

Through the establishment of a structured and standardized framework for characterizing IOM events, our ontology-based tool holds the potential to enhance the quality of documentation, benefiting patient care by improving the foundation for informed decision-making. Furthermore, researchers can leverage the semantically enriched data to identify trends, patterns, and areas for surgical practice enhancement. To optimize documentation through ontology-based approaches, it’s crucial to address potential modeling issues that are associated with the Ontology of Adverse Events.

Introduction

In neurosurgery, it is imperative to differentiate between healthy brain tissue and pathological tissue. Unlike other organs, functional areas and tracts within the brain are not visibly discernible to the neurosurgeon’s naked eye. Injuries sustained in such areas can result in postoperative neurological deficits, such as motor paralysis. Intraoperative neurophysiological monitoring (IOM) enables continuous measurements of neural electrical activity during surgery, thereby helping to mitigate the risk of postoperative deficits [1, 2]. An IOM team has to document patient and surgical information, along with neurophysiological, surgical, and anesthesia events, each timestamped. Additionally, they record neurophysiological baseline measurements of various evoked potentials, followed by tracking changes in signal measurement values, including amplitude and latency [3].

While existing IOM systems (e.g., those provided by the Inomed, Neurosoft or Ionnova Medical companies) may offer basic reporting functions and integration with clinical information systems, they lack the depth and specificity required for comprehensive event documentation. The persistent challenges outlined below have resulted in continued reliance on paper-based documentation and underscored the necessity for the development of an ontology-based approach. Surgical procedures are inherently intricate and dynamic, involving a multitude of interventions, responses, and contextual factors that demand precise documentation. Recording such events presents a formidable challenge, particularly when relying on manual methods and non-standardized descriptions during surgery, where unforeseen complications can emerge. Documenting these events necessitates meticulous attention to details and the ability to capture nuances and context, which can be daunting in the demanding surgical environment. Inconsistencies and gaps in documentation practices impede the utilization of IOM data for decision-making and quality improvement endeavors. Structuring information according to a predefined ontology facilitates the seamless capture of nuanced event details, encompassing specific interventions, responses, and contextual factors. This granular level of documentation is essential for accurate medical decision-making and retrospective analysis, while also serving as the foundation for prospective automatic classification based on signal data.

In our effort to standardize documentation and to switch to a digital framework for intraoperative neurophysiological monitoring (IOM), we have previously developed a JavaScript-based web application known as the IOM-Manager [4]. The primary objective of the IOM-Manager is to enhance the efficiency of data entry during surgical procedures. The efficiency was enhanced by enabling an intelligent hierarchical search and selection of terms for documentation, allowing users to quickly access the desired terms. Of course, this requires a brief training session on the tool and familiarization with the catalog of terms, which, among other things, led to the recognition of deficiencies within it. To facilitate the comprehensive documentation of events, we created a structured catalog comprising four main categories and 25 subcategories. These main categories encompass terms related to IOM, surgical procedures, anesthesia, and other pertinent aspects. For a sample protocol, consult [5], and for a broader context on IOM documentation, see [4].

Although, the IOM-Manager garnered positive feedback, we remained apprehensive about the accuracy and semantic coherence of the terms within the catalog. Employing an ontology customized to our specific use case addresses this concern by precisely defining related concepts and their relationships, enabling consistency checks and semantic inferences. Unlike a taxonomy, which establishes ‘is_a’ relationships, or meronymy, which indicates ‘part_of’ relationships, an ontology offers a more comprehensive semantic framework for documenting events. It facilitates content-related relationships such as causally-related-to or occurs-in, thereby enhancing the precision and coherence of the documentation. Some pitfalls of developing an ontology for IOM were listed in [6]. The focus here lies in elucidating the development of the full ontology and the associated new tool, aspects only briefly mentioned and hinted at in prior publications. We showcase how an ontology-based tool can enhance the documentation of event data in neurosurgery while mitigating certain modeling challenges.

By annotating IOM data with ontology, semantic clarity is ensured through standardized terminology and relationships. This fosters clear communication among healthcare professionals, enhancing understanding of monitored events and their implications. A comprehensive view of the patient status during surgery enables informed decision-making and prompt actions, while improved interoperability ensures accessible patient information across care settings. Researchers can leverage this data to discern trends, patterns, and areas for surgical practice enhancement. Though ontology-based documentation demands initial time and effort, the long-term benefits in patient safety, decision quality, and practice enhancement justify the investment. Furthermore, technological advancements like natural language processing and automated data capture can streamline documentation and make ontology-based approaches sustainable in clinical practice [7,8,9].

An ontology-based documentation tool serves to aid non-technical users in data input and knowledge base querying. To meet this goal, we’ve crafted a user-friendly graphical interface prototype tailored to their needs. While tools like Protégé [10] and ROBOT [11], alongside APIs like OWL API [12], suit ontology experts, they often pose challenges for non-technical users tasked with populating the knowledge base. Simplified user interfaces aligned with domain experts’ expertise can vastly improve ontology population. It is important to note that the software prototype doesn’t target real-time documentation during surgeries, leaving this functionality for future integration projects.

Theoretical background on modeling IOM events

During surgical resection of brain tumors, it is a challenge to distinguish between healthy and pathological brain tissue. Even more important, functional areas such as motor, sensory or language areas and tracts are not visually identifiable for the neurosurgeon. A lesion of these regions may result in a postoperative neurological deficit, e.g., motor paralysis, and affect the quality of life significantly. The use of IOM with continuous measurements of neural activity during surgery reduces the risk of such deficits. IOM includes mapping techniques, which are used to locate functional areas, as well as continuous functional monitoring, which is useful for assessing functional integrity. By measuring evoked potentials, the surgical strategy can be adjusted in real time to prevent impending functional impairment [13,14,15,16].

For example, motor evoked potentials (MEPs) are used to monitor indirectly motor function. They are measured at the beginning of the surgery (so-called baselines) as a reference for subsequent stimulations. If the MEPs remain stable during surgery, it is an indication that there will be no permanent motor deficits. However, if there is signal change or loss (a warning criterion), postoperative motor deficits can occur. MEPs can be measured by transcranial electrical stimulation (TES) through scalp electrodes or by direct cortical stimulation (DCS) through strip electrodes (grids) placed in the central region of the brain cortex [15, 17]. See Fig. 1 for a visualization of the motor pathways.

Fig. 1

Schematic visualization of the pathway with MEPs. Transcranial electrical stimulation (yellow flash) of the motor cortex results in a response that is transmitted through the brain and spinal cord to cause muscle contraction. Here a brain tumor (brown) infiltrates the motor pathways. By using IOM techniques, as much tumor as possible can be removed with the goal of preserving motor function at the same time

Documenting the complexities of IOM events requires precise and rich terminology. Since we already possessed a catalog of terms integral to our ontology, we sought existing ontologies that could readily accommodate these terms. Two candidates were found: the Ontology of Adverse Events (OAE [18]) and the Information Artifact Ontology ($\:\widehat{x}$ [19]). We evaluated OAE and encountered several conceptual issues, particularly with its definition of adverse events as ‘a pathological bodily process that occurs after a medical intervention.’ While this definition may align with the suggestions of BFO founders [20] equating events with processes, we recognized the need to differentiate between basic and complex events due to critical questions surrounding identity criteria and whether events are particulars or universals [21]. This differentiation is essential for our event documentation use case, which predominantly requires precise references to events as particulars, as further elaborated in the subsequent section.

From a philosophical standpoint, the issue of modeling events raises questions about the nature of events and how real-world references to these events can be established. It is crucial to have a solid grasp of the event concept to ensure consistent terminology and to facilitate coherent use of this term in various contexts. The Basic Formal Ontology (BFO), while providing a foundational ontological structure, does not inherently equip one with the tools to identify and comprehend events. A crucial aspect in understanding the nature of events is the establishment of identity criteria to determine when two events are the same. This concurs with Quine’s dictum “no entity without identity” [22, 23]. A recent theoretical paper of Guarino et al. points to the issues involved [24]. To facilitate the discussion, we refer to one of the central resources of the Guarino paper, which outlined five candidates for defining events [25]: a fact, a thing, a temporal part of a thing, a property, and a property-instance (tropes or modes). The related paper accepts only the latter one as appropriate. The rejections of the former candidates are illuminating as well, for example, in the justification to reject that an event is a temporal part of a thing, the paper writes: “a ball’s journey is one event and its rotation another, but the present proposal identifies each event with the very same ball-stage, which makes them not two events but one” [25].

For instance, OAE seemed to classify events as property types (instead of property-instances) based on the justification that this “allows the development and application of new analysis methods to better understanding the mechanisms of adverse events associated with or induced by different medical interventions” [18]. This is similar to identifying events by their causes and effects [26]. However, it is difficult to identify causes and effects, especially because they are also events, exhibiting a tautology in the definition. The authors of OAE were well aware of this fact and stated “the assumption of causality in our preceding ontology would imply too large a gap between the ontology and actual practice. Above all, this assumption would make it difficult to use the term ‘adverse event’ to represent individual cases, since the existence of a causal relation is often hard to verify” [18]. Thereby, they imply to consider events as property instances (in contrast to the initial impression). One could assume that classifying events as BFO processes implies that two events should be considered identical if they have the same position in space-time. However, the same position in space-time is not sufficient for determining the nature of events, one needs contextual relations and details for securing identity [25]. For example, at the same position in space-time two events can happen, e.g., incision into a nerve pathway (perspective of the surgery) and functional impairment of nerve signal transmission (perspective of the body). In that regard, events are different from processes such as measuring blood pressure, for which the same space-time position usually provides an unambiguous identity criterion due to the complex chaining of events that are composing the process [24].

To address these challenges, we will utilize the IAO to model events. Our premise is that within the context of IOM documentation, events present greater complexity and are more suitably represented as information entities rather than real-world entities. In this case, establishing references becomes pivotal, achieved through rigidified descriptions [27], such as employing systems like Ceusters et al.‘s referent tracking [28], initiating a chain of references [29]. In this context, the ontology serves to represent the descriptions of events rather than the events themselves, thus evading commitments regarding the nature of events. The distinction can be summarized by two sets: (i) {documentation, identification, property-instance, continuant} with a focus on identification of event-instances for the purpose of labelling and communicating the events, (ii) {explanation, theoretical understanding, property, occurrent} with a focus on understanding the general dynamics of events. There is still an inference mechanism in the former case, this time regarding the coherence of the descriptions.

Methods

We structured the development of the IOM Documentation Ontology (IOMDO) and the associated tool into three distinct phases. The initial phase focused on the ontology’s creation, drawing from the OBO (Open Biological and Biomedical Ontology) principles and guided by Noy’s guidelines [30, 31]. One implication is, for example, the requirement that all entities must have a unique Internationalized Resource Identifier (IRI), composed of a base, as prefix and a local identifier (e.g., BFO_0000001) As we are not intending to be an OBO foundry ontology, we do not stick to the other principles of it, such as using the IAO for sharing metadata. The subsequent phase involved agile software development, a flexible approach to encapsulate the diverse requirements and swiftly produce a prototype. This method allowed us to capture the essential features within a short timeframe. Basic requirements are based on the central use case “To create an IOM protocol at any time after the surgery based on notes taken during the operation (whenever time permits)”. The last phase entailed practical evaluation within real-world documentation settings. This crucial stage enabled us to gather firsthand insights, assessing the tool’s functionality and efficacy. The observations made during this phase formed the basis for essential adjustments to ensure the tool’s productive utilization.

We selected BFO as the foundational framework for our ontology because it provides a robust ecosystem of related ontologies, specifically designed for events, information artifacts, and clinical contexts [32]. Despite potential challenges within this ecosystem, we have opted for strategies to navigate them rather than exploring alternative approaches. With a global presence in over 250 ontology-driven projects, BFO’s generality and adaptability make it an ideal starting point for crafting subject-specific ontologies. BFO introduces a fundamental distinction between continuants, which endure over time and are fully present whenever they exist (e.g., material entities), and occurrents, which unfold over time and exist only through their constituent parts, as exemplified by disease stages. While alternative options like the Descriptive Ontology for Linguistic and Cognitive Engineering (DOLCE [33]), which formalizes linguistic resources such as WordNet [34], and the Unified Foundational Ontology (UFO [35]), which serves as a foundation for OntoUML [36] and the general formal ontology (GFO [37]), exist, they do not enjoy the same prominence in the biomedical domain as BFO. This can be attributed to BFO’s roots in biomedical expert domains, facilitating its swift adoption [38]. BFO’s focus on bridging theory and practice from the outset, rather than delving into intricate formalizations or philosophical concepts like sortal-relativity or rigid designators, has also contributed to its success. As a testament to its practicality, BFO achieved ISO standardization by the end of 2021 [39].

Ontology development

The ontology development unfolded across seven structured steps. Our primary development environment was the Ontology Editor Protégé Version 5.5.0, which provides support for the web ontology language OWL2 (https://I.stanford.edu/products.php). To assimilate relevant entities from other ontologies, we used the web-based tool Ontofox (http://ontofox.hegroup.org/), after identifying those ontologies with applications such as Ontobee (https://ontobee.org) and BioPortal (https://bioportal.bioontology.org). To ensure the integrity and coherence of the ontology after significant updates, we utilized the HermIT reasoner for consistency checks [40]. In the following, the steps are outlined in detail.

Step 1: Determine the scope of the ontology

The ontology’s scope was determined by formulating two core questions:

• What are the use cases for the ontology?

• Who will use and maintain the ontology?

For first question, competency questions were formulated together with the IOM team at the Inselspital hospital, guiding the roadmap for what the ontology needed to address. The second question of who would use and maintain the ontology was addressed initially also through the competency questions. As we delved into requirement engineering during the software development phase, the definitive users and maintainers were firmly established. This process ensured that the ontology’s structure and functionality were aligned precisely with the practical needs and responsibilities of the intended users.

Despite the broad scope of the ontology, which encompasses numerous use cases, this work was anchored in addressing the specific requirements of the Inselspital, with a strong emphasis on integrating research and treatment. It’s important to note that these requirements may vary in other contexts. To promote broader adoption and acceptance, further steps are necessary, following standardized practices within the OBO Foundry framework. As starting from scratch presents challenges, our preliminary efforts serve as foundational building blocks for developing a community-driven ontology that meets the diverse needs of stakeholders across the IOM landscape.

Step 2: Defining the core terms in the ontology

To define the core terms in IOMDO, we first manually analyzed five IOM protocols, primarily to determine the most relevant documented events. The protocols were analyzed based on their structure and recurring terms. Then, we checked for overlapping with the hierarchical catalog of terms developed in former work [5]. For any terms that did not overlap, we decided together with the physicians whether to skip, redefine or merge them. To gain a better insight into the documentation workflow and to validate our results, we also accompanied four experienced medical-technical assistants during surgery at the Inselspital. Observing when and how events were documented was crucial for understanding the dependencies involved. For defining the core terms, we did not rely on SNOMED CT. It provides extensive coverage of clinical concepts. However, as evidenced by the proliferation of clinical ontologies, it often lacks the granularity or specificity necessary for domains like IOM, e.g., the demand of non-medical terms for event documentation. Therefore, a customized ontology can better accommodate the specific terminology, definitions, and relationships pertinent to IOM data, which we gather in this step.

Step 3: Designing the ontology

In this step, we determined the structure of the ontology. As we took BFO as a base, the general structure was predefined. However, it was crucial to define the location and hierarchy of the events in IOMDO. Here, we addressed the question whether to use the occurrents (process boundaries) or the information object concept (generically dependent continuant) for the events. All other entities in IOMDO must be related to these events in terms of documentation requirements. Regarding other terms in IOMDO, we heavily relied on existing ontologies. For instance, we used IAO through its incorporation in OntoSPM (Ontology of Surgical Process Models [41]) and OGMS (Ontology for General Medical Science¹), both of which rely on the Ontology for Biomedical Investigations (OBI [42]). However, as our requirements necessitated the inclusion of new terms and relationships, we ultimately gave the resulting ontology a new name.

Step 4: Ontology Reuse

Reuse of existing ontologies was based on the MIREOT principle (minimum information to reference external ontology terms), which defines the minimum amount of information required for an imported entity to remain uniquely identifiable in the target ontology. MIREOT suggests importing only needed entities and not entire ontologies. While inferences and consistency checks might suffer from such a minimal strategy, integration of complete ontologies can also lead to inconsistencies and undesired side effects, in addition to inflating the target ontology [43]. Since we only used terms from BFO-based ontologies, the minimal strategy was the right way for us.

Step 5: Write annotations and axioms

To enhance the readability and extensibility of IOMDO, we add a definition to each newly added entity. The annotation property “definition” is used for this purpose. In addition to that, some axioms were defined with the goal of improving consistency checking capabilities.

Step 6: Define Instances and their Object & Data Properties

Based on the available IOM controls, we included instances of the classes in the ontology, extending it to a knowledge base (this is not trivial, as it is no requirement for an ontology development project to include instances). Instantiated entities of the IOM ontology are characterized by object properties as provided by the Relation Ontology (RO [44]). Especially, the relations between objects are important for queries about the documented entities. Data related to patients such as date of birth or case number are defined as data properties.

Step 7: Ontology Evaluation (satisfiability and consistency)

The consistency of IOMDO is checked by HermIT, a highly efficient OWL reasoner that can classify a large range of ontologies.² In addition, the software evaluation of our documentation tool provided insights into further syntactic and semantic problems of the ontology.

Software development

An ontology as presented in Protégé or similar environments cannot be used by a non-technical user without costly training efforts (e.g., to generate queries). To abstract from the complexity of the ontology and to facilitate the documentation of cases, a graphical user interface (GUI) was developed. The decision for the programming framework was mainly driven by its capabilities to integrate the ontology into a user-centered application. Five criteria were defined for this decision: supporting of SPARQL and OWL, portability, available documentation, and being license-free.

For the development itself, we first performed a requirement analysis, after which a software concept for the IOMDO software was formulated, separated into frontend and backend. For the implementation of the frontend, we relied on a simple documentation workflow and context-sensitive dropdown menus to select the relevant entries (see also [5]). Regarding the backend, it was important to decide how to persist new instances, how to map the entries in the user interface to the ontology, and how to query the information within the ontology. The reader will find a sequence diagram (Fig. 8) in the Sect. “The IOMDO-based software”, which depicts the components and their interactions.

Software evaluation

To evaluate the software, we performed two types of tests: an input and a usability test. The former one evaluates whether the mapping between user entries and the ontology is valid, allowing to persist all cases as instances of the ontology. For this purpose, the contents of five IOM protocols were entered via the IOMDO software. In the case of missing or inappropriate terms, these were first noted and then added or corrected within the ontology. In the setting of the Inselspital, completeness is ensured by actively involving domain experts in the development process, thereby minimizing the risk of overlooking important entities, and by aligning with existing ontologies. The software was implemented for this purpose as well—to identify missing entities in the future and contribute to the ongoing evolution of the ontology. Encouraging community participation and contributions play a role in enhancing the completeness of the ontology as well. This article aims to foster the formation of a community.

The usability test was conducted with five members of the IOM team at the Inselspital hospital. A test scenario was defined, and the participants were asked to solve tasks in this scenario with the IOMDO software, while thinking aloud. We wrote down everything that was uttered during the test. Additionally, we required feedback by completing a SUS (system usability scale)-based questionnaire and scoring points [45]. A score above 68 was determined to be desirable [46]. Participants had no prior instruction on how to use the interface, which allowed us to assess whether features were used as intended without explanations. Moreover, it helps to identify what is absolutely needed as instruction for a productive use of the software.

Results

The IOMDO ontology

The final ontology is available via GitHub³. Before we describe the outcome of the seven steps in our ontology development phase, a summary of the IOMDO ontology is given (see also Table 1). IOMDO consists of 547 classes, of which 409 are continuants and 138 are occurrents. Central in the continuants area is the Information content entity (ICE) class of IAO⁴, which has the three subclasses: IAO:’data item’, IOMDO:’document’, and IOMDO:’measurement unit’. IAO:’data item’ consists of six subclasses: IOMDO:’IOM start’, IOMDO:’IOM end’, OGMS:’clinical data item’, OBI:’measurement data’, IOMDO:’process observation data’, and IOMDO:’others’. The OGMS:’clinical data item’ was extended with the two subclasses IOMDO:’diagnostic finding’ and IOMDO:’type of surgery’. In the OGMS:’diagnosis’ class, we included 62 diagnoses, 41 of which are based on our own definitions because of the required specificity in the IOM context. For example, the Cerebrospinal fluid leak syndrome has no specificity in ICD-10 with respect to the chest vertebrae that were affected. For additional metrics, we refer to Fig. 2 and tab “DL query” in Protégé, for which a reasoner has to be started before dragging and dropping items of the ontology to the Query text field.

Table 1 Metrics of IOMDO

Outcome of step 1: Scope of the ontology

The following competency questions are reflecting the central use cases and were derived in collaboration with the IOM team at Inselspital:

• 1. How did MEP thresholds/intensities change during surgery, and what is the postoperative outcome?

• 2. What is a patient’s lowest subcortical mapping threshold in correlation to postoperative outcome?

• 3. How are signal changes related to tumor pathology?

In other words, the ontology’s scope encompasses the documentation of events during surgeries utilizing IOM, thereby enabling inference of postoperative outcomes. The ontology will be used by two types of users: those who derive knowledge from the ontology (mainly physicians) and those who document during surgery (mostly medical-technical assistants). After finishing the IOMDO ontology, the SPARQL queries corresponding to the questions were defined and issued.

Outcome of step 2: The core terms in the ontology

The analysis of the five IOM protocols and the on-site observations during surgeries showed that the core of the documentation are measurements with the following components:

• Time stamp: The time at which the measurement takes place.

• Measurement type: Indicates which potential is measured and how it is stimulated.

• Measured value: The measurement response of the current intensity in mA.

• Measurement location: The muscle or nerve where the measurement takes place.

All other entities are associated with these measurements. The relevant section regarding measurements in the ontology is shown in Fig. 2. Notably, OGMS: ‘clinical data item’ has been expanded to include two additional subclasses, namely IOMDO: ‘diagnostic finding’ and IOMDO:‘type of surgery’. All subclasses within OBI:‘measurement data’ are specific to IOMDO. For instance, IOMDO:‘process observation data item’ denotes an observation during or after live IOM documentation deemed significant by the documenting individual. IOMDO: ‘document’ and IOMDO:‘unit’ represent central document and measurement units, respectively, as defined by IOMDO, such as mA.

Fig. 2

The relevant section regarding measurements related to continuants in IOMDO

Outcome of step 3: The ontology design

The conclusions of the results of Sect. 2 were: not to rely on OAE and to base IOMDO on other BFO-based ontologies. As we decided to model events as generically dependent continuants in BFO, we adopted the ICE class as it is integrated in the ontology for surgical process models (OntoSPM), which itself adopted it from IAO. In the context of documentation, an event is a continuant information content entity with reference to an occurrent real-world entity (an extensive discussion on occurrents versus continuants is provided in [47]). For the purpose of rigid designation, the documented events are described by several hic-et-nunc properties such timestamp, documented_by, and located_in. To further characterize these events, we created the entity ‘diagnostic finding’. In contrast to the entity ‘diagnosis’, each striking event is captured with this entity and not the causal reason for the operation in terms of the ICD code. Processes are used in IOMDO only to indicate actions, which is motivated by the decisions for OntoSPM.

Outcome of step 4: Ontologies reused

For anatomical terms such as nerves and muscles, we referred to the Foundation Model of Anatomy (FMA⁵). For disease terms, the main sources were the Human Phenotype Ontology (HPO⁶) and the Mondo Disease Ontology (MONDO). Terms related to surgery were extracted from OntoSPM, which represents a core ontology for surgical processes in general and focuses on actions and their participants [41]. OntoSPM builds heavily on existing ontologies such as the Information Artifact Ontology (IAO), which itself is using terms the Ontology of Biomedical Investigations (OBI), and the FMA. Due to such reusages, IOMDO includes in the end many concepts of IAO and OBI.

As measuring is a relevant concept, we also considered to re-use ontologies focusing on measurements. EngMath is an ontology for mathematical modeling and engineering, covering aspects of measurements primarily from the perspective of physical and philosophical measurement theory [48]. The Unified Code for Units of Measure (UCUM) is intended to include all units of measures being contemporarily used in international science, engineering, and business, thereby facilitating unambiguous electronic communication and exchange of quantities together with their units, which are “abstract spaces used as a reference metrics for quality spaces, such as physical qualia, and they are counted by some number” [49]. The most promising ontologies for us seemed to be the “Quantities, Units, Dimensions and Types” Ontology (QUDT, see [50] for an embedded presentation of it) and the Ontology of units of Measure (OM [51]). Both provide important insights, e.g., that a basic record consists of (1) the phenomenon: object or event being observed (surgery), (2) the quantity kind (electric signal during surgery), (3) unity of measurement (mA), and (4) a numerical value.

We also performed a search on ontologies covering reporting of events in the biomedical context beyond OAE. Many of these ontologies are basing their efforts on OAE, e.g., with respect to Drugs (ODAE [52]) Vaccine AEs (OVAE [53]), Drug Neuropathy AEs (ODNAE [54]), and Cardiovascular Drug AEs (OCVDAE [55]). For the same reasons provided regarding OAE and due to their specific perspectives, these ontologies were not useful for our case. Other biomedical ontologies are often older, not BFO-based and not maintained, for example, the clinical communication ontology for medical errors (OCCME [56]), ontology of Adverse Drug Reactions for Automated Signal generation Ontologies (OADRAS [57]), and the Adverse Drug Event Ontology (ADEO [58]). In conclusion, there seems to be no alternative to an BFO-based ontology for reporting events, which makes our work important from this point of view. Again: one should differentiate between an ontology for events and an ontology for reporting events, as the entities involved are different.

Outcome of step 5: Annotations and axioms

All entities in the IOM ontology have English and German labels and definitions. If available, the source of the definition is indicated with the annotation ‘definition source’. We added two axioms in Manchester OWL Syntax: for the entity ‘measurement data item’, the axiom “’part of’ (is) some ‘diagnostic finding’” is defined, and for the entity ‘diagnostic finding’ its inverse “’has part’ (is) some ‘measurement data item’” is defined. In addition to that, several AllDisjointClasses axioms were formulated, e.g., the type of the potential evocation must be disjunct. They are referred to by their IRIs.

Outcome of step 6: Instances with Object Properties and Data Properties

Creating instances in Protégé involves navigating the ontology tree to locate the entities to be instantiated. Once an entity is selected, a new instance can be added and named. Establishing the relationships between instances is a meticulous process, susceptible to errors due to the numerous possibilities that must be manually selected. In case of a wrong relation instance, the reasoner will issue an error message and the query will not produce correct results. In our IOMDO software, the user has only to enter the relevant events during an operation as well as the corresponding measured value. Relationship instances are created automatically based on the implemented logic. Hence, during instance creation, we used one important advantage of the IOMDO software compared to Protégé.

Five Object Properties have been defined to allow querying of the instances in the ontology. The ‘Patient’ and ‘Surgery’ entities have Data Properties that contain patient and surgery information, such as case id or the surname. Because some of the data properties are directly related to a patient, we did not rely on the ICE, and because they could be irrelevant or unavailable in other IOM settings, we did not use the preferred OBI approach of defining them as core qualities. We are aware of potential risks, as a consistency check in the initial phase showed an error: the ICE class diagnosis had the data property ‘has_assistent’, which rather belongs to the class surgery. Nevertheless, in analogy to the justification of preferring schema-free over schema-based databases due to the increased flexibility, we stick to our approach.

Outcome of step 7: Ontology evaluation (satisfiability and consistency)

Initially, the relation (object property) “has measurement unit label” from OntoSPM was used to set a relation between a measurement datum and a unit. Due to its functional characteristics, there was a conflict with our modeling (see in Fig. 3). A functional property means that for a given individual, there can be at most one value. However, we decided that a measurement datum (e.g., MEP measurement) can have multiple measured values, because this adequately represents the way the measurements are documented. This inconsistency was immediately detected by HermIT Reasoner when a MEP measurement had more than one measured value with its corresponding measurement units. This led to the solution of a new object property “has measurement unit” without the functional characteristic.

Fig. 3

Results of the HermIT Reasoner in connection inconsistencies

The IOMDO-based software

The IOMDO tool supports the usage of the ontology at two levels: ontology-based guidance of users who document events by dynamically adapting the view based on the relations of the current data item to other entities in the ontology. Second, to formulate queries on the accumulating knowledge base. The focus is on the former level. As the tool is a standalone JAVA-application, it must be installed from the sources.⁷ The tool is designed to support the documentation in a German-speaking Swiss hospital, which required to translate ontological terms from English into German and to add German labels in IOMDO.

The basic functional requirements are:

• The system should allow for the display of previously created protocols, including protocol number, name, surname, diagnosis, and type of operation.

• Users should be able to search for all case data via a search field.

• Protocols should be viewable and editable retrospectively.

• The system should support two different types of queries: simple queries, allowing users to search for a specific class or instance, and complex queries.

• New levels should be added and deleted via a button.

• Users should be able to export created queries for later use.

• The ontology should be expandable by a non-technical user.

• The ontology expansion process should be divided into multiple steps to allow for the categorization of the desired term.

The non-Functional requirements are:

• The software must be available at all times.

• The ontological concepts must be presented in a simple form for the experts.

• Performance and usability have to ensure quick documentation. Protocols should be recorded in less than 30 min.

Table 2 gives an overview of the (backend) frameworks we considered and evaluated according to our five criteria: (i) support of SPARQL, (ii) support of OWL, (iii) portability, (iv) documentation, (v) type of license. Apache Jena was assessed to be the only tool to offer a framework that is flexible enough to cover our use case and allows seamless integration of a GUI with an ontology backend. For the frontend, we opted for JavaFX, which could easily be replaced by other Java frontend frameworks.

Table 2 Evaluation criteria for three frameworks for ontology handling in JavaScript and Java

Based on our requirement analysis for the frontend, we decided to consider five tabs in the GUI: record protocol (default), view protocol, create query, view ontology, and user guide. In the “Record Protocol” view, the data to be recorded is divided into four categories:

• Patient and surgical data: Data related to general patient characteristics and the reasons for the surgery (will be imported in most of the cases).

• Baselines: The basic measurements of evoked potentials (e.g., MEPs) at the beginning of a surgery. Not all of them are needed in every surgery.

• Recorded events: This is where the actual recording takes place during or after surgery. Text can be entered freely in the fields “time”, “value” and “comment”. Dropdown lists are available for the fields “category” and “subcategory. The appearance of the fields “subcategory” and “value” is context dependent.

• Postoperative disposition: A dropdown list is provided to indicate whether a complication has occurred after the surgery, which must be connected to it.

In the example of Fig. 4, the data displayed consists of dummy patient information. On the left side, there are five selectable views. Currently, the ‘record protocol’ view (‘Protokoll erfassen’ in German) is chosen. The data on the right side pertain to the patient (from top to bottom: No., Case ID, Patient ID, Surname, First name, Birthday) and the surgery (Diagnosis, Surgery, Device, Surgery Date, Surgeon, Assistant). Below, collapsed segments allow for the recording of various baselines and reflexes. At the bottom, the expanded area is used to record events in five columns: Time (Zeit), Category (Kategorie), Subcategory (Unterkategorie), Value (Wert), and Comment (Kommentar). In this example, five events are logged: ‘IOM Start’, ‘surgical process’ (Operationsprozess) with the subcategory ‘incision’ (Schnitt), two instances of ‘mapping measurement’ (Mapping Messung) with the subcategory ‘dynamic measurement’ (Dynamisches Mapping) and two measured values in mA, and an ‘IOM End’ (IOM Ende). Additionally, ‘postoperative Hypertension’ is selected as a postoperative complication/ disposition (postoperative Komplikation) at the bottom.

Measurements always require a value entry in mA. To end the recording, “IOM end” is selected. After clicking “Save” the record is saved within the ontology in the RDF/XML syntax. The saved RDF/XML can additionally be viewed in Protégé (see Fig. 5: On the right side are displayed all object properties and their respective instances connected to the instance of the class ‘IOM document’. From top to bottom, the object properties include ‘document of’ (Dokument von) and ‘has data item’ (hat Datenelement). The instances listed on the right side encompass Clinical Data item, DCS MEP Baseline measurement (DCS MEP Baseline Messung), ‘dynamic mapping measurement’ (Dynamisches Mapping Messung), ‘glioma’ (Gliom), IOM Start, ‘IOM End’ (IOM Ende), IOM Document, ‘Craniotomy, resection and nervus opticus decompression’ (Kraniotomie, Resektion und Opticus-Dekompression), mA, ‘observation’ (Observation), patient, postoperative Hypertension, ‘incision’ (Schnitt), and ‘TES MEP Baseline Measurement’ (TES MEP Baseline Messung)).

Fig. 4

Graphical user interface for documenting events in the IOMDO software

Fig. 5

Saved RDF/XML of the documented events in the IOMDO software file as displayed in Protégé

Figure 6 visualizes the core concepts of IOMDO and their application in entering records into the ontology via the GUI. The entity ‘Patient’ is in relation ‘has document’ with the entity ’IOM document’. An IOM document consists of several data elements, to which the items entered in the GUI are mapped. For example, ‘dynamic mapping measurement’ in the GUI is mapped to ‘measurement data item’ in the ontology. The ‘surgery data’ in the GUI is mapped to the entity ‘surgery’, which is subsumed to the “clinical data item” entity. The item ‘incision’ (in German “Schnitt”) in the GUI is subsumed to “surgical process”. The entity “process observation data item” is needed as a link between a process and the data item of the document.

Fig. 6

Core concepts of IOMDO and their application in entering records into the ontology via the GUI

In the “create query” tab of the GUI (mostly used by physicians), two different types of queries are provided: predefined queries for the non-technical user (the query is executed in the background) and SPARQL queries for more technical users. To execute the first query type, a button at the end of a short description must be pressed, producing the results in a text field of the GUI, e.g., a list of all patients for whom adverse events were described. Two queries are predefined queries: (1) Show all patients (name, surname, date of birth), their pathology, their postsurgical outcome and the corresponding DCS-measurements with the names of the muscles (Zeige alle Patienten, ihre Pathologie, ihr postoperativer Outcome und die jeweiligen DCS Messungen inklusive Muskeln) and (2) Show all patients, their diagnosis, their postsurgical outcome and the lowest mapping threshold (Zeige alle Patienten, ihre Diagnose, ihr postoperativer Outcome und die tiefste Mapping Schwelle). In Fig. 7, the second query was executed and leads to information about the patient (last name, first name, date of birth), the diagnosis, the postoperative outcome, and the lowest mapping threshold. For the second query type, the user must formulate a valid SPARQL statement, which is then again executed by pressing the button at the right bottom corner next to the text field. Results can also be exported as csv-files.

Fig. 7

The two query types in the IOMDO tool GUI. Predefined ones at the top and SPARQL queries at the bottom. On the left side, the create query view (Abfrage erstellen) is selected. On the upper right side of the view, the user can execute a predefined query, which will be executed in the backend by clicking the button execute (Ausführen)

The sequence diagram in Fig. 8 shows how the user interaction with the GUI is forwarded to the backend to store and query data in the ontology. The process is designed to be highly adaptable, allowing for the seamless integration of measurement signals into the IOMDO software, e.g., through IEEE 11,073 Service-oriented Device Connectivity (SDC). This integration streamlines data acquisition, leading to more efficient documentation processes, as everything operates within a unified system. In the long term, it is crucial to incorporate signal data into the hospital information system (KIS) using HL7 FHIR, and work has already begun in this direction [59].

Fig. 8

A sequence diagram showing how the user interaction with the GUI is forwarded to the backend to store and query data in the ontology

Software evaluation

The input test and the observations during the usability test allowed us to address deficits in the mapping between user requirements and the ontology, e.g., we initially did not consider that in the case of selecting metastasis as a diagnosis, it is crucial to add the location as additional information. For the usability test, 4 of the IOM team members were included. The average SUS score achieved was 75, with the best individual score being 97.5 and the worst being 50. One central reason for the worst score was a general discomfort with dropdown lists: scrolling down was rated as cumbersome and inconvenient when the list includes more than five items. A searchable list could address this issue in the future. However, all participants agreed that the interface could be used without having extensive technical skills.

Discussion

The development of the ontology benefited from a graphical user interface (GUI) that surpasses the capabilities of Protégé, as the latter, while powerful for knowledge engineers, may present usability challenges for non-technical users. This GUI enables users to perform various tasks, including suggesting new terms or relationships, validating existing concepts, and reviewing ontology changes. Moreover, involving users in ontology development fosters a sense of ownership and investment in the tool, thereby increasing user acceptance and adoption. We offer two ways of issuing queries to the ontology: a simple interface and a SPARQL-based approach. The latter is designed for advanced users or researchers who require more complex querying capabilities. In our experience, statistically oriented users often desire more flexibility in querying without the need for technical ontology creation. Even as ontology developers, we find this to be a quick and efficient querying method. Balancing the tool’s usability for non-technical users with advanced functionalities for power users is a common challenge in software design. However, our long-term goal is to create more templates for SPARQL queries for commonly occurring requests. It is crucial to strike a balance that caters to the diverse needs of the user base.

Was it necessary to develop an ontology for our use case? Our experience with the catalog for IOM documentation prompted us in that direction, and the primary reasons for this decision were twofold: first, to align with established standards such as IAO, thereby leveraging the insights embedded within this standard, and second, to establish comprehensive relationships between entities to enhance internal consistency. Consequently, the mere adoption of a pure data format standard like FHIR proved inadequate. While it may offer practical insights from well-established resources, its emphasis lies more on supporting interoperability alone. In contrast, ontologies offer a formalized representation of knowledge within a specific domain, encompassing concepts, relationships, and axioms. They facilitate cross-domain integration and promote interoperability among various healthcare data sources, enabling more thorough analysis and decision-making. Additionally, an ontology enables more nuanced analysis and interpretation of IOM data, which was highly valued by the clinicians in our project.

One further step in IOM documentation is to allow classification of the time series of evoked potential measurements (not only their manual registrations, which happen in discrete intervals), which would increase the utility of such a system in terms of knowledge discovery and documentation comprehensiveness. Such time-series data can also be analyzed without ontological annotations, and this should be done in a preliminary step. However, without ontological embedding, integration of IOM measurements from different studies and standardization of analysis result interpretations would be difficult. In other words, the advantages of semantic interoperability should be used for data analysis, leading to concepts such as ontology-based machine learning [60]. As a provider of such an all-in-one solution with an analysis component at the end of the whole IOM pipeline, an IOM medical device supplier seems promising, as it already covers many challenging parts of such a solution. Here, the main difficulty lies in translating the expertise with respect to ontologies and in the required change of the existing backend structure into an ontology-based one.

For neurophysiological and neurosurgical researchers in our setting, the requirement to consider time-related measurement data is related to many different use cases. As described above, IOM is intended to provide an early warning signal to determine when tumor tissue removal must be stopped to avoid damage to important structures such as the corticospinal tract. One important use-case for the analysis of time-series data of evoked potentials is the evaluation of such warning signals with respect to maintaining postoperative motor function. For example, Seidel et al. analyzed a cohort of 100 patients undergoing tumor surgery with simultaneous subcortical motor mapping and DCS-MEP monitoring [61]. In a further work, they developed a new acoustic motor-evoked potential alarm as a mapping suction probe, which works like a radar on motor pathways [16, 62]. Even though some warning criteria have been identified and evaluated through such work, there is no consensus regarding the cut-off values and interpretation of MEP signal changes and alarm criteria vary among centers [15]. By creating a knowledge base for such studies conducted at many clinical centers, achieving consensus, and promoting the generation of new insights, e.g., related to repetitive signal changes before certain events, will be much easier.

We have already begun to analyze the data without embedding related annotations into our ontology, for example, by using Fourier transformation for analyzing event patterns. However, without embedding the time-series data structure into our ontology, the potentials for associating different types of information and allowing in-depth interpretations are reduced. Especially for a large amount of data (about 200 MB of measurement data per surgery), knowledge representation facilitates the organization and processing of the data (storing, querying, and analyzing). If the data cannot be efficiently processed with a local computer system, the ontology should be embedded into a big data environment. There are, for example, works related to the use of the big data Hadoop framework with Apache Hive [63] or HBase [64] for storing and querying ontologies.

Conclusion

In summary, through the establishment of a structured and standardized framework for characterizing IOM events, our ontology-based tool holds the potential to enhance the quality of documentation, benefiting patient care by improving the foundation for informed decision-making. Furthermore, researchers can leverage the semantically enriched data to identify trends, patterns, and areas for surgical practice enhancement. The integration of established ontologies ensures alignment with broader medical knowledge frameworks for that purpose. Central to realizing these benefits is the engagement of experts in IOM and the user-friendly nature of the software tool. However, to optimize documentation through ontology-based approaches, it’s crucial to address potential modeling issues that are associated with OAE.