Skip to main content

Replicating medication trend studies using ad hoc information extraction in a clinical data warehouse



Medication trend studies show the changes of medication over the years and may be replicated using a clinical Data Warehouse (CDW). Even nowadays, a lot of the patient information, like medication data, in the EHR is stored in the format of free text. As the conventional approach of information extraction (IE) demands a high developmental effort, we used ad hoc IE instead. This technique queries information and extracts it on the fly from texts contained in the CDW.


We present a generalizable approach of ad hoc IE for pharmacotherapy (medications and their daily dosage) presented in hospital discharge letters. We added import and query features to the CDW system, like error tolerant queries to deal with misspellings and proximity search for the extraction of the daily dosage. During the data integration process in the CDW, negated, historical and non-patient context data are filtered. For the replication studies, we used a drug list grouped by ATC (Anatomical Therapeutic Chemical Classification System) codes as input for queries to the CDW.


We achieve an F1 score of 0.983 (precision 0.997, recall 0.970) for extracting medication from discharge letters and an F1 score of 0.974 (precision 0.977, recall 0.972) for extracting the dosage. We replicated three published medical trend studies for hypertension, atrial fibrillation and chronic kidney disease. Overall, 93% of the main findings could be replicated, 68% of sub-findings, and 75% of all findings. One study could be completely replicated with all main and sub-findings.


A novel approach for ad hoc IE is presented. It is very suitable for basic medical texts like discharge letters and finding reports. Ad hoc IE is by definition more limited than conventional IE and does not claim to replace it, but it substantially exceeds the search capabilities of many CDWs and it is convenient to conduct replication studies fast and with high quality.

Peer Review reports


Reliable information on the use of medication in a hospital and its changes over time is of great importance for many acute and chronic diseases – from a hospital, patient and payor perspective. This is reflected by many studies reporting medication trends: e.g. attention deficit hyperactivity disorder (ADHD) [1], atrial fibrillation (AF) (US [2], Denmark [3, 4]), chronic kidney disease (CKD) [5, 6], rheumatoid disease [7] or hypertension (HT) [8] (England [9], France [10], Germany [11], Sweden [12], US [13, 14]).

However, medical research (like many other disciplines) is affected by the so called replication crisis, addressed in an article in 2012 reporting that only 11% of the pre-clinical cancer studies could be replicated [15]. The Nature Journal conducted a survey of 1500 scientists in 2016, in which 70% of them stated that they had failed to reproduce another scientist’s experiment [16].

The ability to reproduce findings reported in a clinical study is a cornerstone of scientific progress. Replication of medication trend studies can be performed using a CDW, which is an important, albeit little exploited and published use case.

CDWs can deal with structured data very well. Unfortunately, a lot of the patient information in the electronic health record (EHR) is still stored in free text. E.g. Jensen et al. retrieved on average 146 unstructured text documents for each patient from EHR of their hospital for their study [17]. Medication, too, is usually documented as free text within the discharge letter. As a solution, advanced CDW systems offer a query language that can extract data from free text (e.g. in [18]).

The conventional approach is to perform information extraction (IE) in the ETLFootnote 1 process. A well-known system for IE of medication is MedEx [19]. Beside other rule based-systems like [20], hybrid systems exist using machine learning techniques [21]. A good overview on IE from free text is given by Wang et al. [22].

Rule based systems require a high volume of handcrafted rules and learning systems need a large amount of manually labeled training data. Either way, a lot of expert work is necessary. Besides high developmental efforts, another disadvantage of conventional IE is its slow promptness and non-adaptability by users [18].

A novel way to retrieve information from plain text is ad hoc IE. Ad hoc IE is described as extracting the existence of any concepts (e.g. chronic kidney disease) or any numbers, like the left ventricular ejection fraction (LVEF) value, from textual sources in real-time. The Boolean ad hoc IE queries the existence (yes/no) of a medical concept. A medical concept is a named entity that may have a feature/property or a numeric value. Examples of Boolean concepts are single findings or assessments (e.g. moderate mitral insufficiency, severe aortic stenosis), drugs (e.g. Aspirin, beta blocker) or diagnoses (e.g. appendicitis, myocardial infarction). Numeric IE extracts the value as number of a numerical concept. That could be for example the value of a laboratory finding (e.g. cholesterol, glucose, LEVF) or a derived values/indexes (e.g. BMI, age). A numerical condition can be defined optionally, like LVEF <45, matching all mentions of LVEF with a value lower than 45. In some finding reports, the exact value of a concept is not given but there is a formulation indicating an interval or an inequality of a value (e.g. “LVEF lower than 45"). These statements can be queried in conjunction with numeric ad hoc IE exploiting both qualitative and quantitative information from textual reports e.g. for checking inclusion or exclusion criteria of studies. In addition to count queries, which only asses the presence of a concept or the validity of constraints (e.g. BMI >25), the actual values can also be returned for further processing.

This technique showed good results and requires little developmental effort, since the text is indexed efficiently and can be queried with powerful features [18].


This work introduces ad hoc IE for medication and their daily dosage from hospital discharge letters. We present and evaluate query features for a CDW. As an example of use, we show medication trend estimations. Therefore we replicate existing studies from the literature in a large CDW of the University Hospital of Würzburg using ad hoc IE. The results will be compared with the corresponding published data describing similarities and differences.


The developmental steps included extensions and features for the data integration process and the development of new data query tools. For study replication, the drug names had to be acquired and transformed.

CDW system design

We implemented our features in the PaDaWaN CDW [23], which uses the full-text-search engine Apache SolrFootnote 2 as storage engine, based on the index library Apache LuceneFootnote 3. The PaDaWaN-CDW contains both, unstructured text data and structured data, including core data (e.g. age, sex etc.), coded data (e.g. ICD10 and OPS etc.) and numerous other types of information of the clinical information system (CIS) (e.g. lab data) [18]. The data integration process of the PaDaWaN-system contains analyzers for the respective data types. At the end of the pipeline, all values are stored in the Lucene index and can be queried from physicians in the PaDaWaN Web GUI [23]. We modified and extended generic tools for text analysis in the import pipeline (see below). We also added new query features to the framework, which can be used in the front end GUI during runtime.

Data integration development

Lexical analysis

The text analysis tool for discharge letters splits the text into sections like diagnoses, medications, and laboratory values. Figure 1 shows an example for a medication section. We added a sentence splitter for medication extraction that separates the individual medication instructions from each other. Furthermore, we deactivated the stemmer because the word endings of the medications should not be touched. Finally, a custom tokenizer ensures that the quantity, strength and dosage information of the medication instructions are correctly decomposed. Table 1 shows an example of the lexical analysis.

Fig. 1

Example of a medication section of a hospital discharge letter. a German. b English

Table 1 Lexical analysis of the medication section in the discharge letter

Context of information

The context of information in a discharge letter is an important topic. Many pieces of information are negated [24] (e.g. “no fever”, “dizziness is denied”) or they relate to other persons (e.g. within the context of family history). Some information like medications within in the discharge letter have a temporal context and may not be valid any longer (e.g. medication might have been stopped at hospital entry or during hospitalization, like Ramipril in Fig. 1). Depending on the application or evaluation, different types of information are relevant or must be excluded. In most cases, physicians are interested in the confirmed and current findings of a patient.

The PaDaWaN data integration process already identifies negations in the texts with an extended version of the NegEx-algorithm [25]. These negations can be excluded in the GUI for certain queries like medication extraction [18]. We extended this NegEx-version to a ConText [26] implementation. This algorithm handles not only negations but also the context of an information. It is implemented using Apache UIMAFootnote 4. Furthermore, we added several trigger tokens for the patient history.Footnote 5 Using these modifications, the non-currently used drugs are excluded from the text. The remaining, relevant medications remain retrievable at runtime by user queries.

Text query features

Spelling error tolerant query

PaDaWaN already contains several text query features like token, phrase and regular expression queries. Since medical reports are often manually entered, some names of medications are misspelled. For such typos we added a spelling error tolerant query feature that makes use of the Damerau-Levenshtein distance. It is a string metric for measuring the edit distance between two sequences and can thus be employed to assess how much two medication names differ. The distance measures includes a transposition operation (transposition of two adjacent characters) in addition to three edit operations, i.e. insertion, deletion, and substitution [27]. Table 2 shows selected examples of misspellings and their Damerau–Levenshtein distance to the product name.

Table 2 Examples of misspelled medication names and their Damerau–Levenshtein distance

Dose extraction with proximity search

Although most medication trend studies only consider the use of a drug, we also strived to extract the daily dosage of the medication. This requires two pieces of information: the strength and the cumulative daily amount of the drug. The strength is given in digits with a standard unit (usually milligrams or micrograms) with the drug name. The dosing interval is usually coded by a number-hyphen notation like 1/2-0-1/2. The numbers represent the units that must be taken in the morning, at noon and in the evening. A optional fourth digit refers to the number before going to bed. The daily dose is obtained by adding these three or four numbers and then multiplying by the strength. We added a feature that makes it easier to query the daily dose. The proximity query searches the given tokens next to each other. The order of these tokens is irrelevant. Proximity queries do not match across sentence boundaries. Since each medication instruction is provided in a segmented fashion as a single sentence during the import, proximity queries do not match dosage information of other medications. Table 3 shows an example of how a daily dose can be extracted. The corresponding request is displayed as well as matching and not matching text snippets. With this technique, queries can be made for the different drug strengths and daily dosages.

Table 3 Example for promximity searches to query the daily dose of a medication instruction

Query token generation

The Anatomical Therapeutic Chemical (ATC) Classification System is an international classification of active ingredients of drugsFootnote 6. In the literature, ATC codes are used to encode drugs and active agents groups. In order to get all brand, drug and agent group names of an ATC-group like C07 Beta Blocking Agents, we use the ABDA-DBFootnote 7, which contains all names in English and German. Since medical reports rarely contain the full name of a drug, we processed the names from the ABDA-DB in various ways: a) names were simplified by omitting the names of the manufacturers and the strength of the drug; b) other unnecessary words were removed; that includes modifiers concerning the effect like forte and the administration form like oral; c) abbreviations and alternative spellings were considered. Table 4 shows examples of the processing of drug names. The resulting tokens were used for the queries. Hyphens do not need to be treated because they are removed by the tokenizing procedure.

Table 4 Example for the processing of the drug names


We performed tests to evaluate our development and conducted case studies aiming to replicate findings reported in selected medication trends studies.

Medication extraction

Since medication studies only consider the use of drugs, the replication requires just Boolean IE. Therefore we carried out a comprehensive test. We further evaluated the requests for the daily dosage using ad hoc IE. To protect privacy, these texts were de-identified and in addition they must not leave the clinical network.

Extraction of drugs. For the evaluation of the medication extraction 600 documents were randomly selected from the disease domains hypertension, atrial fibrillation and chronic kidney disease. From each domain, 100 medication sections from 2005 and 100 sections from 2015 were sampled, resulting in a total of 600 documents. A manually annotated gold standard was created for these documents. All medications, brands, drug and substance names were annotated using the Apache UIMA CAS type system. In order to save time, the text was first automatically pre-announced using the medication tokens gained in “Query token generation” section. Then, the texts were manually corrected to obtain the gold standard. The ATHEN environmentFootnote 8 was used to perform this work [28]. Afterwards the original texts were imported into the PaDaWaN-CDW with the data integration pipeline. Then queries were made with all drug names and the hits detected were annotated. At the end, all hits found by the system were compared to the gold standard.

Daily dosage. The extraction of the daily medication dosage was evaluated with several drugs: Antihypertensive drugs: Esidrix (Thiazide-Diuretika, ATC: C03A), Concor (β-blocker, C07A), Delix (ACE inhibitor C09A) and novel oral anticoagulants (NOAC) used for atrial fibrillation: Eliquis, Pradaxa, Xarelto. For each drug, 100 medication sections containing this drug from 2015 were selected. For the antihypertensive drugs another 100 units were selected for the year 2005. This was not possible for the NOACs, since they did not exist at that time. Queries were made in the PaDaWaN system and evaluated manually. For the evaluation, all dose strengths were extracted. The proximity query feature was used to extract the dose.

Study replication

To evaluate the quality of the study replication, we chose five studies from the literature covering three domains (hypertension, atrial fibrillation, chronic kidney disease) and compared the major and sub-findings with the results of the University Hospital of Würzburg in total, respectively restricted to its Department of Internal Medicine I (Med1) using the ad hoc query feature with of the CDW. The drugs were extracted from the medication section of the discharge letter. That contains in almost every case the medication at discharge representing the recommended / prescribed medication. Additionally the medication at admission is described in 18% (Med1: 13%) of all cases. At discharge from hospital, patients receive 8% (Med1: 19%) more medication than at admission, while nearly all medications from admission were continued at discharge. (Tested for the main drug agent groups for hypertension.) We used the whole medication section with all medication descriptions as data source to identify weather a drug is taken or not.

This was conducted with the PaDaWaN-CDW including about 1 million patients with 5 million patient cases and more than 600 million pieces of single information. We applied the same in- and exclusion criteria as in the respective publications. However, we did not compute age-adjusted values. Not every single evaluation in the publications was reproduced; we rather focused on the main statements and central result tables of the studies or took the most interesting parts of the publications to show the power of our approach.

Hypertension We chose [13] as first drug trend study, because it is a highly cited study addressing a large population. The analyzed data was acquired during the National Health and Nutrition Examination Survey (NHANES) [29]. We further aimed to replicate the results of Shah and Stafford [14] concerning the findings on systolic blood pressure. These authors used data from the National Disease an Therapeutic Index (NDTI), a nationally representative physician survey. We extracted this information from the discharge letter via numeric ad hoc IE [18].

Atrial Fibrillation. In the replication of the study for atrial fibrillation [3] the ad hoc IE from unstructured texts was combined with structured data from the CDW and differentiated according to these. Subgroups such as comorbidity and age groups were investigated by Gadsbøll et al. [4]. The data sources of these studies were the Danish National Patient Registry, the (Danish) National Prescription Registry and the (Danish) Civil Registration System, containing various information on all prescriptions dispensed in Danish pharmacies since 1995.

Chronic Kidney Disease. We also selected a study to examine temporal trends and treatment patterns by patients with CKD and type 2 diabetes mellitus (T2DM) [5]. In this work, medication groups are evaluated. In a more detailed analysis, CKD was broken down into different severity levels (stages), and the medicative effect of the medication groups was considered [5]. This study also used the data from NHANES.

Tables 5 and 6 map all drug and diagnostic group designations used in respective publications to ATC and ICD10 codes, respectively. These codes were used for the replication of these studies. Table 7 summarizes the replicated studies and shows their inclusion and exclusion criteria.

Table 5 Mapping between diagnostic group designations used in the literature and ICD10 codes used for the replication
Table 6 Mapping between drug group designations used in the literature and ATC codes used for the replication
Table 7 Overiew of replicated studies and their inclusion and exclusion criteria


Ad hoc IE evaluation

Extraction of drugs

Table 8 shows the performance of the ad hoc extraction of medications with an overall F1-score of 0.983 (precision 0.997 and recall 0.970).

Table 8 Performance of the ad hoc extraction of medications

Most errors were caused by abbreviations. The misspelling based errors could be significantly reduced by the error tolerant query feature. Table 9 shows the error analysis of the ad hoc extraction of medications. The most common occurrences of the error groups are shown below. Abbreviation Fraxi (20), Tiotropium (6), Mg Verla (4), Dreisavit (3), Dabigatran (2), Insuman (2), Isosorbid (2) Not in DB Eunerpan (9), Polybion (4), Aclidinium (2), Calcetat (2), Natriumperchlorat (2), Cranoc (2), Calcetat (2) Alternative notation Glycopyrronium (2), Dikalium Clorazepat (2), Humaninsulin (1), Diuretikum (1), Ca Carbonat (1) Misspelling Ferrosanol (4), Eins alpha (2), Amphomoronal (2), Beclometasondipropionat (2), Klazid (2), Rehnagel (2), Cardular (2), Calciumdiacetat (2) Search to fuzzy diabetes ≈ diabetex (4), diagnostik ≈ diagnostika (1), antihypertensiven ≈ antihypertensives (1) Incorrect extracted medication thrombozyten (1), cholesterin (1), albumin (1), kalium (1), natrium (1)

Table 9 Error analysis of the ad hoc extraction of medications

Extraction of daily drug dose

An analysis on the data set for the daily dose, that contains 900 mentions of selected drugs, revealed that 5% of the mentioned drugs were discontinued or reduced. 90% had an indicated strength, 92% an instruction and 89% a strength and an instruction. See Table 10.

Table 10 Presence of strength and instruction application of medication in the evaluation set

The most common daily taken dose was one unit (57%) followed by two units (31%), see Table 11.

Table 11 Summed daily dose of the medication units in the evaluation set

The overall F1-score for the extraction of the daily medication dose was 0.974. The precision was the same or slightly higher than the recall in all tests. The extraction results were slightly better on the antihypertensive drug set (F1: 0.982) than on the NOACs drug set (F1: 0.958). The documents from 2015 also showed slightly better results than those of 2005 (F1: 0.977 vs 0.968). The complete results can be found in Table 12.

Table 12 Performance of the ad hoc extraction of the daily medications dose

Most errors were caused by an unusual notation. See Table 13 and listing below. Other error sources were supplements, which contained numbers, incorrect splitting of the tokenizer, double mentions in same document, segmentation faults, and a too wide gap between the drug name and the instructions. Notation Esidrix 1x1, Pradaxa 150-0-150 mg Supplement Pradaxa 110 mg 1-0-1 (bitte 1 Tag vor stationären Aufnahmetermin pausieren); Tokenizer EuthyroxDouble mention Medikation bei Entlassung: Esidrix 12,5 mg 1-0-0; Medikamente bei Entlassung: Esidrix 25 pausiert Segmentation Gap Concor 5 mg (bei Bedarf) 1 – 0 – 0 – 1

Table 13 Error analysis of the ad hoc extraction of the daily medications dose

Study replication

The presented results for the University Hospital of Würzburg (UKW) and the Department of Internal Medicine I (Med1) were computed via ad hoc IE (see “Study replication” section). Since the ad hoc IE had an F1 score of 0.974, there may be small deviations from the exact values.


Study: Trends in antihypertensive medication use and blood pressure control among United States adults with hypertension Table 14 shows the results of the replication of the medication trend study to hypertension for the years 2000 to 2010. The findings of the referenced paper and their reproducibility by our results are listed in Table 15. The computation time to query the data for Table 14 from the CDW was 2 min 26 s.

Table 14 Replication of the medication group trend study for hypertension [13]
Table 15 Findings of the replicated studies compared to our results

Current trends of hypertension treatment in the United States.

Table 16 shows the grouped systolic blood pressure of hypertensive patients and Table 18 lists their the use of drug agent groups. The findings of the referenced paper and their reproducibility by our results are listed in Table 17. The computation time to query the data for Tables 16 and 18 from the CDW was aggregated 49 min 55 s.

Table 16 Systolic blood pressure (SBP) in mm Hg of hypertensive patients compared to [14]
Table 17 Findings of the replicated studies compared to our results
Table 18 Use of drug agent groups and systolic blood pressure (SBP, measured in mm Hg) groups of hypertensive patients compared to [14]

Chronic kidney disease

Study: Understanding CKD among patients with T2DM: prevalence, temporal trends, and treatment patterns – NHANES 2007-2012 Figure 2 is an additional evaluation showing all severity levels of CKD over time. The computation time to query the data from the CDW was 14 s.

Fig. 2

Temporal trend of CKD stages in the UKW. The severity degrees of CKD-patients are shown over time

Figure 3 shows the hypertension medication agent groups by degrees of severity of CKD for all patients with hypertension and CKD for the years 2013-2016. The computation time to query the data from the CDW for Fig. 3 was 1 min 3 s.

Fig. 3

Medication agent groups by degrees of severity of CKD in the UKW of CKD patients with hypertension

Tables 19 and 21 compare the findings of Wu et al. [5] to our findings for the UKW and the Med1 concerning medication and agent groups for patients with CKD and T2DM. It shows the medication for diabetes as well as the hypertension. The findings of the referenced paper and their reproducibility by our results are listed in Table 20. The computation time to query the data from the CDW was 3 min 16 s for Table 19 and 5 min 9 s for Table 21.

Table 19 Medication and agent groups for CKD with T2DM compared to [5]
Table 20 Findings of the replicated studies compared to our results
Table 21 Medication and agent groups for CKD with T2DM compared to [5]

Atrial fibrillation

The studies on atrial fibrillation (AF) investigate the characteristics and the temporal trend of the use of oral anticoagulants (OAC).

Study: Increased use of oral anticoagulants in patients with atrial fibrillation: temporal trends from 2005 to 2015 in DenmarkGadsbøll et al. investigate the increased use of oral anticoagulants in patients with atrial fibrillation [3]. Figure 4 shows the temporal trend of VKA and OACs compared to [4]. The findings of the referenced paper and their reproducibility by our results are listed in Table 22. The computation time to query the data from the CDW for Fig. 4 was 25 s.

Fig. 4

Temporal trend of VKA and OACs compared to [4]. a UKW. b Paper

Table 22 Findings of the replicated studies compared to our results

Figure 5 shows the temporal trend for AF patient age groups using OACs like in [4]. The computation time to query the data from the CDW for Fig. 5 was 55 s.

Fig. 5

Temporal trend of OAC clustered by age groups compared to [4]. a UKW. b Paper

Study: Non-vitamin K antagonist oral anticoagulation usage according to age among patients with atrial fibrillation: Temporal trends 2011–2015 in Denmark Staerk et al. made a detailed research for the years 2011 and 2015, since NOAC became relevant [4]. Figures 6 and 7 is a detailed analyses of the temporal trend OACs listing its representatives: Dabigatran, Rivaroxaban, Apixaban. The computation time to query the data from the CDW was 36 sec for Fig. 6 and 29 sec for Fig. 7.

Fig. 6

Temporal trend of VKA and OAC usage of all AF patients compared to [4]. a UKW. b Paper

Fig. 7

Temporal trend of VKA and NOACs of AF patients aged ≥85 compared to [4]. a UKW. b Paper

Table 24 shows the distribution among sex and age groups. Table 25 analyses the comorbidities and Table 26 lists the concomitant medication. The values in the referenced paper refer to the time period between 22.8.2011 and 1.1.2016. We computed the values for the same period (named UKW_11) and for the period 1.1.2016 - 1.1.2018 (named UKW_16). The computation time to query the data from the CDW was 1 min 10 s for Table 24, 1 min 40 s for Table 25 and 2 min 10 s for Table 26. The findings of the referenced paper and their reproducibility by our results are listed in Table 23.

Table 23 Findings of the replicated studies compared to our results
Table 24 Characteristics of patients with atrial fibrillation using VKAs or OAC medications compared to [4]
Table 25 Comorbidities of patients with atrial fibrillation using VKAs or OAC. (Continuation of Table 24)
Table 26 Concomitant medication of patients with atrial fibrillation using VKAs or OAC. (Continuation of Table 24)

Table 27 summarizes the results of the study replication. Main findings were replicated and confirmed by us to 93%, sub-findings to 68% and overall to 75%.

Table 27 Summary of the of the study replication results, including main, sub and overall findings

Daily medication dose extraction.

As an additional evaluation, we extracted the daily dose of patients with AF using ad hoc IE. All three OACs agent groups with their drugs where analyzed: Xarelto (Rivaroxaban) (see Table 28), Eliquis (Apixaban) (see Table 29) and Pradaxa (Dabigatran) (see Table 30).

Table 28 Extraction of the daily medication dose of Xarelto for patients with AF
Table 29 Extraction of the daily medication dose of Eliquis for patients with AF
Table 30 Extraction of the daily medication dose of Pradaxa for patients with AF

The average daily dose was 19,31 mg of Xarelto, 7,4 mg of Eliquis and 232,3 mg of Pradaxa.


First, the results of the replication studies are discussed, and second, the ad hoc IE tests and the system itself are compared to other approaches.

Study replication

Major result & comparison. One study (AF Trend from 2005 to 2015 [3]) could be completely replicated, i.e., all main findings and sub-findings were confirmed by us. Overall, 93% of the main findings, 68% of other detailed findings and 75% of all findings could be replicated. Table 27 lists the results of the individual replications. As mentioned in “Background” section, many researchers have tried to reproduce other researchers work, but 70% failed. 24% researchers reporting a successful replication of experiments were able to publish their work. In case of unsuccessful reproduction this proportion was only 13% [16]. Of course, when conducting replication experiments, some deviations have to be expected. Concerning the sources of variation, not only the exact reproduction of the study design is important, but also the population under study and time trends observed regarding diagnosis and therapy matter. E.g., Gu et al. reported that the control of blood pressure (BP) levels “varied greatly between recent publications” [13]. Staerk et al. mentioned that the most frequently used NOAC agent in their study was different to a previous study owing to changes in prescription patterns over time [4].

Study details. The distribution among the groups of active substances for hypertension in the UKW was slightly different compared to the paper [13]. In Med1, patients got substantially more drugs, probably indicating treatment preferences of a certain clinic.

In the CKD study, 75% of all findings agreed with our results, but there were also some deviations. Some observations differed only in stage 5 of CKD. This could be explained with different sizes of population of the subgroups with level 1, 4 and 5. These were caused by the basic population (population-based sample vs. hospital patients). The trends in the studies of atrial fibrillation could be replicated by us, however with a surprisingly small temporal shift. The comorbidities and the concomitant medication differed slightly, but many agreed.

Data acquisition & study population. The studies differed regarding the data acquisition approach: The hypertension [13] and CKD [5] studies were based on NHANES, the AF studies [3, 4] on the Danish National Prescription Registry and the hypertensive study with SBP used a physician survey. The medication in NHANES was "self-reported data (via a patient survey questionnaire)" [5]. We took the medication information from the discharge letter written by physician, which should be reflected in higher accuracy. NHANES is a representative sample of the U.S., i.e. both healthy and sick people, whereas a CDW collects information on hospitalized or ambulatory patients. There are even differences within a hospital. The medication use was found higher in almost all cases at the Med1 compared to the entire clinic. This is comprehensible, because hypertension, atrial fibrillation and chronic kidney diseases are usually treated there. The studies also differed regarding the number of analyzed cases. The AF studies used a nation-wide data source, i.e. three to four times more patients than which were present in the local CDW. For the hypertension study, we analyzed eight times more cases, in the CKD even 25 times more cases.

Analysis duration. While our queries took only a few minutes, it probably took a few weeks or months to conduct the studies for the referenced papers.

Ad hoc IE

Ad hoc IE possesses features of a conventional IE and query functions of CDWs. Therefore, the evaluation results and the system itself are compared with other approaches.

Comparison of evaluation results

According to [22] MedEx is the most widespread used tool for extracting medication information from clinical texts. In their original paper they achieved an F1-score of 93,2% for extracting drug names, a score of 94,6% for the strength and 96,0% for the frequency [19]. Two years later they published a case study around the medication warfarin and pushed the F1 score to 95% (recall 99,7%, precision 90,8%) for extracting the daily dosage [30]. In another study, they tried to calculate the daily dosage for the drug tacrolimus with an extended MedEx version and reported precisions of 90-100% and recalls of 81-100%. For discharge summaries they achieved F1 measures of 96% for strength and 88% for daily dosage [31].

Some papers mention, that they had to deal with more complex medication instructions like dosing in 2 h intervals [19, 3032]. This may complicate the calculation of the dosage and explain the inferior results compared to ours (F1 97,4%, precision 97,7%, recall 97,2%).

The results of the extraction of the drug names alone were only partially comparable with ours. First, no lists of medications were used in the literature, and second, these are all conventional IEs. We applied ad hoc IE, which extracts the information on the fly during runtime.

Conventional versus ad hoc IE

Conventional IE. IE turns unstructured information embedded in texts into structured data [33]. More precisely, it is the automatic extraction of concepts, entities and events, as well as their relations and associated attributes [22]. It consists of subtasks, i.e. entity recognition, relation extraction, event extraction (including time and date), and template filling [33]. In a conventional IE application information are computed by many expensive processing steps [34]. Therefore, each text is annotated several times, e.g. with parts of speech tagging, syntactic or dependency parsing or word list labeling. The output of a tagging process is the input for the next step. Thereafter rule-based systems apply rules on these annotations to extract information. Machine learning approaches use additional features and a trained model for the extraction step.

Ad hoc IE. In ad hoc IE, a segmentation separates non-related concepts. On these segments, a one-step annotation can be made effectively. But this step is quite fast, due to the index, and in contrast to the conventional IE, there are not “many of expensive processing steps” [34]. Thus, ad hoc IE is suitable for domains that can be handled with a one-step annotation. A survey revealed that 65% of clinical information extraction systems are rule-based and often use a regular expression as a search pattern [22]. Hence, they are interesting for ad hoc IE and could possibly be implemented with it. Ad hoc IE shifts the time of extraction from the data-integration phase to runtime, enabling a flexible IE at runtime for all users.

Ad hoc IE does not address all sub-tasks of a conventional IE application. However, the tasks important to the medical domain are supported: Named entity recognition is ensured by the query functions, relation extraction for medical concepts is accomplished by segmentation and for patient identification by context detection.

Comparison In summary, the ad hoc IE was found to be very well suited for this task. It yielded as good results as the conventional IE but was characterized by a much lower developmental effort, promptness of results and intuitive adaptability by users. In domains with complicated structure, conventional IE might be superior in terms of confidence and accuracy [18]. However, ad hoc IE does not claim to replace conventional IE, it rather should be considered a supplement for quick analysis to get a good and detailed overview for further investigations. An additional advantage of ad hoc IE is its ability not only to return the number of hits, but also to retrieve hit snippets from texts. This addresses two points: 1) Queries can be refined iteratively and 2) the system can also be used as an evaluation environment.

Query Features of other CDWs

Text query features are poorly supported in CDWs [18]. Most of them, like the well known i2b2, store their data in SQL-DBs and just support the like-operatorFootnote 9 a SQL full text index. Other CDW index their textual data with index libraries as Apache Solr (e.g. tranSMART [35] or Roogle [36]) or with SQL full text index (e.g. STRIDE [37]). Dr. Warehouse performs an negation detection as well and excludes negated findings from the search [38]. However, no system has query features that exceed a token search.

Comparison to SQL Many CDWs use a SQL-Server as storage engine. Texts can be queried via the like-operator, which is used to perform wildcard queries. However, this is limited in many ways: Error tolerant queries, which deal with misspellings, are not supported. Drug names that consist of several words are difficult or cumbersome to find with SQL methods. Especially, if these words are not next to each other and, e.g., separated by a brand name.

Extracting dose information reliably using SQL is next to impossible. Several words can be between the drug name and the instruction, e.g. additional information about the application. A segmentation of the drugs would be necessary in any case. Additionally, an SQL-based approach is much slower than a text index based system.


Limitations for conducting medication trend studies in a CDW relate to complex inclusion and exclusion criteria that can not appropriately be mapped, like complex temporal constraints. Some techniques frequently used in clinical analyses are more difficult to apply like adjustment for important confounders, e.g. sex and age. This is not a technical limitation, but it would require a laborious recalculation.

The feasibility of replication studies depends as well on the data embedded in the CDW. Only integrated concepts or texts can be queried. The populations of studies are always different, so the population of a specific hospital department does not correspond to the overall population.


With the presented approach of the ad hoc IE for medications, which provides equally good results for this task as the conventional approach, it is possible to quickly carry out analyses like the study replications shown here. We combined ad hoc IE with additional filters based on structured and unstructured data: We stratified the data by year and severity of the respective condition, and analyzed subgroups like age, comorbidities and concomitant medication. Furthermore, we used ad hoc IE to transform unstructured data from the discharge letters to structured data (e.g. systolic blood pressure groups) and extracted the daily dosage per drug on the fly.

To calculate daily medication dosages, each strength unit combination must still be queried individually. It is intended to calculate this automatically, e.g. with the use of function queries.


  1. 1.

    Extract, Transform, Load

  2. 2.

  3. 3.

  4. 4.

  5. 5.

    The complete trigger set is available at:

  6. 6.

  7. 7.

  8. 8.

  9. 9.



Attention deficit hyperactivity disorder


Atrial fibrillation


Anatomical Therapeutic Chemical classification system


Body mass index


Blood pressure


Clinical data warehouse


Clinical information system


Chronic kidney disease


Electronic health record


Graphical user interface


International Classification of Diseases, version 10


Information extraction


Left ventricular ejection fraction


Department of Internal Medicine I


National Disease and Therapeutic Index


National Health and Nutrition Examination Survey


Novel oral anticoagulants


Oral anticoagulants


Operationen- und Prozedurenschlüssel


Systolic blood pressure


Type 2 diabetes mellitus


University Hospital of Würzburg


Vitamin K antagonist


  1. 1

    Zoega H, Furu K, Halldorsson M, Thomsen PH, Sourander A, Martikainen JE. Use of adhd drugs in the nordic countries: a population-based comparison study. Acta Psychiatr Scand. 2011; 123(5):360–7.

    CAS  PubMed  Article  Google Scholar 

  2. 2

    Fang MC, Stafford RS, Ruskin JN, Singer DE. National trends in antiarrhythmic and antithrombotic medication use in atrial fibrillation. Arch Intern Med. 2004; 164(1):55–60.

    PubMed  Article  Google Scholar 

  3. 3

    Gadsbøll K, Staerk L, Fosbøl EL, Sindet-Pedersen C, Gundlund A, Lip GY, Gislason GH, Olesen JB. Increased use of oral anticoagulants in patients with atrial fibrillation: temporal trends from 2005 to 2015 in denmark. Eur Heart J. 2017; 38(12):899–906.

    PubMed  Google Scholar 

  4. 4

    Staerk L, Fosbøl EL, Gadsbøll K, Sindet-Pedersen C, Pallisgaard JL, Lamberts M, Lip GY, Torp-Pedersen C, Gislason GH, Olesen JB. Non-vitamin k antagonist oral anticoagulation usage according to age among patients with atrial fibrillation: Temporal trends 2011–2015 in denmark. Sci Rep. 2016; 6:31477.

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  5. 5

    Wu B, Bell K, Stanford A, Kern DM, Tunceli O, Vupputuri S, Kalsekar I, Willey V. Understanding ckd among patients with t2dm: prevalence, temporal trends, and treatment patterns—nhanes 2007–2012. BMJ Open Diabetes Res Care. 2016; 4(1):000154.

    Article  Google Scholar 

  6. 6

    Komaroff M, Tedla F, Helzner E, Joseph MA. Antihypertensive medications and change in stages of chronic kidney disease. Int J Chronic Dis. 2018; 2018:10.

  7. 7

    Katada H, Yukawa N, Urushihara H, Tanaka S, Mimori T, Kawakami K. Prescription patterns and trends in anti-rheumatic drug use based on a large-scale claims database in japan. Clin Rheumatol. 2015; 34(5):949–56.

    PubMed  Article  Google Scholar 

  8. 8

    Bromfield S, Muntner P. High blood pressure: the leading global burden of disease risk factor and the need for worldwide prevention programs. Curr Hypertens Rep. 2013; 15(3):134–6.

    PubMed  PubMed Central  Article  Google Scholar 

  9. 9

    Falaschetti E, Mindell J, Knott C, Poulter N. Hypertension management in england: a serial cross-sectional study from 1994 to 2011. Lancet. 2014; 383(9932):1912–9.

    PubMed  Article  Google Scholar 

  10. 10

    Godet-Mardirossian H, Girerd X, Vernay M, Chamontin B, Castetbon K, de Peretti C. Patterns of hypertension management in france (enns 2006–2007). Eur J Prev Cardiol. 2012; 19(2):213–20.

    PubMed  Article  Google Scholar 

  11. 11

    Sarganas G, Knopf H, Grams D, Neuhauser HK. Trends in antihypertensive medication use and blood pressure control among adults with hypertension in germany. Am J Hypertens. 2015; 29(1):104–13.

    PubMed  Article  Google Scholar 

  12. 12

    Wallentin F, Wettermark B, Kahan T. Drug treatment of hypertension in sweden in relation to sex, age, and comorbidity. J Clin Hypertens. 2018; 20(1):106–14.

    CAS  Article  Google Scholar 

  13. 13

    Gu Q, Burt VL, Dillon CF, Yoon S. Trends in antihypertensive medication use and blood pressure control among united states adults with hypertensionclinical perspective: The national health and nutrition examination survey, 2001 to 2010. Circulation. 2012; 126(17):2105–14.

    CAS  PubMed  Article  Google Scholar 

  14. 14

    Shah SJ, Stafford RS. Current trends of hypertension treatment in the united states. Am J Hypertens. 2017; 30(10):1008–14.

    PubMed  Article  Google Scholar 

  15. 15

    Begley CG, Ellis LM. Drug development: Raise standards for preclinical cancer research. Nature. 2012; 483(7391):531.

    CAS  Article  Google Scholar 

  16. 16

    Baker M. 1500 scientists lift the lid on reproducibility. Nature. 2016; 533:452–4.

  17. 17

    Jensen K, Soguero-Ruiz C, Mikalsen KO, Lindsetmo R-O, Kouskoumvekaki I, Girolami M, Skrovseth SO, Augestad KM. Analysis of free text in electronic health records for identification of cancer patient trajectories. Sci Rep. 2017; 7:46226.

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  18. 18

    Dietrich G, Krebs J, Fette G, Ertl M, Kaspar M, Störk S, Puppe F. Ad hoc information extraction for clinical data warehouses. Methods Inf Med. 2018; 57(01):22–9.

    Google Scholar 

  19. 19

    Xu H, Stenner SP, Doan S, Johnson KB, Waitman LR, Denny JC. Medex: a medication information extraction system for clinical narratives. J Am Med Inform Assoc. 2010; 17(1):19–24.

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  20. 20

    Spasić I, Sarafraz F, Keane JA, Nenadić G. Medication information extraction with linguistic pattern matching and semantic rules. J Am Med Inform Assoc. 2010; 17(5):532–5.

    PubMed  PubMed Central  Article  Google Scholar 

  21. 21

    Sohn S, Kocher J-PA, Chute CG, Savova GK. Drug side effect extraction from clinical narratives of psychiatry and psychology patients. J Am Med Inform Assoc. 2011; 18(Supplement_1):144–9.

    Article  Google Scholar 

  22. 22

    Wang Y, Wang L, Rastegar-Mojarad M, Moon S, Shen F, Afzal N, Liu S, Zeng Y, Mehrabi S, Sohn S, et al. Clinical information extraction applications: A literature review. J Biomed Inform. 2018; 77:34–49.

    PubMed  Article  Google Scholar 

  23. 23

    Dietrich G, Fell F, Fette G, Krebs J, Ertl M, Kaspar M, Störk S, Puppe F. Web-padawan: Eine web-basierte benutzeroberfläche für ein klinisches data warehouse. In: HEC 2016, Joint Conference of GMDS, DGEpi, IEA-EEF, EFMI.Munich: German Association for Medical Informatics, Biometry and Epidemiology (GMDS) e. V.: 2016. p. 421.

  24. 24

    Chapman WW, Bridewell W, Hanbury P, Cooper GF, Buchanan BG. Evaluation of negation phrases in narrative clinical reports. In: Proceedings of the AMIA Symposium. Washington, DC: American Medical Informatics Association.2001. p. 105.

    Google Scholar 

  25. 25

    Chapman WW, Bridewell W, Hanbury P, Cooper GF, Buchanan BG. A simple algorithm for identifying negated findings and diseases in discharge summaries. J Biomed Inform. 2001; 34(5):301–10.

    CAS  PubMed  Article  Google Scholar 

  26. 26

    Harkema H, Dowling JN, Thornblade T, Chapman WW. Context: an algorithm for determining negation, experiencer, and temporal status from clinical reports. J Biomed Inform. 2009; 42(5):839–51.

    PubMed  PubMed Central  Article  Google Scholar 

  27. 27

    Bard GV. Spelling-error tolerant, order-independent pass-phrases via the damerau-levenshtein string-edit distance metric. In: Proceedings of the Fifth Australasian Symposium on ACSW frontiers-Volume 68. Ballarat: Citeseer: 2007. p. 117–24.

    Google Scholar 

  28. 28

    Krug M, Tu NDT, Weimer L, Reger I, Konle L, Jannidis F, Puppe F. Annotation and beyond – using athen annotation and text highlighting environment. In: DHd 2018.Cologne: Digital Humanities im deutschsprachigen Raum e.V.: 2018.

    Google Scholar 

  29. 29

    National Center for Health Statistics. Analytic and Reporting Guidelines: The National Health and Nutrition Examination Survey (NHANES). Accessed May 2018.

  30. 30

    Xu H, Jiang M, Oetjens M, Bowton EA, Ramirez AH, Jeff JM, Basford MA, Pulley JM, Cowan JD, Wang X, et al. Facilitating pharmacogenetic studies using electronic health records and natural-language processing: a case study of warfarin. J Am Med Inform Assoc. 2011; 18(4):387–91.

    PubMed  PubMed Central  Article  Google Scholar 

  31. 31

    Xu H, Doan S, Birdwell KA, Cowan JD, Vincz AJ, Haas DW, Basford MA, Denny JC. An automated approach to calculating the daily dose of tacrolimus in electronic health records. Summit Transl Bioinforma. 2010; 2010:71.

    Google Scholar 

  32. 32

    Sohn S, Clark C, Halgrim SR, Murphy SP, Jonnalagadda SR, Wagholikar KB, Wu ST, Chute CG, Liu H. Analysis of cross-institutional medication description patterns in clinical narratives. Biomed Inform Insights. 2013; 6:11634.

    Article  Google Scholar 

  33. 33

    Jurafsky D, Martin JH. Speech and Language Processing, vol. 3.London: Pearson London; 2014.

    Google Scholar 

  34. 34

    Sarawagi S, et al. Information extraction. Found TrendsDatabase. 2008; 1(3):261–377.

    Google Scholar 

  35. 35

    Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (redcap)—a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009; 42(2):377–81.

    PubMed  PubMed Central  Article  Google Scholar 

  36. 36

    Cuggia M, Garcelon N, Campillo-Gimenez B, Bernicot T, Laurent J-F, Garin E, Happe A, Duvauferrier R. Roogle: an information retrieval engine for clinical data warehouse. Stud health technol inform. 2011; 169:584–8. ISSN: 0926-9630.

    PubMed  Google Scholar 

  37. 37

    Lowe HJ, Ferris TA, Hernandez PM, Weber SC. Stride–an integrated standards-based translational research informatics platform. In: AMIA Annual Symposium Proceedings. San Francisco: American Medical Informatics Association: 2009. p. 391.

    Google Scholar 

  38. 38

    Garcelon N, Neuraz A, Benoit V, Salomon R, Burgun A. Improving a full-text search engine: the importance of negation detection and family history context to identify cases in a biomedical data warehouse. J Am Med Inform Assoc. 2016; 24(3):607–13.

    Google Scholar 

Download references


We thank the reviewers for their valuable remarks.


This publication was funded by the German Research Foundation (DFG) and the University of Würzburg in the funding programme Open Access Publishing by paying the publication fees of the journal.

This work was supported by the Comprehensive Heart Failure Center Würzburg (BMBF grants: #01EO1004 and #01EO1504). They provided the analyzed data and founded MK, GF and SS.

FP, LL, JK and GD are founded by the chair of artificial intelligence within the computer science department of the Würzburg Unviversity and ME is founded by the Service Center Medical Informatics at the University Hospital of Würzburg.

Availability of data and materials

The list of trigger tokens used for the context algorithm is available on the Web (see “Methods” section). The analyzed patient data must not leave the clinical network in order to protect privacy.

Author information




GD and FP conceived the presented idea. GD carried out the implementation for the tests, designed and performed the experiments and wrote the manuscript. FP contributed to the analysis and the interpretation of the results and technical evaluations. FP also contributed to the refinement of the used techniques and methods. JK made substantial contributions to the design by implementing big parts of the text segmentation used by the context detection. LL implemented big parts of the CDW that were necessary for the study. GF made substantial contributions to the acquisition of data. GF imported the data to be analyzed into the CDW. ME made substantial contributions to the acquisition of data. ME exported the data from the clinical information system of the University Hospital of Würzburg. MK acquired the ABDA-Database, which was used as background knowledge. SS made substantial contributions to the analysis and interpretation of all medical data.

All authors critically revised sections. All authors give their final approval of the version to be published. All authors agree to be accountable for the work.

Corresponding author

Correspondence to Georg Dietrich.

Ethics declarations

Ethics approval and consent to participate

An ethics approval was waived by the corresponding IRB. The used clinical Data Warehouse contains pseudonymized data only.

Consent for publication

The used clinical Data Warehouse contains pseudonymized data only. We only used data for the clinical Data Warehouse as described in ethics approval section. No data is published that relates to an individual person. Therefore, a consent for publication is not necessary.

Competing interests

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (, which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Dietrich, G., Krebs, J., Liman, L. et al. Replicating medication trend studies using ad hoc information extraction in a clinical data warehouse. BMC Med Inform Decis Mak 19, 15 (2019).

Download citation


  • Data warehouse
  • Medication extraction
  • Information extraction