Doublet method for very fast autocoding

Nomenclature

The nomenclature used is neocl.xml, previously described by the author and currently designated as "The developmental lineage classification and taxonomy of neoplasms"[1]. In the context of this manuscript, the purpose of the taxonomy is to provide a listing of all names of neoplasms, with synonyms grouped under a common code number. The current version of the neocl.xml file contains 102,271 unique names of neoplasms. In constructing the taxonomy, enormous effort was made to include every variant name for every known neoplasm of man. Variant names included different terms for the same concept and different ways of expressing an individual term (e.g. variations in word order).

An example of the variety of synonyms encountered for a single tumor is shown for "adenocarcinoma of the colon." There are 44 synonyms listed in the taxonomy. These are: adenoca arising from colon, adenoca arising in colon, adenoca of colon, adenocarcinoma arising from colon, adenocarcinoma arising from large intestine, adenocarcinoma arising from the colon, adenocarcinoma arising from the large intestine, adenocarcinoma arising in colon, adenocarcinoma arising in large intestine, adenocarcinoma arising in the colon, adenocarcinoma arising in the large intestine, adenocarcinoma of colon, adenocarcinoma of large intestine, ca arising from colon, ca arising in colon, ca of colon, cancer arising from colon, cancer arising in colon, cancer of colon, carcinoma arising from colon, carcinoma arising in colon, carcinoma of colon, colon adenoca, colon adenocarcinoma, colon ca, colon cancer, colon cancers, colon carcinoma, colon carcinomas, colon with adenoca, colon with adenocarcinoma, colon with ca, colon with cancer, colon with carcinoma, colonic adenoca, colonic adenocarcinoma, colonic adenocarcinomas, colonic ca, colonic cancer, colonic cancers, colonic carcinoma, colonic carcinomas, large intestine adenocarcinoma, large intestine with adenocarcinoma.

Input file

The input file was created by a PubMed query on "pathology [ad] AND neoplasm [all]", at the U.S. government website [5]. The query gathered all abstracts from the pubmed database in which the term neoplasm occurs somewhere in the pubmed entry, and in which the affiliation of the author contains the word "pathology". The query yielded abstracts that are likely to contain names of neoplasms. The PubMed output file can serve as a good test for an autocoder that uses a neoplasm nomenclature. The PubMed search yielded 66,509 abstracts. All of the abstracts were downloaded into a single file from the PubMed site by setting the "Display" attribute to "Medline" and the "Send to" attribute to "file". This produced a 170,997,880 byte plain-text file. The file was given the filename tumor.txt, and this filename was used by the autocoders as a parsing input file. Although this file is not included with this manuscript, anyone in the world with internet access can obtain a near-identical file by repeating the same PubMed query.

Doublet autocoder

The doublet autocoder is supplied as an open source Perl script (doubcode.pl) with this manuscript [see Additional file 3]. Perl itself is an open source language that is bundled with the Unix and Linux operating systems. In the past decade, Perl has become very popular in the bioinformatics community. Perl interpreters are available at no cost and in versions suitable for virtually all operating systems. Perl can be obtained from the Comprehensive Perl Archives Network [6] or from Active State [7].

The algorithm used by the doublet autocoder is described:

1. Each phrase (term) in the nomenclature (neocl.xml) is converted into intercalated doublets, and each doublet is assigned a consecutive number.

2. Each nomenclature phrase is assigned the concatenated list of numbers that represent the ordered doublets composing the phrase.

3. Every text record (pubmed abstract in this case) is split into an array consisting of the consecutive words in the text record.

4. The text array is parsed as intercalated doublets. Intercalated doublets from the text that match doublets found anywhere in the nomenclature are assigned their numeric values (from the doublet index created for the nomenclature). Runs of consecutive doublets from the text that match doublets from the nomenclature are built into concatenated strings of doublet values. The occurrence of a text doublet that does not match any doublet in the nomenclature cannot possibly be part of a nomenclature term. Such text doublets serve as "stop" doublets between candidate runs of text doublets that match nomenclature doublets.

5. The runs of matching doublets are tested to see if they match any of the runs of doublets that compose nomenclature terms or if they contain any subsumed terms that match nomenclature terms.

6. The array of doublet runs extracted from the text that match nomenclature terms are cached in an external file.

Phrase autocoder

The phrase autocoding script, phrase.pl is included as supplementary file with this article [see Additional file 4]. The script was described in a prior publication [8]. It works by taking text records and parsing the text into all possible ordered phrases of all sizes up to a predetermined limit (5, in this case). The script then examines each resultant phrase to determine if it matches a term in the nomenclature. If there is a match, the term is added to the cache of matched terms.

Speed

The speed of both coders was compared using a 170+ Megabyte text. The doublet method coded the entire text in 211 seconds compared with 1,766 seconds required for the phrase method. Therefore the doublet method coded 8.4 times faster than the phrase method. The text coding rate for the doublet method is 0.8 Megabytes per second on a desktop computer with a modest 1.6 GHz Pentium processor.

Both the phrase method and the doublet method coders used the same Programming Language, the same nomenclature, and the same data structure for the nomenclature (a simple associative array). Both coders grab sequentially occurring chunks of text, each chunk delimited by a double-newline delimiter (roughly the ascii equivalent of a paragraph delimiter) and place the grabbed text chunk into a temporary scalar variables.

The difference between the two coders occurs when the chunks of text are parsed. The phrase coder takes the text and creates an array of every ordered combination of 1,2,3,4 and 5-word phrases contained in the record. This number of elements in the phrase array is about 5 times larger than the number of words in the record. The doublet coder parses the record into the set of all ordered doublet terms. This number of elements in the array of doublets is about the same as the number of words in the record. The phrase coder must try to match about 5 times as many terms as the doublet coder. Also, the phrase coder tries to match each phrase in the record array against the entire nomenclature. The doublet coder tries to match each doublet encountered in the input text against the collection of doublet terms extracted from the nomenclature. Because doublets of text, unlike individual words, tend to have unique meanings, only a small subset of the doublets encountered in the input text will match the set of doublets extracted from the nomenclature. Text doublets that do not match any doublets in the nomenclature are "skipped." Text doublets that match doublets from the nomenclature are concatenated to consecutive matching doublets until a non-matching doublet is encountered. The length of matching doublets sets the length of the candidate term. The algorithmic strength of the doublet method is that it eliminates the need to create and match [against a nomenclature] an array of all possible phrases of all possible lengths found in a textual record.

Autocoding output

The output of the doublet method and phrase method autocoders are provided as supplemental files with this manuscript [see Additional file 5] [see Additional file 6]. The analysis of the autocoding output was performed using another Perl script, doubcode.pl [see Additional file 7]. The analysis output file is also provided with this manuscript [see Additional file 8].

The two autocoder methods parsed a text exceeding 170 megabytes in length. The doublet method coded 4807 different [sometimes called unique] terms. The total number of matching term encountered (which includes replicate matches of terms within a record) by the doublet method is 467,391 terms. The phrase method coded 4557 different terms. There were 250 terms that the doublet method matched and the phrase method missed. There were zero terms that the phrase method matched and the doublet method missed.

The improved performance of the doublet coder is accounted for by its ability to match terms of any length, while the phrase coder is limited to match terms that are no more than 5 words in length. Because the phrase coder parses chunks of input text into all allowed phrase size combinations, the performance of the phrase coder is slowed dramatically when the allowed phrase length is large. For instance, if the theoretical maximum phrase size were permitted for a text chunk 100 words in length, the phrase parser would need to create an array of a size equal to 100 × 100 or 10,000 phrases, matching each phrase against nomenclature. If the phrase size were restricted to 5 words, then the phrase parser would only need to create a phrase array of size 5 × 100 or 500. It is reasonable to restrict the allowed length of coded term phrases if it is known that the number of nomenclature terms with length exceeding the allowed limit is small. It is also reasonable to restrict the allowed length of coded term phrases is it is believed that terms of long length can be successfully represented by subsumed terms within the longer term.

For example, "Refractory anemia with excess blasts in transformation" will be captured as a single term by the doublet method. The phrase method will miss the 7-word term but will code the subsumed terms, "refractory anemia" and "refractory anemia with excess blasts". In this specific instance, the doublet coder outperformed the phrase coder. However, few cancer terms actually exceed 5 words in length, and longer terms almost always contain modifying phrases (so-called term pre-coordinations) that add little to the fundamental meaning of the term.

The following terms are examples of neoplasms extracted by the doublet method and missed by the phrase method. All of the listed terms exceed 5 words in length. "Acute myelogenous leukemia with minimal differentiation, acute myeloid leukemia with minimal differentiation, adenocarcinoma arising in the parotid gland, adenoid cystic carcinoma of submandibular gland, aids-related primary central nervous system lymphoma, atypical glandular cells of undetermined significance, atypical squamous cells of uncertain significance, carcinoid tumor arising from meckel diverticulum, extranodal marginal zone b-cell lymphoma of mucosa-associated lymphoid tissue, giant cell tumor of tendon sheath, giant cell tumour of tendon sheath, glandular malignant peripheral nerve sheath tumor, high-grade malignant peripheral nerve sheath tumor, hurthle cell carcinoma arising from thyroid, hyalinizing spindle cell tumor with giant rosettes, interdigitating dendritic cell sarcoma arising from the spleen, intermediate-grade b-cell non-hodgkin lymphoma, intraductal papillary mucinous neoplasm of pancreas, intraductal papillary-mucinous tumor with moderate dysplasia, kaposi sarcoma arising in the heart, large cell calcifying sertoli cell tumor, leiomyosarcoma arising from the pulmonary artery, low grade cervical glandular intraepithelial neoplasia, low-grade malignant peripheral nerve sheath tumor, lymphoma arising in the thyroid gland, malignant epithelioid peripheral nerve sheath tumor, malignant tumor of peripheral nervous system, malt lymphoma arising in the stomach, metanephric adenosarcoma in a young adult, monomorphic clear cell adenocarcinoma of salivary glands, ovarian serous borderline tumor with micropapillary and cribriform patterns, ovarian serous tumor of low malignant potential, paraganglioma arising in the cauda equina, perineurial malignant peripheral nerve sheath tumor, philadelphia chromosome negative chronic myeloid leukemia, phosphaturic mesenchymal tumor mixed connective tissue variant, pineal parenchymal tumor of intermediate differentiation, pleomorphic adenoma of minor salivary glands, polymorphous low-grade adenocarcinoma of minor salivary gland, polymorphous low-grade adenocarcinoma of minor salivary glands, polymorphous low-grade adenocarcinoma of salivary gland, poorly differentiated malignant peripheral nerve sheath tumor, porokeratotic eccrine ostial and dermal duct nevus, primary cutaneous anaplastic large cell lymphoma, primary gastric diffuse large b-cell lymphoma, primary signet-ring cell carcinoma of lung, primitive neuroectodermal tumor arising in the pancreas, pulmonary immunocytoma with massive crystal storing histiocytosis, rhabdoid transformation of tumor cells in meningioma, serous ovarian tumor of low malignant potential, sex cord tumor with annular tubules, signet ring cell adenocarcinoma of prostate, signet-ring cell mucin-producing adenocarcinoma of minor salivary glands, sinonasal desmoplastic small round cell tumor, smooth muscle tumor of uncertain malignant potential, smooth muscle tumor with an uncertain malignant potential, splenic lymphoma with circulating villous lymphocytes, squamous cell carcinoma of fallopian tube, submandibular gland carcinoma ex pleomorphic adenoma, tall cell variant of papillary carcinoma of thyroid, testicular large cell calcifying sertoli cell tumor, uterine tumor resembling ovarian sex cord tumor."

The long terms found by the doublet coder and missed by the phrase coder represent a very small percentage of the different terms found by either coder in the long 170+ Megabyte text (250/4807 = 5%). A cursory review of these terms indicates that they represent rare lesions or rare variants of common lesions.

Accuracy issues for lexical parsers

The performance of autocoders is always a contentious issue. The reflexive approach to measuring autocoder performance usually involves selecting a small corpus of text and allowing a human expert in a language domain to carefully read through the lines of the text, creating a so-called gold standard against which the autocoder can be compared. The problems with this seemingly straightforward approach are many and have been explained by myself and others. The many limitations of "precision and recall" as measurements of indexing performance have been reviewed elsewhere [9, 10].

It would be a useful exercise to re-examine what autocoding performance means to different people in contrast to what autocoding performance means within the context of lexical parsers [11, 12]. First, every human coder is biased by their different perceptions of the knowledge domain. One coder may prefer "parsimonious" coding. In "parsimonious" coding, there is a "best" code that represents the ideas contained in a defined section of text. A review article on "liver cirrhosis" may contain many different terms, but the parsimonious coder may only preserve a single code for the entire article. Another coder, also a parsimonious coder, may not be so strict, but may want only the best term from among a group of subterms. So if "adenocarcinoma of endometrium" appears in text, she may want to preserve this term but omit the so-called atomic inclusive terms, "adenocarcinoma" and "endometrium." Another coder may want a complete listing of every matching term in a text, including terms that occur within larger terms. Still another informatician may want to include all ancestral terms for each term found in the text (i.e. terms not present in the original text but related to the textual concept).

Finally, some coders seek to create "concept signatures" from text. A concept signature is the list of the concepts contained in a section of text. The relationship of one section of text to another section of text is determined by a quantitative representation of how closely their signatures match. Concept signatures are used to retrieve or organize related documents, not specific concepts [13].

Lexical coders are, in a sense, perfect autocoders because they don't use grammatic rules, they have no "exception" lists, and they never guess or interpret text. The only thing that lexical parsers do is to parse text, examining strings to see if they contain exact matches to terms from a nomenclature. When well-constructed, they do not make mistakes. If they miss a valid term that is present in the text, it is because the term was missing from the nomenclature. Developers of lexical autocoders place enormous demands on the curators of nomenclatures.

The most common cause of missed terms arises when the text contains a modifying word that breaks the term into a phrase that no longer matches anything found in the nomenclature. For instance, "adenocarcinoma of the left lung," would be missed by a lexical parser if the nearest term in its terminology list is "adenocarcinoma of the lung." A rule-based parser or a semantic parser may have successfully teased out the "left" from the term and found the match. In this instance, the nomenclature failed the lexical parser because it did not list terms that indicated tumor laterality. Lexical parsers do not match terms that are absent from the text (i.e. no false positives). False-positive terms are possible in rule-based and semantic parsers if they create word patterns not present in the original text. Also, since lexical parsers strictly match phrases in text with phrases in nomenclatures, it is possible to achieve accurate results of dubious value. For instance, a lexical parser would parse "adenocarcinoma of the lung is not seen" and find a match against the neoplasm term, "adenocarcinoma of the lung." The concept is present in the text, even if though it is absent from the patient. It is a misleading but "true" positive.

The doublet autocoder creates a string of words from a chunk of text. In the case of the example corpus (PubMed abstracts), all of the sentences from the abstract are squeezed into a single string of words, obliterating sentence boundaries. This means that if a sequence of words crossing a sentence boundary happens to match a term in the reference nomenclature, the coder will register a "pseudo-positive" term. Empirical evidence suggests that this theoretical error in the doublet autocoder simply does not occur. A 4 megabyte collection of abstracts chunked as whole abstracts or as chunked sentences (delimited by period-space-space) contained no instances of pseudo-terms created by the obliteration of sentence boundaries in the whole-abstract text chunks. For fastidious developers who wish to ensure that their parsers respect sentence boundaries, it is possible to pre-process text into sentences with a sentence parser [14, 15].

Lexical parsers can be modified to provide an output that preserves the intended sense of the term as used in its context. For example, if the text includes the sentence, "Adenocarcinoma of the lung is not present." The "sensible" lexical parser may be modified to tag the "adenocarcinoma of the lung" with a negation modifier, preserving the intended sense of the term. The author has previously published an "in place" method of inserting codes directly into sentences, preserving modifier terms (including negations) [16].

Comparing parsers

The two lexical parsers used the same programming language (Perl), the same nomenclature, the same data structure to hold the nomenclature, the same text corpus, and the same method for breaking the complete text into text chunks. Differences between the performance of the two autocoding methods were accounted for entirely by the algorithmic processes by which chunks of text are parsed into phrases that are matched against the common nomenclature. It is easy to compare speeds. The doublet parser required 211 seconds to parse the 170+ Megabyte file, while the phrase parser required 1,776 seconds. Three algorithmic properties contribute to the speed advantage of the doublet coder over the phrase coder: 1) The doublet coder parses text into a number of phrases that is 5-fold smaller than the number of phrases produced by the phrase coder; 2) Doublets that are not found in any of the nomenclature terms can be quickly excluded, 3) Successive text doublets that match doublets from the nomenclature can be quickly concatenated and tested for matches in the nomenclature. These are the algorithmic differences between the phrase coder and the doublet coder and presumably account for the improved speed of the doublet method.

Comparing coding accuracy is somewhat trickier. As discussed, lexical parsers have no "false positive" matches. Lexical parsers, unlike natural language processors, do not construct context-based new word phrases from native text. Lexical parsers only match exact phrases found in text with exact nomenclature terms. Therefore, neither the doublet coder nor the phrase coder have false-positive term matches. Coding output was compared by examining the different terms matched by the two coders. The number of different terms found by the doublet method was 4,807, compared with 4,557 terms found by the phrase method. There were no terms found by the phrase method and missed by the doublet method. There were 250 terms missed by the phrase method and found by the doublet method (about 5% of the number of different terms found by the doublet method). For the most part, these terms represented rare neoplasms with pre-coordinated modifiers. The reason that such rare terms were encountered is due to the large size of the text (exceeding 170 Megabytes). When the doublet and phrase methods are judged by their ability to extract nomenclature from text, their output would be similar for small text sizes or when the primary purpose of the coding is to extract and count commonly occurring terms.