Combining entity co-occurrence with specialized word embeddings to measure entity relation in Alzheimer’s disease

Abstract

Background

Extracting useful information from biomedical literature plays an important role in the development of modern medicine. In natural language processing, there have been rigorous attempts to find meaningful relationships between entities automatically by co-occurrence-based methods. It has been increasingly important to understand whether relationships exist, and if so how strong, between any two entities extracted from a large number of texts. One of the defining methods is to measure semantic similarity and relatedness between two entities.

Methods

We propose a hybrid ranking method that combines a co-occurrence approach considering both direct and indirect entity pair relationship with specialized word embeddings for measuring the relatedness of two entities.

Results

We evaluate the proposed ranking method comparatively with other well-known methods such as co-occurrence, Word2Vec, COALS (Correlated Occurrence Analog to Lexical Semantics), and random indexing by calculating top-ranked entities related to Alzheimer’s disease. In addition, we analyze gene, pathway, and gene–phenotype relationships. Overall, the proposed method tends to find more hidden relationships than the other methods.

Conclusion

Our proposed method is able to select more useful related entities that not only highly co-occur but also have more indirect relations for the target entity. In pathway analysis, our proposed method shows superior performance at identifying (functional) cross clustering and higher-level pathways. Our proposed method, resulting from phenotype analysis, has an advantage in identifying the common genotype relating to phenotypes from biological literature.

Background

With the recent exponential growth of biomedical literatures, extracting useful information from these literatures has come to play an important role in the development of modern medicine. In the biomedical domain, information extraction (IE) is focused mainly on automatically identifying entities and their relationships from biomedical literatures as an aspect of natural language processing (NLP). Traditionally, detecting biomedical relationships between entities commonly involves adopting co-occurrence methods, which are based on the assumption that if two entities appear in the same sentence, paragraph, or abstract, these entities would be relevant to each other and helpful for biomedical knowledge discovery such as gene–gene interaction and gene–drug association. However, co-occurrence methods have posed the problem of generating many false positive relations, since they do not consider contextual information in a specific text [1].

In addition to simple co-occurrence-based approaches to measuring the relationship between entities, rule-based methods using syntactic patterns [2,3,4,5] and machine learning methods [6, 7] have been proposed in order to tackle this false positive issue. Measures of semantic similarity and relatedness have been developed to identify ontological relationships between two entities, such as WordNet [8] and UMLS (Unified Medical Language System) [9]. Recently, models of semantic word representations, or word embeddings, have been developed constructing semantic spaces based on large-scale corpora. This line of research adopts deep learning approaches [10,11,12,13,14,15,16] such as Word2Vec [17] for automatically learning optimal feature representation. However, these studies focus only on learning word embeddings by maximizing raw-text probability, which does not perfectly capture both similarity and relatedness [18].

As indicated by previous studies [18,19,20,21], incorporating two or more knowledge sources (e.g. thesaurus, ontology, and corpus) into word embedding approaches can produce better results for ranking the results for relationships between two entities. The present paper was motivated by the concept of utilizing knowledge sources for enriching word embeddings. To our best knowledge, no attempt has previously been made to combine word embedding based on multiple knowledge resources with co-occurrence of entity pairs, while classifying the type of relation by reflecting contextual information in biomedical literature. Moreover, there is no previous study that considers both direct and indirect relationships of entity pairs when calculating co-occurrence of entity pairs.

Therefore, in this study, we propose a hybrid semantic relatedness algorithm for biological knowledge discovery. Our proposed method combines co-occurrence between entities with specialized word embeddings [18] to calculate the semantic similarity of two entities by capturing both similarity and relatedness for semantic words, learning from both a corpus and a thesaurus. In the proposed method, we also consider both direct and indirect scores for each entity pair so as to find a more complex relationship considering not only explicit but also hidden relationships. We select Alzheimer’s disease (AD) as a case study for analysis and evaluation. Alzheimer’s disease is a degenerative brain disorder, whose cause is hard to diagnose accurately. As the number of AD patients has increased, researchers have striven by means of medical experiments and literature analysis to understand the disease’s pathophysiology so as to improve its diagnosis and treatment. For entity extraction, we used two approaches, PKDE4J [22] and SemRep [23]. PKDE4J is an integrated system designed to extract entity and relation from unstructured biomedical text corpora, whereas SemRep, a UMLS-based entity and relation extraction application, can identify semantic relationships in biomedical literatures. To evaluate the performance of the proposed method, we compared it with several well-accepted techniques, namely co-occurrence, Word2Vec [17], COALS (Correlated Occurrence Analog to Lexical Semantics) [24], and random indexing (RI) [25]. In addition, to evaluate the usefulness of the proposed method for other types of knowledge discovery, we conducted the following analyses 1) pathways analysis on the Reactome Pathway database [26] and 2) gene–phenotype relationships analysis on OMIM (Online Mendelian Inheritance in Man) [27]. Overall, the proposed method is able to identify more related genes for pathways than the other methods by differentiating rankings for each gene. The proposed method also finds genes like APOE, which is strongly associated with familial early-onset AD and coronary heart disease [28], through analyses of AD-related genes and the gene–phenotype relationship.

Methods

The present study comprises four steps: data collection, entity relation extraction, semantic relatedness scoring calculation, and evaluation. For semantic relatedness scoring, we consider both direct and indirect connection; in terms of evaluation, we employ four kinds of analyses, namely algorithm comparison, AD related–gene analysis, pathway analysis, and gene–phenotype relation analysis. Figure 1 illustrates the overall design of this study. A detailed description of the proposed approach is provided in subsequent sections.

Fig. 1

Overview of the proposed approach

Data collection

Using ‘Alzheimer disease’ or ‘Alzheimer’s disease’ as search terms, we retrieved 118,167 abstracts from PubMed, a search engine indexing more than 29 million citations for biomedical literature from MEDLINE. The exact query formulation is “Alzheimer disease [Title/Abstract] OR Alzheimer’s disease [Title/Abstract]”.

We did not limit publication by year, so as to get as much data as possible for our analysis. Figure 2 shows the distribution of the number of papers by publication year from 1990 to January 2019.

Fig. 2

Number of papers by publication year

Entity relation extraction

For PKDE4J [22], the algorithm used for entity relation extraction can identify the verb located between the two entities in a sentence and capture relational characteristics. In order to decrease unnecessary indirect connections, we selected entity by type. Since we focus on Alzheimer’s disease, we limited the entity type to gene, drug, and disease. Thus, for entity extraction, we used the following dictionaries: drug dictionaries, the gene dictionary collects from UniProt [29], MeSH (Medical Subject Headings) for disease [30], KEGG (Kyoto Encyclopedia of Genes and Genomes) for genetics [31], and DrugBank for medications [32]. We used the same data collection as the input for SemRep. As output, we extracted 969,341 entity relations using PKDE4J and 630,054 entity relations using SemRep [23].

Semantic relatedness scoring calculation

We considered both direct and indirect scoring for each entity pair. For the direct score, after we extracted the relations of an entity pair, we looked at the same entity pairs with different relation types appearing in one abstract. An example is shown below: the first column is the PMID (PubMed unique identifier), the second column is sentence location in that abstract, and the last column is entity relations:

pmid | sentence location | entity1 | type | entity2 | type| relations.

19,395,124 | 8 | MCI | DISEASE | depression | DISEASE | CO-OCCUR |.

19,395,124 | 17 | MCI | DISEASE | depression | DISEASE | RESULT_OF |.

Next, we considered only the co-occurrence frequency of entity pairs. There are two different kinds of direct relations: 1) co-occurrence of an entity pair in one abstract with frequency greater than one as noted as ‘sum_same’ in Tables 1 and 2) one-time co-occurrence of an entity pair in one abstract as noted as ‘sum_different’ in Table 1. If an entity pair only co-occurs once in an abstract, the co-occurrence number is the same as the number of abstracts. Biomedical literatures, like any other literatures, have skewed distribution. In other words, much research tends to follow popular diseases, drugs, and genes. Due to this tendency, it is hard to identify a new relation by the co-occurrence method. Thus, we aim to find less visible information from biological texts. If two-entity pairs co-occur in several abstracts, it indicates these relations are more popular and we can infer they are well-known entity pairs. We give them a low weight, while assigning entity pairs found in the same abstract a higher weight. Table 1 represents pseudocode for our algorithm.

Table 1 Pseudocode for our algorithm.

Table 2 Alzheimer’s disease–APP direct entity pairs

Therefore, the direct score can be calculated as Formula (1):

$$ {\mathrm{Y}}_{\mathrm{direct}}={\mathrm{S}}_{\left(\mathrm{A},\mathrm{C}\right)}\ast \left(2\upalpha +\upbeta \right) $$

(1)

where S _{(A, C)} is a semantic relatedness score between entity A and entity C. The semantic relatedness score is the cosine similarity calculated by corpus- and thesaurus-trained word embedding, per Kiela et al. [18]. In their method, Kiela et al. use additional contexts (such as a thesaurus) to supplement the Skip-Gram. For each target word, they modify the object to include an additional context, so that each word is sampled uniformly from the set of additional contexts. In this case, the corpus consists of AD-related articles collected from PubMed, while our thesaurus is derived from PharmGKB’s Variant, Gene and Drug Relationship Data [33] and a gene synonym thesaurus from UniProt [29] used to construct a word embedding model for biological relations.

We denote by α the frequency of entity pairs that co-occur in one abstract more than once, while β is the frequency of entity pairs that co-occur in one abstract only once.

Table 2 shows the direct score of the (Alzheimer’s disease, APP) entity pair, where APP (for amyloid precursor protein) an Alzheimer’s-related gene.

Next, we consider the indirect score for each co-occurrence (Entity A, Entity C). This time we need to calculate the semantic relatedness score of each indirect entity pair using Formula (2), with the indirect semantic relatedness for each intermediate entity B defined as a weighted average of the direct semantic relatedness scores:

$$ {\mathrm{S}}_{\mathrm{indirect}\ \left(\mathrm{A},\mathrm{C}\right)}=\left[{\mathrm{S}}_{\left(\mathrm{A},\mathrm{B}\right)}\ast \mathrm{a}+{\mathrm{S}}_{\left(\mathrm{B},\mathrm{C}\right)}\ast \mathrm{b}\right]/\left(\mathrm{a}+\mathrm{b}\right) $$

(2)

where a and b are the co-occurrence frequencies between entities A and B and between entities B and C, respectively.

Then we calculate the indirect averaged semantic relatedness score over all possible intermediates B for the entity pair. We used indirect averaged similarity multiplied by the links count (a + b) for each indirect link path score X.

As shown in Table 3, the entity pair of Alzheimer’s disease and APP has 1834 “B” entities which are intermediate for them. Note that, for convenience, we only show five indirect link paths in Table 3. For example, if we apply the proposed method to (Alzheimer’s disease, BACE1), they co-occur 692 times (the a value), whereas (BACE1, APP) co-occurs 1294 times (the b value).

Table 3 Indirect entity pairs scores

Then we accumulate the score of all indirect link paths of two entities as the indirect score, using Formula (3):

$$ {Y}_{indirect}={\sum}_{i= 1}^n{X}_i={\sum}_{i= 1}^n\raisebox{1ex}{$ 1$}\!\left/ \!\raisebox{-1ex}{$ 2$}\right.\left[{S}_{\left(A,{B}_i\right)}\ast {a}_{\left(A,{B}_i\right)}+{S}_{\left({B}_i,C\right)}\ast {b}_{\left({B}_i,C\right)}\right] $$

(3)

where n is the number of indirect paths or the number of intermediate entities, B_i is the intermediate entity, ${S}_{\left(A,{B}_i\right)}$ is the semantic relatedness score between entities A and B_i, and ${S}_{\left({B}_i,C\right)}$ is the semantic relatedness score between entities B_i and C; ${a}_{\left(A,{B}_i\right)}$ is the co-occurrence frequency between A and B_i, while ${b}_{\left({B}_i,C\right)}$ is the co-occurrence frequency between B_i and C.

Finally, we sum the direct score Y_direct and indirect score Y_indirect together as a semantic relatedness score for each entity pair (Formula (4)):

$$ \mathrm{Y}={\mathrm{Y}}_{\mathrm{direct}\ \left(\mathrm{formula}\ 1\right)}+{\mathrm{Y}}_{\mathrm{indirect}\ \left(\mathrm{formula}\ 3\right)} $$

(4)

where Y_direct and Y_indirect are calculated using Formulas (1) and (3), respectively.

Results and discussion

To measure the performance of the proposed method, we compared it on the top 20 entity pairs with co-occurrence, Word2Vec similarity, COALS, and random indexing. Rohde et al. [24] proposed a model of semantic relatedness based on lexical co-occurrence, known as COALS. COALS is a vector space method for deriving word meanings from large corpora. First, co-occurrence counts are gathered. Next, common words are selected to create a co-occurrence matrix with word pair correlations converted to counts, setting negative values to 0 and taking square roots of positive values. After that, they summed the correlation of each word line in the matrix as the semantic similarity. Sahlgren [25] introduced a random indexing word space approach. Random indexing achieves high processing efficiency by only requiring a small amount of calculation. It uses context information to express the word vector of the characteristic word. However, the randomness of the vector elements (− 1, + 1, 0) may lead to additive subtraction in the calculation of feature word context vectors, with a resulting loss of potential semantic information. For comparison with the proposed method, we used COALS and random indexing to calculate semantic relatedness scores for each entity pair.

We analyzed the relation results between AD and genes by five methods: the proposed method, co-occurrence, Word2Vec [17], COALS [24], and random indexing [25]. In addition, we conducted pathway analysis and gene–phenotype relationship analysis to examine whether the proposed approach can be applied for other types of biological knowledge discovery.

Top 20 entity pairs analysis

We calculated co-occurrence between entities extracted by PKDE4J. Table 4 shows the top 20 entities relating to Alzheimer’s disease by our proposed method. We used a min-max normalization method to generate each ranking score.

Table 4 Alzheimer’s disease top 20 related entity scores (PKDE4J)

From Table 4, we can see that Tau (No. 1), CSF (No. 5), APOE (No. 7), and MRI (No. 20) have high semantic relatedness. In order to show the difference clearly we list the top 20 Alzheimer’s disease-related entities by each method in Table 5.

Table 5 Top 20 Alzheimer’s disease–related entities by each method (PKDE4J)

As shown in Table 5, we can see that for Tau (No. 1), CSF (No. 5), APOE (No. 7), cognitive impairment (No. 11), and MRI (No. 20) the proposed method achieves a higher ranking than other methods. Specifically, among the top 20 entities lists, MRI (magnetic resonance imaging) only appears in our proposed method. This is attributed to the fact that these entities are either core proteins, genes related to AD, or diagnostic methods for AD, all of which may have many intermediate entities helping them link with AD so that they tend to gain a higher semantic relatedness score. Tau protein is a microtubule-associated protein (MAP) involved in microtubule stabilization. It is also a multifunctional protein that plays a key role in certain neurodegenerative diseases such as AD [34]. AD and Tau have 3568 co-occurrences in our dataset, with 236 different intermediate entities to help them link together. For CSF (cerebrospinal fluid), there is strong evidence that special CSF tests may be helpful in diagnosis. AD and CSF have 1968 co-occurrences, with 288 different intermediate entities. APOE gene polymorphism is closely related to AD, coronary heart disease, hyperlipidemia, cerebral infarction, and other diseases. Through the detection of APOE gene type, the incidence probability of senile dementia, cardiovascular and cerebrovascular diseases, and other diseases can be predicted at an early stage, to achieve early detection and intervention and to maximize a patient’s survival period. Studies have found that APOE is closely related to the incidence of AD, and the E4 allele of APOE is a high-risk factor for AD, especially in female patients [24]. AD and APOE entity pairs have 1401 intermediate entities to link them together.

While some entities rank higher by other methods, senile plaques only show in the co-occurrence top 20 results. The top Word2Vec results are all phrases containing “Alzheimer’s disease.” Regarding COALS and random indexing methods, the COALS-ranked terms Huntington’s disease (No. 10), diabetes (No. 12), schizophrenia (No. 13), and stroke (No. 14) only appear in these two rankings. COALS and RI seem to have better performance, yet their calculation principles allow the top 20 entities to have almost identical semantic relatedness scores; thus, it is hard to use COALS and RI to rank the entities.

We also examined the top 50 entities by each method, omitted here due to space limitations; the results are publicly available at http://informatics.yonsei.ac.kr/semantics/Top_50_entity_pair_result.xlsx.

For the top 20 entities, the APOE gene is 7th by our method. However, the APOE gene is not shown by COALS, Word2Vec, or random indexing in the top 20 ranking list. For the top 50 entities, dementia with Lewy bodies and FTD (frontotemporal dementia) are ranked high only by our proposed method. Alzheimer's disease, vascular dementia, and Lewy body dementia are seen as the top three most common causes of dementia. However, memantine (a drug) is shown by co-occurrence only. Multiple sclerosis (MS; a disease) only appears through COALS and random indexing.

Regarding the SemRep results, since we did not select the entity type, there are many words in common in the top 20 and top 50 lists. For example, as shown in Table 6, brain (No. 3), Alzheimer’s disease can affect memory in the patient’s brain; these entity pairs are already well known.

Table 6 Alzheimer disease top 20 related entity scores (SemRep)

As shown in Table 7, APP gene (No. 11), Apolipoprotein E (No. 14), and Parkinson’s disease (No. 15) have a higher score by the proposed method than by the other methods, due to intermediate entities. The top 20 entities in the Word2Vec ranking are all disease-related entities. However, there are many drug names that only appear in random indexing methods, such as donepezil (No. 6) and rivastigmine (No. 17).

Table 7 Top 20 Alzheimer’s disease–related entities by each method (SemRep)

Alzheimer’s disease-related gene analysis

For the PKDE4J results, we identified 8696 entities which co-occur with Alzheimer’s disease. For evaluation, we collected the related genes for Alzheimer’s from KEGG and calculated the ranking of each rated gene using co-occurrence frequency, our ranking method, and Word2Vec [17], COALS [24], and random indexing [25]. Figure 3 shows the Alzheimer’s disease-related gene ranking in each ranking list. The horizontal axis shows the gene name, while the vertical axis shows the ranking of each entity.

Fig. 3

Alzheimer’s disease–related gene ranking (PKDE4J)

Since the vertical axis shows the ranking for each gene, a small number means the ranking is high. With Word2Vec, gene rankings are always lower than for other methods; the COALS and random indexing methods have similar gene rankings. For COX1 and PSENEN genes, our method shows a higher ranking than the others. For the (AD, COX1) entity pair, the number of co-occurrences is only 6 in our database. However, there are 72 different intermediate entities to help them link together. For the (AD, PSENEN) entity pair, the number of co-occurrences is only 3 in our dataset with 12 intermediate entities (so there are 12 indirect paths).

We summarize the results of the gene analysis shown in Fig. 3 in Table 8. At first sight, our proposed method seems not much different from co-occurrence, COALS, and random indexing, but these methods share the weakness that many entities have the same score. Thus, it is hard to interpret these entities’ rankings effectively.

Table 8 Comparison of Alzheimer’s disease–related gene ranking (PKDE4J)

In the PKDE4J result, with 19 AD-related genes, we found 8696 entities co-occurring with AD. To analyze how many AD-related genes occur in the top 20% of (top 1740 ranked) entities, we calculated precision, recall, and F-measure for each method. As shown in Table 9, our proposed method achieved the joint highest F-measure of 65.94% together with the co-occurrence method.

Table 9 Quantitative evaluation in PKDE4J

Pathway analysis

In bioinformatics research involving an intricate network of interactions, pathways analysis is often quite useful. Pathways can help to explain gene function in the context of biological processes.

We applied the proposed method, co-occurrence, COALS [24], and Word2Vec [17] to select the top 20 genes in each ranking list, and used Reactome to do pathway analysis; Fig. 4 shows a genome-wide overview for each method.

Fig. 4

Genome-wide overview

Figure 4 shows that a series of genes are involved in pathways. The yellow marks are pathways that at least have one of the genes in our gene list. Pathways in Reactome are arranged in a hierarchy, the center of each cluster being the root of one top-level pathway. Each step away from the center represents the next level lower in the pathway hierarchy.

In this genome-wide overview, Word2Vec has fewer pathways than the other methods. In the developmental biology cluster, only the proposed method and COALS have related pathways. However, in the hemostasis cluster, only COALS has genes which can be mapped to pathways. In addition, in the signal transduction and immune system clusters, our proposed method has pathways from the root or top-level pathway, and also has a pathway which connects the signal transduction cluster to the metabolism cluster.

The top 20 gene list genome-wide overview shows that from the pathway perspective, our proposed method has better performance for genes identified in (functional) cross clustering and higher-level pathways.

We now analyze these pathways in detail. Table 10 shows the top five most significant pathways by p-value in each gene list [26]. “Entities” are the input genes. “Reactions” can be regarded as the ‘steps’ of pathways: any biological event that changes the state of a biological molecule. “Entities found” is the number of common entities between the submitted data set and the pathway. “Entities ratio” is the proportion of pathway molecules represented by this pathway. “p-value” is the result of statistical testing for over-representation of entities. “Reactions found” counts pathways with at least one molecule in the submitted data set represented. “Reactions ratio” is the proportion of all reactions represented by reactions from this pathway.

Table 10 Top five important pathways sorted by p-value for each gene list

We can see that our proposed method has a higher probability that many entities are found in the same pathways. For example, “Signaling by interleukins” has 7 genes (input gene is 20). However, the gene list selected by Word2Vec has greater dispersion. This may imply that if a disease-related gene is over-represented in the same pathway, then other genes in that pathway may have an impact on the disease.

Table 11 shows the gene rankings by five methods for the top 10 gene rankings by number of pathways.

Table 11 Gene rankings for each method, ordered by pathway number

We summarize the results as shown in Table 11 as Table 12. Our proposed method has clear advantages in selecting genes that can act through more pathways. Genes in the same pathway may have proximate gene expression. Gene expression provides a fundamental basis for genotype to trigger phenotype. Our analysis seems to imply that similar gene expressions may have a homogeneous impact on gene–phenotype association. These genes with similar phenotype associations tend to have a higher chance of co-occurrence in the biological literature. Since our method also considers indirect relations, which can help to link these co-occurrences, the genes which participate in many pathways get higher scores by our method.

Table 12 Top 10 genes ordered by pathway number

Gene–phenotype relationship analysis

Phenotype is the result of comprehensive regulation of molecular events at all levels. Different genotypes can produce the same phenotype, while the same genotype can produce different phenotypes, which makes the scientific problem of genetic regulation from genotype to phenotype highly complex. Therefore, studying the genotype-to-phenotype aspect of genetic regulation is of critical scientific significance, particularly as the biological literature continues to grow exponentially. Genes with the same phenotype are more likely to be researched in one paper, which increases the possibilities for co-occurrence. Our proposed method considers both direct and indirect relations and semantic relatedness for entity pairs, which makes it easier to find genes controlling the same phenotype; this kind of knowledge discovery can help biologists to find new regulatory pathways and mechanisms. Moreover, summarizing the genetic “rules” of disease allows targeting to improve prevention, treatment, and comprehensive measures to reduce morbidity.

For example, the presence of the APOE4 allele is strongly associated with the onset of early-onset familial Alzheimer’s disease. The APOE4 allele is also an important gene for coronary artery disease; in other words, APOE4 has an impact on two phenotypes. Therefore, Alzheimer’s disease and coronary artery disease may share some relations.

Table 13 shows the indirect score for Alzheimer’s disease and coronary artery disease entity pairs by our proposed method. We show the top 20 results by co-occurrence between intermediate entity B and entity C.

Table 13 Indirect relations for Alzheimer’s disease and coronary artery disease

APOE has high co-occurrence with these two diseases, implying that our method can be used to find the related genes for a given phenotype.

We collect the phenotypes of Alzheimer’s disease co-occurrence genes from the OMIM database, ranking by number of common-phenotype genes. Table 14 shows an example. The second column is the co-occurrence gene. The last column is the number of genes.

Table 14 Gene–phenotype ranking by number of common-phenotype genes (PKDE4J)

In Table 15, we compare the top 10 gene rankings, ranked by number of common phenotype genes, and can clearly see that our proposed method has obvious advantages.

Table 15 Top ten gene rankings, ranked by number of common-phenotype genes

Conclusion

With the growth in biomedical literature, how to identify meaningful information effectively from this literature becomes a crucial question. In this paper, we proposed a new semantic relatedness scoring algorithm for entity pairs by incorporating co-occurrence with consideration of both direct and indirect relations via specialized word embeddings. In addition, we used corpus and thesaurus to train word embeddings in order to calculate the semantic relatedness of each entity pair for ranking. We conducted evaluation in four ways:

• 1) We analyzed the top 20 and 50 entities ranked by our proposed method and compared them with co-occurrence, Word2Vec, COALS, and random indexing. The proposed method was able to select the entities that not only highly co-occur but also have more indirect relations for the target entity (in this paper, we used Alzheimer’s disease). For example, the APOE gene is top-ranked by our method but not by the other methods.

• 2) We collected the Alzheimer’s disease related genes from the KEGG database and examined them for ranking positions generated by the five approaches. Our method does not have a great advantage over the others, but it does generate distinct scores for entity pairs, whereas the other methods such as COALS and random indexing produce the same ranking scores, making it difficult to differentiate the degree of association of one entity pair from another.

• 3) We adopted pathway analysis for the top 20 genes listed by four different methods. Pathways allow us a macro perspective on the gene list. Our proposed method achieves better performance at identifying (functional) cross clustering as well as higher-level pathways.

• 4) We also conducted gene–phenotype relationship analysis to examine whether our method has an advantage. We found that the APOE4 gene plays a role in two phenotypes: Alzheimer’s disease and coronary artery disease. The results show that an indirect relation exists between the common gene and these two phenotypes. This means that if phenotypes are given, their common genotype can be identified by our method, which helps to uncover the genetic laws of heredity disease, and can offer better treatments.

This study has two major limitations. First, selection of entity type in the SemRep results was not properly done so as to reduce unnecessary indirect relations. Due to this, the most common entity, brain, achieves a higher score by our ranking method, which considers indirect relations, than by the other methods. Another major limitation is the lack of in-depth analysis of pathway. In a follow-up study, we plan to conduct laboratory experiments on the results identified by the proposed method. In addition, we plan to improve the quality of semantic relatedness scores by incorporating other lexical properties and contextual information for entities buried in biomedical literatures.