Skip to main content
  • Research article
  • Open access
  • Published:

Isotypes of autoantibodies against novel differential 4-hydroxy-2-nonenal-modified peptide adducts in serum is associated with rheumatoid arthritis in Taiwanese women

Abstract

Background

Rheumatoid arthritis (RA) is an autoimmune disorder with systemic inflammation and may be induced by oxidative stress that affects an inflamed joint. Our objectives were to examine isotypes of autoantibodies against 4-hydroxy-2-nonenal (HNE) modifications in RA and associate them with increased levels of autoantibodies in RA patients.

Methods

Serum samples from 155 female patients [60 with RA, 35 with osteoarthritis (OA), and 60 healthy controls (HCs)] were obtained. Four novel differential HNE-modified peptide adducts, complement factor H (CFAH)1211–1230, haptoglobin (HPT)78–108, immunoglobulin (Ig) kappa chain C region (IGKC)2–19, and prothrombin (THRB)328–345, were re-analyzed using tandem mass spectrometric (MS/MS) spectra (ProteomeXchange: PXD004546) from RA patients vs. HCs. Further, we determined serum protein levels of CFAH, HPT, IGKC and THRB, HNE-protein adducts, and autoantibodies against unmodified and HNE-modified peptides. Significant correlations and odds ratios (ORs) were calculated.

Results

Levels of HPT in RA patients were greatly higher than the levels in HCs. Levels of HNE-protein adducts and autoantibodies in RA patients were significantly greater than those of HCs. IgM anti-HPT78−108 HNE, IgM anti-IGKC2−19, and IgM anti-IGKC2−19 HNE may be considered as diagnostic biomarkers for RA. Importantly, elevated levels of IgM anti-HPT78−108 HNE, IgM anti-IGKC2−19, and IgG anti-THRB328−345 were positively correlated with the disease activity score in 28 joints for C-reactive protein (DAS28-CRP). Further, the ORs of RA development through IgM anti-HPT78−108 HNE (OR 5.235, p < 0.001), IgM anti-IGKC2−19 (OR 12.655, p < 0.001), and IgG anti-THRB328−345 (OR 5.761, p < 0.001) showed an increased risk. Lastly, we incorporated three machine learning models to differentiate RA from HC and OA, and performed feature selection to determine discriminative features. Experimental results showed that our proposed method achieved an area under the receiver operating characteristic curve of 0.92, which demonstrated that our selected autoantibodies combined with machine learning can efficiently detect RA.

Conclusions

This study discovered that some IgG- and IgM-NAAs and anti-HNE M-NAAs may be correlated with inflammation and disease activity in RA. Moreover, our findings suggested that IgM anti-HPT78−108 HNE, IgM anti-IGKC2−19, and IgG anti-THRB328−345 may play heavy roles in RA development.

Peer Review reports

Background

Rheumatoid arthritis (RA) is an autoimmune disorder with systemic inflammation and may be induced by oxidative stress that affects inflamed joints [1]. Age, sex, antinuclear antibodies, rheumatoid factor (RF), environmental factors and smoking have been investigated as risk factors of the RA etiopathogenesis [2,3,4]. Several autoantibodies against neoepitopes, including peptide modified with glycation, citrullination, carbamylation, and oxidation and malondialdehyde (MDA), have been measured in RA patients. In clinical practice, Anti-cyclic citrullinated peptide (anti-CCP) antibodies and RF are clinical biomarkers in patients with RA [5, 6]. One of the lipid peroxidation-generated highly bioactive electrophilic oxidation products, 4-hydroxy 2-nonenal (HNE), is produced through oxidative stress-generated reactive oxygen species (ROS)-targeted n-6 polyunsaturated fatty acids that contribute to the pathogenesis of various diseases, including RA, diabetes mellitus (DM), systemic lupus erythematosus (SLE), alcoholic liver disease (ALD), aging, neurodegenerative diseases, chronic obstructive pulmonary disease (COPD), cardiovascular diseases (CDs), and inflammation-driven cancers [7,8,9,10].

HNE has two reactive electrophilic groups, an aldehyde group and an alkene bond, and can react with residues in amino acid. The C = C double bond in HNE can be targeted via Michael addition and has a mass addition at 156 Da in its non-reduced form [alanine (A), arginine (R), cysteine (C), glutamine (Q), histidine (H), lysine (K) and leucine (L)] or 158 Da in its reduced form (CHKRQ) [11,12,13]. The aldehyde group in HNE can react by forming Schiff base adducts and increase mass of 138 Da in the non-reduced form (CHKAL) or 140 Da in the reduced form (CHKR) [11,12,13]. The non-reduced form of the Schiff base adducts (CHKR) further spontaneously rearranges to form a pyrrole adduct with a mass increase of 120 Da [14, 15].

Oxidation-specific epitopes (OSEs) include MDA, HNE, 2-(ω-carboxyethyl) pyrrole, oxidized phosphatidylserine, oxidized cardiolipin, oxidized phosphatidylethanolamine, and phosphocholine-oxidized phospholipids, which react with lipids or free amino groups in side chains of protein [5]. Eggleton et al. indicated that HNE-protein adducts present OSEs and are excellent immunogens to induce autoantibodies [8, 16]. Binder et al. revealed that chronic inflammation can be triggered by accumulation of OSEs [5]. Chou et al. suggested that many anti-OSEs are immunoglobulin M (IgM)-natural autoantibodies (IgM-NAAs) [17], and Gronwall et al. proposed that IgM-NAAs provide protection against pathogenesis of autoimmunity [18].

Levels of HNE-protein adducts in diseased states are higher than those in healthy controls (HCs), including Alzheimer's disease (AD), CDs, Menkes kinky hair disease (MKHD), hyperthyroidism, preeclampsia, mild cognitive impairment (MCI), RA, SLE, and breast cancer [7, 8, 19,20,21,22,23,24,25]. Luczaj et al. reported that amounts of HNE-protein adducts were significantly 1.21-fold greater in the plasma of patients with RA than levels in HCs [24]. Autoantibodies against HNE-derived epitopes are present in a variety of diseases comprising SLE, ALD, and AD [26,27,28]. However, to date, only few studies have reported autoantibodies against HNE-modified human serum albumin in RA [29].

In the present report, four differential novel HNE-modified peptides were re-analyzed via acquired tandem mass spectrometry (MS/MS) data (ProteomeXchange: PXD004546) using PEAKS 7 software (Bioinformatics Solutions, Waterloo, Canada) [6]. Acquired MS/MS spectra were obtained through concanavalin (Con) A affinity chromatography, one-dimensional (1-D) sodium dodecyl sulfate–polyacrylamide gel electrophoresis (SDS-PAGE), in-gel digestion, and nano-liquid chromatography tandem mass spectrometry (nano-LC–MS/MS) in patients with RA versus HCs [6]. We further validated HNE modifications of proteins and examined proteins level in serum and HNE-protein adducts. Moreover, we evaluated the performance of novel diagnostic autoantibodies against unmodified and HNE-modified peptides, which can possibly be used as diagnostic biomarkers for patients with RA, osteoarthritis (OA), and HCs. Herein, we aimed to determine correlations of IgM and IgG autoantibody titers against unmodified and HNE-modified peptide adducts with disease activity and clinical variables in RA patients. Further, the association between higher levels of serum autoantibodies in RA patients with a risk for RA development was assessed compared to HCs. Lastly, to thoroughly evaluate the potential of serum autoantibodies for biomarker development, we incorporated three machine learning algorithms and performed feature selection with WEKA (version 3.8.3) to further classify our subjects.

Methods

Patient samples

Serum samples from 155 female patients [60 with RA (54.8 ± 10.47 years old), 35 with OA (56.2 ± 11.44 years old), and 60 HCs (54.3 ± 8.70 years old)] were obtained from the Division of Allergy, Immunology, and Rheumatology, Department of Internal Medicine and the Department of Laboratory Medicine, Shuang-Ho Hospital (New Taipei City, Taiwan). Patients with RA had received a diagnosis from a rheumatologist and followed the appropriate criteria for classification—either the 2010 American College of Rheumatology (ACR)/European League Against Rheumatism classification criteria [30] or 1987 ACR classification criteria [31]. Further, patients with RA received a disease activity score in 28 joints for C-reactive protein (DAS28-CRP) (4.4 ± 1.67) assessment when they were diagnosed as RA. RA patients included in this study had suffered from this disease for a duration of 5.4 ± 6.41 years (Additional file 1: Table S1). OA patients had been diagnosed according to clinical symptoms with assistance from OA criteria by the ACR [32, 33]. Therapies were given to patients with OA (65% non-steroidal anti-inflammatory drugs (NSAIDs), 11.4% disease-modifying anti-rheumatic drugs (DMARDs)) and RA (42.9% NSAIDs, 99.3% DMARDs) by clinicians (Additional file 1: Table S1). The institutional review board of the study hospital approved this study, and informed consent was provided by all volunteers before participating. Four novel differential HNE-modified peptide adducts were re-identified using PEAKS 7 software (Bioinformatics Solutions) from previous MS/MS data (ProteomeXchange: PXD004546) [6]. Their protein levels were examined by Western blotting with individual randomly age paired serum from 32 patients with RA and 32 HCs. HNE modifications of HNE-modified peptide adducts were assessed through immunoprecipitation (IP) and Western blotting using the pooled Con A-captured serum samples from above-mentioned 32 pairs of serum samples. Further, serum levels of HNE-protein adducts were determined, and isotypes of autoantibodies against unmodified and HNE-modified peptides were evaluated among individual serum of 60 RA and 35 OA patients and 60 HCs. Clinical and demographic characteristics of patients with RA and OA and HCs are summarized in Additional file 1: Table S1. Serum was stored at -20 °C until analyzed.

Novel differential HNE-modified peptide adducts were re-analyzed using PEAKS 7 software

The Peaks PTM module of PEAKS 7 software (Bioinformatics Solutions) was used to identify sequences of HNE-modified peptide from acquired MS/MS spectra against the Universal Protein Resource Knowledgebase, a human protein database (UniProt; http://www.uniprot.org/) containing 157,433 protein entities (UniProt, 2016/11), and those sequences are shown in Additional file 2: Figure S1A. MS/MS data are available through ProteomeXchange with the identifier PXD004546 [6]. S-Pyridylethylation (C)/ + 105.057849 Da was set as the fixed modification, whereas oxidation (M)/ + 15.994915 Da and the following HNE modifications were specified as variables: CHKRQAL/ + 156.11504 Da, CHKRQ/ + 158.13068 Da, CHKAL/ + 138.10446 Da, CHKR/ + 140.12012 Da, and CHKR/ + 120.1916 Da in Additional file 2: Figure S1B. All of the modified MS spectra were identified manually, and fragmented ions were labeled as y, b, y-NH3, and b-H2O ions. Details are provided in the "Additional file 3: Supplementary Information" section.

Con A affinity chromatography and IP-Western blotting

Serum-derived Con A-captured serum proteins were purified using the protocol of Uen et al. [34]. Protein concentrations were examined using a Pierceâ„¢ Coomassie Plus (Bradford) Assay Kit (Thermo Scientific, Waltham, MA, USA) following to the protocol from manufacturer. Con A-captured proteins were used in IP. IP-Western blotting of pooled Con A-captured proteins was used to confirmed modifications of HNE-modified proteins. We used antibodies in IP including complement factor H (CFAH), haptoglobin (HPT), the Ig kappa chain C region (IGIK), and prothrombin (THRB), following to the protocol of Liao et al. [35]. HNE modifications of proteins were detected using a goat polyclonal anti-HNE antibody. Details of experiments are provided in the "Additional file 3: Supplementary Information" section.

Detection of proteins and HNE-protein adducts

Levels of CFAH, HPT, IGKC, and THRB were detected using Western blotting. HNE-protein adducts were quantified using an enzyme-linked immunosorbent assay (ELISA) [36]. All samples were detected in duplicate. Details of the protocol are provided in the "Additional file 3: Supplementary Information" section.

Measurement of autoantibodies against unmodified and HNE-modified peptides

Polypeptides were synthesized and used in the ELISA [35]. Unmodified peptides are presented as CFAH1211−1230, HPT78−108, IGKC2−19, and THRB328−345. HNE-modified peptides, marked as CFAH1211−1230 HNE, HPT78−108 HNE, IGKC2−19 HNE, and THRB328−345 HNE, were prepared using HNE (CAS 75899–68-2, Millipore, Darmstadt, Germany) [37]. Then, CFAH1211−1230 HNE and HPT78−108 HNE were reductively stabilized using NaBH4 [11]. In total, 155 serum samples were evaluated for the presence of IgG and IgM isotypes of anti-unmodified and anti-HNE-modified peptide autoantibodies. All of the samples were detected in duplicate. The protocol details are provided in the "Additional file 3: Supplementary Information" section.

Statistical analysis

The significance of blot densitometric differences, and levels of serum proteins and HNE-protein adducts were determined using Student's t-test. A one-way analysis of variance (ANOVA) was used to examine levels of autoantibody isotypes against unmodified and HNE-modified peptides between RA and OA patients and HCs. Scheffe’s post-hoc test was applied to evaluate the difference of mean between any two groups, as well as a post-hoc test using the Bonferroni method with a 0.0167 adjusted significance level. We used GraphPad Prism (vers. 5.0; GraphPad Software, San Diego, CA, USA) to evaluate differences in Student's t-test between groups, correlations between measurements, and generated receiver operating characteristic (ROC) curves to evaluate the diagnostic performance of autoantibodies. Pearson’s or Spearman’s rank correlation coefficients were used to assess correlations among different parameters. To estimate multivariate-adjusted odds ratios (ORs) and their 95% confidence intervals (CIs) for RA risk, Logistic regression models were performed in this study. The positivity of autoantibody isotypes and HNE-protein adducts was decided by ROC curves. The cut-off value for an ROC curve was determined by Youden index, which represents the sum of sensitivity and 1-specificity, and the maximum value of Youden index is the suitable cut-off point for that curve. Pair-wise comparisons of ROC curves were assessed using MedCalc Statistical Software (vers. 15.4; MedCalc Software, Ostend, Belgium). One-way ANOVA and power were determined using SAS (vers. 9.3; SAS Institute, Cary, NC, USA), and power estimations were calculated according to the ROC analysis. The area under the ROC curve (AUC), sensitivity, and specificity were calculated at a 95% confidence level. The significance level of all statistical tests was set to p < 0.05. For feature selection, we first used ‘Information Gain’ as the attribute evaluator with ‘Ranker’ as the search method in WEKA (version 3.8) [38] to select discriminative features in identify RA patients. Next, we incorporated ten-fold cross-validation to evaluate our model based on decision trees (DT) [39], random forests [40], and support vector machines (SVM) [41] in scikit-learn (version 0.21.3) [42]. Parameter tuning was performed for each training and validation. During model selection, a forward selection algorithm was used to select the most effective combination of features for classification. In forward selection, a feature was selected in to the optimal feature set if adding the feature into the prediction model improved the AUC. To evaluate the predictive performance, we applied a confusion matrix to calculate the accuracy, precision, sensitivity, specificity, and AUC for assessments.

Results

Identification of differential HNE-modified peptide adducts

MS/MS spectra of the four HNE-modified peptides of patients were RA-specific as shown in Additional file 2: Figure S1 and Additional file 1: Table S1. HNE modifications were identified through manual examination. Among HNE-modified peptides, 1211-SHTLRTTCWDGKLEYPTCAK-1230 (CFAH1211−1230, Additional file 2: Figure S1C), 78-AVGDKLPECEADDGCPKPPEIAHGYVEH SVR-108 (HPT78−108, Additional file 2: Figure S1D), 2-TVAAPSVFIFPPSDEQLK-19 (IGKC2−19, Additional file 2: Figure S1E, upper panel), and 328-TFGSGEADCGLRPLFEKK-345 (THRB328−345, Additional file 2: Figure S1F) were identified. The HNE reduced form of Michael adducts of CFAH (at K1230) and HPT (at C92) corresponded to a mass increase of 158.13068 Da. The HNE Michael adducts of IGKC (at A5) and THRB (at K344) had a 156.11504 Da shift in the two residues. Moreover, four HNE-modified peptides that were HC-specific were shown in Additional file 2: Figure S1 and Additional file 1: Table S1: 284− HRTGDEITYQCRNGFYPATRGNTAK−308 (CFAH284−308, Additional file 2: Figure S1G), 162−ILGGHLDAK−170 (HPT162−170. Additional file 2: Figure S1H), 83−VYACEVTHQGLSSPVTKSFNR−103 (IGKC83−103, Additional file 2: Figure S1I), and 328−TFGSGEADCGLRPLFEK−344 (THRB328−344, Additional file 2: Figure S1J) were identified. The HNE Schiff base adducts of CFAH (at K308) corresponded to a mass increase of 138.10446 Da. The HNE Michael adducts of HPT (at A169) and THRB (at L341) caused a 156.11504 Da shift in the two residues, and the HNE-reduced form of Michael adducts of THRB (at C336) corresponded to a mass increase of 140.12012 Da. The HNE-reduced form of Michael adducts of IGKC (at Q91) corresponded to a mass increase of 158.13068 Da. In particular, 2-TVAAPSVFIFPPSDEQLK-19 (IGKC2−19, Additional file 2: Figure S1E, bottom panel) was identified from RA and HC, respectively. The HNE Michael adducts of IGKC (at A4) corresponded to a mass increase of 156.11504 Da.

Validation of HNE modifications on HNE-modified peptide adducts

HNE modifications in four differential HNE-modified peptide adducts were validated by IP-Western blotting with two pooled Con A-captured serum samples from RA and HCs, which detected signals of approximately 184, 43, 25, and 80 kDa, respectively, indicating CFAH, HPT, IGKC, and THRB (Fig. 1).

Fig. 1.
figure 1

4-Hydroxy-2-nonenal (HNE) modification of proteins was validated using IP and Western blotting (a). Proteins were immunoprecipitated from pooled concanavalin (Con) A-captured serum samples (32 healthy controls (HCs) and 32 patients with rheumatoid arthritis (RA)) using anti-complement factor H (CFAH), anti-haptoglobin (HPT), anti-immunoglobulin kappa chain C region (IGKC), and anti-prothrombin (THRB) antibodies and then subjected to Western blotting with anti-HNE antibodies (upper panel). Individually selected random serum samples (HCs and RA patients) were used as controls; these were simultaneously used for Western blotting with anti-HNE antibodies. Percentages of the SDS-PAGE gel and IP loading amounts of Con A-captured serum proteins were 8% and 20 µg, 10% and 20 µg, 12% and 5 µg, and 8% and 20 µg for CFAH, HPT, IGKC, and THRB, respectively. A duplicate gel was stained with Coomassie brilliant blue (CBB) as a loading control, including CFAH (b), HPT (c), IGKC (d), and THRB (e). Red arrows indicate immunoprecipitated proteins

Detection of protein levels and HNE-protein adduct levels

Protein levels of CFAH, IGKC, and THRB from 32 randomly paired individual serum samples from RA patients and HCs showed no significant differences (Fig. 2a, c, d). However, levels of HPT in patients with RA were greatly higher than the levels in HCs (1.24-fold, p < 0.041, Fig. 2b). Serum levels of HNE-protein adducts in RA patients were significantly higher than the levels in OA patients (1.16-fold, p = 0.0311) and HCs (1.20-fold, p = 0.0062, Additional file 1: Table S1).

Fig. 2
figure 2

Protein levels of complement factor H (CFAH), haptoglobin (HPT), immunoglobulin kappa chain C region (IGKC), and prothrombin (THRB) in serum were respectively examined using anti-CFAH (a), anti-HPT (b), anti-IGKC (c), and anti-THRB (d) antibodies through Western blotting. Average densitometric values were calculated from duplicate data. Percentages of the SDS-PAGE gel and loading amounts of serum proteins used in Western blotting were 8% and 20 µg, 10% and 5 µg, 12% and 1 µg, and 8% and 20 µg for CFAH, HPT, IGKC, and THRB, respectively. Membrane Coomassie brilliant blue (CBB) staining was used as a loading control (bottom panel). Student’s t-test was used to determine the significance of blot densitometric differences, levels of serum proteins, and HNE-protein adducts. Values of mean ± 1 standard deviation are also indicated in the figure as horizontal lines

Measuring autoantibodies against unmodified and HNE-modified peptides

The AUC, sensitivity, and specificity were used to assess clinical performances of IgG and IgM that against unmodified and HNE-modified peptides. The ANOVA analysis indicated that differences in all autoantibodies against unmodified and HNE-modified peptides were significant among patients with RA, OA, and HCs (Fig. 3, Table 1).

Fig. 3
figure 3figure 3

Dot plots and receiver operating characteristics (ROC) curves of serum concentrations (absorbance units at 450 nm) of autoantibody isotypes: IgG anti-complement factor H (CFAH)1211–1230 (a), IgG anti-CFAH1211−1230 4-hydroxy-2-nonenal (HNE) (b), IgM anti-CFAH1211−1230 (c), IgM anti-CFAH1211−1230 HNE (d), IgG anti-haptoglobin (HPT)78–108 (e), IgG anti-HPT78−108 HNE (f), IgM anti-HPT78−108 (g), IgM anti-HPT78−108 HNE (h), IgG anti-immunoglobulin kappa chain C region (IGKC)2–19 (i), IgG anti-IGKC2−19 HNE (j), IgM anti-IGKC2−19 (k), IgM anti-IGKC2−19 HNE (l), IgG anti-prothrombin (THRB)328–345 (m), IgG anti-THRB328−345 HNE (n), IgM anti-THRB328−345 (o), and IgM anti-THRB328−345 HNE (p) in healthy controls (HCs), osteoarthritis (OA) patients, and rheumatoid arthritis (RA) patients using an ELISA. OD450, optical density at 450 nm. ANOVA was used to test levels of autoantibody isotypes among patients with RA and OA, and HCs. Scheffe’s post-hoc test was applied to compare the mean difference between any two groups. Values of mean ± 1 standard deviation are also indicated in the figure as horizontal lines

Table 1 Differential 4-hydroxy-2-nonenal (HNE)-modified peptide adducts in rheumatoid arthritis (RA) patients compared to healthy controls (HCs)

Levels of IgG against CFAH1211−1230 in patients with RA were greatly higher than those in patients with OA (1.36-fold, p = 0.0002) and HCs (1.25-fold, p = 0.0007) (Fig. 3a, left panel), and AUC values were 0.66 (with 63.3% sensitivity and 51.7% specificity), and 0.58 (with 60.0% sensitivity and 60.0% specificity) for detecting patients with RA and OA (Table 2). Levels of IgG against CFAH1211−1230 HNE in RA patients were greatly higher than the levels in OA patients by 1.32-fold (p = 0.0006), and HCs by 1.32-fold higher (p = 0.0001) (Fig. 3b, left panel), and AUC values were 0.71 (with 76.7% sensitivity and 55.0% specificity) and 0.52 (with 51.4% sensitivity and 50.0% specificity) for detecting patients with RA and OA (Table 2). HC-RA versus HC-OA showed a statistically significant difference (p = 0.0135, Fig. 3b, right panel, Table 2) in pair-wise comparisons of ROC curves. Further, levels of IgM against CFAH1211−1230 in RA patients were greatly higher than the levels in HCs by 1.21-fold (p = 0.0103) (Fig. 3c, left panel), and AUC values were 0.68 (with 81.7% sensitivity and 53.3% specificity) and 0.68 (with 77.1% sensitivity and 60.0% specificity) for detecting patients with RA and OA (Table 2). Levels of IgM against CFAH1211−1230 HNE in RA patients were greatly higher than the levels in HCs by 1.40-fold (p = 0.0002, Fig. 3d, left panel), and AUC values were 0.70 (with 85.0% sensitivity and 41.7% specificity) and 0.59 (with 80.0% sensitivity and 36.7% specificity) for detecting patients with OA and RA (Table 2). Further, HC-OA versus HC-RA showed a significant difference (p = 0.0421, Fig. 3d, right panel, Table 2) in pair-wise comparisons of ROC curves.

Table 2 Comparisons of the area under the receiver operating characteristic curve (AUC), sensitivity, and specificity of autoantibody isotypes against unmodified and 4-hydroxy-2-nonenal (HNE)-modified peptides in rheumatoid arthritis (RA) patients and osteoarthritis (OA) patients compared to healthy controls (HCs)

Levels of IgG against HPT78−108 were significantly 1.32- (p = 0.0008) and 1.26-fold (p = 0.0003) greater in patients with RA than the levels in OA patients and HCs (Fig. 3e, left panel), and AUC values were 0.67 (with 71.7% sensitivity and 55.0% specificity) and 0.53 (with 51.4% sensitivity and with 46.7% specificity) for detecting patients with RA and OA (Table 2). Levels of anti-HPT78−108 HNE IgG in RA patients were greatly higher than the levels in OA patients and HCs by 1.46- (p < 0.0001) and 1.35-fold (p = 0.0002), respectively (Fig. 3f, left panel), and AUC values were 0.66 (with 71.7% sensitivity and 48.3% specificity) and 0.58 (with 62.9% sensitivity and 55.0% specificity) for detecting patients with RA and OA (Table 2). Levels of IgM against HPT78−108 were 1.35-fold (p = 0.0013) greatly higher in RA patients than HCs (Fig. 3g, left panel), and AUC values were 0.67 (with 71.7% sensitivity and 56.7% specificity) for detecting RA and 0.53 (with 51.4% sensitivity and 50.0% specificity) for detecting OA (Table 2). Levels of anti-HPT78−108 HNE IgM in RA patients were greatly higher than the levels in OA patients by 3.45-fold (p < 0.0001), while RA patients versus HCs was 3.83-fold higher (p < 0.0001) (Fig. 3h, left panel), and AUC values were 0.89 (with 86.7% sensitivity and 66.7% specificity) for detecting RA and 0.58 (with 65.7% sensitivity and 55.0% specificity) for detecting OA (Table 2). Further, HC-RA and HC-OA were statistically significantly different (p < 0.0001, Fig. 3h, right panel, Table 2) in pair-wise comparisons of ROC curves.

Levels of IgG against IGKC2−19 were greatly higher in patients with RA than in patients with OA (1.29-fold, p = 0.0157) and HCs (1.47-fold, p = 0.0003) (Fig. 3i, left panel), and AUC values were 0.72 (with 80.0% sensitivity and 60.0% specificity) for detecting RA and 0.59 (with 65.7% sensitivity and 53.3% specificity) for detecting OA (Table 2). Levels of IgG against IGKC2−19 HNE in RA patients were greatly higher than the levels in OA patients by 1.37-fold (p = 0.0085) and HCs by 1.46-fold (p = 0.0002) (Fig. 3j, left panel), and AUC values were 0.70 (with 78.3% sensitivity and 56.7% specificity) for detecting RA and 0.55 (with 65.7% sensitivity and 50.0% specificity) for detecting OA (Table 2). Levels of IgM against IGKC2−19 in RA patients were greatly higher than the levels in OA patients by 2.24-fold (p < 0.0001) and HCs by 2.48-fold (p < 0.0001) (Fig. 3k, left panel), and AUC values were 0.83 (with 81.7% sensitivity and 60.0% specificity) for RA detection and 0.56 (with 57.1% sensitivity and 50.0% specificity) for detecting OA (Table 2). HC-RA versus HC-OA was statistically significant (p < 0.0001, Fig. 3k, right panel, Table 2), in pair-wise comparisons of ROC curves. Levels of IgM against IGKC2−19 HNE were significantly 1.93- (p < 0.0001) and 2.12-fold (p < 0.0001) greater in RA patients than in OA patients and HCs, respectively (Fig. 3l, left panel), and AUC values were 0.86 (with 83.3% sensitivity and 66.7% specificity) for detecting RA and 0.58 (with 68.6% sensitivity and 51.7% specificity) for detecting OA (Table 2). Further, HC-RA versus HC-OA was statistically significant (p < 0.0001, Fig. 3l, right panel, Table 2) in pair-wise comparisons of ROC curves.

Levels of IgG against THRB328−345 were greatly higher in RA patients than OA patients (1.20-fold, p = 0.0021) and HCs (1.26-fold, p < 0.0001) (Fig. 3m, left panel), and AUC values were 0.70 (with 66.7% sensitivity and 53.3% specificity) for detecting RA and 0.53 (with 62.9% sensitivity and 48.3% specificity) for detecting OA (Table 2). Thus, HC-RA versus HC-OA statistically significantly differed (p = 0.0086, Fig. 3m, right panel, Table 2) in pair-wise comparisons of ROC curves. Levels of IgG against THRB328−345 HNE were significantly 1.20- (p = 0.0011) and 1.35-fold (p < 0.0001) greater in RA patients than in OA patients and HCs (Fig. 3n, left panel), respectively, and AUC values were 0.75 (with 80.0% sensitivity and 53.3% specificity) and 0.60 (with 68.6% sensitivity and 50.0% specificity) for detecting RA and OA (Table 2). HC-RA versus HC-OA was statistically significant (p = 0.0087, Fig. 3n, right panel, Table 2) in pair-wise comparisons of ROC curves. Further, levels of IgM antibodies against anti-THRB328−345 in RA patients was greatly higher than HCs by 1.20-fold (p = 0.0149, Fig. 3o, left panel), and AUC values were 0.63 (with 63.3% sensitivity and 53.3% specificity) and 0.56 (with 60.0% sensitivity and 53.3% specificity) for detecting RA and OA (Table 2). Levels of IgM against anti-THRB328−345 HNE were 1.33- (p = 0.0008) and 1.52-fold (p < 0.0001) greater in RA patients than in OA patients and HCs, respectively (Fig. 3p, left panel), and AUC values were 0.78 (with 76.7% sensitivity and 71.7% specificity) and 0.56 (with 65.7% sensitivity and 46.7% specificity) for detecting RA and OA (Table 2). HC-RA versus HC-OA was statistically significant (p < 0.0001, Fig. 3p, right panel, Table 2) in pair-wise comparisons of ROC curves.

Correlations of serum anti-unmodified and anti-HNE-modified peptide autoantibodies with clinical variables in RA patients

A correlation analysis was conducted of autoantibody reactivities against unmodified and HNE-modified peptides with DAS28-CRP measurements and serum clinical variables (RF, anti-CCP, CRP, ESR, and the HNE-protein adduct) in patients with RA. In Table 3, there were significant positive correlations between DAS28-CRP scores and autoantibodies, including IgM anti-HPT78−108 HNE (r = 0.2703, p = 0.0367), IgM anti-IGKC2−19 (r = 0.2816, p = 0.0293), and IgG anti-THRB328−345 (r = 0.2703, p = 0.0367). RF vs. autoantibodies exhibited significant positive correlations, including IgM anti-HPT78−108 HNE (r = 0.6140, p < 0.0001), IgM anti-IGKC2−19 (r = 0.5674, p < 0.0001), IgM anti-IGKC2−19 HNE (r = 0.5404, p < 0.0001), IgG anti-THRB328−345 (r = 0.614, p < 0.0001), IgG anti-THRB328−345 HNE (r = 0.3072, p = 0.019), and IgM anti-THRB328−345 HNE (r = 0.2845, p = 0.0304). Anti-CCP versus autoantibodies exhibited significant positive correlations, including IgG anti-HPT78−108 HNE (r = 0.2782, p = 0.0314) and IgG anti-THRB328−345 HNE (r = 0.2549, p = 0.0494). ESR vs. autoantibodies exhibited significant positive correlations, including IgM anti-IGKC2−19 (r = 0.2692, p = 0.0376), IgM anti-IGKC2−19 HNE (r = 0.2985, p = 0.0205), and IgM anti-THRB328−345 HNE (r = 0.2597, p = 0.0451). Moreover, HNE-protein adduct vs. autoantibodies exhibited significant positive correlations, including IgM anti-IGKC2−19 (r = 0.2667, p = 0.0394) and IgM anti-IGKC2−19 HNE (r = 0.2709, p = 0.0363). However, autoantibodies exhibited significant negative correlations between DAS28-CRP scores, including IgG anti-IGKC2−19 (r = − 0.3538, p = 0.0056) and IgG anti-IGKC2−19 HNE (r = − 0.3432, p = 0.0073). HNE-protein adducts vs. IgM anti-THRB328−345 (r = − 0.2796, p = 0.0305) had a significantly negative correlation. Otherwise, there were no significant correlations between CRP and the other autoantibodies (Table 3).

Table 3 Correlations between autoantibody isotypes against unmodified and 4-hydroxy-2-nonenal (HNE)-modified peptides and clinical variables in rheumatoid arthritis (RA) patients (n = 60)

Associations of serum anti-unmodified and anti-HNE-modified peptide autoantibodies with RA patients compared to HCs

As shown in Table 4, an age adjusted logistic regression analysis demonstrated that ORs of RA development were significantly associated with levels of autoantibodies against unmodified and HNE-modified peptides in patients with RA compared to HCs: IgG anti-CFAH1121−1230 (OR 3.293, p = 0.013, power = 0.874), IgG anti-CFAH1121−1230 HNE (OR 2.808, p = 0.005, power = 0.870), IgM anti-CFAH1121−1230 (OR 5.204, p < 0.001, power = 0.979), IgM anti-CFAH1121−1230 HNE (OR 2.700, p = 0.004, power = 0.838), IgM anti-HPT78−108 (OR = 2.695, p = 0.005, power = 0.841), IgM anti-HPT78−108 HNE (OR 5.235, p < 0.001, power = 0.985), IgG anti-IGKC2−19 (OR 4.679, p < 0.001, power = 0.974), IgM anti-IGKC2−19 HNE (OR 3.206, p < 0.001, power = 0.905), IgM anti-IGKC2−19 (OR 12.655, p < 0.001, power > 0.999), IgM anti-IGKC2−19 HNE (OR 8.095, p < 0.001, power > 0.999), IgG anti-THRB328−345 (OR 5.761, p < 0.001, power = 0.951), IgG anti-THRB328−345 HNE (OR 9.524, p < 0.001, power = 0.962), and IgM anti-THRB328−345 HNE (OR 5.043, p < 0.001, power = 0.992). IgM anti-IGKC2−19 carried the highest risk of RA (Table 4). The OR results were not considered because of power values less than 0.7, including IgG anti-HPT78−108, IgG anti-HPT78−108 HNE, and IgM anti-THRB328−345. Further, HNE-protein adducts (OR 2.413, p = 0.014, power = 0.743) also demonstrated a high risk of RA development (Table 4).

Table 4 Associations of 4-hydroxy-2-nonenal (HNE)-protein adducts and autoantibody isotypes against unmodified and HNE-modified peptides in rheumatoid arthritis (RA) patients vs. healthy controls (HCs)

Using serum anti-unmodified and anti-HNE-modified peptide autoantibodies to identify RA patients from HCs and OA patients

Experimental results from feature selection indicated that IgM anti-HPT78−108 HNE (HC vs. RA 0.3496, OA vs. RA 0.3496), IgM anti-IGKC2−19(HC vs. RA 0.2967), and IgM anti-IGKC2−19 HNE (HC vs. RA 0.2921) showed discriminative power in identifying RA patients from HC and OA patients (Table 5). Predictive performance of decision trees, random forests, and support vector machines based on all 16 autoantibodies and only forward-selected autoantibodies were summarized in Tables 6 and 7, respectively . For both groups (HC vs. RA and OA vs. RA), we observed that using only forward-selected autoantibodies consistently performed better than all autoantibodies, which supports our assumption that feature selection is effective to identify RA. For HC versus RA, the decision tree achieved an AUC of 0.86, random forest achieved AUC of 0.92, and support vector machine achieved an AUC of 0.82. For OA versus RA, decision tree achieved an AUC of 0.84, random forest achieved an AUC of 0.92, and support vector machine achieved an AUC of 0.88. The ROC plots of these algorithms were presented in Additional file 4: Figure S2. Our results demonstrated that random forest performed better than the other algorithms for predicting RA from HC or OA.

Table 5 Feature importance ranking of 4-hydroxy-2-nonenal (HNE)-protein adducts and autoantibody isotypes against unmodified and HNE-modified peptides in rheumatoid arthritis (RA) patients versus healthy controls (HCs) and osteoarthritis (OA) patients
Table 6 Predictive performance using all 16 features in decision trees, random forests, and support vector machines for healthy controls (HCs) versus rheumatoid arthritis (RA) patients and osteoarthritis (OA) versus RA patients
Table 7 Improved predictive performance using only selected features in decision trees, random forests, and support vector machines for healthy controls (HCs) versus rheumatoid arthritis (RA) patients and osteoarthritis (OA) versus RA patients

Discussion

To the best of our knowledge, this is the first study to investigate autoantibodies isotypes against unmodified and HNE-modified peptides, its correlation with activity of disease in Taiwanese women with RA, and associations of risks for RA compared to HCs. However, a critical limitation should be noted due to our samples used in this study were not strictly selected during the disease progression of RA. Therefore, the efficacy of this test may be affected. Levels of HNE-protein adducts in RA patients were greater than the levels in HCs (Additional file 1: Table S1), which is consistent with results from a previous study [24]. Barrera et al. suggested that HNE-protein adducts also featured a pathogenic contribution of oxidative stress [43]. HNE-protein adducts, OSEs, are recognized as danger signals by innate immune receptors, such as the lectin-like oxidized LDL receptor 1 (LOX1) [44, 45]. HNE-protein adducts (OR 2.413) showed a risk for RA development (Table 4). Chronic inflammation can be triggered by the accumulation of OSEs, an important target of innate immunity, which increases the risk of developing chronic inflammation [5]. Binder et al. indicated that IgM isotypes against OSEs can enhance the clearance and neutralization of proinflammatory effect [5, 16, 17]. If OSEs cannot be efficiently cleared, OSEs would act as damage-associated molecular patterns (DAMPs) that trigger sterile inflammation [46]. Pattern recognition receptors (PRRs) can recognize DAMPs and activate the innate immune response to trigger sterile inflammation [47]. LOX1, a cellular PRR, can recognize and bind to HNE to mediate its uptake and inflammatory effect in atherosclerosis [48]. Macrophages can take up the IgM-NAA-HNE complex by C1q-calreticulin-CD91-dependent or mannose-binding lectin (MBL) and MBL receptor-dependent mechanisms in chronic inflammatory diseases and atherosclerosis [49]. Siloşi et al. reported that B-1 cells produce NAAs (IgM > IgG > IgA) and pathogenic autoantibodies (IgG > IgM > IgA) [50]. However, the boundary line between natural immunity and pathogenic autoimmunity is unclear [50]. Further, IgM-NAAs can control IgG-NAA activity and regulate expression of natural IgG autoreactive repertoire by F(ab')2 fragments of IgG-NAAs in human and mice serum [51, 52]. Chen et al. proved that inhibition of Toll-like receptor (TLR) and IgG-immune complex-mediated inflammatory responses mediate anti-inflammatory features of IgM-NAAs [53]. Moreover, IgG-NAAs may be involved in autoimmune disease pathogenesis, including SLE, Sjögren’s syndrome, and Graves’ disease [50, 54], and anti-OSE NAAs themselves may have a protective effect. Further, we deduced that elevated levels of anti-OSE NAAs may be a risk index of RA development based on protein function and disease activity when oxidative stress occurs over a sustained period in patients with RA (Additional file 1: Table S1, Table 4). In this study, four different novel HNE-modified peptide adducts were identified: CFAH1211−1230, HPT78−108, IGKC2−19, and THRB328−345 (Table 1, Fig. 1).

The biological function of CFAH, a soluble inhibitor of the alternative complement pathway, is to inhibit the inflammatory response through oxidative stress and to protect host tissues from complement-mediated damage [55,56,57]. Okroj et al. reported that complement activation contributes to the pathological process of RA [58]. The complement system is a central innate immune system that participates in eliminating pathogens and promotes inflammatory responses [55]. Weismann et al. indicated that CFAH is able to bind MDA, and as an MDA-binding protein, it blocks the proinflammatory effects that induced by MDA in vivo in mice [59]. Moreover, it was reported that the HNE modification was unbound by CFAH [59]. However, we found that CFAH was modified with HNE at K1230 in RA patients (Table 1, Additional file 2: Figure S1C). Trojnár et al. identified three linear epitopes on serum CFAH (CFAH1157−1171, CFAH1177−1191, and CFAH1207−1226) in atypical hemolytic uremic syndrome (aHUS) [60]. Interestingly, CFAH1211−1230 is also an autoantigen in RA (Fig. 3a, c). The HNE-modified CFAH1211−1230 peptide can enhance autoantibody levels in patients with RA compared to patients with OA and HCs (Fig. 3b, d). Several studies demonstrated that aHUS was associated with the presence of autoantibodies against CFAH [56, 57]. Autoantibodies against CFAH are also present in significant ratios in RA [57]. Insufficient inhibition of CFAH activity may be caused by pathology- associated autoantibodies [56, 58]. Thus, high autoantibody titers against CFAH are not specific to RA, but may be important for pathologic processes in RA. However, IgG anti-CFAH1121−1230, IgG anti-CFAH1121−1230 HNE, IgM anti-CFAH1121−1230, and IgM anti-CFAH1121−1230 HNE were not significantly correlated with DAS28-CRP, RF, CRP, ESR, or HNE-protein adducts, respectively (Table 3); but, IgG anti-CFAH1121−1230 (OR 3.293), IgG anti-CFAH1121−1230 HNE (OR 2.808), IgM anti-CFAH1121−1230 (OR 5.204), and IgM anti-CFAH1121−1230 HNE (OR 2.700) showed risks for RA development (Table 4).

HPT is a hemoglobin-binding protein that can prevent oxidative damage to organs and participates in activating innate and adaptive immune responses [61]. Increased synovial fluid (SF) and serum HPT levels found in RA patients were associated with inflammation and tissue destruction [62]. Yildirim et al. indicated that serum HPT was an acute-phase protein and significantly correlated with disease activity in patients with RA [63]. In this study, we identified one novel HNE modification at C92 on HPT78–108 in RA patients (Table 1, Additional file 2: Figure S1D). Korngold indicated that the HPT-anti-HPT reaction can block HPT-hemoglobin-binding action [64]. Muta et al. reported that the level of the anti-HPT antibody in serum increased and the level of HPT decreased after febrile non-hemolytic transfusion reactions (FNHTRs) [3]. In this study, higher levels of HPT in serum were greatly 1.24-fold higher (p = 0.041) in RA patients than in HCs (Fig. 2b). IgG and IgM against HPT78−108 and HPT78−108 HNE were greatly higher in RA patients than in HCs (Fig. 3e–h). Thus, we inferred that high levels of autoantibodies against HPT may inhibit HPT's function and play a role in the risk of developing RA. Interestingly, IgM anti-HPT78−108 HNE was significantly positively correlated with DAS28-CRP (r = 0.2703) and RF (r = 0.614), and IgG anti-HPT78−108 HNE was greatly positively correlated with anti-CCP (r = 0.2782), but IgG anti-HPT78−108, IgG anti-HPT78−108 HNE, and IgM anti-HPT78−108 were not significantly correlated with DAS28-CRP, RF, anti-CCP, CRP, ESR, or HNE-protein adducts (Table 3). Further, IgM anti-HPT78−108 (OR 2.695) and IgM anti-HPT78−108 HNE (OR 5.235) exhibited a risk of RA development (Table 4, Additional file 5: Table S2).

RFs are autoantibodies against the fragment crystallizable (Fc) region of IgG that, via antigenic stimulation, acts against an abnormal immune response from the host’s natural antibody repertoire [65]. The IgM RF is commonly measured in clinical practice and serves as a marker of RA, other rheumatic diseases, and chronic infections [66]. Sidorov et al. found that the human regulatory RF (regRF) can be induced by the hinge region of Fc fragments of homologous IgG and can prevent rheumatic diseases [2]. RF production can also be stimulated by modified IgG, including agalactosyl IgG, or advanced glycated end-product (AGE)-damaged IgG that are associated with more-severe RA and can play a meaningful role in pathogenesis of RA [67, 68]. In this study, we identified two novel HNE modifications at A4 and A5 on IGKC2−19, which is located on the IgG light chain in RA patients (Table 1, Additional file 2: Figure S1E). IgG and IgM against IGKC2−19 and IGKC2−19 HNE were greatly higher in patients with RA than in HCs (Fig. 3i–l). Thus, we inferred that high levels of autoantibodies against IGKC may promote the risk of developing RA. Indeed, IgM anti-IGKC2−19 was greatly positively correlated with DAS28-CRP (r = 0.2816), RF (r = 0.5674), ESR (r = 0.2692), and HNE-protein adducts (r = 0.2667). However, IgM anti-IGKC2−19 HNE was significantly correlated with RF (r = 0.5404), ESR (r = 0.2985), and HNE-protein adducts (r = 0.2709) (Table 3). Interestingly, IgG anti-IGKC2−19 (r = − 0.3538) and IgG anti-IGKC2−19 HNE (r = − 0.3432) were significantly negatively correlated with DAS28-CRP (Table 3). Additional research is required to determine whether both IgG-NAAs and IgG anti-HNE NAAs have functions similar to regRF. Further, IgG anti-IGKC2−19 (OR 4.679), IgG anti-IGKC2−19 HNE (OR 3.206), IgM anti-IGKC2−19 (OR 12.665), and IgM anti-IGKC2−19 HNE (OR 8.095) showed risks for RA development (Table 4).

RA is characterized by activation of both inflammatory and coagulation processes resulting in erosion of the joints [69]. THRB is transformed into thrombin by a prothrombinase when injury occurs to tissues and then changes via fibrinogen to form fibrin in a coagulation process [70]. Ohba et al. suggested that high levels of thrombin activity in SF via strong mitogenic activity toward synovial fibroblast-like cells play a significant role in the RA pathogenesis [71]. Yang et al. reported that anti-THRB autoantibodies can display prothrombinase activity and contribute to thrombosis in anti-phospholipid syndrome (APS) and SLE [72]. In this study, novel HNE modifications at K344 on THRB 328–345 in RA patients were identified (Table 1, Additional file 2: Figure S1F). Next, levels of IgG and IgM against THRB328−345 and THRB328−345 HNE were greatly higher in patients with RA than in HCs (Fig. 3m–p). Thus, high levels of anti-THRB autoantibodies may be considered as a risk factor for RA. Interestingly, IgG anti-THRB328−345 was greatly positively correlated with DAS28-CRP (r = 0.2703) and RF (r = 0.6140), IgG anti-THRB328−345 HNE was significantly positively correlated with RF (r = 0.3072) and anti-CCP (r = 0.2549), IgM anti-THRB328−345 HNE was greatly positively correlated with RF (r = 0.2845) and ESR (r = 0.2597), but IgM anti-THRB328−345 was significantly negatively correlated with the HNE-protein adduct (r = − 0.2796) as shown in Table 3. Further, IgG anti-THRB328−345 (OR 5.761), IgG anti-THRB328−345 HNE (OR 9.542), and IgM anti-THRB328−345 HNE (OR 5.043) exhibited risks for RA development (Table 4).

Several previous studies showed the feasibility in early diagnosis of autoimmune diseases using a machine learning application for RA. Rodrigo Torres et al. suggested that feature selection can be a powerful tool in biomarker discovery [73]. Therefore, we believe that with appropriate proteomic data and machine learning algorithms, the biomarker candidates we developed can be optimized into a set of highly accurate features. In our experiment, we incorporated feature ranking and a forward selection method to identify IgM anti-HPT78−108 and HNE-protein adducts that can identify RA from HC combined with random forest algorithm. We then compared our results with the accuracy of anti-CCP which was considered a well performance biomarker in RA. Other studies reported that the sensitivity and specificity of anti-CCP were 63% ~ 91.4% and 69.7% ~ 97.6%, respectively [74,75,76,77]. However, we found only 50% of positive anti-CCP in RA samples. Moreover, consistency was observed between statistical analyses of odds ratio and prediction results of classification. Our results from statistical models, feature selection, and machine learning classifiers supported that IgM anti-HPT78−108 HNE, IgM anti-IGKC2−19, and IgM anti-IGKC2−19 HNE showed potential to be developed as biomarkers for RA.

Conclusions

In the present study, we found that some IgG- and IgM-NAAs and anti-HNE IgM-NAAs may be correlated with activity of disease and inflammation in RA. We concluded that increased levels of IgM anti-HPT78−108 HNE, IgM anti-IGKC2−19, and IgM anti-IGKC2−19 HNE in serum can be used as diagnostic biomarkers of RA, and high levels of IgM anti-HPT78−108 HNE, IgM anti-IGKC2−19, and IgG anti-THRB328−345 are related to increased disease activity during RA development.

Availability of data and materials

The data that support the findings of this study are available from Shuang-Ho Hospital (New Taipei City, Taiwan) but restrictions apply to the availability of these data, which were used under license for the current study, and so are not publicly available. Data are however available from the authors upon reasonable request and with permission of the hospital.

Abbreviations

1-D SDS-PAGE:

One-dimensional sodium dodecylsulfate polyacrylamide gel electrophoresis

ACR:

American College of Rheumatology

AD:

Alzheimer's disease

aHUS:

Atypical hemolytic uremic syndrome

ALD:

Alcoholic liver disease

AUC:

Area under the receiver operating characteristic curve

CBB:

Coomassie brilliant blue

CD:

Cardiovascular disease

CFAH:

Complement factor H

Con A:

Concanavalin A

COPD:

Chronic obstructive pulmonary disease

CRP:

C-reactive protein

DAMP:

Damage-associated molecular pattern

DAS28-CRP:

Disease activity score in 28 joints

DM:

Diabetes mellitus

DMARD:

Disease-modifying anti-rheumatic drug

ELISA:

Enzyme-linked immunosorbent assay

ESR:

Erythrocyte sedimentation rate

FDR:

False detection rate

HC:

Healthy control

HNE:

4-Hydroxy-2-nonenal

Ig:

Immunoglobulin

HPT:

Haptoglobin

IGKC:

Immunoglobulin kappa chain C region

IP:

Immunoprecipitation

MBL:

Mannose-binding lectin

MCI:

Mild cognitive impairment

MKHD:

Menkis kinky hair disease

MDA:

Malondialdehyde

NAA:

Natural autoantibody

nano-LC–MS/MS:

Nano-liquid chromatography-tandem mass spectrometry

NSAID:

Non-steroidal anti-inflammatory drug

OA:

Osteoarthritis

ANOVA:

Analysis of variance

OR:

Odds ratio

CI:

Confidence interval

OSE:

Oxidation-specific epitope

PRR:

Pattern recognition receptor

PTM:

Post-translational modification

RF:

Rheumatoid factor

Anti-CCP:

Anti-cyclic citrullinated peptide

ROC:

Receiver operating characteristic

SF:

Synovial fluid

SLE:

System lupus erythematosus

THRB:

Prothrombin

References

  1. Quinonez-Flores CM, et al. Oxidative stress relevance in the pathogenesis of the rheumatoid arthritis: a systematic review. Biomed Res Int. 2016;2016:6097417.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  2. Kochi Y, et al. Ethnogenetic heterogeneity of rheumatoid arthritis-implications for pathogenesis. Nat Rev Rheumatol. 2010;6(5):290–5.

    Article  CAS  PubMed  Google Scholar 

  3. Kuo CF, et al. Rheumatoid arthritis prevalence, incidence, and mortality rates: a nationwide population study in Taiwan. Rheumatol Int. 2013;33(2):355–60.

    Article  PubMed  Google Scholar 

  4. Turesson C, et al. Extra-articular disease manifestations in rheumatoid arthritis: incidence trends and risk factors over 46 years. Ann Rheum Dis. 2003;62(8):722–7.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  5. Binder CJ, Papac-Milicevic N, Witztum JL. Innate sensing of oxidation-specific epitopes in health and disease. Nat Rev Immunol. 2016;16(8):485–97.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  6. Liao CC, et al. Isotypes of autoantibodies against differentially expressed novel malondialdehyde-modified peptide adducts in serum of Taiwanese women with rheumatoid arthritis. J Proteomics. 2018;170:141–50.

    Article  CAS  PubMed  Google Scholar 

  7. Shoeb M, et al. 4-Hydroxynonenal in the pathogenesis and progression of human diseases. Curr Med Chem. 2014;21(2):230–7.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  8. Eggleton P, et al. Detection and isolation of human serum autoantibodies that recognize oxidatively modified autoantigens. Free Radic Biol Med. 2013;57:79–91.

    Article  CAS  PubMed  Google Scholar 

  9. Jaganjac M, et al. Reactive aldehydes–second messengers of free radicals in diabetes mellitus. Free Radic Res. 2013;47(Suppl 1):39–48.

    Article  CAS  PubMed  Google Scholar 

  10. Gargiulo S, et al. Oxysterols and 4-hydroxy-2-nonenal contribute to atherosclerotic plaque destabilization. Free Radic Biol Med. 2017;111:140–50.

    Article  CAS  PubMed  Google Scholar 

  11. Zhao J, et al. Mass spectrometric evidence of malonaldehyde and 4-hydroxynonenal adductions to radical-scavenging soy peptides. J Agric Food Chem. 2012;60(38):9727–36.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  12. Lv L, et al. Effect of 4-hydroxy-2-nonenal treatment on the IgE binding capacity and structure of shrimp (Metapenaeus ensis) tropomyosin. Food Chem. 2016;212:313–22.

    Article  CAS  PubMed  Google Scholar 

  13. Uchida K. 4-Hydroxy-2-nonenal: a product and mediator of oxidative stress. Prog Lipid Res. 2003;42(4):318–43.

    Article  CAS  PubMed  Google Scholar 

  14. Sayre LM, et al. Protein adducts generated from products of lipid oxidation: focus on HNE and one. Drug Metab Rev. 2006;38(4):651–75.

    Article  CAS  PubMed  Google Scholar 

  15. Shireman LM, et al. Glutathione transferase A4–4 resists adduction by 4-hydroxynonenal. Arch Biochem Biophys. 2010;504(2):182–9.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  16. Binder CJ. Natural IgM antibodies against oxidation-specific epitopes. J Clin Immunol. 2010;30(Suppl 1):S56-60.

    Article  CAS  PubMed  Google Scholar 

  17. Chou MY, et al. Oxidation-specific epitopes are important targets of innate immunity. J Intern Med. 2008;263(5):479–88.

    Article  CAS  PubMed  Google Scholar 

  18. Gronwall C, et al. Depressed serum IgM levels in SLE are restricted to defined subgroups. Clin Immunol. 2017;183:304–315.

    Article  PubMed  CAS  Google Scholar 

  19. Jin H, et al. Oxidatively modified proteins as plasma biomarkers in breast cancer. Cancer Biomark. 2013;13(3):193–200.

    Article  PubMed  CAS  Google Scholar 

  20. Yokoyama A, et al. Cerebellar expression of copper chaperone for superoxide, cytosolic cu/zn-superoxide dismutase, 4-hydroxy-2-nonenal, acrolein and heat shock protein 32 in patients with menkes kinky hair disease: immunohistochemical study. Yonago Acta Med. 2014;57(1):23–35.

    CAS  PubMed  PubMed Central  Google Scholar 

  21. Lampka M, et al. Oxidative stress markers during a course of hyperthyroidism. Endokrynol Pol. 2006;57(3):218–22.

    PubMed  Google Scholar 

  22. Hnat MD, et al. Heat shock protein-70 and 4-hydroxy-2-nonenal adducts in human placental villous tissue of normotensive, preeclamptic and intrauterine growth restricted pregnancies. Am J Obstet Gynecol. 2005;193(3 Pt 1):836–40.

    Article  CAS  PubMed  Google Scholar 

  23. Uchida K, et al. Michael addition-type 4-hydroxy-2-nonenal adducts in modified low-density lipoproteins: markers for atherosclerosis. Biochemistry. 1994;33(41):12487–94.

    Article  CAS  PubMed  Google Scholar 

  24. Luczaj W, et al. The onset of lipid peroxidation in rheumatoid arthritis: consequences and monitoring. Free Radic Res. 2016;50(3):304–13.

    Article  CAS  PubMed  Google Scholar 

  25. Wang G, et al. Markers of oxidative and nitrosative stress in systemic lupus erythematosus: correlation with disease activity. Arthritis Rheum. 2010;62(7):2064–72.

    CAS  PubMed  PubMed Central  Google Scholar 

  26. Al-Shobaili HA, et al. Antibodies against 4-hydroxy-2-nonenal modified epitopes recognized chromatin and its oxidized forms: role of chromatin, oxidized forms of chromatin and 4-hydroxy-2-nonenal modified epitopes in the etiopathogenesis of SLE. Dis Markers. 2012;33(1):19–34.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  27. Mottaran E, et al. Lipid peroxidation contributes to immune reactions associated with alcoholic liver disease. Free Radic Biol Med. 2002;32(1):38–45.

    Article  CAS  PubMed  Google Scholar 

  28. Takeda A, et al. In Alzheimer’s disease, heme oxygenase is coincident with Alz50, an epitope of tau induced by 4-hydroxy-2-nonenal modification. J Neurochem. 2000;75(3):1234–41.

    Article  CAS  PubMed  Google Scholar 

  29. Khan F, et al. Unsaturated aldehyde, 4-hydroxynonenal (HNE) alters the structural integrity of HSA with consequences in the immuno-pathology of rheumatoid arthritis. Int J Biol Macromol. 2018;112:306–14.

    Article  CAS  PubMed  Google Scholar 

  30. Aletaha D, et al. 2010 Rheumatoid arthritis classification criteria: an American College of Rheumatology/European League Against Rheumatism collaborative initiative. Arthritis Rheum. 2010;62(9):2569–81.

    Article  PubMed  Google Scholar 

  31. Arnett FC, et al. The American Rheumatism Association 1987 revised criteria for the classification of rheumatoid arthritis. Arthritis Rheum. 1988;31(3):315–24.

    Article  CAS  PubMed  Google Scholar 

  32. Altman R, et al. The American College of Rheumatology criteria for the classification and reporting of osteoarthritis of the hand. Arthritis Rheum. 1990;33(11):1601–10.

    Article  CAS  PubMed  Google Scholar 

  33. Altman R, et al. Development of criteria for the classification and reporting of osteoarthritis. Classification of osteoarthritis of the knee. Diagnostic and Therapeutic Criteria Committee of the American Rheumatism Association. Arthritis Rheum. 1986;29(8):1039–49.

    Article  CAS  PubMed  Google Scholar 

  34. Uen YH, et al. Comparative proteomics, network analysis and post-translational modification identification reveal differential profiles of plasma Con A-bound glycoprotein biomarkers in gastric cancer. J Proteomics. 2013;83:197–213.

    Article  CAS  PubMed  Google Scholar 

  35. Liao CC, et al. Comparative analysis of novel autoantibody isotypes against citrullinated-inter-alpha-trypsin inhibitor heavy chain 3 (ITIH3)(542–556) peptide in serum from Taiwanese females with rheumatoid arthritis, primary Sjogren’s syndrome and secondary Sjogren’s syndrome in rheumatoid arthritis. J Proteomics. 2016;141:1–11.

    Article  CAS  PubMed  Google Scholar 

  36. Weber D, et al. Measurement of HNE-protein adducts in human plasma and serum by ELISA-Comparison of two primary antibodies. Redox Biol. 2013;1:226–33.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  37. Fenaille F, Tabet JC, Guy PA. Identification of 4-hydroxy-2-nonenal-modified peptides within unfractionated digests using matrix-assisted laser desorption/ionization time-of-flight mass spectrometry. Anal Chem. 2004;76(4):867–73.

    Article  CAS  PubMed  Google Scholar 

  38. Smith TC, Frank E. Introducing Machine Learning Concepts with WEKA. Methods Mol Biol. 2016;1418:353–78.

    Article  PubMed  Google Scholar 

  39. Quinlan JR. Induction of decision trees. Mach Learn. 1986;1(1):81–106.

    Article  Google Scholar 

  40. Breiman L. Random forests. Mach Learn. 2001;45(1):5–32.

    Article  Google Scholar 

  41. Burges CJC. A tutorial on support vector machines for pattern recognition. Data Min Knowl Disc. 1998;2(2):121–67.

    Article  Google Scholar 

  42. Pedregosa F, Varoquaux G, Gramfort A, Michel V, et al. Scikit-learn: machine learning in Python. Mach Learn Res. 2011;12:2825–30.

    Google Scholar 

  43. Barrera G, et al. Role of 4-hydroxynonenal-protein adducts in human diseases. Antioxid Redox Signal. 2015;22(18):1681–702.

    Article  CAS  PubMed  Google Scholar 

  44. Sokolov AV, et al. Thrombin inhibits the anti-myeloperoxidase and ferroxidase functions of ceruloplasmin: relevance in rheumatoid arthritis. Free Radic Biol Med. 2015;86:279–94.

    Article  CAS  PubMed  Google Scholar 

  45. Kumano-Kuramochi M, et al. Identification of 4-hydroxy-2-nonenal–histidine adducts that serve as ligands for human lectin-like oxidized LDL receptor-1. Biochem J. 2012a;442(1):171–80.

    Article  CAS  PubMed  Google Scholar 

  46. Chen GY, Nunez G. Sterile inflammation: sensing and reacting to damage. Nat Rev Immunol. 2010;10(12):826–37.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  47. Miller YI, et al. Oxidation-specific epitopes are danger-associated molecular patterns recognized by pattern recognition receptors of innate immunity. Circ Res. 2011;108(2):235–48.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  48. Kumano-Kuramochi M, et al. Identification of 4-hydroxy-2-nonenal-histidine adducts that serve as ligands for human lectin-like oxidized LDL receptor-1. Biochem J. 2012b;442(1):171–80.

    Article  CAS  PubMed  Google Scholar 

  49. Chou MY, et al. Oxidation-specific epitopes are dominant targets of innate natural antibodies in mice and humans. J Clin Invest. 2009;119(5):1335–49.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  50. Silosi I, et al. The role of autoantibodies in health and disease. Rom J Morphol Embryol. 2016;57(2 Suppl):633–8.

    PubMed  Google Scholar 

  51. Adib M, et al. IgG autoantibody activity in normal mouse serum is controlled by IgM. J Immunol. 1990;145(11):3807–13.

    Article  CAS  PubMed  Google Scholar 

  52. Hurez V, Kaveri S-V, Kazatchkine MD. Expression and control of the natural autoreactive IgG repertoire in normal human serum. Eur J Immunol. 1993;23(4):783–9.

    Article  CAS  PubMed  Google Scholar 

  53. Chen Y, et al. IgM antibodies to apoptosis-associated determinants recruit C1q and enhance dendritic cell phagocytosis of apoptotic cells. J Immunol. 2009;182(10):6031–43.

    Article  CAS  PubMed  Google Scholar 

  54. Bagavant H, Fu SM. Pathogenesis of kidney disease in systemic lupus erythematosus. Curr Opin Rheumatol. 2009;21(5):489–94.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  55. Ferreira VP, Pangburn MK, Cortes C. Complement control protein factor H: the good, the bad, and the inadequate. Mol Immunol. 2010;47(13):2187–97.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  56. Foltyn Zadura A, et al. Factor H autoantibodies in patients with antiphospholipid syndrome and thrombosis. J Rheumatol. 2015;42(10):1786–93.

    Article  PubMed  CAS  Google Scholar 

  57. Foltyn Zadura A, et al. Factor H autoantibodies and deletion of Complement Factor H-Related protein-1 in rheumatic diseases in comparison to atypical hemolytic uremic syndrome. Arthritis Res Ther. 2012;14(4):R185.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  58. Okroj M, et al. Rheumatoid arthritis and the complement system. Ann Med. 2007;39(7):517–30.

    Article  CAS  PubMed  Google Scholar 

  59. Weismann D, et al. Complement factor H binds malondialdehyde epitopes and protects from oxidative stress. Nature. 2011;478(7367):76–81.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  60. Trojnar E, et al. Analysis of linear antibody epitopes on factor H and CFHR1 using sera of patients with autoimmune atypical hemolytic uremic syndrome. Front Immunol. 2017;8:302.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  61. Tan W, et al. High serum level of haptoglobin is associated with the response of 12 weeks methotrexate therapy in recent-onset rheumatoid arthritis patients. Int J Rheum Dis. 2016;19(5):482–9.

    Article  CAS  PubMed  Google Scholar 

  62. Niedermeier W, Cross R, Beetham WP Jr. The concentration of haptoglobin in synovial fluid of patients with rheumatoid arthritis. Arthritis Rheum. 1965;8:355–60.

    Article  CAS  PubMed  Google Scholar 

  63. Yildirim K, et al. Associations between acute phase reactant levels and disease activity score (DAS28) in patients with rheumatoid arthritis. Ann Clin Lab Sci. 2004;34(4):423–6.

    PubMed  Google Scholar 

  64. Korngold L. The effect of hemoglobin on the haptoglobin-anti-haptoglobin reaction. Immunochemistry. 1965;2(2):103–6.

    Article  CAS  PubMed  Google Scholar 

  65. Carayannopoulos MO, et al. Evidence that human immunoglobulin M rheumatoid factors can Be derived from the natural autoantibody pool and undergo an antigen driven immune response in which somatically mutated rheumatoid factors have lower affinities for immunoglobulin G Fc than their germline counterparts. Scand J Immunol. 2000;51(4):327–36.

    Article  CAS  PubMed  Google Scholar 

  66. Ingegnoli F, Castelli R, Gualtierotti R. Rheumatoid factors: clinical applications. Dis Markers. 2013;35(6):727–34.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  67. Das H, et al. Diagnostic value of antiagalactosyl IgG antibodies in rheumatoid arthritis. Clin Rheumatol. 2004;23(3):218–22.

    Article  PubMed  Google Scholar 

  68. Newkirk MM, et al. Advanced glycation end-product (AGE)-damaged IgG and IgM autoantibodies to IgG-AGE in patients with early synovitis. Arthritis Res Ther. 2003;5(2):R82-90.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  69. Kern A, et al. Alterations of the thrombin generation profile in rheumatoid arthritis. J Thromb Thrombolysis. 2016;41(3):359–64.

    Article  CAS  PubMed  Google Scholar 

  70. Tomczyk M, et al. Bidirectional functions of thrombin on fibrinolysis: Evidence of thrombin-dependent enhancement of fibrinolysis provided by spontaneous plasma clot lysis. Thromb Res. 2016;143:28–33.

    Article  CAS  PubMed  Google Scholar 

  71. Ohba T, et al. Thrombin in the synovial fluid of patients with rheumatoid arthritis mediates proliferation of synovial fibroblast-like cells by induction of platelet derived growth factor. J Rheumatol. 1996;23(9):1505–11.

    CAS  PubMed  Google Scholar 

  72. Yang YH, et al. Identification of anti-prothrombin antibodies in the anti-phospholipid syndrome that display the prothrombinase activity. Rheumatology (Oxford). 2010;49(1):34–42.

    Article  CAS  Google Scholar 

  73. Torres R, Judson-Torres RL. Research Techniques Made Simple: Feature Selection for Biomarker Discovery. J Invest Dermatol. 2019;139(10):2068-2074e1.

    Article  CAS  PubMed  Google Scholar 

  74. Meyer O, et al. Serial determination of cyclic citrullinated peptide autoantibodies predicted five-year radiological outcomes in a prospective cohort of patients with early rheumatoid arthritis. Arthritis Res Ther. 2006;8(2):R40.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  75. Fernandez-Suarez A, et al. Efficacy of three ELISA measurements of anti-cyclic citrullinated peptide antibodies in the early diagnosis of rheumatoid arthritis. Clin Chem Lab Med. 2005;43(11):1234–9.

    Article  CAS  PubMed  Google Scholar 

  76. Sauerland U, et al. Clinical utility of the anti-CCP assay: experiences with 700 patients. Ann N Y Acad Sci. 2005;1050:314–8.

    Article  CAS  PubMed  Google Scholar 

  77. Ates A, Karaaslan Y, Aksaray S. Predictive value of antibodies to cyclic citrullinated peptide in patients with early arthritis. Clin Rheumatol. 2007;26(4):499–504.

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

The authors thank Ms. Tzu-Yun Yu (Research Center of Biostatistics, College of Management of Taipei Medical University, Taipei, Taiwan) who provided consultant services for power calculations.

Funding

This study and the APC were supported by the Ministry of Science and Technology (MOST) in Taiwan under grant numbers MOST104-2314-B-038-026 and MOST109-2221-E-038-018 to Ching-Yu Lin and Emily Chia-Yu Su, respectively. The sponsor had no role in the research design or contents of the manuscript for publication.

Author information

Authors and Affiliations

Authors

Contributions

CYL, CCC, YSC, KLT, SHL and ECYS conceived and designed the experiments; YYL, CCT, HWC, IJT and YFL performed the experiments; JHC and YYL analyzed the data; and CYL, CCC and ECYS wrote the paper. All authors have read and approved the manuscript.

Corresponding authors

Correspondence to Ching-Yu Lin or Emily Chia-Yu Su.

Ethics declarations

Ethics approval and consent to participate

This study was approved by the Taipei Medical University-Joint Institutional Review Board, and all volunteers signed an informed consent form before participating in the study (No. 201104003 (2011/06/22) and 201501059 (2015/05/09)).

Consent of publication

Not applicable.

Competing interests

The authors declare that there are no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1: Table S1.

Demographic and clinical characteristics of individual subjects contributing to serum for healthy controls (HCs), and patients with osteoarthritis (OA) and rheumatoid arthritis (RA).

Additional file 2: Fig. S1.

Differential 4-hydroxy-2-nonenal (HNE)-modified peptide adducts were re-analyzed through PEAKS 7 using previous MS/MS spectra (ProteomeXchange: PXD004546). Acquired MS/MS spectra were obtained through pooled concanavalin (Con) A-captured serum proteins (nine rheumatoid arthritis (RA) and nine healthy control (HC) pooled samples), 1-D SDS-PAGE, in-gel digestion, and nano-LC-MS/MS (A). HNE reacts with amino acid residues of proteins to form HNE-protein adducts by Michael addition and Schiff base adducts, respectively (B). Representative MS/MS spectrum of the 1211-SHTLRTTCWDGKLEYPTCAK-1230 peptide sequence and the modified peptide bearing an HNE modification at the K1230 residue in RA patients (C). A representative MS/MS spectrum of the peptide sequence 78-AVGDKLPECEADDGCPKPPEIAHGYVEH SVR-108 and the modified peptide bearing the HNE modification at the C92 residue in RA patients (D). The MS/MS spectrum 2-TVAAPSVFIFPPSDEQLK-19 and the modified peptide bearing the HNE modification at the A5 residue in RA patients (E, upper panel); 2-TVAAPSVFIFPPSDEQLK-19 and the modified peptide bearing the HNE modification at the A4 residues in RA patients and HCs (E, bottom panel). Representative MS/MS spectrum of 328-TFGSGEADCGLRPLFEKK-345 and the modified peptide bearing the HNE modification at the K344 residue in RA (F). The MS/MS spectrum 284-HRTGDEITYQCRNGFYPATRGNTAK-308 and the modified peptide bearing the HNE modification at the K308 residue in HCs (G). A representative MS/MS spectrum of the peptide sequence 162-ILGGHLDAK-170 and the modified peptide bearing the HNE modification at the A169 residues in HCs (H). Representative MS/MS spectrum of 83-VYACEVTHQGLSSPVTKSFNR-103 and the modified peptide bearing the HNE modification at the Q91 residue in HCs (I). The MS/MS spectrum 328-TFGSGEADCGLRPLFEK-344 and the modified peptide bearing the HNE modification at the C336 and L341 residues in HCs (J).

Additional file 3.

Supplementary methods.

Additional file 4: Fig. S2.

Comparison of receiver operating characteristics (ROC) curves from unselected features and selected features in (A) decision tree, (B) random forest classifier, and (C) support vector machine classifier.

Additional file 5: Table S2.

Sequences of unmodified and 4-hydroxy-2-nonenal (HNE)-modified peptides.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tsai, KL., Chang, CC., Chang, YS. et al. Isotypes of autoantibodies against novel differential 4-hydroxy-2-nonenal-modified peptide adducts in serum is associated with rheumatoid arthritis in Taiwanese women. BMC Med Inform Decis Mak 21, 49 (2021). https://doi.org/10.1186/s12911-020-01380-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12911-020-01380-y

Keywords