Method | Embedding | Classifier | Precision | Recall | Accuracy | F1 |
---|---|---|---|---|---|---|
Fraser et al. [20] | 35 Hand-Crafted Features | LR | – | – | 81.92 | – |
Yancheva et al. [21] | 12 Cluster-Based Features + LS&A | Random forest | 80.00 | 80.00 | 80.00 | 80.00 |
Sirts et al. [22] | Cluster+PID+SID Features | LR | 74.4 ±1.5 | 72.5 ±1.2 | - | 72.7 ±1.2 |
Hernández et al. [48] | 105 Hand-Crafted Features | SVM | 81.00 | 81.00 | 79.00 | 81.00 |
Fritsch et al. [30] | One-Hot Word Embedding Sequence | – | – | 86 | 85.6 | – |
Pan et al. [31] | GloVe Word Embedding Sequence | Bi-LSTM\(\mid\)GRU Hierarchical Attention | 84.02 | 84.97 | – | 84.43 |
Li et al. [26] | 185 Hand-Crafted Features | LR | – | – | 77 | – |
Fraser et al. [27] | Info and LM Features | SVM | – | – | 75 | 77 |
CNN + SSA | GloVe Word Embedding Sequence | CNN | 76.38 ±8.49 | 77.47 ±8.97 | 76.48 ±5.88 | 76.36 ±5.91 |
BiLSTM + SSA | GloVe Word Embedding Sequence | Bi-LSTM | 74.71 ±1.92 | 75.00 ±14.82 | 75.51 ±5.77 | 74.22 ±8.71 |
BiLSTM + CA | GloVe Word Embedding Sequence | Bi-LSTM | 78.40 ±6.60 | 73.95 ±12.96 | 77.36 ±6.19 | 75.43 ±7.83 |
T-BERTBase-LR | BERTBase (Text Level) | LR | 85.09 ±3.11 | 78.69 ±8.35 | 82.76 ±3.74 | 81.51 ±4.73 |
T-BERTLarge-LR | BERTLarge (Text Level) | LR | 88.21 ±5.33 | 80.86 ±7.58 | 85.10 ±3.43 | 84.04 ±3.93 |
T-XLNetBase-LR | XLNetBase (Text Level) | LR | 84.74 ±6.31 | 79.26 ±7.72 | 81.92 ±5.88 | 81.75 ±6.19 |
T-XLNetLarge-LR | XLNetLarge (Text Level) | LR | 82.30 ±5.15 | 83.83 ±4.34 | 82.87 ±3.14 | 82.86 ±2.60 |
T-XLM-LR | XLM (Text Level) | LR | 80.31 ±5.29 | 79.13 ±8.43 | 80.21 ±4.94 | 79.49 ±5.76 |
S-BERTBase-LR | BERTBase (Sentence Level) | LR | 90.31 ±7.36 | 76.52 ±8.06 | 84.46 ±6.31 | 82.72 ±7.21 |
S-BERTLarge-LR | BERTLarge (Sentence Level) | LR | 90.57 ±3.18 | 84.34 ±7.58 | 88.08 ±4.48 | 87.23 ±5.20 |
S-BERTLarge-LR-BiLSTM | BERTLarge (Sentence Level) | LR | 89.06 ±5.19 | 77.71 ±7.33 | 85.19 ±4.92 | 83.61 ±5.69 |
S-BERTLarge-BiLSTM | BERTLarge (Sentence Level) | BiLSTM | 87.98 ±5.31 | 75.03 ±5.99 | 83.43 ±5.51 | 81.49 ±5.31 |
S-XLNetBase-LR | XLNetBase (Sentence Level) | LR | 83.19 ±6.39 | 74.34 ±8.12 | 80.00 ±5.48 | 78.32 ±6.16 |
S-XLNetLarge-LR | XLNetLarge (Sentence Level) | LR | 76.95 ±6.62 | 71.30 ±8.29 | 75.31 ±5.56 | 73.75 ±6.14 |
S-XLM-LR | XLM (sentence level) | LR | 84.00 ±4.74 | 73.47 ±9.80 | 80.21 ±5.47 | 78.14 ±6.72 |