Table 3 Hyper-parameters of BERTSUM, in the case of multiple candidate parameter values, the ultimately chosen parameter value is displayed in bold
Parameters | Values |
|---|---|
encoder | (classifier/transformer/rnn) |
batch size | (1000/2000/3000) |
train steps | 10,000 |
dropout | 0.1 |
learning rate | \(2e^{-3} \cdot min\left( step^{-0.5}, step \cdot warmup^{-1.5} \right)\) |
warmup | (1000/10,000) |
optimizer | adam |