Table 4 Pre-training parameters in Chinese humanity and social science
From: HsscBERT: pre-training domain model for the full text of Chinese humanity and social science
HsscBERT_e3 | HsscBERT_e5 | |
|---|---|---|
max_seq_length | 512 | 512 |
learning_rate | 2e-05 | 2e-05 |
per_device_train_batch_size | 32 | 32 |
per_device_eval_batch_size | 16 | 16 |
num train epochs | 3 | 5 |
