Table 9 Continuous pretraining performance (full dataset) on Yi-1.5-9B.

From: Localized large language model TCNNet 9B for Taiwanese networking and cybersecurity

Base model

Perplexity

Yi-1.5-9B

3.724