Introduction

The global pharmaceutical industry is a rapidly evolving sector shaped by ongoing innovation and discovery. However, regulatory affairs remain one of the most crucial factors driving this industry, which ensure regulatory compliance. Dossier submissions and marketing applications to agencies, such as the Food and Drug Administration in the US (FDA), the European Medicines Agency (EMA), and National Medical Products Administration (NMPA) in China and other governmental agencies that oversee drug development1,2,3,4 are time-consuming. Undoubtedly, a new drug project for any pharmaceutical business is a costly program that is both full of risks and challenges5. In its essence, despite the seamless early-stage research and development of a new drug, the necessity for companies to invest a significant amount of time and resources into the preparation and translation of registration dossiers stems from the differences in languages and in international regulations across countries, which hinder the new drug market access6.

Large language models (LLMs) have demonstrated exceptional performance in natural language processing tasks and play a prominent role across multiple critical stages of the pharmaceutical lifecycle. From drug discovery and design to clinical trials and patient management, these models provide robust support for enhancing operational efficiency, reducing costs, and optimizing decision-making processes7,8,9. For instance, the SeeTrials10, a tool developed by Intelligent Medical Objects(US), leverages OpenAI’s GPT-4 to efficiently extract safety and efficacy data from clinical trial abstracts. This assists researchers and study designers to quickly grasp the methodologies and outcomes of analoguous studies, thereby supporting decision-making in clinical trials. Beyond trial design, LLM-powered tools like TrialGPT11 utilize end-to-end retrieval, matching, and ranking capabilities to accurately predict patient eligibility while providing explanations. This not only shortens patient screening timelines but also significantly improves the efficiency of matching individuals with appropriate clinical trials.

In new drug application (NDA) submissions, LLMs automate the review of submitted materials to ensure completeness, consistency, and regulatory compliance. By enabling companies to proactively identify and address deficiencies in their submissions, these models streamline the approval workflow and mitigate delays in regulatory review12. In patient care and management, the Foresight model13 — trained on UK hospital patient data—exhibits strong potentials in disease risk prediction, differential diagnosis, and personalized medication recommendations. Moreover, LLMs also excel in translation tasks spanning literature and law. They provide efficient support for cross-lingual knowledge transfer, legal translation and case study analysis14,15.

Although LLMs have showcased impressive applications in the pharmaceutical sector, they face notable challenges16,17,18. Specifically, large-parameter LLMs impose substantial demands on computational resources for training and inference—resulting in high costs—while general-purpose LLMs exhibit strong dependence on the quality and quantity of training data. Compounding these issues, inherent biases and errors in LLMs, alongside concerns about legal compliance and data privacy related to their training corpora, further hinder their reliable application in this field. To address these limitations, several key technical advancements have emerged. Retrieval-Augmented Generation (RAG)—a technique first proposed by Facebook AI Research (FAIR) in 2020—has gained widespread adoption in the LLM era19. This approach integrates vector databases as external knowledge repositories, enabling dynamic retrieval of relevant information during text generation, which effectively mitigates errors stemming from knowledge gaps. In addition to RAG, two foundational capabilities have further enhanced LLM utility: in-context learning and prompt engineering. Tom B. Brown et al. first identified the in-context learning ability of GPT-series models, demonstrating that these models can guide generation outputs via contextual cues without additional training20. Building on this, Pengfei Liu et al. introduced prompt engineering: a methodology that steers LLM behavior toward specific objectives by designing tailored prompts—without updating model weights—thereby alleviating domain-specific biases and errors21.

However, current similarity-based retrieval mechanisms exhibit limitations, particularly in specialized industry translation tasks such as pharmaceutical regulatory documentation between Chinese and English, which is highly confidential, time-critical, and expensive, and should strictly stick to the terminology22,23,24. For example, a NDA submitted to US FDA generally consists of over 200 documents with as many as 60,000-100,000 pages. Its high-quality translation with quick response streamlines subsequent review and approval25. The translation issues regarding these dossiers in China can be broadly categorized as follows: (1) Slow Speed: The completion of more than 200 registration documents typically takes 1–2 months at least. (2) Poor quality: Multidisciplinary information including chemistry or biology, medicine, pharmacy, and production, is written and compiled by multiple departments in the company. All of the original data, charts, figures, protocols, procedures, and records should be translated completely into clear and consistent language. However, presently, the translations fail to meet the requirements of regulatory authorities. (3) High cost: The translation cost up to hundreds of thousands of US dollars, which adds heavy burden to the high-risk journey of a new drug. Lastly, the dearth of professional translators has long been overlooked by the industry26. These tasks demand not only semantic consistency but also strict adherence to precise terminology translation. Vector databases primarily rely on semantic similarity during retrieval, often yielding semantically matched content while failing to guarantee accurate identification and translation of domain-specific terms. Consequently, the key to the wider implementation of LLMs is to develop an efficient and cost-effective application for specific vertical domains27,28,29.

Following the wave of LLM, lightweight LLMs surge in domain-specific researches to meet the fast, easy distributed and secure demands of stakeholders30,31,32. It is expected to be specially trained to target pharmaceutical regulatory affairs. In particular, this study created the PhT-LM to improve the quality, efficiency and reduce the cost in bilingual translation of pharmaceutical regulatory affairs.

Materials and methods

The original materials employed to establish a corpus were obtained from the official website of NMPA, the official regulatory agency in China responsible for the review and approval of drugs, as well as bilingual pharmaceutical textbooks used in universities and colleges and other official published guidelines both in Chinese and English. Following the cleansing and reviewing processes, an English-Chinese bilingual pharmaceutical text translation dataset was generated and stored in a knowledge base comprising two components: a document database and a vector database. Subsequently, a general-purpose LLM, Qwen-1_8B-Chat model, was optimized on the aforementioned translation dataset through the Low-Rank Adaptation (LoRA) technique. Next, the Retrieval-augmented Generate (RAG) technique was employed to input the regulatory affairs document to be translated into the fine-tuned model, along with similar translation examples retrieved from the knowledge base, with the objective of further improving the quality of the model’s translation33. Finally, the whole workflow of creating the lightweight LLM, PhT-LM procedure to created PhT-LM is illustrated in Fig. 1.

Fig. 1
Fig. 1
Full size image

The overall workflow of PhT-LM construction.

Data

Data collection

In order to guarantee the authority and comprehensiveness of the model’s data sources, the data for this study were primarily derived from the following sources: (1) Chinese-English bilingual texts of laws and regulations regarding drug registration published on the NMPA website (https://www.nmpa.gov.cn/) paralleled with those on its English website (https://english.nmpa.gov.cn/); (2) Chinese-English bilingual terms of Pharmacy issued by the China National Committee for Terminology in Science and Technology (CNCTST) in 2014; and (3) relevant teaching materials in the field of pharmacy.

In data collection, automated scripts were written using the Selenium library in Python for content from the NMPA websites to crawl relevant text, including titles, document contents, and attachments. Data such as textbooks related to the international registration of medicines and other sources were used directly in their electronic versions. The final corpus was compiled based on 1,506 documents.

Data pre-processing

Following the collection of documents, a series of rigorous cleaning procedures was carried out to ensure that the resulting dataset thoroughly cover the terminology and key concepts in regulatory affairs. The data cleaning procedures included document screening, establishing the pairs between Chinese and English texts, data deduplication and content validation.

The raw data were pre-processed as follows: Step 1: A manual screening method was used to pair source documents by content to ensure that each Chinese document aligned with its English version. Step 2: The Docx library of Python was employed to read the Chinese and English document pairs that have been exchanged. And each paragraph of the documents was mapped to construct the Chinese and English translation dataset, which was stored as an Excel file. Step 3: The dataset was subjected to de-duplication, with the objective of eliminating redundant data pairs and preventing the occurrence of identical textual content within documents originating from disparate sources. Step 4: A manual review of the format and content of the data was conducted, including, but not limited to, the examination of blanks, garbled codes, and the translation of Chinese and English text. This process aims to confirm the accuracy and reliability of the translated data. Step 5: In case of any potential order effects during subsequent model fine-tuning, it was imperative to randomize the dataset in case of any potential sources of bias. Finally, a dataset comprising 34,769 Chinese-English pairs pertaining to regulatory affairs with a total of 912,826 tokens was created.

Retrieval module

The overall framework of the retrieval module was depicted in Fig. 2. In the knowledge base construction stage, the preprocessed Chinese and English translation data were imported into the document database according to the fields. Concurrently, the text embedding model was imported to encode the text into vectors, which were then imported into the vector database. Both the document database and the vector database were stored in Elasticsearch (hereafter referred to as ES) in a manner corresponding to two independent indexes in the ES cluster34. During retrieval, the input text was searched by document database and vector databases separately, and the query results of the both databases were obtained. The results of the two routes were then proportionally fused as the final query result.

Fig. 2
Fig. 2
Full size image

Framework of the retrieval module.

ES retrieval consists of three main components: text processing, query statement generation, and matching processing. For the input text, the text is first pre-processed to generate query statements by segmenting, removing stop words, setting word weights, etc., and then the query statements are used for multi-field matching. The text segmentation uses IK participle, which supports two participle modes: fine-grained participle and intelligent participle. To optimize retrieval performance, the fine-grained disambiguation mode is used in data storage to improve the hit rate of retrieval by increasing the diversity of disambiguation. In text retrieval, the intelligent participle mode was used to reduce unnecessary participles and improve retrieval accuracy34.

Vector database retrieval, or semantic retrieval, is a search method based on natural language processing and semantic analysis techniques that analyzes the semantic content of documents to determine their relevance to the search query. The key to the effectiveness of semantic retrieval lies in the encoding capability of the text embedding model. The embedded model text vector review list, MTEB (Massive Text Embedding Benchmark, https://huggingface.co/spaces/mteb/leaderboard), is a benchmark that spans 8 embedding tasks covering a total of 56 datasets and 112 languages. Considering the model size, inference time and other practical factors, the mxbai-embed-large-v1 model, ranked No. 4 in the English list, and the stella-large-zh-v3-1792d model, ranked No.3 in the Chinese list, were chosen (as of March 15, 2024) as the embedded models35.

This study employed proportional fusion as the foundation of its retrieval strategy, viz. combining the multiplexed recall results from ES retrieval and vector retrieval in equal proportion, enabling the retrieval of translation examples for source texts across both keyword matching and semantic alignment dimensions. The retrieval results followed a straightforward rule: for any specified number n of results, the top n/2 highest-ranked results are extracted from the ES and the vector database retrievals equally as the final query results. The method backed up these retrieved examples synergistically combine lexical matching-based precision with semantic understanding-driven relevance, not only capturing direct lexical correspondences at the vocabulary level but also effectively leveraging contextual information to enhance translation naturalness, fluency, and contextual adaptability.

Fine-tuned model

The selection of basic model

Table 1 lists the commonly used general purpose lightweight LLMs, such as RedPajama-INCITE-Chat-3B, Firefly-Bloom-1B4, Qwen-1_8B, and OpenBuddy-3B. Among them, Qwen-1_8B-Chat is developed by Aliyun with 1.8 billion parameters, based on Transformer architecture, and pre-trained with over 2.2 trillion tokens of data. In addition, on the English downstream evaluation task MMLU (Massive Multitask Language Understanding), the Qwen-1_8B-Chat model excels existing open-source models of similar size, such as OpenLLaMA-Chinese-3B, OpenBuddy-3B, etc. On the C-Eval validation set, the model is superior to the chat models like ChatGLM2-6B and LLaMA2-7B. Finally, all its weights are completely open for academic research36. Hence, this study took Qwen-1_8B-Chat as the basic model.

Table 1 Comparison of general-purpose lightweight LLMs.
Full size table

Experiments

  1. (1)

    Constructing training and test datasets

First, the texts to be translated were extracted from the constructed translation data set. These texts were then fed into the retrieval module to identify the translation examples that are most similar to the text to be translated. Then, this study constructed templates of prompts integrating translation examples and texts to be translated. The template styles are shown in Tables 2 and 3, respectively, aiming to train the model to extract feature information from contextual translation examples and improve the translation quality. A total of 34,769 datasets with contextualized translation examples were constructed, and 32,769 of them were classified as the training set and the remaining 2,000 as the test dataset.

Since short texts failed to fully demonstrate the superiority of the translation model, the texts to be translated with a length of less than 10 after word splitting were deleted using code. Finally, an English-to-Chinese test dataset containing 658 data items was constructed in parallel with a Chinese-to-English test dataset of 628 data items.

Table 2 The prompt of English-Chinese translation.
Full size table
Table 3 The prompt of Chinese-English translation.
Full size table
  1. (2)

    Fine-tuning

This study used the LoRA technique to fine-tune the interested LLM. This technique allows the model to be more adapted to a specific task by introducing a small number of trainable parameters while leaving most of the model’s parameters unchanged37. All fine-tuning was performed on Ubuntu 22.04 operating system equipped with an 80GB A100 graphics card to ensure the stability of the experimental environment. The hyperparameters recommended by Qwen-1_8B-Chat were employed in fine-tuning: cutoff_len = 2048, batch_size = 4, lr = 5e-5, num_train_epochs = 2, lora_rank = 8. Their excellent performance has been demonstrated in multiple studies38,39,40,41.  Considering its target, the fine-tuned model was constructed and named as PhT-LM.

Retrieval-augmented generation

Retrieval-Augmented Generation, (RAG) integrates the advantages of information retrieval systems and LLMs, aiming to enhance the quality and relevance of generated text33. This study adopts the RAG paradigm, with the specific process outlined in Fig. 3: First, the retrieval framework proposed in Retrieval module is employed as the retrieval module, utilizing a combination of keyword matching and semantic matching to start searches and obtain potentially useful examples. Next, the retrieved translation examples are integrated according to the translation templates shown in Tables 2 and 3. Finally, the PhT-LM fine-tuned in Fine-tuned model serves as the generation model, producing more accurate and contextually appropriate translation results based on the constructed prompt information. This method effectively integrates external translation databases and terminological knowledge into the generation process of LLMs, notably enhancing their performance in domain-specific translation tasks.

Fig. 3
Fig. 3
Full size image

The algorithm process of RAG.

Evaluation metrics

Bilingual Evaluation Understudy (BLEU) is a commonly used metric for evaluating machine translation42. This study adopts Jieba Segmentation (a popular Chinese text segmentation tool developed by Sun Junyi from Baidu Company) to deal with Chinese text segmentation in translations when calculating BLEU scores. And we choose to utilize the sentence_bleu function in the NLTK library. By passing the weights parameter weights= [(1,), (1/2, 1/2), (1/3, 1/3, 1/3), (1/4, 1/4, 1/4, 1/4)], i.e., the n-gram weights are set equally in calculating the BLEU scores at 1-gram, 2-gram, 3-gram, and 4-gram, individually. In order to deal with the problem that the BLEU score is zero when the n-gram that is not presented in the reference translation appears in the machine translation, the smoothing function (). Method4 in the NLTK library is employed as the smoothing technique in BLEU score calculation.

Character n-gram F-score (CHRF) is another metric used to evaluate the quality of machine translation42. It assesses the quality of machine translation by calculating the number of n-gram matches between the reference translation and the resulting translation. In this study, when calculating the CHRF score, this study utilized the sentence_chrf function in the NLTK library to assess the similarity between the resulting machine translation and the reference translation.

Given that the translation quality of regulatory affairs dossiers not only depends on lexical and grammatical correctness, but also needs to achieve accurate matching at character-level, such as terminology. Therefore, this study integrates the BLEU and CHRF scoring criteria to provide a more comprehensive, objective, and precise evaluation of this model. BLEU focuses on n-gram matching at the lexical level and CHRF focuses on n-gram matching at the character level. The integrated scoring criteria thoroughly measure the consistence of translations at all levels, resulting in more accurate, consistent, and professional translations.

Code availability

The retrieval module source code, and fine-tuning model built in this study are publicly available on GitHub (https://github.com/chent-1928/PhT_LM).

Results

The study constructs a new corpus for regulatory affairs in pharmaceutical industry. It comprises 1,506 documents officially published in China, both in English and Chinese. After web crawling, cleaning, and verifying the bilingual documents from the official websites of competent regulatory authorities in China and international organizations, a translation dataset containing 34,769 bilingual data was constructed. Based on the corpus, we design a tailored and impactful lightweight large language model (LLM), PhT-LM. Next, the open-source Qwen-1_8B-Chat model was chosen as the basic model, which was then fine-tuned in the above translation dataset using the low-rank adapter technique. Finally, the retrieval-augmented generate technique was utilized to further polish the model’s translation performance. When compared to popular general-purpose large language models, our model achieved a BLEU-4 mean score of 36.018 and a CHRF mean score of 58.047 based on a self-constructed training corpus, with improved scores ranging from 16% to 65%. In addition, PhT-LM speeds up regulatory affairs translation, compared to the conventional translation by skilled translators. Its excellent performance in translation also cuts down translation fees.

Translation performance test of PhT-LM

This study compares PhT-LM with the general-purpose LLMs including Qwen-1_8B-Chat, ChatGLM3-6B, Chinese-Alpaca-2-7B and GPT-3.5-turbo from the perspective of translation performance. As Table 4(a) shows, in English-Chinese translation test dataset, PhT-LM achieves the highest scores in all five evaluation indices, viz. BLEU-1, BLEU-2, BLEU-3, BLEU-4, and CHRF, with the score of 69.053, 56.726, 47.744, 40.741, and 51.115, respectively, which apparently surpasses the rest four models. Similarly, in Chinese-to-English translation task, Table 4(b) presents PhT-LM’s superior performance with each score improvements ranging from 16% to 65%.

In addition, PhT-LM is also compared with the current mainstream translation services, such as Google Translate and DeepL. It still ranks the top in all five evaluation metrics for both English-Chinese and Chinese-English translation tasks, further verifying its significant advantages in translation performance.

Table 4 Score of all models in the test dataset.
Full size table

In this study, the retrieval module is also added to the Qwen-1_8B-Chat model, the ChatGLM3-6B model, and the Chinese-Alpaca-2-7B model, and the results are shown in Table 5. Comparing the scores in Tables 4 and 5, addition of the retrieval module has greatly improved all index scores of the three models, ranging from 25% to 37% separately.

Table 5 Score of all models with retrieval module in the test dataset.
Full size table

Impact of the number of translation examples on the translation quality of PhT-LM

In order to reveal the importance of translation examples for the model’s in-context learning, and explore the impact of different numbers of translation examples on the performance of the fine-tuned model, this study adjusts the number of query results returned from the retrieval module by modifying the number of translation examples. Consequently, test datasets with 1, 2, 8, and 16 translation examples were constructed, separately, and the test dataset with 4 translation examples acts best.

Fig. 4 shows the impact of different numbers of contextual translation examples on the BLEU-1, BLEU-2, BLEU-3, BLEU-4 and CHRF metrics of the model. It could be seen that the overall trends of all metrics are similar in both English-to-Chinese and Chinese-to-English tasks, i.e., when the number of translation examples is raised from 1 to 4, the BLEU-1 to BLEU-4 scores and the CHRF scores show an upward trend and later, becoming stable. In contrast, when more translation examples are added, e.g., the number of examples is raised up to 16, and the BLEU scores and CHRF scores show a decreasing trend. This figure depicts that the number 4 is the turning point in both English to Chinese and Chinese to English translation quality. Unlike generic LLMs, the number of contextual translation examples shows a different feature in domain-specific lightweight model.

Fig. 4
Fig. 4
Full size image

The impact of different numbers of contextual translation examples on the model.

Ablation experiments

This study examines the effectiveness of the retrieval strategy based on proportional fusion through ablation experiments. Two test groups are constructed in the study, covering Chinese-to-English and English-to-Chinese tasks, separately. Each test group contains 2 test files, and the contextual translation examples of the text to be translated in each group are individually generated by ES retrieval alone and vector database retrieval only.

Table 6 shows a notable improvement in the translation quality of the model, regardless of ES retrieval or vector database retrieval. In the English-to-Chinese translation test database, the model supported by ES retrieval alone reached 68.349, 56.031, 47.118, 40.086 and 50.521 in BLEU-1 to BLEU-4 and CHRF index scores, respectively, and the five evaluation indexes surpassed the scores of the model supported by vector database retrieval alone, which were 67. 661, 54.969, 46.051, 31.199, and 49.684. These were all lower than the scores of 69.053, 56.726, 47.744, 40.471, and 51.115 presented by the model using the proportional fusion retrieval strategy.

Similarly, the five metrics of the model to evaluate its performance using ES retrieval alone in the Chinese-English task (56.246, 44.749, 36.711, 30.881, and 64.436) were higher than those of the model supported by vector database retrieval (56.024, 43.883, 35.744, 29.290, and 63.310), and were lower than those of the model with proportional fusion retrieval strategy (57.212, 45.475, 37.246, 31.295, and 64.978). Therefore, the model translation quality has been further improved by adopting the retrieval strategy of proportional fusion, i.e., the fusion of the ES and the vector database retrieval, complementing each other’s feature information, is a promising strategy in this domain-specific lightweight model.

Table 6 Score of PhT-LM with different retrieval strategies in test dataset.
Full size table

Translation efficiency and cost

Table 7 lists the PhT-LM ‘s translation speed in test dataset following two different retrieval strategies, ES only and proportional fusion. Apparently, the way of ES retrieval responds much quicker than that of proportional fusion in every 1000 word. For English to Chinese translation, ES retrieval strategy takes 30.49 seconds and 56.28 seconds with proportional fusion way. In Chinese-to-English translation tasks, the results are even more outstanding. The ES retrieval strategy and proportional fusion strategy takes merely 19.40 seconds and 38.11 seconds respectively, which is nearly only two-thirds of the time required for English-to-Chinese translation tasks.

PhT-LM represents a cost-effective option for business handling translation tasks. The model could be deployed on most smartphones or personal computers, imposing no extra hardware requirements on companies to perform translation work. In contrast, regulatory affairs translation currently demands that translators available possess a high level of specialization and professional training. Meanwhile, the total cost of engaging translators or translation companies for such work can reach several hundred thousand dollars.

In general, hundreds of thousands of pages in an NDA submission must be submitted to US FDA. Taking 100,000 pages for instance, the whole translation process can be completed iwithn around 50 h—an amazing speed in comparison with conventional human translation speed.

Table 7 Speed of PhT-LM with different retrieval strategies in test dataset.
Full size table

Use-case validations

To evaluate the generalization of the model in real-world application scenarios, this study selected three regulatory guidance documents issued by the International Council for Harmonization of Technical Requirements for Pharmaceuticals for Human Use (ICH), EMA and FDA as external test cases, namely: the “ ICH S1A: Guideline on the need for carcinogenicity studies of pharmaceuticals” issued by, the “Guideline on the clinical investigation of medicines for the treatment of Alzheimer’s Disease” issued by the EMA, and the Guidance for Industry “Allergic Rhinitis: Developing Drug Products for Treatment ” issued by the U.S. FDA. None of these documents were included in the model’s knowledge base or training corpus, and they come from different regulatory systems covering highly specific topics and linguistic complexity, making them the suitable benchmarks for assessing the model’s generalization.

The final translation quality evaluation results, as shown in Table 8, demonstrate that the PhT-LM model achieved superior overall performance. Both in English-to-Chinese and Chinese-to-English tasks, this model achieved the highest scores on the BLEU-3 and BLEU-4 metrics; meanwhile, its scores on the CHRF metric are also close to those of the top-performing GPT-3.5-turbo.

Notably, BLEU-1 and BLEU-2 primarily measure unigram and bigram matching accuracy. Given that GPT-3.5-turbo is an ultra-large-scale language model trained on massive and diverse corpora covering extensive domains and styles, it exhibits greater lexical flexibility. PhT-LM, however, its smaller-scale knowledge base and training data limite its exposure to diverse linguistic patterns during training, which partially impacts its low-order n-gram matching rates and leads to slightly lower BLEU-1 and BLEU-2 scores compared to GPT-3.5-turbo, respectively.

To more intuitively demonstrate the translation effect, Tables 9 and 10 provide paragraph translation examples. PhT-LM model successfully translate the word consistency to 一致性 instead of 共识 (“consensus” in English). And it correctly translates the phrase worldwide regulatory assessments to 全球监管评估 rather than 全球注册评价监管机构. Both the general word and technical term translation confirmed the model’s capability in bilingual translation which not only accurately completes the translation task but also effectively preserves the meaning of the original text and the accuracy of professional terms.

Table 8 Score of all models in the case.
Full size table
Table 9 Translation examples of PhT-LM model in English-Chinese task.
Full size table
Table 10 Translation examples of PhT-LM model in Chinese-English task.
Full size table

Domain-expert assessment

To comprehensively assess the model’s applicability in real-world pharmaceutical regulatory contexts, this study further conducts domain-expert assessment to carry out an in-depth examination of the model’s translation quality from a professional perspective, focusing on three key dimensions: semantic accuracy, terminology consistency, and linguistic fluency.

The study design is as follows: 100 English-to-Chinese and 100 Chinese-to-English bilingual sentence pairs were randomly selected from the constructed test set, and manual scoring was performed by professionals with 3–5 years’ experience in translating dossiers of regulatory affairs. A 1–3 point scale was adopted for scoring (with higher scores indicating better performance), covering the following three dimensions:

  • Accuracy: Whether the target text accurately conveys the semantic meaning of the source text;

  • Consistency: Whether professional terms and expressions remain unified and conform to the conventional norms of regulatory documents;

  • Fluency: Whether the target text is natural and coherent.

Notably, the models to be evaluated were limited to those that demonstrated excellent performance in terms of the BLEU and CHRF metrics, namely, GPT-3.5-turbo (representing general-purpose large models), DeepL (representing mainstream commercial translation services), and the PhT-LM proposed in this study.

In the English-to-Chinese task, results came out to be consistently with automated metric evaluation (as shown in Table 11), PhT-LM demonstrated notable advantages, achieving scores of 242, 263, and 279 in the three dimensions of accuracy, consistency, and fluency—all significantly outperforming GPT-3.5-turbo and DeepL. This fully reflects its professional strengths in terminology processing and contextual adaptation within the regulatory affairs.

In the Chinese-to-English task, PhT-LM performed on par with GPT-3.5-turbo and DeepL in terms of consistency, while it was slightly inferior to the latter two in accuracy(235) and fluency(278). This may be attributed to the complex implicit semantic structures and highly concise sentence patterns in Chinese regulatory texts, which impose higher demands on the model’s capabilities in semantic decoding and English reconstruction. In contrast, general-purpose large models still maintain advantages in the flexibility and naturalness of language generation.

Nevertheless, PhT-LM retained overall competitiveness, with particularly outstanding performance in semantic fidelity and terminology standardization under professional contexts. This demonstrates its practical value and irreplaceability in translation tasks within the pharmaceutical regulatory field.

Table 11 Scores of the expert evaluation.
Full size table

Discussion

Increasing requirements for the quality of document translation in the field of pharmaceutical regulation have been acknowledged by the industry43,44. Particularly in the preparation of regulatory submission materials, high-quality translation has become one of the key factors ensuring the smooth progress of approval processes.

The linguistic review process of product information in the centralised procedure, published by EMA, clearly states that low-quality translations may lead to delays when application materials are submitted to the European Commission45,46. Similarly, FDA has issued multiple guidances to standardize translation practices. Guidance for Industry-Translation of GLP Study Reports: Questions and Answers, requires that translators should possess relevant educational backgrounds and experience in medical document translation, and that written procedures be established to ensure translation quality47.

Data quality requirements

Research has shown that data quality is critical when modeling LLMs in these vertical industries48. Whether it is a LLM with more than 7 billion data or a lightweight language model with only 1.8 billion data, they all possess the ability of in-context learning. Therefore, it is cautious in retrieving, crawling, cleaning and verifying the source data. Since the training data sources of the versatile LLMs are wide, their primary goal is to accomplish a variety of tasks and scenarios generalization, which is quite different from domain-specific LLM49. However, their applications in vertical domains lack accuracy, authority, and coherence, especially in the highly regulated pharmaceutical sector. PhT-LM is particularly constructed based on authoritative, accurate and reliable data. It excels in translation quality through retrieval enhancement. Empirical experiments conducted on real-world texts confirm the effectiveness of our model.

Selection of retrieval strategies

It is worth noting that although the retrieval module using the proportional fusion strategy has the greatest effect on improving the quality of model translation, this improvement comes with increased cost. In particular, the text embedding model must be used to encode the sentences to be translated when performing vector database retrieval, a process that consumes a certain amount of GPU resources and leads to an increase in inference cost. Therefore, in resource-constrained settings, such as relatively low-end mobile phones or PCs with limited video memory, it is more efficient to use the ES retrieval strategy to support the model for text translation. This route is more applicable for most of the small and medium businesses. Further, ES retrieval strategy offers superior translation support without strengthening the computational burden.

In fact, the model retrieved by ES alone is noninferior to the retrieval module using the proportional fusion strategy in terms of translation quality. When compared to GPT-3.5-turbo, PhT-LM retrieved by ES shows an improved performance by 14% to 31%. Therefore, considering the cost of deploying models in the pharmaceutical industrial setting, the ES retrieval strategy is highly recommended as an effective solution for balancing performance and resource consumption, requiring only 4G of video memory for inference.

Currently, generic LLMs fail to meet the accurate demands in vertical domain translation, especially in the translation of regulatory affairs documents which are claimed with poor professionality, consistency and accuracy50,51. This study suggests that an easily deployed lightweight LLM like PhT-LM, is practical and economic for vertical domain translators or business entities to use. It is convenient for them to establish their own models with appropriate number of contextual examples after validation52.

Performance differences between Chinese-English translation

English is a global lingua franca, and a large part of open-source datasets are presented in English53. As a result, the generic LLMs are exposed to a large amount of English data during the pre-training phase, allowing it to demonstrate superior English comprehension54. Similarly, This PhT-LM model also performs better on the English-to-Chinese translation task than on the Chinese-to-English task. Interestingly, the model’s CHRF indicator scores on the Chinese-to-English task are higher than those on the English-to-Chinese task. This comes from the fact that CHRF mainly focuses on the character-level overlap in the translated text. Due to the inherent difference between Chinese and English language, the model will be more likely to generate character sequences similar to the reference translation in the Chinese-to-English task, thus achieving a higher CHRF score. In follow-up studies, the training data can be further expanded in the pharmaceutical domain to further improve the model’s performance in the Chinese-to-English translation task.

Cost-benefit analysis

As Table 12 shows, in general, the cost of translation for regulatory affairs depends on the complexity of texts, language pairs and task urgency. Human translators are required to have specialized knowledge and language proficiency due to the technical nature of the content. For urgent translation service, the cost is typically 25% to 50% higher. According to a professional translation company that has been operating internationally for 20 years. the price for English to Chinese is US$0.10 per English word, and vice versa. For example, translating an NDA can result in total translation costs of several hundred thousand dollars over several months.

Furthermore, DeepL’s pricing structure imposes a significant financial burden on enterprise users. For instance, its Individual Plan costs $8.74 per month, while its Team and Business Plans are priced at $28.74 and $57.49 per user per month, respectively. As enterprise translation needs grow, these costs increase proportionally.

In contrast, PhT-LM is fully open-source and can be deployed locally with just 4GB of VRAM, drastically reducing both initial investment and long-term operational costs for enterprises. While the model may not match the speed of commercial solutions, it excels in translation accuracy and domain-specific expertise. For large-scale translation tasks, PhT-LM outperforms human translators in response speed and scalability, making it especially suitable for high-frequency, standardized translation scenarios.

Although human translation remains indispensable for context comprehension, cultural adaptability, and professional precision, PhT-LM has demonstrated superior translation quality in the domain of pharmaceutical regulatory documents. Its ability to deliver accurate, contextually relevant translations makes it an attractive alternative, particularly when high-quality translations are required at a lower cost.

With its advantages in cost control, translation efficiency, and information security, PhT-LM stands out as a highly efficient and cost-effective solution for pharmaceutical regulatory document translation, offering a compelling alternative to both human and commercial translation services.

Table 12 Cost comparison of different translation methods.
Full size table

Prospective analysis

To further enhance PhT-LM’s adaptability in real-world scenarios, future research can explore in-depth improvements in two directions: data expansion and model optimization. At the data level, PhT-LM is currently trained and evaluated based on Chinese-English bidirectional translation tasks. It can be extended to multilingual capabilities, building a multilingual translation platform for international drug registration and compliance review on high-quality raw data. At the model level, more advanced lightweight large language models (LLMs) can be adopted to further optimize the model size, enabling efficient operation on edge devices and meeting the deployment needs of pharmaceutical enterprises across different IT environments.

Furthermore, PhT-LM can be integrated with existing pharmaceutical regulatory systems in the future, such as eCTD submission platforms and GMP document management systems. This integration would enable automated and intelligent scheduling of translation tasks. Combined with technologies like information extraction and entity recognition, it could also support the development of auxiliary functions55. In the long term, PhT-LM is expected to evolve into a professional intelligent agent for the pharmaceutical regulatory field, and to support compliance writing. For instance, based on existing templates or structured content, it could provide terminology-consistent and format-standardized language support in line with regulatory guidelines, thereby assisting authors in more efficiently drafting initial versions of application materials56.

In brief, PhT-LM holds significant advantages in current pharmaceutical regulatory translation scenarios. Through continuous optimization and expansion in the aforementioned forward-looking directions, it is expected to deliver broader application value in the intelligent processing of pharmaceutical regulatory documents, bringing greater efficiency and consistency to compliance processes in the pharmaceutical industry.

Potential applications in other domains

The PhT-LM framework proposed in this study integrates the features of lightweight model, high-precision retrieval, and domain knowledge base to broaden its potential applications. Beyond its Chinese-English bidirectional translation of pharmaceutical regulatory documents, it is applicable in other vertical domains that are highly specialized, strongly confidential, and have extremely high accuracy requirements, such as multilingual clinical trial protocols and informed consent forms translation, research papers and patents translation. By replacing it with a domain-specific knowledge base and combining it with task-oriented lightweight fine-tuning, specialized translation systems or intelligent auxiliary tools targeting different application needs can be quickly constructed. It sets a good model by its excellent generalization and transferability.

Data access and ethic consideration

Although the bilingual data used in this study are primarily derived from publicly available regulatory texts issued by the National Medical Products Administration, ICH, FDA, and EMA—none of which pertain to personal privacy or business secrets—ethical considerations must still be addressed in the model’s training and application. When such public data are incorporated into a private knowledge base and embedded into LLMs, the model may inadvertently “memorize” and reproduce sensitive expressions, potentially leading to the dissemination of misleading information or disputes over liability attribution in specific contexts. Moreover, as regulations in the rapidly evolving pharmaceutical industry continue to emerge worldwide, over-reliance on the model’s outputs without timely manual review could result in misinterpretation of regulatory requirements or errors in translation.

Even if the data sources are compliant, the auxiliary role of the model should be clearly defined to prevent it from replacing human professional judgment. Therefore, in practical deployment, it is recommended to establish standard operation practices. Its decision-making weight should be strictly set, with domain experts retaining final review responsibility. Meanwhile, the source of the knowledge base data should be specified during deployment to maintain transparency, which is conducive to establishing a sustainable balance between technological innovation and public trust.

Limitations

Despite its impressive performance in regulatory affairs translation, the lightweight LLM still has certain limitations. Future researches could explore the following directions:

Knowledge base: The data relied on by the current model is primarily sourced from Chinese regulatory authorities or the translated regulatory guidelines of ICH, FDA and EMA by Chinese professionals, which inevitably introduces cultural and linguistic biases. Collecting officially issued translations of pharmaceutical regulations from other languages into Chinese is extremely difficult, as such materials are rarely released by the regulatory authorities of other countries or international organizations. The singularity of data sources may lead to the dominance of linguistic patterns and expression conventions specific to Chinese regulatory contexts in the model; this, in turn, affects the model’s translation performance in non-Chinese contexts, resulting in issues such as performance degradation or insufficient cultural adaptability.

To alleviate such biases, future research should prioritize expanding data sources by systematically collecting, cleaning, and integrating official documents from regulatory authorities of multiple countries and regions (e.g., Japan, Spain, and the European Union) to enrich the knowledge base. By integrating regulatory texts with multilingual and multicultural backgrounds, the model can not only be trained to enhance its multilingual translation capabilities but also promote the balanced representation of linguistic features across different regulatory systems. This will thereby significantly improve the model’s generalization ability and cultural adaptability in international application scenarios.

Base model: PhT-LM is built on the Qwen-1_8B model. With the rapid development of LLMs, more advanced lightweight LLMs, such as Qwen2.5-1.5B and Llama-3-1B, keep emerging one after another. A more advanced base model may further enhance the overall performance.

Training strategy: As an efficient parameter-efficient method, LoRA fine-tuning relies on low-rank approximation, which may impair the model’s performance. Therefore, the coming study could combine secondary pre-training with vertical corpora to enhance the model’s understanding. Then, full fine-tuning could be followed according to specific task requirements to maximize its performance.

Translation speed: Although the translation speed of PhT-LM currently satisfies the existing requirements, the integration of more advanced efficient inference optimization techniques such as vLLM, together with more efficient retrieval algorithms to expedite the information retrieval process within a large-scale knowledge base, has the potential to further curtail the translation time.

Conclusion

The study customizes a novel and practical lightweight LLM, PhT-LM, to tackle the problems of slow, poor and high cost of regulatory affairs translation between Chinese and English in pharmaceutical industry. The model notably improves the translation quality of registration documents, which is superior to the available general-purpose LLMs. In addition, it contains fewer parameters and requires 4G of video memory for inference, which could be easily and quickly deployed on personal computers or cell phones. It provides great convenience for and protects pivotal confidential information of pharmaceutical businesses. Last, it unleashes the potentials of domain-specific LLM to improve the quality and speed up the bilingual translation and hence to help these companies stand out from fierce competition in pharmaceutical industry.