Introduction

Polymer materials are essential to a wide range of applications, from packaging and construction to advanced biomedical devices, due to their versatility and tunable properties1,2,3,4,5. A comprehensive understanding of the complex relationships between polymer structures and their properties is vital for the rational design of materials tailored to specific applications. While traditional experimental and computational methods have been employed to study these structure–property relationships, they are often time-consuming and resource-intensive because of the immense diversity within the polymer chemical space. Recently, machine learning (ML) models have emerged as powerful tools for uncovering polymer structure–property relationships, demonstrating remarkable success. These ML-based strategies necessitate the development of effective and robust representations of polymer structures to ensure accurate predictions and meaningful insights. Early methods utilized molecular fingerprints6,7,8,9, which encode structural information into fixed-length bit vectors. While these fingerprints effectively capture polymer substructures, their fixed format limits end-to-end learning capabilities. Advances in deep learning techniques have introduced learned representations, including those based on SMILES sequences10,11,12,13,14,15,16, graph-based representations17,18,19,20,21,22,23,24,25, and 3D geometric data26,27,28,29,30,31. Although these approaches have shown promising results, existing studies often have not considered on multimodal methods, limiting their ability to integrate the diverse and complementary information provided by different data types. This limitation typically restricts their effectiveness to specific properties, making it challenging to achieve general-purpose applicability. For example, a recent study by Zhu32 demonstrated that different representation methods exhibit varying strengths depending on the target property: for 2D molecular structures, SMILES-based representations excel in predicting chirality, whereas graph-based representations outperform others in capturing ring-related features. These findings emphasize the modality-specific strengths and limitations of current approaches and highlight the need for a comprehensive framework capable of integrating multiple information sources into a unified and informative representation.

Beyond molecular structures, recent advancements in language models have introduced an emerging representation method: text-based descriptions. Studies in the field of small molecules have demonstrated that leveraging molecular-related captions, i.e., domain-specific knowledge embedded in textual descriptions, can enhance the molecule modeling33,34,35. These captions provide contextual information that is often difficult to encode using conventional molecular descriptors alone. However, applying textual knowledge to polymer science remains challenging due to the absence of consistent, large-scale, and high-quality datasets of polymer captions. The advent of large language models (LLMs) offers a promising solution to this issue. General-purpose LLMs, trained on extensive scientific corpora, implicitly store vast amounts of domain knowledge. Recent works36,37 have successfully extracted domain-specific insights and material data from these models, showcasing their capacity to generate meaningful, context-rich captions for polymers. This advancement moves beyond purely structure-based predictions, enabling the integration of human-like domain reasoning into polymer modeling.

This work introduces Uni-Poly, a novel unified framework for generating multimodal and multi-domain polymer representations. The primary contributions of this work are as follows: (i) leveraging large language models and structured data, construct a dataset Poly-Caption that containing over 10,000 textual descriptions of polymers through knowledge-enhanced prompting; (ii) developing the Uni-Poly framework, which integrates multiple polymer data modalities—including SMILES, 2D molecular graphs, 3D molecular geometries, molecular fingerprints, and textual descriptions—into a unified representation. To the best of our knowledge, this is the first framework to systematically integrate multimodal and multi-domain information for polymers; and (iii) conducting a comprehensive evaluation of the Uni-Poly framework across a diverse set of polymer property prediction tasks. The results demonstrate Uni-Poly’s superiority in predictive performance compared to existing single-modality and multimodal approaches, highlighting the effectiveness of combining diverse modalities and domain-specific textual knowledge, offering a more comprehensive approach to polymer informatics.

Results

Overall performance

We compared the property prediction performance of Uni-Poly with other single- and multi-modality models. As shown in Table 1, overall, the glass transition temperature (Tg) emerged as the best-predicted property, achieving an R² of ~0.9, highlighting the strong correlation between Tg and polymer structure. This was followed by thermal decomposition temperature (Td) and density (De), with R² values ranging from 0.7 to 0.8. In contrast, properties like electrical resistivity (Er) and melting temperature (Tm) exhibited lower R² values, typically between 0.4 and 0.6, reflecting the challenges of predicting these properties based solely on monomer unit structures.

Table 1 A comparison of R² score across different models and properties
Full size table

Among all evaluated models, Uni-Poly consistently outperformed all baselines across the evaluated properties, achieving at least a 1.1% improvement in R² over the best-performing baseline in various tasks. Notably, for Tm, Uni-Poly demonstrated a significant 5.1% increase in R², underscores the advantage of integrating complementary modalities, particularly for challenging properties where structural data alone is insufficient.

Moreover, we observed that no single-modality model achieved optimal performance across all evaluation metrics. For single-modality models, Morgan excelled in predicting Td and Tm, ChemBERTa performed best for De and Tg, while Uni-mol achieved the highest R² for Er. This finding supports our earlier assertion that single-modality models are constrained by their limited representational capacity, making it challenging to excel across diverse tasks. In contrast, multimodal models generally exhibit superior representational capacity, leading to their outperformance of single-modality models in most tasks, showcasing the benefits of leveraging multiple data modalities.

Notably, the Text+Chem T5 model demonstrated the weakest overall performance, likely due to the challenges of capturing precise structural and physicochemical information through textual descriptions alone. Despite this limitation, the model achieved R² values exceeding 0.44 across all tasks and performed notably well in predicting Tg, achieving an R² of 0.745. Remarkably, for Tm, it outperformed SchNet, illustrating the rich informational content of textual data—a resource often overlooked in polymer research.

Although Uni-Poly achieves higher accuracy than existing models in predicting polymer properties, its performance remains insufficient for engineering applications. For instance, considering the glass transition temperature (Tg), the best-predicted property, the mean absolute error is approximately 22 °C, which still exceeds the error tolerance generally accepted in the industry. This limitation arises from two major constraints. First, differences in experimental measurement methods (e.g., variations in heating rates in DSC(Differential Scanning Calorimeter) tests could lead to differences in Tg exceeding 10 °C38) introduce inherent noise in the data, setting a theoretical limit on prediction accuracy that is difficult to surpass. More importantly, current polymer representation in Uni-Poly lacks a multi-scale representation of polymer structures. Polymer properties are intrinsically determined by structural features spanning multiple scales, including monomer structure, molecular weight distribution, chain entanglement, and aggregated structures. However, due to limitations in data availability, the current model extracts features solely from the monomer-level inputs. To overcome the accuracy bottlenecks, it is promising to incorporate multi-scale structural information into predictive models. For example, studies such as BigSMILES39, have extended polymer representation by encoding monomer sequence information and have achieved certain improvements in prediction accuracy. Integrating such representations into Uni-Poly holds potential for further enhancing its performance.

Poly-Caption and the impact of multi-domain input

We first examine the generated caption dataset, Poly-Caption. Figure 1 illustrates the overall length distribution of the captions. The generated text lengths generally follow a Gaussian distribution, with most captions falling within the range of 90–135 words.

Fig. 1: Distribution of caption lengths in the Poly-Caption dataset.
figure 1

The histogram shows the normalized frequency distribution of caption lengths, measured by the number of words, in the Poly-Caption dataset.

Full size image

To further analyze the dataset’s content at a macro level, we created a word cloud to visualize the high-frequency terms appearing in the captions, as shown in Fig. 2. The word cloud highlights key aspects of the captions, including applications (e.g., “application”, “coating,” and “device,”), properties (e.g., “thermal,” “electronic,” and “mechanical”), structures (e.g., “backbone,” “ring,” and “aromatic”), and synthesis processes (e.g., “polycondensation” and “polymerization”).

Fig. 2: Word cloud generated from the polymer captions.
figure 2

The word cloud visualizes the most frequent terms appearing in the captions from the Poly-Caption dataset. Larger words indicate a higher frequency of occurrence.

Full size image

The accuracy of captions is crucial for demonstrating the effectiveness of our approach. We manually evaluated the captions in the Poly-Caption dataset in two aspects. The first aspect focuses on whether the captions accurately retain the information we provided during generation. This includes fundamental polymer details such as polymer names, monomer molecular formulas, monomer molecular weights, and polymer synthesis information. We randomly sampled 100 captions from the Poly-Caption dataset and compared them with the input JSON data. Our analysis confirmed that all captions accurately reproduce these fundamental details. This finding indicates that the LLM demonstrates a precise understanding of the provided information. The verification of the remaining information posed additional challenges, primarily due to the inherent non-uniqueness and subjectivity in describing polymer structures, properties, and applications. Given the inherent challenges in evaluating such captions, the accuracy assessment of certain rare polymers is difficult even for polymer experts. Consequently, we selected captions corresponding to the 30 most common polymers from the dataset for a comprehensive analysis (sample captions are provided in the SI). Following a thorough review and evaluation by polymer experts, it was observed that while certain terms, such as “high-performance,” were general or vague, the captions overall accurately captured the actual performance of the polymers and offered valuable insights into the field of polymer science. This finding validates the effectiveness of our knowledge-enhanced prompting in generating factually accurate captions that integrate domain-specific knowledge.

To assess the specific contribution of the textual modality to prediction performance, we evaluated a variant of Uni-Poly, namely Uni-Polyw/o-text, which integrates multiple structural representations but excludes captions. The results, as shown in Fig. 3, indicate that while Uni-Polyw/o-text achieves performance comparable to Uni-Poly for Tg prediction, it demonstrates a notable decrease in R² for the other properties, with declines ranging from ~1.6 to 3.9%. These findings suggest that the inclusion of textual data provides valuable domain-specific insights, enhancing predictive accuracy.

Fig. 3: Impact of the textual input on model performance.
figure 3

Bar chart comparing the R² score of the Uni-Poly model and its variant without textual input (Uni-Polyw/o-text) across five prediction tasks.

Full size image

For example, captions highlight areas of application, such as advanced devices or coatings, and environmental conditions like high-temperature usage or chemical exposure. These references offer critical information about performance requirements and context, suggesting properties such as resistivity, thermal performance, mechanical strength, and chemical stability. Beyond applications, captions enrich the understanding of chemical structure. While structural representations reveal functional groups, captions incorporate domain-specific knowledge, offering experiential insights into the roles and impacts of these groups on polymer properties. This combination enhances the predictive power of structural data by linking it to practical expertise. Captions also reveal the connection between synthesis methods or processing conditions and resulting material properties. By detailing how fabrication processes influence functional outcomes, they provide a deeper understanding of the relationship between material preparation and performance. Overall, textual information complements structural representations, especially for properties shaped by real-world usage and processing scenarios.

To further quantify the impact of specific words or phrases on predictions and to inform future efforts in more effective prompt engineering, we conducted an in-depth analysis of the attention weights assigned to tokens by the text encoder in Uni-Poly for the best-performing Tg prediction task. These attention weights can be interpreted as indicators of the importance of tokens in the subsequent embedding and final prediction results. Figure 4 presents the top 20 tokens with the highest average attention weights in the Poly-Caption dataset. We observed that high-importance tokens can be broadly categorized into four groups:

Fig. 4: Important tokens identified from captions in the Poly-Caption dataset.
figure 4

The bar chart shows the 20 tokens with the highest average importance scores assigned by the model.

Full size image

Chemical structure-related tokens

This category constitutes the largest proportion of high-importance tokens, including “aromatic”, “benz”, and “phenol”, which primarily reflect the influence of polymer monomer structures or precursor reactants on Tg. Notably, “aromatic” stands out with a significantly higher importance weight than other tokens. This aligns with chemical intuition, as the aromaticity of a polymer affects chain rigidity and intermolecular interactions, which are strongly correlated with Tg.

Application-related tokens

Certain words, such as “aerospace” and “adhesive”, indicate that the intended use of a polymer may implicitly correlate with its Tg. For instance, polymers used in aerospace applications often require high thermal stability, which translates to a higher Tg. Similarly, adhesives may have Tg implications depending on their required flexibility and operating temperature. The presence of these tokens in high-importance rankings suggests that the model captures such contextual information and integrates it into its predictions.

Descriptors of polymer properties

Tokens like “versatility”, “organic”, and “intricate” represent broader or more abstract descriptions of a polymer’s characteristics. These words do not directly specify chemical structures or applications but rather encapsulate generalizable features that may be associated with properties.

Adverbs and emphasizing words

Examples include “paramount” and “robust”. While it is difficult to infer their direct connection to the property from the tokens alone, we examined their usage within the Poly-Caption dataset by retrieving sentences containing these keywords. We found that such words are typically used to describe key property properties, such as “…with paramount heat resistance”, indicating a specific property, which is correlated with the predicted property.

The results offer insights for future prompt engineering. For instance, strategically emphasizing words related to structure, application, or property description in input prompts could optimize model performance and improve interpretability.

Task-modality attention weights analysis

To further explain the contribution of different modalities to specific property prediction tasks, we analyzed the attention weight matrix learned by the Uni-Poly model, as visualized in Fig. 5.

Fig. 5: The contribution for each modality across prediction tasks.
figure 5

The heatmap visualizes the modality-specific attention weights learned by the Uni-Poly model for each prediction task.

Full size image

Among all modalities in the Uni-Poly representation, SMILES and FP consistently emerge as the dominant contributors across various properties. This underscores that two-dimensional structural features are the primary determinants for recognizing backbone and functional group types, which are essential for polymer property prediction. Other modalities, such as Graph, Geom, and Text, serve primarily complementary roles. Notably, Geom demonstrates higher relevance only in Tm prediction, while Text shows the lowest overall contribution, functioning mainly as supplementary information.

Density (De)

SMILES and FP effectively capture key chemical features, such as monomer mass and side-chain size. Graph and Geom provide moderate contributions, reflecting topological and spatial configurations that influence packing efficiency.

Electronic resistance (Er)

SMILES and FP dominate by identifying conductive motifs like conjugated systems and functional groups. Due to the relatively low prediction accuracy based solely on structural information, Text becomes more valuable, offering application-related insights into electronic properties.

Decomposition temperature (Td)

FP exhibits strong contributions by detecting thermally stable substructures, such as aromatic groups and cyclic structures, while SMILES complements this by providing backbone information. Text contributes moderately by capturing branching features, which restrict chain segment mobility and enhance thermal stability.

Glass transition temperature (Tg)

The attention weights for SMILES and FP are significantly higher compared to other modalities, while those for Text and Geom are notably lower. This is likely because Tg is influenced by factors such as chain segment mobility, side-group bulkiness, and polar group interactions—all closely related to monomer structure and functional groups. Consequently, two-dimensional descriptors like SMILES and FP, which excel at identifying structural motifs, can largely determine Tg, leaving minimal room for supplementary contributions from Text and Geom.

Melting temperature (Tm)

For Tm, FP remains the most significant modality, but Geom contributes above the average. This reflects its ability to emphasize spatial configurations and molecular symmetry, which govern crystallinity and play a critical role in determining Tm.

Ablation study

In this section, we present ablation experiments to evaluate the effectiveness of various components within the Uni-Poly framework. Specifically, we assess how performance changes relative to the full Uni-Poly model when individual components are removed or altered. The configurations tested are as follows:

“w/o pre-train”

This variant skips the pretraining phase that aligns features across modalities. It evaluates the importance of contrastive learning for feature alignment.

“w/o multi-head attention”

The multi-head attention mechanism for feature refinement is removed, testing the impact of inter-modal feature fusion.

“Mean pooling”

The attention-based weighted pooling layer is replaced with simple average pooling, assessing the impact of attention-based weighting on modality aggregation.

“Frozen encoder”

Encoder parameters are frozen during training, restricting the model to optimize only the downstream components.

The results in Table 2 show that freezing the encoder parameters leads to the most significant performance degradation, particularly for properties Tm and Er. This underscores the necessity of end-to-end training in fine-tuning the pre-trained encoders to adapt them effectively to downstream tasks. Skipping the pretraining phase also causes a substantial decline, emphasizing the importance of contrastive learning in aligning features across modalities and adapting encoders—originally trained on small-molecule SMILES datasets—to the polymer domain. Similarly, replacing the attention-based weighted pooling layer with mean pooling results in notable performance reduction, confirming that dynamic weighting enhances feature aggregation by prioritizing informative components. Although removing the multi-head attention mechanism results in relatively smaller accuracy reductions, its absence consistently lowers performance across all properties. This indicates that multi-head attention provides beneficial enhancements to inter-modal feature fusion and refinement, even if it is not a critical component. These findings highlight the importance of each component in the Uni-Poly framework, where the interplay of these elements collectively ensures robust and accurate predictions of polymer properties.

Table 2 R² deviations of various ablation configurations compared to the full Uni-Poly framework
Full size table

Applicability to copolymers

Regarding its current applicability, this study focuses exclusively on homopolymers, while the extension to copolymer systems, which are of greater relevance in many engineering applications, remains an open area for further exploration. Common approaches to handling copolymers can be broadly categorized into two types. The first treats the copolymer chain as an oligomer or a periodic structure, which is then considered as a monomer for feature extraction and analysis. This method has demonstrated strong applicability, particularly in graph-based representations22,40. This method can be integrated into the Uni-Poly framework by treating the resulting unit as an equivalent “monomer.” However, a significant limitation of this method is that it results in excessively long chain structure, which substantially increases computational costs in the SMILES encoding and 3D coordinate branches of Uni-Poly. As a result, this approach is currently computationally impractical within our framework. The second approach involves computing monomer-level descriptors independently, followed by aggregation based on their composition ratios. In the context of Uni-Poly, this would involve generating embeddings for each monomer across all modalities and then linearly combining them to represent the copolymer. While this strategy has been shown to work well for handcrafted features such as molecular fingerprints41, this approach is less suitable for learned representations, particularly the textual captions, which encode contextualized chemical semantics. A simple linear averaging of such representations lacks clear physical or chemical meaning. Instead, a more promising direction would be to generate copolymer-specific captions that explicitly reflect monomer composition and sequence arrangement. Future work will therefore focus on developing tailored prompt templates and architectural adjustments to more effectively encode copolymer-specific features, such as monomer composition and sequence.

Discussion

In this work, we introduced Uni-Poly, a comprehensive multimodal framework designed for polymer property prediction by integrating structural representations (SMILES, 2D molecular graphs, 3D geometries, and fingerprints) with textual descriptions embedding domain-specific knowledge. Uni-Poly represents a paradigm shift in polymer informatics by systematically leveraging diverse data modalities to achieve a unified representation of polymers.

To validate Uni-Poly’s effectiveness, we first constructed a polymer caption dataset, Poly-Caption, which incorporates domain knowledge through large language models. Experimental results demonstrate that Uni-Poly consistently outperforms single-modal and existing multimodal approaches across a range of polymer property prediction tasks, including glass transition temperature, density, thermal decomposition temperature, melting point, and electrical resistivity. Notably, the inclusion of textual descriptions provides domain-specific insights that enhance performance, particularly for properties influenced by practical applications or processing conditions. A detailed analysis of modality-specific contributions reveals the complementary strengths of different data types, with SMILES and molecular fingerprints consistently showing the strongest impact, while text and geometry add critical contextual and structural layers for specific properties.

These findings underscore the transformative potential of multimodal and multi-domain approaches in polymer science. By offering a more comprehensive representation of polymers, Uni-Poly enables enhanced predictive accuracy, supporting high-throughput polymer screening and the targeted discovery of novel materials.

Future research will focus on the integration of more comprehensive and information-rich polymer structural features to further enhance predictive accuracy and expand the applicability of our framework to a broader range of properties and more complex polymer systems, such as copolymers. By incorporating these advancements, we aim to improve the model’s capability in predicting diverse polymer behaviors, ultimately enabling more reliable and versatile applications in polymer informatics.

Methods

Polymer representation

Polymer representation aims to convert the chemical information of polymer into forms that can be processed by machine learning algorithms. A variety of methods have been developed to represent polymers. In Fig. 6, we illustrate several representation methods using polystyrene as an example. These polymer representations are categorized into two types: structural representations and textual representations, which are briefly introduced below.

Fig. 6: Different representations for polystyrene in the structural domain and in the textual domain.
figure 6

Illustration of various input representations used in the Uni-Poly framework, including a the structural domain, SMILES notation, molecular geometry, fingerprints, and molecular graphs and b the textual domain, a natural language description provides domain-related context.

Full size image

Due to the complexity of polymer chain segments, structural representations of polymers are often simplified by focusing only on the monomer unit. Commonly, four types of structural representations are used for monomers: molecular fingerprint, sequential representation, 2D graph, and 3D geometry.

Molecular fingerprint

Molecular fingerprints6 encode chemical information into binary vectors that indicate the presence or absence of predefined chemical features and substructures. Their primary advantage lies in their simplicity and computational efficiency, enabling rapid similarity searches and clustering. However, fingerprint-based representations heavily depend on predefined substructures, which can limit their ability to capture the full complexity of diverse polymer structures.

Sequential

SMILES (simplified molecular input line entry system)42 is the most popular linear notation used to describe chemical structures. For polymers, SMILES notation often replaces the connection points between repeating units with an asterisk (*) to indicate where monomers are linked. SMILES representations are compact, easily interpretable, and well-suited for sequence-based machine learning models, such as BERT15 and Transformers16. However, due to the limitations of its encoding rules, the linear format may separate adjacent atoms into distant parts of the string, making it challenging to capture some global structural information.

2D graph

In 2D topology graphs, atoms are represented as nodes and bonds as edges, providing a detailed depiction of molecular connectivity and bonding relationships. Graph neural networks, including graph convolutional networks (GCNs)17, GraphSAGE18, graph isomorphism networks (GINs)19, and graph attention networks (GATs)20, are commonly employed to encode these graphs into feature vectors that capture both local and global structural information of polymer monomers. Compared to SMILES, 2D graphs explicitly represent bonds, making them more effective at capturing connectivity information.

3D geometry

3D geometry graphs incorporate atomic coordinates, offering a representation of the spatial arrangement of atoms. Geometric deep learning techniques, such as spatial graph networks, are often used to process these 3D geometries. The primary advantage of 3D geometry graphs lies in their ability to capture stereochemistry and conformational details critical for predicting properties related to molecular conformation, such as energy and molecular interactions. However, obtaining accurate 3D geometries can be computationally intensive, and these representations are often sensitive to variations in molecular conformation.

Textual representation

Text-based representations use natural language descriptions to convey domain-specific knowledge about polymers, as illustrated in Fig. 6b. There is no strict definition of what information a caption must contain; captions may include details about chemical composition and structure, morphology, application scenarios, preparation methods, and more. Although captions cannot describe the detailed chemical structure of a monomer as precisely as structural representations, their scope is significantly broader and often closely related to the polymer’s physicochemical properties. These textual descriptions are processed using language models such as BERT33,35 and Transformer34 models, which generate semantic embeddings that encapsulate the context and knowledge expressed in the text. Another critical consideration is data availability. While the small-molecule domain benefits from publicly available caption databases like ChEBI-2043, no large-scale and publicly accessible datasets are currently available for polymers, presenting a significant challenge for utilizing text-based representations.

Multimodal representation

While single-modality methods have shown promise in capturing specific aspects of polymer features, they are limited in their ability to comprehensively represent polymers. Multimodal representations aim to utilize the strengths of different data modalities to create an integrated representation of polymers. Multimodal methods have been explored in various chemical domains, including reaction prediction, molecular property prediction and drug–drug interaction33,44,45,46,47,48. In the context of polymers, recent studies49,50 have proposed to combine both SMILES and 3D geometry representations. However, restricting integration to only a subset of modalities prevents these methods from fully exploiting the diverse structural information available for polymers. Moreover, no current attempts have been made to incorporate text-based representations into polymer modeling, further limiting the expressive power of these approaches.

Datasets

We conducted a comprehensive evaluation of Uni-Poly using datasets collected from the largest publicly available polymer database, PolyInfo51 (The copyrights of this database are owned by the National Institute for Materials Science [NIMS]). Specifically, we focused on five key polymer properties: glass transition temperature (Tg), density (De), thermal decomposition temperature (Td), melting point (Tm), and electrical resistivity (Er). These properties span a wide range of physical and chemical behaviors, involving different mechanisms such as thermal stability, phase transition, and electronic properties. As such, they provide versatility and robustness of the Uni-Poly model in capturing diverse property relationships. For a given monomer, different samples can exhibit significant variations in their properties due to variations in experimental conditions, synthesis routes, or measurement methods. Such variability introduces inherent noise into the data, which poses an additional challenge for accurate prediction. To reduce noise and provide a more consistent target for model training, we grouped samples for the same monomer and computed the average value for each property to serve as a representative value. After processing, our dataset comprised a total of 15,443 structure–property data points, providing a diverse and comprehensive foundation for evaluating Uni-Poly. The distribution and number of samples for each property are illustrated in Fig. 7.

Fig. 7: Dataset overview.
figure 7

The bar plot (top left) shows dataset sizes for various polymer tasks. Other subplots show the distributions of the properties, including density (De), electronic resistivity (Er), thermal decomposition temperature (Td), glass transition temperature (Tg), and melting temperature (Tm).

Full size image

We propose Uni-Poly, a comprehensive framework designed to unify multiple data modalities relevant to polymer materials into a single embedding. In this section, we introduce the core components of Uni-Poly, as shown in Fig. 8.

Fig. 8: Overview of the Uni-Poly framework.
figure 8

The process begins with knowledge-enhanced prompting using a large language model to generate polymer captions. Multimodal data, including text, geometry, graph representations, fingerprints, and SMILES, are encoded through modality-specific encoders to generate unified modality embeddings. These embeddings are then projected into a shared space and pre-trained using contrastive learning. Subsequently, the framework employs a multi-head attention mechanism and a feed-forward network to facilitate refinement. Finally, an attention-based weighted pooling layer aggregates the refined embeddings into the final Uni-Poly embedding, which can be used for polymer property prediction tasks.

Full size image

Structural and textual representation generation

The structural and textual representation generation module generates various representations, including SMILES, 2D molecular graphs, 3D geometries, molecular fingerprints, and textual descriptions. For structural representations, we employ established methodologies to extract data directly from available datasets or utilize specialized tools such as RDKit52.

Generating textual representations for polymers is challenging due to the absence of large-scale polymer-specific caption datasets. To address this gap, we adopted a synthetic data approach and constructed Poly-Caption, a comprehensive dataset that integrates extensive domain-specific knowledge in polymer science.

General-purpose LLMs, such as GPTs53, possess vast domain-specific knowledge from pretraining on extensive text corpora. However, these models face issues such as hallucination54,55, where factually inaccurate information is generated, and output variability, which can undermine their reliability for scientific applications. To mitigate these limitations, we implemented a knowledge-enhanced prompting strategy. This approach employs in-context learning to guide the generation process through carefully designed prompts. Our prompting strategy is structured into three key components:

  • Role definition and task specification: We first define the model’s role and explicitly outline the task of caption generation. Additionally, we impose constraints on the content and style of the captions to ensure relevance and consistency.

  • Structured polymer knowledge: We provide the model with a JSON dataset containing essential polymer information, including SMILES, polymer type, structural name, molecular formula, molecular weight, processing conditions, and synthesis procedures. The selection of this information is critical to balance its reference utility and avoid data leakage. This structured input enhances the factual accuracy of the generated captions.

  • Polymer caption examples: To facilitate few-shot learning, we manually created three polymer caption examples that clarify the expected scope and format of the captions, improving output consistency. Considering our ultimate goal of predicting polymer properties, we incorporated various descriptive information directly related to polymer properties.

By adopting this approach, we generated a caption for each polymer that combines the inherent domain knowledge of the LLM with the additional structured information provided. The resulting captions include details about the polymer’s name, type, structural characteristics, synthesis methods, general properties, and potential applications, which provides a domain-view for polymer representation. The complete prompt template used in this process is provided in the SI, offering readers a detailed understanding of the specific steps and methodology involved.

Embedding and projection

In this module, each modality is independently encoded to generate an initial embedding that captures its unique information, which is then projected into a joint representation space for further processing. A high-level overview of each encoder is provided below, while detailed architectures are available in the referenced literatures.

Given a polymer monomer represented by its SMILES \(\left(S\right)\), graph \(\left(G\right)\), geometry \(\left({\mathcal{G}}\right)\), fingerprints \(\left(F\right)\) and caption \(\left(C\right)\), we define the polymer representation as follows:

$${\mathcal{P}}=\left(S,G{,}{\mathcal{G}}{,}F,C\right),$$
(1)

For modality \(r\in \left(S,G,{\mathcal{G}}{,}C\right)\), we use a specific encoder \({{\mathscr{E}}}_{r}\) to form the corresponding embedding \({h}_{r}\).

SMILES

We employ ChemBERTa14, a transformer model pre-trained on a large SMILES dataset using a masked language modeling strategy, as the SMILES encoder. Following established practices, the embedding of the first token from the encoder’s last hidden state is used as the SMILES embedding, denoted as: \({h}_{S}\),

$${h}_{S}={{\mathscr{E}}}_{S}\left(S\right).$$
(2)

Graph

The graph isomorphism network (GIN)19 is utilized for encoding graph-based representations. GIN takes atom attributes (\({X}_{G}\)), bond attributes (\(E\)), and the adjacency matrix (\(A\)) as inputs. Through multiple interaction layers, it iteratively updates node features by aggregating neighbor information. A readout function aggregates the updated node features into the graph embedding \({h}_{G}\),

$${h}_{G}={{\mathscr{E}}}_{G}\left(G\right),\,G=\left({X}_{G},E,A\right).$$
(3)

Geometry

SchNet26, designed for quantum chemistry tasks, is employed to generate geometry-based embeddings. SchNet processes atom attributes (\({X}_{{\mathcal{G}}}\)) and atom positions (\(R\)) using continuous filter convolutions and interaction blocks to generate the geometry embedding \({h}_{{\mathcal{G}}}\),

$${h}_{{\mathcal{G}}}={{\mathscr{E}}}_{{\mathcal{G}}}\left({\mathcal{G}}\right),{\mathcal{G}}=\left({X}_{{\mathcal{G}}},R\right).$$
(4)

Caption

For caption-based representations, we use Text+ Chem T556, a language model based on the T557 architecture that has been specifically trained for chemistry-related tasks, including caption generation and generating chemical representations from text. Similar to the SMILES encoder, the embedding of the first token from the last hidden state is used as the caption embedding, denoted as \({h}_{C}\), which captures the semantic meaning of the caption,

$${h}_{C}={{\mathscr{E}}}_{C}\left(C\right).$$
(5)

Fingerprints are inherently vectorized representations of molecular structures. We directly use the Morgan fingerprint as its embedding representation,

$${h}_{F}=F.$$
(6)

These embeddings reside in distinct feature spaces due to the differences in their original representations. To obtain alignment and consistency across modalities while preserving their intrinsic information content, we introduce a projection layer \(\rho\) for each modality. The projection layer is modality-specific, transforming the individual embeddings into a shared space representation. The respective embeddings after projection can be represented as follows:

$${H}_{r}={\rho }_{r}\left({h}_{r}\right),\forall r\in {\mathcal{P}},$$
(7)

where \({H}_{r}\in {{\mathbb{R}}}^{d}\) represents the embedding vector of the modality \(r\) with dimension \(d\). The stacked embedding matrix for all modalities is denoted as \(H\in {{\mathbb{R}}}^{\left|{\mathcal{P}}\right|\times d}\), which is used for subsequent processing.

Fusion module

To effectively utilize information from all available modalities, we employ a multi-head attention mechanism58 and attention-based pooling to enhance the integration and fusion of inter-modal information. The embedding matrix \(H\) is first updated through multi-head attention:

$${H}_{\text{refined}}=\text{Multihead}\left(H\right),$$
(8)

where \(\text{Multihead}(\cdot )\) denotes the multi-head attention mechanism that captures complex inter-modal relationships by computing attention scores across modalities and generating refined feature representations. The output is refined through residual connections and layer normalization to stabilize training, followed by an attention-based weighted pooling mechanism that determines the contribution of each modality. The importance of a modality \(r\) is computed through an attention network:

$${\alpha }_{r}=\frac{\exp \left({\omega }_{r}\cdot {H}_{\text{refined},r}+{b}_{r}\right)}{\sum _{{r}^{{\prime} }\in {\mathcal{P}}}\exp \left({\omega }_{{r}^{{\prime} }}\cdot {H}_{\text{refined},{r}^{{\prime} }}+{b}_{{r}^{{\prime} }}\right)}$$
(9)

where \({\omega }_{r}\) and \({b}_{r}\) are learnable parameters. The final Uni-Poly embedding \({H}_{{\rm{Uni}}-{\rm{Poly}}}\in {{\mathbb{R}}}^{d}\) is computed as the weighted sum of each individual modality embeddings:

$${H}_{{\rm{Uni}}-{\rm{Poly}}}={\sum }_{r\in {\mathcal{P}}}{\alpha }_{r}\cdot {H}_{\text{refined},r}.$$
(10)

Output module

The output module generates the model’s final outputs. In our work, we followed a common practice with a two-stage paradigm of pretraining and finetuning.

In this work, we adopt cross-modal contrastive learning59 as self-supervised pretraining. This method aligns data across different modalities by ensuring that embeddings of the same data point are closely positioned in the vector space, while embeddings of distinct data points are pushed farther apart without the need for explicit annotations. Specifically, we utilize the InfoNCE60 loss to maximize the cosine similarity between positive pairs (representations of different modalities of the same polymer) while minimizing the similarity between negative pairs (representations of different polymers). This enables efficient learning of shared representations across modalities, providing a robust foundation for downstream tasks.

After pretraining, we utilize a simple Multi-Layer Perceptron (MLP) as the prediction head for estimating polymer properties. The output from the fusion module, denoted as \({H}_{{\rm{Uni}}-{\rm{Poly}}}\), is fed into the MLP to make final predictions:

$$y\,=\text{MLP}\left({H}_{{\rm{Uni}}-{\rm{Poly}}}\right),$$
(10)

where \(y\) is the target property.

Baselines

For a comprehensive comparison, we evaluated Uni-Poly against both single-modal models and multimodal baselines. Specifically, the single-modal models included Morgan Fingerprints6, SMILES-BERT11, ChemBERTa14, GCN17, GIN19, SchNet26, Uni-mol31, 3D Infomax45, GraphMVP46, DVMP47, and MMPolymer49. Additionally, we develop a text-based model that leverages embeddings generated from Text+Chem T561 for the textual descriptions baseline. For a fair comparison, all baseline models are trained with the default settings.

Implementation details

For the SMILES representations in the dataset, we first convert all SMILES into canonical SMILES, followed by tokenization using ChemBERTa’s tokenizer to ensure consistent and meaningful input sequences. For graphs, we utilized atomic properties, including atom type and chirality tag, as node features, while bond type and bond direction were used as edge features to capture the bonding context. Regarding 3D molecular structures, we replaced connection points represented by asterisks (*) with hydrogen atoms to ensure stability, followed by geometry optimization using the MMFF9462 force field in RDKit to achieve minimum energy conformations and obtain the final atomic coordinates. For molecular fingerprints, we generated 1024-bit binary vectors with a radius of 2 using RDKit’s fingerprinting functionality. For the text captions, considering the capabilities of existing large language models, we utilized the state-of-the-art GPT-453 as the backbone to generate domain-specific captions, leveraging its advanced proficiency in understanding and producing scientific language. A caption was generated for each unique SMILES entry in the database, resulting in a total of 10,142 SMILES-caption pairs. Subsequently, the structural and textual representations were encoded into modality-specific embeddings and projected into the Uni-Poly space with \(d=256\) for further processing.

During the contrastive pretraining phase, the model was optimized for ten epochs on the entire polymer dataset, using a batch size of 32 and the Adam63 optimizer with an initial learning rate of 1e-5. The scale parameter \(\tau\) for contrastive learning was set to 0.07. In the fine-tuning stage, 10% of the dataset was used for testing, while the remaining 90% was allocated for training and validation. To enhance numerical stability, Z-score normalization was applied to each property prediction task. The model was trained for 100 epochs with early stopping, using a batch size of 32 and the Adam optimizer with an initial learning rate of 1e-4. Additionally, a cosine annealing learning rate scheduler was used to adjust the learning rate throughout the training process. For further details regarding the software and hardware platforms, model training procedures, and hyperparameter specifications, please refer to SI.

Evaluation metrics

In our property prediction tasks, all of which are regression tasks, we primarily evaluate model performance using the R-squared (R²) metric. Additionally, root mean square error (RMSE) and mean absolute error (MAE) are also reported in the SI. To ensure the robustness and reliability of our results, we performed fivefold cross-validation and presented both the mean and standard deviation of the metrics.