TransPolymer: a Transformer-based language model for polymer property predictions
Accurate and efficient prediction of polymer properties is of great significance in polymer design. Conventionally, expensive and time-consuming experiments or simulations are required to evaluate polymer functions. Recently, Transformer models, equipped with self-attention mechanisms, have exhibited superior performance in natural language processing. However, such methods have not been investigated in polymer sciences. Herein, we report TransPolymer, a Transformer-based language model for polymer property prediction. Our proposed polymer tokenizer with chemical awareness enables learning representations from polymer sequences. Rigorous experiments on ten polymer property prediction benchmarks demonstrate the superior performance of TransPolymer. Moreover, we show that TransPolymer benefits from pretraining on large unlabeled dataset via Masked Language Modeling. Experimental results further manifest the important role of self-attention in modeling polymer sequences. We highlight this model as a promising computational tool for promoting rational polymer design and understanding structure-property relationships from a data science view.
Code (1)
Tasks
Language ModelingLanguage ModellingMasked Language ModelingPredictionProperty PredictionRepresentation LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Transferring a molecular foundation model for polymer property predictions
Transformer-based large language models have remarkable potential to accelerate design optimization for applications such as drug development and materials discovery. Self-supervised pretraining of transformer models req…
Data AugmentationTransfer LearningMulti-View Polymer Representations for the Open Polymer Prediction
We address polymer property prediction with a multi-view design that exploits complementary representations. Our system integrates four families: (i) tabular RDKit/Morgan descriptors, (ii) graph neural networks, (iii) 3D…
Representing Polymers as Periodic Graphs with Learned Descriptors for Accurate Polymer Property Predictions
One of the grand challenges of utilizing machine learning for the discovery of innovative new polymers lies in the difficulty of accurately representing the complex structures of polymeric materials. Although a wide arra…
Chemistry Integrated Language Model using Hierarchical Molecular Representation for Polymer Informatics
Machine learning has transformed material discovery for inorganic compounds and small molecules, yet polymers remain largely inaccessible to these methods. While data scarcity is often cited as the primary bottleneck, we…
Multimodal machine learning with large language embedding model for polymer property prediction
Contemporary large language models (LLMs), such as GPT-4 and Llama, have harnessed extensive computational power and diverse text corpora to achieve remarkable proficiency in interpreting and generating domain-specific c…
Property Prediction