Exploring Data-Driven Chemical SMILES Tokenization Approaches to Identify Key Protein-Ligand Binding Moieties
Machine learning models have found numerous successful applications in computational drug discovery. A large body of these models represents molecules as sequences since molecular sequences are easily available, simple, and informative. The sequence-based models often segment molecular sequences into pieces called chemical words (analogous to the words that make up sentences in human languages) and then apply advanced natural language processing techniques for tasks such as $\textit{de novo}$ drug design, property prediction, and binding affinity prediction. However, the chemical characteristics and significance of these building blocks, chemical words, remain unexplored. This study aims to investigate the chemical vocabularies generated by popular subword tokenization algorithms, namely Byte Pair Encoding (BPE), WordPiece, and Unigram, and identify key chemical words associated with protein-ligand binding. To this end, we build a language-inspired pipeline that treats high affinity ligands of protein targets as documents and selects key chemical words making up those ligands based on tf-idf weighting. Further, we conduct case studies on a number of protein families to analyze the impact of key chemical words on binding. Through our analysis, we find that these key chemical words are specific to protein targets and correspond to known pharmacophores and functional groups. Our findings will help shed light on the chemistry captured by the chemical words, and by machine learning models for drug discovery at large.
Code (0)
등록된 구현이 없습니다.
Tasks
Drug DesignDrug DiscoveryProperty PredictionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
VQ-Atom: Semantic Discretization of Local Atomic Environments for Molecular Representation Learning
Large language models succeed by combining large-scale pretraining with meaningful discrete tokens. In molecular machine learning, SMILES is widely used as a token representation, but it is primarily a linearization form…
Representation LearningThe Tokenization Bottleneck: How Vocabulary Extension Improves Chemistry Representation Learning in Pretrained Language Models
The application of large language models (LLMs) to chemistry is frequently hampered by a "tokenization bottleneck", where tokenizers tuned on general-domain text tend to fragment chemical representations such as SMILES i…
Representation LearningFARM: Functional Group-Aware Representations for Small Molecules
We introduce Functional Group-Aware Representations for Small Molecules (FARM), a novel foundation model designed to bridge the gap between SMILES, natural language, and molecular graphs. The key innovation of FARM lies …
Contrastive LearningDrug DiscoveryLanguage ModelingLanguage Modelling+3Impact of SMILES Notational Inconsistencies on Chemical Language Model Performance
Chemical language models (CLMs), inspired by natural language processing (NLP), have recently emerged as powerful tools for various cheminformatics tasks such as molecular property prediction and molecule generation. The…
feature selectionLanguage ModelingLanguage ModellingMolecular Property Prediction+1TSSR: Two-Stage Swap-Reward-Driven Reinforcement Learning for Character-Level SMILES Generation
The design of reliable, valid, and diverse molecules is fundamental to modern drug discovery, as improved molecular generation supports efficient exploration of the chemical space for potential drug candidates and reduce…
Reinforcement LearningDrug Discovery