paper-with-me

Papers

Property Enhanced Instruction Tuning for Multi-task Molecule Generation with Large Language Models

2024-12-24 · Xuan Lin, Long Chen, Yile Wang, Xiangxiang Zeng, Philip S. Yu

Large language models (LLMs) are widely applied in various natural language processing tasks such as question answering and machine translation. However, due to the lack of labeled data and the difficulty of manual annotation for biochemical properties, the performance for molecule generation tasks is still limited, especially for tasks involving multi-properties constraints. In this work, we present a two-step framework PEIT (Property Enhanced Instruction Tuning) to improve LLMs for molecular-related tasks. In the first step, we use textual descriptions, SMILES, and biochemical properties as multimodal inputs to pre-train a model called PEIT-GEN, by aligning multi-modal representations to synthesize instruction data. In the second step, we fine-tune existing open-source LLMs with the synthesized data, the resulting PEIT-LLM can handle molecule captioning, text-based molecule generation, molecular property prediction, and our newly proposed multi-constraint molecule generation tasks. Experimental results show that our pre-trained PEIT-GEN outperforms MolT5 and BioT5 in molecule captioning, demonstrating modalities align well between textual descriptions, structures, and biochemical properties. Furthermore, PEIT-LLM shows promising improvements in multi-task molecule generation, proving the scalability of the PEIT framework for various molecular tasks. We release the code, constructed instruction data, and model checkpoints in https://github.com/chenlong164/PEIT.

📄 PDF Abstract BibTeX arXiv:2412.18084

Code (1)

chenlong164/peit 공식 구현 pytorch

Tasks

Machine TranslationMolecular Property PredictionMolecule CaptioningProperty PredictionQuestion Answering

Methods 이 논문이 사용한 방법론

ALIGN In the ALIGN method, visual and language representations are jointly trained from noisy image alt-text data. The image and text encoders are learned via contrastive loss…

Similar Papers 제목 키워드 기반

CrystalFormer-RL: Reinforcement Fine-Tuning for Materials Design

2025-04-03 · Zhendong Cao, Lei Wang

Reinforcement fine-tuning has instrumental enhanced the instruction-following and reasoning abilities of large language models. In this work, we explore the applications of reinforcement fine-tuning to the autoregressive…

Band GapDielectric ConstantInstruction Followingmultimodal generation+2

Large Language Models for Controllable Multi-property Multi-objective Molecule Optimization

2025-05-29 · Vishal Dey, Xiao Hu, Xia Ning

In real-world drug design, molecule optimization requires selectively improving multiple molecular properties up to pharmaceutically relevant levels, while maintaining others that already meet such criteria. However, exi…

Drug Design

What Makes a Good Natural Language Prompt?

2025-06-07 · Do Xuan Long, Duy Dinh, Ngoc-Hai Nguyen, Kenji Kawaguchi 외

As large language models (LLMs) have progressed towards more human-like and human--AI communications have become prevalent, prompting has emerged as a decisive component. However, there is limited conceptual consensus on…

Position-Enhanced Visual Instruction Tuning for Multimodal Large Language Models

2023-08-25 · Chi Chen, Ruoyu Qin, Fuwen Luo, Xiaoyue Mi 외

Recently, Multimodal Large Language Models (MLLMs) that enable Large Language Models (LLMs) to interpret images through visual instruction tuning have achieved significant success. However, existing visual instruction tu…

cross-modal alignmentPosition

WaveCoder: Widespread And Versatile Enhancement For Code Large Language Models By Instruction Tuning

2023-12-20 · Zhaojian Yu, Xin Zhang, Ning Shang, Yangyu Huang 외

Recent work demonstrates that, after instruction tuning, Code Large Language Models (Code LLMs) can obtain impressive capabilities to address a wide range of code-related tasks. However, current instruction tuning method…

Code Generation