paper-with-me

Papers

Pairing interacting protein sequences using masked language modeling

2023-08-14 · Umberto Lupo, Damiano Sgarbossa, Anne-Florence Bitbol

Predicting which proteins interact together from amino-acid sequences is an important task. We develop a method to pair interacting protein sequences which leverages the power of protein language models trained on multiple sequence alignments, such as MSA Transformer and the EvoFormer module of AlphaFold. We formulate the problem of pairing interacting partners among the paralogs of two protein families in a differentiable way. We introduce a method called DiffPALM that solves it by exploiting the ability of MSA Transformer to fill in masked amino acids in multiple sequence alignments using the surrounding context. MSA Transformer encodes coevolution between functionally or structurally coupled amino acids. We show that it captures inter-chain coevolution, while it was trained on single-chain data, which means that it can be used out-of-distribution. Relying on MSA Transformer without fine-tuning, DiffPALM outperforms existing coevolution-based pairing methods on difficult benchmarks of shallow multiple sequence alignments extracted from ubiquitous prokaryotic protein datasets. It also outperforms an alternative method based on a state-of-the-art protein language model trained on single sequences. Paired alignments of interacting protein sequences are a crucial ingredient of supervised deep learning methods to predict the three-dimensional structure of protein complexes. DiffPALM substantially improves the structure prediction of some eukaryotic protein complexes by AlphaFold-Multimer, without significantly deteriorating any of those we tested. It also achieves competitive performance with using orthology-based pairing.

📄 PDF Abstract BibTeX arXiv:2308.07136

Code (1)

bitbol-lab/diffpalm 공식 구현 pytorch

Tasks

Language ModelingLanguage ModellingMasked Language ModelingProtein Language Model

Methods 이 논문이 사용한 방법론

Multi-Head Attention 설명 없음
Attention 설명 없음
Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.
Absolute Position Encodings Absolute Position Encodings are a type of position embeddings for [Transformer-based models] where positional encodings are…
Layer Normalization Unlike batch normalization, Layer Normalization directly estimates the normalization statistics from the summed inputs…
Adam 설명 없음
Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…
Label Smoothing Label Smoothing is a regularization technique that introduces noise for the labels. This accounts for the fact that datasets may have mistakes in them, so maximizing the…

Similar Papers 제목 키워드 기반

DiffPaSS -- High-performance differentiable pairing of protein sequences using soft scores

2024-09-24 · Umberto Lupo, Damiano Sgarbossa, Martina Milighetti, Anne-Florence Bitbol

Identifying interacting partners from two sets of protein sequences has important applications in computational biology. Interacting partners share similarities across species due to their common evolutionary history, an…

Combining phylogeny and coevolution improves the inference of interaction partners among paralogous proteins

2022-08-24 · Carlos A. Gandarilla-Perez, Sergio Pinilla, Anne-Florence Bitbol, Martin Weigt

Predicting protein-protein interactions from sequences is an important goal of computational biology. Various sources of information can be used to this end. Starting from the sequences of two interacting protein familie…

Statistical physics of interacting proteins: impact of dataset size and quality assessed in synthetic sequences

2020-03-16

Identifying protein-protein interactions is crucial for a systems-level understanding of the cell. Recently, algorithms based on inverse statistical physics, e.g. Direct Coupling Analysis (DCA), have allowed to use evolu…

Generative Antibody Design for Complementary Chain Pairing Sequences through Encoder-Decoder Language Model

2023-01-06 · Simon K. S. Chu, Kathy Y. Wei

Current protein language models (pLMs) predominantly focus on single-chain protein sequences and often have not accounted for constraints on generative design imposed by protein-protein interactions. To address this gap,…

DecoderLanguage ModelingLanguage Modelling

MeMDLM: De Novo Membrane Protein Design with Masked Discrete Diffusion Protein Language Models

2024-10-22 · Shrey Goel, Vishrut Thoutam, Edgar Mariano Marroquin, Aaron Gokaslan 외

Masked Diffusion Language Models (MDLMs) have recently emerged as a strong class of generative models, paralleling state-of-the-art (SOTA) autoregressive (AR) performance across natural language modeling domains. While t…

Language ModelingLanguage ModellingProtein Design