paper-with-me

Papers

Reverse-Complement Consistency for DNA Language Models

2025-09-23 · Mingqian Ma arxiv

A fundamental property of DNA is that the reverse complement (RC) of a sequence often carries identical biological meaning. However, state-of-the-art DNA language models frequently fail to capture this symmetry, producing inconsistent predictions for a sequence and its RC counterpart, which undermines their reliability. In this work, we introduce Reverse-Complement Consistency Regularization (RCCR), a simple and model-agnostic fine-tuning objective that directly penalizes the divergence between a model's prediction on a sequence and the aligned prediction on its reverse complement. We evaluate RCCR across three diverse backbones (Nucleotide Transformer, HyenaDNA, DNABERT-2) on a wide range of genomic tasks, including sequence classification, scalar regression, and profile prediction. Our experiments show that RCCR substantially improves RC robustness by dramatically reducing prediction flips and errors, all while maintaining or improving task accuracy compared to baselines such as RC data augmentation and test-time averaging. By integrating a key biological prior directly into the learning process, RCCR produces a single, intrinsically robust, and computationally efficient model fine-tuning recipe for diverse biology tasks.

📄 PDF Abstract BibTeX arXiv:2509.18529

Code (0)

등록된 구현이 없습니다.

Tasks

Data Augmentation

Similar Papers 제목 키워드 기반

LEDOM: Reverse Language Model

2025-07-02 · Xunjian Yin, Sitao Cheng, Yuxi Xie, Xinyu Hu 외 arxiv

Autoregressive language models are trained exclusively left-to-right. We explore the complementary factorization, training right-to-left at scale, and ask what reasoning patterns emerge when a model conditions on future …

ReverseEOL: Improving Training-free Text Embeddings via Text Reversal in Decoder-only LLMs

2026-06-04 · Ailiang Lin, Zhuoyun Li, Yusong Wang, Keyu Mao 외 arxiv

Recent advances in Large Language Models (LLMs) have opened new avenues for generating training-free text embeddings. However, the causal attention in decoder-only LLMs prevents earlier tokens from attending to future co…

Time-Reversal Provides Unsupervised Feedback to LLMs

2024-12-03 · Yerram Varun, Rahul Madhavan, Sravanti Addepalli, Arun Suggala 외

Large Language Models (LLMs) are typically trained to predict in the forward direction of time. However, recent works have shown that prompting these models to look back and critique their own generations can produce use…

Passage RetrievalRe-Ranking

Watermarking Autoregressive Image Generation

2025-06-19 · Nikola Jovanović, Ismail Labiad, Tomáš Souček, Martin Vechev 외

Watermarking the outputs of generative models has emerged as a promising approach for tracking their provenance. Despite significant interest in autoregressive image generation models and their potential for misuse, no p…

Image GenerationLanguage ModelingLanguage Modelling

Prompt-Reverse Inconsistency: LLM Self-Inconsistency Beyond Generative Randomness and Prompt Paraphrasing

2025-04-02 · Jihyun Janice Ahn, Wenpeng Yin

While the inconsistency of LLMs is not a novel topic, prior research has predominantly addressed two types of generative inconsistencies: i) Randomness Inconsistency: running the same LLM multiple trials, yielding varyin…

Language ModelingLanguage Modelling