Boolean-aware Attention for Dense Retrieval

2025-03-03 · Quan Mai, Susan Gauch, Douglas Adams

We present Boolean-aware attention, a novel attention mechanism that dynamically adjusts token focus based on Boolean operators (e.g., and, or, not). Our model employs specialized Boolean experts, each tailored to amplify or suppress attention for operator-specific contexts. A predefined gating mechanism activates the corresponding experts based on the detected Boolean type. Experiments on Boolean retrieval datasets demonstrate that integrating BoolAttn with BERT greatly enhances the model's capability to process Boolean queries.

📄 PDF Abstract BibTeX arXiv:2503.01753

Code (0)

등록된 구현이 없습니다.

Tasks

Retrieval

Methods 이 논문이 사용한 방법론

Attention 설명 없음

Adam 설명 없음

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Weight Decay 설명 없음

Dropout Dropout is a regularization technique for neural networks that drops a unit (along with connections) at training time with a specified probability $p$ (a common value is…

Attention Dropout Attention Dropout is a type of dropout used in attention-based architectures, where elements are randomly dropped out of the…

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

Residual Connection 설명 없음

BoolQuestions: Does Dense Retrieval Understand Boolean Logic in Language?

2024-11-19 · Zongmeng Zhang, Jinhua Zhu, Wengang Zhou, Xiang Qi 외

Dense retrieval, which aims to encode the semantic information of arbitrary text into dense vector representations or embeddings, has emerged as an effective and efficient paradigm for text retrieval, consequently becomi…

Retrieval Text Retrieval

AttentionRetriever: Attention Layers are Secretly Long Document Retrievers

2026-02-12 · David Jiahao Fu, Lam Thanh Do, Jiayu Li, Kevin Chen-Chuan Chang arxiv

Retrieval augmented generation (RAG) has been widely adopted to help Large Language Models (LLMs) to process tasks involving long documents. However, existing retrieval models are not designed for long document retrieval…

Query-Aware Graph Neural Networks for Enhanced Retrieval-Augmented Generation

2025-07-25 · Vibhor Agrawal, Fay Wang, Rishi Puri arxiv

We present a novel graph neural network (GNN) architecture for retrieval-augmented generation (RAG) that leverages query-aware attention mechanisms and learned scoring heads to improve retrieval accuracy on complex, mult…

Graph Neural Network Question Answering Knowledge Graphs

Search, Inspect, Fetch: Exploiting Structure-Aware Boolean Retrieval for Deep-Search Agents

2026-08-03 · Shuai Wang, Haodong Chen, Yu Yin, Shengyao Zhuang 외 hf

Existing deep-search agents use a Search-Visit workflow that retrieves whole webpages without considering the structure they expose through titles, headings, sections, and metadata. This prevents agents from directly con…

Exploring new Approaches for Information Retrieval through Natural Language Processing

2025-05-04 · Manak Raj, Nidhi Mishra

This review paper explores recent advancements and emerging approaches in Information Retrieval (IR) applied to Natural Language Processing (NLP). We examine traditional IR models such as Boolean, vector space, probabili…

Argument Mining Hate Speech Detection Information Retrieval Retrieval