An Information Theoretic Measurement of Topical Relevance in Learner Essays

2021-11-16 · ACL ARR November 2021 11 · Anonymous

We present a new approach to assess topical relevance in learner essays, leveraging recent advances in pretrained language models. Our approach is to generate features by calculating the normalized pointwise mutual information (NPMI) between prompts and sentences in an essay using a finetuned version of GPT-2. We demonstrate various desirable properties of this approach, including its strong performance under a variety of evaluation settings, its generalizability to novel prompts, and its interpretability.

📄 PDF Abstract BibTeX

Code (0)

등록된 구현이 없습니다.

Methods 이 논문이 사용한 방법론

Refunds@Expedia|||How do I get a full refund from Expedia? “How do I get a full refund from Expedia? How do I get a full refund from Expedia? – Call ☎️ +1-(888) 829 (0881) or +1-805-330-4056 or +1-805-330-4056 for Quick Help &…

Multi-Head Attention 설명 없음

Attention 설명 없음

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Cosine Annealing Cosine Annealing is a type of learning rate schedule that has the effect of starting with a large learning rate that is relatively rapidly decreased to a minimum value before…

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Residual Connection 설명 없음

Unsupervised Modeling of Topical Relevance in L2 Learner Text

2016-06-01 · WS 2016 6 · Ronan Cummins, Helen Yannakoudakis, Ted Briscoe

Information Retrieval

Sentence Similarity Measures for Fine-Grained Estimation of Topical Relevance in Learner Essays

2016-06-09 · WS 2016 6 · Marek Rei, Ronan Cummins

We investigate the task of assessing sentence-level prompt relevance in learner essays. Various systems using word overlap, neural embeddings and neural compositional models are evaluated on two datasets of learner writi…

Sentence Sentence Similarity Word Embeddings

Iterative Utility Judgment Framework via LLMs Inspired by Relevance in Philosophy

2024-06-17 · Hengran Zhang, Keping Bi, Jiafeng Guo, Xueqi Cheng

Utility and topical relevance are critical measures in information retrieval (IR), reflecting system and user perspectives, respectively. While topical relevance has long been emphasized, utility is a higher standard of …

Answer Generation Information Retrieval Passage Retrieval Philosophy+4

Enhancing Health Information Retrieval with RAG by Prioritizing Topical Relevance and Factual Accuracy

2025-02-07 · Rishabh Uapadhyay, Marco Viviani

The exponential surge in online health information, coupled with its increasing use by non-experts, highlights the pressing need for advanced Health Information Retrieval models that consider not only topical relevance b…

Information Retrieval Misinformation RAG Retrieval+4

The Anatomy of Relevance: Topical, Snippet and Perceived Relevance in Search Result Evaluation

2015-01-26 · Chuklin Aleksandr, de Rijke Maarten

Currently, the quality of a search engine is often determined using so-called topical relevance, i.e., the match between the user intent (expressed as a query) and the content of the document. In this work we want to dra…

Anatomy Retrieval