Scientific Opinion Summarization: Paper Meta-review Generation Dataset, Methods, and Evaluation
Opinions in scientific research papers can be divergent, leading to controversies among reviewers. However, most existing datasets for opinion summarization are centered around product reviews and assume that the analyzed opinions are non-controversial, failing to account for the variability seen in other contexts such as academic papers, political debates, or social media discussions. To address this gap, we propose the task of scientific opinion summarization, where research paper reviews are synthesized into meta-reviews. To facilitate this task, we introduce the ORSUM dataset covering 15,062 paper meta-reviews and 57,536 paper reviews from 47 conferences. Furthermore, we propose the Checklist-guided Iterative Introspection approach, which breaks down scientific opinion summarization into several stages, iteratively refining the summary under the guidance of questions from a checklist. Our experiments show that (1) human-written summaries do not always satisfy all necessary criteria such as depth of discussion, and identifying consensus and controversy for the specific domain, and (2) the combination of task decomposition and iterative self-refinement shows strong potential for enhancing the opinions and can be applied to other complex text generation using black-box LLMs.
Code (1)
Tasks
Opinion SummarizationReview GenerationText GenerationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Aspect-Aware Decomposition for Opinion Summarization
Opinion summarization plays a key role in deriving meaningful insights from large-scale online reviews. To make this process more explainable and grounded, we propose a modular approach guided by review aspects which sep…
Opinion SummarizationA Sentiment Consolidation Framework for Meta-Review Generation
Modern natural language generation systems with Large Language Models (LLMs) exhibit the capability to generate a plausible summary of multiple documents; however, it is uncertain if they truly possess the capability of …
Review GenerationText GenerationGLIMPSE: Pragmatically Informative Multi-Document Summarization for Scholarly Reviews
Scientific peer review is essential for the quality of academic publications. However, the increasing number of paper submissions to conferences has strained the reviewing process. This surge poses a burden on area chair…
Document SummarizationMulti-Document SummarizationNoisy Pairing and Partial Supervision for Stylized Opinion Summarization
Opinion summarization research has primarily focused on generating summaries reflecting important opinions from customer reviews without paying much attention to the writing style. In this paper, we propose the stylized …
Opinion SummarizationSelf-Supervised Multimodal Opinion Summarization
Recently, opinion summarization, which is the generation of a summary from multiple reviews, has been conducted in a self-supervised manner by considering a sampled review as a pseudo summary. However, non-text data such…
DecoderOpinion Summarization