DomainSum: A Hierarchical Benchmark for Fine-Grained Domain Shift in Abstractive Text Summarization
Most research on abstractive summarization focuses on single-domain applications, often neglecting how domain shifts between documents affect performance and the generalization ability of summarization models. To address this issue, we introduce DomainSum, a hierarchical benchmark designed to capture fine-grained domain shifts in abstractive summarization. We categorize these shifts into three levels: genre, style, and topic, and demonstrate through comprehensive benchmark analysis that they follow a hierarchical structure. Furthermore, we evaluate the domain generalization capabilities of commonly used pre-trained language models (PLMs) and large language models (LLMs) in in-domain and cross-domain settings.
Code (1)
Tasks
Abstractive Text SummarizationDomain GeneralizationText SummarizationSimilar Papers 제목 키워드 기반
DiningBench: A Hierarchical Multi-view Benchmark for Perception and Reasoning in the Dietary Domain
Recent advancements in Vision-Language Models (VLMs) have revolutionized general visual understanding. However, their application in the food domain remains constrained by benchmarks that rely on coarse-grained categorie…
Visual Question AnsweringPartNet: A Large-scale Benchmark for Fine-grained and Hierarchical Part-level 3D Object Understanding
We present PartNet: a consistent, large-scale dataset of 3D objects annotated with fine-grained, instance-level, and hierarchical 3D part information. Our dataset consists of 573,585 part instances over 26,671 3D models …
3D Instance Segmentation3D Semantic SegmentationBenchmarkingInstance Segmentation+2Hierarchical Fine-Grained Image Forgery Detection and Localization
Differences in forgery attributes of images generated in CNN-synthesized and image-editing domains are large, and such differences make a unified image forgery detection and localization (IFDL) challenging. To this end, …
AttributeClassificationImage Forgery DetectionRepresentation LearningDrive&Act: A Multi-Modal Dataset for Fine-Grained Driver Behavior Recognition in Autonomous Vehicles
We introduce the novel domain-specific Drive&Act benchmark for fine-grained categorization of driver behavior. Our dataset features twelve hours and over 9.6 million frames of people engaged in distractive activities dur…
Action RecognitionActivity RecognitionAutonomous VehiclesMulti-Granularity Feature Calibration via VFM for Domain Generalized Semantic Segmentation
Domain Generalized Semantic Segmentation (DGSS) aims to improve the generalization ability of models across unseen domains without access to target data during training. Recent advances in DGSS have increasingly exploite…
parameter-efficient fine-tuningSemantic SegmentationDomain Generalization