paper-with-me

Papers

Measuring Dataset Diversity from a Geometric Perspective

2026-02-10 · Yang Ba, Mohammad Sadeq Abolhasani, Michelle V Mancenido, Rong Pan arxiv

Diversity can be broadly defined as the presence of meaningful variation across elements, which can be viewed from multiple perspectives, including statistical variation and geometric structural richness in the dataset. Existing diversity metrics, such as feature-space dispersion and metric-space magnitude, primarily capture distributional variation or entropy, while largely neglecting the geometric structure of datasets. To address this gap, we introduce a framework based on topological data analysis (TDA) and persistence landscapes (PLs) to extract and quantify geometric features from data. This approach provides a theoretically grounded means of measuring diversity beyond entropy, capturing the rich geometric and structural properties of datasets. Through extensive experiments across diverse modalities, we demonstrate that our proposed PLs-based diversity metric (PLDiv) is powerful, reliable, and interpretable, directly linking data diversity to its underlying geometry and offering a foundational tool for dataset construction, augmentation, and evaluation.

📄 PDF Abstract BibTeX arXiv:2602.09340

Code (0)

등록된 구현이 없습니다.

Similar Papers 제목 키워드 기반

Measuring Diversity in Synthetic Datasets

2025-02-12 · Yuchang Zhu, Huizhe Zhang, Bingzhe Wu, Jintang Li 외

Large language models (LLMs) are widely adopted to generate synthetic datasets for various natural language processing (NLP) tasks, such as text classification and summarization. However, accurately measuring the diversi…

ClassificationDiversitytext-classificationText Classification

We Need to Measure Data Diversity in NLP -- Better and Broader

2025-05-26 · Dong Nguyen, Esther Ploeger

Although diversity in NLP datasets has received growing attention, the question of how to measure it remains largely underexplored. This opinion paper examines the conceptual and methodological challenges of measuring da…

Diversityvalid

Hierarchical Re-estimation of Topic Models for Measuring Topical Diversity

2017-01-16 · Azarbonyad Hosein, Dehghani Mostafa, Kenter Tom, Marx Maarten 외

A high degree of topical diversity is often considered to be an important characteristic of interesting text documents. A recent proposal for measuring topical diversity identifies three elements for assessing diversity:…

DiversityTopic Models

Geometric Remove-and-Retrain (GOAR): Coordinate-Invariant eXplainable AI Assessment

2024-07-17 · Yong-Hyun Park, Junghoon Seo, Bomseok Park, Seongsu Lee 외

Identifying the relevant input features that have a critical influence on the output results is indispensable for the development of explainable artificial intelligence (XAI). Remove-and-Retrain (ROAR) is a widely accept…

Explainable artificial intelligenceExplainable Artificial Intelligence (XAI)

emb-diversity: A Tool for Embedding-Based Measurement of Data Diversity

2026-07-22 · Cantao Su, Menan Velayuthan, Esther Ploeger, Dong Nguyen 외 arxiv

There is growing evidence that data diversity is crucial for developing fair and robust NLP models. However, current approaches to measure diversity remain inconsistent and fragmented: While there exist a number of tools…