GeoDE: a Geographically Diverse Evaluation Dataset for Object Recognition
Current dataset collection methods typically scrape large amounts of data from the web. While this technique is extremely scalable, data collected in this way tends to reinforce stereotypical biases, can contain personally identifiable information, and typically originates from Europe and North America. In this work, we rethink the dataset collection paradigm and introduce GeoDE, a geographically diverse dataset with 61,940 images from 40 classes and 6 world regions, and no personally identifiable information, collected through crowd-sourcing. We analyse GeoDE to understand differences in images collected in this manner compared to web-scraping. Despite the smaller size of this dataset, we demonstrate its use as both an evaluation and training dataset, highlight shortcomings in current models, as well as show improved performances when even small amounts of GeoDE (1000 - 2000 images per region) are added to a training dataset. We release the full dataset and code at https://geodiverse-data-collection.cs.princeton.edu/
Code (0)
등록된 구현이 없습니다.
Tasks
ObjectObject RecognitionSimilar Papers 제목 키워드 기반
Multilingual Diversity Improves Vision-Language Representations
Massive web-crawled image-text datasets lay the foundation for recent progress in multimodal learning. These datasets are designed with the goal of training a model to do well on standard computer vision benchmarks, many…
DiversityText RetrievalGeodesic-based Salient Object Detection
Saliency detection has been an intuitive way to provide useful cues for object detection and segmentation, as desired for many vision and graphics applications. In this paper, we provided a robust method for salient obje…
Objectobject-detectionObject DetectionRGB Salient Object Detection+3Are Video Generation Models Geographically Fair? An Attraction-Centric Evaluation of Global Visual Knowledge
Recent advances in text-to-video generation have produced visually compelling results, yet it remains unclear whether these models encode geographically equitable visual knowledge. In this work, we investigate the geo-eq…
Text-to-Video GenerationTeraSim-World: Worldwide Safety-Critical Data Synthesis for End-to-End Autonomous Driving
Safe and scalable deployment of end-to-end (E2E) autonomous driving requires extensive and diverse data, particularly safety-critical events. Existing data are mostly generated from simulators with a significant sim-to-r…
Autonomous DrivingVideo GenerationDoes Object Recognition Work for Everyone?
The paper analyzes the accuracy of publicly available object-recognition systems on a geographically diverse dataset. This dataset contains household items and was designed to have a more representative geographical cove…
ObjectObject Recognition