MineralImage5k: A benchmark for zero-shot raw mineral visual recognition and description
Mineral image recognition is a challenging computer vision problem. Without external tools, even a human expert cannot distinguish some mineral species accurately. Previous research was mainly focused on processed mineral recognition. This is considered to be a simplified statement of a problem because processed minerals are more visually expressive. On the contrary, in a raw sample, the target mineral can appear in the form of thinly represented inclusions. In real life, the raw samples usually require automatic mineral species identification. Another difficulty in raw mineral recognition is the shortage of publicly available training and validation data. It is impossible to compare different deep learning approaches when the results are evaluated on dissimilar data. The main contribution of this paper is providing an open benchmark for zero-shot raw mineral visual recognition. Besides the evaluation-only zero-shot classification dataset, we publish subsets for segmentation, mineral size estimation, and few-shot classification. For all of the provided computer vision problems, we publish baseline solutions we offer for the community to beat.
Code (1)
Tasks
zero-shot-classificationZero-Shot LearningSimilar Papers 제목 키워드 기반
Learning Zero-Shot Material States Segmentation, by Implanting Natural Image Patterns in Synthetic Data
Visual recognition of materials and their states is essential for understanding the physical world, from identifying wet regions on surfaces or stains on fabrics to detecting infected areas on plants or minerals in rocks…
Material RecognitionSegmentationZero-Shot LearningZero Shot SegmentationZero-Shot Semantic Segmentation via Spatial and Multi-Scale Aware Visual Class Embedding
Fully supervised semantic segmentation technologies bring a paradigm shift in scene understanding. However, the burden of expensive labeling cost remains as a challenge. To solve the cost problem, recent studies proposed…
Domain AdaptationLanguage ModelingLanguage ModellingScene Understanding+4Towards Automated Petrography
Petrography is a branch of geology that analyzes the mineralogical composition of rocks from microscopical thin section samples. It is essential for understanding rock properties across geology, archaeology, engineering,…
Zero-Shot Visual Reasoning by Vision-Language Models: Benchmarking and Analysis
Vision-language models (VLMs) have shown impressive zero- and few-shot performance on real-world visual question answering (VQA) benchmarks, alluding to their capabilities as visual reasoning engines. However, the benchm…
BenchmarkingLarge Language ModelQuestion AnsweringVisual Question Answering+3Semantically Grounded Visual Embeddings for Zero-Shot Learning
Zero-shot learning methods rely on fixed visual and semantic embeddings, extracted from independent vision and language models, both pre-trained for other large-scale tasks. This is a weakness of current zero-shot learni…
Zero-Shot Learning