Evaluating Vision-Language Models for Zero-Shot Detection, Classification, and Association of Motorcycles, Passengers, and Helmets

2024-08-05 · Lucas Choi, Ross Greer

Motorcycle accidents pose significant risks, particularly when riders and passengers do not wear helmets. This study evaluates the efficacy of an advanced vision-language foundation model, OWLv2, in detecting and classifying various helmet-wearing statuses of motorcycle occupants using video data. We extend the dataset provided by the CVPR AI City Challenge and employ a cascaded model approach for detection and classification tasks, integrating OWLv2 and CNN models. The results highlight the potential of zero-shot learning to address challenges arising from incomplete and biased training datasets, demonstrating the usage of such models in detecting motorcycles, helmet usage, and occupant positions under varied conditions. We have achieved an average precision of 0.5324 for helmet detection and provided precision-recall curves detailing the detection and classification performance. Despite limitations such as low-resolution data and poor visibility, our research shows promising advancements in automated vehicle safety and traffic safety enforcement systems.

📄 PDF Abstract BibTeX arXiv:2408.02244

Code (0)

등록된 구현이 없습니다.

Tasks

Zero-Shot Learning

Vision-Language Models Performing Zero-Shot Tasks Exhibit Gender-based Disparities

2023-01-26 · Melissa Hall, Laura Gustafson, Aaron Adcock, Ishan Misra 외

We explore the extent to which zero-shot vision-language models exhibit gender bias for different vision tasks. Vision models traditionally required task-specific labels for representing concepts, as well as finetuning; …

image-classification Image Classification object-detection Object Detection+3

FADE: Few-shot/zero-shot Anomaly Detection Engine using Large Vision-Language Model

2024-08-31 · Yuanwei Li, Elizaveta Ivanova, Martins Bruveris

Automatic image anomaly detection is important for quality inspection in the manufacturing industry. The usual unsupervised anomaly detection approach is to train a model for each object class using a dataset of normal s…

Anomaly Detection Anomaly Segmentation Language Modeling Language Modelling+2

Evaluating Cascaded Methods of Vision-Language Models for Zero-Shot Detection and Association of Hardhats for Increased Construction Safety

2024-10-16 · Lucas Choi, Ross Greer

This paper evaluates the use of vision-language models (VLMs) for zero-shot detection and association of hardhats to enhance construction safety. Given the significant risk of head injuries in construction, proper enforc…

VL-Taboo: An Analysis of Attribute-based Zero-shot Capabilities of Vision-Language Models

2022-09-12 · Felix Vogel, Nina Shvetsova, Leonid Karlinsky, Hilde Kuehne

Vision-language models trained on large, randomly collected data had significant impact in many areas since they appeared. But as they show great performance in various fields, such as image-text-retrieval, their inner w…

Attribute Image-text Retrieval Retrieval Text Retrieval+1

Evaluating Multimodal Large Language Models on Spoken Sarcasm Understanding

2025-09-18 · Zhu Li, Xiyuan Gao, Yuqing Zhang, Shekhar Nayak 외 arxiv

Sarcasm detection remains a challenge in natural language understanding, as sarcastic intent often relies on subtle cross-modal cues spanning text, speech, and vision. While prior work has primarily focused on textual or…

Natural Language Understanding Sarcasm Detection