Papers Zero-Shot Environment Sound Classification

“Zero-Shot Environment Sound Classification” 태그가 달린 논문 6편 · 필터 해제

LanguageBind: Extending Video-Language Pretraining to N-modality by Language-based Semantic Alignment

2023-10-03 · Bin Zhu, Bin Lin, Munan Ning, Yang Yan 외

The video-language (VL) pretraining has achieved remarkable improvement in multiple downstream tasks. However, the current VL pretraining framework is hard to extend to multiple modalities (N modalities, N>=3) beyond vis…

Audio Classification Contrastive Learning Multimodal Deep Learning Scene Classification (unified classes)+11

ONE-PEACE: Exploring One General Representation Model Toward Unlimited Modalities

2023-05-18 · Peng Wang, Shijie Wang, Junyang Lin, Shuai Bai 외

In this work, we explore a scalable way for building a general representation model toward unlimited modalities. We release ONE-PEACE, a highly extensible model with 4B parameters that can seamlessly align and integrate …

1 Image, 2*2 Stitchi Action Classification AudioCaps Audio Classification+18

Alternating Gradient Descent and Mixture-of-Experts for Integrated Multimodal Perception

2023-05-10 · NeurIPS 2023 11 · Hassan Akbari, Dan Kondratyuk, Yin Cui, Rachel Hornung 외

We present Integrated Multimodal Perception (IMP), a simple and scalable multimodal multi-task training and modeling approach. IMP integrates multimodal inputs including image, video, text, and audio into a single Transf…

Classification image-classification Image Classification Mixture-of-Experts+7

ImageBind: One Embedding Space To Bind Them All

2023-05-09 · CVPR 2023 1 · Rohit Girdhar, Alaaeldin El-Nouby, Zhuang Liu, Mannat Singh 외

We present ImageBind, an approach to learn a joint embedding across six different modalities - images, text, audio, depth, thermal, and IMU data. We show that all combinations of paired data are not necessary to train su…

All Cross-Modal Retrieval Multimodal Deep Learning Retrieval+10

WavCaps: A ChatGPT-Assisted Weakly-Labelled Audio Captioning Dataset for Audio-Language Multimodal Research

2023-03-30 · Xinhao Mei, Chutong Meng, Haohe Liu, Qiuqiang Kong 외

The advancement of audio-language (AL) multimodal learning tasks has been significant in recent years. However, researchers face challenges due to the costly and time-consuming collection process of existing audio-langua…

Audio captioning Event Detection Language Modelling Large Language Model+4

AudioCLIP: Extending CLIP to Image, Text and Audio

2021-06-24 · Andrey Guzhov, Federico Raue, Jörn Hees, Andreas Dengel

In the past, the rapidly evolving field of sound classification greatly benefited from the application of methods from other domains. Today, we observe the trend to fuse domain-specific tasks and approaches together, whi…

Classification Environmental Sound Classification Sound Classification Zero-Shot Environment Sound Classification