Anticipating Future Object Compositions without Forgetting

2024-07-15 · Youssef Zahran, Gertjan Burghouts, Yke Bauke Eisma

Despite the significant advancements in computer vision models, their ability to generalize to novel object-attribute compositions remains limited. Existing methods for Compositional Zero-Shot Learning (CZSL) mainly focus on image classification. This paper aims to enhance CZSL in object detection without forgetting prior learned knowledge. We use Grounding DINO and incorporate Compositional Soft Prompting (CSP) into it and extend it with Compositional Anticipation. We achieve a 70.5% improvement over CSP on the harmonic mean (HM) between seen and unseen compositions on the CLEVR dataset. Furthermore, we introduce Contrastive Prompt Tuning to incrementally address model confusion between similar compositions. We demonstrate the effectiveness of this method and achieve an increase of 14.5% in HM across the pretrain, increment, and unseen sets. Collectively, these methods provide a framework for learning various compositions with limited data, as well as improving the performance of underperforming compositions when additional data becomes available.

📄 PDF Abstract BibTeX arXiv:2407.10723

Code (0)

등록된 구현이 없습니다.

Tasks

AttributeCompositional Zero-Shot Learning image-classification Image ClassificationObjectobject-detection Object Detection Zero-Shot Learning

Methods 이 논문이 사용한 방법론

Attention 설명 없음

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Residual Connection 설명 없음

Layer Normalization Unlike batch normalization, Layer Normalization directly estimates the normalization statistics from the summed inputs…

Focus 설명 없음

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Multi-Head Attention 설명 없음

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

Anticipating Visual Representations from Unlabeled Video

2015-04-29 · CVPR 2016 6 · Carl Vondrick, Hamed Pirsiavash, Antonio Torralba

Anticipating actions and objects before they start or appear is a difficult problem in computer vision with several real-world applications. This task is challenging partly because it requires leveraging extensive knowle…

Anticipating Next Active Objects for Egocentric Videos

2023-02-13 · Sanket Thakur, Cigdem Beyan, Pietro Morerio, Vittorio Murino 외

This paper addresses the problem of anticipating the next-active-object location in the future, for a given egocentric video clip where the contact might happen, before any action takes place. The problem is considerably…

Object

Learning without Forgetting for 3D Point Cloud Objects

2021-06-27 · Townim Chowdhury, Mahira Jalisha, Ali Cheraghian, Shafin Rahman

When we fine-tune a well-trained deep learning model for a new set of classes, the network learns new concepts but gradually forgets the knowledge of old training. In some real-life applications, we may be interested in …

Knowledge Distillation

Prompt-Based Continual Compositional Zero-Shot Learning

2025-12-09 · Sauda Maryam, Sara Nadeem, Faisal Qureshi, Mohsen Ali arxiv

We tackle continual adaptation of vision-language models to new attributes, objects, and their compositions in Compositional Zero-Shot Learning (CZSL), while preventing forgetting of prior knowledge. Unlike classical con…

Compositional Zero-Shot Learning Continual Learning

Input anticipating critical reservoirs show power law forgetting of unexpected input events

2014-04-25 · Norbert Michael Mayer

Usually, reservoir computing shows an exponential memory decay. This paper investigates under which circumstances echo state networks can show a power law forgetting. That means traces of earlier events can be found in t…