Technical Report for ICCV 2021 Challenge SSLAD-Track3B: Transformers Are Better Continual Learners

2022-01-13 · Duo Li, Guimei Cao, Yunlu Xu, Zhanzhan Cheng, Yi Niu

In the SSLAD-Track 3B challenge on continual learning, we propose the method of COntinual Learning with Transformer (COLT). We find that transformers suffer less from catastrophic forgetting compared to convolutional neural network. The major principle of our method is to equip the transformer based feature extractor with old knowledge distillation and head expanding strategies to compete catastrophic forgetting. In this report, we first introduce the overall framework of continual learning for object detection. Then, we analyse the key elements' effect on withstanding catastrophic forgetting in our solution. Our method achieves 70.78 mAP on the SSLAD-Track 3B challenge test set.

📄 PDF Abstract BibTeX arXiv:2201.04924

Code (0)

등록된 구현이 없습니다.

Tasks

Continual Learning Knowledge Distillation object-detection Object Detection

Methods 이 논문이 사용한 방법론

Attention 설명 없음

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

Position-Wise Feed-Forward Layer 설명 없음

Multi-Head Attention 설명 없음

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Absolute Position Encodings Absolute Position Encodings are a type of position embeddings for [Transformer-based models] where positional encodings are…

BPE Byte Pair Encoding, or BPE, is a subword segmentation algorithm that encodes rare and unknown words as sequences of subword units. The intuition is that various word…

2nd Place Solution for SODA10M Challenge 2021 -- Continual Detection Track

2021-10-25 · Manoj Acharya, Christopher Kanan

In this technical report, we present our approaches for the continual object detection track of the SODA10M challenge. We adapt ResNet50-FPN as the baseline and try several improvements for the final submission model. We…

Autonomous Driving object-detection Object Detection Position+2

A Baseline Framework for Part-level Action Parsing and Action Recognition

2021-10-07 · Xiaodong Chen, Xinchen Liu, Kun Liu, Wu Liu 외

This technical report introduces our 2nd place solution to Kinetics-TPS Track on Part-level Action Parsing in ICCV DeeperAction Workshop 2021. Our entry is mainly based on YOLOF for instance and part detection, HRNet for…

Action Parsing Action Recognition Pose Estimation

"Knights": First Place Submission for VIPriors21 Action Recognition Challenge at ICCV 2021

2021-10-14 · Ishan Dave, Naman Biyani, Brandon Clark, Rohit Gupta 외

This technical report presents our approach "Knights" to solve the action recognition task on a small subset of Kinetics-400 i.e. Kinetics400ViPriors without using any extra-data. Our approach has 3 main components: stat…

Action Recognition Optical Flow Estimation

Learning Effective NeRFs and SDFs Representations with 3D Generative Adversarial Networks for 3D Object Generation: Technical Report for ICCV 2023 OmniObject3D Challenge

2023-09-28 · Zheyuan Yang, Yibo Liu, Guile Wu, Tongtong Cao 외

In this technical report, we present a solution for 3D object generation of ICCV 2023 OmniObject3D Challenge. In recent years, 3D object generation has made great process and achieved promising results, but it remains a …

Decoder Object

Pinpointing Trigger Moment for Grounded Video QA: Enhancing Spatio-temporal Grounding in Multimodal Large Language Models

2025-11-04 · Jinhwan Seo, Yoonki Cho, Junhyug Noh, Sung-eui Yoon arxiv

In this technical report, we introduce a framework to address Grounded Video Question Answering (GVQA) task for the ICCV 2025 Perception Test Challenge. The GVQA task demands robust multimodal models capable of complex r…

Video Question Answering