Shuffle and Attend: Video Domain Adaptation

2020-08-01 · ECCV 2020 8 · Jinwoo Choi, Gaurav Sharma, Samuel Schulter, Jia-Bin Huang

We address the problem of domain adaptation in videos for the task of human action recognition. Inspired by image-based domain adaptation, we can perform video adaptation by aligning the features of frames or clips of source and target videos. However, equally aligning all clips is sub-optimal as not all clips are informative for the task. As the first novelty, we propose an attention mechanism which focuses on more discriminative clips and directly optimizes for video-level (cf. clip-level) alignment. As the backgrounds are often very different between source and target, the source background-corrupted model adapts poorly to target domain videos. To alleviate this, as a second novelty, we propose to use the clip order prediction as an auxiliary task. The clip order prediction loss, when combined with domain adversarial loss, encourages learning of representations which focus on the humans and objects involved in the actions, rather than the uninformative and widely differing (between source and target) backgrounds. We empirically show that both components contribute positively towards adaptation performance. We report state-of-the-art performances on two out of three challenging public benchmarks, two based on the UCF and HMDB datasets, and one on Kinetics to NEC-Drone datasets. We also support the intuitions and the results with qualitative results.

📄 PDF Abstract BibTeX

Code (0)

등록된 구현이 없습니다.

Tasks

Action Recognition Domain Adaptation Temporal Action Localization Unsupervised Domain Adaptation

Temporal Attentive Alignment for Video Domain Adaptation

2019-05-26 · Min-Hung Chen, Zsolt Kira, Ghassan AlRegib

Although various image-based domain adaptation (DA) techniques have been proposed in recent years, domain shift in videos is still not well-explored. Most previous works only evaluate performance on small-scale datasets …

Domain Adaptation

Temporal Attentive Alignment for Large-Scale Video Domain Adaptation

2019-07-30 · ICCV 2019 10 · Min-Hung Chen, Zsolt Kira, Ghassan AlRegib, Jaekwon Yoo 외

Domain Adaptation Unsupervised Domain Adaptation

Source-free Video Domain Adaptation by Learning Temporal Consistency for Action Recognition

2022-03-09 · Yuecong Xu, Jianfei Yang, Haozhi Cao, Keyu Wu 외

Video-based Unsupervised Domain Adaptation (VUDA) methods improve the robustness of video models, enabling them to be applied to action recognition tasks across different environments. However, these methods require cons…

Action Recognition Domain Adaptation Source-Free Domain Adaptation Unsupervised Domain Adaptation

Multi-Source Video Domain Adaptation with Temporal Attentive Moment Alignment

2021-09-21 · Yuecong Xu, Jianfei Yang, Haozhi Cao, Keyu Wu 외

Multi-Source Domain Adaptation (MSDA) is a more practical domain adaptation scenario in real-world scenarios. It relaxes the assumption in conventional Unsupervised Domain Adaptation (UDA) that source data are sampled fr…

Domain Adaptation Unsupervised Domain Adaptation

Partial Video Domain Adaptation with Partial Adversarial Temporal Attentive Network

2021-07-11 · ICCV 2021 10 · Yuecong Xu, Jianfei Yang, Haozhi Cao, Qi Li 외

Partial Domain Adaptation (PDA) is a practical and general domain adaptation scenario, which relaxes the fully shared label space assumption such that the source label space subsumes the target one. The key challenge of …

Domain Adaptation Partial Domain Adaptation