Transformed ROIs for Capturing Visual Transformations in Videos
Modeling the visual changes that an action brings to a scene is critical for video understanding. Currently, CNNs process one local neighbourhood at a time, thus contextual relationships over longer ranges, while still learnable, are indirect. We present TROI, a plug-and-play module for CNNs to reason between mid-level feature representations that are otherwise separated in space and time. The module relates localized visual entities such as hands and interacting objects and transforms their corresponding regions of interest directly in the feature maps of convolutional layers. With TROI, we achieve state-of-the-art action recognition results on the large-scale datasets Something-Something-V2 and EPIC-Kitchens-100.
Code (0)
등록된 구현이 없습니다.
Tasks
Action RecognitionVideo UnderstandingSimilar Papers 제목 키워드 기반
RITUAL: Random Image Transformations as a Universal Anti-hallucination Lever in Large Vision Language Models
Recent advancements in Large Vision Language Models (LVLMs) have revolutionized how machines understand and generate textual responses based on visual inputs, yet they often produce "hallucinatory" outputs that misinterp…
HallucinationMMEObject HallucinationDiscovering States and Transformations in Image Collections
Objects in visual scenes come in a rich variety of transformed states. A few classes of transformation have been heavily studied in computer vision: mostly simple, parametric changes in color and geometry. However, trans…
ObjectGIFT: Learning Transformation-Invariant Dense Visual Descriptors via Group CNNs
Finding local correspondences between images with different viewpoints requires local descriptors that are robust against geometric transformations. An approach for transformation invariance is to integrate out the trans…
Pose EstimationDual-Flow Transformation Network for Deformable Image Registration with Region Consistency Constraint
Deformable image registration is able to achieve fast and accurate alignment between a pair of images and thus plays an important role in many medical image studies. The current deep learning (DL)-based image registratio…
Image RegistrationRobustness to Transformations Across Categories: Is Robustness To Transformations Driven by Invariant Neural Representations?
Deep Convolutional Neural Networks (DCNNs) have demonstrated impressive robustness to recognize objects under transformations (eg. blur or noise) when these transformations are included in the training set. A hypothesis …