Leveraging Static Relationships for Intra-Type and Inter-Type Message Passing in Video Question Answering
Video Question Answering (VideoQA) is an important research direction in the field of artificial intelligence, enabling machines to understand video content and perform reasoning and answering based on natural language questions. Although methods based on static relationship reasoning have made certain progress, there are still deficiencies in the accuracy of static relationship recognition and representation, and they have not fully utilized the static relationship information in videos for in-depth reasoning and analysis. Therefore, this paper proposes a reasoning method for intra-type and inter-type message passing based on static relationships. This method constructs a dual graph for intra-type message passing reasoning and builds a heterogeneous graph based on static relationships for inter-type message passing reasoning. The intra-type message passing reasoning model captures the neighborhood information of targets and relationships related to the question in the dual graph, updating the dual graph to obtain intra-type clues for answering the question. The inter-type message passing reasoning model captures the neighborhood information of targets and relationships from different categories related to the question in the heterogeneous graph, updating the heterogeneous graph to obtain inter-type clues for answering the question. Finally, the answers are inferred by combining the intra-type and inter-type clues based on static relationships. Experimental results on the ANetQA and Next-QA datasets demonstrate the effectiveness of this method.
Code (0)
등록된 구현이 없습니다.
Tasks
Question AnsweringVideo Question AnsweringSimilar Papers 제목 키워드 기반
Detecting Human-Object Relationships in Videos
We study a crucial problem in video analysis: human-object relationship detection. The majority of previous approaches are developed only for the static image scenario, without incorporating the temporal dynamics so …
Human-Object Relationship DetectionObjectRelationship DetectionJointly Modeling Inter- & Intra-Modality Dependencies for Multi-modal Learning
Supervised multi-modal learning involves mapping multiple modalities to a target label. Previous studies in this field have concentrated on capturing in isolation either the inter-modality dependencies (the relationships…
Redundancy-optimized Multi-head Attention Networks for Multi-View Multi-Label Feature Selection
Multi-view multi-label data offers richer perspectives for artificial intelligence, but simultaneously presents significant challenges for feature selection due to the inherent complexity of interrelations among features…
Multivariate Time Series Anomaly Detection via Dynamic Graph Forecasting
Anomalies in univariate time series often refer to abnormal values and deviations from the temporal patterns from majority of historical observations. In multivariate time series, anomalies also refer to abnormal changes…
Anomaly DetectionTime SeriesTime Series AnalysisTime Series Anomaly Detection+1Multi-Grained Query-Guided Set Prediction Network for Grounded Multimodal Named Entity Recognition
Grounded Multimodal Named Entity Recognition (GMNER) is an emerging information extraction (IE) task, aiming to simultaneously extract entity spans, types, and corresponding visual regions of entities from given sentence…
Grounded Multimodal Named Entity RecognitionMachine Reading Comprehensionnamed-entity-recognitionNamed Entity Recognition+2