Constructing Interpretive Spatio-Temporal Features for Multi-Turn Responses Selection
Response selection plays an important role in fully automated dialogue systems. Given the dialogue context, the goal of response selection is to identify the best-matched next utterance (i.e., response) from multiple candidates. Despite the efforts of many previous useful models, this task remains challenging due to the huge semantic gap and also the large size of candidate set. To address these issues, we propose a Spatio-Temporal Matching network (STM) for response selection. In detail, soft alignment is first used to obtain the local relevance between the context and the response. And then, we construct spatio-temporal features by aggregating attention images in time dimension and make use of 3D convolution and pooling operations to extract matching information. Evaluation on two large-scale multi-turn response selection tasks has demonstrated that our proposed model significantly outperforms the state-of-the-art model. Particularly, visualization analysis shows that the spatio-temporal features enables matching information in segment pairs and time sequences, and have good interpretability for multi-turn text matching.
Code (1)
Tasks
Text MatchingMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Kronecker Mask and Interpretive Prompts are Language-Action Video Learners
Contrastive language-image pretraining (CLIP) has significantly advanced image-based vision learning. A pressing topic subsequently arises: how can we effectively adapt CLIP to the video domain? Recent studies have focus…
Action RecognitionInductive BiasSpatio-Temporal-based Context Fusion for Video Anomaly Detection
Video anomaly detection aims to discover abnormal events in videos, and the principal objects are target objects such as people and vehicles. Each target in the video data has rich spatio-temporal context information. Mo…
Anomaly DetectionOptical Flow EstimationVideo Anomaly DetectionSpatio-Temporal Adversarial Learning for Detecting Unseen Falls
Fall detection is an important problem from both the health and machine learning perspective. A fall can lead to severe injuries, long term impairments or even death in some cases. In terms of machine learning, it presen…
BIG-bench Machine LearningPhilosophyCSTrack: Enhancing RGB-X Tracking via Compact Spatiotemporal Features
Effectively modeling and utilizing spatiotemporal features from RGB and other modalities (\eg, depth, thermal, and event data, denoted as X) is the core of RGB-X tracker design. Existing methods often employ two parallel…
Multi-scale Restoration of Missing Data in Optical Time-series Images with Masked Spatial-Temporal Attention Network
Remote sensing images often suffer from substantial data loss due to factors such as thick cloud cover and sensor limitations. Existing methods for imputing missing values in remote sensing images fail to fully exploit s…
Cloud RemovalImputationMissing ValuesTime Series