Developing hierarchical anticipations via neural network-based event segmentation
Humans can make predictions on various time scales and hierarchical levels. Thereby, the learning of event encodings seems to play a crucial role. In this work we model the development of hierarchical predictions via autonomously learned latent event codes. We present a hierarchical recurrent neural network architecture, whose inductive learning biases foster the development of sparsely changing latent state that compress sensorimotor sequences. A higher level network learns to predict the situations in which the latent states tend to change. Using a simulated robotic manipulator, we demonstrate that the system (i) learns latent states that accurately reflect the event structure of the data, (ii) develops meaningful temporal abstract predictions on the higher level, and (iii) generates goal-anticipatory behavior similar to gaze behavior found in eye-tracking studies with infants. The architecture offers a step towards the autonomous learning of compressed hierarchical encodings of gathered experiences and the exploitation of these encodings to generate adaptive behavior.
Code (1)
Tasks
Event SegmentationInductive LearningSimilar Papers 제목 키워드 기반
A Review of Mechanistic Models of Event Comprehension
This review examines theoretical assumptions and computational models of event comprehension, tracing the evolution from discourse comprehension theories to contemporary event cognition frameworks. The review covers key …
Event SegmentationRepresentation LearningSimultaneous Machine Translation with Tailored Reference
Simultaneous machine translation (SiMT) generates translation while reading the whole source sentence. However, existing SiMT models are typically trained using the same reference disregarding the varying amounts of avai…
Machine TranslationSentenceTranslationHierarchical Multi-label Classification for Fine-level Event Extraction from Aviation Accident Reports
A large volume of accident reports is recorded in the aviation domain, which greatly values improving aviation safety. To better use those reports, we need to understand the most important events or impact factors accord…
Event ExtractionHierarchical Multi-label ClassificationMulti-Label ClassificationMUlTI-LABEL-ClASSIFICATIONEvent-Aware Instructed Assistant for Referring Video Segmentation
Existing referring video segmentation methods often treat a video as a single event consisting of multiple images, overlooking the fact that a video typically contains multiple distinct events. Under such a mechanism, th…
Video SegmentationMouseSIS: A Frames-and-Events Dataset for Space-Time Instance Segmentation of Mice
Enabled by large annotated datasets, tracking and segmentation of objects in videos has made remarkable progress in recent years. Despite these advancements, algorithms still struggle under degraded conditions and during…
Instance SegmentationSegmentationSemantic SegmentationVideo Instance Segmentation