Event-Centric World Modeling with Memory-Augmented Retrieval for Embodied Decision-Making
Autonomous agents operating in dynamic environments increasingly demand decision-making systems that are both efficient and interpretable. Hence we propose the Event-Retrieve-Action (ERA) framework, an alternative formulation for embodied decision-making that bridges the gap between black-box imitation and interpretable memory retrieval while enabling online refinement without retraining. The environment is represented as structured semantic events encoded into an interpretable latent representation, and decisions are generated by retrieving relevant prior experiences from a knowledge bank of event-action pairs. Final actions are produced through weighted aggregation of retrieved maneuvers, enabling transparent and physically consistent decision-making. Experiments in UAV navigation demonstrate real-time performance and adaptive behavior in dynamic environments as a representative embodied decision-making application scenario.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
EgoCITE: Context-Augmented Indexing and Time-Aware Retrieval for Long-Horizon Egocentric Memory
Long-horizon egocentric memory transforms continuous first-person video and audio into a searchable record of past experiences. We demonstrate two bottlenecks in existing systems: indices built from context-poor captions…
E2(GO)MOTION: Motion Augmented Event Stream for Egocentric Action Recognition
Event cameras are novel bio-inspired sensors, which asynchronously capture pixel-level intensity changes in the form of "events". Due to their sensing mechanism, event cameras have little to no motion blur, a very hi…
Action RecognitionEvent-based visionOptical Flow EstimationVinci2: Providing Proactive Assistance in Continuous Egocentric Videos
When should an intelligent assistant speak up without being asked? Continuous egocentric video offers rich, evolving context that enables a new form of assistance: one that is proactive rather than merely reactive. Yet e…
E$^2$(GO)MOTION: Motion Augmented Event Stream for Egocentric Action Recognition
Event cameras are novel bio-inspired sensors, which asynchronously capture pixel-level intensity changes in the form of "events". Due to their sensing mechanism, event cameras have little to no motion blur, a very high t…
Action RecognitionEvent-based visionOptical Flow EstimationTowards Unified World Models for Visual Navigation via Memory-Augmented Planning and Foresight
Enabling embodied agents to imagine future states is essential for robust and generalizable visual navigation. Yet, state-of-the-art systems typically rely on modular designs that decouple navigation planning from visual…
Visual Navigation