Universal Embeddings for Spatio-Temporal Tagging of Self-Driving Logs
In this paper, we tackle the problem of spatio-temporal tagging of self-driving scenes from raw sensor data. Our approach learns a universal embedding for all tags, enabling efficient tagging of many attributes and faster learning of new attributes with limited data. Importantly, the embedding is spatio-temporally aware, allowing the model to naturally output spatio-temporal tag values. Values can then be pooled over arbitrary regions, in order to, for example, compute the pedestrian density in front of the SDV, or determine if a car is blocking another car at a 4-way intersection. We demonstrate the effectiveness of our approach on a new large scale self-driving dataset, SDVScenes, containing 15 attributes relating to vehicle and pedestrian density, the actions of each actor, the speed of each actor, interactions between actors, and the topology of the road map.
Code (0)
등록된 구현이 없습니다.
Tasks
BlockingTAGTemporal TaggingSimilar Papers 제목 키워드 기반
CRNNs for Urban Sound Tagging with spatiotemporal context
This paper describes CRNNs we used to participate in Task 5 of the DCASE 2020 challenge. This task focuses on hierarchical multilabel urban sound tagging with spatiotemporal context. The code is available on our GitHub r…
Audio ClassificationAudio TaggingEnvironmental Sound ClassificationLWM-Temporal: Sparse Spatio-Temporal Attention for Wireless Channel Representation Learning
LWM-Temporal is a new member of the Large Wireless Models (LWM) family that targets the spatiotemporal nature of wireless channels. Designed as a task-agnostic foundation model, LWM-Temporal learns universal channel embe…
Representation LearningUniCtrl: Improving the Spatiotemporal Consistency of Text-to-Video Diffusion Models via Training-Free Unified Attention Control
Video Diffusion Models have been developed for video generation, usually integrating text and image conditioning to enhance control over the generated content. Despite the progress, ensuring consistency across frames rem…
DiversityVideo GenerationScalable Self-Supervised Representation Learning from Spatiotemporal Motion Trajectories for Multimodal Computer Vision
Self-supervised representation learning techniques utilize large datasets without semantic annotations to learn meaningful, universal features that can be conveniently transferred to solve a wide variety of downstream su…
Representation LearningSemantic SegmentationCzech Text Processing with Contextual Embeddings: POS Tagging, Lemmatization, Parsing and NER
Contextualized embeddings, which capture appropriate word meaning depending on context, have recently been proposed. We evaluate two meth ods for precomputing such embeddings, BERT and Flair, on four Czech text processin…
Dependency ParsingLemmatizationnamed-entity-recognitionNamed Entity Recognition+5