3ET: Efficient Event-based Eye Tracking using a Change-Based ConvLSTM Network
This paper presents a sparse Change-Based Convolutional Long Short-Term Memory (CB-ConvLSTM) model for event-based eye tracking, key for next-generation wearable healthcare technology such as AR/VR headsets. We leverage the benefits of retina-inspired event cameras, namely their low-latency response and sparse output event stream, over traditional frame-based cameras. Our CB-ConvLSTM architecture efficiently extracts spatio-temporal features for pupil tracking from the event stream, outperforming conventional CNN structures. Utilizing a delta-encoded recurrent path enhancing activation sparsity, CB-ConvLSTM reduces arithmetic operations by approximately 4.7$\times$ without losing accuracy when tested on a \texttt{v2e}-generated event dataset of labeled pupils. This increase in efficiency makes it ideal for real-time eye tracking in resource-constrained devices. The project code and dataset are openly available at \url{https://github.com/qinche106/cb-convlstm-eyetracking}.
Code (1)
Tasks
Pupil TrackingSimilar Papers 제목 키워드 기반
Weakly Supervised Convolutional LSTM Approach for Tool Tracking in Laparoscopic Videos
Purpose: Real-time surgical tool tracking is a core component of the future intelligent operating room (OR), because it is highly instrumental to analyze and understand the surgical activities. Current methods for surgic…
Instrument RecognitionObject DetectionSurgical tool detectionVideo Object Tracking+1SEN2DWATER: A Novel Multispectral and Multitemporal Dataset and Deep Learning Benchmark for Water Resources Analysis
Climate change has caused disruption in certain weather patterns, leading to extreme weather events like flooding and drought in different parts of the world. In this paper, we propose machine learning methods for analyz…
JaneEye: A 12-nm 2K-FPS 18.9-$μ$J/Frame Event-based Eye Tracking Accelerator
Eye tracking has become a key technology for gaze-based interactions in Extended Reality (XR). However, conventional frame-based eye-tracking systems often fall short of XR's stringent requirements for high accuracy, low…
Convolutional LSTM Neural Networks for Modeling Wildland Fire Dynamics
As the climate changes, the severity of wildland fires is expected to worsen. Models that accurately capture fire propagation dynamics greatly help efforts for understanding, responding to and mitigating the damages caus…
BIG-bench Machine LearningUnsupervised CD in satellite image time series by contrastive learning and feature tracking
While unsupervised change detection using contrastive learning has been significantly improved the performance of literature techniques, at present, it only focuses on the bi-temporal change detection scenario. Previous …
Change DetectionContrastive LearningTime Series