Modality-Buffet for Real-Time Object Detection
Real-time object detection in videos using lightweight hardware is a crucial component of many robotic tasks. Detectors using different modalities and with varying computational complexities offer different trade-offs. One option is to have a very lightweight model that can predict from all modalities at once for each frame. However, in some situations (e.g., in static scenes) it might be better to have a more complex but more accurate model and to extrapolate from previous predictions for the frames coming in at processing time. We formulate this task as a sequential decision making problem and use reinforcement learning (RL) to generate a policy that decides from the RGB input which detector out of a portfolio of different object detectors to take for the next prediction. The objective of the RL agent is to maximize the accuracy of the predictions per image. We evaluate the approach on the Waymo Open Dataset and show that it exceeds the performance of each single detector.
Code (0)
등록된 구현이 없습니다.
Tasks
Decision MakingObjectobject-detectionObject DetectionReal-Time Object DetectionReinforcement Learning (RL)Sequential Decision MakingSimilar Papers 제목 키워드 기반
The convergent Indian buffet process
We propose a new Bayesian nonparametric prior for latent feature models, which we call the convergent Indian buffet process (CIBP). We show that under the CIBP, the number of latent features is distributed as a Poisson d…
Physics-Assisted Reduced-Order Modeling for Identifying Dominant Features of Transonic Buffet
Transonic buffet is a flow instability phenomenon that arises from the interaction between the shock wave and the separated boundary layer. This flow phenomenon is considered to be highly detrimental during flight and po…
Non-exchangeable feature allocation models with sublinear growth of the feature sizes
Feature allocation models are popular models used in different applications such as unsupervised learning or network modeling. In particular, the Indian buffet process is a flexible and simple one-parameter feature alloc…
Learning invariant features using the Transformed Indian Buffet Process
Identifying the features of objects becomes a challenge when those features can change in their appearance. We introduce the Transformed Indian Buffet Process (tIBP), and use it to define a nonparametric Bayesian model t…
Gibbs-type Indian buffet processes
We investigate a class of feature allocation models that generalize the Indian buffet process and are parameterized by Gibbs-type random measures. Two existing classes are contained as special cases: the original two-par…
Vocal Bursts Type Prediction