Sim2Real Bilevel Adaptation for Object Surface Classification using Vision-Based Tactile Sensors

2023-11-02 · Gabriele M. Caddeo, Andrea Maracani, Paolo D. Alfano, Nicola A. Piga, Lorenzo Rosasco, Lorenzo Natale

In this paper, we address the Sim2Real gap in the field of vision-based tactile sensors for classifying object surfaces. We train a Diffusion Model to bridge this gap using a relatively small dataset of real-world images randomly collected from unlabeled everyday objects via the DIGIT sensor. Subsequently, we employ a simulator to generate images by uniformly sampling the surface of objects from the YCB Model Set. These simulated images are then translated into the real domain using the Diffusion Model and automatically labeled to train a classifier. During this training, we further align features of the two domains using an adversarial procedure. Our evaluation is conducted on a dataset of tactile images obtained from a set of ten 3D printed YCB objects. The results reveal a total accuracy of 81.9%, a significant improvement compared to the 34.7% achieved by the classifier trained solely on simulated images. This demonstrates the effectiveness of our approach. We further validate our approach using the classifier on a 6D object pose estimation task from tactile data.

📄 PDF Abstract BibTeX arXiv:2311.01380

Code (1)

hsp-iit/sim2real-surface-classification 공식 구현 pytorch

Tasks

6D Pose Estimation using RGB Pose Estimation

Methods 이 논문이 사용한 방법론

SET Dynamic Sparse Training method where weight mask is updated randomly periodically

Diffusion Diffusion models generate samples by gradually removing noise from a signal, and their training objective can be expressed as a reweighted variational lower-bound…

ALIGN In the ALIGN method, visual and language representations are jointly trained from noisy image alt-text data. The image and text encoders are learned via contrastive loss…

Domain Adaptive Object Detection via Dual-Stream Bilevel-Cycle Optimization

2026-06-30 · Yannan Chen, Wei Wang, Wenqiang Wang, Ruoyu Chen 외 arxiv

Cycle self-training (CST) breaks the shared classifier assumption of the standard self-training framework, which is effective for unsupervised domain adaptation and exploits unlabeled target data by training with target …

Unsupervised Domain Adaptation Image Classification Object Detection

Selective classification using a robust meta-learning approach

2022-12-12 · Nishant Jain, Karthikeyan Shanmugam, Pradeep Shenoy

Predictive uncertainty-a model's self awareness regarding its accuracy on an input-is key for both building robust models via training interventions and for test-time applications such as selective classification. We pro…

Bilevel Optimization Classification Domain Adaptation Medical Diagnosis+2

Out-of-Domain Human Mesh Reconstruction via Dynamic Bilevel Online Adaptation

2021-11-07 · Shanyan Guan, Jingwei Xu, Michelle Z. He, Yunbo Wang 외

We consider a new problem of adapting a human mesh reconstruction model to out-of-domain streaming videos, where performance of existing SMPL-based models are significantly affected by the distribution shift represented …

3D Absolute Human Pose Estimation 3D Human Pose Estimation Bilevel Optimization

qNBO: quasi-Newton Meets Bilevel Optimization

2025-02-03 · Sheng Fang, Yong-Jin Liu, Wei Yao, Chengming Yu 외

Bilevel optimization, addressing challenges in hierarchical learning tasks, has gained significant interest in machine learning. The practical implementation of the gradient descent method to bilevel optimization encount…

Bilevel Optimization Hyperparameter Optimization Meta-Learning

Bilevel Late Acceptance Hill Climbing for the Electric Capacitated Vehicle Routing Problem

2026-04-14 · Yinghao Qin, Mosab Bazargani, Edmund K. Burke, Carlos A. Coello Coello 외 arxiv

This paper tackles the Electric Capacitated Vehicle Routing Problem (E-CVRP) through a bilevel optimization framework that handles routing and charging decisions separately or jointly depending on the search stage. By an…

Bilevel Optimization