Diffusion Features for Zero-Shot 6DoF Object Pose Estimation

2024-11-25 · Bernd Von Gimborn, Philipp Ausserlechner, Markus Vincze, Stefan Thalhammer

Zero-shot object pose estimation enables the retrieval of object poses from images without necessitating object-specific training. In recent approaches this is facilitated by vision foundation models (VFM), which are pre-trained models that are effectively general-purpose feature extractors. The characteristics exhibited by these VFMs vary depending on the training data, network architecture, and training paradigm. The prevailing choice in this field are self-supervised Vision Transformers (ViT). This study assesses the influence of Latent Diffusion Model (LDM) backbones on zero-shot pose estimation. In order to facilitate a comparison between the two families of models on a common ground we adopt and modify a recent approach. Therefore, a template-based multi-staged method for estimating poses in a zero-shot fashion using LDMs is presented. The efficacy of the proposed approach is empirically evaluated on three standard datasets for object-specific 6DoF pose estimation. The experiments demonstrate an Average Recall improvement of up to 27% over the ViT baseline. The source code is available at: https://github.com/BvG1993/DZOP.

📄 PDF Abstract BibTeX arXiv:2411.16668

Code (0)

등록된 구현이 없습니다.

Tasks

ObjectPose Estimation

Methods 이 논문이 사용한 방법론

Latent Diffusion Model Diffusion models applied to latent spaces, which are normally built with (Variational) Autoencoders.

Diffusion Diffusion models generate samples by gradually removing noise from a signal, and their training objective can be expressed as a reweighted variational lower-bound…

ADOPT Please enter a description about the method here

HomeDiffusion: Zero-Shot Object Customization with Multi-View Representation Learning for Indoor Scenes

2026-06-29 · Guoqiu Li, Jin Song, Yiyun Fei arxiv

Recently, zero-shot object customization generation methods have rapidly developed and shown tremendous potential for applications. For instance, in the e-commerce domain, consumers can observe the visual effect of furni…

Representation Learning

DiffCut: Catalyzing Zero-Shot Semantic Segmentation with Diffusion Features and Recursive Normalized Cut

2024-06-05 · Paul Couairon, Mustafa Shukor, Jean-Emmanuel Haugeard, Matthieu Cord 외

Foundation models have emerged as powerful tools across various domains including language, vision, and multimodal tasks. While prior works have addressed unsupervised image segmentation, they significantly lag behind su…

Image Segmentation Segmentation Semantic Segmentation Unsupervised Image Segmentation+3

Conditional Latent Diffusion Models for Zero-Shot Instance Segmentation

2025-08-06 · Maximilian Ulmer, Wout Boerdijk, Rudolph Triebel, Maximilian Durner arxiv

This paper presents OC-DiT, a novel class of diffusion models designed for object-centric prediction, and applies it to zero-shot instance segmentation. We propose a conditional latent diffusion framework that generates …

Instance Segmentation

Unlocking Diffusion Hierarchies: Adaptive Timestep Selection for Zero-Shot Segmentation

2026-06-14 · Ramin Nakhli, Mahesh Ramachandran, Luca Ballan arxiv

Zero-shot segmentation has recently shown notable improvement by leveraging the rich visual priors in large-scale text-to-image diffusion models, such as Stable Diffusion. However, current diffusion-based methods often f…

Image Diffusion Models Exhibit Emergent Temporal Propagation in Videos

2025-11-25 · Youngseo Kim, Dohyun Kim, Geonhee Han, Paul Hongsuck Seo arxiv

Image diffusion models, though originally developed for image generation, implicitly capture rich semantic structures that enable various recognition and localization tasks beyond synthesis. In this work, we investigate …

Video Object Segmentation Image Generation Object Tracking