PIPsUS: Self-Supervised Dense Point Tracking in Ultrasound
Finding point-level correspondences is a fundamental problem in ultrasound (US), since it can enable US landmark tracking for intraoperative image guidance in different surgeries, including head and neck. Most existing US tracking methods, e.g., those based on optical flow or feature matching, were initially designed for RGB images before being applied to US. Therefore domain shift can impact their performance. Training could be supervised by ground-truth correspondences, but these are expensive to acquire in US. To solve these problems, we propose a self-supervised pixel-level tracking model called PIPsUS. Our model can track an arbitrary number of points in one forward pass and exploits temporal information by considering multiple, instead of just consecutive, frames. We developed a new self-supervised training strategy that utilizes a long-term point-tracking model trained for RGB images as a teacher to guide the model to learn realistic motions and use data augmentation to enforce tracking from US appearance. We evaluate our method on neck and oral US and echocardiography, showing higher point tracking accuracy when compared with fast normalized cross-correlation and tuned optical flow. Code will be available once the paper is accepted.
Code (1)
Tasks
Data AugmentationLandmark TrackingOptical Flow EstimationPoint TrackingSimilar Papers 제목 키워드 기반
Pixel-level Correspondence for Self-Supervised Learning from Video
While self-supervised learning has enabled effective representation learning in the absence of labels, for vision, video remains a relatively untapped source of supervision. To address this, we propose Pixel-level Corres…
Contrastive Learningimage-classificationImage ClassificationOptical Flow Estimation+3MAST: A Memory-Augmented Self-supervised Tracker
Recent interest in self-supervised dense tracking has yielded rapid progress, but performance still remains far from supervised methods. We propose a dense tracking model trained on videos without any annotations that su…
Semantic SegmentationSemi-Supervised Video Object SegmentationUnsupervised Video Object SegmentationVideo Object Segmentation+1Joint-task Self-supervised Learning for Temporal Correspondence
This paper proposes to learn reliable dense correspondence from videos in a self-supervised manner. Our learning process integrates two highly related tasks: tracking large image regions \emph{and} establishing fine-grai…
Object TrackingSelf-Supervised LearningSemi-Supervised Video Object SegmentationUnsupervised Video Object SegmentationDINO-Tracker: Taming DINO for Self-Supervised Point Tracking in a Single Video
We present DINO-Tracker -- a new framework for long-term dense tracking in video. The pillar of our approach is combining test-time training on a single video, with the powerful localized semantic features learned by a p…
Point TrackingSelf-supervised Video Object Segmentation
The objective of this paper is self-supervised representation learning, with the goal of solving semi-supervised video object segmentation (a.k.a. dense tracking). We make the following contributions: (i) we propose to i…
ObjectOne-shot visual object segmentationRepresentation LearningSegmentation+4