CSS: Combining Self-training and Self-supervised Learning for Few-shot Dialogue State Tracking

2022-10-11 · Haoning Zhang, Junwei Bao, Haipeng Sun, Huaishao Luo, Wenye Li, Shuguang Cui

Few-shot dialogue state tracking (DST) is a realistic problem that trains the DST model with limited labeled data. Existing few-shot methods mainly transfer knowledge learned from external labeled dialogue data (e.g., from question answering, dialogue summarization, machine reading comprehension tasks, etc.) into DST, whereas collecting a large amount of external labeled data is laborious, and the external data may not effectively contribute to the DST-specific task. In this paper, we propose a few-shot DST framework called CSS, which Combines Self-training and Self-supervised learning methods. The unlabeled data of the DST task is incorporated into the self-training iterations, where the pseudo labels are predicted by a DST model trained on limited labeled data in advance. Besides, a contrastive self-supervised method is used to learn better representations, where the data is augmented by the dropout operation to train the model. Experimental results on the MultiWOZ dataset show that our proposed CSS achieves competitive performance in several few-shot scenarios.

📄 PDF Abstract BibTeX arXiv:2210.05146

Code (0)

등록된 구현이 없습니다.

Tasks

Dialogue State Tracking Machine Reading Comprehension Question Answering Reading Comprehension Self-Supervised Learning

Methods 이 논문이 사용한 방법론

DST Dynamic sparse training methods train neural networks in a sparse manner, starting with an initial sparse mask, and periodically updating the mask based on some criteria.

Dropout Dropout is a regularization technique for neural networks that drops a unit (along with connections) at training time with a specified probability $p$ (a common value is…

A Survey of Self-Supervised and Few-Shot Object Detection

2021-10-27 · Gabriel Huang, Issam Laradji, David Vazquez, Simon Lacoste-Julien 외

Labeling data is often expensive and time-consuming, especially for tasks such as object detection and instance segmentation, which require dense labeling of the image. While few-shot object detection is about training a…

Few-Shot Object Detection Instance Segmentation Object object-detection+3

Self-Supervised Learning in Deep Networks: A Pathway to Robust Few-Shot Classification

2024-11-19 · Yuyang Xiao

This study aims to optimize the few-shot image classification task and improve the model's feature extraction and classification performance by combining self-supervised learning with the deep network model ResNet-101. D…

Classification Few-Shot Image Classification image-classification Image Classification+1

Edge-Based Self-Supervision for Semi-Supervised Few-Shot Microscopy Image Cell Segmentation

2022-08-03 · Youssef Dawoud, Katharina Ernst, Gustavo Carneiro, Vasileios Belagiannis

Deep neural networks currently deliver promising results for microscopy image cell segmentation, but they require large-scale labelled databases, which is a costly and time-consuming process. In this work, we relax the l…

Cell Segmentation Segmentation

SLIP: Self-supervision meets Language-Image Pre-training

2021-12-23 · Norman Mu, Alexander Kirillov, David Wagner, Saining Xie

Recent work has shown that self-supervised pre-training leads to improvements over supervised learning on challenging visual recognition tasks. CLIP, an exciting new approach to learning with language supervision, demons…

Multi-Task Learning Representation Learning Self-Supervised Learning

Improving Few-Shot Learning with Auxiliary Self-Supervised Pretext Tasks

2021-01-24 · Nathaniel Simard, Guillaume Lagrange

Recent work on few-shot learning \cite{tian2020rethinking} showed that quality of learned representations plays an important role in few-shot classification performance. On the other hand, the goal of self-supervised lea…

Few-Shot Learning Representation Learning Self-Supervised Learning