Segmentation for Efficient Supervised Language Annotation with an Explicit Cost-Utility Tradeoff
In this paper, we study the problem of manually correcting automatic annotations of natural language in as efficient a manner as possible. We introduce a method for automatically segmenting a corpus into chunks such that many uncertain labels are grouped into the same chunk, while human supervision can be omitted altogether for other segments. A tradeoff must be found for segment sizes. Choosing short segments allows us to reduce the number of highly confident labels that are supervised by the annotator, which is useful because these labels are often already correct and supervising correct labels is a waste of effort. In contrast, long segments reduce the cognitive effort due to context switches. Our method helps find the segmentation that optimizes supervision efficiency by defining user models to predict the cost and utility of supervising each segment and solving a constrained optimization problem balancing these contradictory objectives. A user study demonstrates noticeable gains over pre-segmented, confidence-ordered baselines on two natural language processing tasks: speech transcription and word segmentation.
Code (0)
등록된 구현이 없습니다.
Tasks
Active LearningSegmentationSimilar Papers 제목 키워드 기반
Processing and Segmentation of Human Teeth from 2D Images using Weakly Supervised Learning
Teeth segmentation is an essential task in dental image analysis for accurate diagnosis and treatment planning. While supervised deep learning methods can be utilized for teeth segmentation, they often require extensive …
Keypoint DetectionSegmentationWeakly-supervised LearningDisFaceRep: Representation Disentanglement for Co-occurring Facial Components in Weakly Supervised Face Parsing
Face parsing aims to segment facial images into key components such as eyes, lips, and eyebrows. While existing methods rely on dense pixel-level annotations, such annotations are expensive and labor-intensive to obtain.…
Semantic SegmentationFace ParsingSemi-supervised Active Learning for Instance Segmentation via Scoring Predictions
Active learning generally involves querying the most representative samples for human labeling, which has been widely studied in many fields such as image classification and object detection. However, its potential has n…
Active Learningimage-classificationImage ClassificationInstance Segmentation+5EAUWSeg: Eliminating annotation uncertainty in weakly-supervised medical image segmentation
Weakly-supervised medical image segmentation is gaining traction as it requires only rough annotations rather than accurate pixel-to-pixel labels, thereby reducing the workload for specialists. Although some progress has…
Image SegmentationMedical Image SegmentationSemantic SegmentationWeakly supervised segmentationPyMIC: A deep learning toolkit for annotation-efficient medical image segmentation
Background and Objective: Open-source deep learning toolkits are one of the driving forces for developing medical image segmentation models. Existing toolkits mainly focus on fully supervised segmentation and require ful…
Deep LearningImage SegmentationMedical Image SegmentationSegmentation+2